Skip to content
3 min read · Tags: csv datetime filtering regular-expressions correlations plotting

BW #107: Consumer confidence

Get better at: CSV files, dates and times, filtering, regular expressions, correlations, and plotting

BW #107: Consumer confidence

How is the economy doing? There's no one answer to that question, since you can measure economic health in many different ways. And of course, different businesses, people, countries, and demographics can experience very different things at the same time. For many years, the University of Michigan has surveyed American consumers (http://www.sca.isr.umich.edu/), asking them to indicate how they feel about the current state of the economy, as well as where they think it's going over the coming months and years. Answers to the survey allow us not just to know how Americans are feeling overall, but how different populations within the United States are feeling.

According to the survey (https://edition.cnn.com/2025/02/21/economy/consumer-sentiment-inflation-tariffs/index.html?cid=external-feeds_iluminar_msn), Americans aren't feeling that great about the economy. Certainly, the fact that prices on many goods are still high, even though Trump promised repeatedly during his campaign to bring them down, hasn't helped.

People are particularly incensed over egg prices; Slate Money (https://slate.com/slatemoney) now has a regular "Eggwatch" feature, from which I have learned about the theft of 100,000 eggs (https://www.msn.com/en-us/news/us/egg-heist-in-the-us-police-never-heard-of-a-hundred-thousand-eggs-being-stolen/ar-AA1zH5kS) and an online egg exchange. Also, the Daily Show recently asked New Yorkers how they're dealing with the high price of eggs (https://www.youtube.com/watch?v=PJAopYB75dQ).

This week, we'll analyze data from the Michigan survey of consumer sentiment, whose latest report came out on Friday. We'll see how Americans are thinking about the economy, and whether some groups are feeling better about it than others.

Data and seven questions

The data comes from the "Surveys of Consumers" site at the University of Michigan. The university performs a number of different surveys, the best-known as of which measures consumer sentiment, with three measures:

The numbers are relative figures, so a measure of 80 is better than one of 70, but worse than one of 90. The idea is that we can compare measurements at two points in time, and compare the sentiments.

The data itself can be downloaded from the overall data page, at

https://data.sca.isr.umich.edu/data-archive/mine.php

We'll be looking at a few different parts of the data, so you'll want to download the "all tables" data in CSV format.

You can also get a data dictionary (the "Time series variable codebook"), explaining the column names and what they measure, from

https://data.sca.isr.umich.edu/technical-docs.php

This week, I have seven tasks and questions for you to answer. Learning goals include working with date-time data, regular expressions, filtering, resampling, window functions, and plotting.

Paid subscribers can download the data file and data dictionary from a link at the bottom of this message.

I'll be back tomorrow with my solutions, including (for paid subscribers) a downloadable copy of the Jupyter notebook I used to solve these problems, as well as a one-click link to open that notebook (and the data) in Google Colab.

Here are my seven tasks and questions: