[Don't forget that my new "Hands-on Projects with Python" course starts later this month! I'm holding a free webinar tomorrow, where I'll lay out the course in full detail, answering any questions you might have. Sign up at https://store.lerner.co.il/about-hoppy-1 and you'll get a Zoom link 10 minutes before we start, as well as a link to the video recording after we're done.]
For the last few years, most economic indicators showed that the US was doing well, especially when compared with the rest of the world. Kyla Scanlon (https://kyla.substack.com/) famously called this a "vibecession," in that people *felt* like things were bad, even if the hard data was generally positive. (She also argues that we need new measures for what constitutes a "good" economy.)
Trump has scrambled things on many fronts, including the economy. But it'll be a few months before we'll really know precisely what is happening. In a Marketplace interview on March 17th, economist Mohamed El-Erian said that soft data comes faster than hard data, and that it can often be a good predictor of where things are going (https://www.marketplace.org/2025/03/17/el-erian-on-doge-tariffs-and-recession-chances-fed-inflation-employment-trump/).
This week, we got data from the New York Federal Reserve. This data came from the Survey of Consumer Expectations at the NY Fed's Center for Microeconomic Data (https://www.newyorkfed.org/microeconomics/sce#/).
The credit-access module in the SCE (https://www.newyorkfed.org/microeconomics/sce/credit-access/) asks people how much credit (e.g., credit cards, student loans, and mortgages) they currently have, and what their plans and expectations are for the coming year. The survey publishes results in February, June, and October of each year.
Data and six questions
We want to download the "chart data" for the survey, an Excel file that you can also get via the following link:
I found it helpful and interesting to look through the entire survey questionnaire, available in PDF from here:
Paid subscribers can download both files from a link at the bottom of this message.
Learning goals for this week include: Working with Excel files, manipulating dates and times, and pivot tables.
I'll be back tomorrow with my solutions, including (for paid subscribers) a downloadable copy of the Jupyter notebook I used to solve these problems, as well as a one-click link to open that notebook (and the data) in Google Colab.
Meanwhile, here are my six questions and tasks:
- Read data from both the the `overall` and `demographics` sheets into data frames. In both cases, use the date as the index, parsing it into a `datetime` value.
- A common question to ask people about their financial security is whether they could come up with a certain amount of money, given the need. Questions N24 and N25 in the survey ask participants to indicate the percentage chance that they would have an unexpected $2,000 expense in the coming 12 months, and also the percentage chance that they could come up with $2,000 if they needed to. Using the `overall` data frame, produce a line plot comparing these values over the course of the data set. What does this indicate? If we use subplots, does the clarity of the plot change? Does setting the y axis to be 0-100, rather than set automatically, change our interpretation?