BW #31: Poverty

First and foremost: Thanks to the many of you who sent warm, supportive messages in the wake of the unexpected passing of my father, Rabbi Barry Dov Lerner, two weeks ago. I learned much from my father, chief among them to love reading and learning — and then to share what I had learned with others. My father came up with the name for Bamboo Weekly, and frequently gave me feedback and comments, even though he wasn't able to code.

You can read the eulogy I gave at the funeral here: https://docs.google.com/document/d/1PxSo9kGwIrkNK4FWatDEEWzXSP2R3OGdEn17eAeEZWU/edit?usp=sharing

And now, back to our regularly scheduled Pandas and data analysis.

While considering a topic to examine this week, I saw an article in the New York Times, "Poverty Rate Soared in 2022 as Aid Ended and Prices Rose". The article cited a report from the US Census Bureau indicating that while poverty levels in the US had declined in 2020 and 2021, they had risen in 2022. The Census Bureau's page describing the study is at https://www.census.gov/library/publications/2023/demo/p60-280.html , including a full report at /content/files/content/dam/Census/library/publications/2023/demo/p60-280.pdf .

Many countries measure poverty levels, and you might have heard mentions of these numbers in the news. But how exactly do you measure poverty? What amount of money is enough to put someone over the poverty line? Does that vary from place to place? What if the family receives government assistance — should that be included in the amount that the family receives, or not?

These and many other questions have led to the Census Bureau measuring poverty in two different ways. The original poverty measure (OPM) compares "pretax money income to a poverty threshold that is adjusted by family composition." That's fairly straightforward to compute, but it can miss a lot of factors that change whether a family should be considered poor. For that reason, the Census Bureau created the supplemental poverty measure (SPM), which includes government assistance, health expenses, and taxes. It's a newer measure, having started only in 2011, but the idea is that it more fully captures the nuances of poverty.

I was wondering what we can learn about poverty from the data. Have things really gotten better in the last few decades? Can we see the downturn in poverty from the last few years, followed by the uptick? Do we see differences in poverty rates for people of different ages and races? And while we're looking at census data, how many elderly vs. children live in the US, and how is that changing?

Data and eight questions

This week, we'll look at a small part of the data from the latest poverty survey, which was published just a few days ago. The overall download page is at:

https://www.census.gov/library/publications/2023/demo/p60-280.html

The data itself is broken up across several different Excel files. We'll only look at one of those files, whose download URL is:

/content/files/programs-surveys/demo/tables/p60/280/tablea3_hist_pov_by_all_and_age.xlsx

I have eight questions and tasks for you this week. The learning goals include working with Excel, cleaning data, grouping, plotting, and selecting rows and columns. I’ll be back tomorrow with detailed solutions, as well as the Jupyter notebook I used to solve these problems:

• Read the "History POV by all and age" file into a data frame. We're only interested (for now) in the "ALL RACES" section. Turn the years into an index, and remove the "percent" measures.
• In 2022, what was the number of Americans living below the poverty line?