Census Bureau

Direct Measures of Poverty as Indicators of Economic Need: Evidence From The Survey of Income and Program Participation

Kurt Bauman

Population Division
U.S. Bureau of the Census
Washington, D.C. 20233,8800

November 1998

POPULATION DIVISION TECHNICAL WORKING PAPER NO. 30


An earlier version of this paper was presented at the annual meetings of the Southern Demographic Association, Annapolis MD, October 30, 1998.

This paper reports the results of research and analysis undertaken by Census Bureau Staff. It has undergone a more limited review than official Census Bureau publications. This report is released to inform interested parties of research and to encourage discussion.


Abstract

There has been increasing interest in using direct measures of economic well being to keep track of how people are getting by. One set of indicators developed for this purpose includes questions on paying bills, ability to get needed health care and food sufficiency. Data gathered by the Census Bureau in the Survey of Income and Program Participation represent the first attempt to gather such data from a nationally-representative sample. The evidence presented here supports the use of hardship measures as a valid and useful measure of household well-being. They are strongly related to other factors correlated with poverty, and have a significant influence on high school dropout. However, there are other dimensions to hardship that are not strictly correlated with poverty, and there is some evidence that hardship might not be reliably measured over time. Those who use it as an outcome measure or as a way to calibrate other measures of poverty and well-being need to proceed with caution.


Direct Measures of Poverty as Indicators of Economic Need: Evidence from
the Survey of Income and Program Participation

Introduction

There has been increasing interest in finding new ways to measure people's well being and keep track of how they are getting by. One reason for this interest is two rounds of Federal welfare reform that have focused directly on the transition from income support programs to self-support. Measured income has often failed to provide an adequate characterization of well being in these circumstances, because income levels don't measure the direct and indirect costs of working. More generally, there has been an interest in the means by which poor people support themselves. Using money income as a measure of well being is constraining for those who want to understand how and why people combine both monetary and non-monetary elements of support.

A similar set of issues is involved in discussions of poverty measurement. Critics have asserted that the current poverty measure distributes poverty across segments of the population in ways that poorly reflect underlying material conditions. In addition, there is controversy over how to account for family size and changes in living standards over time. As a result, there has been an increasing interest in ways to measure poverty in terms of "objective" criteria.

Interest in these topics have been addressed by research on the levels of material hardship among single mothers and others in the population. However, this research has not used nationally representative samples, focusing instead on the city of Chicago and in several selective populations in other cities. Data gathered by the Census Bureau in the Survey of Income and Program Participation address this gap. Topical modules on extended measures of well being and meeting of basic needs have been administered in three panels of this survey. Using these data, it is possible to learn more about the reliability and validity of these measures as benchmarks for a deeper understanding of poverty processes and dynamics. The purpose of this paper is to assess the usefulness of hardship measures along these lines.


Background

Recent research by Kathryn Edin and Laura Lein (1997a, 1997b, Edin 1991) has drawn attention to the ways in which poor mothers balance time, work, and resources from friends, relatives and others to balance a family budget. An important conclusion from this work is that entering the labor force involves costs which make the comparison of the well-being of working and non-working mothers more problematic than has been evident from earlier research. A second conclusion is that the nature of single mothers' "survival strategies" strongly influences their ability to substitute work for public assistance.

These conclusions have a potentially profound influence on how we interpret the course of events that unfold as income support programs are devolved from the Federal government to the states. Money income appears to bear even less relation to actual material circumstances than has been recognized. In addition, typical "welfare to work" programs that focus on skills development may not address the critical issues facing women who often need to use numerous strategies to support their families (Edin and Lein 1997a).

Up until recently , most research on these issues has made use of the current "official" poverty measure, which takes account of before-tax income and family size. The measure has been criticized for inadequate accounting of a number of factors that are increasingly critical to the material well-being of families. These include: the cost of work -- especially for single mothers, the effect of health status, the cost of health care, taxes, and non-cash benefits from government and other sources. In addition, the current standard is often accused of taking inadequate account of family size and changes in living standards over time. The standard also makes no adjustment for geographic differences in cost of living (Citro & Michael 1995; Ruggles 1990).

Improving the measurement of poverty along each of these dimensions involves difficult measurement issues. It is tempting, therefore, to look for additional measures of material well-being to provide a basis for comparisons between groups who differ along these dimensions. For example, expenditure data has been used to calibrate family size adjustments (Betson 1990; Lazear & Michael 1988). It has also been used to track changes in poverty and inequality over time (Mayer & Jencks 1993; Johnson & Smeeding 1998). Subjective estimates of the amount of money needed to get by have been used to calibrate changes in poverty thresholds over time (Citro & Michael 1995).

In a study of material hardship in Chicago, Mayer and Jencks introduced a set of questions designed to directly assess the degree to which families experienced financial problems and budget shortfalls (1989; Cook et al. 1986). They argue that these measures are more than just another way of assessing the well-being of families with low incomes. Rather, they say, their measures closely embody the central concern of poverty policy -- the degree to which families are able to meet their basic needs.

Mayer and Jencks use their measures of material hardship to test hypotheses about family size adjustments (1989). A similar set of measures were used to compare the income contributions of married and cohabiting couples (Bauman 1997).

Most of the work examining hardship has relied on special or limited samples from one or a few geographic locations. And while some work has been done to validate these data, we still don't have a good picture of what they tell us about people's actual living conditions. The present analysis makes use of the Survey of Income and Program Participation (SIPP). SIPP has the advantage of being a national sample. It also has extensive measures of other circumstances that would contribute to well-being. Finally, it is collected at more than one point in time.

The present analysis will focus on many of the same issues covered in earlier work with the Chicago data. These include basic measures of reliability and validity, and the relationship of hardship measures to other related characteristics. The analysis will take place in four steps.

These four sections will be followed by a brief discussion and conclusion. Before the analysis begins, however, it is necessary to briefly describe the data being used.


Data

The data for this study are from the 1991, 1992 and 1993 panels of the U.S. Census Bureau's Survey of Income and Program Participation. The focus is on the topical module on "Extended Measures of Well-being" administered in wave 6 of the 1991 panel and wave 3 of the 1992 panel and the "Basic Needs" topical module administered in 1993 wave 9. The 1991 and 1992 topical modules were identical, and the 1993 topical module contained a subset of the questions asked in the previous two panels. The focus here will be on those questions asked in all three questionnaires.

The topical modules were administered to the reference person in each household (or his/her proxy), providing data on material hardship. The 1991 and 1992 panels overlapped so that both sets of questionnaires were in the field at the same time -- winter of 1992-1993. There were 12,508 households in the 1991 panel and 17,818 in the 1992 panel. The 1993 topical module was administered exactly three years later, in the winter of 1995-1996. The 1993 panel provided 17,572 cases for analysis. Overall, there are for a total of 47,898 cases in the three data sets. The SIPP is based on a multistage sample, rendering standard statistical tests inappropriate. In the tabulations and regressions presented below, standard errors are adjusted to allow for an assumed design effect of 3.0.

The questions on material hardship asked in the 1991 and 1992 SIPP were very similar to those used by Mayer and Jencks in their analysis of poverty and material hardship in Chicago (Mayer & Jencks 1989). The survey questions themselves appear in the appendix, below. Briefly, household heads1 were asked, "During the past 12 months, has there been a time when your household did not meet its essential expenses? By essential expenses, I mean things like the mortgage or rent payment, utility bills, or important medical care." They were then asked about instances when the household did not pay the full amount of rent or mortgage, did not pay the full amount of utility bills, had telephone service cut off due to nonpayment. Next they were asked if someone had needed to go to the doctor or hospital but didn't go, and if someone had needed to see a dentist but didn't go. Finally, household heads were asked to categorize the food eaten in their household as "enough of the kinds of food we want," "enough but not always the kinds we want to eat," "sometimes not enough to eat," or "often not enough to eat."

Three summary indicators of material hardship were created for the present research. First, a general indicator of hardship was recorded for everyone who reported any type of hardship aside from inability to see a doctor or dentist. Second, an indicator was created indicating whether there was an instance when someone in the household had not seen the doctor or dentist when needed. Finally, an indicator of "multiple hardships" was created for every household where more than one type of hardship was reported.

Because there is only one respondent per household, characteristics of the household itself (household income, home ownership) and characteristics of the household head (race, sex, education) were often used in place of individual characteristics. (Individual characteristics were still used for some overall counts of individuals experiencing different types of hardship in their households.)

Most of the control variables presented below are fairly self-explanatory. Variables such as income and employment that refer to a time period are measured with respect to the four-month time period before the interview. Employment status refers to continuous employment over the four-month reference period. Health insurance status was recorded as the absence of health insurance during any of the four months prior to interview for any household member.


Face validity

Before turning to some of the statistical tests, it is well to examine the questions for their content. Much of the argument for using these hardship measures by Mayer and Jencks rested on their face validity (1989). Mayer and Jencks point out, for example, that the trend in legislation since the early days of the war on poverty has been towards replacing cash benefits with in-kind benefits, implying a concern over specific types of hardship rather than overall availability of material resources.2 Mayer and Jencks go on to assert that there is little or no concern over the locus of responsibility for the hardship:

"... if families do not get adequate food, shelter or medical care, most Americans seem to think the government should try to help in some way, even if the problem is caused by incompetence, profligacy, perversity, mental illness, or alcoholism rather than low income." (p. 89)

There are two problems with this argument. First, they present no evidence to support this interpretation beyond the evidence on legislative trends. Clearly, their reading of this evidence is open to second guessing. More important for current purposes, the locus of responsibility is not simply an issue for interpretation, but an issue in data collection. For example, the question on rent payments is:

"In the past 12 months, has there been a time when your household did not pay the full amount of the rent or mortgage?"

There are at least three reasons the rent might not have been paid. First, there may not have been enough money on hand due to unforseen circumstances. Second, the respondent may not have had money on hand due to inadequate planning or budgeting. Third, the money may have been on hand, but the rent neglected.

In the first circumstance, there may not be very close correspondence between hardship and what is often thought of as "need." For example, a solidly middle class or even reasonably well-to-do family could face hardship -- for example, if a self-employed person faced a delay in payment from a major customer, or a family faced a one-time catastrophic expense.

In either of the other two circumstances, the respondent must bear at least part of the blame for the lack of rent payment, and may be reluctant to admit blame. There is some degree of ambiguity in the question (i.e. how late does the payment need to be to count as "not paid?"), and this may provide room for the respondent to avoid reporting missed payments should they wish to do so.

There are two pieces of evidence that the admission of responsibility for mishandling finances might have served as a disincentive for affirmative responses to these questions. First, in directly observing respondents' reactions to these questions as they were administered, it was apparent that there was some degree of reluctance to admit this type of financial problem. Second, people often give inconsistent answers to the first question in the battery, "During the past 12 months, has there been a time when your household did not meet its essential expenses? By essential expenses I mean things like the mortgage or rent payment, utility bills or important medical care." In theory, all those who said "yes" to specific questions about rent, utilities and medical care should also answer in the affirmative to this question. Yet of those who said they missed a rent or mortgage payment, 14 percent said "no" to the initial question. Of those who said they missed a utility payment, 23 percent said "no" to the initial question. Of those who said someone needed to see the doctor but didn't go, 35 percent said "no" to the initial question. While there may have been other reasons for this discrepancy, the reluctance to admit financial problems was most likely a strong contributing factor.

In sum, the set of questions used by Mayer and Jencks and adopted by the Census Bureau contain an element of ambiguity that makes their interpretation slightly more problematic than would be evident from earlier discussions. In all likelihood, the major factor influencing answers to these questions is true hardship or need. However there are other dimensions potentially present that don't lend themselves to such a clear and straightforward interpretation.


Hardship and other measures of well-being

Table 1 shows the number and percentage of people living in households with hardships in 1995. The most common of the hardships reported here were related to paying the bills -- not meeting essential expenses, not paying the full rent or mortgage or not paying the full amount of a gas, electric or oil bill. It was somewhat less common to reside in a household that didn't get food, medical care or dental care. Least common were situations where the household got far enough behind in paying bills that they had their utilities or phone service cut off, or were evicted from their apartment or home. The results from the other two panels (not shown) agree closely with those from the 1993 panel, with exceptions noted below.

Table 1 Percent and Number of People in Households Experiencing Material Harship, 1995 (1k)

Edin and Lein examined the "survival strategies" of families with limited budgets and noted that they often play one type of hardship off against another (1997). They might scrimp on food to buy Christmas presents, or forestall paying one bill in order to pay another. This implies that, over the course of a year, those who have limited resources would experience more than one type of hardship. Table 2 shows that this relationship is evident in the data collected in the SIPP. Fifty-four percent of those experiencing hardship experienced more than one hardship. (This excludes those who reported their household "didn't meet essential expenses" and reported only one of the specific types of expenses about which they were subsequently asked.) Of people in households that didn't meet essential expenses, 64.2 percent experienced more than one type of hardship. For each of the other types of hardship, at least 70 percent of people lived in households with two or more types of hardship.

Table 2 Percent Reporting Two or More Hardships Of Those with at Least One Hardship (1k)

Table 3 shows the relationship between income and hardship by household. A little over 40 percent of those in the lowest income category experience hardship, and around one-quarter experience more than one type of hardship. At the other extreme, less than 5 percent of those in the highest income category experience hardship and 2 percent experience more than one. Thus, there is a strong relationship between income and hardship. On the other hand, these figures indicate that hardship is not a clear indicator of what one might call "true need." One in five households with income between $30,000 and $40,000 reported hardship. Ten percent of households with income between $50,000 and $100,000 reported hardship.

Table 3 Percent experiencing hardships by household income (1k)

One of the original motivations for turning to non-income based measures of poverty was to avoid misunderstanding the situation of the poor (and non poor) who under-report income. However, judging from the proportion of high income households reporting hardship, these hardship indicators also do not capture material distress in an unambiguous way.

Compared to reports of any hardship, reports of multiple hardship are slightly more concentrated among those with low income. The odds of reporting multiple hardships is around half the odds of reporting a single hardship among those in the lowest income categories. At the highest categories, the odds of reporting multiple hardships falls to around 35 percent of the odds of reporting a single hardship.

To further evaluate the hardship measures, it is useful to examine how they are related to other measures of economic well-being that have proven their validity in previous research. Households headed by people with characteristics often associated with poverty -- low education, minority group membership, single parenthood -- should also be households with need. At the same time, a somewhat anomalous finding from previous research needs to be explored. Mayer and Jencks found age to be negatively correlated with hardship, despite the prevalence of low income levels among the aged population. It is of interest to see if this relationship also turns up in the current data.

Table 4 shows the results of a set of regressions designed to examine these relationships. The dependent variable is a binary variable coded as one if the household reported multiple hardships, zero otherwise. Shows are the results of a logistic regression of hardship on the income-poverty ratio, poverty, age, several additional characteristics, and for panel response patterns (these variables will be explained later). The poverty ratio has a strong and significant effect on hardship, as might be expected from the income-hardship relationship just explored. (The main difference between the income measure and the poverty ratio is that the latter is adjusted to account for family size, while the income level is not.) Households whose heads are male, Hispanic or other race, married, employed, well educated and not disabled have lower levels hardship than those without these characteristics. Higher levels of hardship are found among households with children, receiving government transfers, in rental housing and lacking health insurance for one or more members. Many of these influences are related to poverty, and adding these controls decreases the effect of poverty by nearly one-half (the zero order effect of poverty ratio is -.46, compared to the value of -.27 when controls are added).

Table 4 Effect of Panel, Poverty, Age and Other Economic and Demographic Factors on Experience of Multiple Hardships (2k)

That some of these variables have significant effects, above and beyond the effect of poverty, may reflect several factors. First, it may simply be that error in income reporting results in measurable effects of other aspects of poverty. Second, some of these effects may serve as indicators of lack of access to resources, such as savings or ability to borrow funds. This may explain the significance of home ownership status, and may be part of the reason that blacks have higher levels of hardship even with controls. Since lack of assets is generally a condition for participation in government support programs, this may explain the positive effect of transfers on hardship.

Third, the ability to appropriately manage resources might have an impact on hardship, independent of the level of resources available. This may help explain the effect of education. It may also contribute to the large effect of work disability on hardship, since those with disabilities must overcome barriers to successfully manage their affairs. It should be evident, though, that a combination of factors is probably responsible for these effects, since those with less access to resources often have less ability to manage resources for other reasons, and vice-versa.

A fourth potential influence on the patterns observed here is the willingness to report hardship. This may be the reason households with children appear to have significantly greater levels of hardship with these controls in place. Those with children may perceive these issues more acutely than those without children, and may feel more justified in not paying bills in order to meet other expenses when the latter are for the benefit of others within the household. However, this hypothesis is quite speculative at this point.

The effect of age may be partially explained by some of the factors just listed, but it is clearly not explained by poverty (or any of the other control variables). Moving from zero order effects (not shown) to the regression in table 4, the effect of age barely shifts, except for the youngest age category (age 15 to 25). When age is interacted with any of a number of other variables, the same pattern remains. One might, for example, imagine that the effect of age would have to do with the lower costs of owning a home. Older people may pay off mortgages based on a pre-inflationary home purchase price, or own a home free and clear. However, as shown in figure 1, the effect of age is almost exactly the same for renters as for homeowners. More than any of the other variables found significant in these regressions, the effect of age indicates that hardship, as measured here, is not a simple function of material need.

Figure 1 Effect of Age on Multiple Hardship Among Renters and Homeowners (14k)

Table 5 shows nine separate regressions of material hardship measures regressed on the full set of variables just considered. Each of these nine hardship measures are coded to 1 if the person experienced hardship, 0 if they did not. The hardships are: (1) inability to meet essential expenses, (2) non-payment of rent or mortgage, (3) eviction, (4) failure to pay utility bills, (5) utilities being cut off, (6) phone service disconnected, (7) foregoing needed medical care, (8) foregoing needed dental care, and (9) not enough food to eat in the household.

Table 5 Effect of Demographic and Economic Factors on Nine Types of Hardship (7k)

What stands out from the table is the near equality of regression coefficients across all nine hardship measures. This provides an indication that all these measures tap into a single basic phenomenon, justifying the summary measures used earlier.

A few variables are somewhat different in their effects on some outcomes than their effects on other outcomes. Visiting the doctor, visiting the dentist and lack of food are dependent variables characterized by a number of coefficients differing from those in other regressions. So there is some evidence that being unable to visit a health professional is a slightly different sort of hardship than the others.3 The effect of having moved is much greater for eviction than for other hardships. This is probably due to the effect of eviction on moving rather than the other way around. The effect of being in the 1993 panel raises the probability of reporting lack of food, while it decreases the probability of reporting other kinds of hardship. The reason for the change in effect on food shortages was a change in question wording between the two surveys.

From the evidence in this section, it is clear that these material hardship measures hang together reasonably well, and are closely related to traditional measures of income and poverty. On the other hand, material hardship does not function exactly in the same way poverty does. Many high income households report material hardship, and one of the most basic variables that predicts poverty -- age -- predicts lower, rather than higher levels of hardship. Together, these bits of evidence seem to show that there are aspects of material hardship that are not related to poverty as it is traditionally understood. Although income may provide an imperfect basis for measuring of "poverty," material hardship may not be any better. This leaves the analyst with a hanging question -- which provides the more reliable indicator? A way to answer this is to look at how well hardship measures an outcome traditionally associated with poverty -- high school dropout.


Dropout

One of the major reasons that many are concerned about poverty is that poverty demonstrably contributes to a number of other circumstances that are seen as especially undesirable, both from the individual's point of view and from that of society as a whole. These include such things as disease, mortality, divorce, lack of education, mental instability, depression, and drug and alcohol abuse. Perhaps the strongest test of hardship measures is to examine how well they predict one of these undesirable outcomes. If hardship is a useful measure of disadvantage, it should be able to predict such negative outcomes. If hardship has greater validity and reliability than poverty, it should predict these negative outcomes even more strongly than poverty does. In this paper, this test is accomplished by examining the effect of hardship on high school dropout among young people.

Table 6 shows the results of a logistic regression of dropout status on hardship and other variables. Included in this regression are the 1501 households from the 1992 panel with children aged 15 to 20 enrolled in high school at the time of the interview when hardship status was determined. The sample 1992 panel followed respondents a full year after reports of hardship. Dropouts were those teenagers who were enrolled in the first interview, but one year later were not enrolled in school and not graduates. The regression controls household education, poverty, food stamp receipt and several relevant demographic characteristics, but the main interest is in the effect of hardship. Of various hardship indicators tested, the experience of multiple hardships is the one that seemed to have the strongest effect after controls. The value of the parameter is reduced about one third from the value without the poverty, education and food stamp controls (this value is .73, see table 7), but remains significant at the 5 percent level. By contrast, the effect of poverty is essentially unchanged (a reduction of 14 percent) from the value it takes without hardship in the regression. If hardship were as strong a predictor as poverty, one would expect to see a more or less equal amounts of change in these variables. That this is not so provides some indication that hardship is not as strong a predictor.4

Table 6 Effect of Hardship, Economic and Demographic Characteristics on High School Dropout (2k)

Table 7 shows the effects of four ways of measuring hardship. In these regressions there are controls only for sex and race of household heads and number and age of children at risk of dropout. Experiencing any problem paying bills, includes all types of hardship aside from inability to visit a doctor or dentist. The second measure, being unable to visit the doctor or dentist, is the complement of the first. The last is whether the household experienced more than one type of hardship. All these three have large, significant effects on the probability of dropping out. This confirms the basic validity of these measures. When the three are put together in a single regression, the multiple hardship indicator is the one that comes closest to retaining significant predictive power.

Table 7 Effect of Hardship, Economic and Demographic Characteristics on High School Dropout (4k)

The next regression tests the reasonableness of an approach suggested by Mayer and Jencks and adopted by others (Mayer and Jencks 1993; Short and Shea 1995; Bauman 1997). They added the number of different hardships experienced by a household into a scale running from 0 for those who experienced no hardships, to 7 for those who experienced all seven types of hardship. The assumption underlying this approach is that successively higher numbers of hardships are indicative of qualitatively worsening levels of material well-being. This would presumably be reflected in an increased probability of experiencing dropout among households with higher numbers of hardships. This pattern is confirmed by the regression shown in table 7. The coefficients reflecting the effect of hardship on dropout increase monotonically through the first four levels. Those with a higher number of hardships have varied coefficients, reflecting the small number of cases involved. However, the hypothesis that all coefficients form a monotonic and equal-size step function can't be rejected.

These regression results show that basic needs measures do indeed predict high school dropout, but that they do not mediate the effects of poverty. In fact, poverty and education of household head explain much -- though not all -- of the effect of hardship on high school dropout. Material hardship therefore, seems to be, in part, a weak measure of poverty, and, in part, an influence on dropout that works independently of poverty and low education.


Reliability

Up to this point, the discussion has focused on the substantive meaning of hardship measures. Before reaching any final conclusions, it is worthwhile to take a look at how reliably they perform as measures. One way to do this is to observe their consistency across different administrations. Table 8 compares selected results from the three panels of SIPP basic needs questions to similar questions asked of a representative sample of the City of Chicago in the mid 1980s, and a study of low-income single mothers in four cities in the early 1990s (Cook et al. 1989; Edin and Lein 1997). Because neither of the comparison surveys is nationally representative, it is best to examine relative levels of prevalence within each sample, rather than overall prevalence. In general, these figures match up fairly well. Being evicted is an uncommon type of hardship in all surveys. By comparison, having utilities or phone shut off, foregoing needed medical/dental care, or going without food is somewhat more prevalent.

Table 8 Comparison of Basic Needs Measures with Other Studies of Hardship (2k)

Despite the broad agreement found in table 8, there is evidence that answers from the SIPP do not track well over time, and that they are sensitive to sample selection and attrition bias. Table 9 shows the effect of being from different panels on the probability of experiencing multiple hardships. The third column repeats results that were shown in table 4, above. The first column shows similar results with no other controls besides the panel indicators. The middle column shows the same results as in the first column, but with an added variable indicating whether the household respondent was also interviewed in the last wave of his or her panel.

Table 9 Effect of Panel, Poverty, Age and Other Economic and Demographic Factors on Experience of Multiple Hardships (1k)

Column 1 of table 9 shows that those who responded in the 1992 panel had a slightly higher probability of experiencing hardship compared to those interviewed in the 1991 panel. Those interviewed in the 1993 panel had a much lower probability of experiencing hardship. This is notable, because there was strong agreement in the results from the two panels with the same time of administration (the hardship questions in the 1991 and 1992 panels were administered in fall of 1992, the hardship questions in the 1993 panel were administered in fall of 1995).

Aside from time of administration there are several possible explanations for this effect. One has to do with the age of the panel. The 1993 results are based on interviews in the last wave (wave 9) of the panel -- nearly 3 years after the first round of interviews. Substantial attrition had taken place by that time. By contrast, the interviews for the 1991 and 1992 panels took place much earlier. To check the possibility of attrition bias, a dummy variable was added for those members of the 1991 and 1992 panels that were present at the completion of those panels. Since 1993 respondents were all present at the completion of their panel, they were also given a value of one on this dummy variable. The result is shown in column 2. Being a late-panel respondent was associated with a significantly lower probability of experiencing hardship. However, the coefficient on 1993 panel membership remained essentially unchanged. Attrition, while significant, had no real effect on differences between panels. The explanation is that attrition from the 1991 and 1992 panels wasn't large enough to make much of a difference in overall hardship rates. Those who dropped out of the panels had higher hardship, but there were too few to affect things greatly.

Another possible explanation for differences between panels is differences in economic circumstances. The period from 1992 to 1995 was one of economic recovery. However, it is uncertain how much economic gain was experienced by those with incomes low enough to report substantial rates of hardship. For example, figures from the Consumer Expenditure Survey show that income increased 6 percent and expenditures 5 percent from 1993 to 1995. However, among those households with the lowest levels of expenditures, income actually fell. Expenditures did rise for the group with lowest consumption levels, but much of that rise was due to increased housing expenditures. During the same period, food expenditures fell slightly (U. S. Bureau of Labor Statistics undated).

Many of the hardship measures examined here focus on housing -- paying rent and utilities. Two reports using data from the 1995 American Housing Survey concluded that the low-income housing market worsened during the early 1990s. For example, the Department of Housing and Urban Development reported that the percentage of renters and homeowners paying greater than half their income for housing increased slightly between 1991 and 1995 (though the change is not significant), and they report that there was a loss of 9 percent of rental units affordable to very low income families (U.S. Department of Housing and Urban Development 1998). A study by the Center on Budget and Policy Priorities reached similar conclusions (Daskal 1998). However, the respondents in the SIPP panels showed a significant decrease in problems paying rent or mortgage, in evictions, in paying utilities and in utility shutoffs in 1995 compared to 1992 (see table 5).

To test whether changes in income might explain the difference in reported hardship across panels, one can examine this difference after control for income and other characteristics that affect hardship. The third column in table 9 shows this result. The coefficient for being a late panel respondent goes to near zero, but the coefficient for the 1993 panel is unchanged. The reason that attriting panel members had higher hardship was that they had lower income (and other measurable characteristics associated with lower hardship). However, this explanation does not apply to differences between members of the 1993 panel and members of the 1991 and 1992 panels. The lower hardship reported by members of the 1993 panel were not produced by improvements in the economic conditions in which people found themselves.5

Another explanation for the lower levels of hardship expressed in the 1993 panel is an improvement in outlook. Although the level of income may not explain changes in hardship, households may have been less likely to report hardship if they were feeling more optimistic about their economic circumstances. For example, a recent late rent payment may be reported in a family that has continuing problems making the rent, but may be overlooked by a family that has new income sources that make timely rent payment in the future more likely. There is considerable evidence that the period 1992 to 1995 was characterized by rising economic optimism in the general population. Table 10 shows percentage responses to questions posed in a number of national polls (Dougherty et al. 1997; Newport 1996; National Election Studies undated). There is a clear trend towards favorable opinions, which are probably based either on improved personal economic circumstances, or on observation of improved economic conditions generally. It is possible that this optimism had an effect on hardship reports, but there is no direct evidence linking the two.

Table 10 Responses to Public Opinion Survey Questions on 1992 to 1996 (2k)

Overall, the hardship measures do exhibit a broad consistency from one application to another, but do not seem to be reliable enough to track trends over time. The reason for the shift between panels remains something of a mystery. It may be that there is an element of subjectivity that enters into these measures. But for whatever reason, they seem less desirable than might be hoped as a "benchmark" for measuring change, and maybe for measuring other types of differences as well.


Conclusion

The evidence here indicates that hardship measures provide a valid and useful measure of household well-being. They are strongly related to poverty and to other factors correlated with poverty, such as education, race, ethnicity, employment status, homeownership status and health insurance status. And material hardship, in addition to being undesirable in and of itself, clearly also leads to other undesirable outcomes. In this paper it has been established that one such outcome -- high school dropout -- is influenced by hardship with and without controls for poverty and other related factors.

On the other hand, hardship measures cannot substitute for poverty as a measure of well being. There are other dimensions to hardship that are not strictly correlated with poverty, and there is some evidence that hardship might not be reliably measured over time. Hardship should be considered alongside other measures of well-being. Those who use it as an outcome measure or as a way to calibrate other measures of poverty and well-being need to use care.

Some of the problems may have to do with the specific set of questions used in the SIPP. Respondents who feel responsible for bringing on their financial problems have an incentive to minimize their reported problems paying bills. It will be difficult or impossible to eliminate these problems, but it may be worthwhile to work with question wording to minimize this problem.

Several factors seem to influence hardship in addition to income and willingness of respondents to report hardship. These include access to resources besides income, ability to manage resources and ability to live within means. This is evidenced especially by significant effects of factors such as homeownership, education, and age even when poverty level is controlled. Age, especially, seems to play an independent role in its influence on hardship. Coefficients of age are relatively unaffected by other controls or interactions. It is also notable that households with children report higher hardship than other households even with other controls in place. Those who wish to use material hardship to compare the well being of families of different sizes, types and ages should be careful not to draw overly broad conclusions, at least until the reasons for these age and family patterns are better understood.

Hardship has a significant impact on high school dropout. Much, but not all of this effect is explained by poverty, and little of the effect of poverty seems to be explained by hardship. That hardship retains a significant effect even with other controls in place shows its value as an independent measure of well being. Despite the overlap in effects of poverty and hardship, the latter should probably not be treated as a proxy for the former.

Finally, there was a shift in responses between panels that do not correspond well with shifts in other measures of need. There is also a correlation between hardship and panel attrition that can affect results. These aspects of unreliability need to be considered along with some of the differences found across population groups, including age differences. All point to the fact that more needs to be known about these measures if they are to be used broadly for comparisons among groups and across time.

In sum, measures of material hardship provide valuable insight into the material circumstances of people in the population, but must be used with some caution, and don't show great promise of standing in for traditional poverty measures, at least at this stage of development.


References

Bauman, Kurt. 1997. Shifting Family Definitions: The Effect of Cohabitation and Other Nonfamily Household Relationships on Measures of Poverty. Discussion Paper, 1123-97. Madison, Wisconsin: Institute for Research on Poverty.

Betson, David M. 1990. "Alternative Estimates of the Cost of Children from the 1980-86 Consumer Expenditure Survey." Report to the US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

Citro, Constance F. and Robert T. Michael. 1995. Measuring Poverty: A New Approach. Washington, D.C.: National Academy Press.

Cook, Fay Lomax, et al. 1986. Stability and Change in Economic Hardship: Chicago 1983-1985. Evanston, IL: Center for Urban Affairs and Policy Research, Northwestern University.

Daskal, Jennifer. 1998. In Search of Shelter: The Growing Shortage of Affordable Rental Housing. Washington, D.C.: Center on Budget and Policy Priorities.

Dougherty, Regina, Everett C. Ladd, David Wilber and Lynn Zayachkiwsky. 1997. America at the Polls: 1996. Storrs, Connecticut: Roper Center, University of Connecticut.

Edin, Kathryn. 1991. "Surviving the Welfare System: How AFDC Recipients Make Ends Meet in Chicago." Social Problems, 38(4), November, 462-474.

Edin, Kathryn and Laura Lein. 1997a. Making Ends Meet: How Single Mothers Survive Welfare and Low-Wage Work. New York: Russell Sage Foundation.

Edin, Kathryn, and Laura Lein. 1997b. "Work, Welfare and Single Mothers' Economic Survival Strategies." American Sociological Review, 62(2), April, 253-266.

Johnson, David S. and Timothy M. Smeeding. 1998. "Measuring the Trends in Inequality of Individuals and Families: Income and Consumption." Washington, DC, March.

Lazear, Edward P. and Robert T. Michael. 1988. Allocation of Income Within the Household. University of Chicago Press.

Mayer, Susan E. and Christopher Jencks. 1989. "Poverty and the Distribution of Material Hardship." Journal of Human Resources 24(1):88-114.

Mayer, Susan and Christopher Jencks. 1993. "Recent Trends in Economic Inequality in the United States: Income Vs. Expenditures Vs. Material Well-Being." In Poverty and Prosperity in the USA in the Late Twentieth Century, edited by Dimitri Papadimitriou and Edward Wolff. London: MacMillan.

Newport, Frank. 1996. "Public Satisfied with Way Things Are Going." The Gallup Poll Monthly. 374(November):23-29.

The National Election Studies. [Undated.] The NES Guide to Public Opinion and Electoral Behavior (http://www.umich.edu/~nes/nesguide/nesguide.htm). Ann Arbor, MI: University of Michigan, Center for Political Studies [producer and distributor].

Ruggles, Patricia. 1990. Drawing the Line : Alternative Poverty Measures and Their Implications for Public Policy. Urban Institute Press ; Lanham, MD : Distributed by University Press of America.

Short, Kathleen, and Martina Shea. 1995. "Beyond Poverty, Extended Measures of Well-Being: 1992." Current Population Reports, Series P20-50RV.

U.S. Department of Housing and Urban Development. 1998. Rental Housing Assistance -- The Crisis Continues. The 1997 Report to Congress on Worst Case Housing Needs. Washington, D.C.: Office of Policy Development and Research, U.S. Department of Housing and Urban Development.

U.S. Bureau of Labor Statistics. [Undated.] Consumer Expenditure Survey Standard Bulletin Tables (http://stats.bls.gov/csxstnd.htm). Washington, D.C.: Bureau of Labor Statistics.


Notes

  1. The terms "household head" and "reference person" are used interchangeably in this paper. It would be more proper to use the latter term, but the former term is probably more widely understood. In the SIPP, the reference person is the person in whose name a dwelling is owned or rented. If there is more than one person in this position, the first person mentioned by the original household respondent is taken to be the reference person.

  2. It is debatable whether legislative trends were designed to address specific hardships rather than prevent certain stereotyped misuses of funds aimed, at least in theory, at children's well being. Current policies do not focus solely on provision of food and shelter, but on provision of food and shelter to families that meet an income threshold. If we were to take the policy at face value, then, we would conclude that the public's concern is not with hunger, but with hunger among those with low income. It is difficult to understand the rationale for such a concern.

    In either case, though, provision of food and shelter could be thought of as "proxies" for the desired policies, within the constraints of the current political system. If so, it might be argued that it is reasonable to adopt the same set of proxies in social science research. On the other hand, better proxies may indeed exist in social science research, and researchers need not settle for definitions of hardship and poverty formed within these constraints.

  3. Exploratory factor analysis on this set of variables from the 1991 and 1992 panels showed that visiting the doctor and the dentist loaded on a separate factor from that which united the other variables examined here. However, this pattern was not confirmed with the 1993 panel.

  4. A formal test could be undertaken using Cox's non-nested testing approach, but the binary form of the dependent variable limits its applicability. Likewise, formal significance test of the change in coefficients could be undertaken using the approach suggested by Clogg and his colleagues, but application to binary dependent variables requires the assumption of fixed regressors, which is doubtful in this case.

  5. It is possible that some of the improvement in hardship had to do with changes in economic circumstances, rather than the circumstances themselves. This would happen if people avoided hardship when they were accustomed to a certain living standard, and experienced hardship only when their incomes fell. It would also happen if people reported hardship only when they felt, subjectively, that their situation was worsening. A test for this is to look at the effects of lagged values of income on hardship. If lagged values are substantially smaller, then part of the explanation for a change in coefficients could be the effect of improved circumstances. Unfortunately, there are numerous other reasons that coefficients could grow between a lagged and current income value, making this test less than perfectly clean. Nonetheless, such a test was undertaken, and a substantial difference between coefficients was found. However, adding the lagged value of income explained none of the between-panel change in measured hardship.

Appendix

Population Division Working Papers

Source: U.S. Census Bureau, Population Division,
Education & Social Stratification Branch

Author: Kurt Bauman
Last Revised: October 31, 2011 at 10:03:11 PM