|
Supporting Document Census Bureau Standard
Authored by: Examples of Revisions of Survey Descriptions to Meet Standard Three examples of how current Census Bureau news releases and reports might be revised to comply with the new standard are given below. The first two describe demographic surveys, and the third describes an economic survey. The first and third are news releases, while the second is a longer report. It is anticipated that the amount of information provided to meet the standard would be adjusted to the length of a report or news release. Most survey details (except response rates) are readily available in complete summaries of both demographic (Survey Abstracts, Demographic Surveys Division, June 2002) and economic programs (Economic Data Programs, Jan. 2002, 3rd edition). 1. "Nation's Household Income Stable in 2000, Poverty Rate Virtually Equals Record Low, Census Bureau Reports" (October 10, 2001 News Release) Survey Description included in news release: "These data are from the March 2001 Current Population Survey. As with all survey data, they are subject to sampling and nonsampling error." Issues: No information about the survey is presented, except its name and date. The statement about error is uninformative. A reader may not understand that "sampling" is meant to modify "error." The reader isn't told what "sampling error" or "nonsampling error" is. Proposed modification intended to meet minimum standard: "The Current Population Survey is based on a national probability sample of housing units. About 50,000 households were interviewed the week of March 18-25, 2001. The margin of sampling error for the national poverty rate gives a 90 percent confidence interval around the estimate of plus or minus 0.3 percentage points. For further information, visit http://www.bls.census.gov/cps/bmethdoc.htm." 2. "Did You Know? Homes Account for 44 Percent of All Wealth" Findings from the SIPP: 1995. Current Population Reports (CPR) P70-75, issued May 2001. Survey Description included in the CPR: More Information: [...] The data [...] were collected by the Survey of Income and Program Participation (SIPP). The sample of households in the SIPP is divided into four interview groups called rotation groups. Each month, one of the four rotation groups is interviewed about the previous 4 months (the reference period). The asset and liability data in this report were collected from the first rotation group in February 1995 and refer to the last day of January 1995; the second rotation group was interviewed in March 1995, and their data refer to February 1995, and so on. As a result, the data presented in this report represent an average of the assets and liabilities of people at the end of January, February, March, and April 1995 [...]. Accuracy and Reliability of the Data. This report is one of a series that presents information of current policy interest. Statistics from sample surveys are subject to sampling and nonsampling error. All comparisons presented in this report have taken sampling error into account and meet the Census Bureau's standards for statistical significance. Nonsampling errors in surveys may be attributed to a variety of sources, such as how the survey was designed, how respondents interpret questions, how willing and able respondents are to provide correct answers, and how accurately answers are coded and classified. The Census Bureau employs quality control procedures throughout the production process - including the overall designing of surveys, testing the wording of questions, reviewing the work of interviewers and coders, and conducting statistical review of reports. For further information on statistical standards and the computation and use of standard errors, contact (name). Issues: No basic information is provided about the survey, except a long explanation of rotation groups. The "Accuracy and Reliability" statement is general and uninformative about the survey. It does not provide useful information about the accuracy and reliability of the data. Its first sentence is misplaced, and seems to belong at the beginning of the report. It refers to "the Census Bureau's standards for statistical significance" without saying what they are or where to find them. Proposed modification intended to meet minimum standard: Source of Data: The data are from the Survey of Income and Program Participation (SIPP), a continuing longitudinal survey conducted and sponsored by the Census Bureau. The survey is based on a national probability sample that represents the U.S. civilian, noninstitutionalized population. Results reported here are based on the panel of 16,500 sample households first interviewed in 1993. Interviews were conducted in person or by telephone every four months for 2= years. All household members age 15 and older were eligible to be interviewed, with proxy response permitted for household members not available for interviewing. Respondents provided information on income, program participation, and other topics for the four-month period preceding the interview. The data presented in this report are based upon the seventh interview for people in the 1993 panel, and represent an average of peoples' assets and liabilities at the end of January, February, March, and April 1995. The response rate for interviews used in compiling this report was 76 percent, which reflects the cumulative loss over the seven interviews. The Census Bureau employs quality control procedures throughout the survey process, including sample design and selection, testing the wording of questions, reviewing the work of interviewers and coders, and conducting statistical review of reports to minimize errors in reporting, coding, and processing the data. As is the case for all surveys, the statistics from this survey are subject to sampling error. They may also be affected by missing data because households did not participate or provided incomplete information. Data are imputed and weighted to account for sampling probabilities and to adjust for missing information. All comparisons in this report have undergone statistical testing and are significant at the 90 percent confidence level unless otherwise noted. More information about the survey sampling errors and the quality of the data may be found at www.sipp.census.gov/sipp . 3. "Advance Monthly Sales for Retail and Food Services: January 2002," (February 13, 2002, News Release) Survey Description included in news release: Reliability of Estimates: The Advance estimates are based on a small subsample of the Census Bureau's full retail sales sample. Estimates from the Advance and the subsequent full survey can differ because of the earlier reporting in the Advance and because of sampling variability present in each survey. The margin of sampling error, as used on page 1, gives a range about the estimate which is a 90 percent confidence interval. If, for example, the trend estimate is +1.2 percent and the standard error is 0.9 percent, then the margin of sampling error is 11.65 x 0.9 percent or 11.5 percent, and the 90 percent confidence interval is -0.3 percent to +2.7 percent. If the interval contains 0, it is uncertain whether there was an increase or decrease. For a monthly level, the coefficient of variation (CV) is given. The resulting confidence interval is the estimated value 1 1.65 W CV W (the estimated value). Estimates of sampling variability are given in Table 3. Estimates from the Advance and the full survey are subject to nonsampling errors. Such errors can occur because of nonresponse, insufficient coverage of the universe of retail businesses, and response errors. Additionally, estimates of sales prior to March 2001 have been restated from SIC-based estimates. The restatement methodology may have introduced additional nonsampling error. Precautionary steps are taken to minimize nonsampling errors, but their magnitude is not directly measured. Issues: Although the news release includes a careful and comprehensive discussion of confidence intervals for the estimates, there is virtually no descriptive information about the survey itself, beyond a statement that the estimates are affected by errors due to nonresponse, coverage, response errors, and the effects of converting from SIC-based estimates. It is uncertain how well readers understand the information about confidence limits. Proposed modification intended to meet minimum standard: Source of data: The Advance Monthly Retail Trade Survey (MARTS) is conducted by the Census Bureau each month to provide an early indication of sales of retail and food service companies. The MARTS is a mail-out/mail-back survey of a sub-sample of about 5,000 firms selected from the Monthly Retail Trade Survey (MRTS). Responding firms account for approximately X percent of the MARTS estimate. Estimates may be affected by missing or erroneous data due to nonresponse, insufficient coverage of the universe of retail businesses, and response errors. Errors may have been introduced by the method used to restate the SIC-based estimates of sales prior to March 2001. Although precautions are taken to minimize nonsampling errors, their magnitude is not directly measured. More information about the survey is at www.census.gov/svsd/www/advtable.html. Reliability of Estimates: The margin of sampling error, as used on page 1, gives a range about the estimate which is a 90 percent confidence interval. If, for example, the trend estimate is +1.2 percent and the standard error is 0.9 percent, then the margin of sampling error is 11.65 x 0.9 percent or 11.5 percent, and the 90 percent confidence interval is -0.3 percent to +2.7 percent. If the interval contains 0, it is uncertain whether there was an increase or decrease. For monthly level, the coefficient of variation (CV) is given, and the confidence interval is the estimated value 1 1.65 W CV W (the estimated value). Estimates of sampling variability are given in Table 3. Document Management & Control
The most current version of
this document is maintained on the Census Bureau Intranet and may Category:
Standard |