U.S. Department of Commerce

Information Quality

Skip top of page navigation
Census.gov Information Quality Main Statistical Quality Standards › Producing Estimates and Measures: Appendix D3-A

Appendix D3-A: Requirements for Calculating and Reporting Response Rates: Demographic Surveys and Decennial Censuses


1.  Terms and Variables

The variables needed to calculate demographic survey and decennial census response rates are based on classifications suggested by the American Association for Public Opinion Research (AAPOR), 2008. This effort helps to ensure consistency to external standards while allowing the Census Bureau to adapt the classification to our specific circumstances.

The terms and variables are partitioned into three sections. The first section describes eligibility status. Variables in this section distinguish among sample units that are known to be eligible for data collection, are known to be ineligible for data collection, or have an unknown eligibility for data collection. The data collection target population guides the distinction between eligible and ineligible units. The second section describes the response status for eligible sample units. The third section provides detail on nonrespondents by identifying the type of (or the reason for) the nonresponse.

1.1 Eligibility Status

The total number of units selected for a sample is defined as n. These units can be classified by their eligibility status: eligible for data collection (E), ineligible for data collection (I), or of unknown eligibility (U). The target population determines the classification of a unit as eligible or ineligible. The target population refers to persons, households, or other units upon which inferences (estimates) are made. Specific units may be considered eligible for one census or survey but ineligible for another, depending upon the target population. For example, in a survey of housing, vacant units may be part of the target population, but these same vacant units may be outside the target population in an income survey and would therefore be classified as ineligible.


Variable pi (Probability of selection)
Definition Probability of selecting a unit for the sample, including all subsampling, even subsampling for nonresponse follow-up.

 

Variable wii (Sample weight)
Definition The inverse of the final probability of selecting a unit for the sample, including all subsampling, such as subsampling for nonresponse follow-up. formula

 

Term E (Eligible)
Definition The weighted count of sample units that are eligible for data collection. A person, household, or other unit is eligible if an attempt has been made to collect data and the unit is confirmed to be a member of the target population. Both occupied and vacant units can be considered eligible.
Variable ei – An indicator variable for whether a unit selected for the sample is eligible for data collection. If a sample unit is eligible, ei = 1, else ei = 0.
Computation Sum of the sample weight for all eligible units.
formula
Reference Equivalent to the sum of AAPOR "Interview" disposition code (1.0) and “Eligible, non-interview” disposition code (2.0).

 

Term I (Ineligible)
Definition The weighted count of sample units that are ineligible for data collection. This is the number of units for which an attempt has been made to collect data and it is confirmed that the unit is not a member of the target population.
Variable ii – An indicator variable for whether a unit selected for the sample is confirmed as not being a member of the target population at the time of data collection. Information confirming ineligibility may come from observation, from a respondent, or from another source. Some examples of ineligible units include: demolished structure, entire household in armed forces, unit under construction, unit screened out, nonresidential unit, fax/data line or disconnected number (in random-digit dial surveys), and vacant unit. If a sample unit is ineligible, ii = 1, else ii = 0.
Computation Sum of the sample weight for all ineligible units.
formula
Reference Equivalent to AAPOR “Not Eligible” disposition code (4.0).

 

Term U (Unknown eligible)
Definition weighted count of sample units for which eligibility is unknown.
Variable ui – An indicator variable for whether the eligibility of a unit selected for the sample could not be determined.  This occurs if data are not collected from a unit and there is no information available about whether or not the unit is a member of the target population.  Some examples of units with unknown eligibility include: unable to locate unit, unable to reach/unsafe area, address never assigned/worked, or number always busy or call screening/blocking (in random digit dial surveys). If a sample unit is of unknown eligibility, ui = 1, else ui = 0.
Computation Sum of the sample weight for all units with an unknown eligibility. formula
Note Surveys that have large number of units with unknown eligibility (e.g., random-digit-dial surveys) may estimate the proportion of cases of unknown eligibility that are eligible, ee.  This estimated proportion may be used to adjust the estimates of I and E.  The survey must have a defensible basis for estimating ee (e.g., assume that the ratio of eligible to not eligible cases among the known cases applies to the unknown cases).  Without a defensible basis, ee may not be used to adjust the estimates of I and E.  The number of eligible units may be adjusted by adding (ee * U) to E.  The number of ineligible units may be adjusted by adding (U ‑ (ee U)) to I.  The basis for estimating ee must be stated explicitly and the justification described clearly.
Reference Equivalent to AAPOR “Unknown Eligibility, Non-Interview” disposition code (3.0).

 

Term T (Total count)
Definition The weighted count of all units (eligible, ineligible, and of unknown eligibility) selected for the sample.
Computation Sum of the sample weights for the eligibility status outcome of all units. formula
The relationship between E, I, U, and T is T = E + I + U.  For the ith unit ei + ii + ui = 1.

1.2 Response Status

Response status is determined only for eligible sample units. The definition of sufficient data for a unit to be classified as a response will vary across surveys and will impact the count of responding units.


Term R (Response)
Definition The weighted count of eligible sample units with sufficient data to be classified as a response. In a multi-mode survey or census, responses may be obtained by mail, Internet, telephone, fax, touch-tone data entry/voice recognition, or personal visit.
Variable ri – An indicator variable for whether an eligible unit selected for the sample responded to the survey and provided sufficient data.  If a unit responded, ri = 1 else ri = 0 (note ri = 0 for units classified as U or I and units that did not respond with sufficient data).
Computation Sum of the sample weights for all response outcomes.
formula 
Reference Equivalent to AAPOR I+P (complete interviews + partial interviews) disposition codes (1.1) and (1.2).

1.3 Reasons for Nonresponse

To improve interpretation of the response rate and better manage resources, it is recommended that whenever possible, reasons for (or types of) nonresponse be measured. Six specific terms describing nonresponse reasons are defined below. These terms (REF, NOH, TA, LB, INSF, and OTH) define specific nonresponse reasons for sample units. 


Term REF (Refusal)
Definition The weighted count of eligible sample units that refused to respond to the survey.
Variable refi – An indicator variable for whether an eligible sample unit refused to respond to the survey. If a unit refused to respond, refi = 1, else refi = 0.
Computation Sum of the sample weights for all “refusal” outcomes.
formula
Reference Equivalent to AAPOR “R” (refusal and break-off) – disposition code (2.10).

 

Term NOH (No one home)
Definition The weighted count of eligible sample units that did not respond because no one was found at home during the interviewing period.
Variable nohi – An indicator variable for whether an eligible sample unit did not respond to the survey because no one was found at home during the interviewing period.  If a unit was “no one home,” nohi = 1, else nohi = 0.
Computation Sum of the sample weights for all “no one home” outcomes. formula
Reference Equivalent to AAPOR “No one at residence” – disposition code (2.24).

 

Term TA (Temporarily absent)
Definition The weighted count of eligible sample units that did not respond because the occupants were temporarily absent during the interviewing period.
Variable tai – An indicator variable for whether an eligible sample unit did not respond to the survey because the occupants were temporarily absent during the interviewing period.  If a unit was “temporarily absent,” tai = 1, else tai = 0.
Computation Sum of the sample weights for all “temporarily absent” outcomes. formula
Reference Equivalent to AAPOR “Respondent away/unavailable” – disposition code (2.25).

 

Term LB  (Language barrier)
Definition The weighted count of eligible sample units that did not respond because an interviewer or interpreter was not available to conduct the interview in the required language.
Variable lbi – An indicator variable for whether an eligible sample unit selected for the sample did not respond to the survey because an interviewer or interpreter was not available to conduct the interview in the required language.  If a unit did not respond due to a language barrier, lbi = 1, else lbi = 0.
Computation Sum of the sample weights for all “language barrier” outcomes.
formula
Reference Equivalent to AAPOR “Language” – disposition code (2.33).

 

Term INSF  (Insufficient data)
Definition The weighted count of eligible sample units selected for the sample that participated but did not provide sufficient data to qualify as a response.
Variable insfi - An indicator variable for whether an eligible sample unit that was selected for the sample returned a questionnaire, but did not provide sufficient data to qualify as a response.  If a unit returned a questionnaire but fails to provide sufficient data to qualify as a response, insfi = 1, else insfi = 0.
Computation Sum of the sample weights for “insufficient data” outcomes. formula
Reference Equivalent to AAPOR “Break off” and “Break-off questionnaire too incomplete to process” – disposition code (2.12).

 

Term OTH  (Other nonresponse)
Definition The weighted count of sample units that did not respond for a reason other than refusal, no one home, language barrier, temporarily absent, insufficient data, or if a unit was classified as unknown eligibility.
Variable othi – An indicator variable for whether a unit selected for the sample was a nonresponse for a reason other than refusal, no one home, language barrier, temporarily absent, or insufficient data or if the unit was classified as unknown eligibility.  If a unit does not respond for reasons other than refusal, no one home, language barrier, temporarily absent, insufficient data, or if a unit was classified as unknown eligibility, othi = 1, else othi = 0.
Computation Sum of the sample weights for “other nonresponse” outcomes. formula
Reference Equivalent to AAPOR “Other,” “Dead,” “Physically or mentally unable,” and “Miscellaneous” – disposition codes (2.30), (2.31), (2.32), and (2.35).

2.  Unit Response and Nonresponse Rates

2.1  Primary Response Rates


Rate UR rate (Unit Response Rate)
Definition The ratio of responding units to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation UR rate = [R/(E+U)] * 100
Reference Equivalent to AAPOR Response Rate 2 (RR2).

 

Rate AR rate (Alternative Response Rate)
Definition The ratio of responding units to estimated eligible units (expressed as a percentage).
Computation AR rate = [R/[(E)+ee*U] * 100 where: ee = estimated proportion of cases of unknown eligibility that are actually eligible. The survey must have a defensible basis for estimating ee. If such a basis does not exist, then ee may not be used to adjust the estimates of I and E and the survey may not estimate the AR rate.
Reference Equivalent to AAPOR Response Rate 3 (RR3).

 

Rate URM rate (Cumulative Unit Response Rate for multistage surveys)
Definition The product of unit response rates for all stages of the survey
Computation URM rate = formulawhere, URj is the unit response rate at stage j of the survey and k is the total number of stages. If another equation yields a more accurate estimate of the cumulative unit response rate because it uses additional information about the frame, then that equation should be used. If the cumulative response rate is misleading or inaccurate, an explanation of the problems must be documented.

2.2  Detailed Eligibility and Nonresponse Rates

Rate REF rate (Refusal Rate)
Definition The ratio of units classified as “refusals” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation REF rate = [REF/(E+U)] * 100
Reference Equivalent to AAPOR Refusal Rate 1 (REF1).

 

Rate NOH rate (No One Home Rate)
Definition The ratio of units classified as “no one home” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation NOH rate = [NOH/(E+U)] * 100
Reference No AAPOR equivalent.

 

Rate TA rate (Temporary Absent Rate)
Definition The ratio of units classified as “temporarily absent” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation TA rate= [TA/(E+U)] * 100
Reference No AAPOR equivalent.

 

Rate LB rate (Language Barrier Rate)
Definition The ratio of units classified as “language barriers” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation LB rate = [LB/(E+U)] * 100
Reference No AAPOR equivalent.

 

Rate INSF rate (Insufficient Data Rate)
Definition The ratio of units classified as having “insufficient data” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation INSF rate = [INSF/(E+U)] * 100
Reference No AAPOR equivalent.

 

Rate OTH rate (Other Reason for Nonresponse Rate)
Definition The ratio of units classified as “other nonresponse” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation OTH rate = [OTH/(E+U)] * 100
Reference No AAPOR equivalent.

 

Rate U rate (Unknown Eligibility Rate)
Definition The ratio of units classified as having an “unknown eligibility” to the sum of eligible units and units of unknown eligibility (expressed as a percentage).
Computation U rate = [U/(E+U)] * 100
Reference No AAPOR equivalent.

3.  Item Response and Allocation Rates

3.1  Item Response Rates


Term IREQA (Weighted total of responses required for data item A)
Definition The weighted count of sample units for which a response to item A is required. A response is required for item A unless it is a valid skip item.
Variable ireqAi – An indicator variable for whether a response to item A is required. If a response is required, ireqAi = 1, else ireqAi = 0
Computation Sum of the sample weight for all units requiring a response to item A. formula

 

Term ITEMA (Total valid responses for data item A)
Definition The weighted count of sample units for which a valid response to item A is obtained.
Variable itemAi – An indicator variable for whether a valid response to item A is obtained. If a valid response is obtained, itemAi = 1, else itemAi = 0
Computation Sum of the sample weight for all units requiring a response to item A for which a valid response is obtained.
formula

 

Rate IRA rate (Item response rate for data item A)
Definition The ratio of the weighted count of units with a valid response to item A to the weighted count of units that required a response to item A.
Computation IRA rate = ITEMA / IREQA

 

Rate TIRA rate (Total item response rate for data item A)
Definition The product of the weighted item response rate for item A and either the unit response rate, reflecting the response rate to item A after accounting for both unit nonresponse and item nonresponse, or the cumulative unit response rate for multistage surveys.
Computation TIRA rate = IRA * UR or
TIRAM rate = IRA * URM

3.2  Item Allocation Rates

Item nonresponse is measured through the calculation of allocation rates.  Allocation involves using statistical procedures, such as within-household or nearest neighbor matrices populated by donors, to impute for missing values.


Term ALLOA (Total number of responses allocated for item A)
Definition The weighted count of sample units for which a response is allocated to item A.
Variable alloAi – An indicator variable for whether a response is allocated to item A. If a response is obtained, alloAi = 1, else alloAi = 0
Computation Sum of the sample weight for all units requiring a response to item A for which a response is allocated.
formula

 

Rate IAA rate (Item allocation rate for data item A)
Definition The ratio of the weighted count of units with an allocated response to item A to the weighted count of units that required a response to item A.
Computation IAA rate = ALLOA / IREQA = 1 - IRA rate

References

The American Association for Public Opinion Research. 2008. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th edition. Lenexa, Kansas: AAPOR.



Back to Main


Source: U.S. Census Bureau | Methodology and Standards Council |  Last Revised: August 04, 2011