WPC :O }eyO]׍[-rlѲb/2O%CY&cH;}=NAh:1qV"]H,e1GIyK Cr0ߩ^sTU: *$_C-84tn![оHdqֆ`+]y]KCM쿗0< Ӏ>aT3G5>B\:aT9 /   ii  @ FH:\MURAWSKI\LATESTMO\1CPOSTAG.TIFWPC   !!!"""###$$$%%%&&&'''((()))***+++,,,---...///000111222333444555666777888999:::;;;<<<===>>>???@@@AAABBBCCCDDDEEEFFFGGGHHHIIIJJJKKKLLLMMMNNNOOOPPPQQQRRRSSSTTTUUUVVVWWWXXXYYYZZZ[[[\\\]]]^^^___```aaabbbcccdddeeefffggghhhiiijjjkkklllmmmnnnooopppqqqrrrssstttuuuvvvwwwxxxyyyzzz{{{|||}}}~~~o}sϏO/??oL߯O? ?O??O???̅{3OD_/_3߬.7_/o.ߓ3o/_2;)ߜ?A?>2_?_O/+.? ?5_/o+_?1oo//_7) O:+?AO=?0/=?$o-+?._ O??4+0?/"4?)$_8+o@o=??O/;?'?.+?. ?3_,/?/$3?O*&_6+@=?OO. /? ?: O_/?.?*?. 2_ _o/ ,.o Oo? o0&_2O*/&6+@<O- O_?8_  _ .O*?.o1_? ,??- O 0&_1*O'5*? ?<￟? ߿_. ?7/ ??0Ͽ? ߂0o __2/ _-߿/ ?Ͽ. 2 ￿O 1￟? ߿_.Ͽ? OO 6 ς?_ <￟? ߿_ ??0??6/ ?2 o3 /3/O0 0o4? _ 3 ??2  7? ?> o? ?? ?O/__5/ /2 3 3?0 /0o/4? ? ?3 ?O2 / _7? ?> ? ?_ _0O5O _?2 3 20 ?/?4 _ 2 _2 / 6 > > O /4 _?2 3 ?3?/?0 ??/_o4 ? _2 3  _6 = O>  /?/3 ?2 ￏO?3? 2_1 ??/o4  2 3  5 ??=_>  .?3/ 2 Q? ?21 ?./4 _ 2 3 ? 5 Z>  ?.2 ?/2 Q????2?߿1 o.o￯4  2 ?3  _5 Z> ??.oM o$3 Qo??2?M .?Q  O2?4  /5 Z/?? =???.?N [ Q??1?N .?Q /?2??4 ? 5 Y O=????.?M \ QO??1?N .Q ??2??4 O 5 Y_ <????.?L \ Q/ ??1/N . Q ??2??4  4 X__ <????.?L/ o] Q? ??1 M . P ?O2??4  4 X <????.?/K ] Q?? o??1 M . oP ?2??4  4 X?// ;????.O J O^ Q? ??2 M .O 2??4 o 5 W O;????. ?J/ ^ Q? ??2 L .? oO  2??4 ? 5 Wo :????. H ^ Q?/ ??2_ K . N O3??4 /?5 W__ :????/ G ?_ Q? ??2 J . M _3??4  o5 V :????/ Fo _ Q? /??3 I /? L /4??4  5 V? 9????/ _D/ _ Q? ??3 H / K o4??4 _6 U _9????0 C ` Q? ??4/ OF 0O ?I o5??4 _6 U 8????1/ o@ ?` Q?o ???4 E 0 H /6??4  6 UO _ 8????1 ? _` Q? ??5?C 1F //6??4 _7 T  8????2= ` Q? ??6__A 2?D /7??4 O7 TO  7????3?; ` Q?O O??7_? 3oB __8??4 _ 8 So  _7????49O ` 8? ??8o> 4A ߿/:??4 _ _9 :  6????5/8? a!8?  ??9?< 5/?_;??4 / ?:!:? O 6????6_?6? a!8? ? o ??:/: 6?_=_=??4 ? O:!9  /6????85? a!8?  ??;o9 8<????4 ? o<!9_ 5????9_3? a!8?   ??=/O7 9??:o<??4=!9o _5????;/2? a!8? /  ???6 ;9:??4o>!9 4????=?0? a ߃9?  ??@_5 <_7 O/8??4o@ σ9/O 4????>/? a Q?  / ??B?_3 >O/6 o O6??4B P /4????@O.? a Q?  ??D/2 @?/5 _ 5??4?B P_ 3????B/- a Q?  ??F_2 B/4 ? 4??4  A O_o _3????D /, ` Q?o ???G_1 C_4  3??4 o @ O 2????Eo o, ` Q? ??I/ 0 E? 3 / 2??4  ?@ O/? 2????G + ` Q? ??K ?0 G 3 / O2??4  ? N ?2????H/ _+ O` Q?O O??L / H 2  1??4 ? > No 1????I_ +` Q? ??M O/ I/ 2  1??4  ?> N(o1????J/ +O _ Q? ??N . J 1 O 0??4 o = M)1????K O+ _ Q?? o??O . K 1  _0??4  < M?*0????L + O_ Q? ??P? o. L_ ?1 /0??4  O< L+?0????M + ^ Q? ??P ?. L 1  /??4 ? ; Lo,/??OM +_ ^ Q?/ ??Q./MO 1  /O4  : L?? o/_oM + /^ Q? ??Qo .?M 1  /4 _ O: K /o?N , ] Q? /??Q .OOM 1  /?4  9 KO? .??N, ?] Q? ??Q .o?O 1  /?4  8 Jo ?./ ON?, \ Q? ??Q.?O 1  / O4 / O8 J - OO?- \ Q?o ???Q?.OO1  05  7 J/o ./OOO. [ Q??Q_/N?1 / 0OO5 _ 6 I .OO_._?/ Z Po/?1o/Oo.ooo1  0_5  O6 I ? -OO.0/ X 1?O?1?O///??o1  /1O4  5 )_! ?- O/O/_0 /W _?1??1?/0/?2  2/O5 / 4 O?(?" -/O_/_?2 /#1 1??11_?/?2 O 2_5  /4 '# ?-_O?0?_3_ / ?1 1? ?12 o02  _4O?6 _ 3 '?o# -O? 0?o?4/ 1 1 ?1O3_ 0o3o  4 6  2o ?&$O O- ? o1?_5/ o1 o1 ?0_?413? O 6? o7  1? ?&?/% -? O _1?7/ 1 /1/ ?0?_5 oO24?  _ ?7 _7 / O1? %& _- _  ?2 O _9/ ߏ/-Ͽo ? O-_? ?0 ? 7 ?/2 ? ? 1߿ Ͽ_ /8 ?4￟/ o ?(ϯ_ ?/ "/_߿/ o+ ?  ? 3 o_ _;_/oￏ_ -?,?,o ?0 /? 9_ ?ߏ2 O 2(/: ? 5?o_(-"O o+ ?  OO/ 4?? >?/-/,?,_ ?0/_o? ; /_O 3Oo? 3'o= OO/ 6?O(-"? ?, O_/  /5!@O$/-?,? ?0!O=?4!/3%?/7/?'-" /-/ o6oC? 0-?,__?0@o65#?Co7 '-O"/o _9ϏO_G__2-,??1o?C_o9?6/_F_9'-?!o0__/?ϟo?/Lo/?5﮿,ϐ?6O/_HOO?_/O9ߝN_/;'߭!ߒ3_/ϟO?/?_G߿??/?ORo????_Ƈ;o???_OϏ????oHϯo???OϟO?/?_lϟO?/?_AJPO?[?_o/Oo?P/[_O Po[??/   $.O]  TO)F 0ooo/?_No%?c o  O&/-] _ TO+G O/?/L??$c  &/.] OU*OG .?Jo$c /'o._oWO,I0Ooϯ/H$c O(.O/`/_YO-I?0/ _G$Od ?(.a_o o?Z?-J0??O F%d ?(o /Z-??JO0o/ /)??O [oo.oJ/0O_?)? /\?O/o??-J/0_O?o(O] /o-I_0o__O ?/o _?߃?/o//__?/o? ߏ?/ _?o?? /o/?oO?? ?ߏ?O?O o_ o?? ߂o?__O?_ _o//?/O//o?ߏ?__/Oo? O?o _?_/O _?o___/O/_o o /??_ ?  ?_/  ?? ??__    OO_O__?/ _ O  o? ? ?_?/ ? ??? ??? _?O_ O /_o_? O? _ o _ // o  /  O?_/ oO   ?/ ??/  / ? ???? oo/ oo/ o_? __/?/ _ߟ/_? ?_?oO   O_/ o/ o o/o/  o _ OOo?? //_ _?߯_/o _/o __? _ ? _o /  ?oO  o / o_ ?/? ? /o ? ? ?o_ //? __ _//? _oO /o_? /? /oo/??  ? ?O/ _ /  oO ? O/  O o/ _  _/ o  _/? o//_? _ o??oo_? _ / Oo/_? /?O? ? /_O //? ?/ O o /?     _ ? _ ?OO_o   ?// ?_o_ /o_   ?/߿/?/_  / O??/? O  / /o _ o_O/ O//  ?? O_   !_/ / _ Oo/o   _O ?/ ? _O _ o/ ??// O _/ O?/_   o    /_?? O߿? /_ _ o/ ?_? O o" O ? ?/_? /? O/ o O/  /?   //O/  // O? __ o /  ?ooO _  ??#/O _ ? /o  _ /? ?_ O_   ?O_Oo_ ?  / _ ? O ?_O/?  /?O /O  #  ??? /? ?   _o   oo O OoO_? /O? O_O/ _/ o _o ?_ _ # / ?o O?o?O _  ?/OOo_ ?/??  _ _?o?/??__  ??_? ?? ? Oo _O? ?/  // / #o O  ?o   O?O_O//  _  //__ ? OO_o/_/_  ?? /_?_/o _ oo  ?/ O$   ?O? _o ? ?/  /?  /  _// _/ o/ oo  _ ooo/  ?/ _  _  ?$   o O  ?_ OO   _? OO _?ߏ _/ O_ o _ _/o ?? _ _oO  _? /? ? /O o$? O/ ?/ O_O  o_??   o  o OO?o _?_/o  ?o?? / /? O  /o /o   o  O ?$? ? O__??/O  ?/O? _"_  ?O   _  /O   ??_ _ ?// o /o o ???o? ?    _ # o _?oo o O/O%/ ?  _  ?? _ ?_O O_??oO/  /  ?? o?O o /  ? o$  /o?O?_  /+/   /?  _ _ /? _O/o/?/  O  o ?? ? _ _ _/O?$o  _?oo_? ?O ?+_ O    O  _/? oO   _ o o _ //  /__O?o$/  / ?Oo_?//  ?+ O _   _ O  _o/O?? ?/?  _ //__o   _  ?_$? ? O O///_ ? o_  /O    _  /OOoO??//o??_ O_ / /? o/ / /  ??  o ?#_  o /__O ?_ O?/  o/   _ O  ? O  ?/_?/ ? oo_??o  ?  /?O O/  _?? o /  // ?O$ / o?/ ? ?/ __o O O/  o  /? o o O   O  o / _? ?/   O /O _ ??  ? _ ? ?O/O/$? _O/ o O O ?_/  _?? /  _ /  o O?? ?/ OO/?o? oo? o ?o _ O/    _oo/_$ / ? ߿  ?_/? OOOߏ/ O ? _ oO  ? oOO_?ߟ?o ?/?O/oO// _O / ?_ _ ??????O?   /!o O_ߟ/O  oO/oO? _o/??? o/Ϗo / _ oO? /o?o????/o??__oo ? ?OO/      ? ?  _!O / ? ?O ?_  o _  ?  /  o /?_ _??/_   ?  /?? / _   _ o_?/ Oo  ? oo O? _?  o ? /?/ ? O  ? ? //? o  O  ?  o  _   //_ ?O/?߄  O?/!ߊOO/oτ?o ?OO?_  _? o?/_ ߄  OO?o?oO _?O  //o ?? 4߿%_V _@aD\߿m߿`Ͽ*Ͽ)Ͽ@\bi\Ooj_\_Poj?\?Q /jo[oQ_? j[Qoko[oQ??l/[/Sn?Z?S_ /noZX߯oq???o?????/??_O??/?__????????_??o????o?/???/???????_/???o??__???????/OO?o??_????/o??o???/??O_????_????//??_???????__?/??OO??????OO??_o__??_o_????O???o߿/??__????/?????/_??/??O????_????o??/?__???O/????????o????o_????_????/O??o??o_????/??/????o??????߿???__O/??oϿ/O??????/???_O/oO??/????O????/?/_???_????/???_/???_o_????/?o???__??o????oo???/?/???/?????????/O???????/_O?OO?????o????_???O/???o????/O???O??o????O??o??_???????O????o_?/??O????????????o??O_?/??/OO??o_o/???o?_Oo??/??OO?O???????o??o?/O?????O/OO??o????O/o/OO??_?/OO_OO__?o/?oOoooo_O?OO/?O??O/??o/O_?O/O_/_????/?/O/O??/O__?/??_??O_/?_O??__/???o_O?_?oO??o?/?O_ooo?OO?o?_/oo?_??O?o???/?O_?///??_oO?_?_/O?__?O_/ߏ/Ͽo?????????????????O_????/??߿Ͽ_/?￟/o?ϯ_????????????????//_߿/o??o___/oￏ_??o?/?_?ߏO/??o_Oo?OO/??????//?_?/_o?/_OOo?oOO/?O??O_//O???O?///?//oo??__?o?oOo_ϏO___??o?_o?/__?o__/ϟo?/o/?￿Ͽ?O/_OO_/O߿_/￿߿￿߿_/ϟO?/?_߿??/?Oo????_o???_Ϗ????oϯo???OϟO?/?_ϟO?/?_O??????????_o/Oo?????/_Oo??/OOooo/?_o?oO/_OO?/??/OO?o/ooOOoϯ/OO//_O?/_O?_oo????O?o/??Oo//??Oooo/O_??/?O/o??/_O?oO/o_o__O????/o_????????/o//__?/o?ߏ?/_????o?????/o/?oO?????ߏ?O???Oo_o??o????__O???__o//?/O//o?ߏ?__/Oo????O?o_????_/O_???o___/O/_oo/??_??_/????__OO_O__?/_Oo???_?/???????_?O_O/_o_?O?_o_//o/O?_/oO?/??//?????oo/oo/o_?__/?/_ߟ/_??_?oOO_/o/oo/o/o_OOo??//__?߯_/o_/o__?_?_o/?oOo/o_?/??/o???o_//?___//?_oO/o_?/?/oo/????O/_/oO?O/Oo/__/o_/?o//_?_o??oo_?_/Oo/_?/?O??/_O//??/Oo/?_?_?OO_o?//?_o_/o_?/߿/?/_/O??/?O//o_o_O/O//??O__//_Oo/o_O?/?_O_o/??//O_/O?/_o/_??O߿?/__o/?_?OoO??/_?/?O/oO//?//O///O?__o/?ooO_??/O_?/o_/??_O_?O_Oo_?/_?O?_O/?/?O/O???/??_oooOOoO_?/O?O_O/_/o_o?__/?oO?o?O_?/OOo_?/??__?o?/??__??_????Oo_O??////oO?oO?O_O//_//__?OO_o/_/_??/_?_/o_oo?/O?O?_o??//?/_//_/o/oo_ooo/?/__?oO?_OO_?OO_?ߏ_/O_o__/o??__oO_?/??/Oo?O/?/O_Oo_??ooOO?o_?_/o?o??//?O/o/ooO???O__??/O?/O?__?O_/O??__?//o/oo???o??_o_?oooO/O/?_??_?_OO_??oO//??o?Oo/?o/o?O?_///?__/?_O/o/?/Oo???___/O?o_?oo_??O?_OO_/?oO_oo_///__O?o//?Oo_?//?O__O_o/O???/?_//__o_?_??OO///_?o_/O_/OOoO??//o??_O_//?o///??o?_o/__O?_O?/o/_O?O?/_?/?oo_??o?/?OO/_??o///?O/o?/??/__oOO/o/?ooOOo/_??/O/O_???_??O/O/?_O/oOO?_/_??/_/oO???/OO/?o?oo?o?o_O/_oo/_/?߿?_/?OOOߏ/O?_oO?oOO_?ߟ?o?/?O/oO//_O/?__????????O?/oO_ߟ/OoO/oO?_o/???o/Ϗo/_oO???/o???o????/o????__oo??OO/??_O/??O?_o_?/o/?__??/_?/??/__o_?/Oo?ooO?_?o?/?/?O??//?oO?o_//_?O/?O?/OO/o?????o?OO????__?o?/_OO?????o?oO_?O//o??߿__߿߿ϿϿϿo__o??/oo_?ooo?????//??_/o߯oE@DH:\MURAWSKI\LATESTMO\1CPOSTAG.TIFH:\Murawski\Latest Models\1cpostag.tifT4b̤"ͤͤnJ' ']&I{s u& d(.3$ !USUS.,   I&mage <=8C HKKKK'dxd( $ Figure  1  Level 1Level 2Level 3Level 4Level 5 !USUS.,  _  80XXdd8<P?EA'+@4 RQXXCensus2000Testing,Experimentation, : andEvaluationProgram0 h 0h(#h(#0(#(#0p(#(#0p(#p(##X0X RQ#&%XX0July15,2003#X0X%&# RQXX0$t(#(# __________________________________________________________     `     h        `     h      p #X0X RQ## RQXX0  #X0X RQ#TopicReportSeries,No.2 RQXX0Ԁ    #X0X RQ#XX0#X0X_#~XX0AUTOMATIONOF   CENSUS2000 8  PROCESSES#X0X~#QL XX0  #X0X QL5#L JXX0#X0X JL巌#Qualityassuranceprocedureswereappliedthroughoutthecreationofthisreport. D Thistopicreportintegratesfindingsandprovidescontextandbackgroundforinterpretationof  resultsfromCensus2000evaluations,tests,andotherresearchundertakenbytheU.S.Census  Bureau.Itispartofabroadprogram,theCensus2000Testing,Experimentation,andEvaluation  program,designedtoassessCensus2000andtoinform2010Censusplanning.XX0  #X0X#XX0   `     h   0 @ 0@` (#(#0@ ` (#` (#0@ (# (#0@h(#(##X0X3# RQXX0PreparedbyJudithA.Dawson !, andDennisW._Stoudt_ #f #X0X RQ #XX0AU) xdEO$xA P$   #X0X # RQXX0IndependentContractorand _$ DecennialSystemsand %  ContractsManagementOffice#X0X RQ #XX0 @  @`  @  @  @h #X0XB # RQXX0 &#" #X0X RQ #\ nE:*&`z0'$  `(*EX,$ `$ `e , n (#(#(#(##-(#(#(# #<P <  <    ̀IntentionallyBlank    h  $D #    XX0  -), ?E ?> ?  Contents     1.0  Introduction.....................................................................................................X0X .1# XX0#&(#(# 2.0  Background.....................................................................................................X0X 1# XX0#J(#(# 0  X0X 2.10` (#(#DevelopmentStaff...................................................................................................1 \` (#` (#   2.20 ` UseofCommercialOfftheShelfSoftware............................................................2 H` (#` (#   2.30 ` FutureImplications..................................................................................................2 4` (#` (#  XX0 3.0  ScopeandLimitations...................................................................................X0X ...2# XX0# (#(#   X0X 3.10 ` Limitations...............................................................................................................3 ` (#` (#   3.20 ` Interviews.................................................................................................................4 ` (#` (#  XX0 4.0  RequirementsDefinition.................................................................................X0X .4# XX0y# (#(#   X0X 4.10 ` IdentificationofSystemsforStudy..........................................................................5 ` (#` (#   4.20 ` ImpactofSystemsStudied.......................................................................................5` (#` (#   4.30 ` OtherCensus2000Systems.....................................................................................5|` (#` (#   4.40 ` StaffResources.......................................................................................................5h` (#` (#  XX0 5.0  ResearchQuestions..........................................................................................X0X 6@(#(#    XX0 6.0  ResultsofAnalysis..........................................................................................X0X 7>(#(#   6.10 ` TheRightRequirementsandFunctionality............................................................7P` (#` (#   6.20 ` DataQualityandManagementInformation...........................................................8<` (#` (#   6.30 ` RequirementsandSystemDocumentation.............................................................8(x` (#` (#   6.40 ` SystemObjectives,Plans,andFunctionalPerformance.........................................9d` (#` (#   6.50 ` ScheduleandTesting..............................................................................................9P` (#` (#   6.60 ` RiskMitigationandUnplannedChange................................................................10<` (#` (#   6.70 ` SystemInterfaces...................................................................................................10(` (#` (#   6.80 ` RoleofUsers..........................................................................................................11 ` (#` (#  XX0 7.0  Recommendations..........................................................................................X0X 11# XX07 #"(#(#   X0X 7.10 ` ProcessImprovement.............................................................................................11# ` (#` (#   7.20 ` Environment...........................................................................................................12$!` (#` (#   7.30 ` Support...................................................................................................................13% "` (#` (#   7.40 ` SpecificRequirements...........................................................................................13r&!#` (#` (#    XX0 8.0  TopicReportAuthorsRecommendation.....................................................X0X 14# XX0$#J(#%(#(#   9.0  ActionstoDate..............................................................................................X0X 16# XX0 %#n*%'(#(#  +&( A'+ A@> A    %10.0  Summary........................................................................................................X0X 17(#(# # XX0f&#References................................................................................................................X0X 19# XX0D'# 6 Acknowledgments...................................................................................................X0X .21# XX0(#  Z Appendix..................................................................................................................X0X 22# XX0(# . ~ @* 8+&$ @4 C@     eAutomationofCensus2000Processes  &   1.0  Introduction N(#(#      m    X0X TheCensus2000Testing,Experimentation,andEvaluationsProgramprovidesmeasuresof $ t effectivenessfortheCensus2000design,operations,systems,andprocessesandprovides  ` informationonthevalueofnewordifferentmethodologies.Theresultsandrecommendations  L fromtheseanalysesprovidevaluableinformationcrucialtoplanningthe2010Census.By  8  providingmeasuresofhowwelltheCensus2000wasconducted,thisprogramfullysupportsthe $  CensusBureausstrategytointegratethe2010planningprocesswithongoingMasterAddress   File/TIGERenhancementsandtheAmericanCommunitySurvey.Thepurposeofthereportthat   followsistosynthesizeresultsfromrelatedCensus2000evaluations,experiments,andother   assessmentstomakerecommendationsforplanningthe2010Census.Census2000Testing,   Experimentation,andEvaluationreportsareavailableontheCensusBureausInternetsiteat: p 4; e:O  5  http://www.census.gov/pred/www/6/:O/  7 Q/. u  \      XX02.0  Background 4(#(#      m  X0X TheCensus2000wascharacterizedbytheautomationoffunctionspreviouslyperformed 4 clericallyorusingrelativelysimpletoolsofautomation.Similarly,manyofthefunctionsthathad  p beenperformedinanautomatedfashionbyinhousestaffwereturnedovertocontractstaff.  \ Twelvesystemswerethesubjectofstudyandcoveredthefollowingareas:telephone H questionnaireassistanceandtelephonefollowupforcoverageeditfailures,Internetquestionnaire 4 assistanceanddatacollectionforshortformquestionnaires,operationscontrolforallfield   operations,thefieldpersonnelandpayrollsystem,systemsfortheAccuracyandCoverage   Evaluationprogram,includingthecontrolsystemforfieldoperations,theuseoflaptopstocollect  evaluationdataandthematchingsystemusedtocomparethatevaluationdatatocorresponding  censusdata,themanagementinformationsystem,thedatacapturesystemforrespondent   questionnaires,andfinally,datadissemination. l! 0  2.10 (#(#DevelopmentStaffD#  (# (# 0  Developmentstaffswereamixtureofcontractandinhousestaff.Withinthecontract %l " staff,therewereatleastthreedistinctarrangementsthatmayhaveinfluencedtheextentto &X!# whichmorerigorousanddisciplinedrequirementsidentificationandmanagement &D"$ processesthaniscustomaryattheCensusBureauwereemployed.Thosearrangements '0#% includedcontractstaffthatwereembeddedwithinhousestaffandwereusedmoreorless ($& asinhousestaff,suchasthedevelopersoftheOperationsControlSystem2000;contract )%' staffthatworkedonsitebutworkedmoreindependentlyoftheinhousestaff,suchas *%( developersofthematchingsystemusedfortheAccuracyandCoverageEvaluation +&) system;andexternalcontractstaffthatdevelopedsystemsoffsiteindependentofinhouse  staff,suchastheTelephoneQuestionnaireAssistancesystem.TheInternetquestionnaire  assistanceanddatacollectionsystemsweredevelopedbyinhousestaff.Allothersystems  weredevelopedbyteamsofinhouseandcontractdevelopmentstaffs.t(#(# 0  2.20 (#(#UseofCommercialOfftheShelfSoftware0% (# (#L %(#%(# 0  Aconcertedeffortwasalsomadetoutilizecommercialofftheshelf(COTS)productsas $ t muchaspossibleinmanyofthesystemsdevelopmentefforts.Insomecases,COTS  ` productsforthedatacaptureofrespondentquestionnaireswerenotviewedassufficiently  L  robusttomeetCensusBureauneeds,(_Brinson_ԀandFowler,December,2001).Inother  8  cases,thedatarequirementscouldnotbemetwiththetypicalCOTSapplicationusedin $  thetelephonecallindustry,(_Furno_,November,2001).Thefieldpayrollandpersonnel   systemusedaCOTSproductwithcustomization.Theinitialestimatewastheproduct   wouldmeetapproximately90percentoftherequirements;thefinalassessmentwas   approximately50percentoftherequirementsweremetwiththeproductbefore   customization,(Eaton,September,2002).ThetrendtouseCOTSproductsand p contractorsisexpectedtocontinueatanacceleratedrateforthenextcensustomakethe \ overallcensusprocessmoreefficientandeconomical,whilemaintainingthedesiredlevels H ofdataqualityandcompleteness.4(#(# 0  2.30 (#(#FutureImplications \ (# (# 0  Itisimportanttounderstandthereasonsforsuccessorthelimitationsofthesystemsifthe 4 intentistocontinueautomationofbasiccensusprocesses.Iffutureimprovementsareto   bemade,akeytojudgingthesuccessesorlimitationsofsystemsistoanalyzehowthe   functionsofthesystemsweredefined,andhowwellthesystemsperformedthose  functions.Asecondimportantpointistodetermineifthesystemswereaskedtoperform  thecorrectfunctions.ThisreportwillprovidetheCensusBureauwithanoverall  assessmentoftheautomatedsystemsandprocesseslistedinSection3.Inaddition,the l reportwilloffersuggestionsonimprovementsforthedevelopmentprocessforfuture X  Censusprograms.D!(#(#   XX03.0  ScopeandLimitations X0X Ԁ#l!(#(#  Thepurposeofthisreportistosummarizethefindingsandrecommendationsfromtheformal %l # studiesofautomatedCensus2000processesandtheoperationalassessmentsperformedforthose  &X!$ processeswhenavailable.Alistofthosedocumentsmaybefoundinthesection,References. &D"% ThisstatedpurposewasmadeeasierwhenTitanSystemsCorporation/SystemResourcesDivision '0#& (Titan)produceda ProgramSummaryReportofCensus2000AutomatedSystemsEvaluation, ($' bringingtogethertheirthoughtsandfindingsfromeachoftheirtwelvestudies.Thoseresultsare )%( providedintheRecommendationssection.Thefollowingisalistoftheevaluationstudiesthey *%) conductedandsummarized: +&* Ї0  *1.R.1.aTelephoneQuestionnaireAssistance(_TQA_)SystemRequirementsStudy(#(#   *2.R.1.bCoverageEditFollowup(_CEFU_)SystemRequirementsStudy  0  *3.R.1.c.InternetQuestionnaireAssistance(_IQA_)SystemRequirementsStudy(#(#   *4.R.1.d.InternetDataCollection(IDC)SystemRequirementsStudy t   *5.R.2.a.OperationsControlSystem2000(OCS2000)SystemRequirementsStudy ` 0  6.R.2.b.LaptopComputersfortheAccuracyandCoverageEvaluation(LC/_A.C.E._Ԁ) L  SystemRequirementsStudy8 (#(# 0  7.R.2.c.AccuracyandCoverageEvaluation2000(ACE2000)SystemRequirements $ t Study   m  `(#(# 0  8.R.2.d.MatchingandReviewCodingSystem(_MaRCS_)fortheAccuracyand  L  CoverageEvaluationSystemRequirementsStudy 8 (#(# 0  *9.R.3.a.PreAppointmentManagementSystem/AutomatedDecennialAdministrative $  System(_PAMS_/ADAMS)SystemRequirementsStudy (#(# 0  10.R.3.b.American_FactFinder_Ԁ(_AFF_)SystemRequirementsStudy (#(# 0  11.R.3.c.ManagementInformationSystem2000(MIS2000)SystemRequirements   Study (#(#   *12.R.3.d.Census2000DataCaptureSystem(DCS2000)RequirementsStudy p Theasterisk(*)indicatesthosesystemsforwhichanoperationalassessmentwasalsoavailable H forthisreport. 4 0  3.10 (#(#Limitations \ (# (# 0  AfewcommentsontheprocessusedtoanalyzethesystemsstudiedbyTitanseem 4 pertinent.Interviewsofkeypersonnelbysystemwereconductedutilizingasetof   questionsdevelopedforinhousestaffandaseparatesetforcontractors.Thequestions   wereprovidedtointervieweesinadvance.Underthe LIMITS sectionofeachstudyare  thefollowingstatements:(#(#  (Ay2"d"  0  0 (#(#dAT2A3  0m  (# (#  Theperceptionofthosepersonsparticipatingintheinterviewprocesscan p significantlyinfluencethequalityofinformationgathered.dATT݌\ m (#m (# Ќ  "d"  0  0 (#(#dU2A3  0m  (# (#  Insomecases,interviewswereconductedseveralmonths,evenyears,after 4"  theparticipanthadbeeninvolvedinsystemdevelopmentactivities.dUV݌ #p!m (#m (# Ќ  "d"  0  0 (#(#d