Applications to Census Undercount Estimation Conclusions

4. Prior Literature

Notwithstanding the large literature on methods for small-area estimation, there have been comparatively few evaluation studies and even fewer attempts to quantify errors due to heterogeneity. The literature on methods, building on [Deming 1948], is summarized by [Purcell and Kish 1979, Platek et al. 1987, Ghosh and Rao 1994]. Uniform ratio estimators like the ones considered in this study are the oldest and most widespread of all small-area estimators. They are sometimes themselves called “synthetic estimators,” though that name, coined in [National Center for Health Statistics 1968], is more properly applied when such estimators have been summed up within areas over strata or groups.

Parametric evaluations based on variance-component models have been applied and studied [Battese, Harter and Fuller 1988, Prasad and Rao 1990]. For that work, unlike P-12 and the PES, each of the small areas for which estimates are needed contains sampled units; parameters governing heterogeneity are identifiable without the presence of a census or evaluation sample like P-12. When direct comparisons and parametric estimates are not feasible, evaluations of small-area estimates generally take the form of sensitivity analyses and simulation studies.

The literature on evaluation of small-area estimates tends to focus, like our report, on problems of census adjustment. There is a simulation study of synthetic estimation using two demographic groups [Schirm and Preston 1987]. The areas are states plus the District of Columbia; the variable is the 1980 net Census undercounts. Lacking information about levels of heterogeneity of the kind given in the present report, a stylized model is used. Group-specific state effects are assumed to be independent and identically distributed lognormal variables, with variances set to levels loosely suggested by Census Bureau work on 1970 undercounts.

A form of evaluation that has come to be called “artificial population analysis” has been pursued with 1980 Census data [Isaki et al. 1987]. Related, as yet unpublished, work by Census Bureau staff has been conducted with 1990 data. Both “across-the-board” (unstratified uniform ratio estimates) and synthetic estimates have been studied, also with 1980 data [Wolter and Causey 1991]. The areas are states, counties, and 1980 Census enumeration districts (with typical populations of a thousand or so). The variable under study is the Census substitution rate (also studied in P-12), rescaled within strata to match certain 1980 national net undercount estimates. The “across the board” studies use six strata defined by place-type within New England. The synthetic studies use 24 strata defined by age, sex, and race within the whole United States. Results are presented in terms of several aggregate “measures of closeness” for adjusted versus unadjusted values. A discussion of these studies can be found in [Freedman and Navidi 1992].

Using block-level data for components of undercount from the 1990 PES, within-group heterogeneity across blocks has been compared to within-block heterogeneity across groups [Hengartner and Speed 1993]. In a study of Australian unemployment rates, small-area estimates are evaluated by a direct comparison with tabulations from a contemporaneous census [Feeney 1987]. Our work with P-12 is an approximate version of this direct strategy, in which an extra-large sample from the census plays the role of the census itself.

The Bureau has analyzed the P-12 data, concentrating on the statistical significance of state-to-state heterogeneity [Kim 1991, Kim, Blodgett and Zaslavsky 1993]. Several approaches were used, including log-linear modeling of the P-12 variables, estimating state effects separately for post-stratum groups. The test statistics measure excess heterogeneity from state to state after dividing out the observed heterogeneity from local area to local area. This confounds the effects of local heterogeneity with the effects of sample design. Given the high level of local heterogeneity, this analytic strategy has little power for detecting state-to-state heterogeneity.

Methods like those of the present study have been applied to measure state-to-state heterogeneity in six Census coverage indicators including the four studied here [Freedman and Wachter 1994]. That work is based on the whole Census, not on a sample like P-12, and it uses a post-stratification with 357 strata instead of the 1392 used here. For various state-by-state tallies, the impact of heterogeneity on loss-function analyses is quantified. The impact of other omitted or underestimated sources of error on the Census Bureau’s loss function analyses for 1990 has been reviewed [Freedman et al. 1994].

Previous investigators have detected residual heterogeneity in probabilities of enumeration by the 1990 census [Alho et al. 1993]. The investigation focused on minorities in central cities across the four census regions, and used logistic regression. One explanatory variable was the multi-unit housing rate, which turned out to be strongly associated with capture in the census, at least in two regions. Substitutions and allocations were excluded from the model, but were also strongly associated with capture in the census. Overall, the impact of heterogeneity is estimated as being roughly half the size of the net undercount. Geographic heterogeneity at state or substate levels was not explicitly represented: the modeling was done at the level of individuals within broad groups of post strata, some explanatory variables being defined at the post-stratum level.

Many observers favor census adjustment; illustrative citations are [Schirm and Preston 1987, Ericksen, Kadane and Tukey 1989, Wolter and Causey 1991, Mulry and Spencer 1993, Zaslavsky 1993, Belin and Rolph 1994, Steffey and Bradburn 1994, Anderson and Fienberg 1999, Cohen, White and Rust 1999, Prewitt 2000]. Other observers find that census adjustment would introduce more error than it removes [Freedman and Navidi 1992, Hengartner and Speed 1993, Freedman et al. 1993, Breiman 1994, Freedman et al. 1994, Freedman and Wachter 1994, Brown et al. 1999, Darga 1999, Wachter and Freedman 2000, Skerry 2000, Stark 2000]. There are a priori reasons to favor adjustment; on the other hand, there are substantial biases in estimated adjustment factors, and heterogeneity is pervasive. What is difficult to determine from available data is the extent to which biases reinforce each other or cancel, even at the state level; the bottom-line impact of heterogeneity on accuracy is another major issue.

On the wider question of amounts of heterogeneity to be expected for variables of various kinds at local levels, we are aware of no systematic empirical studies. The analysis of local Census data as a field of study is summed up by [Myers 1992]. Better empirical knowledge about geographical heterogeneity in demographic behavior is important not only for small-area estimation but also for the modeling of long-term demographic change. Parish-to-parish variability in English historical data has been analyzed [Wachter 1992]. Stochastic demographic models which recognize geographic levels of randomness in human population processes are the eventual goal.


Applications to Census Undercount Estimation Conclusions

Measuring Local Heterogeneity with 1990 U.S. Census Data
Kenneth W. Wachter, David A. Freedman
© 2000 Max-Planck-Gesellschaft ISSN 1435-9871