Volume 32 - Article 26 | Pages 797–828
Do low survey response rates bias results? Evidence from Japan
By Ronald R. Rindfuss, Minja K. Choe, Noriko O. Tsuya, Larry L. Bumpass, Emi Tamaki
References
American Association, for Public Opinion Research (AAPOR) (2011). Standard definitions: Final dispositions of case codes and outcome rates for surveys. AAPOR.
Atrostic, B.K., Bates, N., Burt, G., and Silberstein, A. (2001). Nonresponse in U.S. government household surveys: Consistent measures, recent trends and new insights. Journal of Official Statistics 17(2): 209−226.
Axinn, W.G., Link, C.F., and Groves, R.M. (2011). Responsive survey design, demographic data collection, and models of demographic behavior. Demography 48: 1127−1149.
Beaumont, J.-F. (2005). On the use of data collection process information for the treatment of unit nonresponse through weight adjustment. Survey Methodology 31(2): 227−231.
Bethlehem, J., Cobben, F., and Schouten, B. (2011). Handbook of nonresponse in household surveys. Hoboken, NJ: John Wiley and Sons.
Biemer, P.P., Chen, P., and Wang, K. (2013). Using level-of-effort paradata in non-response adjustments with application to field surveys. Journal of the Royal Statistical Society: Series A 176(1): 147−168.
Brick, J.M. and Williams, D. (2013). Explaining rising nonresponse rates in cross-sectional surveys. The ANNALS of the American Academy of Political and Social Science 645: 36−59.
Casas-Cordero, C., Creuter, F., Wang, Y., and Babey, S. (2013). Assessing the measurement error properties of interviewer observations of neighborhood characteristics. Journal of the Royal Statistical Society: Series A 176(1): 227−249.
Choe, M.K., Bumpass, L.L., Tsuya, N.O., and Rindfuss, R.R. (2014). Non-traditional family-related attitudes in Japan: Macro and micro determinants. Population and Development Review 40(2): 241−271.
Couper, M.P. (1997). Survey introductions and data quality. Public Opinion Quarterly 61(2): 317−338.
Couper, M.P. (2000). Usability evaluation of computer-assisted instruments. Social Science Computer Review 18(4): 384−396.
Couper, M.P. and Kreuter, F. (2013). Using paradata to explore item level response times in surveys. Journal of the Royal Statistical Society: Series A 176: 271−286.
Couper, M.P. and Lyberg, L. (2005). The use of paradata in survey research. In: Proceedings of the 55th Session of the International Statistical Institute, Sydney. Voorburg: International Statistical Institute.
Curtin, R., Presser, S., and Singer, E. (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly 69(1): 87−98.
Curtin, R., Presser, S., and Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly 64(2): 413−428.
De Leeuw, E. and de Heer, W. (2002). Trends in household survey nonresponse: A longitudinal and international comparison. In: Groves, R.M., Dillman, D.A., Entinge, J.L., and Little, R.J.A. (eds.). Survey nonresponse. NY: John Wiley and Sons: 41−54.
Dijkstra, W. and Smit, J.H. (2002). Persuading reluctant recipients in telephone surveys. In: Groves, R.M., Dillman, D.A., Entinge, J.L., and Little, R.J.A. (eds.). Survey nonresponse. NY: John Wiley and Sons: 121−134.
Dillman, D.A., Smyth, J.D., and Christian, L.M. (2009). Internet, mail and mixed-mode surveys: the tailored design method. Hoboken, NJ: John Wiley and Sons.
Durrant, G.B. and D’Arigo, J. (2011). Using paradata to predict best times of contact, conditioning on household and interviewer influences. Journal of the Royal Statistical Society: Series A 174: 1029−1049.
Durrant, G.B., Groves, R.M., Staetsky, L., and Steele, F. (2010). Effects of interviewer attitudes and behaviors on refusal in household surveys. Public Opinion Quarterly 74(1): 1−36.
Fitzgerald, J., Gottschalk, P., and Moffitt, R (1998). An analysis of sample attrition in panel data. The Journal of Human Resources 33(2): 251−299.
Fricker, S. and Tourangeau, R. (2010). Examining the relationship between nonresponse propensity and data quality in two national household surveys. Public Opinion Quarterly 74(5): 934−955.
Groves, R.M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly 70(5): 646−675.
Groves, R.M. (2011). Three eras of survey research. Public Opinion Quarterly 75(5): 861−871.
Groves, R.M. and Couper, M.P. (1998). Nonresponse in household interview surveys. New York: John Wiley and Sons.
Groves, R.M., Couper, M.P., Presser, S., Singer, E., Tourangeau, R., Acosta, G.P., and Nelson, L. (2006). Experiments in producing nonresponse bias. Public Opinion Quarterly 70(5): 720−736.
Groves, R.M., Fowler Jr., F.J., Couper, M.P., Lepkowski, J.M., Singer, E., and Tourangeau, R. (2009). Survey methodology. Hoboken, NJ: John Wiley and Sons.
Groves, R.M. and Heeringa, S.G. (2006). Responsive design for household surveys: Tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society: Series A 169(part 3): 439−457.
Groves, R.M. and Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly 72(2): 167−189.
Groves, R.M., Pressor, S., Tourangeau, R., West, B.T., Couper, M.P., Singer, E., and Toppe, C. (2012). Support for the survey sponsor and nonresponse bias. Public Opinion Quarterly 76(3): 512−524.
Groves, R.M., Singer, E., and Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly 64(3): 299−308.
Guilkey, D.K. and Murphy, J.L. (1993). Estimation and testing in the random effects probit model. Journal of Econometrics 59(3): 301−317.
Hox, J. and de Leeuw, E. (2002). The influence of interviewers’ attitude and behavior on household survey nonresponse: An international comparison. In: Groves, R.M., Dillman, D.A., Entinge, J.L., and Little, R.J.A. (eds.). Survey nonresponse. NY: John Wiley and Sons: 103−120.
Inaba, A. (2007). Problems relating to declining response rates to social surveys in Japan: Trends after 2000. International Journal of Japanese Sociology 16: 23−34.
Japanese General, Social Survey (JGSS). Social Science Japan Data Archive [electronic resource]. Institute of Social Science, University of Tokyo.
Judge, G.G., Griffiths, W.E., Hill, R.C., Lutkepohl, H., and Lee, T.-C. (1985). The theory and practice of econometrics. New York: Wiley.
Keeter, S. (2012). Assessing the Representativeness of Public Opinion Surveys [electronic resource].
Keeter, S., Kennedy, C., Dimock, M., Best, J., and Craighill, P. (2006). Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Public Opinion Quarterly 70(5): 759−779.
Keeter, S., Miller, C., Kohut, A., Groves, R.M., and Presser, S. (2000). Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly 66(1): 125−148.
Lepkowski, J.M. and Couper, M.P. (2002). Nonresponse in the second wave of longitudinal household surveys. In: Groves, R.M., Dillman, D.A., Entinge, J.L., and Little, R.J.A. (eds.). Survey nonresponse. NY: John Wiley and Sons: 121−134.
Lessler, J.T. and Kalsbeck, W.D. (1992). Nonsampling error in surveys. New York: Wiley.
Lillard, L. and Panis, C.W.A. (1998). Panel attrition from the panel study of income dynamics. The Journal of Human Resources 33(2): 437−457.
Lin, I.-F. and Schaeffer, N.C. (1995). Using survey participants to estimate the impact of nonparticipation. Public Opinion Quarterly 59(2): 236−258.
Little, R.J.A. and Rubin, D.B. (2002). Statistical analysis with missing data. Hoboken, NJ: John Wiley and Sons.
MaCurdy, T., Mroz, T., and Gritz, R.M. (1998). An evaluation of the national longitudinal survey of youth. Journal of Human Resources 33(2): 345−436.
Maddala, G.S. (1983). Limited-dependent and qualitative variables in econometrics. Cambridge: Cambridge University Press.
Matsumoto, W. (2007). A study on technical and cultural differences in survey methodology between Japan and the United States. International Journal of Japanese Sociology 16: 10−22.
Maynard, D.W., Freese, J., and Schaeffer, N.C. (2010). Calling for participation: Requests, blocking moves, and rational (inter)action in survey introductions. American Sociological Review 75(5): 791−814.
Merkle, D.M. and Edelman, M. (2002). Nonresponse in exit polls: A comprehensive analysis. In: Groves, R.M., Dillman, D.A., Entinge, J.L., and Little, R.J.A. (eds.). Survey nonresponse. NY: John Wiley and Sons: 243−258.
Mouw, T. and Kalleberg, A.L. (2010). Occupations and the structure of wage inequality in the United States, 1980s to 2000s. American Sociological Review 75(3): 402−431.
Office of Personal Information Protection, Cabinet Office (2006). Act on the Protection of Personal Information. Tokyo: Cabinet Office, Government of Japan.
Olson, K. (2013). Do nonresponse follow-ups improve or reduce data quality?: A review of the existing literature. Journal of the Royal Statistical Society: Series A 176(part 1): 129−145.
Olson, K. (2006). Survey participation, nonresponse bias, measurement error bias, and total bias. Public Opinion Quarterly 70(5): 737−758.
Peytcheva, E. and Groves, R.M. (2009). Using variation in response rates of demographic subgroups as evidence of nonresponse bias in survey estimates. Journal of Official Statistics 25(2): 193−201.
Plewis, I., Calderwood, L., and Ketende, S. (2012). Assessing the accuracy of response propensities in longitudinal studies. Survey Methodology 38(2): 167−171.
Rindfuss, R.R., Choe, M.K., Bumpass, L.L., and Tsuya, N.O. (2004). Social networks and family change in Japan. American Sociological Review 69(6): 838−861.
Rindfuss, R.R., Piotrowski, M., Thongthai, V., and Prasartkul, P. (2007). Measuring housing quality in the absence of a monetized real estate market. Population Studies 61(1): 35−52.
Rubin, D.B. (1976). Inference and missing data. Biometrika 63(3): 581−592.
Sakamoto, K. (2006). Datsuraku ni kansuru bunseki (sample attrition analysis). The Japanese Journal of Labour Studies 551(June): 55−70.
Sampson, R.J. and Raudenbush, S.W. (1999). Systematic social observation of public spaces: A new look at disorder in urban neighborhoods. American Journal of Sociology 105(3): 603−651.
Sikkel, D., Hox, J., and de Leeuw, E. (2009). Using auxiliary data for adjustment in longitudinal research. In: Peter Lynn, P. (ed.). Methodology of longitudinal surveys. Chichester, UK: Wiley: 141−156.
Singer, E. (2006). Nonresponse bias in household surveys. Public Opinion Quarterly 70(5): 637−645.
Smith, H.L. (2009). Double sample to minimize bias due to non-response in a mail survey. Philadelphia, Pa: Population Studies Center, University of Pennsylvania, PSC working paper series, No. 09-05.
Steeh, C.G. (1981). Trends in nonresponse rates, 1952-1979. Public Opinion Quarterly 45(1): 40−57.
Synodinos, N.E. and Yamada, S. (2000). Response rate trends in Japanese surveys. International Journal of Public Opinion Research 12(1): 48−72.
Tsuya, N.O., Bumpass, L.L., Choe, M.K., and Rindfuss, R.R. (2012). Employment and household tasks of Japanese couples, 1994–2009. Demographic Research 27(24): 705−718.
Van der Berg, G.J., Lindeboom, M., and Ridder, G. (1994). Attrition in longitudinal panel data, and the empirical analysis of dynamic labour market behavior. Journal of Applied Econometrics 9: 421–435.
Vercruyssen, A., van de Putte, B., and Stoop, I.A.L. (2011). Are they really too busy for survey participation? The evolution of busyness and busyness claims in Flanders. Journal of Official Statistics 27(4): 619−632.
Voorpostel, M. and Lipps, O. (2011). Attrition in the Swiss household panel: Is change associated with drop-out? Journal of Official Statistics 27(2): 301−318.
Wagner, J. (2010). The fraction of missing information as a tool for monitoring the quality of survey data. Public Opinion Quarterly 74(2): 223−243.
Wagner, J., West, B.T., Kirgis, N., Lepkowski, J.M., Axin, W.G., and Ndiaye, S.K. (2012). Use of paradata in a responsive design framework to manage a field data collection. Journal of Official Statistics 28(4): 477−499.
West, B.T. (2013). An examination of the quality and utility of interviewer observations in the National Survey of Family Growth. Journal of the Royal Statistical Society: Series A 176(part 1): 211−225.
West, B.T. and Olson, K. (2010). How much of interviewer variance is really nonresponse error variance? Public Opinion Quarterly 74(5): 1104−1126.
Yamada, S. and Synodinos, N.E. (1994). Public opinion surveys in Japan. International Journal of Public Opinion Research 6: 118−138.
Ziliak, J.P. and Kniesner, T.J. (1998). The importance of sample attrition in life cycle labor supply estimation. The Journal of Human Resources 33(2): 507−530.