Volume 49 - Article 36 | Pages 983–1020
Reducing uncertainty in Delphi surveys: A case study on immigration to the EU
By Rhea Ravenna Sohst, Eduardo Acostamadiedo, Jasper Tjaden
References
Abel, G., Bijak, J., Findlay, A., McCollum, D., and Wisniowski, A. (2013). Forecasting environmental migration to the United Kingdom: An exploration using Bayesian models. Population and Environment 35: 183–203.
Acostamadiedo, E., Sohst, R.R., Tjaden, J., Groenewold, G., and Valk, H. (2020). Assessing Immigration Scenarios for the European Union in 2030 – Relevant, Realistic and Reliable? Geneva and The Hague: International Organization for Migration and Netherlands Interdisciplinary Demographic Institute. Geneva and The Hague: International Organization for Migration and Netherlands Interdisciplinary Demographic Institute.
Akins, R.B., Tolson, H., and Cole, B.R. (2005). Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Medical Research Methodology 5(37).
Arrow, K.J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., Ledyard, J.O., and Zitzewitz, E. (2008). The promise of prediction markets. Science 320(5878): 877–878.
Badescu, D. and Chen, E. (2014). Identifying Expertise to Extract the Wisdom of Crowds. Management Science 61(2): 249–486.
Belton, I., MacDonald, A., Wright, G., and Hamlin, I. (2019). Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process. Technological Forecasting and Social Change 147: 72–82.
Bijak, J. (2011). Forecasting international migration in Europe: A Bayesian view. Amsterdam: Springer Series on Demographic Methods and Population Analysis.
Bijak, J., Disney, G., A.M, Findlay, Foster, J.J., Smith, P.W.F., and Wisniowski, A. (2019). Assessing time series models for forecasting international migration: Lessons from the United Kingdom. Journal of Forecasting 38(5): 470–487.
Bijak, J. and Wisniowski, A. (2010). Bayesian forecasting of immigration to selected European countries by using expert knowledge. Journal of the Royal Statistical Society 173(4): 775–796 (Series A (Statistics in Society).
Böhme, M.H., A, Gröger, and Stöhr, T. (2020). Searching for a better life: Predicting international migration with online search keyword. Journal of Development Economics 142: 102347.
Bonett, D.G. (2006). Confidence interval for a coefficient of quartile variation. Computational Statistics and Data Analysis 50(11): 2953–2957.
Colson, A. and Cooke, R. (2018). Expert elicitation: Using the classical model to validate experts judgments. Review of Environmental Economics and Policy 12(1): 113–132.
Czaika, M. and Reinprecht, C. (2022). Migration drivers: Why do people migrate? In: Scholten, P. (ed.). Introduction to migration studies. Rotterdam: Springer Cham: 463–482.
Dalkey, N. and Helmer, O. (1962). An experimental application of the Delphi method to the use of experts. Management Science 9(3): 458–467.
Dao, T.H., Docquier, F., Maurel, M., and Schaus, P. (2018). Global migration in the 20th and 21st centuries: The unstoppable force of demography. Clermont-Ferrand: FERDI, Foundation for Studies and Research on International Development (FERDI). Development Policies Working Paper 223.
de Haas, H. (2011). Mediterranean migration futures: Patterns, drivers and scenarios. Global Environmental Change 21(S1): 59–69.
de Haas, H., Vargas-Silva, C., and Vezzoli, S. (2010). Global migration futures: A conceptual and methodological framework for research and analysis. Oxford: International Migration Institute, Global Migration Futures report.
de Valk, H.A., Acostamadiedo, E., Guan, Q., Melde, S., Mooyaart, J., Sohst, R.R., and Tjaden, J. (2022). How to predict future migration: Different methods explained and compared. In: Scholten, P. (ed.). Introduction to migration studies. Rotterdam: Springer Cham: 463–482.
Disney, G., Wisniowski, A., Forster, J., Smith, P., and Bijak, J. (2015). Evaluation of existing migration forecasting methods and models. Southampton: ESRC Centre for Population Change, Report for the Migration Advisory Committee.
Drbohlav, D. (1997). Migration policy objectives for European East–West international migration. International Migration 35(1): 85–108.
Drescher, M. and Edwards, R.C. (2018). A systematic review of transparency in the methods of expert knowledge use. Journal of Applied Ecology 56(2): 436–449.
European Asylum Support Office (2021). Analysis and statistics – Early Warning and Preparedness System (EPS. Valetta: European Asylum Support Office.
European Asylum Support Office (2019). The future of international protection in the EU+ by 2030: A scenario study. Valletta: European Asylum Support Office.
European Commission (2020). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on a New Pact on Migration and Asylum. Brussels: European Commission.
European Commission (2020). Informing European Commission policy making with scientific evidence. The Group of Chief Scientific Advisors 2015–2019. Brussels: European Commission.
European Commission (2020). Standard Eurobarometer 93. Brussels: EC.
European Spatial Planning Observation Network and Netherlands Interdisciplinary Demographic Institute (2010). DEMIFER – Demographic and migratory flows affecting European regions and cities. Final report.
Findlay, A.M., McCollum, D., Abel, G., Wisniowski, A., and Bijak, J. (2012). A Delphi survey of immigration to the UK to 2060, with particular reference to environmental mobility. Southampton: ESRC Centre for Population Change, ESRC Centre for Population Change Working Paper Series 28.
Finkelstein, S. (2019). Don’t be blinded by your own expertise. Harvard Business Review 97(3): 153–158.
Flostrand, A., Pitt, L., and Bridson, S. (2020). The Delphi technique in forecasting – A 42-year bibliographic analysis (1975–2017. Technological Forecasting and Social Change 150: 119773.
Friedrich Ebert Stiftung, Global Future and International Organization for Migration (2017). Tomorrow’s world of migration and mobility. Geneva: FES, Global Future and International Organization for Migration.
Frontex (2016). Risk analysis for 2016. Warsaw: Frontex.
Hasson, F., Keeney, S., and McKennea, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 32(4): 1008–1015.
Helmer-Hirschberg, O. (1967). Analysis of the future: The Delphi method. Papers. Santa Monica: RAND Corporation.
International Labour Organization and European Commission (2009). Operational indicators of trafficking in human beings. Results from a Delphi survey implemented by the ILO and the European Commission.
Jünger, S., Payne, S.A., Brine, J., Radbruch, L., and Brearley, S.G. (2017). Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine 31(8): 684–706.
Kahneman, D., Sibony, O., and Sunstein, C. (2021). Noise: A flaw in human judgment. New York: Little Brown Spark.
Kupiszewski, M. (2002). How trustworthy are forecasts of international migration between Poland and the European Union? Journal of Ethnic and Migration Studies 28(4): 627–645.
Kynn, M. (2008). The heuristics and biases bias in expert elicitation. Journal of the Royal Statistical Society 171(1): 239–264.
Lachmanova, L. and Drbohlav, D. (2004). The probable future development of future East-West migration (the Delphi method revived. European Spatial Research and Policy 11(1): 135–155.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change 73(5): 467–482.
Lau, P., Ryan, S., Abbott, P., Tannous, K., Trankle, S., Peters, K., Page, A., Cochrane, N., Usherwood, T., and Reath, J. (2022). Protocol for a Delphi consensus study to select indicators of high-quality general practice to achieve Quality Equity and Systems Transformation in Primary Health Care (QUEST-PHC) in Australia. PLoS ONE 17(5): e0268096.
Lomax, N., Wohland, P., Rees, P., and Norman, P. (2020). The impacts of international migration on the UK’s ethnic populations. Journal of Ethnic and Migration Studies 46(1): 177–199.
Morgan, M.G. (2014). Use (and abuse) of expert elicitation in support of decision making for public policy. Proceedings of the National Academy of Sciences of the United States of America 111(20): 7176–7184.
Organisation for Economic Co-operation and Development (2016). Perspectives on global development 2017: International migration in a shifting world. Paris: OECD Publishing.
Organisation for Economic Co-operation and Development (2015). Scientific advice for policy making: The role and responsibility of expert bodies and individual scientists. Paris: OECD Publishing, OECD Science, Technology and Industry Policy Papers 21.
Organisation for Economic Co-operation and Development (2020). Towards 2035. Strategic foresight. Making migration and integration policies future ready. Paris: OECD Publishing.
Page, A., Potter, K., Clifford, R., McLachlan, A., and Etherton-Beer, C. (2015). Prescribing for Australians living with dementia: Study protocol using the Delphi technique. BMJ Open 5: 008048.
Raymer, J. (2017). Measuring flows of international migration. IZA, IZA World of Labour 354.
Rowe, G. and Wright, G. (2001). Expert opinions in forecasting: The role of the Delphi technique. In: Armstrong, J. S. (ed.). Principles of forecasting. Boston, MA: Springer: 125–144.
Rowe, G., Wright, G., and Bolger, F. (1991). Delphi: A reevaluation of research and theory. Technological Forecasting and Social Change 39(3): 235–251.
Schmalz, U., Spinler, S., and Ringbeck, J. (2021). Lessons learned from a two-round Delphi-based scenario study. MethodsX 8: 101179.
Shellman, S.M. and Stewart, B.M. (2007). Predicting risk factors associated with forced migration: An early warning model of Haitian flight. Civil Wars 9(2): 174–199.
Sohst, R.R., Tjaden, J., Valk, H., and Melde, S. (2020). A systematic review of the literature on migration scenarios and forecasts. Berlin and Amsterdam: International Organization for Migrations Global Migration Data Analysis Centre and Netherlands Interdisciplinary Demographic Institute.
Sutherland, W.J. and Burgman, M. (2015). Policy advice: Use experts wisely. Nature 526(7573): 317–318.
Szczepanikova, A. and Criekinge, T. (2018). The future of migration in the European Union: Future scenarios and tools to stimulate forward-looking discussions. Luxembourg: European Union.
Tetlock, P.E., Mellers, B.A., and Scoblic, J.P. (2017). Bringing probability judgements into policy debates via forecasting tournaments. Science 355(6324): 481–483.
Tjaden, J., Auer, D., and Laczko, F. (2019). Linking migration intentions with flows: Evidence and potential use. International Migration 57(1): 36–57.
United Kingdom Government Office for Science (2011). Migration and global environmental change: Future challenges and opportunities – Future scenarios. Foresight. London: The Government Office for Science.
Van Nimwegen, N. and Erf, R. (2010). Europe at the crossroads: Demographic challenges and international migration. Journal of Ethnic and Migration Studies 36(9): 1359–1379.
Verdolini, E., Anadon, L.D., Baker, E., Bosetti, V., and Reis, L.A. (2018). Future prospects for energy technologies: Insights from expert elicitations. Review of Environmental Economics and Policy 12(1): 133–153.
Willekens, F. (1994). Monitoring international migration flows in Europe: Towards a statistical data base combining data from different sources. European Journal of Population 10(1): 1–42.
Willekens, F. (2018). Towards causal forecasting of international migration. Vienna Yearbook of Population Research 16: 199–218.
Woudenberg, F. (1991). An evaluation of Delphi. Technological Forecasting and Social Change 40(2): 131–150.
Zwillinger, D. and Kokoska, S. (2000). Standard probability and statistical tables and formulae. Boca Raton: Chapman and Hall/CRC.