Volume 42 - Article 5 | Pages 133–148 Author has provided data and code for replicating results

Traditional versus Facebook-based surveys: Evaluation of biases in self-reported demographic and psychometric information

By Kyriaki Kalimeri, Mariano G. Beiró, Andrea Bonanomi, Alessandro Rosina, Ciro Cattuto

Print this page  Facebook  Twitter

 

 
Date received:30 Jun 2017
Date published:21 Jan 2020
Word count:4750
Keywords:bias, demography, Facebook, moral foundations, personality, psychometrics, recruitment bias, self-reporting bias, self-selection bias, social media, survey
DOI:10.4054/DemRes.2020.42.5
Additional files:readme.42-5 (text file, 3 kB)
 demographic-research.42-5 (zip file, 117 kB)
 

Abstract

Background: Social media in scientific research offers a unique digital observatory of human behaviours and hence great opportunities to conduct research at large scale, answering complex sociodemographic questions. We focus on the identification and assessment of biases in social-media-administered surveys.

Objective: This study aims to shed light on population, self-selection, and behavioural biases, empirically comparing the consistency between self-reported information collected traditionally versus social-media-administered questionnaires, including demographic and psychometric attributes.

Methods: We engaged a demographically representative cohort of young adults in Italy (approximately 4,000 participants) in taking a traditionally administered online survey and then, after one year, we invited them to use our ad hoc Facebook application (988 accepted) where they filled in part of the initial survey. We assess the statistically significant differences indicating population, self-selection, and behavioural biases due to the different context in which the questionnaire is administered.

Results: Our findings suggest that surveys administered on Facebook do not exhibit major biases with respect to traditionally administered surveys in terms of neither demographics nor personality traits. Loyalty, authority, and social binding values were higher in the Facebook platform, probably due to the platform’s intrinsic social character.

Conclusions: We conclude that Facebook apps are valid research tools for administering demographic and psychometric surveys, provided that the entailed biases are taken into consideration.

Contribution: We contribute to the characterisation of Facebook apps as a valid scientific tool to administer demographic and psychometric surveys, and to the assessment of population, self-selection, and behavioural biases in the collected data.

Author's Affiliation

Kyriaki Kalimeri - ISI Foundation, Italy [Email]
Mariano G. Beiró - Universidad de Buenos Aires, Argentina [Email]
Andrea Bonanomi - Università Cattolica del Sacro Cuore, Italy [Email]
Alessandro Rosina - Università Cattolica del Sacro Cuore, Italy [Email]
Ciro Cattuto - ISI Foundation, Italy [Email]

Other articles by the same author/authors in Demographic Research

» The impact of COVID-19 on fertility plans in Italy, Germany, France, Spain, and the United Kingdom
Volume 43 - Article 47

» The effect of the Great Recession on permanent childlessness in Italy
Volume 37 - Article 20

» Lowest-Low Fertility: Signs of a recovery in Italy?
Volume 21 - Article 23

» Intergenerational family ties and the diffusion of cohabitation in Italy
Volume 16 - Article 14

» Interdependence between sexual debut and church attendance in Italy
Volume 14 - Article 19

» Is marriage losing its centrality in Italy?
Volume 11 - Article 6

Most recent similar articles in Demographic Research

» The problematic estimation of "imitation effects" in multilevel models
Volume 9 - Article 2    | Keywords: bias, survey

» The growth of education differentials in marital dissolution in the United States
Volume 45 - Article 26    | Keywords: demography

» Editorial to the Special Issue on Demographic Data Visualization: Getting the point across – Reaching the potential of demographic data visualization
Volume 44 - Article 36    | Keywords: demography

» Mexican mortality 1990‒2016: Comparison of unadjusted and adjusted estimates
Volume 44 - Article 30    | Keywords: demography

» Are sibling models a suitable tool in analyses of how reproductive factors affect child mortality?
Volume 42 - Article 28    | Keywords: bias