Abstract

n

Mobile self-reports are a popular technique to collect participant labelled data in the wild. While literature has focused on increasing participant compliance to self-report questionnaires, relatively little work has assessed response accuracy. In this paper, we investigate how participant context can affect response accuracy and help identify strategies to improve the accuracy of mobile self-report data. In a 3-week study we collect over 2,500 questionnaires containing both verifiable and non-verifiable questions. We find that response accuracy is higher for questionnaires that arrive when the phone is not in ongoing or very recent use. Furthermore, our results show that long completion times are an indicator of a lower accuracy. Using contextual mechanisms readily available on smartphones, we are able to explain up to 13% of the variance in participant accuracy. We offer actionable recommendations to assist researchers in their future deployments of mobile self-report studies.

Authors:
van Berkel Niels, Goncalves Jorge, Koval Peter, Hosio Simo, Dingler Tilman, Ferreira Denzil, Kostakos Vassilis

Publication type:
A4 Article in conference proceedings

Place of publication:
CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland UK

Keywords:
cognition, context, data quality, Ecological momentary assessment, EMA, ESM, Experience sampling method, questionnaires, Smartphones, working memory

Published:

Full citation:
Niels van Berkel, Jorge Goncalves, Peter Koval, Simo Hosio, Tilman Dingler, Denzil Ferreira, and Vassilis Kostakos. 2019. Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). ACM, New York, NY, USA, Paper 51, 12 pages. DOI: https://doi.org/10.1145/3290605.3300281

DOI:
https://doi.org/10.1145/3290605.3300281

Read the publication here:
http://urn.fi/urn:nbn:fi-fe2019050614365