Changing data collection methods means different kind of data

Company: GfK Group

Author: Gerd Meier


Information technology has improved to such an extent that computer assistance is now possible for telephone interviews (CATI) as well as for personal interviews (CAPI). Data quality is increased by using the computer because sequential effects are avoided by randomizing question and answer lists, the interviewer led through the questionnaire (automatic filter) and answers can be checked for consistency while conducting the interview. On the other hand, the interview setting is changed when using the computer. Maybe the interview setting is experienced as less "personal" by the participants. This raises the question whether it is possible to compare the results of studies with different data collection methods and whether it is possible to change the data collection method in ongoing studies without changes in results. This article describes the research design and the results from a method study conducted by GfK in 1994. In a laboratory and a field study with more than 600 interviews, the results of three data collection methods, CATI, CAPI, and the traditional paper-and-pencil-interview (PPI) were compared. As a conclusion one can say that indeed the data collection method has an influence on the results. There are statistically significant differences between PPI, CATI, and CAPI in the number of answers to open questions, in the number of yes responses, and the assessment of scales. These differences could be traced back partly to varying response spontaneity, as measured by response latency time. The method study also showed some ways to minimize these differences.

  • PDF
  • This could also be of interest