Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys

Abstract
Estimates of interviewer effects on survey statistics are examined from nine surveys conducted over a six-year period at the Survey Research Center. Estimates of intraclass correlations associated with interviewers are found to be unstable, given the number of interviewers (30–40) used on most surveys. This finding calls into question inference from earlier studies of interviewer effects. To obtain more reliable information about magnitudes of interviewer effects, generalized effects are constructed by cumulating estimates over statistics and surveys. These generalized correlations are found to be somewhat smaller than those reported in the past literature. Few differences in generalized interviewer effect measures are found between open and closed questions or between factual and attitudinal questions. Small reductions in effects were obtained when a Computer Assisted Telephone Interviewing (CATI) system was used; there was some evidence of elderly respondents being more susceptible to interviewer effects; the number and type of second responses to open questions were affected by interviewer behavior; and changes in interviewing techniques reduced interviewer effects.