Level of effort paradata (especially number of call attempts) can be used with response propensity models to reduce nonignorable nonresponse bias in sample surveys. One approach for incorporating these data in the adjustment process is the callback model (Drew and Fuller, 1980; Biemer and Link, 2007). A number of problems may arise in applying these models. One important problem is the measurement errors in the call-back data which will lead to model misspecification and adjustment bias. This presentation reports on a study that was conducted to evaluate the quality of the callback data and its effects on callback model estimates. The study provides evidence of substantial underreporting errors in callback data. These errors are also subject to considerable interviewer variation which further exacerbates their effects on the modeling process. These effects are demonstrated using simulation techniques informed by real data. Some solutions for reducing the errors as well as more robust methods for incorporating callback information into the nonresponse adjustment process are discussed.

Keywords: Interviewer error; EM algorithm; Probability model

Biography: Paul P. Biemer is RTI Distinguished Fellow, Statistics and Associate Director of Survey Research and Development at the Odum Institute, University of North Carolina, Chapel Hill. He holds a Ph.D degree in Statistics and has over 30 years of experience in complex survey design and data analysis. He is a Fellow of the ASA and the AAAS and Elected Member of the ISI.