The Total Survey Error Approach: A Guide To The New Science Of Survey Research

The Total Survey Error Approach The Total Survey Error Approach A G U I D E T O T H E N E W S C I E N C E O F S U RV E Y R E S E A RC H Herbert F. Weisberg T H E U N I V E R S I T Y O F C H I C AG O P R E S S • C H I C AG O A N D L O N D O N Herbert F. Weisberg is professor of political science and director of the Center for Survey Research at Ohio State University. He is a coauthor of the widely used text Introduction to Survey Research, Polling, and Data Analysis, 3d ed. (1996, with Jon A. Krosnick and Bruce D. Bowen). The University of Chicago Press, Chicago 60637 The University of Chicago Press, Ltd., London © 2005 by The University of Chicago All rights reserved. Published 2005 Printed in the United States of America 14 13 12 11 10 09 08 07 06 05 12345 ISBN: 0-226-89127-5 (cloth) ISBN: 0-226-89128-3 (paper) Library of Congress Cataloging-in-Publication Data Weisberg, Herbert F. The total survey error approach : a guide to the new science of survey research / Herbert F. Weisberg. p. cm. Includes bibliographical references and index. ISBN 0-226-89127-5 (cloth : alk. paper) — ISBN 0-226-89128-3 (pbk. : alk. paper) 1. Social surveys. 2. Social sciences—Research. I. Title. HM538.W45 2005 300.72′3—dc22 2005007816 ⬁ The paper used in this publication meets the minimum requirements of the American National Standard for Information Sciences—Permanence of Paper for Printed Library Materials, ANSI Z39.48-1992. Contents Preface vii PA R T I Survey Error Theory 1 Scientific Survey Research 5 The Development of a Discipline 2 Survey Error 16 The Establishment of a Paradigm 3 Survey Modes 29 Responses to Emerging Technologies PA R T I I Response Accuracy Issues 4 Measurement Error Due to Interviewers 45 The Debate over Interviewing Style 5 Measurement Error Due to Respondents, I 72 Question Wording Problems 6 Measurement Error Due to Respondents, II 114 Questionnaire Issues 7 Nonresponse Error at the Item Level 130 The Scourge of Missing Data PA R T I I I Respondent Selection Issues 8 Nonresponse Error at the Unit Level 159 The Response Rate Challenge 9 Coverage Error 205 Sampling Frame Issues 10 Sampling Error 225 The Need for Representativeness PA R T I V Survey Administration Issues 11 Postsurvey Error 261 The Data Editing Stage 12 Mode Differences 278 Dealing with Sensitive Topics 13 Comparability Effects 297 The Limits of Equivalence PA R T V Total Survey Error 14 Ethics in Surveys 311 Respecting Respondents’ Rights 15 Survey Errors 325 Coping with Survey Constraints Appendix: Meta-analysis 333 Notes 337 References 345 Name Index 371 Subject Index 379 Preface In recent decades, the survey field has been revolutionized with increased attention to both its basis in social psychology and its statistical underpinnings. The “total survey error” approach provides a new paradigm by which to understand and study this field, so it is time to proclaim the true arrival of survey research as a new science. This book uses the total survey error approach to present a unified approach to understanding the components of good survey research. Survey research is an interesting research approach because the trade-offs between theory and practical considerations are quite direct. This has always been evident in survey sampling, where sampling statisticians learned early on to balance statistical theory with the difficulty of sampling in the real world. The initial knowledge of question writing was developed from practical interviewing, but in recent years the understanding of this topic has become more theoretically informed through insights from s