Aric Rindfleisch, Alan J. Malter, Shankar Ganesan, and Christine Moorman
Executive Summary
To understand, explain, and predict marketplace behavior, marketing academics and practitioners ask questions. Although these questions take many forms, they often appear as items in surveys of managers or consumers. In recent years, editors, reviewers, and authors of leading marketing journals have become increasingly concerned about the validity of survey research. Two issues dominate these concerns: (1) common method variance (CMV) (i.e., systematic method error due to use of a single rater or single source) and (2) causal inference (CI) (i.e., the ability to infer causation from observed empirical relationships). Combined, these issues present a serious threat to the validity of findings from survey-based marketing studies.
Although the subject of these concerns is survey research in general, these issues are especially critical for cross-sectional research (i.e., surveys completed by a single respondent at a single point in time). This rising concern about the validity of cross-sectional surveys is an important issue because this method represents the most common form of field research in many areas and thus provides a critical foundation for much of the knowledge of these topics.
To reduce the threat of CMV bias and enhance CI, survey researchers typically recommend three different data collection strategies: (1) employing multiple respondents, (2) obtaining multiple types of data, or (3) gathering data over multiple periods. All three strategies are capable of creating separation between the collection of independent and dependent variables, which in theory should reduce the hazard of CMV and increase CI. Unfortunately, this view is seldom tested because few survey studies employ any of these data collection strategies. Consequently, the marketing literature provides little guidance with regard to the effectiveness of these data collection strategies in terms of reducing CMV or enhancing CI.
The objective of this article is to address this gap in the marketing literature by providing a conceptual and empirical assessment of the value of collecting data over multiple periods (i.e., longitudinal data). Longitudinal data collection as a means of reducing CMV and enhancing CI would appear to be a worthy endeavor. However, longitudinal surveys demand additional expenditures in terms of time and money. These expenses are often prohibitive for academic researchers faced with limited budgets and marketing practitioners faced with limited time. Consequently, longitudinal survey research is easier to advocate than to implement. Moreover, longitudinal studies raise several potential problems, such as confounds due to intervening events and a reduction in sample size due to respondents dropping out over time. Thus, although longitudinal data collection is desirable, it has limitations.
This article examines the relative merits of cross-sectional versus longitudinal data collection. This examination begins by providing a conceptual review of the value of longitudinal data collection in terms of addressing CMV bias and CI. The authors then perform a comparative assessment of these two validity threats for cross-sectional versus longitudinal data using two survey data sets using two approaches. First, they examine these threats using two survey data sets focused on collaborative new product development. Second, they conduct a Monte Carlo simulation that tests a wider range of parameters. Collectively, these analyses highlight the conditions under which longitudinal data collection is likely to be most valuable in terms of reducing CMV or enhancing CI. On the basis of these insights, the authors offer a set of guidelines to help marketing scholars and practitioners decide whether to invest in a longitudinal survey approach.
The article’s conceptual arguments and empirical results suggest that though longitudinal surveys may offer some advantages in terms of reducing these two validity threats, a cross-sectional approach may be adequate in many situations. Specifically, this research reveals that cross-sectional data are most appropriate for studies that examine concrete and externally oriented constructs, sample highly educated respondents, employ a diverse array of measurement formats and scales, and are strongly rooted in theory. In contrast, a longitudinal approach appears most appropriate when the temporal nature of the phenomena is clear, when it is unlikely that intervening events could confound a follow-up study, and when alternative explanations are likely and cannot be controlled through a cross-sectional approach. To maximize the validity of either approach, researchers need to employ a combination of strong theory, careful survey design, and appropriate statistical tools.
Biography
Aric Rindfleisch is Associate Professor of Marketing at the University of Wisconsin–Madison. He teaches courses on New Product Management and Marketing Strategy. Aric’s research, which focuses on understanding interorganizational relationships, consumption values, and new product development, has been published in Journal of Marketing, Journal of Marketing Research, Journal of Consumer Research, Journal of Consumer Psychology, Journal of Public Policy & Marketing, Marketing Letters, Strategic Management Journal, and Business Horizons. He is an area editor for International Journal of Research in Marketing and a member of the editorial review boards for Journal of Marketing, Journal of Consumer Research, Journal of the Academy of Marketing Science, and Journal of International Marketing. More information about Aric can be found at http://research3.bus.wisc.edu/course/view.php?id=156.
Alan J. Malter is Associate Professor of Marketing at the University of Illinois at Chicago. His research examines the changing role of marketing and competitive strategy, industry clusters and the effects of geographic proximity on innovation and knowledge transfer, tacit knowledge in managerial and consumer decision making, and related methodological issues. His research has appeared in Journal of Marketing, MIT Sloan Management Review, International Journal of Research in Marketing, Journal of Consumer Psychology, Journal of Product Innovation Management, and Organizational Dynamics, among other sources. He previously worked as a trade analyst and consultant on export development for The World Bank. He received his PhD from the University of Wisconsin–Madison.
Shankar Ganesan is the Office Depot Professor of Marketing in the Department of Marketing at the Eller College of Management, University of Arizona. Professor Ganesan’s research interests focus on the areas of interorganizational relationships, buyer–seller negotiations, service failure and recovery, and new product innovation. He is the author of several articles that have appeared in leading academic journals, including Journal of Marketing Research, Journal of Marketing, Journal of Retailing, Journal of Personal Selling and Sales Management, Journal of Academy of Marketing Science, Journal of Applied Psychology, Organizational Behavior and Human Decision Processes, and MIT Sloan Management Review. Professor Ganesan has received numerous awards and honors for his research and teaching, including the American Marketing Association’s Louis W. Stern Award for the Best Article on Marketing Channels and Distribution, Robert D. Buzzell/MSI Best Paper Award from the Marketing Science Institute, and Eller College BPA Student Council’s Faculty Appreciation Award. He teaches Sales Management to undergraduates, Customer Relationship Management to MBAs, and a seminar on Marketing Strategy to doctoral students.
Christine Moorman is the T. Austin Finch Professor Sr. of Business Administration in the Fuqua School of Business at Duke University. Chris’s research focuses on understanding the nature and effects of market information utilization activities by consumers, managers, and organizations. She is particularly interested in how information utilization affects the design and implementation of marketing strategies, marketing alliances, and innovation, as well as the effective functioning of markets. Her work has been published in Journal of Marketing Research, Journal of Consumer Research, Marketing Science, Journal of Marketing, Journal of Public Policy & Marketing, Academy of Management Review, and Administrative Science Quarterly. Chris is on the editorial review boards for Journal of Marketing Research (Associate Editor), Journal of Marketing, Marketing Science, Journal of Consumer Research, Journal of Public Policy & Marketing, Marketing Letters, and Journal of Strategic Marketing. Her research has been supported by more than a dozen grants from the Marketing Science Institute and two grants from the National Science Foundation. Chris has served on the board of directors of the American Marketing Association (AMA), as director of public policy for the Association of Consumer Research, and as chairperson of the Marketing Strategy Special Interest Group (AMA), and she is currently an academic trustee for the Marketing Science Institute. Chris’s complete biographical and publication information is located at http://www.fuqua.duke.edu/faculty/alpha/moorman.htm.
Journal of Marketing Research, Vol. XLV, No. 3, June 2008
View Table of Contents.