Executive Summary
Customer churn (i.e., when a customer decides to defect to another company) is a significant concern in many service industries. One way to decrease churn is to identify customers in advance who are at risk of churning and target an incentive to encourage them to stay. However, this requires accurate predictions about which customers are at risk. The purpose of this article is to identify which methods work best for this critical task. The authors focus on three questions: (1) Does method make a difference? (2) Which methods work best? and (3) Do models calibrated today still have predictive power months from now? The authors researched these issues using a churn-modeling tournament. They compiled a database of 100,000 customers, 171 potential predictors of churn, and a record of whether each customer churned. Thirty-three participants from industry and academia used the data to calibrate churn prediction models. They made predictions about two holdout samples that did not contain the churn indicator. The authors then “scored” each entrant for predictive accuracy and found the following:
- Methods do matter. “Top-decile lift,” a commonly used measure of accuracy, averaged 2.14 with a standard deviation of .53. The authors developed a profitability model and, using conservative assumptions, found that a one-standard-deviation improvement in accuracy could increase profits by hundreds of thousands of dollars for a company that contacts 10% of its five million customers for a churn management campaign.
- Model builders use distinct methodological approaches. These methodological approaches entail elements such as the amount of time spent on various tasks, the number of variables included in the model, and the statistical technique. The authors identify five major approaches: “logistic,” “tree,” “practical,” “discriminant,” and “explain.”
- Logistic and tree approaches perform better than the practical, discriminant, and explain approaches. Profitability calculations suggest that the moving toward a logistic or tree approach can improve profits by hundreds of thousands of dollars.
- Churn models have at least a three-month shelf life. The results show that predictive accuracy does not fall off even when a model is used to predict churn roughly three months after it is calibrated.
These findings have several implications for managers. First, it is important to be open to and to seek out new methods. Methods matter, so managers should be willing to test new methods as they become available. Second, model building is a process that needs to be managed. For example, tree approaches require much time for model estimation. This suggests that companies using decision trees should also invest in the fastest possible hardware for their model builders. Third, logistic or tree approaches are good starting points, especially for a company starting up a predictive modeling function. It may be possible to improve on these logistic or tree approaches, but in general, they perform well. Fourth, models last at least three months. There is at least three months’ time after calibrating a model during which model builders can explore new approaches or build models for other tasks.
Overall, a great deal was learned from this research. The efforts of several practitioners and academics in generating the results of this study add generality and realism. The authors encourage others to continue these efforts.
Biography
Scott A. Neslin is Albert Wesley Frey Professor of Marketing in the Tuck School of Business at Dartmouth College. He received his PhD in Management from the Sloan School of Business at the Massachusetts Institute of Technology. His research interests include sales promotion, market response models, and database marketing. He has published articles on these topics in journals such as Marketing Science, Journal of Marketing Research, Management Science, and Journal of Marketing. He is coauthor (with Professor Robert C. Blattberg) of the book Sales Promotion: Concepts, Methods, and Strategies (Prentice Hall), and more recently, he was author of the monograph Sales Promotion (Marketing Science Institute). He is an area editor for Marketing Science and is on the editorial boards of Journal of Marketing Research, Journal of Marketing, and Marketing Letters.
Sunil Gupta is Meyer Feldberg Professor of Business at Columbia Business School. His research interests include choice models, cross-category purchase behavior, and customer value. His articles in these areas have won several awards, including the O’Dell (1993, 2002) and the Paul Green (1998, 2005) awards for Journal of Marketing Research and the best-paper awards for International Journal of Research in Marketing (1999) and Marketing Science Institute (1998, 2000). He serves on the editorial boards of International Journal of Research in Marketing, Journal of Marketing, Journal of Marketing Research, and Marketing Science.
Wagner Kamakura is Ford Motor Company Professor of Global Marketing in the Fuqua School of Business at Duke University. Before joining the Fuqua School of Business, Professor Kamakura taught at the University of Iowa, the University of Pittsburgh, and Vanderbilt University. Before joining academia, he worked in market analysis, forecasting, and planning in Brazil. Professor Kamakura holds a PhD in Marketing from the University of Texas at Austin, an MS in Industrial Engineering from the University of Sao Paulo (Brazil), and a BS in Mechanical Engineering from the Technological Aeronautics Institute (Brazil). He is a coauthor of Market Segmentation: Conceptual and Methodological Foundations, as well as numerous articles in academic journals. He has served as the editor of Journal of Marketing Research, area editor of Marketing Science, and associate editor of the Journal of Consumer Research. He is currently a member of the editorial boards of Journal of Marketing Research, International Journal of Research in Marketing, Journal of Marketing, Journal of Retailing, and Marketing Science. His current research interests are in market segmentation and market structure; database marketing; and the modeling of customer satisfaction, retention, and profitability.
Junxiang Lu is currently a vice president of the Credit Risk Management Department of Comerica Bank in Auburn Hills, Mich. His major responsibilities include building and validating risk-rating models for Basel II compliance. He is also responsible for capital adequacy modeling, loan pricing, and profitability modeling. Before Comerica Bank, he held marketing analytics positions with MBNA America Bank and a major U.S. telecommunications company. His analytical experience includes marketing campaigns, partnership marketing, customer loyalty and rewards, behavior-based churn modeling, customer lifetime value, and pricing. Junxiang Lu received his BS in Mathematics from Zhejiang University in China and his PhD in Agricultural Economics from Oklahoma State University, Stillwater.
Charlotte H. Mason is Associate Professor of Marketing in the Kenan-Flagler Business School at University of North Carolina, Chapel Hill. She received her BS and MS degrees in Industrial Engineering, an MS in statistics, and her PhD in Business Administration from Stanford University. She has also been an adjunct professor in the Fuqua School of Business at Duke University (2002) and in the Terry College of Business at University of Georgia (2005). Her industry experience includes work for Procter & Gamble and Booz, Allen and Hamilton. Her research focuses on the development and testing of marketing models and applications of multivariate statistics to marketing problems. She is currently investigating issues related to the analysis and use of large customer databases and strategic issues regarding customer relationship management. Her research has been published in Marketing Science, Journal of Marketing Research, Journal of Consumer Research, Marketing Letters, and Journal of the Academy of Marketing Science, among other journals. Charlotte is coauthor (with William Perreault) of The Marketing Game!, a strategic marketing simulation.
J Marketing Research, Volume 43, Number 2, May 2006
View Table of Contents.