Executive Summary
The purpose of this article is to find designs that allow for precise predictions of consumers’ choices rather than to search for designs that produce only an efficient estimation of the underlying statistical model. To this end, the authors employ two prediction-based criteria, the G- and V-optimality criteria, and compared them with the estimation-based D- and A-optimality criteria traditionally used in the marketing literature. The authors implement these criteria in a Bayesian manner to account for the fact that the parameters of the statistical model, the multinomial logit model, are unknown.
Two examples, one involving an experiment with two alternatives per choice set and another with three alternatives in each set, illustrate the main points of the article. Both examples comprise the same total number of alternatives. In addition to predictive validity, the authors evaluate the two- and three-alternative Bayesian D-, A-, G-, and V-optimal designs with respect to level overlap, utility balance, estimation performance, and computational effectiveness.
As they expected, the results indicate that the Bayesian V-optimal designs and, to a lesser extent, the Bayesian G-optimal designs are best suited for predictive purposes. The D-optimal designs rank third in this aspect, but the differences in predictive ability compared with the V- and G-optimal designs are rather small. The results also show that the three-alternative optimal designs lead to better predictions than the two-alternative designs and to more accurate parameter estimates. There is no substantial difference in estimation performance among the distinct optimality criteria.
A serious problem with the Bayesian V- and G-optimal designs is that their computation takes a long time compared with the Bayesian D- and A-optimal designs, which are much faster to compute. The large computation times makes the use of the V- and G-optimality criteria for many practical problems almost impossible; thus, it seems preferable to use the D-optimality criterion to build optimal choice designs, even when the prediction of consumers’ choices is of primary concern.
Finally, the Bayesian optimal designs are not utility balanced. Instead, the choice tasks administered to the respondents exhibit a moderate complexity. This finding is counter to the general belief that proper choice designs must be utility balanced. The Bayesian A-optimal designs provide the most difficult choice tasks, whereas the V-optimal designs entail the least complicated choice tasks.
Biography
Roselinde Kessels is a doctoral candidate in the Department of Decision Sciences and Information Management at the Katholieke Universiteit Leuven (Belgium). Her research examines the construction of optimal designs for the measurement of consumer preferences, specifically, designs for conjoint and discrete choice experiments.
Peter Goos is Professor of Statistics at the Universiteit Antwerpen (Belgium) and is currently a vice president of the European Network on Business and Industrial Statistics. His research interests are the design and analysis of experiments in general. Much of his work examines statistical and practical aspects of experimentation in industry. Currently, he is also working on the design of marketing experiments. His work has been published in Technometrics, Journal of Quality Technology, and Journal of Statistical Planning and Inference, among others. He serves on the editorial board of Journal of Quality Technology.
Martina Vandebroek is Professor of Statistics in the Department of Decision Sciences and Information Management at the Katholieke Universiteit Leuven (Belgium). Her research focuses on optimal and sequential experimental design and on efficient design of conjoint and discrete choice experiments. She also has expertise in statistical modeling, multivariate data analysis, and optimization. Her work has appeared in journals such as Technometrics, Journal of Quality Technology, Computational Statistics and Data Analysis, Journal of Statistical Planning and Inference, and European Journal of Operational Research.
J Marketing Research, Volume 43, Number 3, August 2006
View Table of Contents.