Główna
Journal of Business Research Measuring overconfidence: Methodological problems and statistical artifacts
Measuring overconfidence: Methodological problems and statistical artifacts
Olsson, HenrikJak bardzo podobała Ci się ta książka?
Jaka jest jakość pobranego pliku?
Pobierz książkę, aby ocenić jej jakość
Jaka jest jakość pobranych plików?
Tom:
67
Język:
english
Czasopismo:
Journal of Business Research
DOI:
10.1016/j.jbusres.2014.03.002
Date:
August, 2014
Plik:
PDF, 315 KB
Twoje tagi:
Zgłoś problem
This book has a different problem? Report it to us
Wybierz Tak, jeśli
Wybierz Tak, jeśli
Wybierz Tak, jeśli
Wybierz Tak, jeśli
udało ci się otworzyć plik
plik zawiera książkę (dozwolone są również komiksy)
treść książki jest akceptowalna
Tytuł, autor i język pliku odpowiadają opisowi książki. Zignoruj inne pola, ponieważ są one drugorzędne!
Wybierz Nie, jeśli
Wybierz Nie, jeśli
Wybierz Nie, jeśli
Wybierz Nie, jeśli
 plik jest uszkodzony
 plik jest chroniony DRM
 plik nie jest książką (np. xls, html, xml)
 plik jest artykułem
 plik jest fragmentem książki
 plik jest czasopismem
 plik jest formularzem testowym
 plik jest spamem
uważasz, że treść książki jest nieodpowiednia i powinna zostać zablokowana
Tytuł, autor lub język pliku nie pasuje do opisu książki. Zignoruj inne pola.
Are you sure you want to report this book? Please specify the reason below
Change your answer
Thanks for your participation!
Together we will make our library even better
Together we will make our library even better
Plik zostanie dostarczony na Twój email w ciągu 15 minut.
Plik zostanie dostarczony do Twojego Kindle w ciągu 15 minut.
Uwaga: musisz zweryfikować każdą książkę, którą chcesz wysłać na swój Kindle. Sprawdź swoją skrzynkę pocztową pod kątem emaila weryfikacyjnego z Amazon Kindle Support.
Uwaga: musisz zweryfikować każdą książkę, którą chcesz wysłać na swój Kindle. Sprawdź swoją skrzynkę pocztową pod kątem emaila weryfikacyjnego z Amazon Kindle Support.
Trwa konwersja do
Konwersja do nie powiodła się
0 comments
Możesz zostawić recenzję książki i podzielić się swoimi doświadczeniami. Inni czytelnicy będą zainteresowani Twoją opinią na temat przeczytanych książek. Niezależnie od tego, czy książka ci się podoba, czy nie, jeśli powiesz im szczerze i szczegółowo, ludzie będą mogli znaleźć dla siebie nowe książki, które ich zainteresują.
1


2


JBR08039; No of Pages 5 Journal of Business Research xxx (2014) xxx–xxx Contents lists available at ScienceDirect Journal of Business Research Measuring overconﬁdence: Methodological problems and statistical artifacts Henrik Olsson Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany a r t i c l e i n f o Article history: Received 1 April 2013 Received in revised form 1 November 2013 Accepted 1 January 2014 Available online xxxx Keywords: Overconﬁdence Underconﬁdence Format dependence Regression a b s t r a c t Psychological studies are frequently cited in the business and ﬁnance literature to bolster claims that various kinds of economic disasters, from the large proportion of startups that quickly go out of business to the exaggerated conﬁdence of ﬁnancial investors, can be attributed to overconﬁdence. This article reviews some of the problems associated with concluding that people overestimate the accuracy of their judgments based on observed overconﬁdence measured as the difference between mean subjective probability and proportion correct. Methodological and statistical artifacts, such as regression, can explain many of the observed instances of apparent overconﬁdence. © 2014 Published by Elsevier Inc. How conﬁdent are you that the following statement is true: “One of the most signiﬁcant and irrefutable ﬁndings of behavioral psychologists is that people are overconﬁdent in their judgments and overestimate the reliability of their judgments” (Parikh, 2009, p.142). This bold statement was given by Parag Parikh in his recent book Value Investing and Behavioral Finance. And he is not alone. A quick look in books or articles that mention overconﬁdence in cognitive psychology, judgment and decision making, behavioral economics, behavioral ﬁnance and so on, unveils similar statements (see Table 1 for examples). If you have read these books or articles, you would probably be very conﬁdent that Parikh's statement is true. Psycho; logical studies on overconﬁdence are frequently cited in the business and ﬁnance literature to bolster claims that various kinds of economic disasters, from the large proportion of startups that quickly go out of business to the exaggerated conﬁdence of ﬁnancial investors, can be attributed to overconﬁdence. As Grifﬁn and Tversky (1992) emphasized, “The signiﬁcance of overconﬁdence to the conduct of human affairs can hardly be overstated” (p. 432). In this article I will give a brief exposition of realism of conﬁdence research that challenges the prevailing view. literature, overestimation, overplacement, and calibration of subjective probabilities (or realism of conﬁdence). Overestimation is measured by comparing a person's performance with that person's belief of own performance. For example, the number of correct answers a person achieves on a test is compared to that person's estimate of how many questions she thinks she answered correctly. Overplacement is measured by comparing a person's performance with others' performances. For example, a person's actual location in a test score distribution (percentile) is compared with that person's estimate where she is located in the distribution. The calibration of subjective probabilities is measured by comparing subjective probability judgments with the corresponding objective probabilities. For example, a person's mean subjective probability estimate of choosing the correct answer on a test is compared to that person's relative frequency of correct answers. It is currently unknown to what extent these three different forms of overconﬁdence represents the same psychological construct, as only a handful of studies have investigated two or more ways of measuring overconﬁdence (but see Moore & Healy, 2008; Larrick, Burson, & Soll, 2007). In this article I will mainly focus on the last, overconﬁdence as (mis)calibration (for a discussion of variants of overconﬁdence and potential beneﬁts of functional overconﬁdence, see Gigerenzer, Fiedler, & Olsson, 2012; Moore & Healy, 2008; Mousavi & Gigerenzer, 2011). The many faces of overconﬁdence Calibration of subjective probabilities The term overconﬁdence has been used to describe many different phenomena from hubris to “unskilled and unaware of it” effects (Kruger & Dunning, 1999), measured in various ways. At least three different deﬁnitions of overconﬁdence are used in the psychological Studies in the realism of conﬁdence, or calibration, tradition usually employ one of three response formats, the halfrange format, the fullrange format, or the interval estimation format. In a halfrange task, the participants select one of two presented answers and assess the probability that the selected answer is correct on a scale from .5 to 1 (usually expressed as percentages). Introduction Email address: h.olsson@warwick.ac.uk. http://dx.doi.org/10.1016/j.jbusres.2014.03.002 01482963/© 2014 Published by Elsevier Inc. Please cite this article as: Olsson, H., Measuring overconﬁdence: Methodological problems and statistical artifacts, Journal of Business Research (2014), http://dx.doi.org/10.1016/j.jbusres.2014.03.002 2 H. Olsson / Journal of Business Research xxx (2014) xxx–xxx Table 1 Statements on the pervasiveness of overconﬁdence. Source Quote Blake (2009). The art of decisions: how to manage in an uncertain world. Harlow, “Studies have been made of physicians, clinical psychologists, lawyers, negotiators, engineers, and England: FT Prentice Hall. security analysts. The evidence is overwhelming. They all show the same systematic overconﬁdence in the accuracy of their judgments.” (p. 159) Charupat, Deaves, and Lüders (2005). Knowledge vs. knowledge perception: “You, like most people (including the authors of this article), are overconﬁdent. Your perception of implications for ﬁnancial professionals. Journal of Personal Finance, 4, 50–61. your knowledge exceeds your actual knowledge.” (p. 52) Meloy, Russo, and Miller (2006). Monetary incentives and mood. Journal of “The phenomenon of overconﬁdence is one of the most robust ﬁndings in the decision and Marketing Research, 3, 267–275. judgment literature.” (p. 272) Montier (2007). Behavioural investing: a practitioner's guide to applying “The two most common biases that psychologists have documented are overoptimism and behavioural ﬁnance. Chichester, England: Wiley. overconﬁdence […]. [I]f we ask people for a forecast and then ask them for the 98% conﬁdence intervals, so that the true answer should lie outside of the bounds just 2% of the time, it tends to lie outside the bounds 30–40% of the time! People are simply far too sure about their ability to predict.” (p. 109) Odean (1998). Volume, Volatility, Price, and Proﬁt When all traders are above “People are overconﬁdent. Overconﬁdence affects ﬁnancial markets.” (p. 1887) average. Journal of Finance, 53, 1887–1934. Redhead (2008). Personal ﬁnance and investments: A behavioral perspective. “People tend to think they are better than they really are.” (p. 24) New York: Routledge. Schaefer, Williams, Goodie, and Campbel (2004). Overconﬁdence and the big ﬁve. “It has been consistently observed that people are generally overconﬁdent when assessing their Journal of Research in Personality, 38, 473–480. performance.” (p. 473) Sternberg (2008). Cognitive psychology (5th Ed.). Belmont, CA: Wadsworth. “Another common error is overconﬁdence […]. In general, people tend to overestimate the accuracy of their judgments.” (p. 495). Which country has a population with a higher mean life expectancy? a. Indonesia 50% Random 60% b. Sudan 80% 70% 90% 100% Certain The overconﬁdence score is calculated as the difference between the mean subjective probability of a correct answer and the proportion of correct answers. A positive value indicates overconﬁdence and a negative value indicates underconﬁdence. For example, if the participants on average assess the probability that they have chosen the correct answer to be .8 (or 80%), but only have 60% correct, overconﬁdence is .2. In a fullrange task the probability that a proposition is true is assessed between 0 and 1 (again usually expressed as percentages). The population of Indonesia has a higher mean life expectancy than the population of Sudan. What is the probability that this statement is true? 0% 10% 20% Certainly false 30% 40% 50% 60% 70% 80% 90% 100% Certainly true The overconﬁdence score is computed by assuming that participants would have decided that the proposition is true when they give probability ratings larger than .5, and that the statement is false when they give ratings below .5. For probabilities below .5 the inferred halfrange probabilities are 1 minus the estimated probabilities. The probabilities of .5 are randomly assigned as favoring either the truth or the falsity of the proposition. The mean of the inferred probabilities is then compared to the inferred halfrange proportion correct in the manner of halfrange data and the resulting score is interpreted in the same way as for the halfrange method. In an interval estimation task the participant is confronted with a statement such as: Assess the (smallest) interval within which you are 80% certain (probability .8) that the mean life expectancy of the population of Indonesia lies: Between _______ years and_______ years Here overconﬁdence scores are computed from the difference between the stated probability interval and the observed event proportion within this interval. The data from calibration studies are usually summarized in a calibration diagram. Fig. 1A shows a calibration diagram adapted from Juslin (1994) with data from two conditions (the conditions are explained below in the section on overconﬁdence as a consequence of nonrepresentative stimulus sampling). In the selected condition the calibration curve shows the hallmarks of overconﬁdence as it is below the diagonal, which represent perfect calibration, throughout most of the range (the overconﬁdence score is .08). In the random condition the calibration curve is closer to the diagonal with a tendency toward underconﬁdence in the lower part (the overconﬁdence score is − 0.04). Note that miscalibration in the sense of regression of the calibration curve around the midpoint of the scale does not imply overall overconﬁdence for the halfrange format, the difference between mean subjective probability and proportion correct can still be zero even when the calibration curve has a slope less than one. Fig. 1B shows a fullrange calibration curve from Juslin, Winman, and Olsson (2003). Here the curve is very close to the diagonal and there is close to zero overconﬁdence (the overconﬁdence score is 0.02). In contrast to the halfrange calibration curve, regression around the midpoint in the fullrange curve implies more overconﬁdence. These differences between the two response formats are important to keep in mind for the discussion below of regression as an explanation of observed overconﬁdence in fullrange tasks and the ﬁnding that different response formats elicit different amounts of overconﬁdence. The most common ﬁnding with general knowledge items has been the overconﬁdence phenomenon, where the mean subjective probability exceeds the overall proportion correct (see Keren, 1991; Lichtenstein, Fischhoff, & Phillips, 1982; McClelland & Bolger, 1994, and Yates, 1990, for reviews). The early explanations of overconﬁdence revolved around the idea of a general processing bias, either due to motivational factors (Taylor & Brown, 1988) or cognitive factors (e.g., Koriat, Lichtenstein, & Fischhoff, 1980). In the early nineties a new and somewhat provocative explanation of the overconﬁdence phenomenon appeared in terms of what has been referred to as the ecological models (McClelland & Bolger, 1994). The most well known and elaborate theory was the theory of probabilistic mental models (Gigerenzer, Hoffrage, & Kleinbölting, 1991; see also Björkman, 1994; Juslin, 1993, 1994; Juslin, Olsson, & Björkman, 1997). The many problems with overconﬁdence Overconﬁdence as a consequence of nonrepresentative item sampling The main idea incorporated in the ecological models was that in many circumstances, participants may be reasonably calibrated to the probability structure of their environment. This was premised on cognitive adjustment, that is, that participants have had sufﬁcient experience with their environment to acquire accurate cognitive representations of the ecological probabilities (Gigerenzer et al., 1991). For a number of Please cite this article as: Olsson, H., Measuring overconﬁdence: Methodological problems and statistical artifacts, Journal of Business Research (2014), http://dx.doi.org/10.1016/j.jbusres.2014.03.002 H. Olsson / Journal of Business Research xxx (2014) xxx–xxx 3 reasons (cf. Juslin, 1993, 1994), the traditional strategies of item selection have the sideeffect of overrepresenting items where the probabilistic inferences lead to erroneous answers, at the expense of items where the same inferences are successful. Thus, if items are randomly selected from a reference class of objects deﬁned by a natural environment, overconﬁdence should diminish or even disappear. A number of studies provided support for this hypothesis (e.g., Gigerenzer et al., 1991; Juslin, 1993, 1994; Winman, 1997). The difference between selected sampling (i.e., the condition included items that naïve participants had chosen to be good knowledgediscriminating items in a pilot study) and random sampling is illustrated in Fig. 1A. Some later studies challenged this conclusion (e.g., Grifﬁn & Tversky, 1992), but the issue of the effects of different sampling schemas on overconﬁdence in general knowledge tasks with the halfrange format appears to have been resolved with the metaanalytic review in Juslin et al. (2000). They analyzed 130 studies with and without random sampling to see what the evidence really says. They showed that overconﬁdence bias indeed disappeared across all 35 studies with random sampling, with the difference between mean conﬁdence and mean proportion correct being indistinguishable from zero. Overconﬁdence as a consequence of regression A more recent proposal is that overconﬁdence may reﬂect a regressionlike sideeffect of stochastic components of the judgment process (Erev, Wallsten, & Budescu, 1994; Pfeifer, 1994; Soll, 1996; Dawes & Mulford, 1996). The argument is simple. Due to the merely correlative relation between subjective and objective probabilities there will be regression when one of the variables is plotted against the other. An imperfect correlation implies that when the reported conﬁdence ratings are high, the corresponding proportions correct will be smaller, looking like overconﬁdence. For instance, when one looks at all cases where people said that they were “90% conﬁdent that the statement is true,” the mean proportion of correct answers will be lower, such as 80%, depending on the exact correlation between conﬁdence and proportion correct. If one estimates the conﬁdence judgments from proportion correct (rather than vice versa), then one should get the mirror result: a pattern that looks as if there was underconﬁdence bias. So, for instance, when one looks at all items that the participants got 100% correct, one will ﬁnd that the average conﬁdence was lower, such as 80%. This appears to be underconﬁdence. In contrast, when one looks at all items for which participants were 100% conﬁdent, one ﬁnds that the average proportion correct was lower, such as 80%. This appears to be overconﬁdence. Erev et al. (1994) showed for three empirical data sets that regression toward the mean accounted for practically all the effects that would otherwise have been attributed to overconﬁdence or underconﬁdence, depending on how one plotted the data. The hard–easy effect as a consequence of regression Regression and regression like effects can also explain another pervasive ﬁnding in calibration research that people seem to be underconﬁdent in easy tasks and overconﬁdent in hard tasks. In the absence of any bias, regression toward the mean implies that the largest positive difference will be found for easy items, that is, when proportion correct is high. Regression also implies that this difference will become smaller, and eventually turn into a negative difference, when items become more and more difﬁcult. In other words, regression toward the mean alone produces the pattern that has been interpreted as a cognitive hard–easy effect. There are also several other problems associated with the interpreting the hard–easy effect (Juslin et al., 2000). One is the linear dependency between overconﬁdence and proportion correct. As proportion correct is part of the overconﬁdence score, measurement error in proportion correct alone can produce the hard–easy effect. Fig. 1. Panel A shows the halfrange empirical calibration curves for two conditions in Juslin (1994). The dotted line represents perfect calibration. Adapted from “The overconﬁdence phenomenon as a consequence of informal experimenterguided selection of almanac items”, by Juslin (1994) Organizational Behavior and Human Decision Processes, 57, p. 238. Copyright 1994 by Academic Press Inc. Panel B shows a fullrange empirical calibration curve from Juslin et al. (2003). The dotted line represents perfect calibration. Adopted from “Calibration, additivity, and source independence of probability judgments in general knowledge and sensory discrimination tasks”, by Juslin et al. (2003) Organizational Behavior and Human Decision Processes, 92, p. 44. Copyright 2003 by Elsevier Science. Another problem is scaleend effects. For example, in a half range task when proportion correct is .5 or less overconﬁdence can only be 0 or positive and when proportion correct is 1 overconﬁdence can only be less than zero or negative and response error in the use of the probability scale will appear as a hard–easy effect as the error has only one way to go. The hard–easy effect tells us very little about how good or bad people's judgments are. Merkle (2009) derived mathematically necessary and sufﬁcient conditions for observing a hard–easy effect under realistic assumptions. He concluded that “both perfectly calibrated judges (at the item level) and terrible judges (whose conﬁdence is unrelated to the stimulus) exhibit the hard–easy effect” (p. 211). Overconﬁdence as a result of task format We already saw that overconﬁdence decreases or disappears when items are randomly sampled from an environment and turns into underconﬁdence if objective probability is treated as the independent variable. To what extent a study ﬁnds overconﬁdence is also highly dependent on the response format used. Format dependence refers to the ﬁnding that you can simultaneously observe underconﬁdence and Please cite this article as: Olsson, H., Measuring overconﬁdence: Methodological problems and statistical artifacts, Journal of Business Research (2014), http://dx.doi.org/10.1016/j.jbusres.2014.03.002 4 H. Olsson / Journal of Business Research xxx (2014) xxx–xxx overconﬁdence for the same tasks depending on the choice of response format (Juslin, Wennerholm, & Olsson, 1999; Juslin et al., 1997). For example, the two experiments in Juslin et al. (1999) found underconﬁdence for the halfrange format, close to zero or moderate overconﬁdence for the full range format and massive overconﬁdence for the interval estimation format. The difference between the halfrange format and the fullrange format is a straight forward consequence of how response error interacts with the response format to produce overconﬁdence. Simulations in Juslin et al. (1997) show that even perfectly calibrated responses that have been perturbed by response error (e.g., emanating from inconsistencies in how the probability scale is used) exhibit format dependence effects, with more overconﬁdence for the fullrange format than for the half range format. The explanation for this is that the regression, or rotation, of the calibration curve in the full range format is centered approximately around the mean of the full range scale (0.5), whereas with the halfrange format the rotation is centered around the mean of the half range scale (0.75). This means that with the fullrange format the response error effects are perfectly confounded with the standard measure of overconﬁdence, that is, more response error pulls toward more overconﬁdence. With the halfrange format the effect might either be over or underconﬁdence depending on the task difﬁculty. The response error account also predicts more overconﬁdence for the interval estimation format than for the other two formats, but not the extreme levels of overconﬁdence that is usually observed in interval estimation tasks. Recently, however, a new model based on the idea that people na vely use sample estimates of variability to form intervals, predicts the observed levels of overconﬁdence (Juslin, Winman, & Hansson, 2007). Also, it has been shown that the massive overconﬁdence in the interval estimation format can be turned into close to zero overconﬁdence if participants instead evaluate given intervals (e.g., “The population of Thailand lies between X and Y million. Assess the probability that the statement is correct”; Winman, Hansson, & Juslin, 2004). Underconﬁdence in sensory discrimination Another ﬁnding that speaks against the general assertion that people are overconﬁdent is the pervasive underconﬁdence found in sensory discrimination tasks. The sensory domain was only sporadically explored before the nineties (e.g., Keren, 1988; although the results in the classic study by the father of American pragmatism, Peirce & Jastrow, 1884 can be viewed as the ﬁrst demonstration of underconﬁdence in sensory discrimination) and the tasks were not strictly sensory in nature (e.g., Dawes, 1980). The ﬁrst study that truly explored the sensory domain with a weight discrimination task observed pervasive underconﬁdence (Björkman, Juslin, & Winman, 1993). This study and others (e.g., Winman & Juslin, 1993) led to a debate about possible differences in realism of conﬁdence in the cognitive and the sensory domains (e.g., Baranski & Petrusic, 1994; Olsson & Winman, 1996), where some researchers claimed that there was no difference in over/underconﬁdence between the two domains (e.g., Baranski & Petrusic, 1994). By now it seems safe to conclude that there is a difference between the two domains in terms of calibration, with more underconﬁdence in the sensory domain. For example, Juslin, Olsson, and Winman (1998), reviewed aggregated data from 21 sensory discrimination tasks and 44 general knowledge inferential tasks with items randomly sampled from a reference class of objects deﬁned by a natural environment, a clear main effect of sensory versus cognitive tasks revealed more underconﬁdence for sensory discrimination (.01 vs. − .10; see also Juslin & Olsson, 1997, in which the authors present a computational model that predicts underconﬁdence in sensory discrimination). Overconﬁdence in business and ﬁnance So far, I have reviewed studies from psychology and judgment and decision making highlighting some of the problems associated with concluding that people overestimate the accuracy of their judgments based on observed overconﬁdence scores. In business and ﬁnance, however, these studies have had little impact. They continue to cite that researchers take the (apparent) overconﬁdence observed in psychological experiments as evidence that most people are overconﬁdent most of the time and use it as an assumption in their models. For example, Odean (1998) uses the assertion that people are overconﬁdent (conceptualized as overestimation of the precision of private information) as an assumption in his models of traders in ﬁnancial markets and concludes that, among other things, that overconﬁdence increases expected trading volume and decreases the expected utility of overconﬁdent traders. Overconﬁdence in his models, however, is only an untested assumption and in many of the studies that test Odean's models “overconﬁdence is neither directly observed nor manipulated” (Barber & Odean, 2001, p. 264). Direct tests of the link between overconﬁdence and trading volume are rare. Recent evidence, however, suggests that some measures of overconﬁdence are unrelated to trading volume. Glaser and Weber (2007) investigated the relation between trading volume and two measures of overconﬁdence, one derived from an interval estimation task and another derived from a better than average task. Using data from approximately 3000 online brokers they found that overconﬁdence in the interval estimation task was unrelated to trading volume, while an index of the better than average effect traded more. This result is consistent with results from experimental markets, where overconﬁdence measured by an interval estimation task is unrelated to trading volume (Biais, Hilton, Mazurier, & Pouget, 2005). Does overconﬁdence exist? A reader might ask, “So, does overconﬁdence exist?” The message of this article is not that all instances of observed overconﬁdence are more apparent than real. Rather, the message is that researchers need to be careful in making assumptions about overconﬁdence and its effects on behavior without considering the conceptual, statistical, and methodological problems associated with concepts of overconﬁdence. The answer, then, must be: “It depends on what you mean by overconﬁdence and how you measured it”. This might seem unsatisfactory, but the concept of overconﬁdence has so many different meanings and can be measured in so many ways, that a more speciﬁc answer cannot be given without knowing which form of overconﬁdence is studied and what methods were used to elicit answers from people. In the case of overconﬁdence as miscalibration, the topic of most of this article, some concrete advice can be given. 1) Choose your sample of items carefully, preferably by randomly sample a large pool of items. 2) If a researcher is interested in overconﬁdence for different levels of difﬁculty, some of the linear dependency between overconﬁdence and proportion correct should be removed by estimating proportion correct and overconﬁdence on different item sets (i.e., the same estimate of proportion correct never enters twice, both as an independent variable and as part of the dependent variable overconﬁdence, see Klayman, Soll, GonzálezVallejo, & Barlas, 1999). 3) Beware of scale end and regression effects. Random response error pulls higher probability estimates downwards and lower estimates upwards. This is difﬁcult to correct for, but models that include an error parameter can be used (Juslin et al., 2000). 4) The traditional interval estimation format is probably not advisable to use. Instead, has participants evaluate given intervals (e.g., “The population of Indonesia lies between X and Y million. Assess the probability that the statement above is correct”). Conclusion This article reviews some of the problems associated with concluding that people overestimate the accuracy of their judgments based on observed overconﬁdence scores. Methodological and statistical artifacts can explain many of the observed instances of apparent overconﬁdence. Please cite this article as: Olsson, H., Measuring overconﬁdence: Methodological problems and statistical artifacts, Journal of Business Research (2014), http://dx.doi.org/10.1016/j.jbusres.2014.03.002 H. Olsson / Journal of Business Research xxx (2014) xxx–xxx In the business and ﬁnance literature, however, overconﬁdence is for the most part taken as a given. Little consideration is given to the many ways overconﬁdence can be conceptualized and the methodological and statistical artifacts that come along with attempts of measuring some forms of overconﬁdence. There is clearly a need for more research investigating the assumptions in ﬁnance and business models as well as the proxies for overconﬁdence used in the empirical literature. References Baranski, J. V., & Petrusic, W. M. (1994). The calibration and resolution of conﬁdence in perceptual judgments. Perception & Psychophysics, 55, 412–428. Barber, B.M., & Odean, T. (2001). Boys will be boys: Gender, overconﬁdence, and common stock investment. Quarterly Journal of Economics, 116, 261–292. Biais, B., Hilton, D., Mazurier, K., & Pouget, S. (2005). Judgmental overconﬁdence, selfmonitoring, and trading performance in an experimental ﬁnancial market. Review of Economic Studies, 72, 287–312. Björkman, M. (1994). Internal cue theory: Calibration and resolution of conﬁdence in general knowledge. Organizational Behavior and Human Decision Processes, 58, 386–405. Björkman, M., Juslin, P., & Winman, A. (1993). Realism of conﬁdence in sensory discrimination: The underconﬁdence phenomenon. Perception & Psychophysics, 54, 75–81. Blake, C. (2009). The art of decisions: How to manage in an uncertain world. Harlow, England: FT Prentice Hall. Charupat, N., Deaves, R., & Lüders, E. (2005). Knowledge vs. knowledge perception: Implications for ﬁnancial professionals. Journal of Personal Finance, 4, 50–61. Conﬁdence in intellectual judgments vs. conﬁdence in perceptual judgments. Lanterman, E. D., & Feger, H. (Eds.). (1980). Similarity and choice: Papers in honor of Clyde Coombs (pp. 327–345). Bern, Switzerland: Hans Huber. Dawes, R. M., & Mulford, M. (1996). The false consensus effect and overconﬁdence: Flaws in judgment or ﬂaws in how we study judgment? Organizational Behavior and Human Decision Processes, 65, 201–211. Erev, I., Wallsten, T. S., & Budescu, D.V. (1994). Simultaneous over and underconﬁdence: The role of error in judgment processes. Psychological Review, 101, 519–527. Gigerenzer, G., Fiedler, K., & Olsson, H. (2012). Rethinking cognitive biases as environmental consequences. In P. M. Todd, G. Gigerenzer and the ABC Research Group, (Eds.), Ecological rationality: Intelligence in the world (pp. 80–110). New York: Oxford University Press. Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswikian theory of conﬁdence. Psychological Review, 98, 506–528. Glaser, M., & Weber, M. (2007). Overconﬁdence and trading volume. The Geneva risk and insurance review, 32, 1–36. Grifﬁn, D., & Tversky, A. (1992). The weighing of evidence and the determinants of conﬁdence. Cognitive Psychology, 24, 411–435. Juslin, P. (1993). An explanation of the hard–easy effect in studies of realism of conﬁdence in one's general knowledge. European Journal of Cognitive Psychology, 5, 55–71. Juslin, P. (1994). The overconﬁdence phenomenon as a consequence of informal experimenterguided selection of almanac items. Organizational Behavior and Human Decision Processes, 57, 226–246. Juslin, P., & Olsson, H. (1997). Thurstonian and Brunswikian origins of uncertainty in judgment: A sampling model of conﬁdence in sensory discrimination. Psychological Review, 104, 344–366. Juslin, P., Olsson, H., & Björkman, M. (1997). Brunswikian and Thurstonian origins of bias in probability assessment: On the interpretation of stochastic components of judgment. Journal of Behavioral Decision Making, 10, 189–209. Juslin, P., Olsson, H., & Winman, A. (1998). The calibration issue: Theoretical comments on Suantak, Bolger, and Ferrell (1996). Organizational Behavior and Human Decision Processes, 73, 3–26. Juslin, P., Wennerholm, P., & Olsson, H. (1999). Format dependence in subjective probability calibration. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1038–1052. Juslin, P., Winman, A., & Hansson, P. (2007). The naive intuitive statistician: A na ve sampling model of intuitive conﬁdence intervals. Psychological Review, 114, 678–703. Juslin, P., Winman, A., & Olsson, H. (2000). Naive empiricism and dogmatism in conﬁdence research: A critical examination of the hard–easy effect. Psychological Review, 107, 384–396. 5 Juslin, P., Winman, A., & Olsson, H. (2003). Calibration, additivity, and source independence of probability judgments in general knowledge and sensory discrimination tasks. Organizational Behavior and Human Decision Processes, 92, 34–51. Keren, G. (1988). On the ability of monitoring nonveridical perceptions and uncertain knowledge: Some calibration studies. Acta Psychologica, 67, 95–119. Keren, G. (1991). Calibration and probability judgments: Conceptual and methodological issues. Acta Psychologica, 77, 217–273. Klayman, J., Soll, J., GonzálezVallejo, C., & Barlas, S. (1999). Overconﬁdence: It depends on how, what, and whom you ask. Organizational Behavior and Human Decision Processes, 79, 216–247. Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for conﬁdence. Journal of Experimental Psychology: Human Learning and Memory, 6, 107–118. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difﬁculties in recognizing one's own incompetence lead to inﬂated selfassessments. Journal of Personality and Social Psychology, 77, 1121–1134. Larrick, R. P., Burson, K. A., & Soll, J. B. (2007). Social comparison and conﬁdence: When thinking you're better than average predicts overconﬁdence (and when it does not). Organizational Behavior and Human Decision Processes, 102, 76–94. Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of subjective probabilities: The state of the art up to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 306–334). New York: Cambridge University Press. McClelland, A. G. R., & Bolger, F. (1994). The calibration of subjective probabilities: Theories and models 1980–1993. In G. Wright, & P. Ayton (Eds.), Subjective probability (pp. 453–482). Chichester, England: Wiley. Meloy, M. G., Russo, J. E., & Miller, E. G. (2006). Monetary incentives and mood. Journal of Marketing Research, 3, 267–275. Merkle, E. C. (2009). The disutility of the hard–easy effect in choice conﬁdence. Psychonomic Bulletin and Review, 16, 204–213. Montier, J. (2007). Behavioural investing: A practitioner's guide to applying behavioural ﬁnance. Chichester, England: Wiley. Moore, D. A., & Healy, P. J. (2008). The trouble with overconﬁdence. Psychological Review, 115, 502–517. Mousavi, S., & Gigerenzer, G. (2011). Revisiting the “error” in studies of cognitive errors. In D. A. Hofmann, & M. Frese (Eds.), Errors in organizations (pp. 97–112). New York: Taylor & Francis. Odean, T. (1998). Volume, volatility, price, and proﬁt when all traders are above average. Journal of Finance, 53, 1887–1934. Olsson, H., & Winman, A. (1996). Underconﬁdence in sensory discrimination: The interaction between experimental setting and response strategies. Perception & Psychophysics, 58, 374–382. Parikh, M. (2009). Value investing and behavioral ﬁnance: Insights into Indian stock market realities. New Delhi: Tata McGraw Hill. Peirce, C. S., & Jastrow, J. (1884). On small differences of sensation. Memoirs of the National Academy of Sciences, 3, 73–83. Pfeifer, P. E. (1994). Are we overconﬁdent in the belief that probability forecasters are overconﬁdent? Organizational Behavior and Human Decision Processes, 58, 203–213. Redhead, K. (2008). Personal ﬁnance and investments: A behavioural perspective. New York: Routledge. Schaefer, P.S., Williams, C. C., Goodie, A. S., & Campbel, W. K. (2004). Overconﬁdence and the big ﬁve. Journal of Research in Personality, 38, 473–480. Soll, J. B. (1996). Determinants of overconﬁdence and miscalibration: The roles of random error and ecological structure. Organizational Behavior and Human Decision Processes, 65, 117–137. Sternberg, R. J. (2008). Cognitive psychology (5th ed.). Belmont, CA: Wadsworth. Taylor, S. E., & Brown, J.D. (1988). Illusion of wellbeing: A social psychological perspective on mental health. Psychological Bulletin, 103, 193–210. Winman, A. (1997). The importance of item selection in “knewitallalong” studies of general knowledge. Scandinavian Journal of Psychology, 38, 63–72. Winman, A., Hansson, P., & Juslin, P. (2004). Subjective probability intervals: How to reduce overconﬁdence by interval evaluation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 1167–1175. Winman, A., & Juslin, P. (1993). Calibration of sensory and cognitive judgments: Two different accounts. Scandinavian Journal of Psychology, 34, 135–148. Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs, NJ: Prentice Hall. Please cite this article as: Olsson, H., Measuring overconﬁdence: Methodological problems and statistical artifacts, Journal of Business Research (2014), http://dx.doi.org/10.1016/j.jbusres.2014.03.002