Unskilled and Unaware of It: How Difficulties in Recognizing One's Own
Incompetence Lead to Inflated Self-Assessments
Justin Kruger and David Dunning
Cornell University
I will read the paper, but here is the abstract, and a quote from the 1st paragraph which reminded me of an article I read...I'll tell you why after you read the quote.
Abstract: "People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.
(Now you know that I'm wondering if there is a gender breakdown in this assessment--just from having a a fresh dose of that from our last paper. I seldom break down people/issues into categories--liberal/conservative, male/female etc.).
Quote: ""It is one of the essential features of such incompetence that the person
so afflicted is incapable of knowing that he is incompetent. To have
such knowledge would already be to remedy a good portion of the
offense. (Miller, 1993, p. 4)"
--------------------------------------------------------
The beginning quote reminded me of an article that I read (I forget where/when) that stated that depressed people had a more accurate view of themselves than non-depressed people. It had to do with a natural filter that people have (which probably protects them from the mentally scarring cognition of their incompetence. What made me think of that was the quote.....There is a double whammy in the above. First, you are incompetent (and who wants to be labeled as incompetent?) and second, you are so afflicted that you do not realize it.
-------------------------------------------------------
More later. It is early, and no caffeine is coursing through my veins yet.
----------------------------------------------------
9:14 a.m post caffeine, post read and post breakfast:
First: The authors note that gender failed to qualify the results.
Second: The phrase "Unconscious Incompetence" kept resonating in my brain.
Here are some of some first impressions of the article. I certainly have no qualifications for any critical assessment (and Russell, I hope that none of this sounds like I'm dissing your paper) and have sufficient metacognition abilities to recognize such. I have to admit that I have a bit of a bias from a previous life in having to understand study biases related to a very small sliver of the world. But I have NO professional competence in this area.
Before jumping in, let's look at the definition of metacognition. This is from Jennifer Livingston's document that you can find here:
"Metacognition" is one of the latest buzz words in educational psychology, but what exactly is metacognition? The length and abstract nature of the word makes it sound intimidating, yet its not as daunting a concept as it might seem. We engage in metacognitive activities everyday. Metacognition enables us to be successful learners, and has been associated with intelligence (e.g., Borkowski, Carr, & Pressley, 1987; Sternberg, 1984, 1986a, 1986b). Metacognition refers to higher order thinking which involves active control over the cognitive processes engaged in learning. Activities such as planning how to approach a given learning task, monitoring comprehension, and evaluating progress toward the completion of a task are metacognitive in nature. Because metacognition plays a critical role in successful learning, it is important to study metacognitive activity and development to determine how students can be taught to better apply their cognitive resources through metacognitive control.-------------------------------------------------
"Metacognition" is often simply defined as "thinking about thinking." In actuality, defining metacognition is not that simple. Although the term has been part of the vocabulary of educational psychologists for the last couple of decades, and the concept for as long as humans have been able to reflect on their cognitive experiences, there is much debate over exactly what metacognition is. One reason for this confusion is the fact that there are several terms currently used to describe the same basic phenomenon (e.g., self-regulation, executive control), or an aspect of that phenomenon (e.g., meta-memory), and these terms are often used interchangeably in the literature. While there are some distinctions between definitions (see Van Zile-Tamsen, 1994, 1996 for a full discussion), all emphasize the role of executive processes in the overseeing and regulation of cognitive processes.
The term "metacognition" is most often associated with John Flavell, (1979). According to Flavell (1979, 1987), metacognition consists of both metacognitive knowledge and metacognitive experiences or regulation. Metacognitive knowledge refers to acquired knowledge about cognitive processes, knowledge that can be used to control cognitive processes. Flavell further divides metacognitive knowledge into three categories: knowledge of person variables, task variables and strategy variables.
In reading the study, a few things struck me (I'm sure that Russell will come to my rescue in my mishandling of these!)
- Overall, the study design seemed to be a bit flawed. Specifically given how the study groups were designed for each of the studies, there appeared to me a heavy selection bias. For example, if you are designing a study and students are being offered extra credit , the study will have selection bias because among all of the population that could participate, only those motivated by extra credit would participate. While such a bias may not affect conclusions about lower quartiles, it would affect the composition of upper quartiles because the better students would have not opted in. In fact, any study that is opt in suffers from selection bias. I found the study design of "Humor" to be particularly flawed (and to be fair, the authors acknowledge some of the deficiencies in that study).
- I couldn't help but being left with the impression that there was a bit of chicken/egg going on--meaning that people who score poorly most likely have poor metacognition skills; accordingly, you'd still expect them to exhibit poor metacognition skills even after training. And, they did, but the gap between actual v. estimated performance lessened.
- Study three sets out to determine if incompetents would re-rate themselves relative to others (incompetents tend to overestimate their performance). Specifically: "We reasoned that if the incompetent cannot recognize competence in others, then they will be unable to make use of this social comparison opportunity.) (p 1126). Now, I'm not quite sure how you would parse out lack of skill v. metacognition. Specifically, if you have no skill yourself, particularly in logical problems, how in the heck would you even be able to grade another's paper and recognize that s/he performed better than you? Because of this, how can ANY OTHER conclusion other than the one drawn materialize?
- The study and conclusion that training would help the incompetent become more competent (that's fairly obvious). The authors note that it lead to a paradox that by making them more competent they recognized their own incompetence. A logical paradox I think. Now I found some irreconcilable differences. Specifically, the authors conclude that for the bottom quartile, incompetents did better on the tests, but they still lacked the ability to estimate the abilities of others. My quibble is in the study study design to reach this conclusion. The participants only had access to their own tests (before and after). They did not, in so far as I could tell, have the opportunity to look at the tests of others, but rather just made estimates. What I would have preferred to see (hey, and I'm fully incompetent in study design, but I'm just dealing with logic, I think) is that they were both (1) given training and (2) saw his/her test AND the tests of others. It would be interesting to see how replications of this study using a tweaked study design (both in terms of population and test methodology) how differently the ratings would compare.
- The most interesting point of this entire study to me was the idea that top quartile folks suffered from false consensus-- From Wikipedia: "The false consensus effect refers to the tendency for people to overestimate the degree to which others agree with them. People readily guess their own opinions, beliefs and predilections to be more prevalent in the general public than they really are."
Regardless of my quibbles, I was certainly left with the following admonition:
If we lack skill/knowledge in any undertaking, we need to be vigilant to ensure that we obtain the necessary knowledge and experience to increase our competence (so that we are not unconscious incompetents) and (b) assess our abilities as objectively as possible.
I leave this post with this hope:
That any of us read and discuss such matter provides us with some immunity from being relegated to the bottom quartile of any such study of competence relative to investing!
Russell--thank you so much for this contribution.
7 comments:
Re: depressed people's view of themselves:
Dr. Martin Seligman (Unversity of Pennsylvania, I believe) has done a great deal of work on this subject. He notes that pessimism tends to be linked to depression and may, in fact, be a cause of it. He is a pioneer in cognitive therapy, which helps people change their so-called explanatory style from a pessimistic one to a more optimistic one. This can help people get past their depression.
There is a place for pessimism. Senior executives, who remain mindful that things don't always work out, might be viewed as pessimistic by their brasher, more optimistic colleagues, but the pessimists may also help to keep a company from taking risks that could get it into trouble.
In a study of POWs who spent years in Vietnam prison camps, the surprise result was that the pessimists survived better than the optimists. The reason is curious. At first it's counter-intuitive, but when you think bout it, it makes sense.
The optimists would get their hopes cranked high ("We'll be out by Christmas, I just know it!") and when reality smashed into their happy, rosy scenarios, they sometimes couldn't sustain the shock, pain and dismay. By contrast, the more pessimistic POWs, whose views of reality proved to be more accurate, did not suffer the crushing emotional lows. The best remained gritty and determined throughout the months and, eventually years, of their confinement. Their survival rates were higher. In fact, I believe they were considerably higher.
There is a place for optimists and for pessimists in the work place, provided neither tendency is way, way over on the bell-shaped curve.
But overall, in life, a healthy optimism is best and should be cultivated.
The results of the study are so strong, and across multiple areas of skill that selection bias is not likely to have much of an impact on the general thesis as it stands. It might if you tried to argue that people are worse at judging their joke telling from their driving skill, but that is not the point. The use of quartiles rather then absolute measures eliminates the "magnitude" problem: the scoring is always relative.
Most people who I have shown the study to have noted that it goes a long way toward explaining the behavior of certain people they know. "Bosses", because of their authority in areas outside of their of expertise are frequent culprits in the "unskilled and unaware" problem. Any one who has seen Senior Management in an Engineering firm do interior decoration will know what I speak of.
"But overall, in life, a healthy optimism is best and should be cultivated".
I would argue that is because we are blessed to live in a time period of economic expansion. I am not sure that is a universal truism.
Keep in mind there is an optimism that's completely unrealistic and is therefore unhealthy. That's not the kind that anyone should cultivate. But I wonder if a healthy optimism isn't especially useful during periods of economic stagnation and/or actual contraction.
Extreme pessimism tends to result in an unwillingness to consider options, possible new opportunities, etc. It is also linked to depression; depressed people are notoriously unwilling to consider different options. They tend to wallow.
No matter what the environment, pessimists don't really thrive, but I think that they're even more likely to have problems during stagnant or contracting economic periods.
I suspect that optimists do well in any kind of scenario, but do best during an expansive period.
Great Comments:
Re Seligman: I enjoy his work. I have two of his books: Authentic Happiness and What You Can Change and What You Can't.
Re Optimists/Pessimists in the workplace: "There is a place for optimists and for pessimists in the work place". Personally, I would refine this to say that people need to be realists. There are both optimists and pessimists who are incapable of realism, and both are dangerous in an organization. But if I had to choose between the two, I'd opt for the optimist. I'd agree that an optimist can get into less trouble in an expansive economy--but it is realism, not pessimism that grounds the optimist. And even in the most debilitating of organizational business environments, you need to be realistic in assessing your situation and crafting your plan and optimistic about your attaining it.
Russell: "The results of the study are so strong, and across multiple areas of skill that selection bias is not likely to have much of an impact on the general thesis as it stands." I'm not arguing with the unskilled but unaware premise at all--I agree that selection bias does not impair relative comparisons. But when any is looking at studies, you have to evaluate how selection bias might affect observed results and conclusions drawn. I suppose my real quibble is not really a quibble at all but merely a question. I have every confidence that the authors have a better grasp of study design than I!
I suspect that both healthy optimists and healthy pessimists are quite sure that they're realists. In some situations the optimists are the most realistic, in others, it's the pessimists. And, of course, both can be right for different reasons, which is why both temperments are needed, valuable and wonderful.
I think of them as music: optimists use the major keys; pessimists, the minor.
I've heard beautiful music in both keys, but my preference is almost always for music written in the major keys. I bet I'm not alone.
gemmastar: I think that our operating definitions of optimists v. pessimists in organizational life differ. Specifically, I do not equate the ability to see the downside as equated with pessimism, nor do I equate the ability to see the upside as optimism. But I see that it is handled that way and perhaps I need to recalibrate my view of it. While it is attractive and poetic to see it as a yin/yang, it has never been my experience in organizational life to see it operate as such. In my experience, habitual naysayers are very detrimental to organizational success.
However, to the extent that downside=pessimism and upside=optimism, then I would agree that this balance (provided that it is in the healthy optimist/healthy optimist category that you mention) is critical to organizational health.
"However, to the extent that downside=pessimism and upside=optimism, then I would agree that this balance (provided that it is in the healthy optimist/healthy optimist category that you mention) is critical to organizational health."
Different band, different lead singer, same music.
I think we're playing and singing off the same page, but you have a better band and a much better lead singer!
Post a Comment