Saturday, May 31, 2014

"The Dunning-Kruger Effect"

I've met several people who suffered from the Dunning-Kruger effect. When that person has some authority over others, the problems can be catastrophic. I knew one guy who had it, as did his boss. So the boss could not see how awful this guy was. It took seven years to get rid of him. In the meantime people were suing the place, sabotaging things and walking off, and quitting and taking business with them.

This is from the site The Rawness. (If you want to watch the videos, click on the link.)

"There is a human nature phenomenon called the Dunning-Kruger effect that is discussed and written about quite often these days. To summarize it crudely, it says that incompetent and unqualified people are too incompetent and unqualified to realize just how incompetent and unqualified they are.

"The cognitive defects that keep the incompetent from being competent and skilled also work to keep them from recognizing their incompetence and lack of skill. Additionally, it also keeps them from properly assessing the competence and skills of their parties.

"The blog 'You Are Not So Smart' gave a pretty good summary of the Dunning-Kruger effect. Some highlights:

“‘In the modern world the stupid are cocksure while the intelligent are full of doubt.' - Bertrand Russell

"…The less you know about a subject, the less you believe there is to know in total. Only once you have some experience do you start to recognize the breadth and depth you have yet to plunder…

"Justin Kruger and David Dunning pinpointed the effect in experiments at Cornell around 1999. They had students take grammar and logic tests and then report how well they thought they had scored.

"It’s important to note here, not all people couldn’t predict their skill level. Some people knew they sucked at humor, and they were right.

"The results of the study show all sorts of interesting things. Sometimes people who are really good at something are well aware and can accurately predict their scores, but not always.

"Overall, the study showed people are not very good at estimating their own competence…

"It breaks down like this:

"The more skilled you are, the more practice you’ve put in, the more experience you have, the better you can compare yourself to others. As you strive to improve, you begin to better understand where you need work. You start to see the complexity and nuance; you discover masters of your craft and compare yourself to them and see where you are lacking.

"On the other hand, the less skilled you are, the less practice you’ve put in and the fewer experiences you have, the worse you are at comparing yourself to others on certain tasks. Your peers don’t call you out because they know as much as you do, or they don’t want to hurt your feelings. Your narrow advantage over novices leads you to think you are the shit.

“'Ignorance more frequently begets confidence than does knowledge.' - Charles Darwin

"Whether it’s playing guitar or writing short stories or telling jokes or taking photos (or writing blog posts) – whatever – amateurs are far more likely to think they are experts than actual experts are.

"Education is as much about learning what you don’t know as it is about adding to what you do.

"The blog elaborates further:

"As someone moves from novice to amateur to expert to master, the lines between each stage are difficult to recognize. The farther ahead you get, the longer it takes to progress.

"Yet, the time it takes to go from novice to amateur feels rapid, and that’s where the Dunning-Kruger Effect strikes. You think the same amount of practice will move you from amateur to expert, but it won’t.

"Anyone who has played a role-playing game is familiar with this sort of progression. If there are 100 possible levels of advancement, the first 20 will fly by – but the time to go from levels 50 and 51 may be longer than all the time spent so far.

"The Wikipedia entry on Dunning-Kruger Effect says:

"Kruger and Dunning proposed that, for a given skill, incompetent people will:

"tend to overestimate their own level of skill;
"fail to recognize genuine skill in others;
"fail to recognize the extremity of their inadequacy;
"recognize and acknowledge their own previous lack of skill, if they can be trained to substantially improve.

"This blog also discusses D-K Effect:

"Effectively, this is a strange behaviour where the incompetent do not realize their own incompetence, due to their incompetence. They never recognize they’re doing anything wrong, indeed they outright lack the ability to even reach such a conclusion. Thus, they think they’re doing fine, nay, fantastic, and as such develop a hugely inflated opinion of themselves.

"To put it more crudely, people who are dumb are often too dumb to realize they’re dumb.

"But also important is what the D-K Effect says about competent people:

"The inverse of this is also true.

"The competent do not realize their own competence, because they are competent. They see greater skill in others, recognize their own failures and mistakes, and thus try to improve. Inevitably they do improve, becoming much, much better than where they used to be.

"But the competent (usually) never realize it. They get better, yes, but they still think themselves the abject incompetent failure they used to be. They still make mistakes, they still see others better than they.

"The incompetent unskilled develop massively inflated opinions of their own ability, whereas the competent skilled vastly and consistently underestimate themselves.

"The 'seeing great skill in others' part is an important factor here. As the original study by Dunning and Kruger states:

"[Competent] participants appear to have fallen prey to a false-consensus effect…Simply put, these participants assumed that because they performed so well, their peers must have performed well likewise…

"It thus appears that extremely competent individuals suffer a burden as well. Although they perform competently, they fail to realize their proficiency is not necessarily shared by their peers.

"So while the smart people were guilty to some degree of underestimating their own competency, they were significantly more guilty of overestimating their competition. Because they found the challenges to be easy, they mistakenly assumed everyone else did to a degree as well. They projected their competency levels onto others.

"Incompetent people seem more guilty of overestimating their skill level, while competent people seem more guilty of overestimating the skill level of others.

"There was a so-called 'refutation' of Dunning-Kruger by some scientists named Burson, Larrick and Klayman. On the blog 'Overcoming Bias', Robin Hanson made a post dedicated to the refutation and quoted a passage from the Burson paper as evidence that the Dunning-Kruger Effect was in fact invalid.

"The Burson study that challenged Dunning-Kruger found that on easy tasks, the most competent performers were also the most accurate at assessing their own performance. However, with difficult tasks, the worst, least competent performers were the most accurate as assessing their own performance.

"After Hanson made his post at 'Overcoming Bias", whenever a blog or online article discussed Dunning-Kruger, someone would pop up in the comments saying 'Nuh UHHHHH!! Dunning-Kruger is WRONG WRONG WRONG!' And link to Hanson’s post. If you read the comments to the You Are Not So Smart link above for example, you’ll see the same thing happen there.

"However I don’t think this is necessarily a refutation of Dunning-Kruger if you stop and think about it and try to reconcile both findings instead of just trying to side with a winner. Rather, it’s more of a clarification or fine-tuning than an outright refutation. If you think about it, both studies can very well be right, which I believe is the case.

"When a task is very, very easy, smart people are less likely to have self-doubt about their performance. For example, if you were given a test with simple arithmetic problems like 10+10, even the most humble smart person is going to assume he rocked the test. Because the test is so easy, everyone taking it will rate themselves as performing positively, and the smart people are more likely to be the ones who turn out to be correct about their own performance.

"On the other hand, if you give a test in advanced nuclear physics, even the most delusional dumb person is going to have to admit to himself that he knew absolutely nothing. Because this test is so difficult, everyone will rate themselves as performing negatively, and the incompetent people are more likely to be correct about their own performance.

"All the Burson study shows is that the area of expertise has to be one that is easy enough to grasp that the dumb person thinks he has a chance of excelling at it, but complex enough that a smart person believes he may still have more to learn about it.

"The Burson paper really just shows that when extremes in difficulty are involved, like extremely easy challenges and extremely difficult challenges, the Dunning-Kruger effect doesn’t hold up so well. It has to be an area of expertise easy enough for the simpleton to think he has a shot but complex enough for the smart person to still have room for self-doubt. We’ll call these areas of expertise 'grey areas.' Using this criteria, I think the grey areas where the D-K effect still apply make up 80-90% of challenges we come across in our daily lives, including social dynamics, emotional maturity, psychology and human nature, things this blog specializes in. So expect to see a lot of D-K effect being discussed here soon.

"More observations about Dunning-Kruger: even in the aforementioned 'grey areas,' there are exceptions. Although the incompetent tend to overrate how good they are, if they are unskilled to a shockingly extreme degree, they may end up reaching a level of incompetence where even they can’t deny how unskilled they are. Like the person who goes out multiple times a week to bars and clubs for decades but has never gotten a phone number or a makeout and is still a 40 year old virgin. With such an objectively rockbottom track record in the dating field, even he can’t fall victim to delusions caused by the D-K effect, even though dating is one of those 'grey areas' where reasonable people can differ about how good they are. Take into account also the guy who for decades gets a one night stand almost every single time he goes out, by 9s and 10s to boot. No matter how humble or open-minded to improvement he is, it’s impossible for him to not realize that he is among the best of the best.

"Dunning, Kruger, and others responded to their critics in a follow-up study, which shed even more light on the phenomenon. There were some interesting findings in the follow up study.

"First, the follow-up study suggests an additional reason for the D-K effect in incompetent people:

"One could argue that a goal to preserve a positive, if not accurate, view of the self may be particularly strong among those who have performed poorly precisely because these are the individuals who might suffer the most from admitting the reality of their poor performance. Those who score very well, in contrast, would have considerably less motivation to glorify the quality of their performance. Indeed, they may be motivated instead to be overly modest about their achievement.

"If this is the case, what appears to be an inability to assess the quality of one’s performance on the part of the unskilled might actually be an unwillingness to do so accurately, in that the unskilled prefer to report a rosy view of their performance.

"This touches on something we’ve discussed on the blog previously. Because the incompetent may have deep, underlying fears of being inferior, this need to desperately believe they are actually far better than average is created and manifests itself in what Alfred Adler called the superiority complex, which I discuss in the post 'Theaters of Operation.' So the obnoxious overcompensation arising from superiority complexes may be partially responsible for the D-K effect.

"The follow-up paper also notes that while the incompetent and competent both are guilty of initially misjudging their own competence in comparison to others, there’s a big difference: when smart people get feedback about how they actually did in comparison to others, they become more accurate in future self-assessments. Incompetent people on the other hand, even when getting consistent, repeated feedback about how badly they’re doing, still fail to recognize their incompetence and remain overly confident in predicting future performance:

"Part of why the dramatic overestimation demonstrated by poor performers is so fascinating is precisely because they show dramatic overconfidence on tasks about which they have likely received substantial feedback in the past…It seems that poor performers do not learn from feedback suggesting a need to improve. Hacker, Bol, Horgan, and Rakow (2000) provided direct evidence for this failure to learn from feedback when they tracked students during a semester-long class. As time went on, good students became more accurate in predicting how they would do on future exams. The poorest performers did not—showing no recognition, despite clear and repeated feedback, that they were doing badly. As a consequence, they continued to provide overly optimistic predictions about how well they would do in future tests.

"I feel this is a variation of the Sunk Cost Trap. The more heavily invested you are in something, even if it’s something intangible like a belief in one’s own superiority, the harder it is to cut your losses and let it go, even if its doing you more harm than good. Since these people have invested so much of their self-image into this delusional superiority, they will stick to their guns no matter how much contradicting feedback they get. They’ve reached the point where relinquishing those longtime delusions and facing the truth about themselves would be too crushing to their ego to bear.

"I think this is why so many people have to hit rockbottom before they can come to terms with their shortcomings and admit to themselves that the way they’ve done things their whole lives isn’t actually working. Only such extreme, undeniable feedback can shock them to their senses."


Carnivore said...

“He who knows not and knows not that he knows not is a fool—shun him. He who knows not and knows that he knows not is a child—teach him. He who knows and knows not that he knows is asleep—wake him. He who knows and knows that he knows is wise—follow him.”

outsider said...

My unpopular belief is that most people already have advanced to their maximum possible level of competence, given current social and educational realities.