Chapter of ‘A Skeptic’s HR dictionary’

Published by Guests, on 20/10/2019

By Patrick Vermeren (co-managing director & consultant PerCo)

Falsehood 1: ‘A wrong theory or wrong measurement isn’t important, as long as it leads to a good conversation and useful insights.’

Alternative: ‘It doesn’ matter, it is just a conversation starter.’

The answer is that this is entirely false. This is the brutal truth: garbage in, garbage out! Some people cling to the often-heard fallacy that ‘all models are good, they are just a discussion/conversation starter.’

This myth is the most commonly heard self-justifying rationalization, and most people don’t even attempt to oppose this idea or continue the discussion. After all, the fallacy doesn’t sound that odd, does it? Before I give my opinion on this, consider these analogies:

• You are lost in New York City and you don’t have a city map. You turn to someone for advice and this person replies: ‘I have a map and you can have it for free.’ You are pleasantly surprised by this generous act and you accept the map. Only, it turns out it is a map of… Paris. When the person notices your surprise, they say: ‘Oh well, it’s just a navigation starter.’
• You applied to an online dating site, filled out some quirky personality test, and get back some advice that that doesn’t fit about half the description about you (you recognized about 50% of the description to be fairly true, but you are a person skilled at critical thinking and—as with horoscopes—you realize that confirmation bias might kick in, so you studied the other 50% more carefully). You decide to confront the dating site, and this is their response: ‘It doesn’t matter at all, after all, it’s just a relationship starter.’
• You are building a house and your architect calculated how much concrete should be put into the foundation. You check on the progress of the work and notice your building company has not put in concrete, but Styrofoam balls. The architect notices your bewilderment and shrugs his shoulders saying: ‘It’s only a construction starter.’
• Try this one at home: Tell your partner/spouse you cheated on her/him with a younger, attractive co-worker. Pause for a full two minutes, look guilty, and then say: ‘Just kidding, I only said it as a romantic evening starter…’

Don’t like analogies? Well, try these ‘conversation starters”’:

• ‘Hi, I’ve noticed you around. I’ve never seen a person as ugly as you are. Anyway, it is more important to be beautiful on the inside.’
• ‘It has now been found that the earth is flat, Copernicus, Kepler and his friends were flat-out wrong. Could we brainstorm the consequences of this latest finding for our next aircraft prototype?’
• ‘Did you hear the news today, former U.S. president Bush admitted he ordered the controlled demolition of the Twin Towers (WTC). They laughed at us before and ridiculed us for being conspiracy theorists, but you see now, we weren’t that wrong.’ (The statement that Bush admitted this is a flat out lie.)
• ‘This inkblot test shows that you have committed sexual abuse against your children, therefore we should put you in prison.’ (There are many schools of thought that all provide different interpretations of the Rorschach inkblots.)
• ‘Thanks for participating in this selection test. I am afraid we cannot offer you employment as your GMA (General Mental Ability, or IQ score) is only 80. Do you understand what 80 means?’ (Your actual GMA score is 110 but the test is totally flawed. And a score of 80 is considered intellectually disabled…) 

These discussion starters are not only bad and feeble jokes (sorry about that), but they also demonstrate how flawed the reasoning is. How can you have good discussions when you start off with wrong information? What can you learn from a discussion based on entirely false information or even half-truths? (As is often the case as a result of completely erroneous theories or unreliability of tests or questionnaires.) How can you reach your destination when you put the name ‘Jackson’ into your GPS but select the wrong state in the U.S.? 

We can draw a parallel with the often-heard phrase in computer science or information technology: garbage in, garbage out! Computers can process large amounts of erroneous information, but it is impossible to put in wrong figures or data and get the right answers as the output. Yes, the ‘garbage’ metaphor is the one I want, and I wish to address a possible defense metaphor by stating that garbage can be recycled. The recycling metaphor in this case doesn’t hold up as you cannot ‘recycle’ this garbage and get some renewable raw materials out. Take the MBTI: even the good ‘raw material,’ such as the concept of extraversion-introversion, is measured as a dichotomy (although the latest versions —called ‘steps’—try to deal with this criticism), or is explained in a highly bizarre, even metaphysical manner (‘introverts get their energy from within, extroverts get their energy from others’ —really? And how does this process work? Put a tap on the other person? Or is energy transferred through quantum mechanics, as quacks often like to reference, hoping no one understands a damn thing about quantum mechanics, but knowing full well that it sounds very intellectual…).

One problem with this myth is the mere exposure effect. Just hearing a word (like the incredibly persistent meme 70:20:10), or seeing a logo (like Coca-Cola) very often makes it more familiar to us, thereby making it seem… more trustworthy or credible. That is why this is called a mere exposure effect. Advertisers know this effect all too well and use it in product branding. We have also heard this fallacy so often from HR people who try to avoid strenuous thinking and intellectual discussions, to the point that we may assume it is correct. Hearing is often believing, but that doesn’t make it correct. I think most people are convinced that Napoleon was rather short (and that his ‘short man syndrome’ or ‘Napoleon Complex’ could explain his rather dictatorial nature), but this is a myth that has been repeated so many times, that almost everyone accepts it as a fact. In fact, Napoleon Bonaparte was 1m68 tall, slightly above the average height for a Frenchman at that time! A misunderstanding could have contributed to this myth, as the ‘French inch’ at that time (2.707 cm) is considerably longer than the Canadian (later international) inch (2.540 cm), thus underestimating Napoleon’s height by a full four British inches. 

Technically speaking, this weird reasoning seems to be a special kind of the ad populum fallacy, namely ‘playing to the gallery:’ people who use this fallacy want to arouse prejudices or emotions in their audience instead of presenting good evidence. So indeed, where is your evidence that false statements are good discussion starters? Some consider this a kind of bait-and-switch defense mechanism: person A puts forward a claim, but when driven into a corner (e.g. by someone pointing out the test is flawed), person A backtracks and retreats to a much weaker statement. By saying, ‘it’s only a conversation starter,’ person A is trying to make the real problem seem trivial or uninteresting. It can also be regarded as a ‘reverse switcheroo’ (Kukla, 2000): Person A makes a strong statement (e.g. about the validity of the MBTI), but when they get into trouble and are confronted with the problematic theory and psychometric flaws, person A retreats to a weaker position: the discussion or conversation starter. Person A tries to pretend that this was the only intention. You see, from whatever angle you look at it, it remains a fallacy. Nice try, but no cigar. If you feel happier clinging to this wreck, go ahead! If you really prefer to look right, rather than to be right, that’s your choice.

Oh, and yes, if it really isn’t important which tool you use, why do fanatics of tools like the MBTI, Insights Discovery, the enneagram or DiSC charge a lot of money? Why don’t they use free tools or a free horoscope? Could it be because they simply want to earn money and don’t care about truth and integrity?

Falsehood 2: ‘There are no bad theories, models, or wrong tests and outlandish therapies—only wrong applications.’ 

Some people actually believe there is no such thing as a bad theory. According to their erroneous reasoning, you can only apply them in the wrong manner. I will argue why bad theories, models, and tests will lead to wrong outcomes.

I was once baffled by a false analogy that someone brought up when defending the enneagram. She said: ‘it is not because you can kill someone with a small kitchen knife, that you should throw away the knife.’ This analogy was wrong, but it took me quite a while to find out where the metaphor went wrong, as I was not yet that well-trained in debunking thinking errors and fallacies. By the time I figured out what was wrong, I had already kind of lost the debate. The kitchen knife of course was not designed for killing, but for doing kitchen work, whereas the enneagram was designed for measuring personality. Normally, a kitchen knife can be used for peeling potatoes. If it no longer does that because it is broken, you should sharpen it. There is no need to throw it away if someone used it to kill an animal or fellow human (at least theoretically, I guess most people would want to throw it out, if the justice department doesn’t keep it that is…). The correct analogy then is: if, for example, the MBTI or the enneagram do not measure personality correctly, we should throw them away without any hesitation.

It is completely false to state that one cannot do something wrong with a bad theory. One of the most outrageous examples are studies into the use of the Rorschach inkblot test. There are at least six schools of interpretation in Europe alone. One of the most well-known interpretation methods is called ‘The comprehensive system,’ by John Ernest Exner (1974, 1978). Philip Erdberg—a former instructor of Exner’s Rorschach workshops—and his colleagues published research in 1999 comparing the Rorschach scores of perfectly normal people with the norms published in Exner’s books. Based on these norms, every single person should have been considered seriously mentally ill. On another occasion, PhD-candidate Beatrice Mittman asked 90 alumni of Rorschach trainings to apply their protocols to both psychiatric patients and perfectly healthy adults: again, 75% of the healthy adults were labeled as mentally ill. In 2000, Mel Hamel reported a study of 100 children in California, again all very healthy (including ratings by parents), but they appeared deeply disturbed according to the Rorschach test. 

In my capacity as the president of a not-for-profit organization, visitors often drew my attention to questionnaires that have caused harm, for example people being fired or demoted from positions based on a wrong theory or test. I cannot see how you can rationalize such obvious cases of harm and unfairness to those victims. I think the answer is quite obvious: bad theories can lead to extremely bad decisions, thereby potentially causing harm. 

For more info about and ordering of the book ‘A Skeptic’s HR dictionary’: www.askepticshrdictionary.com