Published by Guests, on 20/10/2019
Falsehood 1: ‘A wrong theory or wrong measurement isn’t important, as long as it leads to a good conversation and useful insights.’
Alternative: ‘It doesn’ matter, it is just a conversation starter.’
The answer is that this is entirely false. This is the brutal truth: garbage in, garbage out! Some people cling to the often-heard fallacy that ‘all models are good, they are just a discussion/conversation starter.’
This myth is the most commonly heard self-justifying rationalization, and most people don’t even attempt to oppose this idea or continue the discussion. After all, the fallacy doesn’t sound that odd, does it? Before I give my opinion on this, consider these analogies:
Don’t like analogies? Well, try these ‘conversation starters”’:
These discussion starters are not only bad and feeble jokes (sorry about that), but they also demonstrate how flawed the reasoning is. How can you have good discussions when you start off with wrong information? What can you learn from a discussion based on entirely false information or even half-truths? (As is often the case as a result of completely erroneous theories or unreliability of tests or questionnaires.) How can you reach your destination when you put the name ‘Jackson’ into your GPS but select the wrong state in the U.S.?
We can draw a parallel with the often-heard phrase in computer science or information technology: garbage in, garbage out! Computers can process large amounts of erroneous information, but it is impossible to put in wrong figures or data and get the right answers as the output. Yes, the ‘garbage’ metaphor is the one I want, and I wish to address a possible defense metaphor by stating that garbage can be recycled. The recycling metaphor in this case doesn’t hold up as you cannot ‘recycle’ this garbage and get some renewable raw materials out. Take the MBTI: even the good ‘raw material,’ such as the concept of extraversion-introversion, is measured as a dichotomy (although the latest versions —called ‘steps’—try to deal with this criticism), or is explained in a highly bizarre, even metaphysical manner (‘introverts get their energy from within, extroverts get their energy from others’ —really? And how does this process work? Put a tap on the other person? Or is energy transferred through quantum mechanics, as quacks often like to reference, hoping no one understands a damn thing about quantum mechanics, but knowing full well that it sounds very intellectual…).
One problem with this myth is the mere exposure effect. Just hearing a word (like the incredibly persistent meme 70:20:10), or seeing a logo (like Coca-Cola) very often makes it more familiar to us, thereby making it seem… more trustworthy or credible. That is why this is called a mere exposure effect. Advertisers know this effect all too well and use it in product branding. We have also heard this fallacy so often from HR people who try to avoid strenuous thinking and intellectual discussions, to the point that we may assume it is correct. Hearing is often believing, but that doesn’t make it correct. I think most people are convinced that Napoleon was rather short (and that his ‘short man syndrome’ or ‘Napoleon Complex’ could explain his rather dictatorial nature), but this is a myth that has been repeated so many times, that almost everyone accepts it as a fact. In fact, Napoleon Bonaparte was 1m68 tall, slightly above the average height for a Frenchman at that time! A misunderstanding could have contributed to this myth, as the ‘French inch’ at that time (2.707 cm) is considerably longer than the Canadian (later international) inch (2.540 cm), thus underestimating Napoleon’s height by a full four British inches.
Technically speaking, this weird reasoning seems to be a special kind of the ad populum fallacy, namely ‘playing to the gallery:’ people who use this fallacy want to arouse prejudices or emotions in their audience instead of presenting good evidence. So indeed, where is your evidence that false statements are good discussion starters? Some consider this a kind of bait-and-switch defense mechanism: person A puts forward a claim, but when driven into a corner (e.g. by someone pointing out the test is flawed), person A backtracks and retreats to a much weaker statement. By saying, ‘it’s only a conversation starter,’ person A is trying to make the real problem seem trivial or uninteresting. It can also be regarded as a ‘reverse switcheroo’ (Kukla, 2000): Person A makes a strong statement (e.g. about the validity of the MBTI), but when they get into trouble and are confronted with the problematic theory and psychometric flaws, person A retreats to a weaker position: the discussion or conversation starter. Person A tries to pretend that this was the only intention. You see, from whatever angle you look at it, it remains a fallacy. Nice try, but no cigar. If you feel happier clinging to this wreck, go ahead! If you really prefer to look right, rather than to be right, that’s your choice.
Oh, and yes, if it really isn’t important which tool you use, why do fanatics of tools like the MBTI, Insights Discovery, the enneagram or DiSC charge a lot of money? Why don’t they use free tools or a free horoscope? Could it be because they simply want to earn money and don’t care about truth and integrity?
Falsehood 2: ‘There are no bad theories, models, or wrong tests and outlandish therapies—only wrong applications.’
Some people actually believe there is no such thing as a bad theory. According to their erroneous reasoning, you can only apply them in the wrong manner. I will argue why bad theories, models, and tests will lead to wrong outcomes.
I was once baffled by a false analogy that someone brought up when defending the enneagram. She said: ‘it is not because you can kill someone with a small kitchen knife, that you should throw away the knife.’ This analogy was wrong, but it took me quite a while to find out where the metaphor went wrong, as I was not yet that well-trained in debunking thinking errors and fallacies. By the time I figured out what was wrong, I had already kind of lost the debate. The kitchen knife of course was not designed for killing, but for doing kitchen work, whereas the enneagram was designed for measuring personality. Normally, a kitchen knife can be used for peeling potatoes. If it no longer does that because it is broken, you should sharpen it. There is no need to throw it away if someone used it to kill an animal or fellow human (at least theoretically, I guess most people would want to throw it out, if the justice department doesn’t keep it that is…). The correct analogy then is: if, for example, the MBTI or the enneagram do not measure personality correctly, we should throw them away without any hesitation.
It is completely false to state that one cannot do something wrong with a bad theory. One of the most outrageous examples are studies into the use of the Rorschach inkblot test. There are at least six schools of interpretation in Europe alone. One of the most well-known interpretation methods is called ‘The comprehensive system,’ by John Ernest Exner (1974, 1978). Philip Erdberg—a former instructor of Exner’s Rorschach workshops—and his colleagues published research in 1999 comparing the Rorschach scores of perfectly normal people with the norms published in Exner’s books. Based on these norms, every single person should have been considered seriously mentally ill. On another occasion, PhD-candidate Beatrice Mittman asked 90 alumni of Rorschach trainings to apply their protocols to both psychiatric patients and perfectly healthy adults: again, 75% of the healthy adults were labeled as mentally ill. In 2000, Mel Hamel reported a study of 100 children in California, again all very healthy (including ratings by parents), but they appeared deeply disturbed according to the Rorschach test.
In my capacity as the president of a not-for-profit organization, visitors often drew my attention to questionnaires that have caused harm, for example people being fired or demoted from positions based on a wrong theory or test. I cannot see how you can rationalize such obvious cases of harm and unfairness to those victims. I think the answer is quite obvious: bad theories can lead to extremely bad decisions, thereby potentially causing harm.
| In sum |
People are very good at rationalizing their irrational beliefs, but: you cannot learn ‘good’ things from ‘bad’ information; the mind is not to be entirely trusted—especially when it comes to ‘hard’ data (such as predicting stock markets, assessing personality, etc.), we need slow and effortful thinking. In the case of our social life as well, our thinking is rife with bias, such as the tendency to follow the majority (social proof or bandwagon); we have freedom of speech, but no freedom of action if this causes harm to other people; scientific research is the best option we have to acquire reliable knowledge. It has built-in self-correction mechanisms and, most of the time, is incremental (progressive and gradual) and not characterized by ‘sudden’ paradigm shifts.
For more info about and ordering of the book ‘A Skeptic’s HR dictionary’: www.askepticshrdictionary.com