Rishon Rishon

Myers Briggs meets the Skeptic

Amritas is skeptical about Myers Briggs:

I have long been suspicious of its four-scale system based on this Skeptic's Dictionary entry:

Furthermore, no matter what your preferences, your behavior will still sometimes indicate contrasting behavior. Thus, no behavior can ever be used to falsify the [MBTI®] type, and any behavior can be used to verify it.

That is reminiscent of Chomskyanism: in theory, no 'surface' structure can be used to falsify a proposed (invisible, inaudible) 'underlying' structure since 'transformations' can account for anything. (In practice, there are constraints on transformations, but even so, there is no hard evidence for them or for the 'underlying' structures that they allegedly affect. Saying that magic spells are not omnipotent still does not address the issue of whether such spells exist at all.) Something that is not falsifiable is not scientific.

I have never looked at the Skeptic's Dictionary before. My instincts are to be partial to it based on its name - I am skeptical by nature, myself. But I am very disappointed by this particular entry. Most of it amounts to little more than an ad hominem attack, i.e. attacking Myers and Briggs (the originators of the test) as individuals because they (may have) made some mistakes, and Jung, who came up with most of the original concepts, but likely wouldn't have supported the way Myers and Briggs developed them. The only comment about the test itself is the one Amritas quoted, and that is demonstrably false. Myers Briggs does make verifiable predictions, the fact that the predictions aren't 100% accurate in no way invalidates them - the question is only whether they are statistically significant, and anyone who has worked with large samples knows that they are.

A coworker of mine once came back from a business seminar and told me about the most amazing experience. The participants of the seminar (about 60 people) were given a Myers Briggs test and divided into groups of 5-6 that were as homogeneous as possible. Each group was then given a task of building something out its component parts - evidently something quite difficult, though I don't know what it was. What made it amazing was not just the variance in how well the different groups performed, but how different their approach to the problem was. "Some of the groups just gave up and chatted at the back of the room", my coworker told me with amazement (my coworker was an SJ, so her reaction is not surprising), "Your group did the best" she added, by which she probably meant NT - I doubt that there were 5-6 INTPs in a group of 60 business seminar attendees. She reported different styles of problem solving and cooperating, e.g. the SJs divided up the task, etc.

There's a certain pseudoscientific idea that if something is hard to measure, it doesn't exist. People who have this notion will often discount soft and fluffy ideas like happiness. I, for one, think happiness is an exceedingly important concept, whether or not it is hard to measure. And though I know of know way to objectively compare the happiness of one person to another, I have no doubt that some people are happier than others, and I trust my own subjective evaluations of the matter to be significantly correlated with the truth. Myers Briggs tests have this problem - they are dependent on peoples' self evaluations regarding a lot of soft-and-fluffy questions like, "Direct-contact group discussions stimulate you and give you energy - Y/N". Nevertheless, according to this paper:

Several researchers have studied the construct validity of the MBTI scores. Carlyn (1977) found evidence indicating that "... a wealth of circumstantial evidence has been gathered, and results appear to be quite consistent with Jungian Theory" (p. 469). Validity of MBTI scores is typically established by correlating the scores with findings from various personality instruments and inventories of interest. Statistically significant correlations have been found between MBTI scores, behaviors reflective of MBTI constructs, and persons' self-assessment of their own MBTI type (DeVito, 1985; Myers & McCaully, 1989). Using factor analysis, Thompson and Borrello (1986) reported that the factors were largely discrete in their sample, and all items had factor pattern coefficients higher than .30. These results supported the structure of the MBTI. More recently, Tischler (1994) noted that "... factor analysis provided unusually strong evidence that the MBTI items are correlated with their intended scales: the scales are almost factorially pure" (Tischler, 1994, p. 30).

Furthermore, if you measure the test-retest accuracy not as a binary outcome (e.g. S or N) but as a quantitative score, you find very high correlations, i.e. people near the middle will often flip, but their numerical scores won't change much. If you still think, like the Skeptic that:

The profiles read like something from Omar the astrologer and seem to exemplify the Forer effect.

Then take my advice in the previous post, and look at the description of your opposite. Ask yourself: which is more like me?

UPDATE: The Skeptic also warns, "There is also a pernicious side to these profiles: they can lead to discrimination and poor career counseling." Of course, if you are talking about individual cases you have to take into account that Myers Briggs tests are not 100% accurate, and assuming that they are can lead to unfortunate results. This in no way falsifies the theory, though it may limit its usefulness. My personal experience is that it's very useful even when I get it wrong (i.e. my subjective impression doesn't agree with the "objective" test). Why? Because it gives me a powerful way to think about the subject: The beginning of wisdom is the definition of terms. A well-chosen terminology helps you to think, and understand. Whether or not I'm right about Razib's personality in the previous post, I'm confident that I'm right about the characteristics I examined, and that I understand something about his personality as a result.

UPDATE: I think that the main reason it's not taken more seriously in academia is that it was developed by non-academics. Who cares that it works in practice, and that thousands of profit-making enterprises that have to explain themselves to their shareholders spend money using it to help them do business. It doesn't have the right credentials!