Finding a way to measure — and perhaps develop — “good thinking.”
There are many ways that a person — even a very intelligent person — can be irrational. We can be bad at assessing probability, overconfident, or uncurious; we can evaluate evidence in biased ways and come up with detailed explanations for random events. Our reasoning can be poor when we don’t consider alternative outcomes. Our preferences can waver because of framing effects. We put too much value short-term rewards, and are distracted by vivid stimuli. Sometimes the very structure of what we do know makes it impossible to think clearly.
For decades, psychologists Keith Stanovich and Richard West have been studying and mapping the various ways that human rationality can go off the rails (Stanovich even coined a term for it, “dysrationalia,” in a 1993 paper). They have argued that current intelligence tests are not good at conveying meaningful information about how rational (or not) a person is.
“Most laypeople are prone to think that IQ tests are tests of, to put it colloquially, ‘good thinking,’” Stanovich says. “Scientists and laypeople alike would tend to agree that ‘good thinking’ encompasses good judgment and decision making — the type of thinking that helps us achieve our goals. Yet assessments of such good thinking are nowhere to be found on IQ tests.”
Rather than viewing rationality as an area of intelligence that tests should be expanded to include, Stanovich and West argue that rationality is best seen as distinct from intelligence, and that what is needed to measure it is not an improved IQ test but a different kind of tool, tailored to the unique features of rationality.
IN SEARCH OF THE RQ
With funding from the John Templeton Foundation and editorial support from MIT Press, Stanovich and West have recently published The Rationality Quotient (RQ): Toward a Test of Rational Thinking, a 480-page exploration of what a Comprehensive Assessment of Rational Thinking (CART) might entail. The book begins with an examination of the theoretical underpinnings for measurements of rationality before going through an outline of the 20 subtests that the authors propose could make up a reliable measure for a rationality quotient, covering things like scientific reasoning, understanding of probability, financial literacy, and “contaminated mindware” (knowledge and beliefs that impair critical thinking). Their goal is to explore both the potential foibles of instrumental or process rationality (whether you use your resources in alignment with your goals) and epistemic or knowledge rationality (how well your beliefs map onto the actual structure of the world).
“We spend a good part of the book talking about how rational thinking involves both cognitive processing issues and knowledge biases,” Stanovich says. Because many tasks of rationality tap into both processing and knowledge, it can be difficult to tease apart their exact relationship. Because of that, Stanovich says he and West were careful to choose tasks to be measured that a rely on a range of different combinations of process and knowledge.
Stanovich suspects that a well-formed measure of rationality will gauge a combination of baseline rational capacity as well of aspects of rationality that can be learned and improved. “There is certainly every a priori reason to think that certain components of rationality might be trainable,” he says. “How much we could possibly train a more generic open-minded thinking attitude is unknown and uninvestigated. Some rational thinking dimensions such as actively open-minded thinking might be more like heritable personality traits” — though he points out that heritability does not mean that a trait can’t also be affected by training.
WHAT MIGHT COME NEXT
Stanovich and West’s work on the rationality test is purposely more of a proof of concept than a finished product ready to be administered widely. In addition to further development (including gathering a more robust understanding of what might constitute a normal or abnormal result on such a test), there is also the diplomatic task of promoting a finished test to psychologists and educators. For example, Stanovich believes that in the future a version of the CART will be useful as a means of testing the effects of coursework not just in students’ knowledge but in their rationality.
“In our society, what gets measured gets valued,” Stanovich notes. By making tools to assess multiple aspects of rationality both more robust and more widely known, it’s possible that a popularized rationality quotient might not just help quantify the problem of irrationality, but help nudge us towards a (slightly) more rational future.
