Among the many, many things we ask people every day are a subset of personality test-type questions such as:
- When making major decisions, do you like to have things settled or keep your options open?
- In social situations are you more outgoing and enthusiastic, or more self-contained and reserved?
- When tackling a new project, do you like to use established skills or prefer to learn new skills?
These questions can provide value to our clients by giving them a window into how their customers make decisions, but it’s also fun to see how they correlate unexpectedly with some of the more tangible attributes we track.
A few years ago we started asking a set of Philip Tetlock-inspired questions with the goal of seeing whether they correlate to people’s ability to predict all sorts of things — election results, pro football games, etc.
Within the span of a week or two, one of these questions caught the eye of two C-level big shots within the company: “Do you typically make decisions quickly and confidently?” Both independently asked me whether the people who say “No” literally take longer to answer this question than those who answer “Yes.”
I’ll save you some suspense: They do!
Since the beginning of 2015 around 5,000 people have answered this question every month. For each of the 77,383 respondents I looked at, I calculated the difference between when they saw the question (we call this an “impression”) and when they answered it. Below are charts that show the mean and median difference for those who answered “Yes” and “No” over time.
As you can see, people who answer “No” take close to half a second longer to respond on average than those who answer “Yes”. That obviously isn’t a very large amount of time, but it’s amazingly persistent. (On a weekly grain, “No” respondents took longer on average to answer the question than the “Yes” group for all weeks but one. The median value for the “No” group was higher than that of the “Yes” group for all but four weeks.)
So what’s driving this? Two subsets of people, primarily — the people who really take their time when answering the question, and older respondents.
For both groups, I divided the respondents up into quartiles based on how long it took them to answer. The chart below shows that, as people take longer to answer the question, the gap between “No” and “Yes” increases, both in absolute and relative terms.
Among people who take longest to answer (delay quartile 4), those in the “Yes” group took 10.3 seconds to answer on average. For the “No” group it took 11.2 seconds on average. Compare this to people in quartile 1, where both groups take a little more than three seconds to answer.
When looking at results by age, it should come as no surprise that the older a person is, the longer they take to answer the question. That could be driven by the fact that reflexes slow with age. But as the chart below shows, the gap between “No” and “Yes” clearly widens with age.
First, respondents aged 13-17 actually conflict with the overall trend; the “Yes” group takes longer to answer the question by a negligible margin. (I suppose self-aware is a word that’s seldom used when describing teenagers.) But once we get to people aged 35 or older, those in the “No” group take about ten percent longer to answer than those in the “Yes” group.
To add one bizarre wrinkle to the age breakdown, there’s a clear relationship between a person’s age and whether they say they make decisions quickly and confidently — and it’s in the opposite direction of how quickly they actually answer.
So to get back to the question that serves as the title of this post — “How self-aware are people?” — the answer is: Kind of self-aware?
