Bad at guessing

Hello there 🙂

Welcome to issue thirty seven of Manufacturing Serendipity, a loosely connected, somewhat rambling collection of the unexpected things I’ve recently encountered.

This newsletter is free to receive, but if you’d like to support me in this odd little endeavour you can buy me a coffee 🙂

Speaking of coffee, grab yourself a suitable beverage my loves, let’s do this thing…

Part I: Things I’ve Encountered Online…

Last week I came across this study from YouGov America. It interested me because I think it’s a great example of a really interesting research write-up, with a disappointingly misleading headline. Here’s how the write-up is framed:

From millionaires to Muslims, small subgroups of the population seem much larger to many Americans

Juicy, huh? Makes you want to read it? Me too.

So what’s my problem with the headline? I’ll get to that in a bit. Let’s unpack the article, friends! Here’s how it kicks off:

“When it comes to estimating the size of demographic groups, Americans rarely get it right. In two recent YouGov polls, we asked respondents to guess the percentage (ranging from 0% to 100%) of American adults who are members of 43 different groups, including racial and religious groups, as well as other less frequently studied groups, such as pet owners and those who are left-handed.
When people’s average perceptions of group sizes are compared to actual population estimates, an intriguing pattern emerges: Americans tend to vastly overestimate the size of minority groups. This holds for sexual minorities, including the proportion of gays and lesbians (estimate: 30%, true: 3%), bisexuals (estimate: 29%, true: 4%), and people who are transgender (estimate: 21%, true: 0.6%).
It also applies to religious minorities, such as Muslim Americans (estimate: 27%, true: 1%) and Jewish Americans (estimate: 30%, true: 2%). And we find the same sorts of overestimates for racial and ethnic minorities, such as Native Americans (estimate: 27%, true: 1%), Asian Americans (estimate: 29%, true: 6%), and Black Americans (estimate: 41%, true: 12%).”

We’re then treated to this visual which neatly shows the gap between Americans’ perceptions of the size of these groups, and reality:

At this point we’re a third of the way through the write-up.

Trouble is, this is the internet, home of short attention spans.

We’re busy right? Do we need to read the rest? The write-up has already delivered versus the headline, so we can safely click away, right?

WRONG.

It’s actually about to get much more interesting, the trouble is, I suspect that most readers have already clicked away. That’s my problem with the headline.

Stick with me folks, the remainder of the article is actually where the good stuff is.

Possibly you looked closely at the visual above and noticed that people are apparently poor at estimating the true size of majority groups too:

“A parallel pattern emerges when we look at estimates of majority groups: People tend to underestimate rather than overestimate their size relative to their actual share of the adult population. For instance, we find that people underestimate the proportion of American adults who are Christian (estimate: 58%, true: 70%) and the proportion who have at least a high school degree (estimate: 65%, true: 89%).”

And maybe you also noticed that estimates appear to be way more accurate when the true proportion is close to 50%:

“The most accurate estimates involved groups whose real proportion fell right around 50%, including the percentage of American adults who are married (estimate: 55%, true: 51%) and have at least one child (estimate: 58%, true: 57%).”

What’s going on there? Read on, friends:

“Misperceptions of the size of minority groups have been identified in prior surveys, which observers have often attributed to social causes: fear of out-groups, lack of personal exposure, or portrayals in the media. Yet consistent with prior research, we find that the tendency to misestimate the size of demographic groups is actually one instance of a broader tendency to overestimate small proportions and underestimate large ones, regardless of the topic.”

Dear reader, in the past, I too have jumped to the conclusion that it’s fear (or something like it) that causes people to overestimate the true proportion of minority groups. Moreover, I’m pretty certain I’ve been involved with creating pieces of content which arrived at this conclusion.

Trouble is, it turns out that this conclusion is pretty flawed.

The write-up continues:

“If exaggerated perceptions of minority groups’ share of the American population are due to fear, we would expect estimates of those groups’ share that are made by the groups’ members to be more accurate than those made by others. We tested this theory on minority groups that were represented by at least 100 respondents within our sample and found that they were no better (and often worse) than non-group members at guessing the relative size of the minority group they belong to.
Black Americans estimate that, on average, Black people make up 52% of the U.S. adult population; non-Black Americans estimate the proportion is roughly 39%, closer to the real figure of 12%. First-generation immigrants we surveyed estimate that first-generation immigrants account for 40% of U.S. adults, while non-immigrants guess it is around 31%, closer to the actual figure of 14%.”

The write-up then goes on to explore the perceptions of group sizes which are arguably less charged, like left-handedness:

“Although there is some question-by-question variability, the results from our survey show that inaccurate perceptions of group size are not limited to the types of socially charged group divisions typically explored in similar studies: race, religion, sexuality, education, and income. Americans are equally likely to misestimate the size of less widely discussed groups, such as adults who are left-handed. While respondents estimated that 34% of U.S. adults are left-handed, the real estimate lies closer to 10-12%.
Similar misperceptions are found regarding the proportion of American adults who own a pet, have read a book in the past year, or reside in various cities or states. This suggests that errors in judgment are not due to the specific context surrounding a certain group.”

Now things are getting really interesting, huh?

If errors in judgement when estimating the size of various groups are not due to fear (or something like fear), what’s going on?

“Why is demographic math so difficult? One recent meta-study suggests that when people are asked to make an estimation they are uncertain about, such as the size of a population, they tend to rescale their perceptions in a rational manner.
When a person’s lived experience suggests an extreme value — such as a small proportion of people who are Jewish or a large proportion of people who are Christian — they often assume, reasonably, that their experiences are biased.
In response, they adjust their prior estimate of a group’s size accordingly by shifting it closer to what they perceive to be the mean group size (that is, 50%).
This can facilitate misestimation in surveys, such as ours, which don’t require people to make tradeoffs by constraining the sum of group proportions within a certain category to 100%.
This reasoning process — referred to as uncertainty-based rescaling — leads people to systematically overestimate the size of small values and underestimate the size of large values. It also explains why estimates of populations closer to 0% (e.g., LGBT people, Muslims, and Native Americans) and populations closer to 100% (e.g., adults with a high school degree or who own a car) are less accurate than estimates of populations that are closer to 50%, such as the percentage of American adults who are married or have a child. ”

For what it’s worth I feel like this part of the write-up is a little unclear, because a couple of explanations are being chucked in simultaneously.

This is how I interpret it:

Problem 1: People are uncertain – i.e. they don’t know the true proportions of these demographic groups, and so they’re forced into guessing.
Problem 2: When people are forced to guess (as they are when confronted with studies like this) they have a tendency to shift their guesses towards something like 50% because that feels more reasonable or rational to them. (This would explain why minority groups are overestimated, majority groups are underestimated, and groups which do account for about 50% of the population appear to be estimated reasonably accurately).
Problem 3: the structure of most studies like this don’t ask respondents to total their demographic estimates so they equal 100% – they ask for each estimate in isolation. I think they’re saying this leads to further inaccuracies as evidenced by the survey above:
- Americans apparently believe that:
  - 27% of the population are Native American
  - 29% of the population are Asian
  - 41% of the population are Black
  - 39% of the population are Hispanic
  - 64% of the population are White
    - These estimates total 200% so can’t possibly be correct. I think the point being made here is that potentially people might be able to make somewhat better guesses if they are forced to reach a total of 100%.

All of this is a very long winded way of saying that we have misconceptions about the true demographic make up of our populations because we don’t know the actual proportions, so are forced to guess, and we are very bad at guessing.

For clarity here, I’m in no way suggesting that racism doesn’t exist, or that media depictions of people from minority groups aren’t frequently extremely problematic and fuel fear.

What I am saying is that it would appear that the reasons people appear to have such skewed perceptions in studies like this one are likely more to do with being bad at guessing stuff, and less about their own fears or biases.

In the final paragraph, (again completely buried so it’s easy to miss), the article asks yet another important question.

If we accept that for the most part, people do not know the true proportions of various demographic groups, (which certainly appears to be the case); might correcting these misconceptions change people’s stance on key issues?

“Does correcting misperceptions of group size change peoples’ attitudes on related issues? Current research suggests it does not.
In a series of studies (one of which used a survey fielded by YouGov), political scientists John Sides and Jack Citrin attempted to correct inaccurate beliefs about the size of the U.S. foreign-born population, both subtly, by embedding the accurate information in a news story, and explicitly, by providing survey respondents with Census Bureau estimates.
They found that while providing this information did somewhat improve people’s knowledge of the number of immigrants in America, they did not make people more supportive of immigration.”

So the answer is no.

Wow. That’s pretty scary huh?

Regardless, I feel like communicating accurate demographic data is still a worthwhile endeavour, but it’s clear that it’s not enough just to do that.

I hope this whole thing hasn’t come off a preachy: “READ THE WHOLE ARTICLE PEOPLE!” rant. If it helps at all, if it has, it’s a rant directed largely at myself that I’ve elected to send on to you, because I too very nearly clicked away a third of the way through because I thought I’d got what I came for.

I think the combination of the misleading headline, plus the fact that the real story was buried so much later in the article will likely have caused many people to click away early, without reading the whole thing.

I think this highlights the importance of both headlines and article structure for pieces like this. As a reminder, this was the original headline: From millionaires to Muslims, small subgroups of the population seem much larger to many Americans

I’m guessing that because that was the headline, the article was shaped as follows:

Juicy, but misleading overall stats about the overestimation of minority groups
Notes on the underestimation of majority groups
Finally the good stuff: a little about the why people overestimate – is it about fear? Actually no
More about why people really overestimate this stuff
A final note on whether or not correcting skewed perceptions might change people’s attitudes to related issues

But let’s imagine the headline was something like this instead: Why are Americans’ estimates of minority groups so unbelievably skewed?

A headline like that might have lead to an article shaped like this:

Cut straight to the why: debunk the “fear” myth early by showing the responses from Black Americans alongside the overall responses
Go on to talk more about why people are actually bad at making these estimates
Share the study which suggests correcting skewed perceptions might not be enough to change people’s attitudes

By moving the spotlight away from the raw results of study, and focusing more squarely on the “why” I think that this article might have been more successful in communicating what I think is the core point: i.e. why we’re so bad at this stuff.

But there’s a further consideration of course. Would my new headline have received the same level of engagement? Hard to know right? Sensational headlines get shares, and is my new headline quite so sensational? Arguably not. Ugh.

That said, even if they kept the original headline, but just adopted an article structure more like the one I’ve outlined above the piece might have better communicated what I think is their core point.

Moar serendipitous finds:

Friends, these emails of mine are becoming increasingly long and unwieldy, so I’m going to try to keep this section more succinct.

Click these links people, there’s some lovely stuff here.

Beth Moon is documenting ancient Baobab trees before they disappear

“In the presence of old trees, I am reminded there is still grace and beauty in the world…” ~ photographer Beth Moon.

A small list of knowable things

I love these weekly reflections from Jonny Sun. Here are my favourites so far:

Ukrainian Stamp Design Contest

After Russia invaded the country, Ukraine’s post office decided to hold a contest to design a stamp that illustrated Ukrainians’ determination to defend their land.

Also check out Polish illustrator Paweł Jońca’s poster, all proceeds from sales are going to humanitarian aid for Ukraine.

A Russian bear steps Ukrainian lego by Paweł Jońca

I would like to be paid like a plumber

17 months after Nevermind was released, Nirvana began to record what would be their final album, In Utero. In November of 1992, shortly before they formally agreed on his involvement, Steve Albini wrote to Nirvana and laid bare his philosophy in this pitch letter.

Who’s keeping whom safe?

Aubrey Hirsch on Florida’s “Don’t say gay” bill.

Part II: Books I’m Reading Right Now

I absolutely loved Good Talk by Mira Jacob, a graphic novel about identity, belonging, family, and love.

Like many six-year-olds, Jacob’s Jewish Indian American son Z has questions about everything: Are white people scared of us? Is it bad to be a Brown person? Can Indians be racist? With humour and heart, Jacob reflects on her own life experiences in order to try to answer Z’s questions honestly. It’s brilliant.

Part III: Things I’ve Been Watching

Here are a couple of things which I’ve watched this fortnight and would recommend:

Top Boy, Netflix – Ronan Bennett’s 2011 Channel 4 drama about London drug gangs is back, and it’s brilliant.
The Andy Warhol Diaries, Netflix – this documentary series doesn’t fully delve into the problematic aspects of Warhol’s artistic practice, but nevertheless I found it fascinating.
Windfall, Netflix – ignore the IMDb rating, this arty psychological thriller is great.

Part IV: What I’ve been up to…

One of my stories is being published in the National Flash Fiction Day 2022 Anthology later this year. I am beyond thrilled. This is a story that’s previously been rejected a couple of times, which goes to show (I think) that just because a story isn’t a good fit for one publication, doesn’t mean it won’t be a good fit for another. Also, if I’d given up when it was first rejected, this story wouldn’t now be being published.

I signed up to this workshop which I really enjoyed, and would highly recommend if you are a person who wants to make a thing (or things), but are struggling to figure out exactly what that thing should look like.

My online short story course with London Lit Lab ends this week – it’s flown by, but it’s been brilliant, and as a result I’ve new story drafted which still needs a bunch of work, but I’m excited about.

What’s next?

I will be going to BrightonSEO, then recovering from BrightonSEO. I’m hoping that I get some more writing done in the next fortnight, and maybe I’ll make another collage.

If you enjoyed this newsletter, you can receive direct to your inbox. Sign up here.