Missingness: How Gaps in Perception, Data, Reasoning Mislead Us

•

 10 min read

YouTube video ID: PqtXiABMj3A

Source: YouTube video — Watch original video

PDF

Kit Yates, a professor of mathematical biology and public engagement at the University of Bath, discussed his book, "You Don't Know What You're Missing," which explores the concept of "missingness" – the idea that the information we use to understand the world is incomplete, and these gaps are often unnoticed. He categorized missingness into three types: intrinsic, extrinsic, and constructive.

Intrinsic Missingness: The Gaps Within Us

Intrinsic missingness refers to the limitations inherent in our own minds and bodies, such as sensory adaptation, limited information processing, and selective attention. Our brains constantly interpret and construct reality rather than passively receiving it.

Reading Exercises and Brain's Interpretations

Yates began with reading exercises to illustrate how our brains fill in gaps. For instance, when presented with a sentence containing a misspelled word like "mitsake," many people quickly identify the intended word "mistake." However, they often miss a second, less obvious mistake, such as a repeated word ("the the"), because their brains are so efficient at processing familiar patterns and context. This highlights how our brains prioritize meaning over precise detail, which can be both a strength (allowing fast reading) and a weakness (hindering proofreading).

Another example involved sentences with jumbled letters, substituted numbers, or missing vowels. Despite these alterations, the brain can still decipher the text, especially when the first and last letters are in place or when context is provided. This demonstrates the brain's ability to use context and expectation to fill in missing information.

A more challenging example, "The pickards found it difficult to discharge patients from their supervision because of the backlogs in accident and emergency," showed that while context helps, longer words and more complex structures make it harder for the brain to fill in the blanks without explicit information.

Perception as an Inventive Act

Our perception of reality is not an objective narration but an "inventive and creative act." Our brains weave sensory signals into a coherent narrative. For example, sound waves don't come with labels like "voice" or "car engine"; our brains interpret them. Similarly, light waves aren't labeled "green" or "blue"; our brains assign these meanings. This process, known as the problem of perception, means that different individuals can construct different realities even from the same stimuli.

Sensory Conflation and Adaptation

Our senses often conflate different properties of reality. Yates provided several examples:

  • Size and Distance: When asked to hold a thumb the same size as a large screen, people instinctively move their thumb closer, demonstrating how our brains conflate size and distance.
  • Sound and Distance: A sonic boom, a loud sound from far away, can be misinterpreted as a quieter, closer sound (e.g., someone falling out of bed or a car backfiring) because our brains use expectations to interpret sounds.
  • Temperature and Thermal Effusivity: A metal spoon feels colder than a wooden spoon at the same room temperature because metal conducts heat away from our hand more quickly. Our senses prioritize the rate of heat transfer (thermal effusivity) over absolute temperature, which is more relevant for survival (e.g., avoiding burns or frostbite).

Sensory adaptation is another form of intrinsic missingness where our senses stop responding to constant stimuli.

  • Visual Adaptation: Staring at a black cross surrounded by colorful patterns can make the colors disappear, leaving a gray background. This happens because photoreceptor cells become saturated and stop sending signals to the brain. Our eyes constantly make tiny, jerky movements (microsaccades) to prevent this from happening in everyday vision.
  • Tactile Adaptation: We stop feeling our clothes after a while because our touch receptors adapt.
  • Olfactory Adaptation: People who wear strong perfume may not smell it themselves after a short period because their sense of smell adapts. Yates shared a tragic story of a contractor who died in a factory explosion because his sense of smell adapted to the increasing natural gas leak, preventing him from realizing the danger.

Inattentional Blindness and Limited Attention

Our attention is limited, akin to a spotlight. We can focus on one part of a scene but cannot process everything simultaneously. This leads to inattentional blindness, where we miss things that are plainly visible because our attention is directed elsewhere.

Yates demonstrated this with a card trick where participants were asked to focus on one card, which then "disappeared." In reality, all the cards on the second slide were different from the first, but the focused attention on a single card prevented participants from noticing the change in the others.

This phenomenon has real-world implications, such as driving while talking on a hands-free phone. While hands-free calls keep eyes on the road and hands on the wheel, the cognitive load of the conversation diverts attention, making drivers as impaired as those at the drink-driving limit. Passengers, however, can mitigate this risk by adapting their conversation to driving conditions and helping spot hazards. Audiobooks, unlike conversations, can be easily tuned out, allowing attention to shift back to driving.

The Power of Expectation

Expectation significantly influences perception. Yates used the classic duck-rabbit illusion to show how the same image can be perceived differently based on whether one expects to see a duck or a rabbit.

The "dress" phenomenon from 2015, where people saw a dress as either blue and black or white and gold, is another example. The actual dress was blue and black, but perception depended on the assumed lighting conditions. Those who subconsciously expected artificial yellow light tended to see blue and black, while those expecting bluer daylight saw white and gold. Our brains perform "color constancy," adjusting for lighting to perceive true colors, but in ambiguous images, this can lead to different interpretations.

Takeaways from Intrinsic Missingness

  • Our senses and brains construct their best guesses of reality; there is no objective reality.
  • Our brains simplify things, deciding what we see and what we don't, leading to inevitable glitches and missed information.
  • The most dangerous situations arise when we believe we've seen everything and don't realize something is missing.
  • Spell checkers cannot catch all mistakes, especially those involving correctly spelled but contextually incorrect words, because they don't understand meaning.

Extrinsic Missingness: Information Filtered from the Outside

Extrinsic missingness occurs when information is filtered, lost, biased, or suppressed before it reaches us, making us believe we have the complete picture when we don't.

Selection Bias

Selection bias happens when the sample used to gather information is not representative of the larger population.

  • Daily Express Headline: A 2016 Daily Express headline claimed "98% say no to EU deal" based on a phone survey. However, to participate, readers had to find the question on page seven of the previous day's paper and call a paid phone line. This process filtered for readers of the Daily Express who felt strongly enough to pay, creating a highly biased sample.
  • Representative Samples vs. Censuses: A census (asking everyone) provides the most accurate data but is often impractical. Surveys rely on representative samples.
  • Margin of Error: Even random sampling can lead to errors, especially with small sample sizes. The margin of error indicates the range within which the true answer lies. To halve the margin of error, the sample size must be quadrupled, not just doubled. Many Brexit polls had insufficient sample sizes to accurately predict the outcome.
  • Self-Selection Bias: This occurs when people choose to participate in a survey, leading to overrepresentation of those with strong opinions. An online survey on immigration showed extreme polarization, but a demographically representative survey revealed a more moderate distribution of opinions.
  • Non-Response Bias: When people don't respond to a survey, and their non-response is related to the survey's topic. In the 2016 US presidential election, many Donald Trump supporters did not respond to polls, leading to an underestimation of his support and inaccurate predictions.
  • Attrition Bias: Common in longitudinal studies (e.g., weight loss trials, educational interventions), this bias occurs when participants drop out, and the reason for dropping out is related to the study's outcome. For example, those struggling in a weight loss trial might drop out, leaving a biased sample of successful participants.

Response Bias

Response bias occurs when people do not respond truthfully to survey questions.

  • Hawthorne Effect (White Coat Effect): Responses change purely because people are being studied. In a 1920s factory study, workers' productivity increased regardless of lighting changes, simply because they knew they were being observed. Similarly, blood pressure can rise at a doctor's office due to the stress of being examined.
  • Acquiescence Bias (Yay-saying/Please-you Effect): Giving a "yes" answer to please the experimenter.
  • Naysaying (Screw-you Effect): Giving the opposite answer to what is expected to undermine the experimenter.
  • Courtesy Bias: Giving a polite but untruthful answer (e.g., saying a bad meal was "nice").
  • Coercion Bias: Giving a desired answer out of fear, often seen in authoritarian regimes where dissent is not anonymous (e.g., North Korean elections).
  • Social Desirability Bias: Conforming to what is perceived as socially acceptable, avoiding admission of illicit, illegal, or taboo behaviors.

Randomized Response Technique

To counter social desirability bias, the randomized response technique provides plausible deniability. Yates demonstrated this by asking the audience about phone use on the toilet. Participants secretly flipped a coin: if heads, they answered truthfully; if tails, they answered "yes." This method allows for statistical estimation of the true proportion of a sensitive behavior without identifying individual respondents. The audience's results suggested a much higher rate of phone use on the toilet (86%) than a YouGov poll (57%), likely because the randomized response technique reduced embarrassment.

Takeaways from Extrinsic Missingness

  • Information is filtered before it reaches us, meaning we often don't see the whole picture.
  • We must ask: "What are we not seeing?" and "Who are we not seeing?"
  • We must also consider why people might not be telling the truth.

Constructive Missingness: Reasoning from Absence

Constructive missingness is the ability to use the absence of information to learn something about the world.

The Dog That Didn't Bark

Sherlock Holmes's famous observation in "Silver Blaze" illustrates this. The fact that the guard dog did not bark when the racehorse was stolen indicated that the thief was known to the dog, leading Holmes to deduce the trainer's involvement. This is reasoning from negative data – the absence of an expected event (the dog barking) provides crucial information.

Absence of Evidence vs. Evidence of Absence

The phrase "absence of evidence is not evidence of absence" is often used, but Yates argues for a more nuanced understanding.

  • Court Cases: A defense barrister might claim "no evidence" to place a client at a crime scene. This could mean the client wasn't there, or it could mean evidence was overlooked, destroyed, or not discoverable.
  • The Coelacanth: For a century, the coelacanth was known only from fossils, leading scientists to believe it was extinct. However, in 1938, a living coelacanth was discovered, proving that the "absence of evidence" for its continued existence was not "evidence of absence."
  • Loch Ness Monster: Extensive searches for the Loch Ness Monster that yield no evidence are not "no evidence." They are strong evidence that the monster is not there, even if not absolute "proof of absence."

The Scientific Method

The scientific method relies on constructive missingness. To prove a new hypothesis, scientists must try to falsify it. The more attempts to falsify a hypothesis fail, the more confident scientists become in its truth, though it is never definitively "proven."

  • Wason Selection Test: This cognitive task demonstrates that people are often poor at identifying the necessary information to falsify a hypothesis. To test "If a card shows an even number on one face, then the opposite face is spotty," one must turn over the card with an even number (to confirm spots) and the card with stripes (to ensure it doesn't have an even number, which would falsify the hypothesis).
  • Mammals and Live Birth: To test "All mammals give birth to live young," simply finding more mammals that give live birth is not enough. One must actively search for mammals that don't give live birth (egg-laying mammals) to potentially falsify the hypothesis. The duck-billed platypus, an egg-laying mammal, falsifies this hypothesis.

Conclusion

The core message is to actively look for what is not there, alongside what is there. Failing to do so means we will miss important absences and literally "not know what we're missing."

  Takeaways

  • Intrinsic missingness shows our brains fill in gaps, leading to both rapid comprehension and overlooked errors such as missed misspellings.
  • Sensory adaptation and inattentional blindness demonstrate that even obvious stimuli can be ignored when attention is focused elsewhere, affecting tasks like driving.
  • Extrinsic missingness arises from biased sampling, response biases, and filtered information, meaning surveys and headlines often present a distorted picture of reality.
  • The randomized response technique can reveal hidden behaviors by protecting respondent anonymity, highlighting how social desirability bias skews self‑reports.
  • Constructive missingness emphasizes learning from what is absent, as illustrated by Sherlock Holmes’s “dog that didn’t bark” and scientific falsification strategies that seek missing counter‑examples.

Frequently Asked Questions

Why does the brain prioritize meaning over precise detail in reading?

The brain favors meaning because it evolved to extract useful information quickly for survival, using context and pattern recognition to predict words. Thus, when reading a misspelled sentence, it fills in the expected word and may overlook less obvious errors, trading accuracy for speed.

How does the randomized response technique reduce social desirability bias?

The randomized response technique reduces social desirability bias by allowing respondents to answer truthfully or give a predetermined random answer that is hidden from the researcher, preserving anonymity. Aggregating many such responses lets statisticians estimate the true prevalence of a sensitive behavior without individual identification, producing more honest data than direct questioning.

Does this page include the full transcript of the video?

Yes, the full transcript for this video is available on this page. Click 'Show transcript' in the sidebar to read it.

on page seven of the previous day's paper and call

paid phone line. This process filtered for readers of the Daily Express who felt strongly enough to pay, creating a highly biased sample. - Representative Samples vs. Censuses: A census (asking everyone) provides the most accurate data but is often impractical. Surveys rely on representative samples. - Margin of Error: Even random sampling can lead to errors, especially with small sample sizes. The margin of error indicates the range within which the true answer lies. To halve the margin of erro

Helpful resources related to this video

If you want to practice or explore the concepts discussed in the video, these commonly used tools may help.

Links may be affiliate links. We only include resources that are genuinely relevant to the topic.

Full transcript is not shown on this page

This page focuses on the summary and original notes. For full verification, refer to the original YouTube video.

PDF