Does multitasking damage your attention?

This is the lesson the course was built around, and the reason isn't the subject. It is that this one literature does something almost no other does: it answers the same question two ways and gets two answers, and both answers have been counted.

The claim you have met

In 2009, Ophir, Nass and Wagner published a study in a major journal that is the study everybody in this subject cites.³ They built a questionnaire called the media multitasking index, split people into heavy and light multitaskers by their scores on it, and reported that the heavy multitaskers did worse on tests of filtering out distraction. What exactly the index asks is not something this course can tell you, because it has not opened the paper.

This course hasn't opened that paper.³ What the lesson holds at search-summary level is the shape of the claim, which is the part that travelled: heavy media multitaskers are worse at filtering distraction. The figures below come from a meta-analysis that read it and 117 other assessments.

What happened next is the ordinary and admirable thing. Other people tried it. Some found the association and some didn't, and after a decade somebody pooled the lot.

What pooling found

Parry and le Roux published the synthesis in 2021.¹ This course has read its abstract, its overall result, its moderator analyses and its conclusion, which is enough for what follows and isn't enough to have weighed every study in it.

The scale, in their words: "Following a systematic search and selection process, 118 assessments were included in the meta-analysis."¹

The overall result, in their words: "Overall, across all 118 assessments, the effect size for the association between media multitasking and cognitive control is small (z = .138, 95% CI [.107, .170], p < .001; with RVE: 95% CI [.102, .174], p < .001) and, as would be expected, highly heterogeneous (I2 = 79.76%, Q(117) = 531.795, p < .001)."¹

One word first, because the rest of this course runs on it. An effect size is a number saying how big a difference is, in units that let you compare one study with another: zero means no difference, and the further from zero the bigger the effect. The letter changes with the measure, and you will meet z here and g in lesson 5. What matters is the sign, the distance from zero, and the interval around it, and this course gives you all three every time.

Three things to take from that sentence and no more. The association is there. It's small. And the studies disagree with each other a great deal, which is what the heterogeneity figure says.

And one thing the sentence does not say, which this course has to. The 118 assessments pool studies that are overwhelmingly student samples, as the research file records.¹ So every figure below is a figure about students, and the sentence "people who multitask are more distractible" is already wrong at the second word.

If the lesson stopped there it would be a dull lesson. What makes it the centre of this course is the next step.

The correction, and what it did

There's one more step, and it's the one that changes the lesson's conclusion.

A pooled estimate is only as good as what got published, and studies that find nothing are less likely to be written up and accepted. There are tests for whether a literature looks like it is missing its null results: one is Egger's test, which asks whether small studies report bigger effects than large ones in a way that a real effect would not produce.

Egger's test was significant for the performance-based assays and not for the self-report ones.¹ So the authors ran a correction called trim-and-fill, which imputes the studies the pattern suggests are missing and re-pools.

In their words: "In this sensitivity analysis, the pooled effect for performance-based assays was no longer statistically significant (z = .032, 95% CI [−.024, .088], p = .260)."¹

Read the interval rather than the estimate. It runs from below zero to above it. Once the pattern of missing nulls is corrected for, the performance-based literature cannot distinguish the effect from nothing.

The whole thing in one table.¹

So which half is right?

This is where a worse course would stop and declare a winner. The performance evidence is the more direct, it doesn't survive correction, therefore the claim is dead.

That reading throws away half the finding and this course won't make it. Here are three explanations for the split, and the lesson's view is that all three are live.⁴

One: the two instruments ask about different stretches of time. A task measures twenty minutes in a quiet room. A questionnaire asks about a life, in which the person's doing things they chose, among other people, with everything else that is true of them.

Two: the self-report is picking up something real that is not attention. Somebody who multitasks heavily may also have more demanding work, more people contacting them, or less control over their day. All of those would show up on a distractibility questionnaire and none would show up on a filtering task.

Three: the belief itself is doing work. A person who has concluded their attention is ruined behaves differently from one who hasn't, and the questionnaire is the only instrument of the two that can see it.

Notice that none of the three requires anybody to be lying or mistaken. And notice that the first two are testable: a study measuring the same people in a quiet room and in their own working week would separate them, and one measuring how much people are interrupted would bear on the second. The third is the hardest to test and this course has no design for it. Nothing this course read separates any of them.

What the authors themselves conclude

In their words: "Ten years on from Ophir et al. (2009) the picture is not any more transparent. Based on the papers reviewed in this study we are no closer to understanding 'cognitive control in media multitaskers'."¹

And, in the same paragraph, the other half, which is why this is not a debunking: "However, complicating matters, studies adopting a different measurement approach than this first investigation have consistently produced results supporting the claim that media multitasking is negatively associated with everyday executive functioning."¹

Both sentences are theirs and they belong together. A meta-analysis that had found nothing would say so; this one says the picture is unclear and that one half of it consistently finds something.

Worked: reading a headline about this

This case is constructed.⁴ The headline is invented and the reasoning is the course's own.

"Heavy multitaskers have measurably worse attention, study finds." What do you now ask?

1. Which instrument? If the study measured attention with a questionnaire, it's in the half where the association consistently holds, and the headline's word "measurably" is doing something misleading: a self-report is a measurement, and it isn't the kind the reader will picture.
2. Is it one study or pooled? One study in this literature tells you very little, because the studies disagree with each other enough to produce an I² of nearly 80 percent.
3. Cross-sectional or not? If the people were measured once, "damage" and "cause" are words the design cannot support, whichever instrument was used.
4. What would change the answer? For this literature, a longitudinal design or an experiment, which the meta-analysts say does not yet exist in what they reviewed.

Four questions, and the headline as written can't answer any of them. That isn't a reason to dismiss it; it is a reason to go and look, which takes about ten minutes and is lesson 7's exercise.

Three things people get wrong about this

"Multitasking rewires your brain." Nothing in what this course read measures anybody's brain, and every study pooled here is cross-sectional.

"The original study was debunked." Too strong. One half of the literature that followed it consistently supports an association, and the meta-analysts say so.

"A small effect means no effect." A small effect that holds across 45 self-report assessments is a finding. What it isn't is a licence to say anybody's attention was damaged.

Practice

Connections

Back. Lesson 1's two instruments are the whole of this lesson, and the predict block is where they pay off. Habits and Self-Discipline lesson 5 introduced confidence intervals and what an interval spanning zero does and does not mean, and this lesson uses it rather than re-teaching it. Reading Well lesson 8 is where reading a paper's methods was taught.

Forward. Lesson 5 is the same move on a smaller and more specific claim, with a meta-analysis that splits by domain and by region instead of by instrument. Lesson 6 asks whether any of this can be trained. Lesson 7 makes the instrument question a habit.

Go deeper

• "Cognitive Control in Media Multitaskers" Ten Years On: A Meta-Analysis (Cyberpsychology, 2021). Open access. Read in substantial part by this course: the abstract, the overall result, the moderator analyses and the conclusion. Read the conclusion first, which is two paragraphs and says both halves of the finding without flinching from either.
• Does the Brain Drain Effect Really Exist? A Meta-Analysis (Behavioral Sciences, 2023), open access and read in substantial part by this course. It is lesson 5's subject and is worth reading beside this one: a second pooled literature on a second famous claim, splitting a different way, which is how you tell a pattern from a coincidence.

Sources

1. Douglas A. Parry and Daniel B. le Roux, "'Cognitive Control in Media Multitaskers' Ten Years On: A Meta-Analysis", Cyberpsychology 15(2), 2021, article 7. Read in part: the abstract, the overall meta-analytic result, the moderator analyses and the conclusion. The full review of the included studies was not read. Supports: the quoted count of 118 assessments; the quoted overall result with its intervals and heterogeneity; the quoted self-report and performance-based pooled effects; that Egger's test indicated asymmetry for performance-based assays and not for self-report; the quoted trim-and-fill result; the quoted statement about cross-sectional designs and causality; and both quoted sentences from the conclusion. The counts of 45 and 73 in the table are from the same moderator analyses.
2. Wiradhany and Nieuwenstein, "Cognitive control in media multitaskers: Two replication studies and a meta-analysis", Attention, Perception, & Psychophysics 79, 2017. Search-summary level; not opened. Named in research/SOURCES.md as the earlier synthesis, superseded in scope by source 1, and nothing in this lesson rests on it.
3. Eyal Ophir, Clifford Nass and Anthony D. Wagner, "Cognitive control in media multitaskers", PNAS 106(37), 2009. Search-summary level; the PDF was located and not read, and the body says so where the study is described. Supports only the shape of the original claim: an index of media multitasking, a heavy and light split, and worse performance on filtering among the heavy group. No figure in this lesson comes from it.
4. The three explanations for the split between instruments are this course's own reasoning, marked inline where they appear, and the lesson says that nothing it read tells them apart. The headline in the worked case is invented and labelled as such, and the four questions asked of it are the course's own.