The phone on the desk

Lesson 4 was the same move at a larger scale: a famous finding, a decade of work, and a synthesis that split by instrument. This lesson is smaller, sharper and more personal, because it is about a decision nearly everybody has made without evidence.

Where should your phone be while you work?

The claim

In 2017, Ward, Duke, Gneezy and Bos published a study that gave this subject its phrase. People did working-memory tasks with their own phone in one of three places: on the desk, in a bag, or in another room. Those with the phone in another room performed best, even though nobody was using it. They called the effect brain drain: the mere presence of the phone was said to consume cognitive capacity.

This course hasn't opened that paper.³ What it holds at search-summary level is the shape of the claim and the name. The figures in this lesson come from a meta-analysis that pooled it and twenty-one other studies.

Then, in 2022, somebody ran it again properly. Ruiz Pardo and Minda published a preregistered direct replication of the second of Ward's experiments, crossing phone location with whether the phone was switched on, six conditions in all, measuring working memory and inhibition. The effect did not replicate: phone location made no difference.⁴

This course hasn't opened that paper either,⁴ and holds the design and the outcome at search-summary level, which is enough for what the lesson does with it: the claim went one way and a careful direct test went the other, which is where a pooled estimate earns its keep.

What pooling found

Böttger, Poschik and Zierer published the meta-analysis in 2023.¹ This course has read its abstract, its overall result, both subgroup analyses and its discussion.

Effect sizes here are reported as g rather than lesson 4's z, and the reading is the same: zero means no difference, the sign says which way, and the interval says what the pooled evidence can distinguish.

The scale, in their words: "In a database search we identified 22 studies with a total of 43 relevant effects that could be assigned to the categories 'memory', 'attention', and 'general cognitive performance'."¹

The overall result, in their words: "The pooled effect across all studies was g = −0.14, with a 95% confidence interval (CI) of −0.24 to −0.03, at p < 0.05."¹ The included effects disagree with each other more than chance would explain, at an I² of 61.84; I² is the share of the variation between studies that is real difference rather than noise, so about three fifths here.

Something is there. The interval sits entirely below zero, the direction is the one the original claimed, and a course that stopped here would say the effect is real and small.

It doesn't stop here, because the authors ran two subgroup analyses and both of them matter more than the headline.

Those three figures again, outside the block, because the rest of the lesson uses them. Memory g = −0.23, interval −0.36 to −0.10. Attention g = −0.07, interval −0.21 to 0.06. General cognitive performance g = 0.10, interval −0.52 to 0.72.¹

The subgroup that makes this lesson hard

The second analysis split by region, because an earlier study had reported different effects for Asian and European participants.¹

Read the bottom row twice. The original study was run in the United States. In the pooled North American samples, the effect is −0.03 with an interval straddling zero, which is as close to nothing as a meta-analysis gets.

What a subgroup analysis is, before the discipline. You take the studies you have already pooled, sort them by some feature, and pool each pile separately. It costs nothing, it can be done after the fact on any feature you like, and that is both why it is useful and why it needs handling.

Now the discipline. A subgroup analysis tells you that something differs between these literatures. It doesn't tell you what. The four candidates below are this course's own list rather than one the authors give, and it can't choose between them.⁵

1. The populations differ in something that matters, which is the reading the phrase "cultural difference" reaches for and is the least supported of the four.
2. The studies differ. Different tasks, different phone conditions, different sample sizes, different years.
3. Publication practice differs, and a literature's null results aren't equally likely to be published everywhere.
4. Chance. Subgroup analyses generate spurious differences for the ordinary reason that they involve multiple comparisons on smaller pools.

The authors do report that the Asian subgroup was much the least heterogeneous of the three, at an I² of 30.58 against 70.71 in Europe and 50.98 in North America.¹ That makes chance less comfortable as an explanation than it would otherwise be, because the studies inside that subgroup are agreeing with each other rather than scattering. It does not make it impossible, and the authors call for more research rather than offering a mechanism. Neither does this course.

What the authors conclude, and why the lesson quotes it

A lesson could stop at the three null intervals and leave a reader thinking the whole thing was a scare. The people who did the pooling don't think that.

Their abstract ends: "In view of the present research results, it seems important that people in general, and especially children and adolescents in schools and classrooms, learn how to deal with the distracting potential of smartphones."¹

That is a policy recommendation from a paper whose attention subgroup crossed zero, and both things are true at once. A significant overall pooled effect, a real effect in memory, and authors who think the practical upshot is caution. Reporting the intervals and not the conclusion would be as partial as reporting the conclusion and not the intervals.

They also report that the funnel plot showed no obvious asymmetry,¹ which is a mild point in the literature's favour and a weaker one than it sounds. A funnel plot is the picture Egger's test in lesson 4 puts a number on, and on 43 effects it is not a powerful test.

And their case for the recommendation is better than the intervals alone suggest, which is the half a sceptical reading loses. There is a real pooled effect in memory. And the measures these studies used are cognitive-capacity tasks, which is not the same thing as what a phone does to a classroom: the distracting potential the authors are worried about is largely not what the pooled measures were testing. That second half is this course's reading of their discussion rather than a sentence they write.⁵ A reader who takes the null attention interval as the end of the argument has answered a narrower question than the one the authors asked.

Worked: reading a subgroup like this one

This case is constructed and the numbers are invented.⁵ It exists because the discipline in the section above is easier to state than to apply.

A meta-analysis of a sleep intervention pools 30 studies and finds g = −0.11 overall, with an interval just below zero. It then splits by whether the studies were run in a university lab or in people's homes. Lab studies: g = −0.34, significant. Home studies: g = 0.01, not.

What would you conclude? A reasoning order, which is the course's own and not a rule anybody has published.⁵

1. Restate what was found, without a cause. Something differs between the lab literature and the home literature in this pool. That is the whole finding.
2. List what could differ, including the boring options: the interventions, the measures, the participants, the sample sizes, the years, the journals.
3. Ask which split was planned. A subgroup the authors said in advance they would test is worth more than one they found by looking, and a meta-analysis usually says which.
4. Ask what the split would predict next. If lab-and-home is the real divide, the next home study should also find nothing. That is a prediction, and it is the thing that makes the subgroup useful rather than decorative.

Applied to the phone case, step three is the one to notice: the regional split was not a fishing expedition. The authors say they ran it because an earlier study had reported a difference between Asian and European participants, which means it was a check on a prior claim rather than a pattern spotted in their own data.¹ That makes it more interesting and it still doesn't say what differs.

Three things people get wrong about this

"The phone drains your brain even face down." The pooled effect is small, it is in memory rather than attention, and it is absent in the North American samples. The phrase outruns the evidence in both the direction it points and the size it implies.

"The effect was debunked." A significant pooled effect exists and the meta-analysts recommend caution. What failed was one direct replication and the attention subgroup.

"A meta-analysis settles it." This one splits three ways by domain and three ways by region, and its own authors ask for more research. Pooling tells you what the literature currently contains, which is a different thing from telling you what is true.

Practice

Connections

Back. Lesson 4 taught pooling and publication-bias correction, and this lesson assumes both rather than re-teaching them. Lesson 1's instrument question is what makes the domain split readable: memory, attention and general performance are measured by different tasks. Habits and Self-Discipline lesson 5 is where an interval spanning zero was first taught in this Core, and it is used here unchanged.

Forward. Lesson 6 asks whether attention can be trained, which is the last of this course's three contested claims and the one where the evidence is clearest. Lesson 7 makes a habit of everything lessons 4 and 5 did by hand.

Go deeper

• Does the Brain Drain Effect Really Exist? A Meta-Analysis (Behavioral Sciences, 2023). Open access. Read in substantial part by this course: the abstract, the overall result, both subgroup analyses and the discussion. Read the two subgroup tables, which are the whole lesson, and notice that the paper reports them without softening either one.
• "Cognitive Control in Media Multitaskers" Ten Years On: A Meta-Analysis (Cyberpsychology, 2021), open access and read in substantial part by this course. Lesson 4's subject, and worth reading beside this one: two pooled literatures on two famous claims, splitting two different ways, which is how you tell a pattern from a coincidence.

Sources

1. Tobias Böttger, Marius Poschik and Klaus Zierer, "Does the Brain Drain Effect Really Exist? A Meta-Analysis", Behavioral Sciences 13(9), 2023, article 751. Read in part: the abstract, the overall result, both subgroup analyses and the discussion. The full review of included studies was not read. Supports: the quoted count of 22 studies and 43 effects with its three categories; the quoted pooled effect and its interval; the heterogeneity figure; all three domain figures with their intervals; all three regional figures with their intervals; the statement that the Asian subgroup showed no heterogeneity; the statement that the funnel plot showed no obvious asymmetry; the reason the regional split was run; and the quoted closing recommendation from the abstract.
2. This lesson introduces no other pooled evidence. The media-multitasking synthesis belongs to lesson 4 and is cited there.
3. Adrian F. Ward, Kristen Duke, Ayelet Gneezy and Maarten W. Bos, "Brain Drain: The Mere Presence of One's Own Smartphone Reduces Available Cognitive Capacity", Journal of the Association for Consumer Research 2(2), 2017. Search-summary level; not opened, and the body says so where the study is described. Supports only the shape of the original claim: three phone locations, working-memory and attention measures, best performance with the phone in another room, and the name. No figure in this lesson comes from it.
4. Juan Pablo Ruiz Pardo and John Paul Minda, "Reexamining the 'brain drain' effect: A replication of Ward et al. (2017)", Acta Psychologica 230, 2022. Search-summary level; not opened, and the body says so. Supports only: that it was a preregistered direct replication of Ward's second experiment, that it crossed phone location with phone power across six conditions, that it used an operation span task and a go/no-go task, and that phone location made no difference. No figure in this lesson comes from it.
5. The four candidate explanations for the regional split are this course's own reasoning, marked inline where they appear, and the lesson says the authors offer no mechanism and neither does it. The sleep-intervention case is constructed and its numbers are invented, which the body states where it appears, and the four-step reasoning order is the course's own rather than a published procedure.