Reading a claim about your attention
90 min
Two hosts talk the lesson through. The voices are synthetic; the script was written from this lesson and checked against it, and asserts nothing the lesson does not.
- Sort a claim about attention into measurable, unmeasurable as stated, or false, and say what would settle it
- Identify which instrument a claim rests on, and say what follows from that
- Decide, for one working arrangement of your own, what to change and what to leave, with what each costs
This is the last lesson, and it is the one that stays useful after the rest has dated.
Everything before it was about attention. This one is about the claims you will meet about it, and you will meet a great many, because this is among the most advised-about subjects in the genre and, on what this course found, one of the least settled.
Habit one: ask what would have had to be measured
The institute's standing sort is three categories, and Digital Literacy lesson 8, Using AI Effectively lesson 11, Time Management lesson 8 and Habits and Self-Discipline lesson 7 all use it. Measurable: a study could check it. Unmeasurable as stated: it uses a term nobody has defined. False: it has been checked and did not hold.
This course adds a question in front of that one, and it is the thing lesson 1 opened with: which instrument would have had to produce this claim? The three habits below are this course's own way of organising the job rather than a procedure anybody has published.4
Work one. "People who check their phone constantly are worse at concentrating." Before sorting it, ask what would have been measured. A questionnaire about phone-checking and a questionnaire about concentration? A questionnaire and a task? Two tasks? The claim is compatible with all three and they aren't the same claim, and lesson 4 is a case where two of them give different answers about the same literature.
Once you have asked that, the sort is easier. The sentence as written is measurable, several different ways, and the useful question is which way somebody did it.
Work a second, and this one is a specimen this course has written rather than a sentence it is quoting: an ability to focus without distraction on a cognitively demanding task.4 It is the shape of thing the books in this subject define, and the course has opened none of them, so it is not quoting one. What would be measured? Nothing yet: there is no criterion for when somebody has the ability and when they merely did the task once. Unmeasurable as stated, which isn't the same as useless. It's a definition, and definitions are conventions.
Work a third. "You get about eight seconds of attention before you lose people." What would be measured, and on whom, and compared with what? This is the category where the instrument question does most of its work, because the claim implies a measurement nobody names.
And a fourth kind, which is the commonest. "Turn off notifications, because every interruption costs you twenty-three minutes." That is two claims: a recommendation and a figure. Time Management lesson 5 traced the figure and found no printed source anybody has been able to produce. The recommendation survives the figure being wrong, on lesson 2's evidence about what an interruption actually costs, and a sorter who threw out both has thrown away the half that holds.
Habit two: check whether the claim is about change
This subject has one failure mode of its own, and naming it is this course's own reading rather than a finding.4 It differs from the previous course's.
Habits and Self-Discipline's numbers failed by arriving stripped of their scope. This subject's claims fail by being about change when nothing measured change.
"Attention spans are shrinking." "Nobody can read a book any more." "Your attention span is now shorter than a goldfish's." Every one of those is a claim about something that used to be different, and a claim about change needs the same people, or at least comparable populations, measured twice on the same instrument.
Nothing this course read does that. Every source here is either a single session or a snapshot of different people at one moment. The 2008 experiment ran for ninety minutes. The mind-wandering study sampled people over days. Both syntheses pool studies that measured people once, and one of the two says in terms that its designs are cross-sectional and its causal inferences therefore limited.
So this course's central refusal isn't modesty, it is arithmetic. A body of evidence made entirely of snapshots can't answer a question about change, however many snapshots there are.
Before the next section. Somebody wants to know whether attention spans have really fallen. What study would settle it, and why do you think nobody has run it?
Show the answer
Two parts, and the second one is more interesting than the first.
The study is easy to describe. Take a sample, measure them on a fixed task, wait ten or twenty years, measure the same people again on the same task. Or measure comparable samples at two dates with the identical instrument.
Why nobody has run it is where the lesson is. Three reasons, and this course's own reading is that the third is the binding one.4
The obvious one is duration. A twenty-year study needs funding across careers and a stable research group.
The second is that instruments change. A task written in 2005 is delivered on hardware nobody uses now, and rewriting it breaks the comparison, which is the whole point.
The third is that nobody would agree what to measure. You would have to settle, in advance and for both dates, what "attention span" means as a number. Lesson 1's three descriptions aren't one thing, and a study measuring one of them would be attacked for not measuring another. Committing to a criterion is what makes a claim checkable and it is also what makes it contestable, which is why so many claims in this genre avoid it.
Habit three: separate the report from the general claim
Somebody tells you a technique works for them. You have looked and found no evidence for it.
Those are two claims and almost every argument about this subject is two people disputing different sentences.
"It works for me" is a report about one person, in one situation, usually without a comparison. It is weak evidence about the world and it isn't nothing, and it is the same kind of evidence as the self-report half of lesson 4's literature, which is the half where the associations hold.
"It works" is a general claim and needs the general kind of evidence.
What this lesson asks of you is to say which one you are disputing, out loud, before you disagree. Most of the time the honest sentence is: I believe you about your mornings, and the general claim hasn't been tested.
The three claims, side by side
What this course did to the three best-known claims in its subject. The first two columns are the syntheses'; the third is this course's own reading of what each leaves standing, and the checkpoint below says how loosely to hold it.1234
| The claim | What the synthesis found | What survives |
|---|---|---|
| Media multitasking damages attention | 118 assessments; overall small; performance-based half fails a publication-bias correction | The self-report association, consistently, at z = .200 |
| A phone's mere presence drains cognitive capacity | 22 studies, 43 effects, g = −0.14; memory yes, attention crosses zero; null in North American samples | A small pooled effect in memory, and the authors' recommendation of caution |
| Attention can be trained | 87 publications, 145 comparisons; no convincing far transfer against treated controls | Large near transfer, which isn't what anybody is buying |
Read the middle column and then the right one. In all three cases something survives, and in all three cases it is narrower than the claim it came from. None of the three is a fraud and none of them is what it is sold as, which is the pattern this course most wants you to carry out of it.
And notice what the three have in common. In each case the synthesis is more informative than the original study, and in each case it is the thing nobody has read. The original is famous. The paper that pooled it is where the answer is.
The books you will be handed
You will meet this subject mostly through two very widely read books: Newport's Deep Work and Mark's Attention Span. Csikszentmihalyi's Flow is the trade statement of the state this course is not teaching, which lesson 1 told you it was leaving out.
This course hasn't opened any of them.5 That is on the unread line of its research file and
enforced by the build, and it means this section isn't an assessment of them. No claim anywhere in
these seven lessons rests on any of the three.
One of them is worth a separate note, on the same search-summary basis. Attention Span is by the researcher whose two papers lessons 2 and Time Management lesson 5 are built on, which means its claims can in principle be checked against work with her name on it. That is a note about where a reading would go furthest rather than a recommendation.5
What to do when somebody hands you one is the three habits above, and the middle one especially. A book about attention is very likely to contain a claim about change, and the question to put to it is what was measured twice.
This course has spent seven lessons being careful about what the evidence supports. Apply that to the course. Which of the things you have been told here should you hold most loosely, and which most firmly?
Show the answer
Worth doing, and the ordering isn't the obvious one.
Hold most firmly: the instrument distinction. It isn't a finding, it is a way of reading, and the evidence for its usefulness is lesson 4, where a single literature answers the same question two ways depending on which instrument asked. Nothing that happens to these figures will touch it.
Hold firmly: lesson 2's result, with its scope. Interrupted tasks finished faster, with the cost in stress, frustration, time pressure and effort. One experiment, 48 people, ninety minutes, and the clearest measured thing in the course.
Hold with its hedges attached: lesson 3's causal claim. The authors wrote that time-lag analyses "strongly suggested" wandering was "generally" the cause of unhappiness. Both hedges are theirs and both matter.
Hold loosely: the three-row table above. It is this course's own assembly,4 and the judgement about what survives in each row is the course's reading rather than any synthesis's conclusion.
Hold most loosely: lesson 6's hour. Two of its six elements have evidence behind them, one of those is borrowed from another course, three are labelled judgement and one is labelled unsupported. It is in the course because leaving nothing constructive would have been worse, not because it is established.
Three things people believe about this literature
"Somebody must have checked." Three of this subject's best-known claims were checked, by people who published syntheses that say so plainly, and almost nobody who repeats the claims has read them.
"The science says." There is less science here than the phrase implies and more than in most of the genre. Say which instrument, which synthesis, and which population, and the sentence usually collapses into something narrower and more useful.
"It's obvious that attention has got worse." It may be. Nothing this course read could show it, and a thing being obvious is a reason to want the measurement rather than to skip it.
Practice
Take 20 minutes, and stop at twenty whether or not you have finished.
Find one claim about attention with a number in it. An app's onboarding, a headline, a slide, a book's jacket.
Write it down word for word, with where you found it. Then follow it back, writing each step.
- What does this source cite? If nothing, write "nothing", and you are finished.
- Open what it cites. Is the number there? Not something like it.
- What instrument produced it? A questionnaire, a task, a log, something else.
- Is it a claim about change? If so, what was measured twice?
Write your finding in one sentence. Five endings are all results: the number is there with its conditions; the number is there stripped of them; the trail ends in nothing; the trail leads somewhere you can't get to; or twenty minutes ran out.
Take 30 minutes. This is the last thing the course asks.
Take one working arrangement of yours: where you work, when, with what open, with whom able to reach you. One that you have wondered about.
Three endings, and all three are legitimate. Write whichever you choose in four lines.
One: change it. What exactly, what you expect to happen, how you would notice, and what it costs somebody else. Lesson 6's six lines are the format.
Two: keep it deliberately. Not by default. Say what it gives you that an alternative would not, and what you are accepting in exchange. An arrangement you have examined and kept is a different thing from one you never looked at, and the difference is the point of this course.
Three: stop deciding about it. Write down that you aren't going to change it, that you aren't going to keep reconsidering it, and what would bring it back onto the table. Carrying an undecided question has a price paid every week and buys nothing.
Then one line, whichever you chose: what would have to be true for this to have been the wrong call?
If you can't choose, that is a result too. Write down what you would need to know, and whether it is something you could find out in an afternoon or something nobody has measured.
Connections
Back. This lesson uses the whole course. Lesson 1's instrument question is habit one. Lesson 2 supplies the surviving recommendation in the fourth sort. Lesson 3's hedges are what the checkpoint says to keep attached. Lessons 4, 5 and 6 are the three rows. From earlier Core courses: the sort is Digital Literacy lesson 8's and Using AI Effectively lesson 11's, Time Management lesson 5 traced the twenty-three minutes, and Habits and Self-Discipline lesson 7 is this lesson's nearest relative in the whole institute.
On. The course's test is eighteen items, thirteen of them on people and claims that appear in no lesson. And the project is lesson 6's hour, run for four weeks, with every element of your design labelled evidence or judgement the way that lesson labelled its own.
Out. Memory and Note-Taking and Personal Knowledge are later courses in this same term and aren't written yet. What survives from here, when every figure in the course has dated, is the instrument question, the change question, and the habit of separating a report from a claim.
Go deeper
- "Cognitive Control in Media Multitaskers" Ten Years On: A Meta-Analysis (Cyberpsychology, 2021). Open access, read in substantial part by this course. If you read one thing from this course's list, read its conclusion: two paragraphs that say both halves of a finding without flinching from either.
- Does the Brain Drain Effect Really Exist? Open access, read in substantial part. Lesson 5 works through its two subgroup tables; come back to it here for its closing recommendation, which is a policy sentence written by people whose own attention subgroup crossed zero.
Sources
- Douglas A. Parry and Daniel B. le Roux, "'Cognitive Control in Media Multitaskers' Ten Years On: A Meta-Analysis", Cyberpsychology 15(2), 2021, article 7. Read in part, as lesson 4's Sources entry records. Supports row 1 of the table.
- Tobias Böttger, Marius Poschik and Klaus Zierer, "Does the Brain Drain Effect Really Exist? A Meta-Analysis", Behavioral Sciences 13(9), 2023, article 751. Read in part, as lesson 5's Sources entry records. Supports row 2.
- Monica Melby-Lervåg, Thomas S. Redick and Charles Hulme, "Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of 'Far Transfer'", Perspectives on Psychological Science 11(4), 2016. Abstract read verbatim; the body was not opened, as lesson 6's Sources entry records. Supports row 3.
- The three-row table is this course's own assembly, and the "what survives" column is the course's reading rather than any synthesis's conclusion; the checkpoint says so. The three habits of reading, the argument that this subject's claims fail by being about change, and the three reasons nobody has run the study that would settle it are the course's own reasoning, marked inline where they appear. The decision exercise's form is taken from Habits and Self-Discipline lesson 7, which is this institute's own earlier handling of the same problem.
- Cal Newport, Deep Work; Gloria Mark, Attention Span; Mihaly Csikszentmihalyi, Flow. None
of the three was opened for this course, and all three are on the
unread:line ofresearch/SOURCES.md, which exempts this lesson so that it can name them. They are cited here for nothing. The note about where a reading would go furthest rests on the same search-summary level as everything this course knows about them, and the body says so.
Check your understanding
This lesson has a 6-question quiz. Pass it and the questions come back on a schedule in Review, so what you learned stays learned. Your progress is saved in your browser; no account needed.