Reading a research paper

A research paper is the most useful genre this course will teach you to open, and the most widely avoided. It's also the genre where reading in the order the words are printed does the most damage.

This lesson is about the order to read one in instead.

Why a paper is not read in its own order

A paper is written in a fixed order, and the order is archival rather than pedagogical. It exists so that a specialist can find a particular thing in a particular place, and so that the record is complete.

That order postpones every question you actually have.

The claim lives in the abstract and the conclusions. The evidence lives in the figures and the results. And whether the evidence can support the claim is decided in the methods, which sit in the middle and which the abstract compresses to a clause at most. Read front to back and you meet the claim first, with no way to weigh it, then a long stretch you can't yet see the point of, and then the discussion, by which time the claim has had half an hour or more to settle in your head unopposed.

There is a second cost, and it is the one that makes people give up. Reading front to back means deciding whether the paper is worth your time only after you have spent the time.

Three passes

The method here is S. Keshav's, from a three-page note written in 2007 for computer scientists and useful well outside it.¹ It has three passes, each with a budget and a defined output, and the first one is where most of the value is.

The first pass, five to ten minutes. Read the title, the abstract and the introduction carefully. Read the section and sub-section headings, and nothing under them. Read the conclusions. Glance over the references, marking any you have already read.¹

At the end you should be able to answer five questions, which Keshav calls the five Cs.¹

• Category. What kind of paper is this? A measurement study, an analysis of an existing system, a description of a new one?
• Context. What other work is it standing on, and what body of theory?
• Correctness. Do the assumptions look sound?
• Contributions. What are the paper's main contributions?
• Clarity. Is it well written?

And then the step that makes the budget worth keeping: you may stop. Keshav gives three reasons you might: the paper doesn't interest you, you do not know enough to follow it, or the assumptions look invalid.¹ In my own use the answer is usually to stop, and finding that out in eight minutes is the point.

The second pass, up to an hour. Read with care, but ignore proofs. Look hard at the figures, and Keshav is specific about what to look at: are the axes properly labelled, are error bars shown? "Common mistakes like these will separate rushed, shoddy work from the truly excellent."¹ Mark the references you have not read, for later.

The output is a test: afterwards "you should be able to summarize the main thrust of the paper, with supporting evidence, to someone else".¹ Not the topic. The thrust, with the evidence attached.

The third pass. Keshav's third pass is to "attempt to virtually re-implement the paper": make the same assumptions the authors made and re-create the work, which is how you find the assumptions they didn't state.¹ It takes hours, it is for papers in your own field, and most readers never need it.

Two honest notes from elsewhere, because this can sound tidier than it is. Carey, Steiner and Petri, writing for people who are not yet fluent, say there's "no correct or incorrect approach" to whether you work through the figures first or read the results text alongside them, and that "early on, it can take a long time to read one article front to back".² The three passes are a way of spending that time deliberately rather than a promise that it will be short.

A first pass, live

Here is a first pass on a real paper, and I have chosen one this course leans on so that you can check our own sourcing while you learn the method. It is Marc Brysbaert's 2019 meta-analysis of reading rate, the paper lesson 4 is built on, and it is free as a PDF.³

Title. "How many words do we read per minute? A review and meta-analysis of reading rate." A question and a method, which is already two of the five Cs half answered.

Abstract. 190 studies, 18,573 participants, 238 words a minute for non-fiction and 260 for fiction, oral reading 183 from 77 studies. The estimates are "lower than the numbers often cited in scientific and popular writings" and "the reasons for the overestimates are reviewed". Ranges given: 175 to 300 for non-fiction, 200 to 320 for fiction.³

Introduction. Opens on Rayner and colleagues' review of speed reading and why more than a thousand words a minute is impossible without severe loss of understanding, then narrows to the ordinary question: how fast do we read normally? It states the article's job in one sentence, which is to discuss "how the number came about and how well it is supported by the available data".³

Headings, and nothing under them. Nineteen before the back matter, counting the introduction you have just read, and the count is itself information. The eighteen after it, in order: the origins of 300 words per minute; hypotheses of why reading is faster than listening; some problems with the estimate of 300 wpm; a meta-analysis of reading rates in silent reading; a meta-analysis of oral reading rates; fiction reading as a critical test of Carver's theory; a new fiction book reading study; reading rates in the wild; reading rate and text difficulty; life span differences in reading rate; individual differences in reading rate; maximum reading rates; reading rate for text recall; reading rates in different languages; reading rates in non-native speakers; general discussion; recommendations; remaining issues. Data availability, acknowledgements, supplementary material and references follow and aren't part of the argument.³

Under "the origins of 300 words per minute" sit named sub-sections, and two of them are just surnames: Tinker, and Carver. A heading that is a bare surname tells you the paper is going to spend a section on what that person claimed, and a paper rarely gives a section to a claim it simply accepts.

Stop there and look at what the headings alone have told you. Two of the eighteen begin "a meta-analysis of", so the pooling is the spine. One says "a new fiction book reading study", so the author has run something of his own and set it beside the pooled data. One names Carver, so there is a theory being tested rather than only a number being estimated. That's three kinds of paper in one, and you have it without reading a word of the body. What the headings will not tell you is how big the new study is. Note the question and carry it.

Conclusions. There is no section called Conclusions, which is common. The General discussion does that job, and its three sub-headings are written as whole sentences, so they are the conclusions: "Normal silent reading rate in English is 238 wpm for non-fiction and 260 wpm for fiction"; "There is no evidence for reading gears except for the distinction between reading and scanning"; "There is no need for different language processes in reading and listening".³ That second one is the rejection of a theory, and lesson 4 rests on it. Sub-headings written as claims are a gift, and the first pass gets them for nothing.

Recommendations. Three of them, addressed to researchers, the first being that "many articles do not contain enough information to calculate reading rate", particularly in eye movement papers.³ The gaps the analysis leaves open are in the section after it, which is headed "Remaining issues".

References. Glance and you find the names lesson 4 also cites: Rayner and colleagues, Carver, Taylor.

That took eight minutes and it is enough to decide three things: whether to read the paper, what to cite it for, and what you would have to check before citing it for anything else.

The second pass, and what it is for

The second pass is where you find out whether the claim is carried by the evidence, and it happens mostly in two places.

The figures. Keshav's questions are the right ones and they are quick: are the axes labelled, with units? Are error bars shown? Does the figure show the data or a model fitted to the data? A figure with unlabelled axes in a published paper is a fact about the paper, not about you.

The methods. This is the section that decides what the evidence can support, and it's the section the abstract gives you least of. An abstract will usually tell you the sample size. It will not tell you who the sample were, what they were actually asked to do, or what they were compared against. How many participants, and who were they? What were they asked to do, and is that the thing the paper claims to be about? Was there a comparison group, and against what?

When the second pass contradicts the first

Here is the case the whole method is for, and this course has one ready-made.

Lesson 3 met Recht and Leslie's 1988 study of sixty-four junior-high students, in which poor readers who knew baseball understood a passage about baseball as well as good readers who did not. First-pass that study, from its abstract and the accounts of it, and you get a clean, striking result about the power of prior knowledge.⁴

In 2025 Reynolds, Hattan and Markham went back to nineteen studies of that kind and looked at what a first pass skips: how baseball knowledge was measured, and what the comprehension passage was actually like. They found that thirteen of the nineteen used the same two knowledge measures, that those measures "focused heavily on vocabulary and baseball trivia", and that "the most common baseball comprehension text was deceptively complex".⁴

And the second matters most, because a passage that is harder than it looks puts text difficulty back inside the very comparison the study was built to hold still. Lesson 3 works that through; here the point is where the finding had to come from.

That is not a story about a bad study. It's a story about what a first pass can and cannot tell you, and about a result that got repeated for nearly forty years by people who had done the first pass and stopped.

It is also not a story about a claim collapsing. The reviewers open by granting that links between knowledge and comprehension "have been widely documented for decades", and close by asking the field to rest on non-baseball work instead.⁴ Lesson 3 works that through. What was weakened is a demonstration, not the thing demonstrated.

One more thing about where findings live. Both charges are in the review's own abstract, so a first pass on the review gets you all of it in ninety seconds. That's the pattern worth keeping: the first pass is cheap on the paper in front of you, and what it cannot do is reach the paper behind it.

It is also a story about this course. Everything lesson 3 says about Recht and Leslie comes from an abstract and from secondary accounts, and lesson 3 says so in its own body. We haven't done the second pass on it either. What we have done is read somebody else's, which is the next best thing and is not the same thing.

What people get wrong

"I cannot read papers, I am not a scientist." The first pass doesn't require you to follow the statistics or the proofs. It requires you to read five parts of the paper and answer five questions, and the five questions are about what kind of thing you are holding. That is a general reading skill and you have been building it since lesson 2.

"The abstract is the paper." An abstract is a compressed advertisement for the paper, written by the authors, usually last. In my experience it is where a claim most often outruns its evidence, because compression is what drops the conditions.

Lesson 4's screen-reading finding is the counter-example worth holding. Delgado's abstract names all three of its moderators and says which way each one runs, which is why lesson 4 could state the conditions on that finding without opening the paper. Most abstracts aren't that good. The test is whether the abstract's headline sentence has the same scope as the sentence in the results, and you will not know until you look.

"Peer reviewed means true." In these fields it usually means two or three people who work in the area read it, wrote down their objections, and an editor judged that the authors had answered them. That is a real filter. It's usually not a check on the data, because reviewers mostly work from the manuscript, and the literature is full of peer-reviewed papers that later work has overturned. Peer review is the reason you can start from a paper rather than from a blog post. It is not a reason to stop reading at the abstract.

"I should understand every sentence before going on." Lesson 2's rule about a first pass through a hard book applies here with more force, because a paper's hardest sentences are often in a section you do not need. Mark it, go on, and see whether the paper turns out to be one you are reading twice.

"The discussion tells me what was found." The results tell you what was found. The discussion tells you what the authors make of it, which is a different thing and is where a correlation most often becomes a cause. When the two disagree, the results win.

Practice

Connections

Lesson 2 gave you the survey, and the first pass is the same idea in a genre with a fixed shape. A book's structure has to be discovered; a paper's is given, which is why the first pass can be budgeted in minutes.

Lesson 3 gave you three obstacles, and on a paper outside your field the usual one is the second: not the words, and not the syntax, but the relations a specialist supplies without noticing. That is why the fix is a review article or a textbook chapter rather than a slower rereading.

Lesson 6's marks are what a second pass produces. The ? on a methods sentence and the X beside a figure with no error bars are the record you will summarise from.

Lesson 7 is the judgement, and the four grounds do real work here, including by not applying. A paper whose abstract claims more than its design can carry isn't a disagreement, and it is not one of the four grounds either: it is the objection lesson 7 sorts onto rule 11, that the support doesn't carry the claim, and what you owe is the narrower claim written down. Where the design instead leaves a question its own subject raises unanswered, that is the fourth ground, and the verdict is agreement on what was argued, suspended judgement on the whole, and the gap named.

Lesson 10 needs this, because reading several papers against one question means first-passing many and second-passing few.

Go deeper

• S. Keshav, "How to Read a Paper". Three pages, free, and the source of everything above about the three passes. Read it; it takes ten minutes and it is the shortest useful thing in this course's bibliography.
• Carey, Steiner and Petri, "Ten simple rules for reading a scientific paper". Open access, peer reviewed, and written for people who aren't yet fluent. Its rule on unpacking every figure is the best short treatment of that I found.
• Brysbaert's reading-rate meta-analysis. Free, and the paper first-passed above, so you can check the pass against the paper.

Sources

1. S. Keshav, "How to Read a Paper", University of Waterloo, 2007, widely circulated in ACM SIGCOMM Computer Communication Review, full text. Read in full. The three-pass method and its budgets; the five Cs, named and defined; the instruction to read headings and ignore what is under them; the figure questions about axes and error bars and the sentence about what separates rushed work from the excellent; the second pass's test, that you can summarise the main thrust with supporting evidence to someone else; and the third pass as an attempt to virtually re-implement the paper.
2. Michaela Carey, Kevin Steiner and William Petri, "Ten simple rules for reading a scientific paper", PLoS Computational Biology 16(7), 2020, e1008032, open access. Read in full. Quoted here for its two honest notes, that there is "no correct or incorrect approach" to whether the figures or the results text come first, and that "early on, it can take a long time to read one article front to back".
3. Marc Brysbaert, "How many words do we read per minute? A review and meta-analysis of reading rate", Journal of Memory and Language 109 (2019), 104047, full text. Read in substantial part, as lesson 4 records, and for this lesson the full heading structure and the Recommendations and Remaining issues sections as well. The first pass above was done on that PDF and every element of it is checkable there: the title, the abstract's figures and its sentence about the numbers often cited, the introduction's statement of the article's job, the nineteen top-level headings before the back matter, in order, and the sub-headings named, the three declarative sub-headings of the General discussion quoted verbatim, and the Recommendations section's first recommendation about articles that do not report enough to calculate a rate. The heading list was read off the PDF's own embedded outline by font, because a first extraction from the two-column text gave seven headings where the paper has nineteen and mixed two heading levels, which is a mistake this lesson invited a reader to catch.
4. Donna R. Recht and Lauren Leslie, "Effect of Prior Knowledge on Good and Poor Readers' Memory of Text", Journal of Educational Psychology 80(1), 1988, and Dan Reynolds, Courtney Hattan and Marissa Markham, "Fair or Foul? Interrogating the Role of Baseball Knowledge in Studies of Knowledge and Comprehension", Reading Research Quarterly 60 (2025), e575. Both as lesson 3 uses them, and both at the read depths lesson 3 records: Recht and Leslie at abstract and secondary level, Reynolds and colleagues at abstract level, with every quoted phrase taken from that abstract.