Jared Stein's grad-school-community blog on teaching and learning.

I’ve begun a harvest of research literature concerned with online (hypertext) reading and cognitive load. I initially ordered my reading list by discovery, but, a half dozen articles later, wish I had collected broadly, first, and began moving backwards from more recent publications.

The following three articles are from the 90s, concerned primarily with comparison of measures of online or hypertext reading with more understood practices in print reading.



Dillon, A. (1992). Reading from paper versus screens: a critical review of the empirical literature. Ergonomics, 35(10), 1297-1326. doi: 10.1080/00140139208967394.

This literature review examines differences between paper and digital document reading through the early 90s. Predictions about online reading had not, by 1992, come to full fruition, though the desire to “make it great” persists. Dillon makes a clear distinction between research focused on outcomes versus processes of reading. Outcomes: accuracy (favors hypertext), speed (may favor paper), comprehension (no difference, or favors hypertext), fatigue (mixed or no difference). Process: eye movements (no gross difference), manipulation (very different, favors paper, but may end up favoring hypertext), navigation (favors linear, print, but effects are mixed or circumstantial). Basic ergonomic issues are explored, including presentation and user characteristics. Manipulation facilities issues were also explored, including display size, scrolling vs. paging, and input devices. The conclusion notes that most experimental work has focused on outcome measures, probably as a means of accounting for reading speed differences. Future studies need to address less limited outcome variables than speed and accuracy. Researchers should recognize that “the ecological validity of many of these studies is low”, and ergonomists should expand their views of reading. “if our desire is to create systems that improve on paper rather than just matching it in performance and satisfaction terms (as it should be) then much more work and a more realistic conceptualisation of human reading is required.

This exhaustive review is nice to have, though I expect a number of the studies’ conclusions may have been affected by users experience reading online, or the quality of available technology. It would be interesting to see some of these studies replicated again, though with Dillons’ concluding warnings taken into consideration. Ultimately, though, we must consider that reading primarily online is now the norm, despite any persistent deficiencies, but that other issues of cognitive load or distraction may now take precedence over matters of simple media comparison.

As a side note, it seems almost quaint now that Dillon writes, “the millions of books that exist now will not all find their way into electronic form”. Thank you, Google Books.

Wenger, M. J., & Payne, D. G. (1996). Human Information Processing Correlates of Reading Hypertext. Technical Communication, 43(1), 51-60. Technical Communication.

Past research exists that relates basic cognitive abilities to reading abilities. General performance measures included working memory span, lexical decision tasks, and choice reaction. The researchers hypothesized that letter matching tasks and spatial processing tasks would additionally be predictive of hypertext reading ability. This research project focused on reading outcomes, and sought to “determine whether cognitive abilities that are predictive of performance while reading linear text are also predictive of performance while reading hypertext” (52). Performance is measured while and after reading hypertext. Performance is defined as speed (wpm), total number of nodes, and number of repeated nodes. The research method was “a correlational study of hypertext reading performance on a set of information processing tasks”, and included 22 native English students (54). Measures of reading performance (recall, comprehension, text structure memory, rate, node measures as above) were compared to the five info processing abilities. Three levels of reliability: p < .1, p < .05, p < .01; "relationships between the information processing measures and the hypertext reading measures replicate those documented" in normal text (57-58), and "lexical decision performance was most strongly related to hypertext reading performance when the stimuli were words" (58). Also, they found a relationship between spatial processing task performance and hypertext reading, both in terms of speed and accuracy. This suggests "knowledge acquired in basic research into reading processes can be applied to optimizing the design of hypertext information" (59), and that relational processing may be "of critical importance in reading hypertext".

This research has implications for future research on hypertext reading behavior, allowing researchers to predict reading performance by cognitive ability tasks, as well as for hypertext document/environment design. I will likely come back to this idea of measuring cognitive ability as my studies advance. I've recently found a 2007 article titled, Cognitive load in hypertext reading: A review, which should give a nice retrospective look back over the last 20 years.

O’Hara, K., & Sellen, A. (1997). A comparison of reading paper and on-line documents. CHI ’97 (pp. 335 – 347). Retrieved from http://portal.acm.org/citation.cfm?id=258549.258787.

This descriptive study did not look at the cognitive processes of reading, but rather at reading-supportive activities and behaviors in print and online documents. The researchers state that this study is a process, as opposed to outcomes, study, and cite a need to carefully design reading experiments to be “ecologically valid”, “representative of real world tasks”(336). The research subjects were 10 volunteers randomly assigned into two groups, the paper readers and the online readers. Both were charged with reading an article, taking notes, then writing a summary. Selected findings tended to focus on the subjects’ own responses take from interviews. Annotations proved popular, easier, and were geographically tied to the source passage for the paper group, though the online group was able to quickly copy and paste whole passages into their notes. Movement through documents again favored paper, which offered speed and automaticity. It also accommodated use of both hands for both efficiency and navigation/place-marking, and the physical presence of the paper itself to create explicit cues. This latter aspect was represented especially well in the researchers look at spatial layout, where they observed subjects arranging documents, notes, and summary sheet for logic and efficiency. Such physical arrangement was not as easy in the digital environment, though windows were available and the screen resolution was, for the time, rather high. The researchers conclude primarily that designers should “look to paper for improving the design of digital reading technologies” (340), recognizing the importance of annotation, quick and automatic navigation, and spatial layout support.

I was disappointed not to find the outcomes of the summaries examined, though the researchers did claim this was a process oriented descriptive study, as the importance of the process observations may hinge on their relative value against the subject’s performance. The implications of this study are primarily for online document design, for the researchers note that paper still offers key advantages in terms of quick navigation and flexibility of spacial layout. Reporting on annotation behavior, the researchers write, “reading was interspersed with long period of editing, or note-taking was done with little reference back to the source document”, but we are to wonder what effect this has on the summaries. I wonder if this behavior carried through into the 21c, and what impact it has on outcomes.Many, but not all, of the deficiencies of the online group may have been overcome with higher resolution, portable computers, modern online document tools such as PDF viewers, social bookmarking tools such as Diigo, and even major word processors. The observations on spatial layout reminded me of a previous usability design case that referenced users reliance on a geography of the text that was often absent or obscured in online texts. (I may have a ref for this in Zotero that I haven’t migrated yet.)There remain numerous deficiencies in online document presentation, annotation, and navigation that make paper an alluring option to many. To explore this further, I look forward to reading Ackerman and Goldsmith’s 2008 article, Learning Directly From Screen? Oh-No, I Must Print It! Metacognitive Analysis of Digitally Presented Text Learning.