Challenges for online reading researchers

Some thoughts re. future challenges and opportunities for researchers interested in online reading that were burbling as I drove to Layton today:

* reading detection without eyetracking
* eyetracking in mobile computer device
* ecologically valid studies of reading for learning; more authentic environments and tasks
** quasi-experimental research of online users in their natural environments engaged in real tasks
* relationship of Csikszentmihalyi’s flow to reading (offline or online)
* usability/ui design for electronic reading (spatial orientation, readability)
* distraction/attention in online reading environments (hyperlinks, media, accessibility of external stimuli, user’s mental models and preconceptions of “online” v. “offline” activity)
* annotation/note-taking interaction (review of current tools)
* permanence of user document metadata (e.g. annotations, note-taking) (permanence, stickiness, context, import/export)
* descriptive research or phenomenology of online reading behavior for learning (students and academics)

Posted in PhD Coursework | Tagged , , , , , , | 2 Comments

Logical arguments of tracking reading

I’ve been thinking about eyetracking as a useful indicator of reading, and how other indicators may be used to track user activity on the web. Many eyetracking studies presume to know when reading occurs via movement patterns. Even accepting the eye-mind hypothesis, Jakob Nielsen reminds us, “By itself, eyetracking can do one thing and one thing only: discover what users look at. (2010, 10)”. And yet our understanding of the automatic and unconscious eye movements in reading is fairly sophisticated, and can even provide prediction of fixation location and saccade lengths based on an analysis of the text itself (this is one way that computational models of reading are tested). So though we have not seen a comprehensive theory of eyetracking reading, there appear to be strong relationships that allow for reliable prediction.

The complexity and automaticity of eye movements when reading is such that it seems unlikely that a human could trick eyetracking into thinking it was reading when it was not. However, I personally have experienced a kind of reading without comprehension, a movement through the text, recognizing the textbase without comprehending, because my mind was on other matters. Friends have described similar phenomenon, when tired, when distracted, when disconnected from the text. I have not examined literature on the question of “reading” without comprehending, and how that might look compared to reading with comprehension; however, I suspect a robust enough algorithm could detect anomalies in oculomotor activity in disconnected reading.

But this particular question is a bit far afield from my original pondering of the logic of the assumptions we make about eye tracking. Thinking through this logic may allow me to then extend these to other possible indicators of reading, such as computer user interactions.

As with all theories, we can not logically verify that reading is occurring by observing user activity; we can only hope to falsify that reading is occurring. However, I am optimistic that through consistent lack of falsification we can come to a reasonable, inductive assumption about when reading is occurring, and substantiate that assumption with assessment of reading comprehension.

Eye tracking falsification syllogism

If one is reading*, one’s eyes will move in “reading patterns”.
Your eyes are not moving in “reading patterns”.
You are not reading*.

*Reading here means the current mode of reading as rauding, where the eye moves through the text fairly consistently and completely while the text remains relatively static. RSVP reading is another matter.

Presuming one had no ability to track oculomotor activity, one would have to rely on other possible indicators, which also can only rely on falsifications. For instance, looking at home PC reading on the web:

If one is reading on the web, …
…the text must remain visible.
…cursor activity will not suggest interest in navigating the site.
…time spent on the page will be no shorter than (formula based on words, difficulty, reading speed).
…windows other than the window containing the text will not gain focus and receive interaction.
…lack of activity on the the page will not reach a “time out” threshold(?).
…scrolling occurs when text extends below “the fold”.

Still not good enough! Even if a method of indicating reading without eyetracking (or other more invasive, lab-based monitoring techniques) may be feasible from a falsification point of view, such a method is still subject to a number of confounding variables. Yet such a method may be substantiated or validated by an assessment of reading comprehension; it may additionally be substantiated by cursor movement if a correlation between eye movement and cursor movement patterns when reading is shown to exist.

Posted in IPT 692R - Research in DE & Blended | Tagged , , , , , | 1 Comment

Foveal vision in reading

As I work my way backwards from eye tracking to the correlation between eye movement and cognition (i.e. to answer the question, how do we know that eye movement means one is reading?) I’ve begun to delve into basic research on eye movements in reading, most recently accessing the influential work “Eye Fixations and Cognitive Processes” by Just and Carpenter (1976). This work goes quite a bit over my head, but coming at it with some background knowledge on eye tracking, and the clear and succinct description of eye movements in reading given by Rayner (2008) in the chapter “Eye Movements in Reading”, I think I understand enough to recognize what Just and Carpenter were trying to achieve.

This brings me to the following illustration, which suggests how the eye can not focus on words outside of a very limited foveal vision area:

This illustration comes from the web site of Dr. Hans Werner Hunziker, and is featured in a Wikipedia article on rauding.

Posted in IPT 692R - Research in DE & Blended, PhD Coursework | Tagged , , , | 2 Comments

Weekend Notes: Eyetracking and Determining Reading

This weeks’ readings took me deeper into eye tracking of reading, as I sought to verify that current research on eye tracking of web site usage is not necessarily related to or relevant for reading research. Nielsen and Pernice’s 2010 book, “Eyetracking Web Usability” is already a classic for web designers, and though Nielsen’s work on the F-shaped scanning pattern of web site users has caused quite a stir amongst critics such as Carr and even educators such as Wear, Nielsen and Pernice are careful to articulate the limitations of their studies to users with “business goals”–quite distinct from learning goals. So this definitely confirmed my sense that (1) we can’t confuse the outcomes of web usability studies with reading or learning research, and (2) there really hasn’t been much work done studying reading for learning behavior on the open web (though the latter still needs to be substantiated).

Another eyetracking paper that rang a bell for me is Campbell and Maglio’s 2001 conference proceedings, “A Robust Algorithm for Reading Detection” from PUI 2001. The work described herein provides a huge stepping stone for me, as Campbell and Maglio not only reinforce the important distinction between “typical” reading (aka rauding, aka deep reading) and skimming, they detail the thresholds by which each can be indicated via eyetracking, and provide the algorithm used to calculate these. I need to follow up to see if they wrote subsequent papers dealing with this topic, or what other work references this paper and substantiates it, but my initial take is their work will be key as I seek to (1) identify when deep reading occurs, and (2) correlate that to computer user input (e.g. mouse tracking). At first I was worried this paper would pre-empt work I want to do, but it merely facilitates it, and will hopefully allow me to focus on different issues.

Finally, I continue to be distracted by the apparent lack of systematic typology for kinds of reading behavior. I’ve traced the term “rauding” to Carver from the 70s, who uses it as a theory of reading that is useful for making predictions about reading. Still, skimming and scanning seem to be the only universally accepted terms for non-reading reading (though Campbell and Maglio in the aforementioned article even break down scanning into more granular categories). When I get a few free hours I’ll plan to sit down and try to map out the different descriptions of reading modes/types/forms/strategies, and seek to understand what distinguishes them from each other.

Posted in IPT 692R - Research in DE & Blended, PhD Coursework | Tagged , , , , , | 1 Comment

Notes on Discussion Reading Tracking Proposal Draft

These are my notes from the in-class discussion of my project proposal first draft. The general conclusion is that the problem needs to be articulated from a methodological gap. An alternative approach is to write this as a stand-alone literature review of reading strategy tracking methods.

Clarity: reading problem or methodological problem? are gaps in research related to reading? or to the methodology?
//it is a methodological problem
(explain research environment) is the problem because its not easy? expensive? ecologically invalid?
articulate more clearly issues of access, time, and cost (scalability) — define and emphasize ecological validity
balance cost and effectiveness

ecological validity: not reading in the natural way(?) because of hardware, hawthorne effect, [in a distracting environment]

has anyone written about the problem of eye tracking set ups on users?

Writing
more sign posting?

literature review needs to call for the technology. why is this technology better than screen capture?

Outline

* set of understood reading behaviors – what is deep reading? how is it different from scanning skimming?
* we need to be able to measure these because (???) reading behaviors for learning are important to study– classifying and importance;
* these have primarily been studied in web design and usability context ? need to study for learning and comprehension contexts ? these haven’t been studied rigorously and extensively in online context ? need to study when user goal is learning vs info finding and task completion —
* here are the available tools & methods
* here are their strengths and limitations (people who have talked about limitations)
** has lack of access to tracking equipment inhibited studies?
** do current methods of tracking allow for ecologically valid studies (e.g. hardware, hawthorne, open web)
* let’s explore an alternative: mouse tracking – cost is lower, ecological validity is higher,
** State assumption/have to argue that eye tracking is accurate indicator of reading, if interface-tracking correlates to eye tracking, it may indicate certain aspects of reading behavior
what about verbal reporting/think alouds?
what about comprehension tests?

methods section –> details of eye tracking and how it will be done!!!

Lit review alternative

? what strategies do people use in online context to get at reading behaviors [skimming, scanning, reading]? (lit review) what are limitations and strengths?

reading behaviors as identified in literature (reading science, usability, L2)
reading for comprehension we call deep reading

To Do

Read literature on reading behaviors/strategies from reading science, L2, web usability
Read literature on tracking methods
Rework as lit review on tracking reading strategies

Posted in IPT 692R - Research in DE & Blended | Tagged , , , | Leave a comment

10-Article Analysis: Rational, theory, questions

I’ve analyzed ten articles on the topic of reading, hypertext, and cognitive load, looking at the articles’ rational, theoretical framework, and research questions. That document is available on Google Docs to invited users.

I found this particular task useful in identifying not only the structure of the articles, but also to narrow focus to these prime motivators. Indeed, this turned out to be more useful than standard annotations, if only because my annotations tend to be overly lengthy and exhaustive.

Posted in IPT 692R - Research in DE & Blended, PhD Coursework | Tagged , , , | Leave a comment

Levy Article on Reading and Attention



Levy, D. M. (1997). I Read the News Today, Oh Boy: Reading and Attention in Digital Libraries. Proceedings of the second ACM international conference on Digital libraries (pp. 202-211). Philadelphia, Pennsylvania.

This paper explores the changing view of attention and reading, contextualizing it in modern technology developments. “…a particular constellation of technology developments and information resources … are making their own unique demands on and shaping our ability to attend. …there is an opportunity to witness — and perhaps, with awareness, to influence — how these new technologies evolve and further shape the attentional dimension of our lives.” Attention is a “highly limited resource” (203) and information superabounds, putting our minds at risk of diffusion and information overload. Choices themselves tax attention. “…there is great value in organizing information resources so that wise and inexpensive choices can be made.”(204) “Libraries and documents have … been shapers of attention…” though we should not forget that there are cultural stereotypes that neglect the diverse reality of reading activities–each with different attentional strategies. In the late 20th century we can say that reading has moved from intensive to extensive and now to expansive or “hyper-extensive”. Levy proposes a framework for conceptualizing reading that orients by asking what, how, where, and why about the attentional activity. Defining a “digital library” can be tricky, as it can be a physical or metaphorical space. The physical space still remains important as a protection from or provider of distraction.

This article nails a lot of the critical concerns re. the future of reading and attention in digital environments. Levy, like all critics of technology’s impact on culture, risks being labeled a technophobe or luddite. The perspective I see in Levy and Carr and others is that we should take control of how technology influences our activity and culture, and not be simply subject to its rapidly changing and contradictory effects. Though the factual information in this article is not foundational for work I hope to do, the perspectives are, and Levy’s ideas suggest research. Indeed, he asks a prime question that I’d love to investigate, “How does place (real, not virtual) affect reading style?” I’m interested in virtual places, too.

Posted in IPT 692R - Research in DE & Blended | Tagged , , , | 1 Comment

DE Research Week 4 Readings

Garrison, D. R. (1997). Computer conferencing: the post-industrial age of distance education. Journal of Open and Distance Learning, 12(2), 3-11.

Recognizes the power of computer conferencing –especially asynchronous online forums–as compared to live, synchronous conversation. This distinguishes a new era of distance education, a post-industrial era. “…computer conferences may be globally accessible but are not mass education.” (10) Proposes cognitive presence and social presence in these environments.

This offers an interesting, historical perspective of the change from independent study to more modern distance learning practices as provided for by the web. One generally salient quote: “Technological world-views must be associated with educational ideals if we are note to be blindly driven by technology.”(9)

Garrison, D., & Arbaugh, J. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10(3), 157-172.


This literature review focuses on the community of inquiry framework and its elements, categories, and indicators. Social presence appears to support learning outcomes and student satisfcation. Cognitive presence may be enhanced “via a variety of online course formats” (162). It is difficulty to move inquiry beyond info exchange or exploration to synthesis and resolution without practical application questions or appropriate tasks, or possibly metacognitive awareness. Teaching presence is shown in numerous studies to be important for “successful online learning” (163) though instructional design, facilitation, and instruction. Finally, numerous gaps in research are identified.

The importance of teaching presence and social presence for positive learning outcomes seems clear. The significance of social presence is more ambiguous, and may not be as clearly related to learning outcomes. I speculate that social presence may motivate students due to shared goals, or may motivate because of reinforcement of feelings of ego gratification or group acceptance and belonging, and may affect learning outcomes because the emotional response–the affective factor that we talk about in language learning–that comes from positive or negative interaction makes information or tasks more memorable.

Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1-2), 5-9.

This “personal perspective” of the development of the Community of Inquiry framework summarizes the originators collaborative work to develop the descriptive and exploratory framework focused on Social Presence, Cognitive Presence, and Teaching Presence. The framework built upon the work of Dewey on social activity, In context of the era, CoI was especially important in recognizing the value of “computer conferening”–asynchronous text-based communication that encourages reflective and critical thought. For much of the 10 years since CoI was developed, research methods centered on quantitative transcript analysis despite its challenges in validity and reliability. A “psychometrically sound CoI instrument” has been recently development to study student perceptions via survey results. The researchers conclude by looking forward to the use of the CoI instrument in blended learning and the framework “as a predictor of learning processes and … outcomes” (9).

This article is helpful in bringing readers up to speed on the short history and current research of CoI. While it doesn’t present new information or provoke questions or research directions, it does make visible the typically unseen developmental process of an important framework and the dedicated collaborative work that has gone into CoI over the years.

Posted in IPT 692R - Research in DE & Blended, PhD Coursework | Tagged , , , , | Leave a comment

Reading in Hypertext Ecologies Draft 1

Based on the exploratory research questions I drafted last week, I’ve composed the following 1p “proposal” on describing reading in hypertext ecologies:

Google Doc: Reading in Hypertext Ecologies Draft 1

Posted in PhD Coursework | Tagged , , , | Leave a comment

Online Reading Research Questions (Take 1)

My area of interest for the Phd in Instructional Psychology and Technology is in online reading behavior in hypertext environments. I’ve been shy to draft real research questions until I was more familiar with the current state of research on this topic. And while I haven’t discovered, let alone read, anywhere near enough to be confident, I’d better try to articulate something before I go any further. Failure to do so will only exacerbate the problem of too many sources in too many directions.

I’ve come up with three possible questions–not unrelated, but, taken together, too much to tackle in a single dissertation. So I need to either trash or combine, if possible. At this stage I’ll appreciate all input, suggestions, questions, or challenges.

Questions

  1. What are the traits of learners’ navigational behavior when reading in authentic hypertext environments? (descriptive: track and analyze user behavior) Or, what is the reading ecology of learners in hypertext environments?
  2. What effect does an authentic hypertext environment have on cognitive load and completion in reading for learning tasks? (experimental: group performance in authentic vs controlled environments)
  3. What does learners’ use of tabbed web browsing in authentic environments suggest about the nature of hypertext, user intention, or cognitive load? (descriptive: track and analyze user behavior re. tabs)

Related questions

  1. How much hypertext is in online course learning materials (9-12? higher ed?) versus comparable outworld, informal learning materials? How much hypertext is internally linked (within site/domain) vs. externally linked (other sites/domains)?
  2. Is the amount of hypertext increasing proportionally to the amount of web pages? I.e. are authors, on average, using more hypertext in their web pages?
  3. If learners spend more time exploring, skimming, scanning than deep reading in hypertext environments, is there a quantifiable gain in breadth of learning?

Preliminary questions

  1. How do we distinguish between different reading task objectives and related behavior? I.e. “deep” reading vs skimming vs scanning
  2. Is there a consensus on the negative impact of hypertext in online reading? Is this clearly different from mere digital reading?
Posted in IPT 692R - Research in DE & Blended, PhD Coursework | Tagged , , , , , | 2 Comments