The Human Factors of journal usage and the design of electronic texts
Andrew Dillon, John Richardson and Cliff McKnight
This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A., Richardson, J. and McKnight, C. (1989) The human factors of journal usage and the design of electronic text. Interacting with Computers, 1(2), 183-189.
The present paper reports on a study of journal usage amongst human factors researchers. The aim of the study was to shed light on how journals are used with a view to making recommendations about the development of a full-text, searchable database that would support such usage. The results indicate that levels of usage vary over time, the range of journals covered is small and readers overlook a large proportion of the contents of articles. Furthermore, three reading strategies are observed which indicate that the presentation of journal articles is not ideally suited to their uses. The implications of these findings for developing suitable computer-based applications are discussed.
Keywords: electronic text, journal usage, hypertext, reading from screens and paper, text manipulation, reading strategies.
Human factors research in the domain of computer presented text has tended to concentrate on possible differences between reading from paper and reading from screen. Early research suggested a speed and accuracy decrement for screen reading (Wright and Lickorish 1984) as well as increased fatigue (Cushman 1986), decreased comprehension (Belmore 1985) and lower preference ratings (Cakir et al. 1980) for computer presented text. Recent research by Gould et al.(1987) has indicated that under optimal conditions -high resolution, black text on white background using anti-aliased characters- the performance deficits could be overcome. Therefore it appears that there is no inherent technological restriction to presenting text on screen (for a detailed review of these issues see Dillon et al. 1988).
Important as this may appear, it is not enough that image quality is improved. Most of the above work involved variants on proof-reading tasks which hardly equate with the type of reading people normally engage in. Furthermore, given a straight choice between screen and paper based texts, readers will almost certainly choose the latter. It is important therefore that screen based presentation exploits the capabilities of technology to support readers and offer them facilities that paper never could. To do this requires a detailed understanding of the nature of the readers' task and their requirements of printed material.
Current work at HUSAT involves the development of a full text, searchable database for use by staff. Given the nature of our work, staff are comparatively frequent users of journals. An in-house library exists to serve immediate needs and the main Loughborough University library on campus acts as a supplement. The proposed database will contain the scanned contents of several years' issues from relevant journals. The present study was motivated by the need to identify pertinent reader characteristics that would affect acceptance or rejection of such a system.
The study takes the form of a task analysis of current reading strategies with paper journals as it is felt that there is much to be gained by understanding current reader practice before developing any electronic alternative. Users of texts have developed a range of strategies and skills for exploring and manipulating paper documents that will have a profound influence on their acceptance of electronic equivalents. Furthermore, as paper documentation is likely to co-exist with electronic text for the foreseeable future, if not forever, these skills and strategies will always remain useful to readers. Technology must be designed to support these skills and strategies, enhancing them where possible, not requiring they be forgotten and new ones learned.
15 subjects took part in the study, all were practising Human Factors researchers currently engaged in a variety of projects.
A pre-task interview collected information relating to frequency of use, range of journals covered, requirements of journals, levels of photocopying and so forth. When the interview ended subjects were presented with a selection of unbound journals from their specified range of interest, asked to imagine this was the first time they had seen these editions and to interact with the journal as normal, articulating what they were attending to as they did so. Two experimenters were present to record protocols. This process was repeated for several journals and the record of the interaction was discussed with the subject to ensure accurate representation of their elicited strategy or usage pattern.
Access rates, range and coverage of journals
Information gathering tends to be problem-driven but is not particularly systematic. There were few mentions of citation indexes, abstracts or on-line searches. Few subjects seemed aware of the range of services offered by the main library.
Two reasons for accessing journals were specified: work-demands (e.g., reviewing literature, rapid familiarisation with a new area) and personal reading (e.g., area of interest, keeping up with new developments). Nine subjects reported accessing journals at least once per week. The highest levels of usage were twice per week (4) and the lowest were once every two months (1).
The range of journals covered was small, four primary journals being cited by eight subjects and another seven journals being cited by two or individual researchers. None of the subjects reported total satisfaction with their coverage of relevant literature and three actually felt that they missed a lot of suitable material.
All subjects felt that the database would be of some use, though more as a supplement to their current journal usage than a replacement. Subjects varied in terms of how far back the contents of the database should go, suggestions ranged from 3 to 8 years (anything over 5 years old was reckoned to be considerably out of date in this area!).
Experimenters' records of subjects' protocols were collated and descriptions of reading strategies derived. The following section discusses the general pattern of journal usage that emerged.
Firstly all subjects look at the table of contents of the issue. A preference was expressed for contents printed on the front or back page which made location of relevant articles possible without opening the journal. At this point readers tend to scan the contents by looking primarily at the titles of papers or the authors. Six subjects reported looking at titles only, four reported looking at all the titles and only then referring to authors and four reported looking at all the authors first and then looking at the titles. Only one subject reported looking at both the title and author of each paper.
If the reader fails to identify anything of interest at this point the journal is put aside and, depending on the circumstances, further journals may be accessed and their contents viewed as above. When an article of interest is identified then the reader opens the journal at the start of the relevant paper. Three subjects reported attending to the author's address in order to gain an impression (e.g. nationality, academic or industrial background and so forth) and it seems likely that early judgements about the probable quality of the paper are occasionally made, rightly or wrongly, on this basis.
The abstract is usually attended to next. However, many of the present sample were very critical of abstracts, describing them as "misleading" or the author's attempt to "sell" the article to the reader. Two subjects actually ignored the abstract and started at the introduction. Only five subjects actually read the abstract fully, the remaining eight all scanned it quickly. At this point a decision may be made about the suitability of the article for the reader's purposes. However, given the poor estimation of abstracts, most readers tended to at least view other parts of the text before fully rejecting an article.
The next phase of reading tends to be a quick scan of the rest of the article. Most subjects reported browsing the start of the introduction at this point before flicking through the article to get a better impression of the contents. At this stage the subjects reported attending to the section headings, scanning the diagrams and tables, noting both the level of mathematical content and the length of the article. Browsing the conclusions seems to be a common method of extracting central ideas from the article and deciding on its worth. Ten subjects reported reading or browsing the conclusions at this point.
References are browsed by some readers in order to further their impressions of the article. This may involve browsing the actual reference list or noting names as they appear in the text. In all, six subjects reported attending to the references at this stage. Doing so was seen as a way of identifying the theoretical perspective of the author and appreciating the relevance of the work to the reader's needs.
By now readers have completed one cycle of interaction with the article and decide whether or not to proceed with it. A number of factors may lead article rejection. The main one is obviously content. The reader by now has a strong impression of the type of material contained in the paper and will be able to make an informed decision on the relevance of it to her needs. How accurate this impression is remains an empirical question. If the article is heavily mathematical it tends to be rejected by the readers in this sample. Poor sectioning, large method and results sections, small discussions and large size in terms of number of pages were all cited as factors that would influence a reader's decision on whether or not to reject an article. The decision to photocopy the article is often made at this point too.
The concept of article size is interesting. Large articles obviously require a significant time-investment which is often seen as a disincentive. However, perceptions of what constituted a large or small article varied e.g., large articles were described as being anything from 6 to more than 30 pages, while small articles were between 3 and 20 pages. Median responses suggest that articles of more than 20 pages are 'large' and those articles that are of about 5 pages are 'small'. Approximately 10 pages is considered to be 'medium' length.
If the article is accepted (or photocopied) for reading it is likely to be subjected to two types of reading strategy. The majority of subjects (10) scan read the article in a non-serial fashion to rapidly extract relevant information. This will involve reading some sections fully and only skimming or even skipping other sections. Typically the method and results sections of experimental papers are skim read while the introduction or introductory sections and the discussion/conclusions are read fully. Readers may highlight points or make notes at this stage.
The second reading strategy is a serial detailed read from start to finish. This was seen as "studying" the article's contents and though not carried out for each article that is selected, 11 subjects reported that they usually read selected articles at this level of detail eventually. Two subjects stated that they rarely did this unless their scanning of the article failed to provide them with the requisite information or the discussion brought up points that required closer reading of the method or results section to fully appreciate. Some subjects (3) expressed a preference for this reading strategy from the outset over scanning though acknowledging it to be less than optimal.
While individual preferences for either strategy were reported most readers seem to use both strategies depending on the task or purpose for reading the article, time available and the content of the article. Original and interesting work is more likely to be read fully than dull or routine papers. Reading to keep up with the literature requires less "studying" of articles than attempting to understand a new area. If reading the article with a view to citing it in one of their own papers, subjects expressed a stronger tendency to read the article fully. However, even when reading at this level of detail some subjects still reported skimming particular sections that were not intrinsically relevant to their particular needs at that time.
Implications for the design of a full-text, searchable database
Merely reproducing the style of the paper version will be of little benefit to the users of any technology designed to support reading. In order to encourage use of such facilities it is important that users can perceive clear advantages for investing time and effort in learning a new system. Obviously, if the database provides rapid access to material not immediately available in the library then it may encourage use. However evidence from users of the innovative Adonis document delivery system (Richardson,1989) show that this may be a necessary but not sufficient condition.
If we note the reading styles outlined above several performance characteristics emerge that would appear important for design purposes. Firstly all readers attend to the Contents page of journals and prefer these to be easily accessible. It would seem therefore that a facility to scan lists of titles and authors would be desirable. These should be grouped as they are on paper i.e., in "issues", but the ability to scan continually should be available.
Secondly, since the full contents of the paper are not attended to immediately, it is better that users are given brief information about the paper and offered the chance of jumping around to various sections of the text. The default mode of article presentation should not be the same as the paper equivalent. A likely presentation style based on the present findings might be: the title of the paper, the author(s), the abstract, a list of section headings that are selectable and the references cited. Further information about the size of the article might also be useful.
Thirdly, rapid browsing facilities are vital. At the initial stage of article browsing fast page-turning is common as readers jump back and forth through the article. The electronic version must support this activity by allowing both serial scrolling of the next page/previous page variety and rapid jumping to particular sections e.g., from the introduction to the method or the discussion. It might be desirable to facilitate jumping to "landmarks" in the text such as tables or figures too. We have implemented a reading interface that will facilitate such interactions and investigations of user performance with, and ratings of, such a system are currently underway.
Fourthly, the ability to print the article at any point would be desirable as obtaining hard copies of selected articles is a major concern of most journal readers. Keeping a record of interesting articles which can be batch printed later may be desirable. Given the observed reading styles of the present sample it might be useful to offer the facility to print sections rather than the full article. For example, readers might choose to print the introduction and discussion sections only. This would have the advantage of reducing costs of obtaining hard copies and save on unnecessary use of paper.
Obviously these are relatively general considerations. It would be wrong to generate specific principles for interface design on the basis of the present study although we have incorporated these findings in the latest version of the system. The optimal structure for electronically presented articles and the most usable way of navigating through this structure are empirical issues which require much detailed research. We are currently addressing these questions in our further research.
It is also likely that the type of interface that would best support the presentation and manipulation of journal articles on screen will be significantly different from the type of interface that would support reading of text books or manuals. As yet, little is known about the user perceived differences between text types and the need for a valid classification of texts has been identified (Brown 1988). Again, this is an issue that we are investigating.
As with all computer based applications, physical access remains a major stumbling block. Virtually all subjects stated that they would like such a database on their desk. Making the effort to use facilities available elsewhere (even if it is only someone else's office in the same building) is seen as a deterrent to use. The facility we are developing at HUSAT will eventually be available on the computer network so users will be able to access it from their desks or offices should they not wish to use the in-house library.
This research was funded by the British Library Research and Development Department and was undertaken as part of Project Quartet.
Belmore, S. (1985) Reading computer presented text. Bulletin of the Psychonomic Society, 23(1), 12-14.
Brown, P.J. (1988) Hypertext: the way forward. In J.C. Van Vliet (ed.) Proceedings of EP88. Cambridge University Press (in press).
Cakir, A., Hart, D. J. & Stewart, T. F. M. (1980) Visual Display Terminals. Chichester: John Wiley and Sons.
Cushman, W. H. (1986) Reading from microfiche, VDT and the printed page: subjective fatigue and performance. Human Factors, 28(1), 63-73.
Dillon, A., McKnight, C. and Richardson, J. (1988) Reading from paper versus reading from screens. The Computer Journal, 31(5), 457-464
Gould, J.D., Alfaro, L., Finn, R., Haupt, B. and Minuto, A.(1987) Reading from CRT displays can be as fast as reading from paper. Human Factors , 26 (5), 497-517.
McKnight, C., Richardson, J. and Dillon, A. (1989) The authoring of hypertext documents. In R. McAleese (ed.) Hypertext: Theory into Practice, Norwood, N.J.: Ablex (in press)
Richardson, J. (1989) The Adonis Document Delivery System: A library trail with end-users. HUSAT Research Centre, Loughborough University of Technology. PQ/LUT/28.
Wright, P. and Lickorish, A. (1983) Proof-reading texts on screen and paper. Behaviour and Information Technology 2 (3), 227-235.