A psychological investigation of researchers' perceptions of texts

Andrew Dillon

This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A. (1990) Accessing information: a psychological analysis of researchers' perceptions of texts. In: M. Feeney and K. Merry (eds.) Information Technology and the Research Process, London: Bowker-Saur, 104-113.

Abstract

An important aspect of any application of new technology is understanding how the proposed users of this technology conceptualise the task domain. In terms of the research process this requires an appreciation of researchers' practices and preferences. Only in this way is it possible to develop technology which will support the researcher. The present paper outlines work carried out within the academic community under the British Library funded Project QUARTET. It reports on research aimed at understanding human interaction with a variety of text-based information sources with a view to specifying user requirements for future technologies. By employing personal construct theory and repertory grid analysis it becomes possible to shed light on researchers' perceptions and uses of current sources of information. Results are presented and the implications of such findings for the development of information storage, retrieval and presentation systems will be discussed.

Introduction

Recent advances in technology have significantly reduced the cost of computer systems and led to increased usage by non-specialists as part of the normal working and leisure activities. In the research domain this is demonstrated by the emergence of increasingly sophisticated on-line search and retrieval systems, document delivery systems, powerful desk-top statistical packages and so forth. While one must doubt the wisdom of enthusiasts predicting the total replacement of paper with electronic text in the next few years (e.g.Jonasson 1982) there is no denying that researchers' interactions with information sources such as textbooks, journals and manuals is being radically altered by IT.

Designing suitable technology for human use has proved to be a more complicated issue than most computer scientists and engineers thought and it is increasingly recognised by computer manufacturers that Human Factors professionals have a valuable contribution to make in their development. The essence of the Human Factors approach is that users have a broad range of skills, strategies, expectations and needs that they bring with them to their interactions with systems and these must be considered and supported in designing any application. The term "user-friendly" might be overworked but it captures the essence of the ergonomic approach.

The present paper reports on work carried out at HUSAT as part of the British Library-funded Project Quartet. This project was set up to investigate how scholarly communication would be affected by advances in technology and as psychologists our remit covered such areas as reading and manipulating electronic text, advanced workstation interfaces and structuring electronic information. In particular the paper details the results of a study looking at researchers' perceptions of text types which was carried out in order to shed light on how electronic versions may best be designed.

Information Technology to support the researcher

Human Factors work in this domain has largely concentrated on issues related to the problems associated with reading from screens. There is some evidence to support the idea that electronic text is slower to read, likely to be read less accurately and is disliked by readers thereby suggesting that electronic versions of most academic texts would not be well received by scholars. However, it transpires that many of the performance differences can be overcome with improved image quality (Gould et al 1987, Dillon et al 1988). There are two problems with this though. Firstly, such high quality screens will not be standard for many years to come and in the meantime users will find themselves faced with sub-optimal technology. This is less serious however than the second problem which is that guidelines relating to information presentation on paper do not readily transfer to screens. In other words merely reproducing the paper format electronically, even on a high-quality screen, will not lead to user acceptance of the system.

These issues have a very real impact on the possible exploitation of IT in the research community. Reporting and archiving of results, presentation of theoretical perspectives and descriptions of work rely heavily on text-based communication channels such as journals, specialist magazines and textbooks. As a group, researchers must rank among the highest users of texts. Indeed without such resources many of the behaviours and activities associated with the research process would not be possible. If providing electronic clones of journals and books for researchers is not good enough (as it now seems) how must they be designed?

The researcher or user is always the best place to start when attempting to answer such a question. Identifying their needs and uses of texts should encourage informed decision making about the design of suitable electronic systems to support their tasks. The present study set out to investigate these issues in the domain of research using a psychological technique known as Repertory Grid Analysis (Kelly 1955), a scaling exercise which requires subjects to consider the similarities and differences they perceive between objects. In this case, texts were provided and researchers required to consider their perceptions of these before quantitatively ordering the texts on self-generated descriptive constructs. The aim was to determine the manner in which researchers construe the information sources they use in order to provide clues to enhancing and supporting their tasks with IT.

Method

Subjects:

6 subjects participated in this study, all were researchers at HUSAT.

Stimulus materials:

Elements (texts) were identical for all subjects and were selected on the basis of likely usage by the sample, as judged by the experimenter and several colleagues. They consisted of 9 texts:

  • a newspaper
  • a manual
  • a text book
  • a novel
  • a journal
  • a catalogue
  • a conference proceedings
  • a magazine
  • a technical report

Such a diversity of texts was chosen in order to provide the best opportunity to elicit a full range of constructs (descriptive terms) pertaining to texts.

Procedure

Subjects were presented with a series of three elements, known as a triad, and asked to think of a way in which two of these were similar and thereby different from the third thus providing a construct on which to rate them. The triads were presented according to a predefined sequence arranged so that no pairings of elements were repeated.

When a meaningful construct was generated the two poles were written on cards and placed either side of a 1-5 rating scale on the desk. Subjects then rated all the texts according to the construct, physically placing texts at some point on this scale according to their perception of its agreement with one or other pole. Once confirmation was obtained that subjects were satisfied with this arrangement the ratings were noted and the next construct elicited.

Results

The results were analysed using the Shaw (1980) FOCUS program. FOCUS involves a two-way cluster analysis that systematically reorders the columns and rows of the data matrices to produce a focused grid with minimal variation between adjacent elements and constructs.

A focused grid for one subject is presented in Fig. 1. The grid consists of the raw ratings made by the subjects with the element list above and the construct list below. The FOCUS program automatically reorders these to give the minimum total distance between contiguous element and construct rating columns. Dendrograms are constructed by joining elements and constructs at their appropriate matching levels.

In Fig. 1. the element dendrogram is on top and the construct dendrogram is to the right of the reordered ratings. The matching levels for both are shown on adjacent scales. The highest match between any two of the N elements or M constructs becomes cluster N +1 or M +1 and so forth until all elements or constructs are matched once by cluster 2N -1 in the case of N elements, and 2M -1 in the case of M constructs.

High matches indicate that the relevant elements share identical or similar ratings on the majority of constructs or the relevant constructs discriminate identically or similarly between the majority of elements. Thus in Fig. 1. it can be observed that elements 5 (journal), 7 (proceedings) and 3 (text book) all match highly providing one cluster and that element 4 (novel) is least similar to the others. Interestingly elements 9 (report) and 2 (manual) are closely matched in this individual's perceptions. Constructs 10 (Work - Leisure) and 1 (To do with I.T. - Not to do with I.T.) offer the highest match among the constructs elicited, while number 7 (Single author selection possible - not possible) is the lowest match. This subject also displays a tendency to rate presence of graphical content with trivial texts (see constructs 5 and 8). By proceeding in this manner it becomes possible to build up a detailed picture of how an individual researcher construes texts.

The analysis was carried out on all subjects' constructs (see Dillon 1989 for full details) and showed a high level of agreement across subjects. In total, 54 different constructs were generated by this sample which produced varying degrees of match. Precise detailing of these matches is beyond the scope of the present paper. Of primary importance for present purposes is the way in which researchers draw distinctions between the texts. Stated simply it seems as if texts are distinguishable on three broad levels:

    How they are read e.g., serially or non-serially, once or repeatedly, browsed or studied in depth etc.

    Why they are read e.g., for professional or personal reasons, to learn or not, out of interest or out of need etc.

    What type of information they contain e.g., technical or non-technical, subject matter, general or specific, textual or graphical etc.

Fig. 1. FOCUSed grid for one subject.

This approach facilitates a classificatory system as in Fig. 2 Here, three texts are distinguished according to their positions relative to the How, Why and What axes. The descriptors study-skim, work-personal and general-specific may vary and are only intended as examples of common constructs employed by the present sample. Other samples may employ very different terms. However, these are still likely to be descriptors that pertain to the attributes How, Why and What.

Obviously there is an individualistic aspect here. The same text may be classified differently by any two readers. Thus a literary scholar is likely to classify novels differently from a casual reader. Both might share similar views of how it is to be read (e.g., serially or in-depth) but differ in their perceptions of why it is read or what information it contains. The scholar will see the novel as related to work while the casual reader is more likely to classify it as a leisure text. What it contains will differ according to the analytic skill of the reader with a scholar viewing e.g., Joyce's Ulysses perhaps as an attempt to undermine contemporary English and the casual reader perhaps seeing it as a powerful, modern work. Neither is wrong; in fact both are correct. Any classification of texts based on psychological criteria must, by definition, allow for such individual differences in perception.

That researchers may vary their classification of texts according to tasks (i.e. within-subject differences) was also apparent from the comments of subjects in this study. Several of them remarked that some texts could possibly be classed as work-related and personal reading depending on the situation. An obvious example of this occurs when someone reads an academic article that is both relevant to one's work and intrinsically interesting in its own right. For individuals whose professional and personal interests overlap such an occurrence was common. The present categorisation of texts allows for this by placing emphasis on the motivation for utilising the text (the Why axis).

Fig. 2. A three-way classification of texts based on How, Why and What attributes.

Implications for IT systems

How does all this relate to the development of IT systems? Obviously the desire for a stable classification of texts that could have their electronic equivalents empirically decided upon and standardised remains unfulfilled. In fact, this study suggests that such a typology is probably not feasible. Theoretically, at least, one could seek to determine the complete membership of each attribute set by identifying all possible Hows, Whys and Whats and subsequently plot the range of texts that match any combination of the three. However, such a level of analysis is probably too fine-grained to be worth pursuing. Even if, for example, all possible reading styles and strategies were identified and documented (no mean feat in itself) thereby specifying the complete set of How variables, it is not at all clear how a similar approach could be usefully employed with the Why and What aspects.

The classification that is proposed in this paper is best seen as a simple representation of the factors influencing researchers' perceptions of relevant texts. Characterising text according to How, Why and What variables provides a useful means of understanding the manner in which a given readership is likely to respond to a text. At a gross level, this classification may serve to guide decisions about the feasibility of developing a usable electronic version of a text type. Where the likely readership is known this can act as a stimulus to meaningful task analysis to identify how best to design such texts. Indeed this approach has been responsible for the work reported in Dillon et al (1988) where Human Factors professionals' requirements of journals were analysed in order to arrive at informed decisions about how best to design a database of articles (see McKnight 1989 for initial description). It may highlight situations where an electronic version would prove unsuitable or where one may only be useful for certain activities associated with that text. For example, from what we know about reading from screens we can see that any text that requires lengthy serial reading (e.g. a novel) is unlikely to prove an acceptable candidate for electronic presentation. However, this same text, when considered in the light of other task variables (e.g. rapid retrieval and comparison of the literary style in certain sections) may be usefully presented electronically. Structuring information gathering so that data concerning each of these attributes are considered ensures a balanced view of a system's utility to relevant users.

Conclusion

For IT applications to have the desired impact on the research process it is vital that they are designed in accordance with the needs of the target users: researchers. The present paper has shown that typical researchers have strong views about texts and their uses of them which can and should be considered before any system is designed.

Acknowledgement

This work was funded by the British Library Research and Development Department under Project QUARTET.

References

Dillon, A. (1989) Towards a classification of text types: a repertory grid approach. HUSAT Memo, Dept. of Human Sciences, Loughborough University of Technology, Leics.

Dillon, A, McKnight, C. and Richardson, J. (1988) Reading from paper versus reading from screen. The Computer Journal 31, (5), 457-464.

Dillon, A., Richardson, J. and McKnight, C. (1988) Towards the design of a full-text, searchable database: implications from a study of journal usage. British Journal of Academic Librarianship 3, (1), 37-48.

Gould, J.D., Alfaro, L., Finn, R., Haupt, B. and Minuto, A. (1987) Reading from CRT displays can be as fast as reading from paper. Human Factors 26, (5), 497-517.

Jonasson, D. (1982) (Ed) The Technology of Text. Englewood Cliffs NJ: Educational Technology Publications

Kelly, G. (1955) The Psychology of Personal Constructs, 2 Vols. New York: Norton

McKnight C (1989) The development and testing of a Hypertext database this volume

Shaw, M.L. (1980) On becoming a personal scientist. London: Academic Press