The Application of Cognitive Psychology to CAD

Andrew Dillon and Marian Sweeney

This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A. and Sweeney, M. (1988) The application of cognitive psychology to CAD. In D. Jones and R.Winder (eds.) People and Computers IV. Cambridge: Cambridge University Press, 477-488.

Abstract

The design of usable human-computer interfaces is one of the primary goals of the HCI specialist. To date however interest has focussed mainly on office or text based systems such as word processors or databases. Computer aided design (CAD) represents a major challenge to the human factors community to provide suitable input and expertise to an area where the users goals and requirements are cognitively distinct from more typical HCI. The present paper is based on psychological investigations of the engineering domain, involving an experimental comparison of designers using CAD and the more traditional drawing board. By employing protocol analytic techniques it is possible to shed light on the complex problem-solving nature of design and to demonstrate the crucial role of human factors in the development of interfaces which facilitate the designers in their task. A model of the cognition of design is proposed which indicates that available knowledge and guidelines alone are not sufficient to aid CAD developers and the distinct nature of the engineering designer's task merits specific attention.

Key words: cognitive psychology, CAD, interface design, protocol analysis

1. Introduction

While a large reservoir of empirical evidence has grown up on the value of certain human computer interface features (see Shneiderman [14] for a detailed review) the concept of usability has become almost inextricably tied-up with the design of text-based office products such as word processors and databases. While these applications represent a significant portion of the IT market for non-specialists, increased technological developments continue to broaden the range of tasks for which computers may offer significant contributions e.g., Computer Aided Design systems (CAD).

Computerisation has aided design in so far as it provides a sophisticated drafting medium, where visualisation of the data is significantly enhanced and where data storage and organisation facilitate analytic procedures such as finite element analysis, weight, stress and mass properties, kinematic analysis and so on.

According to Ullman and Dietterich [16] future development and integration of CAD technology is dependant on an understanding of the engineering design process at the level of the individual and the organisation. The focus of the present paper is to address the cognitive processes involved in mechanical engineering design. The human factors issue here is to develop a model of how designers work and think, in order to determine how CAD technology must progress so as to facilitate those processes.

2. The design task

The term 'mechanical engineering design' is described by Ullman and Dietterich as a creative decision making process for specifying or creating physical devices to fulfil a stated need. This definition gives no indication of how it is actually achieved. It is useful however, to the extent to which it provides a perspective within which to view and interpret the task, i.e., at the level of cognitive processing underlying decision making. Although the literature on the engineering design process is scant, it focuses primarily on a behavioural model of the task (Mostow [10]). It is the cognitive process involved in design which is of interest to the present research.

3. User psychology: the case of designers

The psychology of the CAD user is relatively uncharted territory. While some research has been carried out which attempts to understand the nature of design as a cognitive activity (Whitefield [17] ) our understanding of the users of CAD technology is woefully incomplete. Yet the advances afforded by CAD are arguably among the most important of all IT applications.

In discussions of designer psychology, it is possible to identify two trends. The first of these borrows heavily from the literature on cognitive style (e.g., Tovey [15]), while the second discusses how designers think in terms of the problem-solving strategies employed during task completion (e.g., Lawson [6]).

An individual's cognitive style refers to the characteristic manner in which they respond to and process information. According to Messick et al [9] 19 different cognitive styles have been proposed and investigated. Of these, a few styles have emerged as the major tools for distinguishing between individuals at a cognitive level e.g., field dependence/independence (Witkin et al [18]), holism/serialism (Pask, [12]) and convergence/ divergence (Hudson, [5]). In addition to the more formal cognitive style dimensions outlined, distinctions between left and right brain processing are suggested as a meaningful distinction between individuals (e.g., Tovey [15]).

Given the presumed creative nature of the designer's work it has been suggested, though rarely empirically demonstrated, that designers are field independent (capable of clearly seeing patterns in complex displays), holistic (intuitive, suddenly inspired with a solution) and divergent thinkers.

However, the term designer encompasses a variety of individuals who perform widely varying tasks and it is generally agreed that depending on the nature of the task being performed, a designer may need to employ processes characterised by both extremes of any style dichotomy in order to produce solutions. Furthermore, cognitive style considerations have been investigated with respect to interface design for the more typical "casual user" without any great success ( though see Fowler and Murray [4] for an alternative view) and are unlikely to offer any tremendous insight on their own for the development of CAD interfaces.

The second trend in the literature on the psychology of design attempts to derive the types of problem solving strategies that are commonly employed by designers in their attempts to produce a solution. Lawson's [7] empirical work suggests that designers possess a unique strategy which distinguishes them from scientists. That is, designers tend to have a solution focussed perspective, compared to a scientist's logico-deductive problem focussed reasoning.

Current thought emphasises the "generator-conjecture-analysis" model of design (Darke [1] )which states that designers structure their problems by exploring aspects of possible solutions in an iterative fashion. This is a particularly high-level analysis of designer cognition and by virtue of its generality is severely limited in prescriptive power for the area of interface design. At a lower level there are numerous strategies and sub-strategies that any individual can adopt to solve a problem (Young [19]). Rzevski and Evans [13] list 6 situation dependent strategies that designers employ in order to achieve a design solution: imagination, analogy, heuristics, random search, systematic search and transformation. However, how these are employed is not outlined, information which is essential if any translation into interface guidelines is to be made.

4. Overview of the present study

The present research was carried out as part of a larger collaborative project aimed at developing a user modelling tool for CAD interface design. Our contribution was aimed at analysing the users of these systems with a view to classifying interactive styles and predicting the degree to which individual or group variance would need to be incorporated in the modelling tool. Full details of this work can be found in Dillon et al [2]

The present paper reports on an empirical investigation of designers in situ. The major concern was with increasing the current level of knowledge about the psychology of design beyond speculation around the issues of cognitive style and problem-solving strategy. A secondary concern of the investigation reported here was to shed light on the designers conceptualisation of their task with a view to isolating the relevant human factors issues that need to be addressed for the successful exploitation of CAD technology.

Verbal protocol analysis was selected as the best means of eliciting relevant information for this study. Protocol analytic techniques have been both advocated (Ericcson and Simon [3]) and criticised (Nisbett and Wilson [11]) as a means of gaining insight into an individual's thought processes. Verbal protocol analysis has been used to good effect in the area of user cognition in HCI studies (e.g., Lewis and Mack [8]). Objectivity and reliability are maximised through careful selection of the data that is required ( e.g., concurrent verbal descriptions of what is being attended to are more reliable than retrospective descriptions of how a process was performed) and ensuring that data is scored independently by two or more raters (their combined ratings giving an index of reliability).

4.1. Method

In order to gain insight into the psychology of design, a study involving 16 mechanical engineering designers was carried out at the design offices of British Aerospace (BAe) at Filton. The subjects were made up of two groups, one which carried out the task on a CAD system (n=9) and the other on the more traditional board and t-square (n=7). This ensured that the analysis of the cognitive processes involved in design was not overly influenced by the nature of the drawing interface subjects used. It also enabled designers to make comparative judgements on the effect of technology on the process of design.

The selection of the experimental task was based on two fundamental criteria: firstly it must be realistic i.e., the type of task that the subjects would perform in the course of their normal duties; secondly it had to be possible to complete in 3 hours, so as to be a realistic task. As a result, a task to design a wing-flap mechanism involving a moving carriage was selected with the help of a BAe supervisor. A pilot study revealed that completion time was within the desired range

4.2. Procedure

The subjects were briefed about the aim of the study and each were given an identical task and instructions. A senior designer was on hand throughout, to answer technical questions. The subjects were observed independently and two experimenters sat with each subject and prompted verbal protocols as necessary. After 3 hrs. the session ended. In a post experimental interview each subject was asked to discuss his reactions to the task and views of technology and design in general. All comments were tape-recorded using unobtrusive tie-pin microphones.

4.3. Analysis

A random selection of half of the tape-recorded verbal protocols were examined in order to determine at a global level how designers structured their task. From this, it emerged that the designers' thought processes seemed to chunk themselves primarily about consecutive and often discrete action points. That is, designers displayed a tendency to structure the problem into perceived manageable units and tackle these independently in a series of action cycles. These cycles were invariably initiated and preceded by a phase of decision making where, for example, the designer considered options and task parameters. It was therefore agreed that the protocols best afforded analysis at the level of these action points and their relevant option considerations.

The action cycle was therefore seen as the obvious parsing unit of the protocols while the associated decision making provided a structured means of examining the processing of information during task performance. At this level of operationalisation of designer performance, it was decided that insights could be gained into the designers' cognitive activities by focussing attention more closely on the reasoning underlying their decision making and studying the types of knowledge employed by the designers.

5. Results

A classification of the types of knowledge (found to underlie decisions) referenced in the verbal protocols was constructed. The types of knowledge initially identified included:

  • domain knowledge;
  • task knowledge; and
  • device knowledge;

Domain knowledge relates to the knowledge which a designer has gained through formal training, reading and experience. It encompasses a structured body of knowledge which the designer accesses in order to solve any design problem. It is likewise expanded by new experiences. No difference was observed between subjects on different mediums. That is, both CAD and board designers employed similar domain knowledge at similar milestones in the development of a problem solution.

Task knowledge relates to information which the designer has about the task or about a class to which the task at hand belongs e.g., computer monitor encasement. A designer may successfully tackle a task for which they possess relatively little task knowledge by applying relevant principles from the domain of design in general. The main difference observed between the subjects here, resulted from the users typical work i.e., designers experienced in process design (moving parts) felt more at home with the experimental task than designers who usually worked on structural design.

Device knowledge relates to knowledge which the designer has about the medium on which the task is executed i.e., CAD system or board and t-square. A designer who may be said to have a high level of device knowledge should display a solving procedure which is more concerned with adding elements to and evolving a workable solution. Low levels of device knowledge would be manifest by the expression of concerns with the means by which goals are executed. This was more commonly characteristic of designers working on CAD systems. In the CAD medium this obviously raises important HF issues in terms of the flexibility and transparency of the interface.

6. Discussion

This model of the knowledge underlying activity, provides a useful insight into the overall psychology of the designer. Firstly it seems that the process of design can be understood at a cognitive level as consisting of a series of decision and action phases, resulting from the designer's perception of the problem. It is important to realise that different designers may perceive the problem in different ways and the level at which they break the problem down into managable units was seen to vary in this study.

Secondly, ordering and completion of these phases are controlled by the decision making routines. These decisions are based on three broad knowledge types which collectively contain a variety of rules, procedures, strategies and experiences.

These knowledge types will interact differently depending on their respective proportionality within the designer's knowledge base at any particular time. For example, an experienced board designer who is transferred to CAD will require support with transfer of relevant knowledge, and the acquisition of new device knowledge in order to utilise their task and domain knowledge to the same effect as before. Designers who find themselves working on a new task will utilise the same device knowledge as before but will rely more heavily on domain knowledge as a resource for supplying information pertinent to the task in hand. It is important to note that in terms of the decision-action cycle not all of these were contingent on a rationalisation of the information at hand. Apart from actions resulting from rationalised decision making, others were found to be the result of trial and error or guesswork and others still were the result of personal preference or habit.

In relation to the more traditional literature on the psychology of design, it is not clear what links can be made with the cognitive style school of thought. It is possible that the initial perception of the problem and the subsequent process of task division may correlate with processing trends such as field dependence/independence or serialism/holism. However as this was not a concern of the present investigation, it would be pointless to speculate further. Links with the "strategy" literature are somewhat easier to foster. The protocol data suggest that designers do indeed use strategies such as visualisation and imagination as an aid to problem solving. However it is not clear to what extent a general strategy can represent in any detailed and context-independent way how designers think. Some designers obviously do perform the "conjecture- generator-analysis" routine at certain stages of the design process, but only after they have studied the problem, prioritised their sub-problems and set about the cycle of solution generation.

7. Implications for Interface Design

The most obvious implication of this model for interface design is related to the device knowledge required by a designer to convey a meaningful representation of their ideas to others. The traditional design "interface" of board and t-square appear rather rudimentary in comparison to CAD, but designers have evolved an array of techniques and supplementary tools to aid their powers of visual representation e.g., geometric drawing aids, adjustable set- squares etc. and the board designers in this study demonstrated an impressive range of (mostly automated) skills in drawing and representing complex shapes and relationships.

With regards to the CAD users it was obvious that the system was capable of reducing much of the complexity of the drawing technique to a few supposedly simple keypresses. However the innate logic of the command sequence seemed to cause some users difficulty and comments to the effect that they often used a local expert as a command dictionary were noted. Similarly, movement of parts of the design was often simulated on the board by using cut-outs of the relevant shape and literally moving it around the drawing by hand. This acted as a powerful visual aid to some designers. No directly comparable facility existed on CAD.

A tendency toward a preoccupation with the means to the solution rather than the end was noted with the CAD users. That is to say they could be described as being method oriented as opposed to the board users who were solution oriented. CAD users expressed concerns with being limited in the way they executed the task rather than using the system as a tool. It was also noted that designers working on CAD systems tended to take more risks and hence make more errors. In the final analysis they lost the speed advantage of CAD over board and invariably took the same amount of time to complete the task as the board designers.

Here lie the initial usability issues. The board offers its user direct manipulation facilities that are employed on the back of a life-history of visual representation based on physical movements of the arm and fingers to trace an image. CAD systems on the other hand remove the intrinsic physical relationship between drawer and drawing and replace it with command sequences and joystick/mouse movements. This can prove the source of considerable difficulty for designers as there is often no intuitive link between input and drawing/screen display. While the human is capable of adapting to the situation and experienced CAD users seem to interact effectively with the system, a number of CAD users in this study expressed a preference for the tangibility of the board and stated that in an ideal world a combination of both the efficiency and speed of the CAD system in drafting and the capacity for physical manipulation on the drawing board would be employed.

Obviously research could fruitfully investigate the types of commands and input devices that designers require in order to minimise processing of device knowledge. Much of this work has been carried out by the human factors community in the domain of office systems and the methodological developments gained in that domain are probably applicable here even if the results are not. Advocates of CAD systems list the speed and accuracy of these systems as the major advantages of computerised design. This investigation suggests however that such advantages are likely to be eroded on certain interfaces by increased designer concern with device knowledge when they could be more effectively utilising task and domain knowledge.

Usability issues extend beyond device knowledge however. With respect to the task, CAD systems should be designed to support the user who may lack some task knowledge either through unfamiliarity with the task or forgetfulness etc.. For example, Query-In-Depth help facilities offer a chance to embed contextual information into the system, which the designer consults as and when required. Numerous analysis aids could be made available to the designer to facilitate informed decision making on issues such as stress and tolerance. Given the observed tendency of designers in this study to consider variables which appear beyond the immediate task domain, such facilities would seem a useful method of enhancing the designers cognitive tendencies.

Obviously the development of IKBS offers possibilities to provide the designer with even more powerful information sources. Sophisticated error-trapping routines that identify potential weaknesses in a proposed designs or automatically tailor a design to the stylistic requirements of company policy may be useful. Mass local storage of previous designs on CD-ROM offer the chance to rapidly access a wealth of design data. Developments of this kind may supplement the domain knowledge of designers. However the human factors issues of such advances have no parallels as yet in other areas and it would be difficult to make recommendations for the CAD domain on the basis of current knowledge.

8. Conclusion

The psychology of designers is poorly understood in comparison with other computer users. For the successful exploitation of CAD technology it is important that this is rectified. The present investigation suggests that designers employ three types of knowledge to guide their action sequences and propose a solution. Human factors issues pertain most immediately at the level of device knowledge, where current guidelines and methods have some relevance. However the results of this study indicate that there are at least two other types of knowledge that are employed in the design process that elevate it above the level of a routine cognitive skill. Therefore the usability issues concern the support of a complex, creative task which requires much further investigation in its own right.

Acknowledgements

This work was carried out at HUSAT under the SERC funded Alvey MMI142 Enabling Technology Project.

We would like to thank the other members of the HUSAT team: Val Herring, Enda Fallon and Phil John.

Thanks are also due to British Aerospace for their time and effort, GEC, The Ergonomics Unit at UCL and City University.