Book title: Design Rationale: Concepts, Techniques, and Use
Lawrence Erlbaum Associates, 1996, ISBN: 0-8058-1566-X
This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A. (1997) Review of Carroll and Moran (eds). Design Rationale. Journal of the American Society for Information Science, 48(8) 762-763.
The concept of design rationale has emerged over the last decade as attention focused on the reasons and criteria underlying any application’s design and development. In general terms, a design rationale is a description of the development of an artifact that goes beyond the record of its specification and testing, for example, to include details of why it was designed to operate one way rather than another, what trade-offs were made, what motivated the design team to include certain features or to drop others, or what mistakes or errors were made along the way. Growing out of the field of human-computer interaction’s (HCI) regular concern with understanding designers in order to influence their designs, Design Rationale (the book) could be seen as the culmination of efforts on the part of the HCI community to apply user-centeredness to designers. By focusing on the manner in which design proceeds, and the reasons for any one artifact being designed the way it is, the hope is design rationales can inform our decision making both in designing new information technologies and training future designers.
The present volume is the latest entry in the LEA Computers, Cognition and Work series, and it contains 16 chapters by some of the major thinkers in the field. At nearly 500 pages it is a large text that is divided into 5 major sections developing a bottom-up line of reasoning from details of certain notations for design rationale, to capturing organizational perspectives on designs. Though logical and systematic at the high level, individual chapters within sections deviate from the theme considerably, and six of the chapters have already been published in a special issue of the journal Human-Computer Interaction in 1991 giving the distinct impression that influence between authors was not reciprocal.
As Moran and Carroll put it in their introductory overview, design is the process of making the tangible out of the intangible. As it is classically understood, design is an ill-structured problem solving process, much rooted in craft-based (as opposed to scientific) knowledge. Design rationale is proposed as a means of exploring this process in a relatively formal manner (assuming rationales always existed informally). The logic seems to be as follows: if we identify how artifacts are created, and the reasons for implementing an interface one way rather than the other, over time such knowledge should provide us with insights into how we might design better. Put so simply, this is likely true, but a more than cursory glance at the notion of rationale immediately raises the questions of just what might we capture, and to what use should we put it?
It might seem that there is little enough known about the use and value of rationales to produce such a text yet but the editors raise 13 research issues, ranging from how we make explicit for capture certain management as opposed to technical forces at work in design, to whether or not capturing rationale alters the nature of the design process itself? All fascinating questions, and no doubt a fruitful source of doctoral dissertation topics for years to come, but only some of these questions are (or perhaps even could be) directly tackled by the various authors in the text. The evidence suggests that the concept of design rationale is open enough to interpretation to prevent a cohesive set of issues being formalized. This is not a criticism, it is an indication of the contents of the volume, and readers should be aware of this as they approach the text. It weaves a wide tapestry, frequently appearing repetitive (do so many of the authors need to put their own spin on the concept of design?) and the editors have indulged authors who want to tell us how their chapters evolved before they tell us what their chapters contain.
The text kicks off seriously with Lee and Lai discussing the various ways in which design rationale may be represented, placing emphasis on their own complex Decision Representation Language which I found unnecessarily cumbersome but which the authors argue is more natural in use than it may sound. While they make a strong case for the representative power of their language compared to other forms of rationale description, few of the other authors pick up on their work, and the best case of all is never presented - evidence that a design team has used this language to document a rationale. For a chapter originally written in 1991, and for which the only updated references are other chapters in the present volume, one can only conclude that such a test seems less pressing for the authors.
The tackling of more concrete concerns, such as what a rationale may look like in practice, is handled neatly in MacLean et al’s chapter that immediately follows. Put simply, they argue that the essential components of their design rationale (QOC) are Questions (the key design issues being considered), Options (the alternatives that are presented for answering the questions), and Criteria (the relative assessment issues for weighing options). At once, a notation of design rationale becomes clear even if the designers they show using this notation seem to find it less useful or intuitive than the authors do (a point neatly demonstrated in Buckingham Shum’s chapter in which he examines the usability of this notation with intended users).
Carroll and Rosson view the use of rationale as a means of developing a contextualised science base for HCI that can influence real-world design practice. These authors have argued this point for several years, and they themselves can tie their efforts back to the pioneering work of Francis Bacon in the 17th century. Theirs is perhaps the most ambitious chapter of the present volume, going beyond a concern with representing rationale to advocating its application to the advancement of the state of the art in HCI. This is powerful work that is not without its critics but I feel its importance is likely to be lost on some readers as the chapter is dominated by examples from a SmallTalk environment that these authors have described in detail elsewhere.
The empirical chapters are generally not too encouraging about the immediate prospects for design rationales. Buckingham Shum finds QOC to be as disruptive as it is useful. Olson et al suggest that in practical terms, the capture of rationale requires an enormous amount of work, and most likely, the employment of a dedicated scribe at design meetings. Conklin and Burgess-Yakemovic present a number of practical guidelines for overcoming some of the resistance to capturing design rationales (such as employing a local champion and limiting the rational capture to specific times) but they hint at a cost-benefit analysis that shows rationale capture to save costs in error identification that the software engineering community are most likely to appreciate. Pulling the pros and cons of rationale use together however, is largely left to the reader, and a general summary chapter would have served a useful purpose for readers on a budget.
The section on educational applications of design rationale offers an alternative to the other sections. Here we are introduced to the notion of rationale as means of instructing designers to think about their actions. Casaday invokes the term template to describe the structure he encourages designers to place on their actions, and he provides anecdotal accounts of their use in design meetings at DEC. Carey et al cover the use of scenarios in more formal educational terms, the teaching of HCI to students or at training workshops in industry. These authors are developing a design library that incorporates rationales examples of exemplary designs and usability principles, but again, the evidence of its utility is anecdotal and premature, though it points in the direction of future training and re-use environments that have tremendous potential.
Potts attempts to link the concept of rationale to more traditional design methods and in a tightly argued chapter raises enough doubts mid-text about the utility of the rationale approach in software engineering practice to stem the flood of optimism that dominates the book. Unfortunately few of the other authors react to his analysis. Interestingly enough, it is left to the final chapter in the book by Grudin for the invocation for rationale developers to ‘know their users’ to be raised. In a stimulating analysis of design situations that may differ in their potential to gain from the use of design rationales, Grudin reminds us all not to make the assumption that engineering and science are the same. The very real prospect of generating a formal account of a project that may end in failure is likely to deter many a designer.
I would have liked this perspective to have been tackled more explicitly throughout. The tension between HCI’s scientific aspirations and software engineering’s cost-driven concerns has long concerned researchers and practitioners in this domain. In parallel with Carroll and Rosson’s faith in the rationale approach to science building, other HCI researchers have been concerned with linking science to design in a way that tries to avoid stereotypical distinctions between craft and science, rigor and relevance, the laboratory and the field etc. Underlying rationale is a focused concern with meta-analysis, methodology in its truest sense, and a reflection on our epistemologies. A philosopher of science might have brought a powerful integration to such work, but even in the more philosophical chapters there is little reference to such a perspective - Kuhn is invoked only once in 500 pages, Popper not at all.
Similarly, while a rationale aims to inform designers of the assumptions and criteria they are employing in a design, it seems strange that so much of the work in HCI that has tried to understand how research can influence real-world design practice is never raised in this context. If, for example, we are to encourage designers to consider Questions, Options, and Criteria for any interface component, then it seems relevant to raise the question of how we present criteria that are derived from HCI-research to the people who may best use them. Avoiding such an issue tends to create the (to my mind) false impression that rationales offer a new perspective, far removed from that long-standing debate in HCI about its relevance and its most appropriate form of representation to designers, when it seems more accurate to consider it a natural extension of such work.
In sum, this is a useful and welcome addition to the HCI literature. It takes an important concept and offers multiple perspectives on the nature, use, and value of design rationales. Drawbacks exist in terms of repetition, a loss of focus in some chapters and within sections, and in general I think that too many of the authors labored their points as if to make sure that the reader would understand. While such drawbacks are common to edited volumes, a shorter, more focused text would have resulted if a less inclusive approach, not incorporating previously published papers, had been adopted. But this is not the final word by any means, and students of HCI will need to start somewhere to get a handle on the concept of rationale. The present volume is already on my HCI class reading list, where I expect it to stay for the foreseeable future.