InfoMatters

In the dim distant past, some of us used to distinguish our work from the masses by declaring proudly that we were ‘user-centered’. At one time this actually meant you did things differently and put a premium on the ability of real people to exploit a product or service. While the concern remains, and there are many examples of designs that really need to revisit their ideas about users, I find the term ‘user-centered’ to have little real meaning anymore. It is not just the case that everyone claims this label as representative, after all, who in their right mind would ever declare their work as not user-centered and still expect to have an audience? It is more a case that truly understanding the user seems beyond both established methods and established practices.

I will leave aside there any argument about the term ‘user’. Some people have made careers out of disimissing that term and proposing the apparently richer ‘person’ or ‘human’, but the end result is the same (though I prefer to talk of human-centered than user-centered myself). The real issue is methodological.

First, claiming adherence to user-centered methods and philosophies is too easy; anyone can do it. Ask people what they would like to see in a re-design and you have ‘established’ user-requirements. Stick a few people in front of your design at the end and you have ‘conducted’ a usability test. Hey presto, instant user-centered design process. If only!

Second, and more pernicious, the set of methods employed by most user-centered professionals fails to deliver truly user-centric insights. The so called ’science’ of usability which underlies user-centeredness leaves much to be desired. It rests too much on anecdote, assumed truths about human behavior and an emphasis on performance metrics that serve the perspective of people other than the user. ISO-defined usability metrics refer to ‘efficiency’ and ‘effectiveness’ and ’satisfaction’. These do not correlate so one needs to capture all three. But who gets to determine what constitutes effective and efficient anyhow? In many test scenarios this is a determination of the organization employing the user, or the thoughts of the design team on what people should do, not the user herself. Maybe this should be called organizational-centric or work-centric design. If I wanted to start a new trend I could probably push this idea into an article and someone might think I was serious.

What is often overlooked is that the quality of any method is determined far too much by the quality of the evaluator who employs it.. Evaluation methods are all flawed, that much is a given, but it is the unwillingness of many people to recognize these shortcomings that should give us all real concern. Here’s but one example. The early Nielsen work on heuristic evaluation has given rise to the ‘fact’ that evaluators find about 35% of usability problems following his method, and if you pool several reviewers you can get a better hit rate. What many people overlook in this is that the 35% figure is not calibrated with real user problems but is based on Nielsen’s own interpretation of the problems users will face. So the 35% claim is really a claim that following his method, you will probably find a third of the problems that Nielsen himself estimates are real problems for users. This is a very different thing. It is interesting that in my own tests with students, this 35% figure holds pretty firm, which is impressive, but you cannot lose sight of what that percentage relates to or you will misunderstand what is going on.

Now of course, there are great evaluators out there but even if all evaluators were great, that would not change the problem with user-centeredness as it currently exists. Too much evaluation occurs too late to matter. OK, this is an old story but what’s changed since this story was first heard? Not enough. If user-centered design really is limited to evaluating and designing for a narrowly construed definition of usability then there is little prospect of change. For a limited range of tasks where I want to be efficient (finding a number in my cell-phone, for example, then current practices are fine, as long as I can prototype quickly) but for the type of deeply interactive tasks that I might spend large parts of my day engaged in (reading, communicating, exploring data etc.) then talk of ‘effective’ and ‘efficient’ rings more hollow. But it is preciely this next generation of application opportunity that we need to explore if we are to augment human capabilities. The old usability approach is fine for applications where we are making digital that which used to be performed with physical resources (text editing, mailing, phoning, calculating) but it’s not a great source of help for imagining new uses.

If we could de-couple user-centered design and usability then there might be some benefit but I don’t think this is as important as it might first appear. More important is the very conception we have of users and uses for which we wish to derive technologies and information resources. Designing for augmentation is a very real problem and a great challenge for our field theoretically and practically.

Just back from a fascinating presentation by Aaron Marcus on the importance of culture-centered design. He was a guest here of the ASIST Student Chapter at the iSchool and spoke for almost two hours with questions from the audience. His work leans heavily on Hofstede’s model of cultural dynamics, which he acknowledges has several weaknesses, but he presented an interesting mapping of the general characteristics of cultures (too often for my taste reduced to ‘nationality’) and sample web sites one finds in government, university and large company websites that represent said culture. Fascinating work, but more needs to be done.

The ASIST 2007 conference in Milwaukee this week was also a relatively lively affair. Keynoter Anthea Stratigos from Outsell presented a fast paced look at the world and I believe surprised many of us with the statement that China was fast becoming the leading English-speaking nation in the world. No reference provided but if it is even close to being true, what are the cultural implications? If nothing else, why do relatively recent listings of countries where English is spoken, even if not recognized as an official language, seem to make no mention of this? The best I can do to track this comment is back to the UK Prime Minister, Gordon Brown, who apparently predicted that Chinese speakers of English would outnumber all other English-language speakers in the world by 2025. Sound bite as science?

I also had a chance to talk with Tefko Saracevic and Don Kraft about changes they had seen in submissions to their respective journals (Information Processing and Management and JASIST). Tefko noted that he witnessed as many submissions in the last few years as he had received in the 15 preceding years, a fact he attributes to the emergence of information research in China and India. Could the day come when American and European scholars will compete to submit to Asian journals? If it happens it will seem inevitable and obvious with hindsight, but will it happen? One suspects there are many forces at work here, only some of which we recognize. In the meantime, this submission glut has pushed up the rejection rates for these journals to record highs.

I took part in a panel on Digital Genres that, despite its theme, drew a large and very lively audience at 8.30 am Tuesday morning. This really was a panel (not an attempt to sneak papers past reviewers) to which each presenter was limited to 3 minutes and then had to answer questions. Before we knew it, the audience got stuck in and took us on a tour of the problems with definition, the lack of appreciation in the field for earlier work, the absence of a well-informed archival perspective, the value of genre in searching and the emergence of genre in the digital realm. This was the hightlight of my conference sessions but of course, I am biased.

The special issue of Information Research with the Proceedings of the CoLIS 2007 conference in Sweden has now been published. There’s a lot of interesting reading here but let me point to a couple of papers I like. The Talja and Hartel piece examining the concept of user-centeredness in the information literature is a worthy contribution and should be required reading for those who wish to understand the emergence of this defining orientation within our field. Also, David Bawden’s paper, Information as self-organized complexity, provoked a lot of discussion at the conference itself and is now available to all.

The IEEE Professional Communication Society has awarded Arijit Sengupta and myself the Rudolph J. Joenk, Jr. Award for Best Paper of 2006 in the IEEE Transactions on Professional Communication. The paper, “Query by Templates: Using the Shape of Information to Search Next-Generation Databases,” (see my pubs page for a copy) results originally from Dr. Sengupta’s doctoral dissertation which he completed at Indiana in Computer Science while we were both there. Jit is now an Assistant Professor at the Dept of Information Systems and Operations Management in the Raj Soin College of Business at Wright State University and deserves most of the credit for this - not only was it his research work but he persisted with the paper when others may have been put off by some of the reviewers’ comments which indicated they did not really understand the concept of information shape and when his fellow faculty members wondered just what is a CS guy doing working on this kind of stuff? Congrats Jit!

I learned today of the death of one of the legends of IR, Prof Karen Sparck Jones of University of Cambridge — there’s a nice note about her at:http://www.admin.cam.ac.uk/news/dp/2007040403. The expression above about the need for more women to study CS is hers, and she was right. Her work crossed boundaries from automated language processing to privacy. Gender aside, we just need more thinkers like her in this field.

The Modern Language Association (MLA) Task Force on Evaluating Scholarship for Tenure and Promotion released a new report in December that questions the role of monographs, and more directly, the responsibilities of publishers and university presses in facilitating tenure decisions for scholars in the humanities. The report is available at: www.mla.org/tenure_promotion and raises the spectre of faculty failing to take appropriate responsibility for tenure decisions by placing an undue emphasis on the successful production of published monographs by new professors. Since successful publication of a monograph requires the author to pass the review stage of the press, so the argument goes, then such reviewers have more influence on eventual tenure than the faculty making up the P&T committee at the candidate’s institution. The report contains some disturbing data, suggesting that PhD’s in the fields represented by MLA have only about a 35% chance of getting tenure when viewed as a complete pool, and that the the standards for receiving tenure are becoming ever more demanding. Not only are faculty making decision based on the outcomes of publishers’ reviews of proposals, the report argues that publishers themselves are more concerned with publishing essays, editions and textbooks that they can sell rather than monographs that impress tenure committees. Of course, taking a socio-technical systems perspective here one has to bring the libraries into the equation. Since libraries form a large (the largest?) market for scholarly books, the declining interest of libraries in purchasing monographs, particularly in the humanities, means that utlimately, tenurability of faculty can rest on the decision of a librarian to purchase a work (and how many librarians now graduate from programs that do not require an understanding of research?). This is no idle concern. The Association of Research Libraries reports that monograph purchases are falling as expenditures on serials rise prohibitively. Some where in all this, innovation and quality of research output are squashed, if not lost, when it comes to judging the work of scholars. Of course, everyone would agree that assessment of quality should never rely solely on the judgements of those whose primary motive is profit, but there is a real danger that this is where we are in certain disciplines. New digital scholarship must start finding clearer indices for quality.

If you’ve dabbled in online auctions or sales you will no doubt have wondered just how reliable are the various reputation ratings sellers report to convince you of their integrity. Interesting news this week from researchers at CMU who report that fraudsters in online auctions and selling sites can be reliably identified by the pattern of behaviour they reveal (tinyurl.com/yyr32m). The key seems to be identifying link patterns between group members, with fraudsters tending to display a clear pattern of communicating more with members in another group than the group of legitimate buyers and sellers on the site. This ‘bipartite core’ which can be clearly seen when the various transactions are plotted as a graph seems to be a strong indicator of the perpetrator working with accomplices to maintain a clean record on the original site while engaging in fraud with partners. The precise details are not yet revealed but most interesting is the emergence of this type of behavioral index from a massive data set using data mining techniques. What other indices of human behavior are out there to be uncovered is intruiging to consider. With Ebay reporting over 200 million registered users, the problem of small N for behavioral research might just start to appear quaintly old-fashioned.

The conference was a great event over 6 days, depending on when you started. For me the official kick off was the iSchool party at the Cedar Door where our tab had to be upped several times to handle the thirsty hordes. I had dinner later with our first keynote, Laszlo Barabasi, who is a delightfully engaging guest and speaker. His keynote address was fast paced and pointed to the insights to be gained in viewing human activities on the web as scale-free networks incorporating bursts of activity. He argued that 10% of most networks provide the key to holding the network together and that fitness attracts a disproportionately large number of links from other sites. Of course, the mystery of what makes a site or a linked node super-fit remains something to be discovered (and sold, I suppose). You can find out more about the man and his work here: www.nd.edu/~alb/

Attendance was up and most people seemed genuinely happy with the program and the location - Austin makes for a great conference venue though I needed to work on people to move them beyond the dubious delights of 6th St when seeking entertainment. Several sessions just would not end — a well attended set of presentations on blogs ran 30 minutes over (it was lunchtime) as people just would not stop asking questions of the various presenters. And it was not just new areas that caught the buzz. The panel on historiography was equally in demand even on the last day! I make a point in my program notes that ASIST is one conference where the old and the new mix easily, and it is this type of perspective-mix that keeps me at ASIST year after year. It was also good to see so many PhD students and younger members - ASIST seems to have lost many of the younger set in recent years to the equally-large IA Summits but when President Mike Leach asked at the outset how many people were attending ASIST for the first time, it was good to see so many hands go up.

Peparing a conference program is a long process and I am glad it’s over. I had superb assistance from Dick Hill at ASIST and three executive program committee members (France Bouthillier, Javed Mostafa and Carole Palmer) but it remained a long slog which I am glad to hand over to next year’s committee (see the call for papers: www.asis.org/Conferences/AM07/am07cfp.html). While the society is good about awards events for various members, I think the program committee each year deserves a little more than a piece of paper commemorating their efforts and handed out in a rush at the poorly-attended business meeting. But this is a minor issue - the conference is its own reward, right? I’ll just not be rushing to serve on future program committees.

It was good to see so many faces there, and to talk to several readers of the blog - hello!! More later when I get a chance to think about it all.

Am just back from a trip to GSLIS at McGill (great people, wonderful hospitality) where I spoke last week to faculty and students there on the future of information studies and the need for us to more aggressively position the field through better research and a focus on real world issues. I mentioned, rather bluntly, that I consider the world not to be in need of any more models of information seeking behavior, since I consider there to be far too many of these out there already. Worse, most of these are not really models at all but vague representations involving arrows, boxes and circles that contain little more than common sense. I doubt anyone will really listen to this since one sure way of making a career as an academic in LIS is to find a group that has never been studied explicitly and then describing their behaviors as if these were unique or important. I joke that there really ought to be a model generating algorithm out there rather like those “How to Speak Postmodern” or “Create your own Blues Singer” guides which contain three separate lists of terms that can be combined in any order to give you phrases such as ‘Hyper-modern multivocalities” or names like “Jumping Jake Humperdinck”. For information seeking models it could be as simple as listing age, gender and job characteristics (e.g., the info seeking behavior of middle-aged, male, clergy etc.). We could get more sophisticated and add task or media attributes once we have exhausted the possibilites of three attributes. Maybe we are there already.
Of course, I expect a ‘model’ to have some predictive value in helping us understand what people do, so this is probably a minority concern for now but it has me thinking about the need for a corrective in the LIS literature.

Most models of information seeking behavior look at more than behavior, they consider cognition which is quite natural for information activities, except that behavior and cognition are not the same. I can let this slide and go with Wilson ’s (1999) slightly unwieldy definition of information behavior as ‘those activities a person may engage in when identifying his or her own needs for information, searching for such information in any way, and using or transferring that information”. Should this not be a reasonable concern of information researchers? It could certainly be if the fruits of that research shed real insight but it’s not clear that we have gained much from all this effort. Most models allude to environmental or contextual drivers, some responses by a human, and a state change resulting in feedback. They are presented often in a form of flowchart that seems to indicate a logical human process abstracted by careful examination.

Wilson’s (1999) article Models of Information Behaviour Research synthesizes many of the popular models into a nested framework that reveals many of the similarities among models but in so doing highlights for me the paucity of real content in any one of them. I was surprised nobody challenged my view but then again, the challenge requires an example of a model that really works. Maybe I am just expecting too much here, but for the sake of Information Studies, I hope not!

There is much interest in attracting new students, especially female, to computer science and it has not gone unnoticed by some in that discipline that there is a real image problem. The Computer Research Association, a grouping of some 200 academic departments in computer science and engineering, is doing its best to put the sex appeal back in CS (you mean it was once there?) by inviting anthropologists to give keynotes at their conference (the wonderful Genevieve Bell of INTEL (www.intel.com/technology/techresearch/people/bios/bell_g.htm) who spoke here at the iSchool two years ago) and trying to sell the message that not only can CS give you a high paying job but it really does deal with exciting ideas. Check out the reports from this year’s CRA gathering at: www.cra.org/govaffairs/blog/, particularly the accounts of Rick Rashid, head of Microsoft Research’s address where he argued that there was much excitement still awaiting the CS profession.

I don’t dispute any of this but I would note that the three projects listed as exemplars of wonder are:

–Using any surface as a computing interface
–Human scale storage, where all one’s actions and conversations can be recorded
–Terra scale applications such as mapping the sky and giving multiple attributes to each object

These have real potential for excitement but how much of that stems from the computational aspects that must be solved or from the human and social factors that such innovations might invoke. Unless CS incorporates the necessary methods and theories to handle those aspects then it’s hard for me to get terribly excited. And if CS did incorporate these, then would it still be computer science (and no jokes please about any discipline with ’science’ in its name not being a real science)?

The serious point here (other than growing the recruitment of more and better balanced student cohorts) is what type of knowledge does it take to deliver successful outcomes for such projects? In my view there is no single discipline that could really tackle one of these three wonder projects successfully, only a multi-disciplinary approach could work. Since we tend to divide up universities into discrete disciplines and put buildings around them to keep outsiders from infiltrating their ranks, there seems to be a problem here. What would it take to create a truly new intellectual space to end the isolation at universities? I think the answer to that is far more important to think about than any specific wonder project and the information school movement might be the appropriate vehicle for trying out potential solutions.