Interactions

Authors:

---

  Philosophy of Design

SRA is a typical software development company at which most of the people focus on producing contract-based software and where usability and interaction design are a secondary matter. The design project reported here, therefore, has been a unique project in our institute where we could apply an interaction design-centered software development approach in collaboration with a research institute (Nara Institute of Science and Technology) within the context of two government-funded research projects over the last three years. The first research project was to study computer-supported creativity. Instead of trying to identify what functionality is necessary to support creativity, we argued that software that does not "bother or disturb" a user's thinking process would help the user become creative. And we started the design project, which focused on interaction design for a few interactive systems. The second research project was to construct an environment that supports empirical studies. We continued our design project and applied our interaction-design-centered approach again in this research project, resulting in an interactive system that supports exploratory data analysis tasks.

Throughout the interaction-design project, our claim is that interaction design should be the core of software development; it should be taken into account from the very first step of system development. We particularly emphasize the importance of visual interaction. The first look at a screen shot should tell you everything you can do. A system that requires detailed descriptions of what you can do indicates that the system is designed badly. We worked especially hard on not adding unnecessary functionality. In our experience in developing systems for users, we understand that users tend to request more and more functions during the system design. The task of the interaction designer is to discuss with users why most such requirements would be harmful for the system, by arguing that what matters is not the number of functions but the harmony of functions. In this sense, user-centered design, which might indicate leaving too much responsibility to the users, may not necessarily lead to a good design.

  Design Process

Figure 1 illustrates the four aspects of our interaction-design-centered process: (1) a design principle, (2) a task type and an interaction method that supports the task type, (3) an interaction idiom library, and (4) resulting interactive systems.

A design principle is a concept that guides our overall design process. Design is a series of making compromises coping with conflicting requirements and constraints. This principle guides the interaction designer to elicit and prioritize design requirements.

In our design project, we have focused on a particular task type and a particular interaction method that supports the task type. For instance, as we describe in "Sample Design Project," we have focused on supporting the design of linear information design, such as writing a paper, summarizing notes, making reports for data analysis, and editing a movie. We used the spatial positioning of objects to support the task type. We identified why the spatial positioning is a good way to support early stages of linear information design (based on architectural design theories, theories on the use of externalization from psychological perspectives, and theories of representations, such as semiotics). We have constructed a framework architecture using the method and applied it to different application domains in the area of linear information design.

As we design and prototype interactive systems for different domains in the same task type, we have gradually constructed a software library consisting of fragmented applications supporting essential aspects of interaction. For instance, in supporting the spatial positioning of objects, we need a space for placing objects, manipulation of the objects, and manipulation of the space. These components can be used in different application systems as long as they use spatial positioning as an interaction method. As we iterate prototyping, the library evolves and the cost and time required for design and development have decreased.

As discussed earlier, as we go through our design in different systems, our library evolved, our understanding of the task type and interaction method increased, and our design discipline was refined, resulting in an application system. These four aspects are co-evolving throughout the interaction design project.

  Sample Design Project

Our project is based on the four-tiered process. As a design principle, we used a concept called ART (amplifying representational talkback). Based on Donald Schoen's design theory, we focused on the role and effects of representations that operate during a user's thinking processes. ART emphasizes two points: a user should be able to externalize what she wants with as little cognitive overload as possible, and she should be able to perceive what has been externalized with as little cognitive overload as possible. The concept embraces the role of perception as a complement to cognition and respects the power of paper and pencil. Using paper and pencil seldom bothers our thinking processes. We can externalize what we want to externalize, with various degrees of precision and commitment. The goal of our project, according to the ART concept, was how to go beyond paper and pencil.

We applied the ART design principle in designing interactive systems for linear information design. In linear information design, such as collage-style writing, a user needs to construct parts and frames the whole by trial and error. The whole and the parts depend on each other and co-evolve forming a hermeneutic circle. To support linear information design, instead of allowing a user to directly manipulate the linear information, we provide a space for objects as parts framing the whole. A user can freely place objects in a 2-D space, and the system automatically serializes the objects from top to bottom or left to right, whatever accommodates the "natural" order in the application domain.

The spatial positioning of objects allows users to externalize a variety of "meta" comments for linear information design by using size, distance between objects, alignment, and so on.

This has led to the construction of the architecture, consisting of ElementEditor (EE; for creating an object to be placed in the space), ElementSpace (ES; the space), and DocumentViewer (DV; for showing the serialized information).

Four interactive systems were designed and developed using this framework: collage-style writing, notes summarization, multimedia data analysis, video editing.

All of these are for early stages of linear information design using EE, ES, and DV. For instance, in multimedia design analysis a user can view multimedia data (e.g., a subject's video) in EE and identify an interesting part of the movie. The user can drag and drop the segmented movie and place it in ES. The user can move and resize the object; for instance, he might move important factors toward the top and make interesting parts but unknown factors larger. The user can textually annotate the positioned object. Each object, together with its annotation, is serialized (from top to bottom or left to right as the user specifies) and shown in DV in a table format. The content of DV can be saved in an HTML format.

The Jun-NSN library has evolved as the project produced the four systems. The library is rich in its interaction idioms especially for this particular architecture. For instance, the library has several application fragments to implement a "space" in which a user can place multimedia objects (text, movie, sound, image). We have accumulated multiple code fragments to manipulate the space, for instance, show a tracing line for an object as the user moves it.

The interaction designer's role was to prioritize what is desirable and to compromise about what is possible in terms of programming and computational hardware limitations. In doing so, the interaction designer closely collaborates with the programmer in understanding what is possible in terms of what is desirable.

The interaction designer produces sketches and describes to the programmer what the designer thinks is a desirable interaction. The programmer listens to the designer and implements application fragments, which are executable objects that illustrate computational limitations (such as rendering speed, and algorithms). By using tangible, executable application fragments, the designer and the programmer further discuss what can be and cannot be achieved.

The communication goes both ways. Not only does the designer show the sketch and the programmer implements it (see Figure 2), the programmer sometimes implements "cool stuff" and shows the designer; then the designer tries to understand the implication of the possibility (see Figure 3).

When the designer and the programmer both see that most of the conflicting requirements are solved and desired functions are achievable, application fragments are integrated within a single window, resulting in a prototype system. A user then uses the system and identifies likes and dislikes about the system. Some of the user's opinions are taken into account and reflected in the system design, but most of the dislikes that the user identifies can be described by the designer to persuade the user.

  Authors

Kumiyo Nakakoji^1,2,3
kumiyo@is.aist-nara.ac.jp
Graduate School of Information Science
Nara Institute of Science and Technology

Yasuhiro Yamamoto²
yxy@is.aist-nara.ac.jp

Aoki Atsushi¹
aoki@sra.co.jp

¹SRA Key Technology Laboratory, Inc., Japan
3-12 Yotsuya, Shinjuku
Tokyo, 160-0004, Japan

²Nara Institute of Science and Technology, Japan
8916-5 Takayama, Ikoma,
630-0101, Japan
Phone: +81 (743) 72-5381
Fax: +81 (743) 72-5383

³Japan Science and Technology Corporation
2-2-11 Tameoka, Miyagino, Sendai, 983-0852, Japan

Kumiyo Nakakoji has been working for SRA Inc., for the last 15 years. She received her Ph.D. from the University of Colorado through the company's scholarship program. She has been working in human-computer interaction, especially in the use of computational environments to support creativity.

Yasuhiro Yamamoto, who received his Ph.D. in computer science from the Nara Institute of Science and Technology, has been interested in information philosophy and interaction design. He has been working as an interaction designer for the last several years working with Atsushi Aoki.

Atsushi Aoki, a renowned Smalltalk super-programmer, has been working at SRA Inc. since 1990. He has been working on the open-source, 3-D multimedia Smalltalk library called Jun, which gets hundreds of downloads monthly from all over the world through the company's Web site (www.sra.co.jp/people/aoki/Jun/Main_e.htm).

  Figures

Figure. Kumiyo Nakakoji

Figure. Yasuhiro Yamamoto

Figure. Aoki Atsushi

Figure 1. Interaction-design-centered process.

Figure 2.

Figure 3.

  Sidebar: Practitioner's Workbench

Resources

1. Beaudouin-Lafon, M. Instrumental interaction: An interaction model for designing post-WIMP user interfaces. In Proceedings of Human Factors in Computing Systems (CHI2000), ACM Press, pp. 446–453, 2000.

2. Cooper, A. The Inmates are Running the Asylum SAMS, 1999.

3. Weed, B. Visual interaction design: The industrial design of the software industry. ACM SIGCHI Bulletin 28,3 (July 1996).

Tools

Mostly paper and pencil, sometimes Macromedia Fireworks®, Adobe Illustrator®

Favorite quotations

The first sight of the screen matters.

"Don't think ease of use, think enjoyment of the experience." "Don't think affordances, think irresistibles." "Metaphor sucks" (from J.P. Djajadiningrat, C.J. Overbeeke, and S.A.G. Wensveen, Augmenting fun and beauty: A pamphlet. In Proceedings of DARE 2000: Designing Augmented Reality Environments, Helsingor, Denmark, pp. 131–134; available at www.io.tudelft.nl/idstudiolab/research/pdfPool/ 2000/00DjajDARAugm.pdf

Source of inspiration

Philosophy

Good HCI joke

User evaluation and usability standards are for those people who cannot distinguish good designs from bad ones.

 

©2002 ACM  1072-5220/02/0300  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2002 ACM, Inc.