Preface
There are two major changes in this latest edition of Information Visualization: Perception for Design. The first is intended to make the design implications of research in perception clearer. To this end, 168 explicit guidelines for the design of visualizations have been added to the text in highlighted boxes. These guidelines should be taken as suggestions to support design decisions, not as hard and fast rules. Designing visualizations is a complex task, and it is not possible with a succinct guideline to set out all the circumstances under which a particular rule may apply. Graphic designers must take into account interactions between small symbols and large areas of color and texture as well as shading effects, shape effects, the grouping of symbols, and so on. Different tasks may dictate changes in what should be highlighted and what should be deemphasized. Often a designer must use an existing color scheme or symbol set, and this also constrains the design problem. Because of this complexity, it is important to understand the theory behind a guideline before it is applied; understanding the mechanisms of perception and the processes of visual thinking can make it clear when and how that guideline should be applied and when it does not apply.
The second major change is an increased emphasis on the process of visual thinking. The book now more fully incorporates the modern view that perception is an active process in which every part of the visual system is retuned several times a second to meet the needs of the current visual task. The greatest change is a radical reworking of the final chapter, which now sets out the key components of the architecture of the visual brain and follows this with a description of ten visual thinking algorithms. These describe how people think using common visualization tools and techniques. They are intended to help a designer take a visualization design problem and create a novel and well-designed visual thinking tool.
In addition to these major changes, the book has been revised and updated throughout to take recent research into account. Hundreds of new references have been added, and most of the figures have been redrawn to take advantage of full-color printing.
Now let me tell you how this book came about. In 1973, after I had completed my master’s degree in the psychology of vision, I was frustrated with the overly focused academic way of studying perception. Inspired by the legacy of freedom that seemed to be in the air in the late 1960s and early 1970s, I decided to become an artist and explore perception in a very different way. But after three years with only very small success, I returned, chastened, to the academic fold, though with a broader outlook, a great respect for artists, and a growing interest in the relationship between the way we present information and the way we see. After obtaining a doctorate in the psychology of perception at the University of Toronto, I still did not know what to do next. I moved into computer science, via the University of Waterloo and another degree, and have been working on data visualization, in one way or another, ever since. In a way, this book is a direct result of my ongoing attempt to reconcile the scientific study of perception with the need to convey meaningful information. It is about art in the sense that “form should follow function,” and it is about science because the science of perception can tell us what kinds of patterns are most readily perceived.
Why should we be interested in visualization? Because the human visual system is a pattern seeker of enormous power and subtlety. The eye and the visual cortex of the brain form a massively parallel processor that provides the highest bandwidth channel into human cognitive centers. At higher levels of processing, perception and cognition are closely interrelated, which is the reason why the words “understanding” and “seeing” are synonymous. However, the visual system has its own rules. We can easily see patterns presented in certain ways, but if they are presented in other ways they become invisible. Thus, for example, the word goggle, shown in the accompanying figure, is much more visible in the version shown at the bottom than in the one at the top. This is despite the fact that identical parts of the letters are visible in each case and in the lower figure there is more irrelevant “noise” than in the upper figure. The rule that applies here, apparently, is that when the missing pieces are interpreted as foreground objects then continuity between the background letter fragments is easier to infer. The more general point is that when data is presented in certain ways the patterns can be readily perceived. If we can understand how perception works, our knowledge can be translated into guidelines for displaying information. Following perception-based rules, we can present our data in such a way that the important and informative patterns stand out. If we disobey these rules, our data will be incomprehensible or misleading.
This is a book about what the science of perception can tell us about visualization. There is a gold mine of information about how we see to be found in more than a century of work by vision researchers. The purpose of this book is to extract from that large body of research literature those design principles that apply to displaying information effectively.
Visualization can be approached in many ways. It can be studied in the art-school tradition of graphic design. It can be studied within computer graphics as an area concerned with the algorithms needed to display data. It can be studied as part of semiotics, the constructivist approach to symbol systems. These are valid approaches, but a scientific approach based on perception uniquely promises design rules that transcend the vagaries of design fashion, being based on the relatively stable structure of the human visual system.
The study of perception by psychologists and neuroscientists has advanced enormously over the past three decades, and it is possible to say a great deal about how we see that is relevant to data visualization. Unfortunately, much of this information is stored in highly specialized journals and usually couched in language that is accessible only to the research scientist. The research literature concerning human perception is voluminous. Several hundred new papers are published every month, and a surprising number of them have some application in information display. This information can be extremely useful in helping us design better displays, both by avoiding mistakes and by coming up with original solutions. Information Visualization: Perception for Design is intended to make this science and its applications available to the nonspecialist. It should be of interest to anyone concerned with displaying data effectively. It is designed with a number of audiences in mind: multimedia designers specializing in visualization, researchers in both industry and academia, and anyone who has a deep interest in effective information display. The book presents extensive technical information about various visual acuities, thresholds, and other basic properties of human vision. It also contains, where possible, specific guidelines and recommendations.
The book is organized according to bottom-up perceptual principles. The first chapter provides a general conceptual framework and discusses the theoretical context for a vision-science-based approach. The next four chapters discuss what can be considered to be the low-level perceptual elements of vision, color, texture, motion, and elements of form. These primitives of vision tell us about the design of attention-grabbing features and the best ways of coding data so that one object will be distinct from another. The later chapters move on to discussing what it takes to perceive patterns in data: first two-dimensional pattern perception, and later three-dimensional space perception. Visualization design, data space navigation, interaction techniques, and visual problem solving are all discussed.
Here is a road map to the book: In general, the pattern for each chapter is first to describe some aspect of human vision and then to apply this information to some problem in visualization. The first chapters provide a foundation of knowledge on which the later chapters are built. Nevertheless, it is perfectly reasonable to randomly access the book to learn about specific topics. When it is needed, missing background information can be obtained by consulting the index.
Chapter 1 : Foundation for a Science of Data Visualization A conceptual framework for visualization design is based on human perception. The nature of claims about sensory representations is articulated, with special attention paid to the work of perception theorist J.J. Gibson. This analysis is used to define the differences between a design-based approach and an approach based on the science of perception. A classification of abstract data classes is provided as the basis for mapping data to visual representations.
Chapter 2 : The Environment, Optics, Resolution, and the Display This chapter deals with the basic inputs to perception. It begins with the physics of light and the way light interacts with objects in the environment. Central concepts include the structure of light as it arrives at a viewpoint and the information carried by that light array about surfaces and objects available for interaction. This chapter goes on to discuss the basics of visual optics and issues such as how much detail we can resolve. Human acuity measurements are described and applied to display design.
The applications discussed include design of 3D environments, how many pixels are needed for visual display systems and how fast they should be updated, requirements for virtual-reality display systems, how much detail can be displayed using graphics and text, and detection of faint targets.
Chapter 3 : Lightness, Brightness, Contrast, and Constancy The visual system does not measure the amount of light in the environment; instead, it measures changes in light and color. How the brain uses this information to discover properties of the surfaces of objects in the environment is presented. This is related to issues in data coding and setting up display systems.
The applications discussed include integrating the display into a viewing environment, minimal conditions under which targets will be detected, methods for creating grayscales to code data, and errors that occur because of contrast effects.
Chapter 4 : Color This chapter introduces the science of color vision, starting with receptors and trichromacy theory. Color measurement systems and color standards are presented. The standard equations for the CIE standard and the CIEluv uniform color space are given. Opponent process theory is introduced and related to the way data should be displayed using luminance and chrominance.
The applications discussed include color measurement and specification, color selection interfaces, color coding, pseudocolor sequences for mapping, color reproduction, and color for multidimensional discrete data.
Chapter 5: Visual Salience and Finding Information A “searchlight” model of visual attention is introduced to describe the way eye movements are used to sweep for information. The bulk of the chapter is taken up with a description of the massively parallel processes whereby the visual image is broken into elements of color, form, and motion. Preattentive processing theory is applied to critical issues of making one data object distinct from another. Methods for coding data so it can be perceptually integrated or separated are discussed.
The applications discussed include display for rapid comprehension, information coding, the use of texture for data coding, the design of symbology, and multidimensional discrete data display.
Chapter 6 : Static and Moving Patterns This chapter looks at the process whereby the brain segments the world into regions and finds links, structure, and prototypical objects. These are converted into a set of design guidelines for information display.
The applications discussed include display of data so that patterns can be perceived, information layout, node–link diagrams, and layered displays.
Chapter 7 : Visual Objects and Data Objects Both image-based and 3D-structure-based theories of object perception are reviewed. The concept of the object display is introduced as a method for using visual objects to organize information.
The applications discussed include presenting image data, using 3D structures to organize information, and the object display.
Chapter 8 : Space Perception and the Display of Data in Space Increasingly, information display is being done in 3D virtual spaces as opposed to the 2D screen-based layouts. The different kinds of spatial cues and the ways we perceive them are introduced. The latter half of the chapter is taken up with a set of seven spatial tasks and the perceptual issues associated with each.
The applications discussed include 3D information displays, stereo displays, the choice of 2D versus 3D visualization, 3D graph viewing, and virtual environments.
Chapter 9 : Images, Words, and Gestures Visual information and verbal information are processed in different ways and by different parts of the brain. Each has its own strengths, and often both should be combined in a presentation. This chapter addresses when visual and verbal presentation should be used and how the two kinds of information should be linked.
The applications discussed include integrating images and words, visual programming languages, and effective diagrams.
Chapter 10 : Interacting with Visualizations Major interaction cycles are defined. Within this framework, low-level data manipulation, dynamic control over data views, and navigation through data spaces are discussed in turn.
The applications discussed include interacting with data, selection, scrolling, zooming interfaces, and navigation.
Chapter 11 : Visual Thinking Processes This chapter begins by outlining the cognitive system involved in thinking with visualizations. The second half of the chapter provides ten common visual thinking algorithms that are widely applicable in interactive visualization. These are processes that occur partly in a computer, partly in the visual brain of the user. The output of the computer is a series of visual images that are processed through the visual system of the user. The output of the user is a set of epistemic actions, such as clicking on an object or moving a slider, which result in the visualization being modified in some way by the computer.
The applications discussed include problem solving with visualization, design of interactive systems, and creativity.
These are exciting times for visualization design. The computer technology used to produce visualizations has reached a stage at which sophisticated interactive 3D views of data can be produced on laptop and tablet computers. The trend toward more and more visual information is accelerating, and there is an explosion of new visualization techniques being invented to help us cope with our need to analyze huge and complex bodies of information. This creative phase will not last for long. With the dawn of a new technology, there is often only a short burst of creative design before the forces of standardization make what is new into what is conventional. Undoubtedly, many of the visualization techniques that are now emerging will become routine tools in the near future. Even badly designed things can become industry standards. Designing for perception can help us avoid such mistakes. If we can harness the knowledge that has accumulated regarding how perception works, we can make visualizations become more transparent windows into the world of information.
I wish to thank the many people who have helped me with this book. The people who most influenced the way I think about perception and visualization are Donald Mitchell, John Kennedy, and William Cowan. I have gained enormously by working with Larry Mayer in developing new tools to map the oceans, as well as with colleagues Kelly Booth, Dave Wells, Tim Dudely, Scott Mackenzie, and Eric Neufeld. It has been my good fortune to work with many talented graduate students and research assistants on visualization-related projects: Daniel Jessome, Richard Guitard, Timothy Lethbridge, Sean Riley, Serge Limoges, David Fowler, Stephen Osborne, Dale Chapman, Pat Cavanaugh, Ravin Balakrishnan, Mark Paton, Monica Sardesai, Cyril Gobrecht, Justine Hickey, Yanchao Li, Kathy Lowther, Li Wang, Greg Parker, Daniel Fleet, Jun Yang, Graham Sweet, Roland Arsenault, Natalie Webber, Poorang Irani, Jordan Lutes, Irina Padioukova, Glenn Franck, Lyn Bartram, Matthew Plumlee, Pete Mitchell, and Dan Pineo. Many of the ideas presented here have been refined through their efforts.
Peter Pirolli, Leo Frishberg, Doug Gillan, Nahum Gershon, Ron Rensink, Dave Gray, and Jarke van Wijk made valuable suggestions that helped me improve the manuscript. I also wish to thank the editorial staff at Morgan Kaufmann: Diane Cerra, Belinda Breyer, and Heather Scherer. Finally, my wife, Dianne Ramey, read every word three times (!), made it readable, and kept me going.
Figure P.1 The word goggle is easier to read when the overlapping bars are visible.
Redrawn from Nakayama, Shimono, and Silverman (1989).