Visualization-ism: An Art History

Berglund Authority Level 3

by David Staley <dstaley@heidelberg.edu>

Art historians of the future may very well define ours as one of the most important in the entire history of images, on par with Impressionism, Cubism or the Baroque. The images of our period are composed not with marble or oil on canvas but with pixels and bits of electronic information. Their subject matter is not the human form, religious motifs or inspiring landscapes but relatively mundane topics such as the structure of social relations, the rhythms of the heart or the landscape of numbers. Since the 1980s, the creation of digital graphics such as GIS maps, data mining images, CT scans and mathematical phase spaces have helped to spur on the art historical period I label Visualization-ism.

A visualization is an image that is used to organize and communicate information. The graphic designer Edward Tufte refers to such images as "cognitive art," and includes images such as train schedules, statistical scatterplots, maps, musical scores and engineering schematics as examples. The iconoclastic art historian James Elkins calls these "information images," and includes them in his list of "non-art images," by which he means images that are not paintings, sculptures or architecture, the types images that have long defined the traditional domains of art historical analysis. According to Elkins, non-art images make up by far the bulk of all the images human beings have created, and can be examined by art historians in the same fashion as more traditional "art images." As the examples listed above suggest, cognitive art and other types of and non-art information images have a long and rich history in the sciences; the application of computer graphics has enhanced and expanded this pre-existing feature of science. Indeed, Visualization-ism may well be understood as a renaissance of cognitive art. What distinguishes the last decade as a distinct period in the history of the visualization of information, and indeed in the history of art, is the widespread application of the concept of visualization to many areas outside of the sciences.

The history of Visualization-ism begins in the sciences. Like Giotto's frescos, which anticipated the Italian Renaissance, computer-generated images like the Lorenz Attractor anticipated Visualization-ism. In the 1960s, Edward Lorenz was seeking to find patterns in long-term meteorological data. Lorenz used a three-dimensional coordinate system physicists call "phase space." Lorenz reduced the meteorological data he was examining to three variables which would describe the state of the weather at any given point in time. Those variables formed the dimensions of the phase space. Using an early form of computer graphics and by plotting these points in phase space, Lorenz discerned a striking and heretofore unrecognized pattern in the data, the Lorenz Attractor. (figure 1)

figure 1: The Lorenz Attractor (http://archive.ncsa.uiuc.edu/SCMS/DigLib/stills/0892.hobill.lg.gif)

An attractor means that the phase points coalesce around a particular area of the phase space, like gnats attracted to a light. Lorenz could not predict where the next phase point would alight, but it soon became clear that that point would end up somewhere in the resulting butterfly-like shape. In the 1980s, using the more sophisticated graphics enabled by fourth generation computers, physicists explored similar types of "strange attractors," those complex systems like the weather that seemed chaotic but whose phase space diagrams revealed a coherent, if unpredictable, pattern. Computers were useful not only in crunching the numbers and generating the data, but in organizing those data into a useful visual form. The graphics were not incidental to the work or simply a visual aid or illustration: the visualization of the physical systems was the scientific work.

Using computer graphics, scientists and mathematicians explored the data in similar abstract shapes, such as Poincare maps, bifurcation diagrams and fractal geometries, the most famous of the latter type being the Mandelbrot Set. (figure 2)


figure 2: The Mandelbrot Set (www.math.utah.edu/~alfeld/math/mandelbrot/mandelbrot.html)

Mathematicians used computer graphics to explore other mathematical surfaces and spaces; the mathematicians Gregory and David Chudnovsky, for example, used computers not only to calculate the value of pi to several billion digits but to display their results in a three-dimensional graphic form they called a "pi-scape." Computer graphics were also applied to physical objects and spaces as well. CT scans and MRI scans allowed doctors the opportunity to see the soft tissue of the body without the need for surgery or other invasive procedures. One science writer in the mid 1990's remarked that computer graphics had allowed scientists to wrest the concept of a "map" from geography, since many of these images "mapped" out spaces as diverse as the human body, abstract mathematical spaces and the human genome. These scientists were, in his words, "Mapping the Next Millennium."

The 1990s witnessed an explosion in the production of these computer generated information images. In 1986, the National Science Foundation funded a report on the use of computer graphics for scientific work, the results of which were published a year later. The authors of the report coined the term "scientific visualization" to describe the application of computer graphics and imaging to computational science, the branch of science that uses computers to study scientific or engineering problems, a method somewhere between theory and experiment. "Visualization is a method of computing," wrote the authors of the report. "It transforms the symbolic into the geometric [that is, numbers into shapes], enabling researchers to observe their simulations and computations. Visualization offers a method for seeing the unseen. It enriches the process of scientific discovery and fosters profound and unexpected insights. In many fields, it is already revolutionizing the way scientists do science." The revival in the production of scientific images after 1987 may well have had much to do with the blessings of the NSF, the decline in the support for supercomputing centers, perceived competition from Japanese software manufacturers, and the fact that desktop computing had become widespread. Whatever the causes, Chaomei Chen, the editor of the journal Information Visualization tracked a steady increase throughout the 1990's in the number of journal articles devoted to visualization, to go along with conferences and books devoted to the subject.

Scientific visualization has not remained confined to just science, however, which is what distinguishes this period of Visualization-ism from earlier periods in the visual history of the sciences. Abstract data images are becoming a part of other disciplines. The social sciences were among the first to import the concept of visualization from the hard sciences. This was especially evident in geography and cartography, disciplines that already employed spatial images in their work. Geographers began working with GIS systems, a method of converting data (even data that were not already in spatial form) into maps, in order to discern patterns in that data. (figure 3)


figure 3: GIS map (www.usgs.gov/research/gis/gis-f30.gif)

This GIS visualization from the US Geological Survey maps a portion of the San Andreas Fault in San Mateo County, California. This image was produced by combining a digital elevation model with a LANDSAT image of the same area, to produce the illusion of three-dimensions on the two-dimensional surface of the screen.
In the same way scientists and mathematicians used visualization to organize numerical data, businesses are using computer graphics in order visualize patterns in their data. Data mining is a process similar to GIS, in that it converts database information into visual form, revealing patterns in the numbers that would not have been apparent had the numbers remained in tabular form. At IBM Research, Peter Kirchner's elegantly designed "Results Obtained from a Credit Card Application" maps out six variables of information. (figure 4)


figure 4: Data Mining (www.research.ibm.com/dx/imageGallery/image212.html)

Each sphere represents an approved credit card holder, the size of the sphere indicating the holder's salary and the color identifying their credit limit (see the key at the bottom). These spheres are situated in a three dimensional space, with axes representing debt ratio, work duration and net worth. Such a visualization is not a decorative diversion that draws attention to the real information; the resulting image is the real information.

In this visualization by Martin Wattenberg titled "The Map of the Market," found at SmartMoney.com, the dynamics of capital markets are captured in a Mondrian-like display. (figure 5)


figure 5: The Map of the Market (www.SmartMoney.com)

The markets are divided into sectors, and within each sector there are further divisions into rectangles representing individual companies. The area of the rectangle is equal to the company's market capitalization. Color adds another dimension to this display. Red means a drop in stock price, green means a rise and black means no change. Further, the brightness or dullness of the hue reflects the intensity of the change: bright red or green means a large change in stock price, a darker shade (the addition of black) means less of a change. As with any good visualization, the image displays lots of data, and the viewer is able to easily see the shape of the whole as well as zooming into individual parts of the data.

Designers have applied visualization to domains without numbers as well. Wattenberg has designed a method called the matching diagram that visualizes structures in a sequence of symbols. He has applied this visualization technique to a variety of sequences, such as music, text, encoded data files and DNA. Look, for instance, at a matching diagram for one of Bach's Goldberg Variations: (figure 6)


figure 6: A matching diagram of one of Bach's Goldberg Variations (www.turbulence.org/Works/song/gallery/gallery.html)

The music is laid out in a single line, with arches connecting identical passages. The resulting forest of arches, according to Wattenberg, reveals the "deep structure" of the music, a structure that reveals symmetries and asymmetries, order and complexity in the sequence. Seeing this visual structure is unlike our usual experience of music, which unfolds in sequential time rather than in two-dimensional space. Having seen the whole structure, a viewer can then compare and contrast the structures in several pieces of music. Compare the matching diagram of Bach's music to this one by Chopin, noting the structural differences between the two: (figure 7)


figure 7: A matching diagram of Chopin's Mazurka in F#Minor (www.turbulence.org/Works/song/gallery/gallery.html)

Further, since the method can potentially apply to any sequence of symbols, one could discern visual analogies across domains, comparing the structure of a piece of music to a baseline DNA sequence to a novel. (figure 8)


figure 8: A matching diagram of DNA, 200 base-pair sequence (www.bewitched.com/match/dna.html)

Like many of the most interesting computer visualizations, the matching diagram allows us to visualize information that are not usually thought of as visual.

Like music, we experience written language as a sequence of symbols; the whole of the structure unfolds one symbol at a time, in a line, and never all at once. "Text visualizations" seek to represent a one-dimensional written text as a two- or three-dimensional visual structure. (figure 9)


figure 9: A 3-dimensional text visualization of Dante's Inferno (www.iath.virginia.edu/dante)

In this visualization of the structure of Dante's Inferno, Deborah Parker's text visualization allows a viewer to select names, geographic references or other tagged characters from the poem, and then map these in a three-dimensional representation of the levels of Hell, thus depicting the distribution, density and location of references. When reading the text, this structure is revealed to a reader only after she has coursed through the long line of words. With this visualization, the viewer can discern, with one look, the structure of the whole. George Chang has similarly created a multidimensional visualization of Romeo and Juliet. (figure 10)


figure 10: A text visualization of Romeo and Juliet (www.jupiterstar.com/lunarscribbles/visualization/explanation)

The text of Shakespeare's play is (literally) arranged in a circle surrounding the visualization, read in a clockwise direction from "12:00" at the top. In the center, Chang has broken up the words of the text from their linear chain and arranged them in a three-dimensional space. The position of a word is based on its predominant location in the text. "Romeo" appears throughout, and so it gravitates toward the center of the image, equidistant to all locations in the circle of text, whereas "Mercutio," who appears only in the first half of the play, is attracted toward the lower right-hand part of the circle. The brighter the word, the more frequently it is cited in the play, thus "Romeo" is brighter than "Mercutio." The goal of a text visualization is not to read the text in a new way but rather to view the text as a whole, to comprehend its structure, revealing new insights.

As these examples illustrate, visualizations are useful in depicting whole structures. The sociologist Lothar Krempel has explored the visualization of networks of social structures, or "network visualizations." These are visualizations that depict the connections between social actors, and how power and influence flow through the resulting networks. He has depicted social structures such as the reciprocal gift exchange patterns among the !Kung, cooperation and collaboration between German sociologists, the networks of associations between research organizations and visitor behavior at the Duisburg Zoo, among others. This image depicts "Access" of a German industrial organization to state-sponsored research laboratories. (figure 11)


figure 11: A network visualization by Lothar Krempel (www.mpi-fg-koeln.mpg.de/~lk/netvis/access.html)

Krempel uses simple symbols to depict complex social phenomena: spheres of varying size depict the influence exerted by social actors. These spheres are arrayed in an abstract space where the distance between spheres reflects the force or repellency between the actors (although the distance between spheres can also refer to their position in a hierarchy or their location in geographic space). Lines connect these spheres to show the intensity of the social bonds between the actors. Krempel's network visualizations—like Parker's text visualizations or Wattenberg's matching diagrams-- are all engaging evidence that the concept of visualization has migrated from the sciences into the social scientific and humanistic realms as well.
Although similar, appreciating a visualization is not the same as our experience looking at more traditional art images. For example, visualizations tug at our expected ideas concerning the organization of pictorial space. As with many conventional works of art, visualizations are two- or three- dimensional pictorial spaces. One of the more important creative acts confronting the designer of a visualization is imagining and configuring that space. For some visualizations, this choice requires minimal attention, as the nature of the data themselves suggest a spatial form. GIS maps, and CT and MRI scans, for example, are already representations of some physical terrain, whether that be the surface of the earth or the geographies of the human body. More precisely, visualizations of this type reflect a physical space mapped out according to the dimensions of length, breadth and height. While this representational space must still be composed, the designer does not need to invent the space within which his symbols will be arranged.

The most interesting and expressive visualizations are those whose data do not suggest an obvious physical spatial analogy; instead, the creator must invent the space within which the data will be displayed. The spatial dimensions of these types of visualizations do not correspond to length, breadth and height but rather to other dimensions—intensity, quantity, influence, pitch, repetition-- which must be determined by the creator. Many of the more aesthetically-pleasing visualizations created today are not based on representative spaces like photographs or maps but are more like statistical diagrams or musical scores. Edward Tufte argues that the invention of the first statistical graphics in the 18th century "required replacing the latitude-longitude coordinates of the map with more abstract measures not based on geographical analogy." William Playfair and Johann Heinrich Lambert, whom Tufte credits as the inventors of statistical graphics, defined the x- and y-axes of their diagrams according to statistical measures, in order to discern the relationships between two quantities of data. Tufte refers to these images as relational graphics. The resulting shapes of these relational graphics are not tangible objects in a physical space but rather abstract objects in an abstract space. Such abstract spaces predate even these 18th century images; the historian Alfred Crosby argues that the musical score, standardized in the 11th century, was Europe's first graph, in that it imagined an abstract space defined by pitch along one axis and temporal sequence along the other. The position of a musical note within this space depicts a relationship between those two dimensions; the sequence of notes forming a shape that exists not in physical space but within the abstract space of the musical score. One could easily argue that the inventions of the relational graph and the musical score were just as important to the history of art as the creation of linear perspective in the Renaissance.

The creator of a visualization, like the inventors the relational graphic and the musical score, must imagine a space within which to arrange the information. The spheres in Kirchner's data mining visualization do not relate to each other in a physical space of height and width but rather in an abstract space determined by the dimensions of debt ration and net worth. The social structures of Lothar Krempel or the words in Cheng's text visualizations float in a space determined by dimensions such as frequency of citation or position is a hierarchy. The arcs in Wattenberg's x-graphs do not spiral "upward," at least as we understand that word in the context of physical space, but rather in a space defined by the dimensions of repetition and frequency. In the same way looking at a Cubist painting or a Mondrian forces us to rethink our assumptions about the composition of pictorial space, visualizations confront us with many richly varied abstract spaces.

Although it may seem paradoxical, the beauty of a visualization derives from its utility, another challenge to our traditional understanding of art and art history. The best visualizations are those that allow one to see the structure of the information as a whole, to see the shape of the whole. At the same time, a well-formed visualization allows the eye to "zoom in" in order to examine individual parts of that structure, all the while allowing the eye to see how those parts relate to the whole. The best visualizations are dense with data and information, and are thus deeper and richer than a simple diagram or graph. Visualizations are not decorations used simply to break up blocks of text, nor do they serve merely as dazzling and attention-grabbing devices that draw the eye to something else. Visualizations are not the cartoonish clip-art icons of the type that decorate many websites. Instead, the best visualizations are images that allow one to see, think about and understand multidimensional levels of information that would not have been apparent had it not been so spatially organized.

Visualizations are evocative, expressive visual images worthy of the attention of historians of art. They have altered the way we perceive space and represent ideas in graphic form, and as such belong in the same category as other significant cultural artifacts that have taught us to "see" differently, such as Greek geometry, Renaissance linear perspective and Abstract Expressionism . "Despite the beauty and utility of the best work," Edward Tufte has observed, "design of information has engaged little critical or aesthetic notice: there is no Museum of Cognitive Art." Visualization-ism may not form a chapter in the next edition of Gardner's Art Through the Ages or any other canonical art historical survey any time soon. I can envision a day, however, when art historians approach visualizations with the same critical attention afforded to paintings and sculptures, and place such images within the larger narrative of the history of art. When that day arrives, the Internet will more than likely serve as Visualization-ism virtual Museum of Cognitive Art.
 
About the Author: David J. Staley is an Assistant Professor of History at Heidelberg College, Tiffin, Ohio. He is the author of the forthcoming book Computers, Visualization and History: How New Technologies Will Transform Our Understanding of the Past.
 
References
 
Advisory Group on Computer Graphics, "The History of Visualization in the Social Sciences," April 1999, http://www.agocg.ac.uk/train/review/over1.htm, accessed July 24, 2002.
 
Chen, Chaomei, "Editorial: Information Visualization," Information Visualization 1 (2002), 1-4.
 
Crosby, Alfred, The Measure of Reality: Quantification and Western Society, 1250-1600. (Cambridge University Press, 1997).
 
Elkins, James, The Domain of Images (Cornell University Press, 1999).
 
Hall, Stephen, Mapping the Next Millennium: How Computer-Driven Cartography is Revolutionizing the Face of Science. (Vintage Books, 1992).
 
McCormick, B., DeFanti, T., and Brown, M. "Visualization in Scientific Computing," Computer Graphics 21 (November 1987), special issue.
 
Owen, G. Scott. "Definitions, History, and Goals of Visualization," February 11, 1999, http://www.siggraph.org/education/materials/
HyperVis/visgoals/visgoal0.htm, accessed July 24, 2002.
 
Tufte, Edward, The Visual Display of Quantitative Information (Graphics Press, 1983).
 
___________, Envisioning Information. (Graphics Press, 1990).
 
___________, Visual Explanations. (Graphics Press, 1997).