The Art of Mapping Statistics

Interview with George Legrady

George Legrady Studio is a research and production integrated studio for digital interactive installations with recent public projects currently being realized for the Rem Koolhaas Seattle Public Library (2005), the Richard Meier designed Siemens Headquarters in Munich (1999), Ebner Stolz Corporate Offices, Stuttgart (1999), the Los Angeles Metro Rail (2002, 2005), and numerous exhibitions in museums in the US, Canada and Europe. Emphasis is on a systematic approach to permanent embedded architectural works and interactive installations through the implementation of complex technologies for new forms of content, narratives and analysis.

Legrady - since 2001 Professor of Interactive Media, with joint appointment in the Media Arts & Technology program and the department of Art, UC Santa Barbara - is one of the first generation of artists in the 1980's to integrate computer processes into his artistic work, producing pioneering prizewinning projects in the early 1990's such as the "Anecdoted Archive from the Cold War" (1993), "Slippery Traces" (1995), "Sensing Speaking Space" (2002), and more recently the internationally traveling "Pockets Full of Memories" (2001-2005). He has recently completed a commission, "Making Visible the Invisible" for the Rem Koolhaas designed Seattle Public Library, was featured in November at the Whitney Museum Artport (http://www.artport.whitney.org). Legrady exhibited worldwide and received numerous awards

Roberto Simanowski talked with George Legrady about his work Making Visible the Invisible, a commission by the Seattle Public Library for their new innovative building designed by Rem Koolhaas, about the revelation and beautification of data, about the negotiation between artist and engineer in mapping art, and about the future of art.

Seattle Public Library

RS: In 2005 you finished your project "Making Visible the Invisible" in the Seattle Public Library which visualizes information about the circulation of library patrons' books. How does this piece work?

GL: In brief, we receive an XML formatted list of the circulation of books and media (such as cd's and dvd's) that library patrons check out every hour, process it, then visualize the results in 4 sequenced animations presented on 6 large LCD screens that are positioned horizontally behind the main information desk of the library's open research space called the "Mixing Chamber". The intent of the project is to provide a glimpse as to what the community is thinking based on the things they check out to take home with them. We show this by the hour and over long time as the project will be active for ten years.

The data comes together in real-time from various sources, as patrons check things out online, and at a number of locations in the library building. All items have a RFID (radio frequency identification) tag attached to them which are automatically read by the checkout system, resulting in a chronological, aggregated list that provides a colorful associative sequence of titles, a form of semi-randomly generated browsing list. All personal references to patrons are purged from the list as the library maintains very high standards of privacy protection. In addition to the circulation date and timestamp, we get metadata about the book itself such as Dewey subject classification codes, barcode info, titles, associative keywords defined through the Library of Congress Marc System, so there is a lot of potential for visualization. The last visualization "Keyword Map Attack" uses a list of keywords we create by doing a word frequency index. This is done by parsing the titles and keywords of each checked out item in the list, to produce an hourly record of which words are predominant, and then follow with some statistical analysis of what has come through, for instance, calculating how many books, cds, dvds, and how the checked out items map themselves according to the Dewey categories.

The project proposal was guided by the library's interest in having an artwork that reflected on the library as a system, and one of its key infrastructure components consists of the flow of data. I also wanted to build on the library's high-tech interface which the architect Rem Koolhaas prioritized in his concept for the building. The project is unusual as a public artwork in that it plugs directly into the library's Information Technology (IT)system. We receive data from the IT center which we process and store on our server mounted in the library's server room next to hundred others humming away. Once the data is processed, it is accessed on the next floor by three computers each of which have 2 projectors connected to them. These computers receive the data every 15 minutes, and generate visualization animations continuously from morning till night.

earlier forms of visualization

RS: You have radically changed the interface since you began work on this piece. In your earlier version you planned to visualize the data as vertical lines and as a net of dots with changing brightness, which would have reminded of the starry sky or rather of a radar image with books as airplanes and topics as geographical space. Later the ideas for visualization included interesting graphics and animations in an arcane and minimalistic way.

GL: You are comparing the early visualization studies to what eventually was implemented. The project consists of 3 components: the incoming data, its processing, and then the visualization. Since our first tests in early 2004, the incoming data format sent to us from the library has changed formats a few times, and this has led to a number of fundamental redesigns for the visualization. We have been studying the data, trying to get a sense of what would work best, what metadata information might be most relevant and interesting to keep track of. The initial designs which can be seen at the bottom of the project's website page were based on a different set of metadata. We have requested a much richer set from IT. There are currently 4 animations in the library and these are different from the data tracking page where we study by the hour what is circulating: The internet version consists primarily of statistical analyses for research purposes which we realized in 2004 with Andreas Schlegel, an interaction designer working in Berlin today. Andi is also responsible for the look and feel for the site featured in November at the Whitney Museum's artport site.

In hindsight, I spent 2003 and early 2004 studying the possibilities with Andi and in the summer of 2004, I worked with August Black on visualization experimentations and planning the data processing system. August tested a visualization form called "radar plot" which consists of mapping Dewey category data in the form of spokes from a central point like on a bicycle wheel, where the number of books checked out in any category would be marked on the spoke, then by joining the dots and filling in the inside space with black, a visualization was generated that looked like an explosion from a central point. It is called "Burst" and can be seen at this website. An explanation is available here.

1 1/2 years were spent in planning, and then the project was realized between July and early September (ten weeks) this past summer (2005) with the information architecture and technical work done by Rama Hoetzlein, a doctoral student in the UCSB Media Arts & Technology program who has backgrounds in both Art and Engineering, and his colleague Marko Zifchock. The visualization uses GameX, an open-source DirectX based game engine that Rama and Mark created at Cornell University some years ago.  

Vital Statistics: Shows numerically what has circulated during the last hour and since the morning. Each of the 6 screens feature a specific sampled section of the list of items. Screen 1 shows the total number of items checked out; Screen 2 gives a count of the Dewey items checked out; Screen 3 features the total of Non-Dewey items; Screen 4 lists the Books checked out; Screen 5; the DVD’s checked out, and Screen 6 shows the sum of CD, VHS and tape media checked out. The background color is synchronized to change at each hour. It begins with orange in the morning, transitions to yellow by noon. At 1pm it shifts to green and by late afternoon goes from blue to purple. In this way, the color becomes an indicator of time.

Dot Matrix Rain: The screens are subdivided according to the Dewey Classification System consisting of 10 columns across and 100 bars vertically placed to represent the divisions of each category from 0 to 99. Checked out items’ titles come on the screen chronologically in two fashion, “falling from the sky” if they do not have a Dewey number, otherwise popping on screen at their Dewey location, which has the effect of brightening the bar’s color that represents their Dewey classification. Books are in yellow, DVD in green and other media in blue. By the end of the animation, the bars are color coded to provide an overview as to which Dewey categories received the most circulation.

RS: The final version is much more focused on giving precise information. Now one can actually see which titles have been taken. Why did you change from the more visually spectacular and poetic setting to the more precise, less metaphoric one? 

GL: The challenge is always to arrive at the ideal balance between information and poetics. The initial abstract designs were experiments and eventually evolved into more informational visualizations as it seemed necessary to bring in a certain degree of legibility so that patrons can figure out relatively easily how the visual system works. Also, during the experimentation phase we were working with test data. Once the real data was coming in, (and in real-time), this opened the way for visualizations where we could feature the data more directly without too much of a stylistic interference. In the end, the most fascinating element is the cultural narrative and associative resonances that emerges out of the stream of data itself, so its best to let it speak for itself. 

Another thing to keep in mind is the influence of the collaborative process that takes place when specialists work together. Each of the participants makes decisions at every step of the development, which transforms the outcome to varying degrees. These decisions are shaped by individuals' world views, aesthetic approaches, and logical problem-solving solutions that inadvertently pull and push the process so that the outcome may need to be readjusted if the conceptual and aesthetic goals of the project diverge from the initial artistic concept. During the summer production we had 4 individuals involved. Rama and his colleague Mark were developing the technical production in Ithaca. I was in Santa Barbara, Asia and Oklahoma interacting through telephone and the Internet, and so was Flurin Springer, a design student in Zurich, who was brought in to assist with the screen graphic design lay-outs. The final adjustments took place during the installation week in Seattle were Rama and I worked side-by-side to arrive at the current animations.  

RS: What compromises does the artist have to make if the collaboration includes so many people coming from other areas such as data-design, programming, and engineering? 

GL: It's a fine balancing act between being open to contributions that may enhance the project but potentially shift the work slightly from its original intention, and at the same time being careful to maintain the fundamental principles of the project so that the concept of the work is not compromised. Brigitte Steinheider, an industrial/organizational psychologist and I co-authored a paper on the process of team-based collaborative development and potential consequences. The key ingredients are to make sure that there is an overall understanding of the project through knowledge-sharing, and close interaction throughout the development so that the project stays within the parameters of the initial concept.  

RS: You entitle your piece "Making Visible the Invisible" and said at the beginning of this interview its intent is to provide a glimpse as to what the community is thinking expressed through the things they check out. It seems you see your work in the tradition of art, which intents to foster reflection about society and self (ones own role on this planet). You do so by literally disclosing information otherwise hidden or hardly accessible to the public. This can also be seen in your earlier work "Pockets Full of Memories" from 2001, an installation project that activates the public to contribute information, leading to the creation of a database archive throughout the length of the exhibition.  

GL: "Pockets Full of Memories", commissioned by the Centre Pompidou, has now been exhibited in six European urban communities (Paris, Rotterdam, Linz, Budapest, Helsinki, Manchester) and consists of the public digitizing and then describing an object in their possession through an interactive questionnaire. The digitized image of the object is featured with other contributions on a large projection through alternating animations, letting the public become aware how their semantic description determined where the system's algorithm would situate the object in relation to the others. The underlying intent of both "Pockets Full of Memories " and "Making Visible the Invisible" has been to reveal systems at play that impact on us socially, economically, and politically. I consider data management and processing to be a highly interesting arena of research for artistic work. For instance, how we perform our daily social interactions such as how we shop, how we transact finances, how we are medically diagnosed, and how we displace ourselves geographically, are all studied very closely through data collection, statistical analysis, processing, resulting in some form of actions. Things I have done and choices I have made in the recent past make themselves apparent through the incoming junkmail I receive, letting me know that the system has recognized my actions.

RS: Lev Manovich notes in his article "The Anti-Sublime Ideal in New Media": "data visualisation art is concerned with the anti-sublime" because it transforms a phenomenon, "which goes beyond the limits of human senses and reason" into a "representation whose scale is comparable to the scales of human perception and cognition", it transfers "invisible and 'messy' phenomena […] into ordered and harmonious geometric figures". In this light, mapping seems more to have the effect of beautification of (statistic) data and of a therapeutic appeasement of the audience. What is the essence of mapping and its concept as art?  

GL: My Geography, Library Science and Engineering colleagues consider mapping as a means for clarifying the relationships in a set of information, for instance a map allows me to know the way, and in this sense there is appeasement, as I can have the overview, and understand the limits, boundaries, and relations between elements within a structure, situation, or system. The process of visual mapping also allows the emergence of patterns which would not have been apparent otherwise. Things that are present, active, but have yet to be given a label, a name, or have yet to be pointed at. Things that may reveal themselves now that we are paying attention, and looking closely. It is in this sense that the Seattle project has got its name "Making Visible the Invisible. 

RS: Your perspective is certainly accurate if mapping art indeed provides a better orientation. However, there are examples of mapping art, which don't allow a better understanding of a situation or system but deliver a more or less abstract visualization of data. The intent of those pieces (for example Mark Napier's "Black and White," Greyworld's "Source", and Stelarc's "Ping Body") seems to be a strong visual impact on the audience rather than the diagnose of certain processes in society. Those works appear as a new way of formalistic art, combining ready-made data with formalistic experiments. In contrast, the sociological study of "Making Visible the Invisible" - as well as for example "The Mechanics of Emotion" by Maurice Benayoun and Jean-Baptiste or "They Rule" by Josh On & Futurefarmer - can be seen in the tradition of realism or rather naturalism, which considered writing as scientific experiments and aimed to rule out all subjectivity of the author undertaking the experiment. Albert Camus in his book "The Rebel. An essay on Man in Revolt" (1951) once accused this kind of poetic to be a pure apotheosis of reality, in which the requirement of artistic creation, the requirement of a specific perspective of the artist is renounced. Such charge would not apply to "Pockets Full of Memories", where the alternative ways to organize the data compiled is a strong statement by the artist about the way our memory functions. However, "Making Visible the Invisible" - especially in its less metaphoric, more informative version - does present an account of the underlying structure of data without the interference of the artist's point of view. It seems to be more in the way of photography (mirroring what is in front of the camera) than of painting (depicting what the artist sees or feels). How do you see the role of the artist in mapping as an art form? What do you think about those examples mentioned above, which do not present a reliable account of reality?  

GL: Teaching in the Media Arts & Technology Graduate program, a situation where I am daily engaged in interdisciplinary dialogue with engineers, computer scientists and electronic composers, has probably influenced me towards streamlining the visual representations I do so that the information speaks for itself.  

Of course, this realistic approach is an illusion, as no matter how literal, or metaphorical the representation may be, there is always subjectivity and a cultural voice present. As we know the photograph is a transparent medium that successfully hides the author's ideological intentions, and hence the reason why it has been such a great propaganda tool. Paul Virilio quotes the sculptor Rodin who argues in the late 19th century that the painting may better convey and synthesize an event in time, and therefore is "truer" then the photograph (which Rodin calls the scientific image) as it basically lifts a moment out of it and results in a reduced suspended state which falsely represents. So the question of which is truer, the literal or the metaphorical representation, is still open to debate. There is today a very active interest in visualizing information and scientific data as a form of information analysis. This work is being done across many disciplines such as Geography, Library Science, Computer Science, Biology (bio-informatics) and is of particular interest as it overlaps the visualization experiments currently taking place in digital based artworks and design. What could be further emphasized in the scientific research is an active and conscious awareness of the transformative impact of the form or structure by which the data is organized and visualized. In brief: data that is processed through whatever form, is transformed by it.  

For the kinds of visualizations I generate for projects like the Seattle Public Library, the intent is to create rules according to which data is organized and then results in a visualization. If the outcome does not fulfill my expectations in terms of its conceptual, formal, and aesthetic goals, I change the rules. One could argue that art allows for this kind of flexibility, but then scientists do the same.  

On the one hand, I am very much interested in the poetics that may emerge from presenting a set of data within a structure, as I am interested in letting the voice of the artist/author surface through the structural design of the form itself. For instance in the "Slippery Traces" interactive project that I and Rosemary Comella produced in the mid 1990s (published by ZKM in Artintact 3), the concept is that the viewer creates a narrative that emerges through the navigation from one postcard to another by clicking on one of multiple "hotspots" on the current postcard viewed. The images are grouped in chapters, and organized according to metadata, their relationship defined through rules that have been encoded into the database, so that hopefully the viewer at some point realizes that the narrative sequence they have been weaving through their choices, going from postcard image to the next at some point reveals the author's (my) particular point of view about the meaning of the selected images. In essence, what I provide in this work, is a set of images that have "in" and "out" connectors based on what is in the image, and the viewer connects them one at a time, to eventually arrive at a narrative. 

RS: You started your carrier as a visual artist before getting involved in programming. You have seen - and have been actively involved in this process - how new media changed the hierarchy, system and concept of art during the last 20 years. Do you have an intuition about art and new media 10 years from now? 

GL: I began my artistic career in photography, was introduced to computer programming in the early 1980's and it took some time to figure out how to synthesize digital technological processes into an artmaking influenced by minimalism, and conceptual art, and so I bring that experience to consider how new media may develop in the future. Certain aspects of digitization fit right in to the analysis of technological representation. For instance, questions related to differentiations between the original and the copy, code production as a form of authorship, the unique object, multiples and mass production, the relation between the real and its simulation, etc. Many of the discursive topics of postmodernism have been seamlessly transplanted onto digital media art. What has stood out since the time of photography's integration into the mainstream contemporary art market in the 1980's is that approaches that have continued painting's historical discourse of the grand narrative such as in the works of Thomas Demand, Andreas Gursky, Cindy Sherman, Barbara Kruger, and others, have become integrated to the core of art discourse whereas those experimental works that focused on examining the syntactic language of the medium such as the formal experiments of let's say Rodchenko, Moholy-Nagy, or the "street photography" focused works of photographer's like Winogrand, Friedlander, which address how the medium itself can develop new forms of representations, maintained their status as marginally situated "photographic works." In the end, everything will be digitally produced so the medium itself wont be the issue, but rather how it will be implemented, either to continue certain narratives, or to challenge and arrive at new forms of what my colleague Marcos Novak describes as "new worldmaking".

RS: Thanks a lot for this interview.