Defining Digital Multimedia

by Ken Jordan

Recently Randall Packer and I published an anthology of seminal texts from the history of computer-based multimedia. The book, Multimedia: From Wagner to Virtual Reality (New York, W.W. Norton, 2001), attempts to highlight connections between the medium's roots in the pre-digital era to its use in the arts today. The book is supported by a website on ArtMuseum.net that includes additional in-depth information.

The book and website are part of an ongoing project that is guided by two underlying, interrelated objectives. The first is to offer a working definition of interactive digital media that makes explicit the most radical, and potentially transformative, aspects of the form. The second is to suggest that contemporary new media practice should be grounded on an appreciation of the historical interplay between the arts and sciences that gave birth to this medium. The book presents the conceptual development of interactive digital media through the writings of pioneering figures in both the arts and sciences, dating back to Richard Wagner and the Futurists on the arts side, and to Vannevar Bush and Norbert Wiener in the sciences. By proposing a vocabulary and framework for critical discourse about digital multimedia, and by basing this effort on the landmark achievements of multimedia's pioneers, we hope to help digital media achieve its potential.

In the wake of post-modernist practice, computer-based media has resisted definition -- and for good reason: definitions are confining. They reduce the range of potential in the object defined by drawing attention away from what lies outside the wall of definition. This is a particular concern with new media, because one of its attractions is its fluid, multifarious character, its permeable walls. Digital media's peculiar nature challenges traditional categories; this in itself is an aspect of its radical character. But we believe that there is value in proposing and discussing alternative definitions of digital media -- even if these definitions are contingent, bracketed by circumstances. In fact, it may be best to regard them as contingent, because our experience with digital media is so fresh, and where it leads so unclear. The definitions of today will inevitably be replaced tomorrow, as new applications for digital media emerge over time.

Definitions are meant to establish a shared vocabulary that can focus argument -- and often, covertly, to achieve a politically motivated purpose. Our purpose is overt: If, as Marshall McLuhan suggests, we literally construct the world we inhabit through the design and deployment of our media technologies -- because they enable certain behaviors while discouraging others -- then the social and political ramifications of how we define and address the emerging digital media are undeniable. By identifying a subject's key characteristics, we begin to say what it is and what it is not. For digital media this is particularly critical; if the digital arts community does not lead the discussion about how to define digital multimedia, and the types of behaviors it should or shouldn't encourage, other interests, like governments and corporations, will force a definition on us.

The interests of for-profit entities rarely coincide with those of the creative community. In the case of digital, the multinational media corporations have made clear that their intent is to maintain the legacy paradigms of 20th century media (which are hierarchical, broadcast-based, and author-centered) rather than support the emergence of challenging new media forms (which are, at their best, rhizomatic, peer-to-peer, and interactive). It is in their interest to force compromises from the technology that will protect their traditional businesses -- compromises that effectively gut the most democratic, and creatively engaging, aspects of digital media. If it was up to Disney, 21st century media devices would do no more than act as delivery platforms for the media formats of the last century.

Today's situation is strikingly different from the way new media forms have emerged in the past. Early in the last century, Picasso and Braque were free to invent collage in the privacy of their garret. Their materials were cheap, and the works persisted whether a viewer looked at them or not. Eventually those works found their way to viewers who came to appreciate them, and other artists were able to propagate the collage format however they chose -- there were no technical or intellectual property restrictions to deter them. History has countless examples like this, where artists in isolation, with little to no resources, and little care for the ways of business culture, pioneered forms of expression that came to be widely adopted.

Digital multimedia, in contrast, requires a massive technical infrastructure and user base to come into being. It is expensive. It demands standards agreed to by a broad community. While every artistic innovation that has been embraced by society requires interdependence and collaboration between participants (whether it is overt or invisible, conscious or unconscious), digital media calls for a far greater level of coordination, planning, and deliberate resource commitment than what we are familiar with from the past.

For this reason, there is a need for a definition of digital media that brings attention to its most radical characteristics. If Viacom trumpets the claim that click-to-buy TV shopping networks express digital media's greatest potential, we need a clear way to say why that is not the case.

Much has been written about narrow aspects of the digital media experience. However, little critical work has been done to show how these separate aspects combine into a whole. We wondered if we could identify the core principles that, when bound together, articulate the inherent capabilities in digital media that lead toward new forms of personal expression. We wanted to draw a line between the mainstream media forms of the past, and a possible future. Though the formal implementations of digital media are still in development (and will continue to be, relentlessly, given the freedom to do so), we identified basic concepts that persist, regardless of the technologies being used by an artist or engineer in a specific situation. Could these concepts suggest a trajectory for future development, and provide a way to measure if digital media is achieving what it is capable of?

We identified five characteristics of new media that, in aggregate, define it as a medium distinct from all others. These concepts set the scope of the form's capabilities for personal expression; they establish its full potential:

* Integration: The combining of artistic forms and technology into a hybrid form of expression.

* Interactivity: The ability of the user to manipulate and affect her experience of media directly, and to communicate with others through media.

* Hypermedia: The linking of separate media elements to one another to create a trail of personal association.

* Immersion: The experience of entering into the simulation or suggestion of a three-dimensional environment.

* Narrativity: Esthetic and formal strategies that derive from the above concepts, which result in nonlinear story forms and media presentation.

Together, these five concepts offer a definition of digital media that pushes toward the technical and esthetic frontiers of the form.

Integration, of course, is the backbone of multimedia; the combining of different media into a single work is intrinsic to multimedia practice. In modernism, Richard Wagner made the first critical argument for synthesizing all media into a single, coherent expression -- to which he gave the name Gesamtkunstwerk, in 1848. While technology has always played a role in the development of forms of expression (since all media are technologies in their own right), beginning in the mid-twentieth century there was a deliberate effort to incorporate technology as a thing in itself into artistic practice. This work, championed most visibly by Bell Labs engineer Billy Kluver, made technology an explicit aspect of art. This led, in turn, to artists exploring the formal properties of computers, in order to make an art that is computer-specific. Because the computer can mimic all traditional media, it lends itself to artworks that blur the lines between disciplines, just as in consciousness the distinctions between different media forms (image, text, sound, movement) are less than absolute.

Interactivity is an overused word that is in danger of losing its meaning. However, as originally conceived by Norbert Wiener, Douglas Engelbart, and others, interactivity has extraordinary promise. The term needs to be reclaimed from those who abuse it (by using it to describe home shopping TV channels, for instance). By interactivity we specifically mean: the ability of the user to alter media she comes in contact with, either alone or in collaboration with others. Reading a text is not an interactive experience; interactivity implies changing the words of the text in some way -- adding to them, reorganizing them, engaging with them in a way that effects their appearance on the screen. Digital media is inherently dynamic, changeable. Interactivity exploits this quality, and encourages a creative engagement by the user that leaves its mark on the artwork. Just as a conversation is a two-way experience that effects both parties, interactivity is an extension of our instinct to communicate, and to shape our environment through communication.

Hypermedia may prove to be the most profound contribution that the computer has made to esthetics. By making a persistent link between media objects, the user can now easily share his private path through them. Never before has it been so easy to make your own non-linear method of moving through ideas and information available to others. At the same time, using hypermedia, all traditional media forms intrinsically have the same weight. By writing links you decide how to place emphasis on one media object in relationship to another; context determines relative importance. Text leads to image leads to sound in a way that suggests their significance vis a vis one another, just as in the mind.

But while hypermedia is potent in and of itself, without interactivity hypermedia would be limited to a way of browsing extant items, rather than engaging directly with them. Interactivity is what empowers hypermedia, making it more like the experience of consciousness encountering the world. In life, one thought leads to another, which leads you to your notebook, where you reread a line of text, then cross out one word and replace it with a different one. Without interactivity, hypermedia would place you in a state of continual passivity, frustrating your impulse to engage with what you come across.

Like hypermedia, immersion is a digitally enabled method for mimicking an aspect of consciousness. The arts have long been concerned with accurately reflecting private sensual perceptions. The history of each art form is replete with movements that claim this as their objective; similarly, integration has been led by the desire to combine art forms in a way that reflects our sensual apprehension of the world. Digital technology allows us to pursue this impulse even further through the creation of fully realized virtual environments. Immersion, of course, requires the integration of artistic forms. As with hypermedia, without interactivity an immersive experience would seem confining -- like sitting in your living room unable to take a book off the shelf. Also note that, even when digital media does not suggest a convincing three dimensional virtual space, it encourages the use of spatial metaphors for the arrangement of information. One obvious example is the Web, which lends itself to architectural or geographic methods of "navigation," rather than adhering to linear forms of organization. In this context, immersion's reliance on hypermedia is immediately apparent; without the ability to link in a non-linear fashion between media objects, the immersion suggested by the Web would not be possible.

This inter-reliance between these key characteristics culminates in the wide range of non-linear narrative forms that digital media lends itself to. Our methods for self expression grow out of an ongoing collaboration with the tools we use to give that expression a recognizable shape. Working with these tools, we find ways to capture nuances of personal experience so that we can share them with others. Before digital technology, our tools led us toward linear modes of expression. However, the dynamic nature of databases opens up possibilities for alternative narrative structures that come closer to replicating the internal associative tendencies of the mind. Artists like Lynn Hershmann, Roy Ascott, and Bill Viola saw this potential early on, and have explored approaches to narrativity that make full use of integration, interactivity, hypermedia, and immersion in their digital artworks. The narrative forms pioneered by these artists, and the many others who share their interests, are effectively blueprints for digital communications in the coming century.

*

One reason that digital media have resisted definition to date is that they cannot be adequately described by their materials. Bits of data are elusive things. Because those bits of data are being recombined in media objects through an endless variety of devices, using a constantly expanding range of interfaces, it is a challenge to describe this emerging medium as you would describe traditional forms, such as theater or music. Theater is something that happens on a stage in front of an audience. Music is the organized shaping of sound for esthetic purposes. But new media can come at you through the Web, CD-ROMs, kiosks, CAVE's or other virtual environments, among a seemingly endless string of delivery systems. New interfaces are perpetually in development; many more devices are yet to come.

When we began our project nearly four years ago, Randall and I did not have the benefit of Lev Manovich's landmark book, The Language of New Media (Cambridge, MIT, 2001). Lev, grappling with similar questions, chose an instructive though different route toward an answer. One notable aspect of this new medium is how it can be accurately described in many ways -- like an elephant by a group of blind men -- and that different definitions need not conflict with one another. (In fact, Lev's definition and ours are likely complementary.) This is a consequence of the new medium having encompassed within it both the technology of wired communications and the legacy of modern media forms. New media is the grandchild of the telegraph and the photograph. It is an offspring of unlike disciplines that can sustain within itself the legacy discourses of its constituent parts. Art theory, communications theory, issues of governance and regulation, telecommunications business practice, media business practice -- these are among the intellectual threads that remain relevant. Which only adds to the challenge of definition.

Lev's approach is to look past the delivery devices to the medium's substrata. He focuses on the essential elements that combine to constitute digital media -- the ones and zeros, the bits -- to identify the esthetic attributes of media composed by databases. There are esthetic and social consequences to the fact that we can now shape all media, in an endless variety of formal presentations, from the same fundamental stuff. Ones and zeros give us the opportunity to recast the same content in a multitude of skins, each as an unique experience in itself. At the same time, our entire media record is being digitized, with implications that are only beginning to be addressed. Lev recently dubbed the examination of this territory as the field of "info-esthetics," a suggestive name that introduces art theory to the inner workings of databases. This is important terrain to explore.

But there is one notable drawback to this approach. As technology progresses, and all media forms get digitized and are indexed as bits in databases -- including text, music, images, video, etc. -- the distinction between the dominant forms of traditional media and the new forms enabled by digital technology becomes blurry.

When Moby Dick is delivered to your PDA, does that make it a work of new media? While the delivery system might be of 21st century vintage, the work itself -- the words of Melville -- remains stubbornly of the 19th. If it is relevant that the novel has been saved in digital form at one time or another during the production and distribution process, then the copy of Moby Dick now on my bookshelf should also be considered new media, because the pages of my paperback edition were typeset on a computer. Digital production has been standard in book publishing for more than a decade. Some might say that the critical difference is the surface material the words actually appear on at the end of the production/distribution process; if the words are printed on paper then it's old media, but if the words appear on a screen it becomes new media. Today, certainly, the difference between the two is significant. But what about in twenty years, or sooner, when the technological challenge of electronic paper has been met, and all texts are read on digital devices with pages that effectively replicate today's hardcover book?

Focusing solely on the bits does not address the need for a critical framework that distinguishes between digital facsimiles that mimic the experience of pre-digital media, and emerging media experiences that are uniquely digital. Why does this matter? Because the specific implementation of digital media is still in play. If the public is satisfied by so-called new media that does no more than replicate the old, than we will have missed an extraordinary opportunity to enhance our tools for communication.

Our attempt at a definition started from the opposite direction than Lev's. We began by considering the user experience, and identifying the types of behavior that digital media enable -- particularly those that are less available, or unavailable, in other media forms. Rather than using a microscope to dissect the atomic structure of the digital object, we turned a telescope to the night sky of new media to search for patterns of activity. With a telescope trained on the historic work of pioneering engineers and artists, clear patterns do indeed emerge.

*

For the purpose of this project, the term "digital multimedia" seemed to be the most appropriate -- rather than "new media," "digital media," etc. -- because it emphasizes the form's continuity with efforts in the arts that came before. The word "multimedia" was coined by artists in the 1960s to describe avant-garde practices that not only mix diverse media, but also emphasize audience participation, non-linear narrative structures, and indeterminacy. There is a line in the development of computer-based media that runs parallel to an important trajectory in modernism. We want to make that connection explicit.

This is not to say that digital multimedia grew out of a cohesive, carefully coordinated strategy. But looking back, you can identify a few consistent themes that drove the medium's development over a half century. These themes were pursued concurrently with other, at times conflicting, objectives. But in retrospect the extent of a consistent vision shared by multimedia's pioneers is quite profound. Eventually their diverse efforts coalesced into a meta-medium, to borrow a phrase from Alan Kay. Kay is the man who tied the loose threads of digital multimedia together in the late 1960s, by designing the prototype for the first true multimedia computer, the Dynabook.

Vannevar Bush began it all by proposing a mechanical device that operated literally "as we may think." The challenge, as he discussed it in his famous article of 1945, was to create a machine that supported the mind's process of free association in the act of creation. This aspect of Bush's hypothetical machine, which he dubbed the memex, tends to get overlooked today. What gets attention instead are the many ways the memex foreshadows the personal computer -- particularly its ability to call up media objects from a database. Bush did not use the word "database," because the memex, as he described it, was not a digital device. It was analog: a desktop and storage space that gave access to microfilm, audio recordings, photographs, and movies. It was, in a way, a kind of library -- but with a crucial difference. Libraries arrange information linearly. Bush, however, was interested in rearranging information according to the idiosyncratic paths of personal association that each individual invents during the creative process. He wanted a machine that encouraged spontaneous, associative, stream-of-consciousness thinking, and then left a trail of that thought process behind so that it could be retrieved, not only by the individual who created it, but by others as well. In this way, the memex would allow people to share their private, unconsidered thoughts as they leap between ideas moment by moment.

Bush was interested in identifying a central aspect of consciousness, and making a device that effectively expanded consciousness through mechanical means. If you look at the history of the personal computer from this perspective -- as an ongoing project to create a media machine that enhances the intuitive, associative tendencies of consciousness -- it connects digital media inextricably to important currents that run through modernism.

Bush had taken, essentially, an esthetic position -- an esthetic position that shares remarkable qualities with some unexpected bedfellows. These are contemporaries with whom Bush is never associated, particularly as he was FDR's chief science advisor and the architect of the military industrial complex. Still, as the person who proposed that information should be organized and saved mechanically in a way that captures the spontaneous movement of the mind, it is inevitable that he should be grouped with others who shared similar interests in mid-century.

For example, during the 1940s Charlie Parker was pioneering a new musical vocabulary based on spontaneous improvisation -- one that went far beyond the method established by Louis Armstrong. Parker's radical approach to improvisation, the charts be damned, placed non-linear associative thinking above all else in jazz, and led to the free jazz of John Coltrane and others in the 1960s and 70s. In painting, Jackson Pollock was taking a similar approach, dripping paint in loops following the dictates of his spirit, never following a plan or a sketch. The privileging of spontaneous action was central to Pollock's practice. In literature, during these same years, Jack Kerouac pursued a method of "spontaneous bop prosody" -- as he called it -- that led him to write novels that captured the movement of his mind moment-by-moment in the act of creation; a steady stream of honest personal observation that used associative thinking as its central organizing principle.

The prim bureaucrat Vannevar Bush might have been surprised to find himself in such unkempt, but august, company. However, looking back the similarities between Bush and the mid-century American avant-garde are obvious. They shared an esthetic that treats the individual's private impulse as primary, and that gives people permission to act in a non-linear, irrational way, as society would define it. Bush's interest was to enable each of us to shape data into the form that serves us best, rather than to conform our private thought process to an organization set by others. This opposition between self and society is not absolute, of course (though in mid-century the tension between private impulse and social conformity was an intellectual flash point, especially because of the threats of Fascism and Stalinism, on the one hand, and the theories of Freud, on the other). That digital media can trace its birth to the intent to mine this opposition, however, is significant.

Bush's vision inspired a generation of computer pioneers in the 1960s, and led directly to the personal computer. Douglas Engelbart expanded on Bush's premise by designing an oNLine System that would "augment human intellect," as he put it, based on the insight that the open flow of ideas and information (as represented by texts and pictures) between collaborators was as important to creativity as private free association. At the same time, J.C.R. Licklider envisioned universal networked access to the full "library" of human knowledge. This idea led him to spearhead the early development of the Internet while he was director of the government program, ARPA. Soon after, Ted Nelson followed with a proposal for a "hypermedia" system (he coined the term) that would fulfill Bush's objective to arrange materials from this "library" in a manner that reflects how the mind moves freely from one thought to another.

Central to all these efforts was the notion that the user should not only have access to media objects, so she can organize them as she pleases, but that the computer user should also be able to interact with media objects, and change them to suit the needs of the moment. Editing and recombining digital media was seen as essential to the utility of the computer. Licklider, in his seminal article "Man-Computer Symbiosis," proposed that the computer should act as an extension of the human capabilities for cognition and communication -- which includes, of course, the manipulation of media. Engelbart's oNLine System was designed specifically for the collaborative manipulation of digital media over a wired network. In keeping with Bush's vision of the memex as a way to enhance creativity, these pioneers insisted that the computer user's ability to interact with and change media should be as great as possible. Tim Berners-Lee has often said that he considered the edit function in the first Web browser to be just as important as the ability to link between Web pages; for the Web to be successful, he felt it essential that each reader could also be an author, able to annotate Web pages by adding "private links."

This approach to interactivity paralleled currents in the avant-garde, particularly in performance. In 1948, John Cage introduced the idea of live performance as unscripted event, in which the audience encounters people, objects, and activities within a defined space, in surprising juxtaposition to one another. The audience is encouraged to become creative participants in the work of art as it occurs. This type of performance, which Allan Kaprow later named Happenings, shared many concerns with the way engineers were shaping online interactive environments. Both engineers and artists were addressing the question: how do you encourage the appropriate dynamic encounter between people within a framed situation? And they reached a similar conclusion: give the user/participant as much freedom to act as possible.

Implicit in Bush's memex is the suggestion that a mechanical device can replicate the intimate movement of the mind at play, by representing media objects of all kinds in any order, as the user desires. From this, it follows that a computer might one day effectively mimic the encounter of consciousness with the world through the senses, by arranging media objects in a way that mimics reality. Though Bush himself did not make this leap, engineers influenced by his vision in the early 1960s did, and none more profoundly than Ivan Sutherland.

Sutherland was the first person to propose that bits and bytes could be represented as three-dimensional virtual environments. In his article from 1965, "The Ultimate Display," he began with the idea that by digitizing information -- transforming it into ones and zeros -- all data became subject to the graceful manipulations made possible by mathematics. This, in turn, invites the computer programmer to shape data into a three-dimensional form that mimics the way we encounter information in the physical world. Like Bush, Sutherland's approach to the formal arrangement of information is essentially an esthetic stance. This particular esthetic stance can be traced back to the mid-19th century writings of Richard Wagner, which declared that art should do its best to recreate the full, multi-sensory engagement between the self and the world. To facilitate his vision, Wagner reinvented the conventions of the opera house, and in 1876 opened the Festpielhaus Theater in Bayreuth, Germany. It was the first modern theater to employ Greek amphitheatrical seating, surround-sound accoustics, the darkening of the house, and the placement of musicians in an orchestra pit -- all to focus the audience's attention on the dramatic action, and transport them into an illusionary world staged within the proscenium arch. Wagner's call for an immersive "collective artwork" that fuses all the arts into a single expression -- his "Gesamtkunstwerk" -- is echoed in the last paragraph of Sutherland's 1965 paper:

"The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked."

Sutherland presented this paper at an engineering conference, and it was first published in a technical journal. But it is hard to ignore how much it reads like a manifesto written by an Italian Futurist. There is, in fact, a remarkable similarity between the tone and intention of articles by certain computer media engineers and fiery artistic manifestos. The modernist imperative to "make it new" (in Pound's famous phrase), and the belief that society will be transformed as a result, is very much present in writing by computer scientists. Digital multimedia may well force us to reconsider the entire historic arc of modernism, including its supposed end, since the esthetic stance of modernism has become increasingly relevant in response to digital media.

When Alan Kay designed the prototype for the Dynabook, in the late 1960s, the intellectual foundation was in place for a digital multimedia that synthesized all existing art forms, and presented them in an environment that enabled meaningful interactivity and hyperlinks. With the requisite processing power, it would eventually incorporate Sutherland's experiments with three dimensional representations. This meta-medium, to use Kay's term, carried with it specific, idealistic attitudes and intentions about human creativity and communications. It reflected a commitment to media forms that are nonhierarchical, open, collaborative, and emulate the free movement of the mind at play. It is, in sum, an extraordinary vision.

*

Randall and I started our project by identifying the types of behavior digital multimedia encourages that are most unlike traditional media forms -- like interactivity, hyperlinking, and virtual immersion. We then went back to original source texts by the engineers and artists who introduced and developed these concepts. For the anthology we organized these essays in a way that draws attention to the evolution of these concepts, while at the same time suggesting connections between artistic practice and the inventions of engineers.

It became apparent during this process that nearly all of the people included in the anthology share a strong interest in the specific way that the world is apprehended by consciousness. In their articles, each of them would somehow circle the questions: how is it that we perceive what we perceive, and how can our tools for expression replicate the experience of perception?(1)

These are the questions behind the musings of Vannevar Bush, who recognized that the amount of information being published in scientific journals had exceeded the ability of any scientist to keep pace with his field, and so required a more efficient and intuitive technology for organizing information, based on the inherent mechanics of the mind. The same questions drove JCR Licklider to consider how a computer might be designed to act as a symbiotic extension of human activity, as an agent that expands the work of consciousness by offloading the most repetitive aspects of the creative act to a machine. Douglas Engelbart designed his oNLine System to augment what he identified as the natural inclinations of the human intellect during the creative process, including the tendency we have to share ideas and be inspired through collaboration. Ivan Sutherland's notion of a three-dimensional representation of reality was based on the assumption that the virtual replication of perceived experience is intrinsically of value; that the closer we come to capturing how the mind engages the world through the senses, the more powerful our ability to express our imagination becomes.

For artists throughout history, the effort to accurately mimic perception has often been the pivot around which formal innovation occurs. This became especially true during modernism, as generations of artists founded a parade-worth of new movements, and wrote volumes of brick-throwing manifestos, all with the stated purpose of more accurately capturing, and reflecting, what it is like to live in contemporary times.

Particularly relevant to the history of digital multimedia, Richard Wagner was obsessively concerned with initiating an art form that actively engages all the senses in a convincing representation of an immersive experience. And applying the technology available to the arts at the time to do so. Toward that end, he proposed a theater that incorporates all the arts -- music, poetry, acting, movement, visual imagery, stagecraft -- in equal measure. Wagner's concept of the Gesamtkunstwerk carries within it an implicit answer to the question "How can our tools for expression replicate the experience of perception?" His intent was to find the most effective way of transporting the audience into a believable representation of another world, to engulf the spectator in a higher reality by manipulating the sensual mechanisms that communicate mundane existence to the self.

Wagner's answer presages the technical innovation of Ivan Sutherland and those who followed his lead into creating virtual environments -- including Myron Kreuger, Scott Fisher, Dan Sandin, Char Davies, Jaron Lanier, and others.

Key to this trajectory in modernism is the related question: how does the form of an artwork shape our perceptions, and focus attention on aspects of experience that had not previously attracted much consideration? With Ulysses, James Joyce expanded the notion of what constitutes an accurate observation of the self. His method was to give formal shape to the musings of conscious thought that had been outside the bounds of what previously had been shared between people. Cezanne similarly introduced new elements into personal expression. His paintings announced that artists ought to probe the fundamental properties of vision, and that the sensitivities of the eye to color and composition should be exploited. By doing so, the artist challenges received notions about how we see, and invites consciousness to appreciate a wider range of the visual.

John Cage is similarly interested in countering habitual modes of thinking through the formal inventions of art. This is best illustrated by his infamous piece from 1953, 4'33", in which the pianist David Tudor remained silent at his instrument for the prescribed duration of time. While the piece was initially met as a scandal, Cage's intent was to encourage the listener to contemplate the passing of time freely while listening attentively to the random sounds of the concert hall. By focusing attention on the fundamental elements of the music performance, in an unconventional manner, Cage sought to provoke the audience into a creative engagement with the artwork that challenged received notions of what music is, and how it should be made. For Cage, consciousness reaches its greatest freedom when liberated from the shackles of convention; his experiments with interactivity, which introduced the serendipity of chance into the making of an artwork, were driven by his interest in challenging habitual modes of perception.

Cage's approach -- to awaken perception by increasing an audience's participation in the active creation of an artwork -- has become widely adopted in the arts, and has profoundly influenced digital multimedia. Artists such as Myron Krueger, Nam June Paik, Lynn Hershman, Bill Viola, Marcus Novak, Kit Gallaway and Sherrie Rabinowitz have placed inteactivity at the center of their digital media works. In these cases, among many others, the introduction of chance encounters into the art experience provokes the audience/participant to reflect on her own method of perception, which transforms the familiar into something fresh.

By asking how it is that we perceive what we perceive, and how to best share these perceptions with others, we are effectively asking another, deeper question: what does it mean to be human? Stressing certain qualities of consciousness carries with it an opinion about what specifically constitutes our individuality. When we represent aspects of ourselves through art, or other forms of expression, they become recognized by society as worthy of attention. Conversely, when an aspect of ourselves falls outside of our ability to represent it to others, society is not in a position to recognize it, or respond to it. We know that the tools we use for expression expand our ability to identify aspects of our experience and share them with others.

Digital media will undoubtedly become the dominant form of communication in this century. The formal properties that digital media acquire as they develop carry with them assumptions about what tools are needed for effective communications. These assumptions are based on philosophical notions about who we are as individuals, and how we best express ourselves. These questions, which traditionally belong to the realms of esthetics and spirituality, have become increasingly political because of the issues raised by digital media.

As a society, we are now building the infrastructure that will support the full implementation of digital communications. We are deciding the rules that determine what it can do and what it can't; what behaviors it will encourage and what it will disallow. If it follows that the forms we use to express ourselves carry with them an implicit notion of who we are and how we communicate, then, in important ways, through the building of our media infrastructure we decisively influence what it will mean to be human in the 21st century. What other issues carry such political significance?

With our definition, Randall and I tried to capture the aspects of digital multimedia that make the most complete use of the computer's potential for personal expression. The medium's potential lies in how it makes widely accessible the tenets of modernism most concerned with accurately reflecting the minute workings of consciousness in our communications with one another, and the use of formal methods to free us from habitual modes of thought. This lineage in modernism valued an extreme subjectivity, because it recognized that social forces act against the needs of the individual in a relentless push toward conformity. Digital multimedia, with its emphasis on enabling idiosyncratic personal expression, can be an important counterbalance to social tendencies that drive toward homogeneity. But only if we identify and defend those qualities of multimedia that support the creative spark of the individual.

----

(1)Footnote: This is a different question than: how do we know what we know? It is only indirectly an epistemological concern. Rather, it begins with the practical assumption that by encountering the world we form ideas and opinions, and it narrowly examines what takes place during that encounter.)