The Streitz perspective: computation is ubiquitous, yet must be designed for human use (a festschrift for Norbert Streitz)

Over the course of his long and fruitful career, Norbert Streitz has been a continuing advocate for a particular way of looking at, and designing for, computational systems that are embedded in the world.

Streitz’s research activities have covered a wide range of areas: Cognitive Science, Human–Computer Interaction, Hypertext/Hypermedia, Computer-Supported Cooperative Work (CSCW), Ubiquitous Computing, Ambient Intelligence, Interaction and Experience Design in hybrid worlds, and Smart Cities. One of the basic ideas was the integration of real and virtual worlds [15, 26]. He and his AMBIENTE-Team became well known for the development of Roomware^®, [5, 15, 16, 28, 29] the integration of buildings and furniture with information technology (e.g., interactive tables and walls) and for the design of Smart Artifacts in ambient environments [20, 22, 23, 25]. The roomware components were developed in close cooperation with industry and won several design prices.

During all this work, an important shift has been in place. The change in which computation is handled, moving from large centralized computing, to personal desktops, to small computers distributed throughout the environment has been a part of Streitz’s research agenda since the very beginning of this movement. Having been a visiting scholar at Xerox PARC in 1990, and thus being influenced first hand by Weiser & Brown’s vision of a “calm technology” [30–32] as a uniquely humanistic way of developing and deploying ubiquitous computing technology, Streitz led multiple teams over the next several years to capitalize on a vision of where the future of computing would go. In each of these systems, whether capturing a model of how personal work would be organized and managed in the future, or building ways in which ambient information can be expressed, managed, and organized, a deep concern for the human aspects is clear.

The goal of this paper is not to give a complete list of all the work Streitz has been engaged in, e.g., his early work in cognitive science and on mental models [e.g., 7] is not covered here, but to make the compelling observation that throughout his career, design for humanity has always been a hallmark of his research. Another attribute of Streitz’s research career has been to continue to scale his ideas about working collaboratively. With each successive system, the scale, reach, and extent of the work continued to expand to encompass more work, more styles, more content, and ultimately building systems that can handle very large problem spaces with large communities to enfold and use them, all with the initial goal of creating calming, supportive, highly functional environments [8, 10, 12–17, 23, 27].

Streitz began working in the area of hypertext and hypermedia in the late 1980s when that field was flourishing [6]. As a participant in many of the important meetings^{Footnote 1} that were seeking to define and formalize hypertext as an information and communication technology long before the Web existed, Streitz was drawn toward solving the very real problems associated with people actually using the rich media of hypertext in group activities. This led to his involvement in both the hypertext and the Computer-Supported Cooperative Work (CSCW) communities. As an important player in both domains, it was perhaps inevitable that early in his career, Streitz would begin to investigate the use of hypermedia in a sharing-work context. An early successful system in this area was the SEPIA system.

SEPIA

In the SEPIA (Structured Elicitation and Processing of Ideas for Authoring) system [2, 9, 17, 18], the goal was to create a cooperative hypermedia authoring environment that used a “…hypermedia data model with composites, sophisticated and comprehensive authoring functionality and support for a new rhetoric for cooperative work.” (p. 11). The underlying hypertext model supported a wide range of typed nodes and links [8, 17]. The initial impetus for this work began by identifying the cognitive processes of authoring [18] and the human challenges involved in making the hypermedia systems actually work in real practice [2]. In SEPIA, they pioneered the model of interacting problem spaces (with spaces such as a “content space” a “planning space” an “argumentation space” and a “rhetorical space”) as the mechanism for modeling the dynamics of authoring over a number of different perspectives [18]. They introduced the concept of “activity spaces” as a way to create hyperdocuments through a “construction kit” approach within an activity space. SEPIA has multiple ways to support a number of simultaneous authors and was demonstrated in use as a sophisticated argumentative and rhetorical tool for many different topics involving the parallel construction of complex document structures.

This early concern about how authors could work together to create a complex document quickly showed the limits of doing everything through the hardware and meeting spaces of the time, and led to the development of the Dolphin system.

Dolphin

Dolphin is an electronic meeting-room system to provide computer support for a number of different meeting types. Through the design features of Dolphin, face-to-face meetings were supported with a large interactive electronic whiteboard, mainly with but also without networked computers [14]. Extensions of these meetings could include remote participants at their computers, also provided with an online electronic whiteboard. DOLPHIN supported the creation and manipulation of informal structures (e.g., free-hand drawings, handwritten scribbles), mainly used in brainstorming as well as more formally specified structures (e.g., hypermedia documents with typed nodes and links) [3, 4, 10, 14].

The important design features of Dolphin were the close integration of the large physical, multitouch room-sized displays with local and remote participants. These features, which are now widely accepted as essential, were groundbreaking in the early and mid1990s when Dolphin was being developed and reported.

Streitz was profoundly influenced by Mark Weiser’s notion of the “disappearing computer” [30] and helped to establish and disseminate the corresponding ideas [19]. While being active at a global level, he especially helped to propagate this idea in Europe. Part of these activities was the launch of the proactive research initiative “The Disappearing Computer (DC)” funded by the EU in its Future and Emerging Technology (FET) program. The DC initiative (http://www.disappearing–computer.net) became a cluster of 17 projects covering a wide range of topics [19, 21]. Streitz [11] also introduced the distinction between the physical disappearance of computer devices being achieved by making the computer-based parts very small so that they can fit in the hand, be interwoven with clothing, be attached to the body or even implanted in the body and the mental disappearance of computers, facilitated by becoming “invisible” to the “mental eyes” of the users. It is worth mentioning that the disappearing computer notion is closely related to the approach of Ambient Intelligence [1, 12, 24, 33].

Roomware^®

In the late 1990s, Streitz and his AMBIENTE-Team started to develop smart environments which were populated by what they called Roomware^® components [15], a trademarked term. The first prominent example was the i-LAND environment, an interactive landscape for creativity and innovation [16]. It consisted of a well-thought collage of an interactive wall (DynaWall), an interactive table (InteracTable), several ConnecTables and chairs (CommChairs), all being connected and used with the gesture-based cooperative software BEACH. There were several generations of Roomware [5, 15, 16, 28, 29], and the latter being developed in close cooperation with industrial partners, also winning several design awards in the process. Some of them, e.g., the InteracTable, were made available as commercial products through the spin-off companies and achieved some market success, a remarkable achievement in the ubicomp/pervasive computing research world where the vast majority of research was rarely transferred to large-scale business deployments. Others, such as the DynaWall, are only now becoming common in work environments.

Ambient Agoras

While the initial roomware developments focused on the support for productivity work and meetings including brainstorming, argumentation, and design, the next set of developments investigated informal communication and means of providing awareness and notification about the presence and mood of people. This work in the Ambient Agoras project included so-called ambient displays as, e.g., the Hello, Wall [22]. Using sensors to locate people, e.g., in the hallways, raised the issues of protecting privacy which was provided by a device called the Personal Aura [20, 23, 25].

From rooms to buildings

As the focus of Streitz work shifted toward larger assemblies, he began working on intelligent buildings. The notion of Cooperative Buildings [15, 27] was already the starting point of the AMBIENTE research division at its beginning. This group, with the research goal subtitle of “Smart Environments of the Future,” was founded in 1997 at GMD-IPSI (later Fraunhofer IPSI) in Darmstadt, Germany, to initially build smart rooms and then understand the logical extension of smart rooms as physical collections of artifacts and systems at the next level of abstraction populating buildings. The application area has been based on the notion that “the world around us is the interface” for people’s activities [15]. This important element (designing and building systems that leverage expectations about virtual systems based on their real counterparts) is an old tradition that has been made new with the virtualization of information in the roomware approach. The extension concerned also the application domains. While initially addressing work environments, he and his team extended the scope to smart home environments including leisure and gaming activities as in the AMIGO project, and to public spaces, the latter providing a point of departure to the next level, namely, urban environments [12, 13].

In many ways, a key driver for Streitz’s work has been to continually seek out larger and more comprehensive visions of how people can work and live together. In SEPIA, the focus was on relatively small groups—small teams working together. But over the years as technology has progressed, a continuing expansion of goals becomes clear. And now, it is clear that what is needed is a vision to align humans and technology into a coherent ecology. This work leads inevitably toward city-scale systems—a landscape where people enjoy everyday life and work, while having multiple opportunities to exploit their human potential and lead a creative life.

How can information technology support people in such a Humane City—as it is called by Streitz [12, 13]? The application of information technology in the context of future cities is often indicated by notions of Smart Cities or Ubiquitous Cities. They define new areas for the concepts of Ambient Intelligence and Ubiquitous Computing [12, 13, 24]. In a way, it is a natural consequence of extending the scope of work on smart rooms and smart and cooperative buildings to the next level of complexity and sophistication. Work, play, sharing of all forms at many levels of abstraction, but integrated with public spaces to cover comprehensively all activities related to living and working in an urban environment. One approach is to revisit the Greek notion of the “agora” and take this as a template for the design of future cities.

The research for Smart Cities works by envisioning a number of scenarios of city-wide organization. One such framework is the Urban Life Management [13], which was proposed as an umbrella scenario to identify appropriate research lines to identify deficits and future research agendas in these areas. A piece of this work is the Hybrid City consisting of a real city with its physical entities and real inhabitants and a parallel virtual city of counterparts of real entities and people. Nevertheless, there will be no complete match between real and virtual entities. Analysis of this kind examines all working entities along a continuous dimension with “real” and virtual as its end points, locating other dimensions as they are useful. One such dimension is the distinction between “local context” and “global context”, on the one hand, and between “individual activities” and “group/social activities” on the other hand.

The overarching goal of this entire line of work is to understand how to maximize the intelligence of the entire landscape, from individual, to the tools they use, to the social groups, buildings and the entire infrastructure of the city to support smart work.

The standard practice in technology research is often to follow the “technological imperative,” that is, to build out the technology first and fit human practice into and around the technological requirements. Many research systems have been constructed that were primarily “technology demonstrations” rather than complete environments that were designed to be sensitive to the actual needs of the people working (and living) within the environment.

When viewed as an arc of work from the very beginning, one of the dominant threads in the research and development projects of Streitz has been to always keep the social and cognitive needs of the human as part of the process. Whether in early hypermedia research where the issues centered on keeping the hypermedia systems understandable by their users [18] or in the development of ambient information displays that would calmly display information in the periphery of awareness [22, 23], the ideal of designing for real human uses has always been paramount.

The other important arc in Streitz’ career has been to always expand the range of the vision. Again, in the early days, the focus was on helping to support individuals and small teams with hypermedia systems. And, over time, the level of consideration expanded with each passing year and each subsequent system to incorporate more and more of the overall environment, be it for work, leisure, or entertainment. Work and living practices have changed with advances in technology—large wall-sized displays or interactive tables, once visionary prototypes in roomware systems [15, 16] are now growing increasingly common.

As you can see from the trajectory of Streitz work and the variety of papers in this issue from his colleagues, Streitz’s work has influenced a large number of researchers over a long period of time, and significantly, in a broad range of topics.

The papers collected here reflect the wide ranging impact on architecture, design, esthetics, user-interfaces, roomware, and the future of ubiquitous and ambient computing.

• Till Schümmer, Peter Tandler and Jörg Haake lay out their model for “The next generation business meeting: From i-lands to flexible meeting landscapes” giving an extensive overview of the evolution of meeting practices, as seen through the lens of meeting support technology, and carrying forward the work established by the meeting-room inventions and research from Streitz’s earlier work.

• Carsten Röcker also picks up from the work of Streitz themes of distributed working teams and awareness in his paper “Universal access to awareness information: Using smart artefacts to mediate awareness in distributed teams” by presenting a set of design recommendations for team awareness systems based on fundamental social and technical requirements.

• Carsten Magerkurth picks up another application area, i.e., games, in combination with the theme of keeping the person at the center of the design in his paper “Hybrid gaming environments: Keeping the human in the loop within the internet of things.” Hybrid gaming environments that blend ubiquitous computing with tangible user-interfaces and smart artifacts need their own special design guidelines. A particular game is used to illustrate these important design principles as well.

• Christian Müller-Tomfelde’s paper visits the problems of designing the specifics of interaction on roomware-style horizontal displays in “Interacting with Mouse and Touch Devices on Horizontal Interactive Displays.” Once seen only in research labs such as Streitz’ lab at GMD-IPSI (Darmstadt), tabletop surfaces are now widely available products. Design issues still raise their heads, however, and user-interface design for engagement still matters.

• Adrian Cheok discusses the importance of a different design esthetic, one taken from the Japanese cultural perspective, which has always been very close to the heart of Norbert Streitz having experienced it during his multiple visits in Japan, on “Kawaii/Cute interactive media.” This is a radical variant on traditional design stances, one that takes serious the human value point-of-view on the design of encompassing systems, as seen in the intelligent cityscape design work by Streitz and his colleagues.

• Georgios Metaxas, Panos Markopoulos and Emile H.L. Aarts discuss “Modeling social translucency in mediated environments.” As ambient intelligence begins to pervade our workspaces, social translucency (what information is shared and how visible it is, or is not) will be a driver in determining long-term viability of social, computationally augmented spaces.

• Gilles Privat in his paper “Phenotropic & stigmergic webs: the new reach of networks” introduces a novel model of networks that include not just actors and objects, but models the network-of-objects as a first-class citizen, worthy of design in and of itself. Such networks are a possible next step in the evolution of new kinds of computational fabrics that will support people working in larger groups.

• Daniel M. Russell argues in his paper Ubiquitous search for smart workplaces” that search is now an integral component of twenty-first century work practice. As such, a smoothly integrated search user experience that leaves workers in their flow state needs to be tightly coupled with the entire smart workspace, another element of the Streitzian smart workspace program.