Kihong Ku, Philadelphia University

Jonathan Grinham, Catholic University of America

Originally published: Grinham, J. and Ku, K., (2012). 4D Environments and Design: Towards an Appliance Architecture Paradigm, Digital Aptitudes + Other Openings, Proceedings of the 100th ACSA Annual Meeting Conference, March 1-4, 2012, Boston, MA

INTRODUCTION

In his book ‘Cognitive Surplus: Creativity and Generosity in a Connected Age,’ a study of the new media culture, Clay Shirky (2010) presents an anecdote that illustrates a paradigm shift within our modern culture. Shirky recounts a friend’s story of his four-year old daughter suddenly rising to her feet mid-movie and beginning a vigorous search behind their television screen. The friend, from his own childhood experience, assumed the child was searching behind the screen for the people she was seeing on-screen. When asked, ‘what are you doing?’ the child responded, ‘looking for the mouse.’ This anecdote is used by Shirky to represent the new media culture in which we live; a culture that, in many ways, is more perceivable to a four-year then it is to previous generations. Shirky states, “here’s something four-year olds know: a screen without a mouse is missing something. Here’s something else they know: Media that’s targeted at you but doesn’t include you may not be worth sitting still for.” The four-year-old protagonist represents a societal and generational shift from a culture of media consumers to a culture of media producers. More importantly, the four-year-old represents the inescapable future of a culture whose members expect malleable, interactive and user-oriented environments. Shirky’s story establishes the tone for this paper, which readily accepts that architecture, both professional and academic; currently exist within this new media culture.

The proliferation of household appliances with embedded microprocessors such as fridges, washing machines, or smart handheld devices such as the iPhone or tablet have changed the interaction space with computers beyond the computer screen and mouse. With the notion of ubiquitous computing which Weiser (1991) characterized as the ‘internet of things’ objects occupy both physical and virtual space (Dade-Robertson 2011). Within this context we examine interactive architecture to explore the convergence of robotics, architecture, and open source computing to ask the following questions: Can architecture actively and dynamically change physical environments in real time while becoming a social medium? Can architecture connect the virtual and the physical? Can architecture become an interface to connect what were once thought to be disparate ideas and worlds?

RESEARCH APPROACH

The first author established in the fall of 2009 the PARTeE (Prototyping in Architectural Robotics for Technology enriched Education) Lab as an interdisciplinary design group that explores the implications of interactive architecture through integrating computationally driven physical kinetic systems and components into buildings and spaces to meet changing human needs. The laboratory was initially funded by the Center for Creative Technologies in the Arts and Design at Virginia Tech. This endeavor currently continues at Philadelphia University and its outcomes and methods have been adopted and grown through a series of design studios and projects led by the authors.

Theoretical explorations and case study research on related projects such as the REEF project by Rob Ley and Joshua Stein offered initial conceptual insights. Practical explorations of architectural scenarios involved design experimentation and prototyping through model construction. Various digital media and techniques were utilized for rapid prototyping purposes such as 3D modeling with the Rhino software, laser cutting, solid deposition 3D printing and a variety of robotics technologies (e.g., photoresistors, thermosensors, LEDs, servomechanism, etc.) and architectural materials (felt, polystyrene, acrylic, wood, etc.).

THEORETICAL FRAMEWORK

Research in the field of user-to-user and user-to-object interactivity has established a large body of literature which describes the intricacies and evolution of the many studies of interactivity, responsiveness, and computational intelligence. In the field of architecture interactive design is popularly described as an emerging field, however, it is more accurately described as a ‘re-emergence.’ We must look at a brief history of interactivity as it relates to architecture to understand this re-emergence and its significance to the questions established above.

The term ‘interactive design’ was coined in the late 1980’s by Bill Moggridge of IDEO and Bill Verplank of Xerox. However, the first emergence of interactive design can be found in the studies of cybernetics in the late 1960’s and the formation and continued research of MIT’s Media Lab. In his 1969 article, Toward a Theory of Architecture Machines, Nicholas Negroponte, the founder of MIT’s Media Lab, asked, “Can a machine deduce responses from a host of environmental data?” (1969). This question and others developed in the Media Lab sought to realize the computer and its algorithmically driven logic as a partner or “associate” to its human design counterpart, ultimately developing a theory of “humanism through machines” (Negroponte, 1970). The exchange of data, introduces an essential element of interactive architecture—the feedback loop. Interactivity describes systems that have the ability for a participant (either human or computer) to exchange information while evaluating the received information against a regulatory system, rationalize about the data and produce a given exchange of data. Typically, these exchanges are considered to show intelligence. This reading of intelligence must be clarified. In its most root form the computer shows a form of intelligence, what is described as the ability to solve problems through a codified set of procedures or rules; however, unlike humans the computer is not aware of, nor able to reflect upon this action (Terzidis 2008). The need to program the intelligence by a human counterpart presents a fundamental problem in the study of interactivity. Tristan d’Estrée Sterk of the Office of Robotic Architectural Media & Bureau for Responsive Architecture [ORAMBRA] describes that early studies and development of interactive architectures struggled to find its foundation due to the architect’s inability to construct the computational and structural systems needed to realize the vast complexity of interactive architectures. Instead, the studies found residency in the fields of mechanical, electrical and structural engineering (2003).

Understanding how the architect approaches the opportunity of programming intelligence provides a two-fold framework for this paper.

Intelligence and Interactive Architecture

We first examine two approaches that show the programming of intelligence in an architectural context. Two studies began in the late 1990s. MIT’s House_n and University of Colorado’s Neural Network House established frameworks for understanding a ‘smart’ or intelligent home. Both studies recognized that a completely autonomous, pre-programmed intelligence has potential pitfalls in relation to decision-making and end user operability. As stated by both lead researchers of the projects, Stephen Intille and Michael Mozer, these systems carry a complexity that lacks transparency, are considered too complicated to be programmed by the user, require professionals to adjust and maintain the system and each user (home) requires customization to address the nuances of day to day decision making. For these reasons the two studies strived to provide a more efficient user-to-object exchange.

The University of Colorado’s Neural Network House uses an autonomous system; however, the research focuses on the capacities of a heuristic mechanism within the programming. In this scenario the house’s programmed intelligence is able to learn about the habits and needs of the user through a series of subtle user-oriented tests (i.e., if the lights are left on before entering a room, will the user immediately turn the lights on?). From this data the system measures the needs of the inhabitant against those of conserving energy, in what Mozer calls an ‘optimal control policy’ (Mozer 1998). This framework uses an algorithm to measure the dollar cost of energy conservation against the dollar cost of the relative discomfort of the user. The two data sets are evaluated and a decision is made based on the relative ‘cost’ of a system’s actions. The result is an autonomous home which is capable of learning the habits of its users, associating a real world cost to those habits and making a calculated decision based on opportunity cost.

MIT’s House_n takes a different approach to object-to-user exchange. In addition to the issues of automation identified above, Intille (2002) identifies the need to empower people with information that facilitates decision making while reducing the feeling of loss-of-control, which he explains can be psychologically and physically debilitating (Intille 2002). The House_n, therefore, approaches the smart home as a ‘teaching home’ (Intille). This scenario illustrates a pervasive physical computing system which instead of actively making decisions and producing given responses, uses algorithms to display an indicator of how the home could be working more effectively (i.e., an LED on a window indicating that current conditions would be a good time to use passive cooling). If the user responds to this indicator, the systems then projects information graphics about potentially more efficient configurations. This system works to both produce a more efficient control of the house’s energy as well as allow the user to make decisions which could be too complex for an algorithm to control and ultimately empowers the user to feel a sense of control.

The research of the PARTeE Lab identifies and adopts these two frameworks as important references in the design of user-to-object interactivity. Adapting these models, we developed ‘a2o’ a physical prototype and explain in the next sections how this system weaves object intelligence through a user-driven hierarchy that facilitates user-to-object interactivity. It is also illustrated how the use of social tools, such as Second Life (a free 3D virtual world where users can socialize, connect, and create using avatars, http://secondlife.com/) and Twitter (a social networking tool, https://twitter.com/) enhance user operability.

The New Generators and Open Source

To study the re-emergence of interactive architecture and its social implications, the PARTeE Lab studied the resultant effect of Negroponte’s ‘humanism through machines’ on our modern culture. In one way the re-emergence of interactive design can be thought of as a ‘self-fulfilling-prophecy.’ The study of interactivity seeks to create easier exchanges of information between users and objects. Therefore, one would hope, it could solve its own problem, ‘the architects’ inability to construct the computational and structural systems needed to realize the vast complexity of interactive architectures.’ The solution, or emergence, is the new media culture that has produced communication technologies capable of enabling and facilitating user-to-user interactivity, as well as interactivity between user and information at an exponential rate. Henry Jerkins, a leading researcher in the field of new media notes in his publication (Jerkins et al., 2006), ‘confronting the Challenges of Participatory Culture: Media Education for the 21st Century’.

According to a 2005 study conducted by the Pew Internet and American Life project (Lenhardt & Madden, 2005), more than one-half of all American teens—and 57 percent of teens who use the Internet—could be considered media creators. For the purpose of the study, a media creator is someone who created a blog or webpage, posted original artwork, photography, stories or videos online or remixed online content into their own new creations. Most have done two or more of these activities. One-third of teens share what they create online with others, 22 percent have their own websites, 19 percent blog, and 19 percent remix online content.

Thus interactive architecture resides in a new media culture driven by younger generations. Within the new media culture, a major shift is the decentralization of knowledge to online participatory / user communities. This shift produces a two-fold paradigm change that is essential to the understanding of the re-emergence of interactive architecture. First, it represents a culture, specifically ‘Generation I’ (internet generation) that has grown, or emerged, into an environment of interactivity. The new media culture’s communication technologies have enabled and facilitated user-to-user interactivity resulting in a new generation that has come to expect an open flow of data, social interaction and adaptable user-oriented devices, products and environments. David Marshall (n.d.), Chair of the Department of Communication Studies at Northeastern University in Boston, describes the new media culture, ‘These cultures, in their dynamic relationship with products, networks, hardware, software and practices are constantly changing in sometimes profound and sometimes banal ways’. Architecture, through its design processes, its adoption of computer software and its formation of global design communities, has become a nodal point in the complex network of information exchange within the new media culture. Not only has social media influenced the ideological concepts of architecture but also architectural form has become a host to a culture whose members expect malleable, interactive and user-oriented environments. Ingeborg Rocker of Harvard University’s Graduate School of Design explores social implications of Partick Schumaker’s early writings on parametric architecture (a design ‘style’ within which interactive architecture resides). She states, ‘Architectural and urban form were thus to be comprehended as an aesthetically condensed intelligence as the materialization of the logics of inhabitation, and ultimately as the materialization of the new social relations that those logics began to set forth’ (Rocker 2011). Second, the shift has produced a new perception of who the producers of information are and who possesses authoritative view of its content. This shift is especially important as a research framework adopted by the PARTeE lab. The new media culture, specifically open source hardware and software such as Processing, Grasshopper, Wiring and Arduino have increased accessibility to electronics and physical computing, and their user generated forums and ‘wikis’ have provided architects with a new capacity to design complex systems within interactive design. These computational open source technologies paired with the computerization of fabrication such as computer numerically controlled (CNC) cutting systems and 3D printers allow for a new and fertile architectural research platform— interactive design.

PROTOTYPING

Architectural Scenario

The physical construct, a2o [eh-too-oh], was developed as a full-scale prototype designed using Michael Fox’s classification of dynamic kinetic structure (Fox and Kemp, 2009). These systems are understood to be singular systems able to actively influence localized climates within a building system. In the case of a2o, the design was based on the narrative of a sun-shading interface and focuses on weaving autonomous decision-making intelligence with a user controlled feedback loop. During the course of this research another layer of ‘social-emotive’ interactivity emerged through the use of social media environments, in this specific case the Second Life virtual environment and Twitter. The weaving of these systems required an intelligent, user-oriented hierarchy which produced a series of rule based relationships to real-time sensory data and physiological and psychological needs of the user. However, before understanding this relationship we must first introduce the physical architecture. The actuation of programmed intelligence requires equal physical logic and necessitates interdisciplinary research; the result is new micro-morphologies within the study of architecture. Contemporary architecture can be understood to be the architecture of the diagram (Eisenman 2010). Within our research the superposition of the architectural diagram with the physical computing diagram results in an emergence and synergy wherein the computational structure informs the architectonics of the project.

When approaching the physical design of the prototype the team envisions a bottom up design for the physical construct within a layered hierarchical computational logic. This approach identifies multiple factors. First, a plug and play nodal design is adopted, narrowing the scope of the project from a building to a part of the building. We focus on a fenestration element that collects data from localized spatial and environmental conditions. Second, the part-to-whole diagram matches the computational logic diagram. a2o is composed of a series of units, or ‘pixels,’ containing dedicated sensors [proximity, haptic and light] and dedicated actuators [servomechanism, RGB LED, speaker]. Sensory data collected by each individual unit is relayed to a master controller - in this case an Arduino microprocessor – which controls an array of units. This master controller is itself a ‘slave’ to a master controller at a higher level resulting in a series of pixels within pixels. This laying of physical computing logic structure, described as cellular automata, allows for the partial system to be expanded as each pixel within the system becomes a unit within the subsequent pixel. Ultimately, this structure becomes the base for the computational hierarchy of a2o’s object-to-user exchange.

Passive / Active Autonoumous intellegence

The first level of user-to-object interaction is what we consider a passive /active autonomous system that seeks to produce an architecture that is capable of making low-level decisions relative to spatial conditioning and energy conservation. As stated above these systems tend to lack programming transparency as well as a capacity to encapsulate all the parameters of decision-making. Therefore, the system uses a swarm agent model to produce a low-level passive autonomous response, what could be considered as the system normative state. Swarm intelligence produces collective behaviors of unsophisticated agents interacting locally within their environment, causing coherent, functional global patterns to emerge (Maher & Merrick, 2005). Through localized light sensors the system measures the light levels falling on individual units. Throughout the day as light levels increase individual units respond by contracting embedded linear servomechanisms, resulting in a compression of the polymer shell which produces a differentiated shading pattern across the field of agents, responses that could be associated or read as blossoming or flocking. In turn, the blossoming effect increases the units profile and reduces solar gains falling on the surface beyond.

At a slightly higher level of intelligence, the individualized response of the agent models allows the system’s intelligence to actively respond to the user’s need. In this scenario the system makes an assumption that the proximity of the user to the window infers a desire for a viewing (Figure 1). The system then uses a gestural interface to allow for controlled mitigation of solar gains while also producing isolated views and privacy. The use of a gestural interface produces a novel understanding of phenomenology and anthropomorphic within design. Rather than turning a system ‘ON’ or ‘OFF’ or prescribing a daily routine, the gestural interface allows for an adaptive, playful and user-defined interaction, the result is a kinesthetic, haptic and optical reading of the a2o that is understood as awareness, intelligence and otherness.

Figure 1: a2o prototype reacting to user vicinity and creating shades of gradient

Social Medium

The importance of new media to both cultivate and produce a re-emergence in interactive architecture also begs the question whether architecture is or, can inherently become, a new form of social media. Within this layer of intelligence a2o is capable of being aware of user interaction relative to time. Therefore, if the system has not been active within a given time, or it begins to recognize patterns of low or no interactivity, for example if the user is away, the system can seek out the user’s ‘digital-self.’ Figure 3 illustrates the data connection scheme of a2o and the Second Life and Twitter environments.

The first level of social interface allows the user to tele-operate and tele-monitor the system. a2o uses Pachube, a real time internet data host, to connect to Second Life. The use of the Second Life virtual environment allows users to remotely operate and monitor the status of a2o through their avatar (Figure 2).

Figure 2: Second Life representation of a2o

Second Life, currently the most popular general-purpose 3D virtual world, was used as a proof-of-concept test. Buildings or objects in a 3D virtual environment are more expressive and intuitive because of the one-to-one proportional relationship to the real world, whereas 2D graphic user interface (GUI) methods such as web interfaces or typical computer applications, although popular, provide a less easily visualized environment and therefore lack a transparent one-to-one understanding of virtual and real world stimuli. a2o in Second Life asks how will we virtually interact with physical environments in the future?

The second level of research explores the capacity of emotive data. Connecting to Twitter allows for a new social awareness. Not only can a2o ‘tweet’ emotive statements called from a pre-program vocabulary that is algorithmically prescribed relative to environmental and interactive conditions. For example, when no users are present a2o can playfully [subject] tweet ‘come out come out wherever you are.’ a2o can also ‘follow’ friends (users), parse their tweets for recognizable emotive words, and produces a given spatial response through kinetic movement and RGB kinetics. As an architectural interface, twitter may allow for a proactive physical response. If the system recognizes that the user(s) current Tweets contain a majority of negative words, the system can preemptively open the units of a2o to provide the user a more well lit and inviting environment upon return, potentially improving the user’s physiological well-being. Through the connection of a2o and Twitter we ask how data flows related to social media will be expressed architectonically.

Figure 3: a2o connection to Second Life and Twitter

User Override

The use of interactive design subassemblies required that their program and function be transparent and malleable. Much of the media culture within which interactive research resides presents data and information through GUIs, tablets, pads and screens. Through its program, ubiquity and materiality, architecture invites a more transparent one-to-one interface capable of enabling its users to feel a sense of control.

Kinetic memory allows users to physically train the actuation of a2o. Through a series of kinetic sensors, an action placed on a single unit, such as compression, can be mapped proportionally to the actuation of the servomechanism producing a one-on-one replay of the action by the other units in the field. The kinetic memory is analogous to the simple act closing the blinds. Although a2o’s intelligence may be understood as complicated by a user, kinetic memory is a form of user override and represents the highest level of the system hierarchy (this is not to say the most intelligent, rather it overrides all other controls). Interactions can also be stored in the memory of the Arduino microprocessor allowing the user to record an action placed on one of its units in a given period of time and replay the action over variable time intervals and intensities. The result is a user-oriented physically programmable surface capable of emergent patterning that can be described as fluid, pulsing, wave-like or bubbling. We suggest through kinetic memory new possibilities for forms of individualized creative expression.

CONCLUSION

The PARTeE approach focused on combining computation, robotics, and virtual worlds with rapid prototyping. Through the prototype we presented a way to understand how architecture as 4D environments can be conceived, designed and produced. As architecture seeks out a post-digital ‘ism,’ it realizes the tools that have been developed for architects have allowed its process to become analogous to those of fashion and the new media culture it resides in. a2o and the work of PARTeE does not seek to answer what architecture is, but rather ask what can it do? The ever-expanding toolkit of off -the-shelf robotics, open source computing, and user generated information communities have lowered the barriers-to-entry for designers to explore this question. a2o’s development as an advanced working prototype provides a construct in which questions can be asked: Can architecture actively and dynamically change physical environments in real time while becoming a social medium? Can architecture connect the virtual and the physical? Can architecture become an interface to connect what were once thought to be disparate ideas and worlds?

REFERENCES

Dade-Robertson, Martyn. The architecture of information: Architecture, Interaction Design and The Patterning of Digital Information. London: Routledge, 2011.

Eisenman, Peter. “Diagram An Original Scene of Writing.” In The Diagrams of Architecture, by Mark Garcia, 92-103. West Sussex: Wiley & Sons ltd, 2010.

Fox, Michael, and Miles Kemp. Interactive Architecture. New York: Princton Architectural Press, 2009.

Hegel, Georg Wilhelm Friedrich. Science of Logic. New York: Cambridge Unversity Press, 2010.

Intille, Stephen S. “Designing a home of the future,.” IEEE Pervasive Computing, April-June 2002: 80-86.

Jerkins, Henry, Ravi Purushotma, Kathrine Clinton, Margaret Weigel, and Alice J Robison. “Confronting the Challenges of Participatory Culture: Media Education for the 21st Century.” New Media Literacies. The MacArthur Foundation. 2006. http://www.newmedialiteracies.org/files/working/NMLWhitePaper.pdf (accessed 12 10, 2010).

Leddy, Thomas. “Moore and Shusterman on Organic Wholes.” 49 (1991): 63-73.

Maher, Mary-Lou, and Kathryn Merrick. Agent Models for Dynamic 3D Virtual Worlds .

Marshall, David. New Media Culture. http://www.newmediacultures.co.uk/.

Mozer, Michael C. “The Neural Network House: An Environment that Adapts to its Inhabitants.” American Association for Artificial Intelligence Spring Symposium on Intelligent Environments. Menlo Park, CA: AAAI Press, 1998. 110-114.

Negropnte, Nicholas. The Architecture Machine. Massachusetts: The MIT press, 1970.

Negroponte, Nicholas. “Toward a Theory of Architecture MAchines.” Journal of Architectural Education 23, no. 2 (1969): 9-12.

Rocker, Ingeborg. “Apropos Parametricism; If, in what style should we build?” Log, Winter 2011: 89-100.

Shirky, Clay. Cognitive Surplus. New York: Penguin, 2010.

Sterk, Tristin. “Building Upon Negroponte: A Hybridized Model of Control Suitable for Responsive Architecture. Digital Design.” 21st eCAADe . Graz (Austria), 2003. 407-414.

Terzidis, Kostas. “Design inside the Chinese Room.” International Journal of Architectural Computing, 2008: 361-370.

Weiser, Marc. “The computer for the 21st century,” Scientific America, 265, no. 3 (1991): 94-104