Evaluating ActuAir: Building Occupants' Experiences of a Shape-Changing Air Quality Display

Our prototype is a shape-changing modular display that physicalizes changes in AQ levels in a sensory-rich workplace. The workplace is an open space hosted in a five-story smart building in a city in the UK which is an office, teaching and research facility; with an extensive grid of environmental and occupancy sensors provided for research purposes. Sensory data is logged publicly via an API, allowing open access to real-time and historical (timeseries) climatic data of different areas in the workplace (and the building). Ethical approval was obtained at Newcastle University, UK to use the data and conduct the studies in the building (which is part of the Newcastle University building complex).

The prototype was built as a response to results from a series of exploratory studies [64,65]; which pointed towards physicalizing data that the building collects for awareness and wellbeing purposes. The prime purpose of the display is to surface AQ data collected, focusing on - but not limited to - CO2 measurements. A key research aim through prototyping and testing the display was to understand what data is important for AQ experience in the workplace, and how this data could be represented through inflatable and LED feedback to be intuitively perceived and understood by the building occupants when deployed in place. For the purpose of conducting this research, the prototype is highly customizable; to able to evaluate different physical arrangements and feedback configurations for communicating climatic data with the building occupants of the workplace.

The prototype consists of eight custom silicone rubber pouches with embedded Neopixel LED stripes, attached on eight prefabricated fabric modules (see Figure 1 B); forming a customizable modular room divider (Figure 1 A). The silicone rubber pouches inflate and deflate based on different AQ data levels – choosing between CO2, humidity or temperature data. In the default set up, the amount of inflation is proportional to the selected data stream – e.g. inflation increases when CO2 increases. The inflation effect is enhanced by the embedded LEDs, which obtain a green-to-red color range proportionally to preset AQ levels³. LED animations and gradual color transitions are manually controlled and are not part of the default set up (details in supplementing materials).

Each module operates with a wireless Arduino connected to a local web server built on Raspberry pi (Figure 1 C). The server is fetching AQ data (e.g. CO2, temperature, humidity) from the building's API, and wirelessly communicates with the Arduinos which control the air-pumps and valves for each module - e.g. the amount of air pumped in and out, and the time they stay inflated or pulsing – as well as the Neopixel LEDs function – e.g. LED brightness, color and color animations. All electronics (e.g. Arduinos, air -pumps and valves, preboards, cables) are integrated at the back in each module, enabling them to act as stand-alone and in coordination with other modules.

Being modular, the display is highly configurable; the modules can easily plug in and out the existing modular infrastructure in different positions, can be re-arranged and re-assembled by the building occupants, be moved around and installed in other rooms in the workplace. Feedback produced by each module is also highly customizable, to assist with the research purpose of the display. The server application provides a webpage with a control panel which enables the researchers to choose different data streams – i.e. CO2, humidity, and temperature - from different building sensors; choose to display real time or historical data; choose which modules display which data; and manipulate feedback in a modular way – i.e. manually control inflation/deflation of a given module, manually control LED color range and speed and repetition of animations (see supplementing material). Utilizing the control panel to dynamically configure feedback, the researchers evaluate aspects of the display's perception and interpretation, such as the experience of inflation as CO2, the choice of color range to represent improving and deteriorating AQ; and the timing and the synchronization of these inflation and LED color change.

Biomimicry is conceptually at the core of the design of the intervention, driving the prototype's design and making. The form of the pouches and the choice of materials was heavily inspired by natural organisms (for example the Portuguese Man o’ War); the metaphor of breathing– translated to inflation and deflation – is key behind the conceptualization of the prototype. Apart from the working prototype, the materials and other in-progress prototypes were openly exhibited in the workplace; showcasing the design and fabrication process, which involved experimentation with different color and shape-changing materials (Figure 1 D).

The final prototype is a customizable, modular actuating barrier providing a physicalization of data in place. It's intended use is to raise awareness on climate – specifically AQ - in the workplace; but this purposefully remains vague in some of the studies to explore how the building occupants understand and use it. The broader research scope of the prototype's deployment as an intervention was to build an understanding of the potential of actuating materials and soft robotics in architecture (inspired by biomimicry) to communicate climatic data; with the view on engaging the building occupants in considering air quality, climate and their wellbeing in the buildings.

Three successive studies took place between June and August 2022 to evaluate the above display. Each study was led-by or facilitated-by the prototype's installation in place; addressing feedback awareness, perception and interpretation by the building occupants as described in detail below. Utilizing the display's configurability and modularity, different physical arrangements and feedback sequences – i.e. shape and color change coordination - were evaluated with participants. The studies overlapped with regards to what aspects of the experience they explored, supporting data triangulation; but they also informed each other in terms of set-up, context and focus, based on key findings as described below. All the materials and the final prototype itself remained publicly displayed in the workplace throughout the duration of all of the studies.

3.2.1 Study 01 “ActuAir - Design criteria evaluation workshops” was a series of design criteria workshops led by the prototype's installation in a meeting room (Figure 2 A), aiming to explore the feedback perception of experts and simultaneously occupants of the building in a controlled set up. Nine participants in total – building occupants and design experts - participated in three hybrid workshops (60 minutes in duration) during June 2022. The prototype was installed in a meeting room within the workplace building, showcasing a potential (random) arrangement and feedback functionality. In terms of feedback, both real-time data and historical data were showcased using a non-gradual green-orange-red⁴ LED to represent increasing CO2; increase was also accompanied by inflation. Each workshop started with a briefing of what the intervention does – i.e. displays air quality data - and how –i.e. showcasing feedback control; followed by going through specific design evaluation criteria and addressing relevant questions to prompt critical discussion. Criteria included a) Physical aspects of feedback - shape change, color change, noise experience; b) Temporal aspects of feedback – experience of the coordination between light and inflation in each module and between modules; c) Spatial aspects – physical positioning and arrangement; d) Configurability and user control. Real-time modifications were made to feedback, to test participants’ ideas. The aim of Study 01 was to unpack the promising elements and existing pitfalls of the display's materialization, and then define the conditions for one or more deployments in the building. Findings on biomimetic feedback metaphors framed the focus and setup of Study 02; and findings on shape-change readability, noise and color-inflation coordination informed the setup of Study 03.

3.2.2 Study 02 “Biomimetic Feedback co-creation workshop” was a co-creation workshop facilitated by the prototype's installation in a meeting room during July 2022 (Figure 2 B). The workshop (60 minutes duration) aimed to explore feedback perception and interpretation in a controlled set up. Five participants explored their interpretations of five video-recorded biomimetic feedback configurations using the prototype (see Figure 04) to provide cues on AQ readings while also experiencing the prototype installed in place; inspired by key findings on biomimetic feedback in Study 01. After an initial showcase of all the videos and without communicating the display's purpose (to physicalize air quality data), the participants were asked high-level questions around what data was represented and how they interpreted the different feedback scenarios. This was followed by a second showcase of each video while communicating that air quality data is displayed; and an in-depth discussion of how each configuration and feedback was interpreted. Finally, the participants were asked to vote for which one appealed to them the most and co-design their own modular configurations using paper templates. The aim of Study 02 was to explore how the shape and color change coordination of different modular arrangements inspired by biomimetic metaphors is interpreted; both in absence of and when knowing that air quality is represented. Findings highlighted the preference of circular modular arrangements, and the association of modules with areas in the buildings and spatialized data streams; which then drove the setup of Study 03.

3.2.3 Study 03 “ActuAir - deployment case study” was a longer-term case study deployment of the prototype in a public area in the workplace during August 2022 (Figure 2 C); aiming to explore feedback awareness and experience in-the-wild. Grounded within literature on evaluating lived-in prototypes [75,140]; the intervention was installed in the workplace and was left to operate there for two (2) weeks. The deployment set up was driven by findings of both Study 01 and 02. It included the choice of circular modular arrangement for the deployment (Study 02); the association of each module with different building areas & sensors (Study 02); the gradual green to red color range to represent CO2 increase as it was interpreted intuitively and was preferred over non-gradual color change (Study 01 & 02); the use of inflation when CO2 increases (Study 01 & 02) using noise effectively and with precaution (Study 01); the positioning of the display at an angle to the main working area to make shape-change more readable from multiple viewpoints (Study 01). During the two-week period, the six-module circular arrangement displayed CO2 data from nearby sensors; from the same sensor in week 01, and from the three (3) nearest sensors in week 02 which created variations on the color change pattern of the modules. During week 01, an information poster next to the prototype informed that it displays data from the building – not explicitly stating that CO2 is displayed - and provided a QR code leading to short online experience questionnaire. The poster was updated during week 02 – informing that it is an air quality display. The actual data sources and zones displayed were not revealed during the duration of the study. Data from seven building occupants were collected through semi-structured interviews conducted in front of the display and through the online survey. The aim of Study 03 was to map aspects of shape and color changing feedback experience– including materiality - in-situ; to unpack if and how the circular modular arrangements are associated with localized and spatialized data streams; and to evaluate if the selected color and shape change (and noise) feedback raises awareness in place – if it is being made aware while working in the lab, if it is distracting, and how it is interpreted with regards to climate (both knowing and in absence of knowing that air quality is displayed).

Participants (Table 1) were recruited from occupants of a shared workspace based in the office building part of Newcastle University; which is the building that these research activities (i.e. the making of the prototype and the evaluation studies) were conducted. Participants were employees, researchers and students working in the same workplace for varying amounts of time; recruited through internal communication of the studies via email. For all studies we accommodated to post-pandemic work life limitations⁵; targeting to recruit people that were coming to the workplace at the time of the study (as many worked mostly from home). For each study different participants were chosen, to avoid being knowledgeable on what the display does or biases from previous studies. For Study 01 (design criteria), participants were recruited following a convenience sampling method, we invited HCI researchers, designers and software/hardware engineers working in the space, based on their design expertise and availability. They were organised into three groups based on their specialism, for conducting three workshops⁶. For Study 02, a group of students was randomly selected; who used the workspace for a specific amount of time during the studies. For Study 03 (deployment case study), participants were chosen based on how often and how long they were working in the building for; recruiting full-time researchers and employees that work at least three full-time days in the office and did not participate in Study 01. Participants were not incentivized to take part in the studies.

All workshops (Study 01 and 02) were video-recorded and transcribed using Otter.ai. In Study 03, interviews were audio recorded and transcribed manually. Transcripts (all together) were qualitatively analyzed using Thematic Analysis resulting in thematic insights on users’ perceptions and experiences of the intervention. The analysis produced themes related to physical aspects of feedback – shape change, color change and noise experience; temporal aspects of feedback – experience of coordination between these systems in each module and between modules; spatial aspects – i.e. physical positioning and arrangement in the workplace; configurability and user control. Findings from all studies are presented together clustered under main themes. Raw data is presented (participants’ quotes) to support findings; accompanied by the participant's number in brackets (for Study 01, the workshop number is also noted).

DI01- Modules’ positionality key for shape-change readability. Results from Study 01 and 03 highlighted differences in participants’ awareness and perception of shape-change depending on the display's physical positioning. Study 01 illustrated that the observation angle - i.e. observing the display from a fixed front-facing point of observation - can inhibit shape-change perception; “too subtle (P07-W01)”; “not dramatic or extreme enough (P04-W02)”; “[…]to create an impression and capture attention in distance (P08-W01)”. These findings informed the set-up in Study 03 where the display was installed vertically within a main corridor, to be approached and therefore observed mostly at an angle. The fact that the building occupants were able to approach and freely move around it in Study 03 also reduced some of the difficulties with shape-change readability observed in Study 01.

DI02- Utilizing outlines and shadows to enhance shape change. Participants in Study 01 brainstormed ideas to address shape-change readability issues. Some suggested attaching the inflatable pouches to less-rigid components (acting as muscles to bend a surface), or covering them with material features to further enhance outline's shape change; “soft bendable materials (P07-W01)"; “covering them with soft materials with holes (P03-W02)”; “[…] with geometrical properties that can create a more visible shape change when inflation or deflation is actuated, changing the visible outline (P08-W01)”; pointing towards using materials with geometrical features that further enhance the shape change occurring during inflation (Figure 3). Adding to this, In Study 01, participants suggested the use of spotlights or sidelights to “enhance the outline of the inflatables and utilize their shadows as part of the experience (P08-W01)”, creating a more dramatic effect readable from all observation points and at a distance.

DI03- Light defuse and pulse key for inflation perception. Participants in Study 01 & 03 further acknowledged that the embedded LEDs play a key role in enhancing shape-change awareness and perception; found the light defusing effect (as the pouch inflates) intriguing and suggested utilizing this effect further. The coordination of inflation and LED animations was extensively discussed by participants in Study 01 with a view to better supporting inflation awareness and feedback perception; with participants suggesting effects such as “light pulsating when inflation has reached its maximum (P04-W02)”; or “(the pouches) can stay semi-inflated to diffuse light better (P06-W02)”.

DI04- Light as material In Study 03, participants’ observations highlighted experiences of the color change and the aesthetic aspects of inflation and materiality; with “(inflation) changing light distribution (P15); and even “the color of light changes because of inflation […] I think light becomes brighter when it deflates” (P16). These findings – i.e. linking shape change and materiality with color distribution and feedback perception – are uniquely to the field of soft robotic shape changing displays; and can open up avenues for exploration of the perception of these parameters (and their coordination) for the purpose of communicating data in various contexts.

DI05- Circularity and spatial data associations. Results in Studies 02 and 03 illustrated how the modularity of the display (see Figures 4, 5, 7) inspired thoughts of spatialized data representations. Some participants in Study 02 perceived the circular modular arrangement as representing an area in the building, linking the overall shape of the display with a location of data streams; others (Study 02) viewed each module linked to different rooms or room sensors, both in circular and in non-circular arrangements (see Figures 4,5, 7). These spatial relationships were extended to interpreting feedback variation between modules. When modules were not synchronized, they were perceived as showing different areas in the building (Study 02). With regards to the circular arrangement; “My interpretation is as every module represents a room and it depicts the air quality in the room over a brief period of time (P11)”; “I think it's different blocks in a building rather than rooms -like block A,B,C etc. and if there is less amount of oxygen in block A, it turns red and simultaneously do the others (P13)”. Intersecting modules (modules on the intersections of two circular arrangements) (see Figure 7 B, C) were perceived as displaying the averages between the sensory readings of two areas. In the case where one (central) module was behaving differently from the surrounding circular ones, participants interpreted this as the ‘average AQ’ - i.e. the physical middle point as the data (middle - point) average - or as having different sensitivity to data. “It's more like the outer ring responds to 70% of CO2 and the inner core responds to 100% of CO2 in the atmosphere. (P12)”; “It might depict the gradual rise in CO2 and say 70-80% of rise in CO2 might change the color of the core. (P14)”. These findings connecting modular display elements to spatial data relationships can be key insights to guide the deployment of such awareness displays in the context of smart buildings.

DI06- Form and material key for environmental associations. As the prime aim of the display is to physicalize climatic (AQ) data such as CO2, the participants discussed how well it serves its purpose. Participants in all studies intuitively related to the display of climatic data e.g.“[…] it seems it senses temperature from different zones in the building. (P10)”, based on shape and broad design factors such as color and material properties ‘it is green and has leaf-like shapes (P15)’; “it is nice how ‘material is making light diffuse[…] resembling something from the ocean (P15)”; “gluey texture (P16)”. Study 03 illustrates that materiality drove participants to touch the display; many felt it was pleasant or interesting to touch the ‘gluey texture (P16)’ hinting towards providing more opportunities for genuinely tangible interactions and tactile feedback to create embodied connections to climatic data (Figure 6 ). Across studies, participants expressed views suggesting that the display was evidently and intuitively linked with environmental data, but CO2 was not necessarily an obvious connection “the design is too abstract to be associated directly with CO2 representations (P04-W02); ‘it is too nice to visualize CO2, and it should be more ugly or extreme’ (P06-W02).

DI07- Light and inflation coordination enhances climatic associations. Participants in Study 01, inspired by the prototype's physical properties, saw the coordination of inflation and LEDs (and the adoption of biomimetic metaphors) as a way to better support feedback awareness and intuitive interpretation of the display; and potentially link it better to CO2 data representations. Some participants (Study 01) suggested abandoning use of the ‘traffic-light’ system (non-gradual green-orange-red LEDs) to represent CO2 levels, in favor of using two-color or one-color representations, gradual transitions between colors, and LED animations to draw attention – e.g. light blinking or pulsating at critical moments. Participants in Study 01 further engaged with biomimicry to support their display and feedback design ideas, relating different feedback scenarios with natural processes; including breathing, the life-cycle of corals and the CO2 accumulation by leaves: “red turning to pale red and white as air quality worsens, imitating the life-circle of corals (P03-W02)”; “each module represents a leaf accumulating CO2 as it inflates, turning from green to red (P06-W02)”; “light could be ‘pulsating’ (i.e. fading-in and out at a regular rhythm) to illustrate the rhythm of CO2 concertation, turn red and starting blinking when CO2 reaches a critical level, and then stay red until CO2 decreases (P06-W02)”;“the breathing metaphor […] the pace of breathing (LEDs pulsating) changing when CO2 levels change (P05-W02)”. These metaphor concepts inspired the framing of Study 02; they were translated in specific modular arrangements and specific shape & color feedback sequences (Figure 5) which were evaluated in Study 02. Key findings in Study 02 illustrate that light and inflation feedback (and the synchronization of these two systems) through different modular arrangements designed based on biomimetic metaphors that address climate and CO2 accumulation (or the effects of its accumulation on nature), can provide direct cues for intuitive sense-making and interpretation of modular awareness displays; linking the interpretation of such feedback with climatic aspects and climatic deterioration (Figure 5).

DI08-Color drives feedback interpretation. Across studies, LED color was seen as the dominant mechanism to communicate both absolute (AQ measurement) and relative (AQ change) information, as well as rhythm of AQ change “It's pretty straightforward: Green is good, and red is bad, time to leave the room or open a window (P12)”; “It's green and inflated and turns red and deflates when there is higher than the ideal temperature (P10)”. Inflation acted as complementary mechanism for sense-making directly linked with accumulation in nature (of CO2 or O2) but having a diverse meaning with regards to being positive or negative. “[…] inflation increases for representing CO2 levels (P02-W03)”; “There's more oxygen when the prototype inflates since it's green and deflates when there is more CO2 exhibiting red color. (P11)”; “It should probably go the other way around [.]it would be better if the prototype gets large and makes noise when it goes red to alert the user something is happening (P10)”; “Inflation indicates something positive, whereas shrinking something negative (P15)”. In two-color representations – e.g. the Coral life-cycle in Study 02- participants anchored themselves around red color being ‘bad’ -e.g. high CO2 (see Figure 5)- which again confirmed the dominance of color in feedback interpretation highlighting the green-to-red color scale as the key element for sense-making; with inflation acting as an accompanying aspect, further enhancing or toning-down the color interpretation.

DI09-Rhythm of feedback important for awareness. The tensions between feedback repetition, rhythmicity and momentum were apparent in participants discussions (Study 01). The appropriate coordination of LEDs and inflation was framed both in the context of creating momentum when significant change in AQ levels takes place; as well as in the context of creating a sense of pacing and rhythm in the display's operation. Participants in Study 01 suggested normalizing feedback intervals – mostly referring to LED blinking - to create the feeling of time passing accompanied with specific points of momentum. ‘The modules functioning as digital digits (P01-W03)’ if arranged appropriately; ‘the circular arrangement of the modules is imitating an analogue clock (P02-W03)’; “(at a circular arrangement) a repeated LED blinking at even intervals (like a clock) can draw the attention to specific moments; for instance, when the hour has passed and a new reading is available, or when there is a significant CO2 change” (P02-W03).

DI10-Noise as feedback at key moments. Noise (byproduct of inflation) was seen with mixed feelings by participants in all studies. “I'm not sure if the prototype gives out so much sound or it's just the video, but it'll be really disturbing if it's the prototype. (P10)”. Participants in Study 01 and Study 02 suggested that noise can be an essential interesting feedback parameter to be used with caution “Do you think that people will be aware of it by just seeing the colors? (P14)” implying that its’ noise is essential to draw attention; “Since you can't avoid noise, use it as part of the feedback system, drawing attention when it is needed (P09-W01)”; “(about noise) probably draws attention to the change in the light. (P11)”. These findings further pointed towards using inflation for CO2 increase and informed the set up in Study 03. Inflation noise became particularly annoying when displaying the time-series animations during Study 01, which discouraged further discussion of displaying historical data during deployment Study 03.

DI11- Speed of light animation linked to urgency. LED light animations with varying speed, repetition and color transitions were heavily suggested as ways to create rhythm and momentum in the display (Study 01); which were then materialized and evaluated in Study 02. Results from Study 02 illustrate that these variables – e.g. speed of color transition, the color range of LED animations, and repetition - are key to feedback interpretation. A gradual non-repeating green-to-red transition at a speed ∼30s created a clear perception of slow deterioration; whereas the same animation at the speed <5s and in repetition created a sense of urgency “It feels more like a ticking clock, so you have to get out (P11)” (Figure 5).

DI12- Feedback expectation impacts attention. The design dimensions of feedback rhythmicity, predictability and variation were not easy to navigate in the wild. It became apparent that observations from a controlled set-up do not necessarily apply for in-the-wild deployments. For instance, traffic-light color range and non-gradual color change were seen as limiting in Study 01; with participants expressing that gradual color change could communicate better, and that traffic light color range is over-used. On the contrary, the idea of a traffic-light system was treated positively in Studies 02 and 03 as it was easy to interpret and predict “I expected it to turn red at some point (P21)’; while gradual color change was not perceived on time from most participants in Study 03 “Did it really change since last time? (P21)”; “It's always looked green since I've seen it. (P17)”; “Was it always yellow on some sides? (P19)”(Figure 6).

Participants expressed the view that the display can be successfully used for displaying diverse data sources in different building locations (or regardless of the building location); as it allows flexibility on how it is used, and it blends well in space "the design is successful because it has a clear how –what it does, its functionality and functional properties – but an abstract what (it is for), allowing flexibility in its use participants to use it in different ways (P05-W02)’; “It blends nicely with the space, visually integrated and non-disturbing (P16)”.

DI13-Spatial context influences data use. Results in Studies 01 and 02 show that the display's location in the workplace created different expectations in terms of data used and feedback configurations; as well as different speculative reactions and behaviors from the side of the building occupants “(modules with) the traffic light system can be meaningful outside office spaces; they can notify you if the CO2 is too much (in the room), and you can choose to open the window or go somewhere else (P01-W03)”; or in the meeting rooms, “It will be interesting to see what people will do during the meeting if AQ deteriorates, if they open a window or do a break (P08-W03)”. In Study 02, participants highlighted that, when installed at a central location in a circular arrangement, they expected the display to function as a display of climatic data averages of the main building blocks“ [at a central location] each module represents each block in the building complex. […] The core module in the center is for the whole building complex by itself, or the average of all the (building) blocks in one prototype, hence it's placed in the center (P13)”;” This one is like 3 overlapping circles representing each block in our university. So the overlapping modules (the ones that belong to multiple circles) could be the average of the few blocks combined (P10).”

DI14-Feedback tailoring based on occupancy/ proximity data. Beyond AQ and climate data, there were some ideas around integrating different data-sources, including for human activity. “I felt like it will be based on movement, like with motion sensors […] maybe you have some sensors which brighten up (the display) the more you go closer and dull out the more you go farther (P11)”; “I thought the display also captures occupancy, as a measure relevant to air quality (P16)”; I wonder if it is just building sensors or local sensors too? I don't know, but does it know if people are nearby or even looking at it (P18)”. These ideas can be interpreted as bringing more variation, gamification, and interactivity (and potentially unpredictability) to the feedback from the display; as well as strengthening human – data interactions. It was suggested that motion and occupancy sensors could be used to also control the overall powering of the intervention; and participants also discussed their concerns around energy consumption. “Out of necessity it uses electricity! (P18)”

DI15-Extending modularity/plug n’ play. A further line of thought around strengthening user engagement included enhancing the existing customizability of the display, designing tools to allow configuration and manipulation by building occupants. The existing modules are easy to move and reassemble in different locations in the building; participants in Study 01 suggested mobile-based interaction could provide incentives for people to actively use the modules and further configure the feedback as they see value. Some highlighted the ability to able to select data sources (Study 01 and 02) – different data sources from different building locations, and potentially their averages (Study 02). “mobile interaction could be encouraged to select data streams and appropriate feedback features of the display – e.g. light color, intensity, duration (P02-W03)”; the design of a tool that “allows to visualize your own feedback patterns supporting the drag-n-drop of shapes, modify the feedback (i.e. inflation, light animations) and deploy it in the physical installation (P01-W03)". Differences in perception of what data is important to represent AQ were highlighted – some referring to CO2 and some to O2. This also suggests potential future value in user customizability of data sources.

Results (Study 01 & Study 02) highlighted the importance of coordinating inflation and LEDs, utilizing LED animations to a) to support and further enhance shape change perception (DI03) - e.g. in quotes on pouches staying semi-inflated (for diffusing light) and using light pulsating to enhance inflation (P06-W02), the use of spotlights to utilize inflation shadow outlines (P08-W01), b) create momentum and draw attention to specific moments (DI03, DI11)– e.g. light pulsating to enhance maximum inflation perception (P04-W02) and c) create a visual rhythm based on the temporal dimension of climatic data (DI09)-e.g. light pulsating in regular intervals to highlight time passing (P02-W03). A fourth aspect was noise, discussed in the context of creating momentum on specific moments and supporting the visual experience of shape change with auditory feedback (DI10)(P09-W01). The above aspects (which are further unpacked below) could be framed as key design dimensions for designing with light and inflation for large-scale, modular and non-modular, shape- and color- changing displays; also extendable beyond climatic data representations.

Results from Study 01 highlighted the potential for shape changing materials to successfully provide abstract – yet intuitively understood - visual cues for interpreting state and state change in climatic (and other) data streams; when supported with appropriate lighting (DI02). Participants (Study 01) referred to using ambient light to enhance shape change awareness (or to create the illusion of it) suggesting the use of spotlights; utilizing the produced shadows of the shape outlines as part of the feedback strategy (Study 01). Shadows are heavily unexplored as a feedback strategy in HBI/HCI research; illustrating avenues for further design exploration. One of the very few examples is Shutters [20], where natural/ambient light is used to produce shadow projections to communicate visualizations driven by shape changing SMA wires; illustrating gaps and emerging opportunities for design research. Results (Study 01) highlight the importance of feedback at-scale in the buildings’ public spaces (Study 01, 02, 03) whilst remaining non-disruptive; which further supports the utilization of ambient light and shadow as part of climate feedback, as it is passive, calm and deployable at scale within the buildings (DI02).

Participants’ suggestions of geometrical properties to further enhance shape-changing feedback point towards the emerging space of inflatable auxetics [17,79,133] and programmable metamaterials [54,55,130] and their potential application in large-scale displays (DI01). The work of Alexandra Ion [54,55,130] extends to exploring programmable, passively-changing geometrical structures and the geometry-enabled swarm behavior of artifacts; see for instance passively reconfigurable shape changing materials [106]; DIY pneumatic metamaterials [22] and modular structures [55]. Such examples have interesting potential for modular user-configurable physical displays; acting as potential output configurations for diverse climatic data sources in buildings (DI01).

Although LED color was the driver of sense-making amongst participants; it was seen as inseparable from the inflation and the material dimensions of the display (DI04). The employment of LED light has many manifestations in HBI and soft robotics literature; but often not discussed in the context of materiality (DI04). Results illustrated that materiality enhanced the perception of light (e.g. P15, Study 03); in Study 03, light obtained a ‘material’ dimension itself – e.g. P16 notes the illusion of LED color changing due to materiality and inflation. Materiality and light diffuse are intertwined in architectural design, but very seldomly addressed in the context of public displays (see transparency-changing glass for instance [6,132]). Li-Lo displays [104], is one of the limited works which unpack ambient light use and diffuse aesthetics through a series of material displays and projections in-place; highlighting more potential gaps for understanding the experience of light diffuse as feedback for climate and AQ change (DI06). These results point towards research on material light and color emitting feedback strategies; examples include the use of thermochromics [18]; as well as light-emitting biomaterials – see the use of cyanobacteria [38] or algae [63] in building materials. Both research avenues have interesting complexities with regards to creating programmable composite / biomaterials for climatic feedback purposes.

Noise when inflation is actuated is a novel aspect of pneumatic systems, usually treated as a problem to eliminate and very rarely as a feedback strategy – see projects such as PITAS [18], an attempt of engineering a pneumatic metamaterial that actuates without noise, and FlowIO soft robotics platform [105] which supports a portable approach for pneumatics claiming to limiting noise (DI10). Our findings suggest that there might be opportunities to utilize noise (with caution) for large-scale public displays as part of the feedback strategy. Findings in Study 03 suggest that, in public open-space setting, inflation noise is almost essential to attract attention to specific moments of AQ change; with participants expressing limited concerns about being distracted by the noise. Noise use should therefore be context specific - i.e. might be unnecessary and distracting in a meeting room where there is close visual proximity to the display, and essential to draw attention in specific moments when installed at a public spot. Summarizing, large scale pneumatic displays should subsequently allow a level of programmability for utilizing noise depending on the context of deployment (DI10).

Results (Study 01, Study 02) highlighted that when designing modular shape and color changing feedback, biomimicry can have a direct impact on how intuitively this feedback is perceived and whether it is linked with climatic (and AQ) data (DI07). The imitation of natural processes related with CO2 accumulation (e.g. breathing or leaves’ CO2 accumulation) or the impact of CO2 accumulation in nature (e.g. the death of corals and the expansion of lichens) was translated through modular arrangements and shape and color change sequences in Study 02, informed by Study 01 findings. Study 02 results show how the above biomimetic metaphors translated in modularized inflation and color feedback were intuitively linked with aspects of deteriorating climate (see quotes in Figure 5).

Modularity expanded the possibility of the existing shape and color changing system to obtain different forms at scale in the buildings [56,77,117]; and generate different dynamic behaviors between modules or groups of modules (see Figure 5, the Leaves metaphor A & B for instance) while still behaving together as a whole (see Figure 5, the Lichens metaphor); which can be framed as swarm behavior [103]. Swarm behavior – i.e. collective motion of independent entities – enabled by modular soft robotics is an aspect of biomimetic design with some applications in kinetic architecture – see Ned Kahn's wind façade⁷ - rarely explored in HCI/HBI research [66,78,103]; which can have interesting implications on the development and interpretation of modular, soft robotic displays at large scale [77,117,134]. In this work, different modular arrangements were presented in Study 02; with modules obtaining different inflation and color change sequences either individually or as groups (see Figure 5) as an attempt to imitate different aspects of natural organisms (DI06, DI07). Beyond associations with climatic aspects such as deteriorating climate, participants in Study 02 perceived different modular arrangements in direct association with the spatial location of data streams and through to be representing spaces/building areas and the climate data readings from them– something like a physical data map of the building areas. Participants in Study 02 particularly pointed to the circular 6-module arrangement as the most appealing, associating the circularity with the representation of an ‘area’ in the building; with some replicating circular arrangements to visualize intersecting areas in the buildings during the co-creation activity. Given the number of modules fabricated and used in this study is limited; future work can explore further the installation of different biomimetic feedback arrangements of modular soft robotics at scale, as well as how building occupants perceive them and see value in using them in-situ.

The display was described as aesthetically pleasing (in all studies) and that it provided distinct visual cues to interpret climatic change and climatic levels. The association with specific biomimetic metaphors was picked up by participants in Study 03, when participants suggested that the display shows environmental data based on its ‘leaf -like shapes (P15)’; also ‘resembling something from the ocean (P15)’. These results from Study 03 showed that, beyond modular assembly and inflation and color change coordination, primary aspects contributing to the climatic interpretation of the display were (as intended) the overall form factors of each module and their materiality (DI06); with some expressing that it was interesting to touch- ‘a gluey texture (P16)’- highlighting the possibility of embodied and haptic experience of feedback. Expanding on these findings, materiality could be further explored in the context of designing for climatic data displays, through further research in (material) haptic and embodied experiences; and conceptually, transitioning from nature imitation to nature connectedness (biophilic feedback). Building on relevant literature on architecture and embodied cognition [80,112,116]; material-driven feedback can extend to further imitate bodily reactions and further engage embodied experience of climatic data; for instance, material feedback that imitates sweat [94,129] or broadly skin – see soft robotic skin textures [46,106] - and its responses to the environment [23]. Using such material feedback could also foster climate-sensitive behavior in the buildings [45] – i.e. climate awareness and opportunities to rethink human-climate relationships.

Building on relevant research on biophilic design; experiencing nature can stimulate conservationist attitudes towards the natural environment [58,66,143]; fostering climate awareness and sensitivity on how human actions impact the environment and climate [66]. Along these lines, participants already highlighted concerns over the installation's energy use; questioning why it should be always on (Study 03) (DI14). Responding to participants’ energy concerns, reducing overall energy use of the display falls within emerging visual pollution [32] and sustainability concerns driven by current climate pressures. Bio-material research can support the reduction of LED use in the built environment, replacing them with naturally occurring light feedback [76]. Beyond LEDs, pneumatic action – and broadly - shape change also requires energy; with solutions such as PITAS [18] illustrating early potential of composite materials – e.g. conductive silicone- to provide shape and light change with less electricity while eliminating noise; yet the challenge of zero energy use and self-powered displays remains open. Materials used in passive climatic skins [60] have unexplored potentials in design research for building interiors and physical displays; for the purpose of both reducing energy use and further support biophilia in the workplace. Types of energy-passive climate-actuated materials include thermobimetals (materials responding to heat) [115] and hygromorphic (materials responding to humidity increase) [93,131] require significant amounts of environmental heat or humidity to actuate and currently have limited interior applications. Contemporary advances in biomaterials for architecture - see research by HBBE ⁸ for instance– open the space for using biology to produce electricity [97]; or engineer materials to respond to or capture CO2 [5,21,47,83,143]. Such examples can push the frontiers of color and shape change in soft robotics, utilizing biochemical reactions with CO2 to improve AQ, self-power and provide a naturally occurring feedback [76] – see for instance the work of Goodchild-Michelman [38].

Summarizing, the above results suggest that biomimicry is a meaningful strategy when designing for physical climate-awareness displays; and its success depends on primarily form and material factors; and secondarily on shape and color change coordination. Modularity plays an important role on enabling dynamic visual arrangements and feedback sequences at scale - providing with cues towards spatiotemporal data relationships – with the potentials of swarm behavior of elements to communicate meaning remaining relatively unexplored. Design research on actuating biomaterial feedback is an emerging space for climatic displays; that can potentially support both sustainability and climate regulation while nurturing climate-awareness in the buildings.

Findings on how successful different shape and light feedback was in terms of being brought to the occupants’ attention and awareness, are discussed in the context of Studies 01 and 02 (controlled installation) and Study 03 (deployment in-the-wild). Although successful in the controlled set up with no distractions (Study 02), green-to-red LED animations did not engage building occupants’ peripheral awareness in the wild (Study 03). Unpacking the neuroscience of feedback; timing and priming – to expect feedback to change and therefore anticipate it - within a relevant behavior context [46,116], impacts feedback awareness in place. Seeing the green to red color range, participants were primed to expect a specific color change to red when seeing green, assuming deterioration (DI08, DI12). As AQ did not drop significantly during the first days of the deployment maintaining the color change within neighboring color tones (Study 03), results showed participants’ attention to the display decreased over time. The request to highlight the perception of time passing through regular feedback intervals (DI09)– some referred to a clock ticking (results in Study 01, 02)- highlights the need of rhythmicity in peripheral interventions, which then can attract attention when it changes - becoming fast (sense of urgency, Study 02) or static.

With regards to feedback configurations to drive occupants’ engagement in the long run, there were no straightforward answers to the problem of ‘too repetitive’ and ‘too unpredictable’. Some participants envisioned the potential of integrating human activity data to the display's function (DI14), creating a more human-responsive feedback. For instance, using proximity and occupancy sensors to animate it around people's movements in the building; creating further associations between building occupants and their contribution to the room's CO2 footprint. Reflecting on relevant research on feedback and embodied cognition [80] which supports the idea that the visual rhythm of the environment can affect body motion (e.g., our walking pace), animated CO2 feedback that synchronizes with the pace of people passing by could provide a more ‘personal’ view of CO2 contributions, a more embodied feel of its change. Feedback configurations that establish relationships between the pace of light/inflation animations and pace of walking of people approaching or passing by could significantly enhance engagement with the display and hopefully create a deeper individual and collective connection to climate within the building. Finally, proximity sensors respond to quotes over reducing energy use of the installation; since it can be activated only when (enough) people are around – providing further links between the amount of people in the building and CO2 levels.

There were associations between the display's configuration and building spaces and localized building data (the CO2 data footprint of indoor building areas) – e.g. participants in Study 02 interpreted the circular arrangement as different building rooms or blocks (DI05). In the case where one (central) module was behaving differently from the rest, participants interpreted this as the ‘average’ or ‘middle point’ (both physically, spatially and digitally). These perceptions of spatial-data analogies – e.g. the circular arrangement as a physical area; the middle module as the data (middle - point) average - highlight opportunities to further engage with exploring relationships between the display's modular arrangements and spatial / data representations; to further investigate building occupants’ spatial sense making and provide cues to perceive where data comes from.

Across the studies, participants shared interpretations of what the display does and shaped a collaborative understanding of its feedback and use through iterative discussions. Spatial behavior context further informed occupants’ expectations around the display's function and use in place [39], illustrated when discussing how it can be used in different rooms/areas (DI13). For instance, the traffic light system was suggested if using the display in and outside meeting rooms; circular arrangement were favored in public areas to represent CO2 levels in building blocks or rooms. These results contribute to existing literature on modular, DIY and open-source architecture as a collaborative instrument [12,18], highlighting the importance of designing with configurability and modularity in mind, when thinking of large-scale physical climatic displays and providing tools to enable occupants’ control when shaping collective climatic data representations and use (DI15); creating the basis for rethinking human-climate relationships through data with the prospect of cultivating climate-sensitive behaviors– i.e. prompting building occupants to think how their own behaviors and actions relate to climate, and potential ways to offset their impact (net-zero).

Through mapping building occupants’ experiences of a modular soft-robotic shape and color changing display of climatic (AQ) data in a smart workplace building; we have brought to the fore several design implications (DI) relevant to the application of soft robotics in that context, also extended to the design and deployment of modular (5.2), shape-changing (5.1) and broadly physical displays (5.3.). We contribute to HBI [42] research through opening the design space of modular soft robotics for large physical displays for climate awareness; presenting existing opportunities and challenges for the deployment of such interventions in smart office buildings. Building upon relevant work on biomimicry and biophilic design [58,143] and large-scale soft robotics / actuating materials [98,122] ; but here with a specific focus on lived-in office buildings and modular interventions for climatic awareness, there are several  areas of future work which could be fruitfully explored concerning broadly large scale soft robotic displays (see 1-5 below) and particularly modular displays (see 6-7 below):

Through these various proposed strands of research agenda – a potential new research landscape emerges focusing on biomimetic and biophilic feedback and actuating materials/soft robotics to communicate climatic data. There is much potential for developing new, engaging and ultimately occupant-centered data-interactions to foster climatic awareness; with the scope of creating opportunities to re-consider human-climate relationships within the built environment, and hopefully this work helps to scaffold future work in this area.