Sparsely actuated modular metamaterials for shape changing interfaces

Shape-changing interfaces are a diverse class of interfaces that use various actuation techniques to display a physical shape [3]. Such physical shapes produce strong and expressive visual and haptic feedback to the users, beneficial for tangible interaction. A diverse set of mechanisms and actuation methods for shape-changing interfaces have been demonstrated. For example, desktop-based 2.5D shape change interfaces that change surface topologies were presented using pin-based methods [12, 14, 22, 43] where each pin is raised off the horizontal surface by a motor such that a global shape can be generated by a layout of individually actuated pins. These are generally stationary systems, as they require large space for their actuators, and tend to be energy-intensive.

More portable approaches use motor-embedded modular blocks to achieve shape changes and even locomotion in 3D. For example, LineForm [32] and ChainForm [31] offer linear and modular interactive interfaces driven by identical modules capable of sensing and actuation. Such modular approaches improve the scalability of the interface. Linkage-based [9, 47] shape-changing systems have also been demonstrated to achieve diverse shape changes with only one motor or the user’s manual input.

However, these shape-changing systems are composed of potentially complex mechanical structures that require assembly. On the other hand, metamaterials recently emerged as shape-changing interfaces that can be fabricated in one go to reduce prototyping effort when creating shape-changing systems. In this paper, we thus explore expanding the shape-changing capabilities of metamaterial structures to demonstrate shape-changing interfaces that are easy to fabricate and can achieve complex curvilinear shapes.

Mechanical metamaterials refer to structures whose deformation patterns rely on the arrangement of their interior cell structure rather than the specific material they are made of [6]. Diverse macroscopic behaviors can be achieved by programming the combination of the cell structures. Based on this, past works have demonstrated materials with unique properties or realized certain pre-defined functions that traditional materials cannot achieve. These include auxetic materials that expand in the other direction when stretched [2, 29], locally programmable stiffness in one printed object [38], computational materials that execute logic functions [21, 23, 45], or programming deformation behaviors [7, 13].

Mechanical metamaterials have recently attracted attention in the HCI community as a valuable resource for building shape-changing interfaces through preprogramming the shape change induced by the deformation in the structure. Ion et al. presented a method for controlling the degrees of freedom of a grid structure by selectively constraining its cells [19, 20]. The presented structures have a controllable amount of independent degrees of freedom to achieve a user-defined movement pattern. A recent work presented a pneumatically actuated version of these metamaterial designs [10]. These works focus on planar grid structures that are modeled to have an integer number of unconstrained degrees of freedom. Ou et al. [34] presented a single degree of freedom mechanical metamaterials based on the well-known rotating squares paradigm. They present designs that can perform versatile deformations such as out-of-plane bending, shearing, and twisting which are achieved by changing the joint locations and orientations in the cell structure. Our work uses designs presented by Ou et al. but further explores controlling the cells’ internal flexibility to expand the achievable shape space. We achieve this by fabricating the links in the metamaterials with long and flexible material, then control the localized deformation with sparsely placed actuators. Our designs are considered multiple degrees of freedom because of the added joint flexibility rather than unconstrained degrees of freedom.

Accompanying the presented metamaterial-based shape-changing structures, design tools have been developed to support the customization of the shape change. Kulkarni et al. [28] built a design tool that automatically generates custom wrist exoskeletons fitting the user’s body measurements. Our cell configuration tool, although using a different strategy, also computes cell configurations from user-given shape input. Our design tool goes beyond solving the rigid body kinematics of the metamaterial structure and further includes user decisions on interface flexibility and actuator locations for the simulation of the interface.

The use of flexible joints instead of rigid hinges has been a common method in mass production. As the availability of elastic materials increased, making custom designs with flexible hinges became much easier. In this paper, flexibility and sparse actuation are concepts that essentially go hand in hand. Flexible materials settle at configurations that minimize their elastic energy. This causes the effect of deformations induced by actuators to stay local, as propagating these deformations would mean higher elastic energy. Sparse actuation refers to placing actuators at far enough locations such that the resulting shape of the interface is caused by the localized contribution of a few actuators. This is similar to how several muscles affect one elastic medium at separate locations in nature (e.g., tongue, elephant trunk).

Designing flexible systems is challenging due to the complexity of predicting their movement trajectories. Many researchers presented toolboxes to aid designers in using flexible sections [30, 36, 46]. Various finite element-based methods and tools exist for simulating the motion of thin and flexible strand-like structures [4, 5, 36, 44, 49]. Building onto these models, researchers have developed computational tools for working with deployable grid shells [37], stretched fabric and 3D print combination interface [39]. Relatively faster methods include pseudo rigid body models [18] which discretizes the beam structure even less than the Cosserat rod-based models. Panetta et al. built an optimization algorithm for the location of sparse actuation in grid shell structures and Megaro et al. [30] optimized the flexible structure to meet actuator requirements and avoid material failure. The discrete rod models offer a realistic simulation opportunity, however, they need to minimize an elastic energy function, which poses a highly nonlinear optimization problem that is not fast and stable enough for our purposes. In this work, we find it crucial to enable fast and easy iterations when designing with our metamaterials. Therefore we implemented a simplified elastic energy function which can be solved at one iteration using a Newton Method-based solver. Our algorithm can give the designer immediate visual feedback after configuration.

We base our unit cell designs on Ou et al.’s work which presented cells that contract in-plane and bend out-of-plane when deformed [34]. While  Ou et al. used rigid joints that propagate the deformation along the cells, we take a different approach and deliberately use long and flexible hinges. Figure 2 showcases our basic contracting cells and bending cells. The contracting cells ’close up’ linearly when deformed and the bending cells bend out-of-plane by an angle determined by how much the joint is tilted. The long and flexible hinges deform when the cell is deformed, propagating much less deformation to its neighboring cells thus keeping the deformation largely local.

These cells can be flexibly tiled in 1D or 2D, forming, for example, a linear chain, a rectangular grid, or a branched-out structure. This creates a layout in which actuating one cell would only deform its close neighbors. Based on this, we propose to put a small number of actuators within such a layout and at a distance to each other to sparsely actuate the metamaterials. The actuators can create local and non-interfering movement patterns which together achieve a global shape change. The flexibility in defining the local deformation and orchestrating the actuation patterns enables complex curvilinear shape-changing behaviors. We demonstrate one such example in Figure 3. The sparsely placed actuators at the left and the right ends can work either in sync or with the same amount of actuation (Figure 3-a) to create a shape that is close to the 1 DOF movement of the base metamaterial structure. Or they can be out of sync and with different amounts of deformation to create movement patterns that expand the shape space, (Figure 3-b and c). Through sparse actuation, complex moving wave patterns can be achieved by moving the motors in basic dynamical motion profiles.

The cell layout can be entirely 3D printed with dual material printing - we printed the rigid parts with PLA or ABS and the flexible hinges with TPU materials.

In this section, we explain how we model the metamaterial structure for the fast estimation of its movements given specified joint flexibilities and actuation amounts.

Every cell is made up of nodes and joints depending on their location in the cell layout. As exemplified in Figure 4, we denote the nodes with the letter N and group the joints into vertical and horizontal joints based on their direction in the original state, and denote them as V and H respectively. Subscript i and j refer to the vertical and horizontal locations of the nodes and joints. Horizontal edge with the subscripts i,j, H_{i, j}, combines, N_{i, j} and N_{i, j + 1} whereas the vertical edge, V_{i, j}, combines N_{i, j} and N_{i + 1, j}. Every joint is associated with two nodes. Every node is associated with from 2 to 4 joints.

We simplify the movement of all nodes by attaching them a deformation parameter, \(D_{N_{i,j}}\), which is in the range of 1 and -1. By this parametrization, the deformation of 0 refers to the original alignment of nodes, whereas 1 and -1 refer to their maximum amount of deformation in either direction.

As shown in Figure 4, the same amount of deformation refers to the rotation in opposite directions for neighboring nodes, which is a requirement of the rotating squares kinematic structure. Nodes whose subscripts add up to an even value rotate on the counterclockwise direction for deformation of 1 and those that add up to an odd value rotate on the clockwise direction.

The joints force the nodes to follow the kinematic requirements of the structure while also tending to spring back to their undeformed state due to their elasticity. The distribution of the cell deformations is done by minimizing the elastic energy function of the structure. In this work, we defined a simplified energy function that allows designers to quickly simulate the cell deformations.

We define a per-joint energy function as:

where \(D_{N_1}\) and \(D_{N_2}\) refer to the deformation amounts of the two nodes that share the specific joint. The k₁ parameter is associated with joint flexible length and k₂ is associated with joint thickness. According to Equation 1, having a k₁ value of 1 at a joint refers to completely stiff connections. In this case, deformation at one joint is perfectly propagated to all of the joints regardless of the k₂ values. As the value of k₁ decreases, the per-joint energy function depends more on the individual deformation amounts of nodes. The total elastic energy function is computed by summing up the energies of all vertical and horizontal joints. Equation 2 shows the computation of the entire energy function. The resulting energy function is a second-degree polynomial function of the deformation amounts of each node:

The k₁ and k₂ values corresponding to a joint size were found through an empirical evaluation. For this, we printed a set of layouts and defined deformations between nodes with the angular constraints shown in Figure 5. We then computed the best-fitting k values for the given sizes of the flexible joints.

We compute the gradient and Hessian terms for the overall energy function. Due to the nature of the simplified elastic energy function, Equation 1, the Hessian matrix is constant and does not rely on any deformation terms. Therefore the computation of the deformation terms that minimize the energy function are computed at one step, providing a very fast estimation method for the exploration of available shapes.

Placing an actuator on a joint, between two nodes causes that joint to be overridden by the actuator. The deformation amounts of these nodes, therefore, are decided by the actuator. The Newton method minimizes the energy function for the remainder of the nodes that are not overridden by any actuator. Using this method the designer can quickly go through deformation patterns, caused by the specific set of joint flexibilities and actuation amounts.

Following the fabrication of the metamaterial structure, we propose a physical design stage where the user can easily reconfigure the positions of the sparse actuation to try different shape-changing effects.

We designed the structure such that motors can be easily snapped between the nodes. The physical modular connectivity across all nodes of the metamaterial design allows the designer to physically reconfigure the shape change after fabrication by changing the actuator locations. We also provide additional modules for further customization and scaling of the interface. We provide more details on the physical fabrication and post-fabrication modular design procedure in Section 5.

We used a combined bottom-up and top-down approach. The user defines the number of cells and how they will be placed in the open/flat state. They then define the most actuated shape they would like to achieve. The design tool automatically configures the detailed cell parameters based on the input shape.

In the enumerated list below, we present the design pipeline step by step. Figure 6 demonstrates the main design stages and UI elements for the sample fish application.

Following these steps, the cell configurations are made and the designer can continue with the simulation step.

In this section, we present the user interaction for the simulation of the design and the implementation of the flexible model of the metamaterials. Following the design stage, the joint inclination angles are automatically configured. The designer assigns flexible joint sizes and defines actuators on joints. For the simulation, the user performs the following steps:

Our shape-changing base structure designs can be printed in one pass without needing any assembly. We used a dual-material Ultimaker S5 3D printer with a 0.4 mm nozzle width and 0.1 mm layer height. The nodes were printed with PLA/ABS and the flexible joints were with TPU95A. Both when using PLA and ABS for the rigid part, TPU links would occasionally detach from the rigid node. We avoided this by designing the connection points of the TPU sections larger to hook into the rigid structure and resist the longitudinal stress.

The material choice for the stiff and flexible material can be varied, but the important aspect is that the two materials should provide a decently strong connection when printed. Using another flexible material could mean a different elastic modulus, thus changing the ideal values for the deformation propagation function explained in Section 4.2.

Figure 8 shows the fabrication of the example fish application. All applications are first printed as a metamaterial structure and then enhanced through modular add-ons after fabrication. The modules are connected via snap-fit connections for easy attachment and detachment. We used ABS for the snap-fit connections as PLA is too brittle for such bending structures. The modules allow 1) actuator placement for enabling active control, 2) metamaterial combination for scaling or reusability, 3) connection to stable surfaces for grounding or shape attachment and 4) physical enhancing the prototype for desired tasks. In this work, we explored a subset of many possible modular enhancements. The addition of different actuator types, physical enhancements or sensors could unlock many more application possibilities, which we discuss in Section 7.2.

In this section, we present four types of modules that were essential to the applications presented in Section 6.

Servo motors are connected across a joint. The motor connection hubs are designed to align the shaft of the motor at the middle point of the two actuated nodes. Figure 9 shows our actuator module with snap-fit connections that can be plugged across a joint. When actuated, the motor creates a rotary motion around the joint and actuates the cell.

Metamaterial connection hub allows transferring the movement of the metamaterial along to another metamaterial structure. This helps generate larger structures that can not be printed at once the design and reusing the same fundamental piece in multiple assemblies. Figure 10 demonstrates two metamaterial structures being combined together by two-way snap-fit connectors. These connectors help rigidly connect nodes of two separate metamaterial structures.

Stable connection hubs are used for connecting the metamaterial to a surface to ground it or for connecting a shape that should not be affected by the internal movements of the metamaterial. Figure 11 presents how the stable connection hub is connected to the metamaterial structure and to an external shape. The stable connection hub connects two adjacent nodes of the metamaterial structure to a stable surface through two long flexible beams. This averages out the orientations of the inversely rotating adjacent nodes and keeps the orientation of the external shape unchanged. This is desirable for adding static shapes that are not meant to rotate along with individual nodes during the shape change of the structure.

Physical enhancements can be used for enhancing the physical connection of the metamaterial to the environment. In this work, we present friction pads for enhancing higher contact friction with the surroundings. Figure 12 presents a module for adding onto the node which is used for facilitating designs that move across a distance on a surface or grip an object by adding high-friction material onto the connection hub. Section 6.1 and 6.3 present two applications where the use of the friction module plays an essential role.

In this application example, we introduce two interfaces leveraging a common metamaterial base design to execute two distinct natural modes of movement: crawling and swimming. This application demonstrates the efficacy of a simulation tool capable of facilitating iterative exploration across various actuator configurations and actuation signals.

For the crawling worm application, both actuators are controlled to perform sinusoidal movements. By controlling the phase offset between the sinusoidal motor movement profiles we achieve the crawling behavior. Switching the sign of the phase offset causes the direction of the crawl to reverse. The movement of the worm is facilitated by the addition of friction pad modules. As shown in Figure 13, a single crawling motion can cover a large distance thanks to the agile design of the interface.

In the case of the fish prototype shown in Figure 8, the motor on the tail side of the fish is controlled in a sinusoidal motion profile providing the propelling movement of the fish. The motor on the head side of the fish controls the swimming direction of the fish.

Nature inspiration has been a big part of clothing design. This application was inspired by how the natural flapping motion of butterflies involves the top edge of the wings to lead the bottom edge to be able to push air downwards. Similar to the movement of the crawling worm application, the control of the wings is performed using two motors that move in sinusoidal trajectories that have a phase difference between one another.

In this example, the butterfly model is constructed by combining four separate metamaterial pieces and two actuators, as shown in Figure 14. The building blocks of this prototype can be repurposed for a different design. Through the modularity of the metamaterial structure, the designer can experiment with metamaterial cells with different joint configurations (i.e., print replacement butterfly wings that bend more or less) without needing to reprint one large interface again.

Our sparsely actuated metamaterials provide more flexibility to traditional mechanical structures, for example, a gripper. Instead of having binary closing and opening-up states in a gripper, having a few actuators allows us to control each arm of the gripper separately. The flexibility also allows the links of the gripper to conform with the shape of the object better than a stiff gripper would do. These enable the gripper to produce more fine-tuned motion, enabling more complex functionalities such as adjusting the gripping size based on the object, and selectively opening up arms to control the releasing direction of the object.

With this application, we demonstrate that the simulation of sparse actuation does not need to only model the use of multiple actuators on one interface. Angular constraints, presented in Figure 16, can be used alongside actuators to simulate a constant local deformation. In this application, we present an umbrella-holding mechanism that is already curved in the rest state due to the angular constraint connected over its joint on one end. A motor on the other end of the structure is actuated to fully close the holding mechanism once the user places the umbrella. The grasping style of the umbrella holder was inspired by octopus tentacles. Along these lines, augmenting the umbrella holder with additional friction or suction by providing modular add-ons could increase the grasping quality for various other objects.

Our deformation approximation algorithm is a simplified energy function and it relies on certain assumptions to expedite its simulation. The approximation generates a good enough estimation of the actual deformation behaviour. This, however, can be further improved by using a physics-based method to simulate more accurate and realistic deformation behaviour which can incorporate interactions with the environment. In this paper, we focus on providing the designer with a fast enough algorithm to support their shape exploration. A slower but more accurate simulation could follow the existing one after the designer experiences real-time exploration and finishes the parameter refinement. We leave the implementation of a more physics-based approach to future work.

One benefit of our approach is that its modularity enables further extending many aspects of our method. This can lead to tailored solutions and more functional metamaterial structures in terms of hardware setup, actuation method, etc. The most basic extension based on our current structures is to expand our current set of modules to incorporate sensing (e.g. magnetic, capacitive) for achieving modular interactive displays. The combination of the flexible nature of the structure with internal deformation sensors could provide us with opportunities for active kinesthetic feedback during tangible interactions with the shape-changing interface. External sensing methods can be helpful in interactive applications such as adding proximity sensors to presented gripper applications for object detection. Embedding conductivity in the 3D printed structure [16] can provide more ways of implementing sensing capabilities or seamlessly embedding external sensors. In this work we only presented a friction module as a physical enhancement module. Other types of passive modules such as magnets [11] or suction cups can be added to increase the environmental interaction capabilities of the cells.

Our designs consist of two types of cells: basic and bending cells. Ou et al. [34] presented additional kinematic configurations such as shearing and twisting cells. For this work we decided to work with a smaller group of available primitives to explore wider into what is possible with only these two, since the versatility of our interfaces is not due to the type of movements we can offer but due to the range of movements provided by the joint flexibility and sparse actuation. Expanding into the cell would without a doubt expand the set of achievable shapes.

We used lightweight and small DC servo motors to actuate the cells because they are attainable and fit the requirements of the cell actuation (-90 to 90 degrees movement range). A problem with these motors is that they suffer from higher noise levels than more expensive ones. We observed that the inherent elasticity of the system works as a dampening factor and the structure does not vibrate as much as a rigid structure would. Still for achieving higher torques and more stable actuation, bigger and higher quality motors can be used. Additionally, other actuation methods such as pneumatics or shape memory alloys can be an interesting extension to this work although the simulation tool would need to be adjusted for it. We believe that shape memory alloys complement the lightweight nature of the metamaterial-based structure for applications where their lower actuation speed is not a problem.

During our actuation tests, we did not observe any significant material failure or plastic deformation. These tests included running presented movements for the application cases for extended repetitions. When using angular constraints for a long duration, as in the umbrella holder application, the joints that are most affected by the constraint did not fully bounce back to their original straight form after the removal of the angular constraint. In time they straightened further but a partial deformation was sustained. This deformation due to persistent stress, also known as creep, did not cause failure and the affected joints still operated. This effect can be further studied for analyzing the reusability extent of the presented structures.