NFCSense: Data-Defined Rich-ID Motion Sensing for Fluent Tangible Interaction Using a Commodity NFC Reader
DOI: https://doi.org/10.1145/3411764.3445214
CHI '21: CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, May 2021
This paper presents NFCSense, a data-defined rich-ID motion sensing technique for fluent tangible interaction design by using commodity near-field communication (NFC) tags and a single NFC tag reader. An NFC reader can reliably recognize the presence of an NFC tag at a high read rate (∼ 300 reads/s) with low latency, but such high-speed reading has rarely been exploited because the reader may not effectively resolve collisions of multiple tags. Therefore, its human–computer interface applications have been typically limited to a discrete, hands-on interaction style using one tag at a time. In this work, we realized fluent, hands-off, and multi-tag tangible interactions by leveraging gravity and anti-collision physical constraints, which support effortless user input and maximize throughput. Furthermore, our system provides hot-swappable interactivity that enables smooth transitions throughout extended use. Based on the design parameters explored through a series of studies, we present a design space with proof-of-concept implementations in various applications.
ACM Reference Format:
Rong-Hao Liang and Zengrong Guo. 2021. NFCSense: Data-Defined Rich-ID Motion Sensing for Fluent Tangible Interaction Using a Commodity NFC Reader. In CHI Conference on Human Factors in Computing Systems (CHI '21), May 08–13, 2021, Yokohama, Japan. ACM, New York, NY, USA 14 Pages. https://doi.org/10.1145/3411764.3445214
1 INTRODUCTION
Near-field communication (NFC) systems have been widely deployed for object identification. Recently, NFC readers have also become integrated into many smartphones and smart devices to enable their users to easily and privately access NFC tag information without using their camera. An NFC reader, known as a high-frequency (HF) radio-frequency-identification (RFID) reader, can reliably recognize the presence of a tag at a high read rate (e.g., ∼ 300 tags/second), which makes NFC useful in industrial applications, such as workflow management.
Human–computer interaction (HCI) researchers have also extensively used NFC or RFID tags for prototyping tangible user interfaces (TUIs) [22] because of their low price and ready availability [31, 34, 45]. However, this high-speed mode in single-tag reading has been rarely exploited in the context of HCI applications. Unlike machines, we as humans cannot operate a tag at such a consistently high speed. We may use multiple tags simultaneously. However, unfortunately, an NFC or HF RFID reader is potentially unable to resolve collisions of multiple tags effectively; alternatively, such readers may require an additional time-domain mechanism, such as dynamic framed slotted ALOHA (DFSA) [10], which results in the reader taking extra time to resolve the tag collisions. Furthermore, the order in which tags appear cannot be guaranteed. These reliability and efficiency concerns potentially hinder the user experience. In general, the HCI application of NFC tags is limited by a discrete, hands-on interaction style where one tag is used at a time.
To remedy this problem, in this paper, we present NFCSense, a data-defined rich-ID motion sensing technique for fluent tangible interaction design. The design space is defined based on data collected from commodity NFC tags and an NFC reader operating in the high-speed reading mode, which allows the motion features of an action performed on a tagged object to be extracted and recognized from the rich-ID time-series data. Our investigation began with an analysis of the feasibility of repurposing a commodity NFC reader as a speed sensor for obtaining data on the transient time-domain features (i.e., speed and frequency) of a moving NFC tag. Subsequently, we determined the optimal design parameters for practical use by analyzing data on the three-dimensional (3D) activation map of NFC tags. A tangible interaction design space, including the primitive designs of physical tokens, constraints, and motion sensing algorithms, was defined by these acquired parameters.
The applicability of this approach was demonstrated by a proof-of-concept implementation of a system, which comprised a set of designs for fluent tangible interaction. Figures 1 a and 1 b show that the system can extract the time-domain features from the effortless actions performed by users on the NFC-tagged object. As indicated in Figure 1 c, with a physical constraint that modulates the tag's location and ensures that signal collisions are avoided, batch operations on multiple NFC tags can be performed to accelerate the workflow and increase throughput. In this study, we also explored wearable and non-planar reader antenna designs to extend the scope of use and the range of platforms through which this technique can be employed.
NFCSense not only enables new input modalities but also provides hot-swappable interactivity. Because a commodity NFC reader can reliably identify a virtually unlimited number of IDs in a contactless fashion, the system automatically identifies which set of algorithms to use for a given type of tagged object when the user interchanges the hardware components on the go. The smooth transition warrants fluent user experiences in an extended period of use when multiple tagged objects are involved.
Our main contributions are two-fold, spanning the domains of engineering and design: 1) we realized the data-defined rich-ID motion sensing approach by using a commodity NFC reader, which was validated through technical validation and a proof-of-concept implementation and 2) we formulated a fluent tangible interaction design space that is enabled by hot-swappable interactivity with existing objects. By using simple readily available hardware, HCI researchers and designers can apply our method for rapid prototyping TUI in conjunction with existing objects for a wide range of computing applications in education, entertainment, and health care. We also envision this technique to be further generalized for enriching the design space of mobile human-computer interaction as many smart devices (e.g., smartphones) have a built-in NFC reader nowadays.
The remainder of the paper is organized as follows. First, we present the background of our study and review the literature. Subsequently, we describe our feasibility study and series of data-driven analyses. Thereafter, we present our data-defined rich-ID motion sensing approach, which was based on the aforementioned analyses, as realized in a proof-of-concept TUI system that instantiated our interaction designs in practical applications. Finally, we conclude this study by discussing design implications, limitations of this study, and future research directions.
2 RELATED WORK
2.1 Token+Constraint TUI Systems
Tangible user interfaces (TUIs) seamlessly couple digital information with physical forms [22] and leverage our existing cognitive and spatial knowledge to make digital information embodied and comprehensible. Token+Constraint TUI systems [44] combine the physical/digital artifacts of tokens and constraints, allowing the users to map structured compositions of these tokens onto a variety of computational interpretations. The affordance [14] of physical constraints can facilitate and guide user interaction [13] and limit the possible range of token movements to make the future state of such movement predictable. With physical constraints, researchers can further explore the design space of passive TUIs that are actuated only by gravity, such as the Marble Answering Machine [8].
2.2 Prototyping with NFC/RFID Technologies
Passive NFC/RFID technologies are useful for prototyping TUI systems involving multiple objects with ease. Compared with object sensing technologies that can identify passive objects (e.g., audio recognition [18, 26, 40] and the use of a capacitive sensing matrix [46] and pressure sensors [17]), An NFC tag provides a smaller footprint while yet providing a virtually infinite number of IDs; furthermore, it does not require physical contact for a reliable reading. Unlike the tags used in computer vision, the sensing of NFC tags does not suffer from line-of-sight problems. Therefore, NFC technology provides a reliable identification mechanism that helps the system identify both the object in action and the action set bound to the object [4]; in doing so, the system helps users easily associate the content with the symbolic appearance of the content's original form and functions [35]. Commercial products use NFC tags in toys-to-life applications, such as Nintendo Amiibo 1 and Lego Dimensions 2, where in-game content is unlocked by the player placing NFC-tagged toy figurines on an NFC reader. Researchers have also applied near-field RFID to prototype token+constraint interactions, such as DataTiles [37] or to detect page-flipping events in interactive book applications [3].
Because NFC systems do not primitively support far-field tag identification, the tag and reader must be tightly coupled in the application. Another subset of the literature involves applying ultra-high-frequency (UHF) RFID systems to sensing touch inputs on tags [19, 23, 28, 47, 48], making tangible user interfaces [20, 27, 43], and making wearable input devices [24, 30]. Although UHF RFID systems support a longer distance (∼ 6m) and room-scale deployment using a single reader, the deployment cost of UHF RFID readers is significantly higher than their NFC counterparts. The much higher sensing latency (∼ 200 ms [43]) of UHF RFID systems also limits their application in time domain.
2.3 NFC for Interactivity
Beyond focusing on NFC's basic capability to identify tags, researchers have also enabled interactivity on NFC tags by leveraging their signals to detect presence [3], touch [38], and rotation [11]. Furthermore, advanced techniques have been applied, such as using on–off keying to encode 1D inputs on a tag-embedded widget [42], embedding additional electromechanical switch circuitry for additional input [34], and forming multiple tags as compound widgets [1, 15]. Project Zanzibar [45] explored a broad range of tangible interactions, including multi-object tracking, stacking, translation, and rotation with an NFC antenna array. Nonetheless, the focus among researchers has been directed more toward tracking the static states of tokens rather than tracking their transient states, such as speed and frequency, and the use of physical constraints.
Researchers have also applied sensor fusion to extend the capability of NFC systems, such as incorporating magnetic sensing with the RFID tag to realize a 5-degree-of-freedom (5DoF) input of an RFID tag [31], implementing NFC with capactive sensing [16], or using an extended NFC reader as a two-dimensional antenna array that is coupled with a capacitive touch sensor [45]. By incorporating an additional motion sensor, previous studies have also proposed wearing motion sensing RFID readers in apparel and devices such as gloves [12], bracelets [5, 12], wristbands [9], rings [7], and finger-worn devices [32] for ubiquitous computing applications. Nonetheless, for makers who wish to prototype rich-ID motion sensing systems with some existing set of objects, these delicate sensor fusion systems are more complex and difficult to obtain than is a single commodity NFC reader.
2.4 Technologies for Sensing Fluent Tangible Interaction
In the context of designing a fluent tangible interaction design space, occlusion-free NFC technology can differentiate itself from occlusion-sensitive computer vision [2, 39] by allowing users to use a tagged object with greater freedom. NFC provides a much larger ID space than other contactless and wireless sensing technologies that do not provide rich ID (e.g., magnetic tracking [6, 29]). NFC also reduces the physical contact that is otherwise required by other rich-ID technologies that are dependent on physical contact [17, 18, 20, 46]. NFC technology can do so because it requires no electromechanical connectors that may constrain object movement or material choice. With the rich-ID motion sensing proposed in this paper, users can enjoy not only effortless operation but also hot-swappable interactivity, which provides the user a fluent experience with time when using an occlusion-free, contactless, and rich-ID NFCSense system.
3 BACKGROUND
This section introduces the theoretical background of this study, specifically the operation principles underlying NFC systems, tag activation in motion, and collision detection and avoidance.
Operation Principles of an NFC System. NFC systems use a magnetic dipole antenna for power transfer. Figure 2 a and 2 b depicts a simple representation of an NFC system, which comprises a reader and a passive RFID tag, where B is the vector of the time-varying near-field magnetic field produced by the AC current flowing into the reader antenna coil, S is the normal vector of the surface area of the tag antenna coil, θ is the angle between B and S, and z is the distance from the center of tag antenna to the center of reader antenna. When the tag and reader antennas are a distance z apart from each other, this magnetic field B induces a voltage Vtag that causes a current flow on the tag antenna coil. The B is given by
(1)
where I is the current, a is the radius of the reader's loop antenna, and μ0 is the free-space permeability (4π × 10− 7H/m). Therefore, the Vtag can be represented as
(2)
where N is the number of turns in the tag antenna coil and ϕ is the magnetic flux passing through each turn. B · dS is the inner product that is related to the angle θ between the two vectors. Equations 1 and 2 state that the voltage Vtag is determined by two main factors: 1) the distance z between the reader antenna and the tag antenna, and 2) the angle θ between the magnetic-field vector B produced by the reader antenna and the surface area S of the tag antenna (Figure 2 b). When the distance z is too large and the value of cosθ is too small, the value of Vtag is insufficient for the tag to be activated. Otherwise, when the tag receives sufficient energy from the reader, it activates and transmits its identifying inventory number back to the reader.
Tag Activation in Motion. Figure 2 c illustrates motion involved in tag activation. When a moving tag passes through a reader's sensing range along a linear path L, the tag activation time Ton can be modelled as Ton = Lon/V, where Lon is the path length when the tag is activated, and V is the actual translation speed of the tag. When Ton ≥ 2/fread [41], where fread is the read rate of the NFC reader, the translation speed of tag motion can be sensed and resolved as V = Lon/Ton. However, the Lon depends not only on the tag-to-reader distance z and angle θ but also on the type of tag antenna as well as the shape of the reader antenna. For instance, the magnetic fields of a common on-board square or rectangular antenna are distorted when in close proximity to the coil. These factors make the Lon of a tag-reader pair unpredictable by the aforementioned theories in practice.
Collision Detection and Avoidance. Tag collision occurs when multiple NFC tags are energized and read by an NFC reader simultaneously. The reader detects such cases using the error-correcting code in the received ID data sequence and is unable to differentiate these signals. The common solution is a time-multiplexed collision avoidance mechanism (e.g., slotted ALOHA [10]), which is implemented by sending a signal back to the tags after the reader recognizes that tag collision has occurred. Upon receiving this signal, each tag uses its own random number counter to determine how long to wait before sending its data, and each tag sends their data at different times. Although this solution allows multiple overlapping tags to be detected after some time, this solution introduces random additional delays to the system, and the order of tag detection cannot be guaranteed.
Summary. Although the theory is well-established, estimating a tag's actual activation behavior is nontrivial in practice because the performance is determined by the specifications of the tag and readers (i.e., their antenna design or sensitivity), which vary between commodity devices. In this work, we adopt a data-driven approach to directly acquiring the actual tag activation parameters through a series of empirical measurements on a commodity NFC reader. We first investigate whether the fread of a commodity reader is feasible for supporting tag motion sensing in HCI applications. Then, we explore the tag activation behaviors by using off-the-shelf NFC tags on the reader with respect to the tag-to-reader distance z and angle θ, as well as the tag-to-tag distance dgap, to define the tangible interaction design space for NFC tag motion sensing.
4 FEASIBILITY STUDY: SENSING TAG MOTION SPEED USING AN NFC READER
We first investigated the feasibility of using an NFC reader as a motion sensor, akin to using a tachometer to measure the speed of motion. We aimed to understand whether an NFC reader can measure the rotation speed of a tag mounted on a shaft or a disk by measuring the frequency of the presence signal, akin to how a tachometer measures the rising edge of a square wave. Furthermore, we investigated whether the NFC reader can determine the translation speed of a tagged object from the duration between when a tag enters and when a tag exits an area, which is equivalent to the pulse width of a square wave signal.
We first evaluated the stability and sampling rate of the NFC reader, which was an off-the-shelf 13.56 MHz NXP MFRC522 MIFARE HF RFID reader (Figure 2 a) that supported ISO/IEC 14443 A/MIFARE tags. This $5 reader was evaluated without any hardware or firmware modification, and it was mounted on a breakout printed circuit board comprising a four-loop rectangular planar spiral antenna, which had the dimensions of 39.5 mm (width; W) × 41.5 mm (length; L) and a width circuit trace of 1.2 mm. An NXP S50 Mifare Classic 1K PVC coin RFID tag (Figure 2 a), which had the dimensions of 12.5 mm (radius; r) × 1 mm (thickness; t), was used for measurement. An Arduino Uno board based on an ATMega328 microcontroller was used for signal processing and data transmission to a PC. The reader was scheduled to read a tag every 10 ms in 5 s. During the measurement, the timestamp after each read was recorded, regardless of whether a tag was detected. Figure 3 b shows the results of the tag read duration. When the tag was detected, the mean duration was 3.176 ms (standard deviation [SD] = 0.013ms); when the tag was not detected, the mean duration was 1.950 ms (SD = 0.014ms). According to the results, the NFC reader could achieve fread = 311 tag reads per second with $> 99.7\%$ of tag presence events detected, and a signal of a frequency $\le \frac{f_{read}}{2}$ = 155.5 Hz or a pulse width duration $\ge \frac{2}{f_{read}}$ = 6.4 ms could be captured without information loss [41].
Responsiveness was tested by measuring whether the tag read had a low latency. We tested the ability of the reader to sense rotation speed with the experimental apparatus shown in Figure 3 a, which allowed for a direct comparison with a magnetic sensing system. A Winson WSH136 analog Hall sensor was attached to the center of the reader's antenna. Placed in a 1-mm-thick holder, a 5 mm (r) × 2 mm (t) N35 Neodymium magnet was attached 1 mm above the center of the tag used in the sampling rate study. The tag, which had the NFC and magnet, was attached to the wing of a fidget spinner that was affixed 15 mm above the reader's surface. Based on the previous results, we synchronized the readings of the NFC reader and the analog Hall sensor at a rate of 300 reads per second in 10 s when the spinner was spinning and decelerating from 11 revolutions per second (rps) to 5 rps; these measurements were obtained with a software timer programmed in Arduino. A total of 10 s × 300 reads per second = 3000 readings were collected.
As indicated in Figure 3 c, the readings of both the NFC reader and the analog Hall sensors were positively correlated (Pearson correlation: ρ = 0.70 s), indicating that the latency of the NFC reading rotation was as low as that of magnetic sensing at a read rate of 300 Hz. Nonetheless, at a lower speed, the transition between the presence and absence of a tag did not fully align with the transitions in the Hall sensor output due to the difference in activation areas between the NFC reader and the magnetic sensor. Because we did not know the actual activation area of the NFC tag on this reader, we cannot conclude whether the translation speed of a motion can be determined from the pulse width of a square wave as detected by an NFC-based tachometer.
5 EXPLORATORY STUDY: OBSERVING HOW NFC TAGS GET ACTIVATED
To further investigate the activation area of an NFC tag on the NFC reader used in our study, we conducted a series of measurements to understand the types of design problems that may arise in practice. These measurements were obtained from observations of the activation of these tags under various conditions. These observations were gathered over four sessions.
5.1 Session 1: Activation Map of a Non-Tilted Tag
This session examined the presence signal pertaining to a tag's 3D location. In this session, the angle θ between the tag and the reader surfaces was set to 0; in other words, the tag was not tilted.
Apparatus. Figure 4 a depicts the experiment apparatus. A modified delta 3D printer was used as a robotic arm holding the tags. We replaced its nozzle with a tag holder built from Duplo blocks and 3D printed components, and we put a reader holder, which was also made from Duplo blocks, on the printbed. We programmed the printer to move the tag holder at a precision of 0.2 mm, and we placed an ADXL335 three-axis accelerometer, which had a 250 Hz sampling rate, next to the holder to monitor the motion of the print head. A Mifare S50 PVC coin RFID tag and an MFRC522 NFC reader, which we used in the feasibility study, were mounted on the tag and reader holders, respectively. To eliminate electromagnetic interference, both the tag and reader were placed at least 5 cm away from the metal components of the 3D printer. The centers of the tag holder and reader holder were carefully aligned at a precision that allowed for the stacking of two Duplo blocks together.
Procedures. Because the magnetic field emitted from the rectangular antenna had a reflectional symmetry in the three Cartesian axes with its center as the origin, the tag activation could be described through measurements of only one quadrant. An NFC tag is used in the measurement. We first aligned the center of the tag to the center of the reader antenna at 1 mm above the center of the sensor. Then, we took a measurement as the 3D printer moved a tag to every 1 mm grid location in a 39 (W) × 39 (L) × 40 (height; H) mm3 volume. After the tag was moved to a location, as determined by the motion sensor, the system took 20 samples of the binary presence state, and the tag then moved to the next location until a full traversal had been made. A measurement took 100 ms at each location. In total, 40 (W) × 40 (L) × 40 (H) × 20 = 1,280,000 samples were taken, and the total measurement time was ∼ 1.6 h.
Results. The measurement results are presented in Figures 4 b, in which a 3D scatter plot shows the locations that had at least 95% (19 out of 20) of tag presence instances being detected. The tag presence distribution agreed with the electromagnetic field distribution of the antenna. In the z axis, the cross-sectional area of the tag presence was dependent on the hover height of the tag. Figures 4 d shows the maximum hover height was detected at zmax = 33mm. The maximum cross-sectional quadrant area A = 457mm2 was detected at z = 11mm, where we obtained the maximum distances of Lx = 22mm (x axis) and Ly = 22mm (y axis) from their central component. Notably, when a tag was at a hover height of z ≤ z* = 6mm, the tag was first, activated not only at the center but also at the periphery of the antenna and second, was inactivated between the two activation zones. Within the range of 21 ≥ z ≥ z* = 6mm, the measurement was of similar validity because the distances to the center were ≤ 1mm less than Lx = 22mm.
Figures 4 e and 4 f present the cross sections at z > z* and z < z*, respectively. They both show that Li = Ly = 22mm when i ≤ wz and wz > 0 in the cross section of every z value except for z = 32mm. This result indicates that the activation area formed by the rectangular reader antenna was mostly of a round-rectangular shape; this meant that a valid measurement along the x axis (or y axis) did not have to strictly align with y = 0 (or x = 0).
Discussion. With regard to our application to sensing a moving NFC tag, the most important insight extracted from the results was that knowing where the NFC reader ought to be deployed makes the tag presence signal pattern more predictable. We recommend keeping the tag-to-reader distance steady at either z > z* or z < z* throughout the speed measurement to simplify the algorithm used for motion sensing. The z* value may vary between tags and antennas because their electromagnetic-field coupling can differ.
5.2 Session 2: Activation Map of a Tilted Tag
In this session, we further examined the presence signal of the same tag used in Session 1 with respect to its 3D location and tilt angle θ.
Apparatus. The apparatus used in this session was identical to that in Session 1. An additional 3D-printed adapter was used to affix the tag at seven angles (0°, 15°, 30°, 45°, 60°, 75°, 90°), as shown in Figure 5 a.
Procedures. Because the cylindrical NFC tag was rotationally symmetric, measurements of only pitch angle were sufficient to describe the effect on the angle of the tag. Nonetheless, we had to measure two quadrants to be able to sufficiently describe the tag activation distribution on this antenna. Seven pitch angles were measured in this session. Similar to Session 1, prior to taking measurements, we located the 12.5-mm radius tag 13 mm above the center of the antenna. Subsequently, we took measurements as the 3D printer moved a tag to every 1 mm grid location in a 3D [-40, 39] (W) × [0, 39] (L) × [13, 35] (H) mm3 volume. After the tag was moved to a location, 20 samples of the binary presence state were recorded. Thereafter, the tag was moved to the next location until a full traversal had been made. A measurement took 100 ms at each location. In total, 7 (angles) × 80 (W) × 40 (L) × 23 (H) × 20 = 10,340,000 samples were taken, and the total measurement time was ∼ 14.3 h.
Results. Figure 5 b shows the measurement results. The cross-sectional area of the tag presence along the z axis was dependent on the tilt angle θ of the tag. The maximum hover height zmax = 28 mm was the same when 0 ≤ θ ≤ 30°. However, when θ > θ* = 30°, a blind gap was observable in the antenna area (20 > x > -20 and 20 > y > -20) at every hover height on the z axis. Increasing the tilt angle θ broadened the blind gap, centered the blind gap, and reduced the maximum hover height. The thinnest gap observed at θ = 90° was 16 mm, which lay between x = -8 mm and x = 8 mm.
Discussion. To reliably sense a moving cylindrical NFC tag, the tag should not be tilted more than an angle of θ*. Because a slight ≤ θ* tilt still allows for reliable sensing, the orientation of a tag does not need to be strictly fixed. However, embedding the tag into a 6-degree-of-freedom (6DoF) object, such as a sphere, does not guarantee that the presence signal can be picked up by a rectangular antenna. The angle θ* may vary between tags and antennas because their electromagnetic-field coupling can differ.
5.3 Session 3: Activation Map of a Pair of Separated Tags
In this session, we further examined the presence signal pertaining to the 3D location of a pair of cylindrical NFC tags that were separated at various distances.
Apparatus. The apparatus used in this session was similar to that in Session 1, except for the addition of two different NFC tags, TagA and TagB (both z*=5 mm) for measurement. An additional 3D printed adapter was used to affix the two tags at different gap distances from 0 to 18 mm.
Procedures. We had to take measurements in two quadrants to sufficiently describe the tag activation distribution on this antenna. Nine gap distances dgap between TagA and TagB from 0 to 18 mm (2 mm step) were measured in this session. Prior to taking measurements, we aligned the center of TagA to the center of the reader antenna at z* = 6mm above the center of the sensor. Subsequently, we took measurements as the 3D printer moved a tag to every 1 mm grid location in a [-65, 24] (W) × [0, 24] (L) × [6, 34] (H) mm3 volume. After the tag was moved to a location, 20 samples of the binary presence state were recorded; thereafter, the tag was moved to the next location until a full traversal had been made. A measurement took 100 ms at each location. In total, 10 (gap distances) × 90 (W) × 25 (L) × 29 (H) × 20 = 13,050,000 samples were taken, and the total measurement time was ∼ 18.1h.
Results. Figure 6 shows the measurement results. When dgap ≤ 14mm, intersections were observed between the activation maps of TagA and TagB. Notably, neither TagA nor TagB was reliably detected in these intersections due to the signal collisions, causing a blind gap between the two tags. The smaller the dgap was, the wider the blind gap was. When dgap = 0, the maximum hover height of TagA was also reduced. When dgap = 0 mm and y = 0mm, the widest gap between TagA and TagB was 11 mm, where z = 6mm, and the maximum combined length of the presence of TagA and TagB was 66 mm, where z = 15mm.
Discussion. The detection of the presence of two side-by-side tags does not seamlessly transition from one to another if the gap distance dgap between the two tags is too small. The blind gap turns part of both tags’ active area into an unstable or even inactive area, making the detection not only less efficient but also less reliable. The remaining activated area may still be sufficiently large for the tag to be reliably identified by the reader, but the maximum allowable speed is reduced. Accordingly, the maximum throughput is also reduced. When the dgap is too large, the gap between the two activation areas also reduces the channel utilization and throughput. Hence, an optimized dgap should be sufficiently large to allow the two areas to touch each other without intersecting. The users can define a $d_{gap}^*$ as a threshold of signal quality in their application.
5.4 Session 4: Variance Between Tags
To understand how the aforementioned results generalize, we tested various tags with the sample apparatus and procedures used in Session 1. Figure 7 a depicts the tags that we used for measurements, which were as follows:
- 20 × T0: 12.5 mm (R) × 1 mm (T) Mifare 1k Classic NFC Coins (the same one used in Session 1-3)
- 1 × T1: 12 mm (W) × 22 mm (L) NTAG213 Micro NFC Tag
- 1 × T2: 5.5 mm (W) × 16 mm (L) NTAG203 NFC Tag
- 1 × T3: Keychain embedded an NFC tag with a 13-turn 23 mm (outer diameter) × 18 mm (inner diameter) coil
- 1 × T4: 15 mm (R) × 0.1 mm (T) NTAG215 NFC Tag Sticker
- 1 × T5: 50 (W) × 70 (L) × 1 (T)mm3 NFC Card
We measured a two-dimensional 39 (W) × 39 (L) × 40 (H) mm3 sensing plane, where the (x, y) = (0, 0) point was defined to be 1 mm above the center of the reader antenna.
Figures 7 b and 7 c present the measurement results. Among the six types of tags, T4 and T2 performed the best (zmax = 35mm, z* = 6mm) and worst (zmax = 7mm, z* = 0mm), respectively. This result demonstrates that the performance of the tag is correlated to the form factor of the tag antenna, as depicted in Equation 1. Because tags T1 to T4 exhibited an activation map that was similar to T0, the preceding results applied to them. The different activation map of T5 indicated that our preceding findings cannot be generalized to T5, which was the only tag that was larger than the reader antenna.
Regarding the internal validity of the 20 T0 tags, their mean maximum height was $\overline{z_{max}}$ = 31.55 mm (SD = 1.99 mm), their mean maximum distance to the center was $\overline{L_y}$ = 21.55 mm (SD = 0.89 mm), and their mean minimum hover height for signal activation was $\overline{z^*}$ = 5.55 mm (SD = 0.76 mm). The low variance between these 20 samples indicated that the parameters obtained from a tag can be transferred to another tag with the same specification.
6 DATA-DEFINED RICH-ID MOTION SENSING USING A SINGLE NFC READER
Based on the empirically obtained design parameters in Session 4, we formulated a set of empirically grounded rich-ID sensing approaches called NFCSense, which includes guidelines for designing and implementing the physical form of tag-embedded tokens and physical constraints, and software algorithms for extracting the features of the motion of rich-ID tags.
6.1 Form Primitives for NFC Tagged Tokens
This section introduces the form primitives of the tokens and the physical constraints. Figure 8 presents the four basic form primitives of token design, where the token comprises an NFC coin and a physical support made of nonconductive materials. For single-tag tokens, we recommend a single NFC tag to be deployed under three configurations:
- z < z*: z < z*, θ < θ* (Figure 8 a)
- z > z*: z > z*, θ < θ* (Figure 8 b)
- θ > θ*: z > z*, 90° > θ > θ* (Figure 8 c)
where z and θ are the tag-to-reader distance and angle, respectively, and z* and θ* are empirically determined thresholds of z and θ, respectively. With a linear physical constraint that fixes the θ and z values of a tagged token moving above the reader, these three configurations can be recognized by the different number of pulses (Figure 9).
When using multiple single-tag tokens or a sole multi-tag token, we recommend for the tags to be deployed according to the following configuration:
- $d_{gap}> d_{gap}^*$: $d_{gap}> d_{gap}^*$, z > z*, θ < θ* (Figure 8 d),
where dgap is the tag-to-tag distance and $d_{gap}^*$ is the user-defined signal quality threshold. According to the study results, we recommend a sufficiently large dgap (between the tags) to be maintained to optimize both the usability and the signal-to-noise-ratio. Making each tag z > z* and θ < θ* also reduces unwanted minor components in the activation map.
6.2 Form Primitives for Physical Constraints
NFCSense's input space can be further extended by using physical constraints, which can modulate the primitives’ physical states (z, θ, dgap) and provide a physical reference for manual operation. Figure 10 shows the form primitives of linear, rotational, and simple harmonic motion (S.H.M.) constraints. For linear translation, putting a track above the reader can fix the θ and z values of a tag moving on it. A track-to-reader distance of zp can reduce a z > z* tag's required thickness to z* − zp, thus allowing the tags to be thinner, lighter, and therefore more amenable to further integration. Without compromises to sensing validity, the track can be made two times $w_z< w_z^*$ wider than the primitives, depending on the tag-to-reader distance z. To allow for rotation or simple harmonic motion, a spinner or a swing can be added to modulate the θ and z values of a primitive. A (loose) location constraint is required to ensure that a z > z* tag enters the activation area once every cycle.
6.3 Feature Extraction Algorithms
Figure 11 depicts the workflow of extracting the tag motion features from the rich-ID square waves. The binding between the event trigger's NFC tag ID and its use of algorithm can be predefined, so the system can reliably determine which software algorithms to use without the user manually selecting the algorithms or without a machine learning classifier. Tag motion features can be extracted from the rich-ID square wave of time-series state information using three algorithms.
- I. Linear Translation. When z and θ are known, the linear translation speed VL of a tag sliding through a reader can be computed by VL = L(z, θ)/TM, where L is path length across the major component of the active sensing area at z and θ and where TM is the pulse duration of the major component that has the largest Ton. Furthermore, L(z, θ) can also be determined from the activation map (Figures 4 and 5). When the pulse width of a tag absence, Toff, is longer than two times the tag read duration, the token's primitive type can be identified from the pulse count N obtained from the segment. If a θ > θ* (N = 2) token is identified, the direction of motion can be resolved from the asymmetric pattern (Figure 9 c). A valid segmentation mechanism can be achieved by setting the time-to-live timer tTTL1 larger than the length of Toff between the pulses at a predefined minimal speed.
- II. Rotation or Simple Harmonic Motion. When a tag is identified to be one with constrained rotation or in simple harmonic motion, the rotation frequency f can be computed using $f_{R} = \frac{1}{T_i}$, where Ti is the interval between the rising edges. For simple harmonic motion, the frequency can be computed by $f_{SHM} = \frac{1}{2T_i}$. The acceleration can be derived from the first-order derivative of the frequency. The algorithm outputs either fR or fSHM based on the event trigger's NFC tag ID. The algorithm reset the state of measurement when the time-to-live timer tTTL2 gets expired.
- III. Compound. When a tag is identified to be part of a compound that comprises multiple tags, its linear translation speed $V_L = \frac{\sum ^{N-1}_{n=1}L(d_{gap},D_{tag})/t_i^{n}}{N-1}$, where L = dgap + Dtag (diameter of the tag), or the rotation frequency $f_{R}^n = \frac{1}{T_i^n}$ are resolved using the rising edge interval Ti because doing so yields more robust results. When the same type of tags is used and the dgap is sufficiently large, knowing L(dgap, Dtag) does not require a lookup table. The tag presence order PO also indicates the input mode. For VL, a valid segmentation mechanism can be achieved by setting the time-to-live timer tTTL1 larger than the length of Toff between the pulses at a predefined minimal speed.
7 SENSING USER INPUTS ON TOKENS
Based on the parameters obtained from the exploratory studies, we designed and implemented a system to demonstrate the range of interactivity that can be enabled by such sensing techniques (Figure 12). Instead of making 3D printed objects for embedding physical tags, we 3D printed physical tag holders, which are compatible with conventional physical building blocks (i.e., Lego bricks, Duplo blocks) to make the results replicable and encourage further generalization. The blocks from which the sensing elements were embedded allow for easy (dis)assembly; these blocks were primitively hot-swappable because the NFC tags supported contactless sensing with no need for any electromechanical connectors.
The reader (Figure 12 a) was an NFC reader mounted on a base made of Duplo blocks. The track (Figure 12 b) was 3D printed using nylon, which provided a low-friction 3.4cm(W) × 19cm(L) surface. The top surface of the track was 3 mm away from the reader's surface. The pillar was a supporting structure for tilting the track, allowing an object to slide along the track.
As for objects that supported linear translation (I), the boat (Figure 12 c) had a T0 tag mounted 15 mm above its bottom surface, meaning that the non-tilted tag hovered at a height of z = 18mm > z* above the reader antenna when the boat was sliding along the track. The figurine (Figure 12 d) had a T0 tag mounted on its bottom surface, so that the tag hovered at z = 3mm < z* above the reader antenna at a θ = 0° < θ* tilt angle when the figurine was sliding along the track. The car (Figure 12 h) had a T0 tag mounted on it at z = 18mm > z* above the reader antenna at a tilt angle, θ = 75° > θ*, when the car was moving along the track. The trolley (Figure 12 i) had a 3D-printed tag holder mounted on it with two T0 tags affixed side by side at a gap distance of dgap = 4 mm (III). Both tags were mounted 18 mm above the trolley's bottom surface, meaning that the tags hovered at z = 21mm > z* above the reader antenna plane at a θ = 0° < θ* tilt angle when the trolley was moving along the track. Both the car and the trolley had plastic wheels to reduce friction.
As for objects that supported rotation or simple harmonic motion (II), the white spinner (Figure 12 e) had three T4 tags mounted on each of its 8-mm-thick wings. The three tags had a gap distance of dgap = 20 mm (III). The black spinner (Figure 12 f) had one T4 tag mounted on one of its wings. The swing (Figure 12 g) had a T4 tag mounted at the bottom of its moving part, which was swinging at z = 30mm > z* above the reader antenna when used.
7.1 Sensing Motions
Based on a lookup table constructed from the activation maps in the studies described in Section 5 and on the feature extraction algorithms described in Section 6, time-series signals were collected, which could be used to resolve the frequency and translation speed and to infer the state of a tag from the motion signal patterns.
Frequency. Figure 13 a shows the resolution of the frequency of the black spinner that was spun by a user. As shown in Figures 13 b and 13 c, not only the frequency but also the rotation direction of the white spinner could be resolved based on the fR of each tag and on the order of appearance of the three tags, respectively. Notably, the T4 tag had an activation area with a diameter of >4 cm when 25mm > z > z* = 5mm (Figure 7). Therefore, aligning the tag to the reader is not difficult if the visual affordance of the reader's location is clearly observable. Additional support such as a stabilizer (e.g., an armrest) or visual feedback pertaining to the reader can further improve usability.
Figure 13 d depicts the frequency fSHM of the simple harmonic motion of the swing hanging on the rack was resolvable. By leveraging the ID of NFC tags, the system can automatically switch the detection mode by using the algorithm corresponding to each tag.
Speed. Figure 14 a shows a user sliding a boat along the track to resolve its speed VL. Figure 14 b shows a user releasing the boat from the top of the track to resolve its speed VL. When the angle of incline of the track is known, the location at which the boat is released can be further inferred through the observed speed. Figure 14 c shows a user playing a balancing game with the trolley on the track. The user moves the trolley by tilting the track using both hands. The direction of motion and the acceleration were resolved. These examples demonstrate that both hands-on and hands-off interaction styles are supported by this system.
Additional Tag States from Presence Signal Patterns. Figures 15 a and 15 b show a triple-pulse pattern being generated after a user walked the figurine along the track; this pattern differed from the monopulse pattern generated after a user jumped the figurine over the reader. Figures 15 c and 15 d depict a short-long pattern generated after a user moved the car forward along the track; this pattern differed from the long-short pattern generated after the user moved the car backward along the track. The speed VL of the action was resolved in both cases.
7.2 Extending the Input Space with Physical Constraints
If carefully designed, physical constraints can extend the input space by localizing a tag from various input sources, thus preserving a meaningful tag order, easing the inputs, and increasing the throughput by preventing collisions.
Maintaining the Distance Between Two Tags. The white fidget spinner (Figure 12 e) and the trolley (Figure 12 i) featured a placeholder that maintained the distance between the two tags to reduce interference in tag reading. In the multiplexer example (Figure 16 a), we also added a 30-mm-diameter × 3-mm-thick cylindrical nylon ring to each tag ensure that the tags rolled along the track smoothly.
Localizing a Tag from Various Input Sources. Figure 16 a depicts the multiplexer, which comprised a stack of two tracks that allowed an NFC coin to roll along them. The inner and outer tracks were deployed at a zinner = 4mm < z* and zouter = 10mm < z* distance to the reader, respectively. The signal patterns for tag presence after the user placed the coin tag in each slot differed because the tri-pulse was for the inner track and the mono-pulse was for the outer track. Therefore, the system could recognize where the tag was placed based on these signals. More tracks can be distinguished by adding more outer tracks either with different slopes within a distance zouter < zmax to the reader or at a different angle θouter < θ* to create more speed-related patterns. 3
Preserving a Meaningful Tag Order. Figure 16 b illustrates an educational Montessori-inspired [49] TUI for learning mathematical concepts with a Duplo-block compatible linear tag holder; this was the sorter, which supported four tag-embedded Duplo blocks to be mounted on it in a linear order. Each block, which was embedded in a T0 tag with a Nylon-printed Duplo compatible tag holder, was associated with a certain numerical quantity (e.g., 1, 2, and 3) or an object (e.g., apple, banana, and carrot). The user answers the quiz by putting these blocks together in the correct order, where the block order is read by having the blocks be effortlessly released from the top of the stack. Visual feedback, such as incorrect elements, is shown on the screen, motivating the user to try again.
Easing the Inputs. The funnel (Figure 17 a) extends the entry area of a track and outputs one tag at a time. Users were allowed to throw a tag into the filter to get the tag through instead of placing it carefully on the track. Such easy tag placement can increase the throughput and thus accelerate the workflow. This extension is minimal yet effective, although it only supports a few (e.g., 3–5) tags at a time. Throwing too many tags simultaneously into the funnel may cause jamming, which requires extra time for manual intervention, although the user is also aware of this problem and capable of solving it.
Increasing Throughput by Preventing Collisions. The modulator (Figure 17 b-d) prevents collisions by modulating each tag's location before they are read. We adapted the coin separation mechanism of a mechanical coin sorting machine 4 to the NFC coin tags and constructed a tilted gear-shaped rotating plate with a handle at the back; this plate allowed the users to roll the plate with it. A user was allowed to throw multiple tags into the container with one hand and to resolve any jams by rocking the handle to separate the tag and modulate their locations before they were read.
The modulator requires users to perform simple rolling movements; without the modulator, the user must manually separate tags, which is cognitively taxing and requires fine motor control. Such savings in time and effort are especially apparent in batch operations when many tags are used. Furthermore, this mechanism is inexpensive to deploy and maintain because it requires no electrical power; however, it introduces a body-scale device, which may be too bulky for some mobile applications.
7.3 Extending the Scope of Use with a Custom Antenna
The solid flat-form antenna on the printed circuit boards is not expected to be the bottleneck in applications because many methods are available for flexibly allowing a magnetic dipole antenna to achieve reasonable power transfer efficiency. Several breakout board modules for NFC readers (e.g., Grove NFC readers 5) provide an interface for the users to hook up a custom antenna. Nonetheless, for a module that has no such interface, such as the MFRC522 reader board that we used in this study, antenna modification can be achieved by first cutting the existing circuit trace at both ends of the on-board capacitors, then connecting a low-impedance coil inductor of similar inductance to both ends of the capacitors, and finally experimenting with antenna designs using Equation 1 and an NFC tag.
In this section, we present two examples of custom antennas: 1) a wearable antenna to support mobile applications and embodied interaction and 2) a non-planar antenna to improve the reliability of detecting a tag-embedded marble, which is a 6-DoF object that cannot be monitored with a planar coil when its tilt angle is θ > θ*.
Wearable Reader Antenna for Always-Available Input. Figure 18 a shows a finger-worn NFC reader antenna, which was made of a two-turn 20 mm (W) × 18 mm (H) coil using AWG30 wire. This thin-form antenna had a sensing range of ∼ 2cm for the T4 tag, allowing a user to wear it on their finger as a part of a ring or a skin patch. Figures 18 b and 18 c show a user wearing this antenna to detect the rotation speed and direction of the white spinner. Without being constrained by the location of a fixed NFC reader antenna, users can freely use the spinner in a mobile context and even perform a larger range of bodily inputs, such as blowing on the wing to make the spinning more rapid [33]. These new types of embodied interactions can be captured through the speed pattern, enriched by an additional display on the smartwatch or smartphone, and used as an intervention, in, for example, stress management [33].
Multi-Coil, Non-Planar Reader Antenna for Sensing Tag-Embedded Marbles. Figure 19 a shows a 1 × 3 two-turn 4-cm-diameter round antenna array made of AWG24 single-core wire. Each coil unit had a sensing range of ∼ 3cm for the T0 tag, which we embedded at the center of the 25-mm-diameter marble. The antenna array was deployed on the track. To capture the tag-embedded marble, the middle of the three coils was pointed toward the x axis as the side antenna, whereas the other two faced the z axis (the surface normal of the track) as the bottom antenna. A tag that was in the activation area of any coil could trigger a read. Figures 19 b and 19 c show that the marble movement was captured once by the side antenna when the tag was tilted θ = 90° and was captured twice by the bottom antenna when the tag was tilted θ = 0°. Therefore, the speed of the bottom antenna was calculated using the algorithm III (Figure 11) to approximate the speed at the location of the side antenna calculated by the algorithm I.
Because calculations of speed are less precise for the rolling spheres than for the sliding object, more coil units may be required in the array for statistical estimates of a more precise speed value. Nonetheless, the non-planar antenna design allows designers to use simple hardware to enable fluent interaction experiences for such a spherical form [25].
8 DISCUSSION
Herein, we discuss the design implications, limitations of this study, and directions for future research.
Parameter Acquisition. The form primitive design parameters for the token (z*, θ*, and $d_{gap}^*$) and physical constraint ($w_z^*$) can be simply obtained by manual measurement without a 3D printer or a robotic arm. Nonetheless, designers can use a tag and reader that are different from what we used in this paper. These designers can test a tag by holding it by their fingers and analyze the distance and angle above the reader through real-time feedback on a computer screen or, more simply, an LED that shows the presence of a tag. They can search the local optimum using a gradient descent method based on the guidelines presented. Furthermore, researchers who wish to obtain a higher resolution activation map can also increase the granularity of our experiment apparatus by at least 10 times, because most 3D printer on the market can achieve sub-millimeter precision. The parameter acquisition of each tag-reader combination only has to be conducted once because, as indicated by our study results, the profiles can be transferred to the same type of tags — even automatically using a robotic arm as we demonstrated in our exploratory studies. Hence, the companies who made prototyping kits based on NFCSense can acquire these parameters for their users, so the end-users do not have to do so.
For more experienced and sophisticated users who aim at further hardware customization, such parameter acquisition can be conducted using an electromagnetic-magnetic simulator when the user knows 1) the antenna structures of both the reader and tag and 2) the activation voltage of the NFC chips. Nonetheless, because the coils are not always visible in commodity tags and readers, users should check the specifications beforehand. After the simulation, an experimental apparatus similar to ours can be used to validate the devices’ real-world performance to determine the tolerance of the simulation. Finally, for novice technical users, the learning curve of performing a valid software simulation is steeper than determining the optimal parameters through a hands-on approach.
Reliability. NFC is a reliable technology that has been developed for decades. Thus, the main reliability problems in our context pertain to the physical designs that determine the tag's motion, distances between the tags, and signal quality. With regard to the tags’ motion, the form factor of tags and the constraints should ensure a proper reading distance and a proper angle between the tag and the reader antenna. Because the reader may also lose the tracking of a tag when it is moving too fast for the sampling rate of an NFC reader, the physical design should limit the maximum speed of a moving tagged object when it is passing through the reader, such as adding turns when the object is sliding with wheels or applying rough surfaces in the middle of the path to slow the object down. The tagged object may, however, become stuck if the constraint is improperly designed. As for the distances between the tags, using tags that are too thin (e.g., an NFC sticker) or too flexible may cause unwanted overlaps between the stickers; these induce more collisions that make the tracking less reliable. In vertical constraint design, as realized in the multiplexer example, increasing the thickness of the tag can stabilize the rolling movement. With regard to signal quality, the tags and constraints should be made of nonconductive materials to ensure that they do not block the RF signals at an operation frequency of 13.56 MHz. Conductive materials may shield the electromagnetic fields emitted from the reader, thus disabling the power transfer or interfering with the RF signal transmission. In general, further adaptation and optimization should incorporate these aspects in their physical design. Finally, the validity of our current implementation with respect to speed sensing was based on the lookup table built from the activation map collection from our study, the 3D printed objects and conventional building blocks, and the Arduino-controlled uniform sampling at a submillisecond accuracy, validating the results of the direct speed measurement can be considered for future work.
RF Antenna Optimization. Because the reader antenna coil in the NFC system is mainly used for power transfer, the antenna design only needs to induce a sufficient voltage when based on Equations 1 and 2. Moreover, the reader antenna's resonant frequency should match the one on the tags through an inductive–capacitive coupling to maximize power transfer efficiency. In this work, we simply customized the antenna by using the on-board capacitors and crafted the antenna by using a conductive wire (∼ 5Ω/m) of a similar trace length as that of the on-board antenna to achieve a similar coupling; in doing so, we could focus on the antenna form-giving. For wearable applications, the range and distance of tag reading can be further extended through a lab approach using tools such as a vector network analyzer (VNA) and computational antenna simulators; such extensions allow users to wear the antenna at a more comfortable position, such as the back of the finger or hand.
FaBrickation, Retrofitting, and Beyond. This research was conducted during the Covid-19 pandemic when the authors had limited access to rapid prototyping equipment such as a laser cutter. To make the experiment results replicable while still rapidly prototyping physical objects, we utilized 3D-printed Lego bricks and Duplo blocks, as demonstrated in FaBrickation [36]; this meant that we only needed to print minimal parts and make adaptors with a Duplo brick library 6. The adaptation of these conventional building blocks not only accelerates the manufacturing pipeline without sacrificing precision but also enable hot-swappable interactivity. Retrofitting NFC technology to already-existing objects with rich semantics, such as figurines, and to kinetic mechanisms of specialized blocks and toys allow for effective demonstrations. Nonetheless, precise handcrafting was needed for the more complex mechanisms, such as the modulator, that could not be made by the building blocks or were not readily available in everyday objects. Future work can provide extra support with regard to integration in personal fabrication for users to adapt complex anti-collision mechanisms with NFC systems and FaBrickations.
Applicability. In this paper, we intentionally described simple use cases to focus on the NFC system and its capabilities. We demonstrated the usefulness of our approach primarily through applications featuring rapid prototyping TUI with existing objects. The use cases of input primitives presented in this paper are also highly generalizable because all of them can be used as props in various educational (e.g., storytelling), entertainment (e.g., gaming), and health (e.g., rehabilitation) applications.
User Experiences. This research does not have a user study because we intended to adopt a technical HCI research approach to validate the conceptually novel interaction systems through proof-of-concept implementation [21]. Furthermore, our contribution mainly pertains to engineering and design. Nonetheless, knowing how designers and practitioners use the techniques to explore further designs is the next logical step in our area of interest. Future work can consider running design workshops to collect more artifacts and design processes to determine emerging design patterns that can enrich the proposed design space.
9 CONCLUSION
We present NFCSense, a data-defined rich-ID motion sensing approach for fluent tangible interaction using a commodity NFC reader. We investigated the feasibility of repurposing an NFC reader to measure motion speed through a series of data-driven analyses, in which we determined practical design parameters from the results of technical measurements. Based on the results, we demonstrate an interaction design space through proof-of-concept implementations, which demonstrate the embodiment of fluent, hands-off, and multi-tag tangible interactions through effortless input. We also show how hot-swappable interactivity supports fluent use throughout an extended period. We also detail some design implications for future researchers to surmount current limitations and leverage NFC technology, which is in prevalent use.
ACKNOWLEDGMENTS
We sincerely acknowledge Associate Chairs and the anonymous reviewers for the helpful comments. The first author would like to thank his wife and kids for supporting most of the fabrication and experiments to be completed at home.
REFERENCES
- Daniel Avrahami and Scott E. Hudson. 2002. Forming Interactivity: A Tool for Rapid Prototyping of Physical Interactive Products. In Proc. ACM DIS ’02. 141–146.
- Daniel Avrahami, Jacob O. Wobbrock, and Shahram Izadi. 2011. Portico: Tangible Interaction on and around a Tablet. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 347–356. https://doi.org/10.1145/2047196.2047241
- Maribeth Back, Jonathan Cohen, Rich Gold, Steve Harrison, and Scott Minneman. 2001. Listen Reader: An Electronically Augmented Paper-based Book. In Proc. ACM CHI ’01. 23–29.
- Victoria Bellotti, Maribeth Back, W. Keith Edwards, Rebecca E. Grinter, Austin Henderson, and Cristina Lopes. 2002. Making Sense of Sensing Systems: Five Questions for Designers and Researchers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis, Minnesota, USA) (CHI ’02). Association for Computing Machinery, New York, NY, USA, 415–422. https://doi.org/10.1145/503376.503450
- Eugen Berlin, Jun Liu, Kristof van Laerhoven, and Bernt Schiele. 2010. Coming to Grips with the Objects We Grasp: Detecting Interactions with Efficient Wrist-worn Sensors. In Proc. TEI ’10. 57–64.
- Andrea Bianchi and Ian Oakley. 2013. Designing Tangible Magnetic Appcessories. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (Barcelona, Spain) (TEI ’13). Association for Computing Machinery, New York, NY, USA, 255–258. https://doi.org/10.1145/2460625.2460667
- Keywon Chung, Michael Shilman, Chris Merrill, and Hiroshi Ishii. 2010. OnObject: Gestural Play with Tagged Everyday Objects. In Adjunct Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 379–380. https://doi.org/10.1145/1866218.1866229
- Crampton Smith, G.1995. The hand that rocks the cradle.ID magazine, 60–65..
- Assaf Feldman, Emmanuel Munguia Tapia, Sajid Sadi, Pattie Maes, and Chris Schmandt. 2005. ReachMedia: On-the-move Interaction with Everyday Objects. In Proc. IEEE ISWC ’05. 52–59.
- Klaus Finkenzeller. 2003. RFID Handbook: Fundamentals and Applications in Contactless Smart Cards and Identification(2nd ed.). Wiley Publishing.
- KennethP. Fishkin, Bing Jiang, Matthai Philipose, and Sumit Roy. 2004. I Sense a Disturbance in the Force: Unobtrusive Detection of Interactions with RFID-tagged Objects. In UbiComp 2004: Ubiquitous Computing, Nigel Davies, ElizabethD. Mynatt, and Itiro Siio(Eds.). Lecture Notes in Computer Science, Vol. 3205. Springer Berlin Heidelberg, 268–282. http://dx.doi.org/10.1007/978-3-540-30119-6_16
- K. P. Fishkin, M. Philipose, and A. Rea. 2005. Hands-on RFID: wireless wearables for detecting use of objects. In Proc. IEEE ISWC ’05. 38–41.
- Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. InFORM: Dynamic Physical Affordances and Constraints through Shape and Object Actuation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 417–426. https://doi.org/10.1145/2501988.2502032
- James J Gibson. 1977. The theory of affordances. (1977).
- Jason T. Griffin, Steven Henry Fyke, Christopher Lyle Bender, Santiago Carbonell Duque, David Ryan Walker, and Jerome Pasquero. 2014. Near-field communication (NFC) system providing mobile wireless communications device operations based upon timing and sequence of NFC sensor communication and related methods. U.S. Patent 8,670,709.
- Tobias Grosse-Puppendahl, Sebastian Herber, Raphael Wimmer, Frank Englert, Sebastian Beck, Julian von Wilmsdorff, Reiner Wichert, and Arjan Kuijper. 2014. Capacitive Near-Field Communication for Ubiquitous Interaction and Perception. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Seattle, Washington) (UbiComp ’14). Association for Computing Machinery, New York, NY, USA, 231–242. https://doi.org/10.1145/2632048.2632053
- Changyo Han, Katsufumi Matsui, and Takeshi Naemura. 2020. ForceStamps: Fiducial Markers for Pressure-Sensitive Touch Surfaces to Support Rapid Prototyping of Physical Control Interfaces. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney NSW, Australia) (TEI ’20). Association for Computing Machinery, New York, NY, USA, 273–285. https://doi.org/10.1145/3374920.3374924
- Chris Harrison, Robert Xiao, and Scott Hudson. 2012. Acoustic Barcodes: Passive, Durable and Inexpensive Notched Identification Tags. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology(Cambridge, Massachusetts, USA) (UIST ’12). Association for Computing Machinery, New York, NY, USA, 563–568. https://doi.org/10.1145/2380116.2380187
- Meng-Ju Hsieh, Jr-Ling Guo, Chin-Yuan Lu, Han-Wei Hsieh, Rong-Hao Liang, and Bing-Yu Chen. 2019. RFTouchPads: Batteryless and Wireless Modular Touch Sensor Pads Based on RFID. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 999–1011. https://doi.org/10.1145/3332165.3347910
- Meng-Ju Hsieh, Rong-Hao Liang, Da-Yuan Huang, Jheng-You Ke, and Bing-Yu Chen. 2018. RFIBricks: Interactive Building Blocks Based on RFID. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 189, 10 pages. https://doi.org/10.1145/3173574.3173763
- Scott E Hudson and Jennifer Mankoff. 2014. Concepts, values, and methods for technical human–computer interaction research. In Ways of Knowing in HCI. Springer, 69–93.
- Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’97). ACM, New York, NY, USA, 234–241. https://doi.org/10.1145/258549.258715
- Haojian Jin, Jingxian Wang, Zhijian Yang, Swarun Kumar, and Jason Hong. 2018. WiSh: Towards a Wireless Shape-aware World Using Passive RFIDs. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services(Munich, Germany) (MobiSys ’18). ACM, New York, NY, USA, 428–441. https://doi.org/10.1145/3210240.3210328
- Keiko Katsuragawa, Ju Wang, Ziyang Shan, Ningshan Ouyang, Omid Abari, and Daniel Vogel. 2019. Tip-Tap: Battery-Free Discrete 2D Fingertip Input. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 1045–1057. https://doi.org/10.1145/3332165.3347907
- Han-Chih Kuo, Rong-Hao Liang, Long-Fei Lin, and Bing-Yu Chen. 2016. GaussMarbles: Spherical Magnetic Tangibles for Interacting with Portable Physical Constraints. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4228–4232. https://doi.org/10.1145/2858036.2858559
- Gierad Laput, Eric Brockmeyer, Scott E. Hudson, and Chris Harrison. 2015. Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2161–2170. https://doi.org/10.1145/2702123.2702414
- Hanchuan Li, Eric Brockmeyer, Elizabeth J. Carter, Josh Fromm, Scott E. Hudson, Shwetak N. Patel, and Alanson Sample. 2016. PaperID: A Technique for Drawing Functional Battery-Free Wireless Interfaces on Paper. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). ACM, New York, NY, USA, 5885–5896. https://doi.org/10.1145/2858036.2858249
- Hanchuan Li, Can Ye, and Alanson P. Sample. 2015. IDSense: A Human Object Interaction Detection System Based on Passive UHF RFID. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(Seoul, Republic of Korea) (CHI ’15). ACM, New York, NY, USA, 2555–2564. https://doi.org/10.1145/2702123.2702178
- Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng, Mike Y. Chen, Rung-Huei Liang, De-Nian Yang, and Bing-Yu Chen. 2013. GaussBits: Magnetic Tangible Bits for Portable and Occlusion-Free near-Surface Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 1391–1400. https://doi.org/10.1145/2470654.2466185
- Rong-Hao Liang, Meng-Ju Hsieh, Jheng-You Ke, Jr-Ling Guo, and Bing-Yu Chen. 2018. RFIMatch: Distributed Batteryless Near-Field Identification Using RFID-Tagged Magnet-Biased Reed Switches. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). ACM, New York, NY, USA, 473–483. https://doi.org/10.1145/3242587.3242620
- Rong-Hao Liang, Han-Chih Kuo, and Bing-Yu Chen. 2016. GaussRFID: Reinventing Physical Toys Using Magnetic RFID Development Kits. In Proc. ACM CHI ’16(Santa Clara, California, USA) (CHI ’16). 4233–4237.
- Rong-Hao Liang, Shun-Yao Yang, and Bing-Yu Chen. 2019. InDexMo: Exploring Finger-Worn RFID Motion Tracking for Activity Recognition on Tagged Objects. In Proceedings of the 23rd International Symposium on Wearable Computers (London, United Kingdom) (ISWC ’19). Association for Computing Machinery, New York, NY, USA, 129–134. https://doi.org/10.1145/3341163.3347724
- Rong-Hao Liang, Bin Yu, Mengru Xue, Jun Hu, and Loe M. G. Feijs. 2018. BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174187
- Nicolai Marquardt, Alex S. Taylor, Nicolas Villar, and Saul Greenberg. 2010. Rethinking RFID: Awareness and Control for Interaction with RFID Systems. In Proc. ACM CHI ’10. 2307–2316.
- Einar Sneve Martinussen and Timo Arnall. 2009. Designing with RFID. In Proc. ACM TEI ’09. 343–350.
- Stefanie Mueller, Tobias Mohr, Kerstin Guenther, Johannes Frohnhofen, and Patrick Baudisch. 2014. FaBrickation: Fast 3D Printing of Functional Objects by Integrating Construction Kit Building Blocks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 3827–3834. https://doi.org/10.1145/2556288.2557005
- Jun Rekimoto, Brygg Ullmer, and Haruo Oba. 2001. DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’01). ACM, New York, NY, USA, 269–276. https://doi.org/10.1145/365024.365115
- A.P. Sample, D.J. Yeager, and J.R. Smith. 2009. A capacitive touch interface for passive RFID tags. In 2009 IEEE International Conference on RFID. 103–109.
- Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: Embedded Single-Camera Sensing of Printed Physical User Interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 447–456. https://doi.org/10.1145/2501988.2501992
- Valkyrie Savage, Andrew Head, Björn Hartmann, Dan B. Goldman, Gautham Mysore, and Wilmot Li. 2015. Lamello: Passive Acoustic Sensing for Tangible Input Components. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1277–1280. https://doi.org/10.1145/2702123.2702207
- Claude Elwood Shannon. 1949. Communication in the presence of noise. Proceedings of the IRE 37, 1 (1949), 10–21.
- Timothy M. Simon, Bruce H. Thomas, Ross T. Smith, and Mark Smith. 2014. Adding Input Controls and Sensors to RFID Tags to Support Dynamic Tangible User Interfaces. In Proc. ACM TEI ’14. 165–172.
- Andrew Spielberg, Alanson Sample, Scott E. Hudson, Jennifer Mankoff, and James McCann. 2016. RapID: A Framework for Fabricating Low-Latency Interactive Objects with RFID Tags. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5897–5908. https://doi.org/10.1145/2858036.2858243
- Brygg Ullmer, Hiroshi Ishii, and Robert JK Jacob. 2005. Token+ constraint systems for tangible interaction with digital information. ACM Transactions on Computer-Human Interaction (TOCHI) 12, 1(2005), 81–118.
- Nicolas Villar, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, William Field, and Haiyan Zhang. 2018. Project Zanzibar: A Portable and Flexible Tangible Interaction Platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174089
- Neng-Hao Yu, Li-Wei Chan, Seng Yong Lau, Sung-Sheng Tsai, I-Chun Hsiao, Dian-Je Tsai, Fang-I Hsiao, Lung-Pan Cheng, Mike Chen, Polly Huang, and Yi-Ping Hung. 2011. TUIC: Enabling Tangible Interaction on Capacitive Multi-Touch Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 2995–3004. https://doi.org/10.1145/1978942.1979386
- T. Zhang, N. Becker, Y. Wang, Y. Zhou, and Y. Shi. 2017. BitID: Easily Add Battery-Free Wireless Sensors to Everyday Objects. In 2017 IEEE International Conference on Smart Computing (SMARTCOMP). 1–8. https://doi.org/10.1109/SMARTCOMP.2017.7946990
- Shilin Zhu and Yilong Li. 2018. 2DR: Towards Fine-Grained 2-D RFID Touch Sensing. arXiv preprint arXiv:1808.08808(2018).
- Oren Zuckerman, Saeed Arida, and Mitchel Resnick. 2005. Extending Tangible Interfaces for Education: Digital Montessori-Inspired Manipulatives. CHI 2005: Technology, Safety, Community: Conference Proceedings - Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1054972.1055093
FOOTNOTE
1 https://www.nintendo.com/amiibo
2 https://www.lego.com/en-us/dimensions/products
3Because of the rectangular reader antenna, Lon depends on the track's angle. Therefore, the speed measured in this application requires further calibration.
4 https://www.youtube.com/watch?v=SbWc3bSdDlw&t=301s
5 https://wiki.seeedstudio.com/Grove_NFC/
6 https://www.thingiverse.com/thing:159219
This work is licensed under a Creative Commons Attribution International 4.0 License.
CHI '21, May 08–13, 2021, Yokohama, Japan
© 2021 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8096-6/21/05.
DOI: https://doi.org/10.1145/3411764.3445214