research-article

Public Access

Pervasive Pose Estimation for Fall Detection

Authors:

Jiaqing Luo,

Ruiyu Bai,

Suining He,

Kang G. ShinAuthors Info & Claims

ACM Transactions on Computing for Healthcare (HEALTH), Volume 3, Issue 3

Article No.: 27, Pages 1 - 23

https://doi.org/10.1145/3478027

Published: 07 April 2022 Publication History

All formats PDF

Abstract

Falls are the second leading cause of accidental or unintentional injuries/deaths worldwide. Accurate pose estimation using commodity mobile devices will help early detection and injury assessment of falls, which are essential for the first aid of elderly falls. By following the definition of fall, we propose a Pervasive Pose Estimation scheme for fall detection (P\(^2\)Est), which measures changes in tilt angle and height of the human body. For the tilt measurement, P\(^2\)Est leverages the pointing of the mobile device, e.g., the smartphone, when unlocking to associate the Device coordinate system with the World coordinate system. For the height measurement, P\(^2\)Est exploits the fact that the person’s height remains unchanged while walking to calibrate the pressure difference between the device and the floor. We have prototyped and tested P\(^2\)Est in various situations and environments. Our extensive experimental results have demonstrated that P\(^2\)Est can track the body orientation irrespective of which pocket the phone is placed in. More importantly, it enables the phone’s barometer to detect falls in various environments with decimeter-level accuracy.

1 Introduction

Human falls are a major public health risk. About 646,000 fatal falls are known to occur each year, making it the second leading cause of accidental injury deaths, after road traffic injuries. Over 80% of fall-related fatalities occur in low- and middle-income countries, with regions of the Western Pacific and South East Asia accounting for 60% of these deaths.¹ Falls must be detected in a timely, accurate, reliable, and scalable manner in order to reduce the risk of serious injuries or deaths.

Prior work on fall detection—including monitoring body poses or ambient fluctuations—is either near-body or long-distance. The near-body detection requires the deployment of sensors on the human body. The sensors used for fall detection could be from special or general devices. Special or dedicated (e.g., solely for medical alerts) devices are usually specially packaged and worn at specific positions of the body (e.g., belt, necklace, and shoes) for real-time or/and accurate pose estimation, but may incur extra cost or inconvenience. Most people have a fluke mentality and are reluctant to pay extra for emergencies (falls). Due to the lack of users, existing machine learning methods usually use fabricated data (e.g., [7]) instead of real data for training, making it difficult to guarantee their effectiveness in actual use. On the other hand, general devices like smartphones and watches have high acceptance and incur no additional cost/inconvenience, but their uncontrolled wearing positions (e.g., chest, upper arm, or waist) and angles [23] (caused by different body shapes or clothing styles) may lead to high-level uncertainties in estimating the body pose. The long-distance detection, on the other hand, usually requires the deployment of additional infrastructure (e.g., cameras [36], RF sensors [31, 32], and microphones [18]) in their operational environment. Although infrastructure can reduce users’ costs and restrictions, its deployment and maintenance costs limit the deployment and popularity of applications built thereon, and the use of cameras may also raise privacy concerns. More importantly, unpredictable interference/noise may greatly affect the accuracy of pose estimation.

A fall is said to occur when the line of gravity (LoG) falls outside of the base of support (BoS) (tilt angle) that results in the person coming to rest on the floor or another lower level (thus at a lower height). Although this definition is concise, the process of a fall is uncontrollable and is therefore full of unknowns and uncertainties. Accurate pose estimation is essential for the early detection of falls and assessment of their resultant injuries, as well as refined management of fall risks especially for elderly people. In addition, the use of (generic) commodity mobile devices (i.e., smartphones and watches) for accurate pose estimation can help the doctors assess and improve the care of patients with movement disorders, such as those who are stricken with Parkinson’s disease or have undergone intracranial surgery (e.g., cerebral infarction, cerebral hemorrhage, and vestibular schwannoma), cochlear implant surgery, joint replacement, and so on.

It remains a challenge to make commodity mobile devices to achieve the same accuracy as special devices in detecting falls without degrading the user experience, which is conducive to the popularization of applications and the collection of real data. We propose a Pervasive Pose Estimation scheme for fall detection (P\(^2\)Est), which relies on rotation vector sensor and barometer already built into commodity mobile devices (e.g., smartphones/watches), to achieve high accuracy when tilt angle and height changes are measured/sensed. In particular, P\(^2\)Est takes full advantage of the constraints in some states, such as Walking, Stationary, and Unlocking, to update and calibrate the tilt-angle and height measurement data. Each tilt-angle measurement is a sequence of coordinate transformations. P\(^2\)Est captures the bearing of the body when the user unlocks the phone, and then establishes the relationships among Body, Device, and World coordinate systems. To the best of our knowledge, this is the first attempt to relax position restrictions and improve measurement accuracy without training. The height measurement is a continuous calibration of atmospheric pressure differences. Instead of smoothing the actual measured pressure, P\(^2\)Est smoothes the virtual floor pressure to keep the pressure difference between phone and floor stable or unchanged when the subject is walking or the phone is stationary. This is also, to the best of our knowledge, the first time that a single barometer is shown to be able to measure height in a complex environment with decimeter(dm)-level accuracy. This technique will likely have a wide range of applications other than fall detection, such as indoor localization and navigation.

P\(^2\)Est has four salient features: (i) timeliness — it estimates body pose in real time; (ii) accuracy — it measures body tilt including up-down, left-right, and front-back. (iii) reliability — it can distinguish between different types of falls (e.g., trip, slip, and segmented fall), and achieve dm-level accuracy in complex situations (e.g., spiral stairs, indoor/outdoor); (iv) scalability — it uses popular mobile devices like smartphones, and can adapt to different user habits.

In summary, this article makes the following main contributions:

–

Construction of the body coordinate system with unknown device positions;

–

Improvement of the accuracy of the barometer-based height measurement up to the dm-level;

–

Design of tolerance to the step detector’s delays and false positives; and

–

Extensive evaluation of P\(^2\)Est in complex situations and environments.

The rest of this article is organized as follows. After discussing the related work in Section 2, we present the primers used by, and challenges of, P\(^2\)Est in Section 3. Then, we overview the system design of P\(^2\)Est in Section 4 and present the coordinate transformation for tilt angle calculation and reference update in Section 5, followed by barometer pressure calibration in Section 6. We experimentally evaluate P\(^2\)Est in Section 7 and discuss some of the remaining issues in Section 8. The article finally concludes in Section 10.

2 Related Work

A considerable amount of research and development has been done on fall detection, which can be roughly classified as two categories according to the technology used and the monitored object: near-body sensing and long-distance sensing. Since P\(^2\)Est belongs to the former, we will focus on its related work, which can be divided further into two categories according to the sensors used: special and general devices.

These special devices focus on detection accuracy because their estimation uncertainty is low. The sensors can be attached to various parts of the human body [10], including chest [19], wrist [12], waist [7, 28], thigh [23], ankle [35], and foot [14]. The measurements can be made via accelerometer, gyroscope [7], and many other motion sensors, as well as wireless sensor networks [17]. Physical interpretation of the human body, including pose angle [16], angular velocity [13], pulse shape [12], foot pressure [14] and contextual information [7], and their correlations with fall patterns [7] and fall risks [33], have been statistically and quantitatively investigated further. Machine learning algorithms [8, 10] like support vector machine (SVM), regression tree, random forest (RF), and K-nearest neighbors (K-NN) can be applied to classify the subtle behaviors and detect fall events.

On the other hand, the general devices prioritize the effectiveness of detection owing to their high estimation uncertainty. Unlike special devices, general devices experience less restrictions regarding placement. The body part where the device is to be attached/placed [11, 27] has been identified and the effect of sensor placement on detection [1, 2] has been investigated further. Under high uncertainty, signal vector magnitude [5, 15, 24, 29], acceleration high-frequency feature [9], and impact force [22] are used to detect whether a fall has occurred, vertical velocity [24], heart rate [29], and immobility time [15, 29] are measured to reduce the false-positive rate.

3 Preliminaries

We first introduce the key definition of fall and then describe the sensors used for pose estimation. Finally, we present a series of experiments to explore/identify challenges in analyzing body motions using mobile device sensors.

3.1 Definition of Fall

Before defining a human fall, we need to introduce two key concepts: LoG and BoS. LoG is the gravitational force line which is a vertical line projecting downwards from the center of gravity. BoS is the supporting area beneath the human body includes the points of contact with the supporting surface and the area between them. These points may be body parts (such as feet or hands), or extensions of the body parts (such as crutches or a chair the subject is sitting on).

A fall is a bio-mechanical event in that an external force, gravity, destabilizes the body’s alignment of the torso over the legs. When LoG is within BoS, the person is stable. When LoG falls outside BoS, the person is unstable and may fall. According to this definition, the detection of a fall requires a moving coordinate system associated with the human body to represent and characterize changes in the body’s tilt angle and height over time.

As illustrated in Figure 1, the Body coordinate system of a person, \(X_{[b]i}Y_{[b]i}Z_{[b]i}\), is defined relative to the body’s bearing in the \(i\)th sampling period. The axes are not swapped when the body’s bearing changes. When the person stays upright (e.g., standing and walking), the \(X_{[b]i}\) axis points to the left, the \(Y_{[b]i}\) axis points to the head, and the \(Z_{[b]i}\) axis points toward the outside of the body’s front. The three coordinate axes, taken two at a time, form three anatomical planes, namely sagittal (left and right), transverse (head and tail), and coronal planes (back and front), respectively.

Fig. 1.

When a person stands and falls in sampling periods \(i\) and \(j\), respectively (see Figure 1), his body tilts and transforms to a new coordinate system, i.e., \(X_{[b]j}Y_{[b]j}Z_{[b]j}\), which is obtained from \(X_{[b]i}Y_{[b]i}Z_{[b]i}\) by rotating the coordinate axes. The tilt angles between the \(Y_{[b]j}\) axis and those of \(X_{[b]i}\), \(Y_{[b]i}\), and \(Z_{[b]i}\) indicate the relative motion of the body for left/right, up/down, and front/back tilt. Another important observation is that the height relative to the floor changes from \(h_{i}\) to \(h_{j}\). For early detection of falls, we need to estimate the body pose including tilt and height. The tilt measurement helps us distinguish between different types of fall, such as slip and trip. The height measurement allows us to identify different types of states, such as lying in the bed or on the floor.

3.2 Pose-estimation Sensors

We group pose-estimation sensors into three categories according to their respective purposes: orientation determination, height measurement, and state detection.

3.2.1 Orientation Determination.

The rotation vector sensor and the gravity sensor are the most frequently used sensors for motion detection and monitoring.

A rotation vector sensor reports the device’s orientation relative to the World coordinate system. The World coordinate system, \(X_{[w]}Y_{[w]}Z_{[w]}\), is a fixed coordinate system at which the device is placed. Specifically, the \(X_{[w]}\) axis is tangential to the ground at the device’s current location and roughly points East, and the \(Y_{[w]}\) axis is tangential to the ground at the device’s current location and points towards the magnetic North pole, the \(Z_{[w]}\) axis points towards the sky and is perpendicular to the ground.

The device’s orientation in the \(i\)th sampling period is represented by the rotation necessary to align the World coordinate system with the Device coordinate system, \(X_{[d]i}Y_{[d]i}Z_{[d]i}\), defined relative to the device’s screen. Similarly to \(X_{[b]i}Y_{[b]i}Z_{[b]i}\), \(X_{[d]i}Y_{[d]i}Z_{[d]i}\) is a moving coordinate system embedded in the device. When the device is held in its default orientation (portrait orientation for a phone), the \(X_{[d]i}\) axis is horizontal and points to the right, the \(Y_{[d]i}\) axis is vertical and points up, and the \(Z_{[d]i}\) axis points toward the outside of the screen face.

Given two coordinate systems \(\mathbf {X}\) and \(\mathbf {Y}\), we define the symbol of \(\mathbf {X} / \mathbf {Y}\) to represent the transformation from \(\mathbf {X}\) to \(\mathbf {Y}\). Starting from \(X_{[d]i}Y_{[d]i}Z_{[d]i}\) aligned with \(X_{[w]}Y_{[w]}Z_{[w]}\), three elemental rotations at the \(i\)th period, \(\mathbf {R}_{Z}(\theta _{Z[d/w]i})\), \(\mathbf {R}_{X}(\theta _{X[d/w]i}),\) and \(\mathbf {R}_{Y}(\theta _{Y[d/w]i})\), are defined to reach any target orientation for \(X_{[d]i}Y_{[d]i}Z_{[d]i}\). The Tait–Bryan angles (\(\theta _{Z[d/w]i}\), \(\theta _{X[d/w]i}\), \(\theta _{Y[d/w]i}\)) are the amplitudes of these elemental rotations [4].

For instance, the target orientation can be obtained with the following inputs:

–

\(X_{[d]i}Y_{[d]i}Z_{[d]i}\) rotates about the \(Z_{[w]}\) axis by \(\theta _{Z[d/w]i}\);

–

\(X_{[d]i}Y_{[d]i}Z_{[d]i}\) rotates again about the new \(X_{[w]}\) axis by \(\theta _{X[d/w]i}\);

–

\(X_{[d]i}Y_{[d]i}Z_{[d]i}\) rotates a third time about the new \(Y_{[w]}\) axis by \(\theta _{Y[d/w]i}\).

Unless specified otherwise, P\(^2\)Est selects this default sequence. The rotation matrix in the \(i\)th period, \(\mathbf {R}_{[d/w]i}\), describes \(X_{[d]i}Y_{[d]i}Z_{[d]i}\) relative to \(X_{[w]}Y_{[w]}Z_{[w]}\) and is formally given by

\[\begin{eqnarray} \mathbf {R}_{[d/w]i}&=&\mathbf {R}_{Z}(\theta _{Z[d/w]i})\mathbf {R}_{X}(\theta _{X[d/w]i}) \mathbf {R}_{Y}(\theta _{Y[d/w]i}) \nonumber \nonumber\\ &=&\begin{bmatrix}\cos \theta _{Z[d/w]i} & -\sin \theta _{Z[d/w]i} & 0\\ \sin \theta _{Z[d/w]i} & \cos \theta _{Z[d/w]i} & 0\\ 0 & 0 & 1\end{bmatrix} \nonumber \nonumber\\ &\times &\begin{bmatrix}1 & 0 & 0\\ 0 & \cos \theta _{X[d/w]i} & -\sin \theta _{X[d/w]i}\\ 0 & \sin \theta _{X[d/w]i} & \cos \theta _{X[d/w]i}\end{bmatrix} \nonumber \nonumber\\ &\times &\begin{bmatrix}\cos \theta _{Y[d/w]i} & 0 & \sin \theta _{Y[d/w]i}\\ 0 & 1 & 0\\ -\sin \theta _{Y[d/w]i} & 0 & \cos \theta _{Y[d/w]i}\end{bmatrix}. \end{eqnarray}\]

(1)

The Android sensor framework provides a method to compute the device’s orientation using \(\mathbf {R}_{[d/w]i}\) and returns

–

Azimuth, \(\theta _{A}=\arctan 2(\frac{\mathbf {R}_{[d/w]i}(1,2)}{\mathbf {R}_{[d/w]i}(2,2)})\), is the angle of rotation about the \(-\)\(Z_{[w]}\) axis, and represents the angle between the device’s current compass direction and magnetic North;

–

Pitch, \(\theta _{P}=\arcsin (-\mathbf {R}_{[d/w]i}(3,2))\), is the angle of rotation about the \(X_{[w]}\) axis, and represents the angle between a plane parallel to the device’s screen and a plane parallel to the ground;

–

Roll, \(\theta _{R}=\arctan 2(\frac{-\mathbf {R}_{[d/w]i}(3,1)}{\mathbf {R}_{[d/w]i}(3,3)})\), is the angle of rotation about the \(Y_{[w]}\) axis, and represents the angle between a plane perpendicular to the device’s screen and a plane perpendicular to the ground.

3.2.2 Height Measurement.

Linear acceleration sensor and barometer can be used to measure height variation and altitude, respectively. The former provides a three-dimensional vector representing acceleration along each device axis, except for gravity. Given the vector returned in the \(i\)th sampling period, denoted as the device linear acceleration with respect to \(Z_{[w]}\) axis, \(a_{Z[d/w]i}\), can be written as

\[\begin{eqnarray} \left[\mathbf {R}_{[d/w]i}(3,1),\mathbf {R}_{[d/w]i}(3,2),\mathbf {R}_{[d/w]i}(3,3)\right] \begin{bmatrix}a_{X[d]i}\\ a_{Y[d]i}\\ a_{Z[d]i}\end{bmatrix}. \end{eqnarray}\]

(2)

Taking Figure 1 as an example, the person under watch starts to fall from a standstill state. Assuming the length of each sampling period is \(T\), the number of samples is \(k\), and the initial velocity and acceleration are both 0, the cumulative height changes by the \(k\)th sampling period, \(h_{k/0}\), can be calculated as

\[\begin{eqnarray} h_{k/0}=\sum _{i=1}^{k}\bigg (\sum _{i=1}^{k}a_{Z[d/w]i-1}T+\frac{1}{2}a_{Z[d/w]i}T^{2}\bigg). \end{eqnarray}\]

(3)

The barometer measures the atmospheric pressure in millimeters of mercury (mmHg). Given the measured pressure \(p\) and the pressure at the sea level \(p_{0}\), the altitude \(\hbar\) can be obtained by

\[\begin{eqnarray} \hbar =44300*\left(1-\frac{p}{p_{0}}\right)^{\frac{1}{5.256}}. \end{eqnarray}\]

(4)

3.2.3 State Detection.

Step detector and accelerometer are employed to detect the two most common states, walking and stationary. The step detector counts each step the target person takes by detecting the motion of his hands or hips. The latency is expected to be below 2 seconds.² The latest Apple Watch suggests determination of whether or not the user is stationary, by using the signal vector magnitude [29]. If its magnitude is below a pre-specified threshold for a certain period of time, the user is deemed stationary. If the user is stationary after a fall, this may indicate that the user is likely dazed or injured.

3.3 Challenges in Pose Estimation

We have conducted a series of experiments with Android phones. These experiments were designed to identify and understand the difficulties in measuring the human body’s tilt angle and height.

3.3.1 Tilt-angle Measurement.

Note that the tilt angle is defined only with respect to \(X_{[b]i}\), \(Y_{[b]i}\) and \(Z_{[b]i}\). Although the phone in the user’s pocket is stationary relative to his/her body, the orientation and movement of the phone might not coincide with that of the body, because the phone can be put in different pockets with different orientations.

We have conducted an experiment to test the consistency of the phone’s orientation in different pockets. The sampling rate of the rotation vector sensor is set to 50 Hz. As shown in Figure 1, two test phones, Huawei Nexus 6P, and Google Pixel XL were placed in the chest and waist pockets, respectively. The tester walked down the spiral staircase from the 4th floor to the 3rd floor of our Department building and then bent approximately \(\frac{\pi }{2}\) forward and left, respectively. Figure 2 shows that the orientations of the two phones were not the same even though they were carried by the same person.

Fig. 2.

One can see from Figure 2 that \(\theta _{A}\) turned close to \(2\pi\) when walking down the spiral stairs. While \(\theta _{P}\) can help determine whether the body is bent or not, it is actually difficult to determine the direction of the body’s bending \(\theta _{R}\) mainly because the rotation matrix from \(X_{[b]i}Y_{[b]i}Z_{[b]i}\) to \(X_{[w]}\)\(Y_{[w]}\)\(Z_{[w]}\), i.e., \(\mathbf {R}_{[b/w]i}\), is unknown.

When the testing person is standing or walking, \(\mathbf {R}_{[b/w]i}\) can be initialized by aligning \(Y_{[d]i}\) with \(Z_{[b]i}\). However, this initialization process requires the user’s explicit assistance. If she/he changes a coat or moves the phone to another pocket, \(\mathbf {R}_{[b/w]i}\) needs to be re-initialized, and frequent reminders of re-initialization might annoy the user. A natural question is then: “can \(\mathbf {R}_{[b/w]i}\) be automatically initialized without the user’s awareness?” A rough idea is to determine \(Z_{[b]i}\) by using the walking direction. Although there have been studies [21] to detect the user’s walking direction, there still exist some basic restrictions that have not been considered. For a given linear acceleration \(\mathbf {a}_{[d]i}\), the horizontal acceleration in \(X_{[w]}\)\(Y_{[w]}\)\(Z_{[w]}\), \(\mathbf {a}_{XY[d/w]i}\), can be given by

\[\begin{eqnarray} \begin{bmatrix}\mathbf {R}_{[d/w]i}(1,1) & \mathbf {R}_{[d/w]i}(1,2) & \mathbf {R}_{[d/w]i}(1,3)\\ \mathbf {R}_{[d/w]i}(2,1) & \mathbf {R}_{[d/w]i}(2,2) & \mathbf {R}_{[d/w]i}(2,3)\end{bmatrix} \begin{bmatrix}a_{X[d]i}\\ a_{Y[d]i}\\ a_{Z[d]i}\end{bmatrix}. \end{eqnarray}\]

(5)

Figure 3 shows that the walking direction is along the \(Y_{[w]}\) axis, which differs from \(\mathbf {a}_{XY[w]i}\)’s direction. The main reason for this difference is not only that the human body shakes left and right while walking, but also that the gimbal lock [30] occurs when \(\theta _{P}\) approaches \(\frac{\pi }{2}\) (e.g., the phone is in the chest pocket), which makes it impossible to uniquely determine the phone’s orientation via Tait–Bryan angles.

Fig. 3.

In summary, even if the pocket is stationary relative to the body, it is challenging to initialize and update \(\mathbf {R}_{[b/w]i}\) while taking both the user’s habit and experience into account.

3.3.2 Height Measurement.

We designed an experiment to evaluate the effectiveness of the linear acceleration sensor and barometer in measuring height. The experimental results in Figure 4 show that the linear acceleration can only detect the height changes during a rapid fall and will incur cumulative errors when calculating \(h_{k/0}\) using Equation (3). Surprisingly, the barometer can measure changes in altitude during rapid and slow falls without incurring cumulative errors.

Fig. 4.

Our another experiment evaluated whether the accuracy of the barometer meets the requirements of fall detection. The phone was dropped 50 times from the 1.2 m table to the floor. Figure 5 shows that the measured height of the table generally follows a Gaussian distribution with a mean of 1.21 m and a standard deviation of 0.13 m. The measurement error is below 10% of the average person’s height.

Fig. 5.

Is it good enough to measure the height with a barometer? According to the ideal gas law [3], the atmospheric pressure is directly affected by temperature. From 9 AM to 8 PM, the phone was placed indoors and outdoors every hour to collect atmospheric pressure data. The sampling rate of the barometer is set to 5 Hz. In each period 1,000 samples are taken. The reported values are mean altitudes converted from raw barometric data by using Equation (4). Figure 6 shows that the altitude measured indoors went up slightly from 11 PM to 2 PM (205.96 m – 209.27 m) and then started to fall reaching the lowest point 191.64 m at 7 PM. Interestingly, the indoor/outdoor measured altitudes were not significantly different during the day but differed by 1.75 m at 8 PM.

Fig. 6.

Unfortunately, barometer readings are unstable even during a short continuous observation (33 mins). Three Android phones, Huawei ALP-AL100, Huawei Nexus 6P, and Google Pixel XL, were placed at the same altitude and started measuring at the same time. As shown in Figure 7, there were some fluctuations in the altitude measurements. For the atmospheric pressure measured by Google Pixel XL, the minimum and maximum values were 230.65 m and 233.23 m, respectively. It should also be noted that the device diversity affects the measurement results. The mean altitudes measured by the three phones were 226.88 m, 235.37 m, and 231.65 m, respectively.

Fig. 7.

In summary, while the barometers can capture the core height changes in different types of falls without cumulative errors, it remains challenging to calibrate the barometric pressure in complex environments.

4 Overview of P\(^2\)Est

We first introduce the assumptions we made and their rationality and then describe the basic framework of P\(^2\)Est.

4.1 Assumptions

The design of P\(^2\)Est is based on the following three “rational” assumptions.

A1.

The user places the phone in the pocket of his/her upper body cloth (shirt/jacket). We chose this pocket because (1) the trunk angle indicates the direction of the sternum relative to gravity, which matches well the definition of fall in Section 3.1; (2) to mitigate the measurement errors, the height of the phone should be as close as possible to the user’s height; (3) compared to other positions (e.g., wrist and handbag), the trunk generates less spatial uncertainty.

A2.

When placing the phone in the same pocket, the user will not change the phone’s orientation. P\(^2\)Est does not assume that the user always puts the phone in the same pocket. To reduce the ambiguity in analysis, the user is assumed to maintain his prior habit and not to flip or operate the phone while placing it in the pocket.

A3.

To initialize P\(^2\)Est, the user is asked to place the phone on the floor and keep it still for a few seconds. P\(^2\)Est needs to estimate the measurement error of the accelerometer at rest and measure the floor pressure related to the initial height of the phone. Note that this is a one-time initialization. The floor pressure can be dynamically calibrated according to the basic statistics.

Although these assumptions impose some restrictions on users, they enable us to use general-purpose devices (e.g., commodity smartphones) for an interpretable and reliable analysis, which, in turn, makes users who are in need more willing to participate in data collection. Some patients suffering from movement disorders can monitor their condition at home with mobile phones, which is very important and convenient for follow-up evaluation.

4.2 Framework of P\(^2\)Est

The basic idea of P\(^2\)Est is to accurately detect, in real time, basic states of the subject, and then make full use of the constraints in these states to update and calibrate the measured data in order to finally gain the detailed characterization of a fall. P\(^2\)Est identifies and discriminates other states of interest (e.g., Falling, Standing, and Sitting) based on the constraint and segmentation derived from the following three basic states:

–

Walking is a steady state of movement. The body remains roughly upright, so the height of the phone in pocket remains stable. In Section 6.2, we will discuss how to eliminate the delay and false positives of the step detector.

–

Stationary is a constant steady state. The phone’s height remains unchanged, although the height might be unknown. A new detection method will be presented and detailed in Section 6.1.

–

Unlocking has a certain directionality. When the user awakes or unlocks the phone, the phone is pointing in the same direction as the body.

As shown in Figure 8, P\(^2\)Est consists of two parts: tilt-angle measurement and height measurement.

Fig. 8.

The core of tilt-angle measurement is a sequence of coordinate transformations. Suppose the states in sampling periods \(r\) and \(s\) are, respectively, Walking and Unlocking, and the body’s orientation has not changed from the \(r\)th sampling period to the \(s\)th sampling period. P\(^2\)Est takes the following steps to calculate the tilt angles:

(1)

determines the bearing of the body based on the phone’s pointing in the \(s\)th sampling period, and constructs a reference Body coordinate system, \(X_{[b]}Y_{[b]}Z_{[b]}\), in the \(r\)th sampling period;

(2)

finds the relative orientation of the body and the phone in the \(r\)th sampling period;

(3)

updates the orientation of the body in the \(t\)th sampling period based on the relative orientation of the body and the phone.

(4)

finds the relative orientation of the body in sampling periods \(r\) and \(t\), and calculates the tilt angles between \(Y_{[b]}\) and those of \(X_{[b]t}\), \(Y_{[b]t},\) and \(Z_{[b]t}\).

To track the body’s rotation, P\(^2\)Est updates \(X_{[b]}Y_{[b]}Z_{[b]}\) by considering the following two conditions:

C1.

The state is Walking in the \(t\)th sampling period. As in the \(r\)th sampling period, the body is also upright in the \(t\)th sampling period. If \(X_{[b]}Y_{[b]}Z_{[b]}\) can be transformed to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\) by horizontal rotation, the phone is in the original pocket. \(X_{[b]}Y_{[b]}Z_{[b]}\) is updated from \(X_{[b]r}Y_{[b]r} Z_{[b]r}\) to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\). Otherwise, it is reset to \(X_{[w]}Y_{[w]}Z_{[w]}\) to wait for the next Unlocking.

C2.

The state is not Walking in the \(t\)th sampling period. If \(X_{[b]}Y_{[b]}Z_{[b]}\) can be transformed to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\) by horizontal rotation and \(X_{[b]}Y_{[b]}Z_{[b]}\) has not been reset, the body is upright. \(X_{[b]}Y_{[b]}Z_{[b]}\) is updated to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\). Otherwise, it remains unchanged.

The update process can continue in the (\(t+1\))-th sampling period. The automatic and continuous updates allow P\(^2\)Est to adapt itself to different user habits.

The key to the height measurement is the prediction and calibration of the floor pressure. When the person is Walking or the phone is Stationary, P\(^2\)Est continuously adjusts the floor pressure to maintain the relative pressure difference. The relative pressure difference directly correlates with the height of the phone. It can be obtained by a short-term measurement even when the height is unknown. Suppose that the states in sampling periods \(u\) and \(v\) are Walking and Unlocking, respectively. If P\(^2\)Est measures the floor pressure in the \(o\)th sampling period, and \(o\) and \(u\) are separated by a short time interval, \(h_{u}\) can be measured very accurately, providing fine-grained calibration in the subsequent Walking. As the floor pressure is continuously adjusted, \(h_{v}\) might not be as accurate as \(h_{u}\), providing coarse-grained calibration.

5 Coordinate Transformation

We now detail the coordinate transformation including the calculation of tilt angle and update of the reference coordinate system.

5.1 Tilt Angle Calculation

Reference-to-world transformation (\(s\)th period): P\(^2\)Est first describes and denotes the rotation matrix from \(X_{[b]}Y_{[b]}Z_{[b]}\) to \(X_{[w]}Y_{[w]}Z_{[w]}\), as \(\mathbf {R}_{[b/w]}\). The horizontal rotation of the body and the phone are \(\theta _{Z[b/w]s}\) and \(\theta _{Z[d/w]s}\), respectively. When the state is Unlocking, \(\theta _{Z[b/w]s}\) equals to \(\theta _{Z[d/w]s}\). Here \(\theta _{Z[d/w]s}\) can be calculated as

\[\begin{eqnarray} \theta _{Z[d/w]s}=\arctan 2\left(\frac{-\mathbf {R}_{[d/w]s}(3,1)}{\mathbf {R}_{[d/w]s}(3,3)}\right), \end{eqnarray}\]

(6)

where \(\mathbf {R}_{[d/w]s}(3,1)\) and \(\mathbf {R}_{[d/w]s}(3,3)\) can be inferred from the rotation vector sensor in Equation (1).

Note that the calculation of \(\theta _{Z[d/w]s}\) is not unique as it is sequence-dependent. When the state is Walking, the body is upright, i.e., \(Y_{[b/w]r}\) coincides with \(Z_{[w]}\). Assume that the interval between \(r\) and \(s\) is short, the body’s orientation has not changed so \(\theta _{Z[b/w]r}=\theta _{Z[b/w]s}=\theta _{Z[d/w]s}\). This assumption will be relaxed in Section 5.2. Let \(X_{[b]}Y_{[b]}Z_{[b]}\) be \(X_{[b]r}Y_{[b]r}Z_{[b]r}\), i.e., \(\mathbf {R}_{[b/w]}=\mathbf {R}_{[b/w]r}\). By defining three Tait–Bryan angles (\(\theta _{Z[b/w]r}=\theta _{Z[d/w]s}\), \(\theta _{X[b/w]r}=\frac{\pi }{2}\), \(\theta _{Y[b/w]r}=\pi\)), \(\mathbf {R}_{[b/w]}\) can be represented using a sequence of rotations:

\[\begin{eqnarray} \mathbf {R}_{[b/w]}&=&\mathbf {R}_{[b/w]r}=\mathbf {R}_{Z}(\theta _{Z[d/w]s})\mathbf {R}_{X}\left(\frac{\pi }{2}\right) \mathbf {R}_{Y}(\pi) \nonumber \nonumber\\ &=&\begin{bmatrix}\cos \theta _{Z[d/w]s} & -\sin \theta _{Z[d/w]s} & 0\\ \sin \theta _{Z[d/w]s} & \cos \theta _{Z[d/w]s} & 0\\ 0 & 0 & 1\end{bmatrix} \nonumber \nonumber\\ &\times &\begin{bmatrix}1 & 0 & 0\\ 0 & 0 & -1\\ 0 & 1 & 0\end{bmatrix} \times \begin{bmatrix}-1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & -1\end{bmatrix}. \end{eqnarray}\]

(7)

Body-to-device transformation (\(r\)th period): P\(^2\)Est then finds the rotation matrix from \(X_{[b]r}Y_{[b]r}Z_{[b]r}\) to \(X_{[d]r}Y_{[d]r}Z_{[d]r}\), denoted as \(\mathbf {R}_{[b/d]r}\). Using \(\mathbf {R}_{[d/w]r}\) obtained from the rotation vector sensor, we can calculate \(\mathbf {R}_{[b/d]r}\) as

\[\begin{eqnarray} \mathbf {R}_{[b/d]r}=\mathbf {R}_{[d/w]r}^\mathrm{T}\mathbf {R}_{[b/w]r}. \end{eqnarray}\]

(8)

Body-to-world transformation (\(t\)th period): Next, P\(^2\)Est finds the rotation matrix from \(X_{[b]t}Y_{[b]t}Z_{[b]t}\) to \(X_{[w]}Y_{[w]}Z_{[w]}\), as \(\mathbf {R}_{[b/w]t}\). With \(\mathbf {R}_{[d/w]t}\) obtained from the rotation vector sensor, we can calculate \(\mathbf {R}_{[b/w]t}\) as

\[\begin{eqnarray} \mathbf {R}_{[b/w]t}=\mathbf {R}_{[d/w]t}\mathbf {R}_{[b/d]r}=\mathbf {R}_{[d/w]t}\mathbf {R}_{[d/w]r}^\mathrm{T} \mathbf {R}_{[b/w]r}. \end{eqnarray}\]

(9)

Body-to-reference transformation (\(t\)th period) & tilt angles: Finally, P\(^2\)Est describes the rotation matrix from

\(X_{[b]t}Y_{[b]t}Z_{[b]t}\) to \(X_{[b]}Y_{[b]}Z_{[b]}\), as \(\mathbf {R}_{[b/b]t}\). Since \(X_{[b]r}Y_{[b]r}Z_{[b]r}\) is identical to \(X_{[b]}Y_{[b]}Z_{[b]}\), \(\mathbf {R}_{[b/b]t}\) is equivalent to \(\mathbf {R}_{[b/b]t/r}\), which is the rotation matrix from \(X_{[b]t}Y_{[b]t}Z_{[b]t}\) to \(X_{[b]r}Y_{[b]r}Z_{[b]r}\). Using \(\mathbf {R}_{[b/w]t}\) and \(\mathbf {R}_{[b/w]}\), \(\mathbf {R}_{[b/b]t}\) can be given by

\[\begin{eqnarray} \mathbf {R}_{[b/b]t}&=&\mathbf {R}_{[b/b]t/r}=\mathbf {R}_{[b/w]r}^\mathrm{T}\mathbf {R}_{[b/w]t} \nonumber \nonumber\\ &=&\mathbf {R}_{[b/w]t}^\mathrm{T}\mathbf {R}_{[d/w]t}\mathbf {R}_{[d/w]r}^\mathrm{T}\mathbf {R}_{[b/w]r}. \end{eqnarray}\]

(10)

The tilt angles between \(Y_{[b]}\) and those of \(X_{[b]t}\), \(Y_{[b]t},\) and \(Z_{[b]t}\) can be finally given by

\[\begin{eqnarray} \theta _{X_{[b]t}Y_{[b]}}&=&\arcsin (\mathbf {R}_{[b/b]t}(1,2)), \end{eqnarray}\]

(11)

\[\begin{eqnarray} \theta _{Y_{[b]t}Y_{[b]}}&=&\arccos (\mathbf {R}_{[b/b]t}(2,2)), \end{eqnarray}\]

(12)

\[\begin{eqnarray} \theta _{Z_{[b]t}Y_{[b]}}&=&\arcsin (\mathbf {R}_{[b/b]t}(3,2)). \end{eqnarray}\]

(13)

\(\theta _{X_{[b]t}Y_{[b]}}\), \(\theta _{Y_{[b]t}Y_{[b]}}\) and \(\theta _{Z_{[b]t}Y_{[b]}}\) represent left/right, up/down and front/back tilts, respectively. Unlike \(\theta _{A}\), \(\theta _{P}\), and \(\theta _{R}\) provided by the Android API, \(\theta _{X_{[b]t}Y_{[b]}}\), they are calculated relative to the Body coordinate system instead of the World coordinate system.

5.2 Reference Coordinate System Update

P\(^2\)Est first determines whether \(\mathbf {R}_{[b/b]t}\) only represents horizontal rotation or not. If \(\mathbf {R}_{[b/b]t}\) is a rotation around the \(Y_{[b]}\) axis, \(\mathbf {R}_{[b/b]t}(2,2)\) is close to 1. It is necessary to determine a threshold for \(\mathbf {R}_{[b/b]t}(2,2)\). Suppose that the state is Walking between sampling periods \(r-f\) and \(r\). For the \(i\)th sampling period where \(i\in [r-m,r)\), \(\mathbf {R}_{[b/b]i}(2,2)\) can be calculated by using Equation (10). Denoting the sample mean and standard deviation of \(|1-\mathbf {R}_{[b/b]i}(2,2)|\), \(\hat{\mu }_{R}\) and \(\hat{\sigma }_{R}\), the \(z\)-score of \(|1-\mathbf {R}_{[b/b]t}(2,2)|\), \(z_{R|t}\), can be defined as

\[\begin{eqnarray} z_{R|t}=\frac{|1-\mathbf {R}_{[b/b]t}(2,2)|-\hat{\mu }_{R}}{\hat{\sigma }_{R}}. \end{eqnarray}\]

(14)

The \(z\)-score is a numerical measurement used in statistics of a value’s relationship to the mean of the reference population. If \(z_{R|t}\lt 3\), \(\mathbf {R}_{[b/b]t}\) is a rotation about the \(Y_{[b]}\) axis.

P\(^2\)Est then updates \(\mathbf {R}\) by considering the following two conditions:

Condition 1 : The state is Walking in the \(t\)th sampling period. If the phone is placed in the same pocket as in the \(r\)th sampling period, \(X_{[b]}Y_{[b]}Z_{[b]}\) is updated from to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\). Otherwise, it is reset to \(X_{[w]}Y_{[w]}Z_{[w]}\). \(\mathbf {R}_{[b/w]}\) can be represented as

\[\begin{eqnarray} \mathbf {R}_{[b/w]}= {\left\lbrace \begin{array}{ll} \mathbf {R}_{[b/w]t}, & \mbox{if} z_{R|t}\lt 3; \\ \mathbf {I}_{3}, & \mbox{otherwise}.\end{array}\right.} \end{eqnarray}\]

(15)

Condition 2 : The state is not Walking in the \(t\)th sampling period. If the body is upright and \(X_{[b]}Y_{[b]}Z_{[b]}\) is not reset, \(X_{[b]}Y_{[b]}Z_{[b]}\) is updated to \(X_{[b]t}Y_{[b]t}Z_{[b]t}\). Otherwise, it remains unchanged. \(\mathbf {R}_{[b/w]}\) can be represented as:

\[\begin{eqnarray} \mathbf {R}_{[b/w]}= {\left\lbrace \begin{array}{ll} \mathbf {R}_{[b/w]t}, & \mbox{if} z_{t}\lt 3 \mbox{ and } \mathbf {R}_{[b/w]}\ne \mathbf {I}_{3}; \\ \mathbf {R}_{[b/w]}, & \mbox{otherwise}.\end{array}\right.} \end{eqnarray}\]

(16)

The continuous update \(\mathbf {R}_{[b/w]}\) enables P\(^2\)Est to track the rotation of the body. The effectiveness of the tilt measurement was evaluated with the same experiment as shown in Figure 2. Figure 9 shows that the tilt angles of the two phones were consistent even though they were placed in different pockets. All tilt angles were close to 0 when the person walked down the spiral stairs. Consistent with the actual observation, when the body bent forward and left, the tilt angles measured by the phone in the chest pocket were larger than that in the waist pocket. Interestingly, the sign of \(\theta _{X_{[b]t}Y_{[b]}}\) (left/right-tilt) measured by the two phones were opposite when the body bent forward. The possible reason for this is that the body is not rigid and may deform when bent.

Fig. 9.

The update of \(\mathbf {R}_{[b/w]}\) can also be used to relax the assumption made in Section 5.1. If the person turns by a certain angle between the sampling period \(r\) and \(s\), the measured tilt angles may differ from the actual values. Figure 10 shows that \(\theta _{A}\) changed from \(-\)1.08 to 2.44 as the tester turned \(\pi\) before Unlocking. \(\theta _{Z_{[b]t}Y_{[b]}}\) changed from 0.04 to \(-\)1.12, which indicates that the tester bent backward, but actually he bent forward. Equations (14) and (16) are used to push \(\mathbf {R}_{[b/w]}\) from the \(r\)th sampling period to the \(s\)th sampling period until the sampling period when the phone is taken out of the pocket, called Pushing. After Pushing \(\mathbf {R}_{[b/w]}\), \(\theta _{Z_{[b]t}Y_{[b]}}\) changed from 0 to 1.02 when the tester bent forward.

Fig. 10.

6 Pressure Calibration

We present (i) a simple, yet effective exponential smoothing to predict and smooth the floor pressure, and (ii) a method to eliminate the delay and false positives of the step detector.

6.1 Floor Pressure Smoothing

P\(^2\)Est requires the user to place his phone on the floor for a few seconds to measure the initial floor pressure, \(p_{F}\), and accelerometer errors. Suppose the phone is on the floor in the \(o\)th sampling period. Taking a window of \(n\) samples, \(p_{F}\) is the mean of \(n\) measured pressures. \(p_{F}\) provides a starting value for subsequent dynamic calibrations and only needs to be measured once. As described in Section 3.2, \(a_{Z[d/w]o}\) can be used to measure changes in height, which is defined in Equation (2). The sample mean and standard deviation of \(a_{Z[d/w]o}\) are \(\hat{\mu }_{a}\) and \(\hat{\sigma }_{a}\), respectively.

P\(^2\)Est then estimates the initial walking pressure, \(p_{W}\), when the phone is in Walking state for the first time. Suppose the step detector detects Walking in the \(u\)th sampling period. Taking a window of \(n\) samples, \(p_{W}\) is the mean of \(n\) pressure measurements. If \(u\) and \(o\) are close enough to each other, \(p_{W}\) can be can be converted to \(h_{u}\) by Equation (4).

Next, P\(^2\)Est detects the phone in Stationary state by measuring the static initial pressure, \(p_{S}\). Assume that the phone is on a table of unknown height in the \(v\)th sampling period. The \(z\)-score of \(a_{Z[d/w]v}\), \(z_{a|v}\), can be calculated as

\[\begin{eqnarray} z_{a|v}=\frac{a_{Z[d/w]v}-\hat{\mu }_{a}}{\hat{\sigma }_{a}}. \end{eqnarray}\]

(17)

If \(z_{a|v}\lt 3\), the phone’s state is Stationary. Taking a window of \(n\) samples, \(p_{S}\) is the mean of \(n\) pressure measurements. If the floor pressure in the \(v\)th sampling period is accurately calibrated, then \(p_{W}\) can be converted to \(h_{v}\) by Equation (4).

Finally, P\(^2\)Est adopts a simple exponential smoothing to forecast the floor pressure, \(\hat{p}_{F|i+1}\), when the state is Walking or Stationary. Considering these two cases, the initial pressure difference from device to floor, \(p_{D/F}\), is defined as

\[\begin{eqnarray} p_{D/F}= {\left\lbrace \begin{array}{ll} p_{W}-p_{F}, & \text{if state is}\;\; {Walking}; \\ p_{S}-p_{F}, & \text{if state is}\;\; {Stationary}.\\ \end{array}\right.} \end{eqnarray}\]

(18)

\(p_{D/F}\) in Walking remains unchanged as long as the phone is in the original pocket. \(p_{D/F}\) in Stationary is stable if the phone’s position doesn’t change. Denoting the measured pressure in the \(i\)th sampling period as \(p_{i}\), the floor pressure in the \(i\)th sampling period, \(p_{F|i}\), can be calculated as

\[\begin{eqnarray} p_{F|i}=p_{i}-p_{D/F}. \end{eqnarray}\]

(19)

The forecasted floor pressure in the \((i+1)\)-th sampling period, \(\hat{p}_{F|i+1}\), is equal to a weighted average between the most recent floor pressure, \(p_{F|i}\), and the previous forecasted floor pressure, \(\hat{p}_{F|i}\), i.e.,

\[\begin{eqnarray} \hat{p}_{F|i+1}&=&\alpha p_{F|i}+(1-\alpha)\hat{p}_{F|i} \nonumber \nonumber\\ &=&\alpha (p_{i}-p_{D/F})+(1-\alpha)\hat{p}_{F|i}, \end{eqnarray}\]

(20)

where \(\alpha \in [0,1]\) is a smoothing parameter which can be estimated by minimizing the sum of the squared residuals (SSE). The estimated pressure difference from device to the floor in the \(i\)th sampling period, \(p_{D/F|i}\), can be given by

\[\begin{eqnarray} \hat{p}_{D/F|i}=p_{i}-\hat{p}_{F|i}. \end{eqnarray}\]

(21)

The measured pressure difference from device to floor in the \(i\)th sampling period is expected to be \(p_{D/F}\). Hence, the residuals are specified as \(p_{D/F}-\hat{p}_{D/F|i}\). Taking a window of \(l\) samples, \(\alpha\) minimizes:

\[\begin{eqnarray} SSE=\sum _{i=1}^{l}(p_{D/F}-\hat{p}_{D/F|i}). \end{eqnarray}\]

(22)

If the state is neither Walking nor Stationary, the smoothing will be suspended, and \(p_{F|i+1}\) will be used as the reference floor pressure.

A normalized pressure ratio \(r_{D/W|i}\) is defined to describe the changes in the height of the device in the \(i\)th sampling period relative to the initial height of the pocket measured in Walking:

\[\begin{eqnarray} r_{D/W|i}=\frac{p_{D/F|i}}{p_{W}-p_{F}}=\frac{p_{i}-\hat{p}_{F|i}}{p_{W}-p_{F}}. \end{eqnarray}\]

(23)

Because \(r_{D/W|i}\) describes the change of barometric pressure, it is not affected by the different measurement values of different mobile phones described in Figure 7. Two elementary experiments were designed to evaluate the effectiveness of smoothing. Take the experiment shown in Figure 4 as an example. As shown in Figure 11, compared to the non-smoothing height ratio, \(r_{D/W|i}\) is more stable in Stationary. When the number of fast-falling samples exceeds 70, \(r_{D/W|i}\) fluctuated around 0 while the non-smoothing height ratio went up from 0.01 to 0.18 and then dropped to \(-\)0.21. As a pressure ratio, \(r_{D/W|i}\) is basically the same as the smoothing height ratio. Figure 12 indicates that \(r_{D/W|i}\) can be used to differentiate states. For example, \(r_{D/W|i}\) was 1 and 0.53 in Walking and Sitting, respectively. \(r_{D/W|i}\) was slightly lower in Sitting than in Walking. This might be due to the estimation error of \(p_{F|i+1}\).

Fig. 11.

Fig. 12.

6.2 Improvement of Fault-Tolerance

P\(^2\)Est improves the accuracy of detecting Walking state by eliminating the effect of step detector’s delay and false positives. When Walking suddenly switches to Falling, P\(^2\)Est may continue adjusting \(p_{F|i}\) due to untimely or incorrect state detection. As shown in Figure 13, \(r_{D/W|i}\) was around 0.9 when a fall took place.

Fig. 13.

To tolerate the delay of step detector, P\(^2\)Est in Walking state suspends smoothing for \(l\) sampling periods, called Pending, if both of the following two conditions are met: (C1) the height change exceeds a certain threshold, i.e., \(|p_{i}-p_{i-1}|\gt 0.2|p_{W}-p_{F}|\); and (C2) the change of angle exceeds a certain threshold, i.e., \(z_{R|i}\gt 3\). The reason for choosing both conditions over one is that the second condition is sometimes too sensitive. When P\(^2\)Est wakes up the smoothing after \(l\) sampling periods, the horizontal shaking of the non-upright body, such as kneeling with both hands on the floor, may lead to false positives of the step detector. If \(z_{R|i}\gt 3\), P\(^2\)Est in Walking state sets \(\alpha\) in Equation (20) to \(\cos \theta _{Y_{[b]t}Y_{[b]}}\). If the body is upright, i.e., \(\alpha \approx 1\), more weight is given to \(p_{F|i}\). Otherwise, more weight is given to \(\hat{p}_{F|i}\). Comparing before and after using Pending, \(r_{D/W|i}\) dropped from 0.9 to 0.04.

7 Experimental Evaluation

We prototype and evaluate P\(^2\)Est, including multiple comparative trials. We implemented a prototype of P\(^2\)Est on an Android phone, Google Pixel XL (with Bosch BMI160 inertial measurement unit and Bosch BMP280 barometer). The frequency of the rotation vector sensor was set to 50 Hz (SENSOR_DELAY_GAME). The frequency of all other sensors was set to the lowest frequency (SENSOR_DELAY_NORMAL).

7.1 Down vs. Up Stairs

The absolute value of the pressure changes with altitude, but the pressure difference relative to the building floor should not change in Walking. The height of the tester is 1.72 m. The phone is placed in the chest pocket with the height of 1.4 m. A tester walked downstairs from the 3rd floor to the 2nd floor, during which the phone was placed on the stair/floor in the middle/last. Then, she/he walked again in the opposite direction. Figure 14 shows that the normalized pressure ratio \(r_{D/W|i}\) dropped from 0.94 to 0.07 when the state changes from Walking to Stationary. When the phone was placed on the floor, the height of the phone is supposed to be 0 m, while the one obtained by \(r_{D/W|i}\) was \(0.07\times 1.4\approx 0.1\)m, reaching decimeter accuracy. Figure 15 shows that the measured pressure \(p_{i}\) went up from 984.29 to 984.96 when going downstairs, and it dropped from 984.90 to 984.47 when going upstairs. As illustrated in Figure 16, \(\alpha\) adjusted adaptively in different states.

Fig. 14.

Fig. 15.

Fig. 16.

7.2 Ratio vs. Pressure

Change-point detection is often used to detect falls. We compared the change points of the measured pressure \(p_{i}\) (raw data) and the normalized pressure ratio \(r_{D/W|i}\) (data processed by P\(^2\)Est). First, we used offline change-point detection to find three samples at which the data mean changes most significantly. Figure 17 shows that the change points of \(p_{i}\) and \(r_{D/W|i}\) when going down the stairs were {164,393,552} and {177,350,552}, respectively. We chose {177,350,552} as the ground truth because \(r_{D/W|i}\) is not affected by the increase in pressure when going downstairs. In actual detection, the number of change points is unknown, and only partial data is available. Then, we used online change-point detection to evaluate the effectiveness of P\(^2\)Est. Cumulative Sum (CuSum) algorithm [6] and Auto-Regressive Moving Average (ARMA) model [34] were used to detect outliers in real time. We created a sliding window of 50 samples in which 40 samples overlap. The outliers detected by the CuSum algorithm were the upper and lower CuSums of those samples that have drifted beyond five standard deviations above and below the mean of those samples within the sliding window. As shown in Figure 18, corresponding to the first change point 177, the outliers of \(p_{i}\) and \(r_{D/W|i}\) were 143 and 183, respectively. The outliers of \(p_{i}\) between 394–410 were false positives caused by the pressure rise when going downstairs. We selected the degrees \(p\) and \(q\) of the ARMA model by the Bayesian information criterion (BIC) and calculated the residual between the predicted value and the true value. Figure 19 shows that the outliers of \(r_{D/W|i}\) were more concentrated near the ground truth than the outliers of \(p_{i}\).

Fig. 17.

Fig. 18.

Fig. 19.

7.3 Indoor vs. Outdoor

As shown in Figure 6, atmospheric pressure is affected by temperature. One tester walked from indoor to outdoor, and placed the phone on the ground for 50 seconds, then walked again in the opposite direction. As shown in Figure 20, the normalized pressure ratio \(r_{D/W|i}\) did not change when the tester walked from indoor to outdoor (or in the opposite direction), but can capture the height change when the phone was placed on the floor. Interestingly, Figure 21 shows that the measured pressure \(p_{i}\) fluctuated even more when the phone was placed on the ground. It took time for the barometer to sense changes in the temperature.

Fig. 20.

Fig. 21.

7.4 Falling vs. Normal States

Three types of fall, i.e., Trip, Slip, and Segmented fall, and three types of normal state, i.e., Sitting (chair) on chair, Sitting (couch) on couch, and Lying (couch) on couch were tested sequentially. The Segmented fall contains two steps: kneeling down to support the floor with both hands, then lying down. The height of mat, chair, and couch were 0.12 m, 0.36 m, and 0.43 m, respectively.

We used offline change point detection to divide the time series of the normalized pressure ratio \(r_{D/W|i}\) into multiple segments, and calculated the sample mean of each segment. Figure 22 shows that \(r_{D/W|i}\) can distinguish the three different normal states but not three falls. \(r_{D/W|i}\) for Trip, Slip, and Segmented were 0.311, 0.346, and 0.438, respectively, while \(r_{D/W|i}\) for Sitting (chair), Sitting (couch), and Lying (couch) were 0.773, 0.653, and 0.442, respectively. As shown in Figure 23, \(\theta _{X_{[b]t}Y_{[b]}}\), \(\theta _{Y_{[b]t}Y_{[b]}},\) and \(\theta _{Z_{[b]t}Y_{[b]}}\) can distinguish Trip and Slip. Trip and Segmented were close and different from normal states. As the body leans back in Sitting (couch), \(\theta _{Z_{[b]t}Y_{[b]}}\) for Sitting (chair) and Sitting (couch) were 0 and \(-\)0.34, respectively.

Fig. 22.

Fig. 23.

7.5 Threshold & Detection

In order to detect falls, an appropriate threshold needs to be set for \(\theta _{Y_{[b]t}Y_{[b]}}\) and \(r_{D/W|i}\). To distinguish between different falls, it is also necessary to observe the changes in \(\theta _{X_{[b]t}Y_{[b]}}\) and \(\theta _{Z_{[b]t}Y_{[b]}}\). These thresholds can be set by following the Pareto principle [20]. For instance, the thresholds for \(\theta _{Y_{[b]t}Y_{[b]}}\) and \(r_{D/W|i}\) can be \(0.2\times \frac{\pi }{2}\) and 0.2, respectively. However, in practice, we need to consider more factors, such as the initial tilt angle corresponding to \(\theta _{Y_{[b]t}Y_{[b]}}=0\) and the absolute height corresponding to \(r_{D/W|i}=1\).

We first conducted a set of tests for setting the threshold of \(r_{D/W|i}\). Two states Falling and Lying (couch) were considered. Figure 24 shows that if the threshold for \(r_{D/W|i}\) was set to 0.2, the false positive rate was 0 but the false negative rate was 0.3, possibly because the pocket’s height is lower than the person’s, and the sum of mat height (0.12 m) and chest thickness (around 0.2 m) is around 0.32 m which is close to the couch’s height (0.43 m). Another observation is that the difference between \(r_{D/W|i}\) in Lying (couch) and Falling was quite different in different tests. The reason is that if it was a forward fall, the thickness of the chest was not taken into account.

Fig. 24.

We then conducted another set of tests to observe the effect of changes in body posture and phone position on P\(^2\)Est. We changed the body postures (Crooked and Upright) and the phone positions (Jacket pocket, Belt, and Trouser pocket) to test the changes of \(\theta _{P}\) (provided by Android API) and \(\theta _{Y_{[b]t}Y_{[b]}}\) (calculated by P\(^2\)Est), both of which are used to measure the inclination of the body when falling. Figure 25 shows that in the Walking state, \(\theta _{Y_{[b]t}Y_{[b]}}\) was close to 0 for all cases, while \(\theta _{P}\) was \(-\)0.5989, \(-\)1.3428, 0.1857, and \(-\)1.2826, respectively. As Device and Body coordinate systems may not be consistent with each other (caused by body posture or mobile phone position), the variables measured in the Device coordinate system (e.g., \(\theta _{P}\)) may fail to accurately describe and analyze a fall. The continuous fluctuation of \(\theta _{P}\) for Trouser was due to the shaking of the phone while walking. In the Falling state, \(\theta _{Y_{[b]t}Y_{[b]}}\) was within [0, \(\frac{\pi }{2}\)] for all cases, and \(\theta _{P}\) for Belt was \(-\)1.4153. These results indicate that the different positions and postures may not affect the initial value and range of \(\theta _{Y_{[b]t}Y_{[b]}}\), which is essential for standardized data collection and accurate data analysis. Figure 25 also shows that in the Falling state, \(\theta _{Y_{[b]t}Y_{[b]}}\) for Crooked and Upright were 0.5278 and 1.5358, respectively. If \(\theta _{Y_{[b]t}Y_{[b]}}\) is too small, it may lead to false negatives in fall detection.

Fig. 25.

Overall, after making simple settings, P\(^2\)Est can replace special devices with mobile phones to collect standardized data in complex situations. How to set the thresholds requires further modeling the normal and abnormal behavior of users, which is beyond the scope of this article (and thus part of our future work).

8 Discussion

Some basic principles and algorithms of P\(^2\)Est can be extended to other applications. If the spatial or map information of a building is available, the pressure calibration can find the absolute, instead of relative, values. Combining height with the body orientation information, which can be obtained from \(\mathbf {R}_{[b/w]t}\) defined in Equation (9), it is possible to provide a fine-grained indoor localization/navigation service [25, 26].

When the phone is not in the pocket (e.g., in hand or in a handbag), P\(^2\)Est can set a certain amount of time to observe whether the phone is stationary or not, and then provide a manual input option to choose if the user needs emergency service like what Apple Watch does [29]. Upon detection of a fall, P\(^2\)Est will prompt the user first, and then call or send a text message, if the user doesn’t respond, to a pre-specified third party, such as the care center, relatives, or friends.

There still exist some minor issues associated with P\(^2\)Est as described in the assumptions. If users flip or operate the phone while placing it in their pocket, the bearing of the body might be ambiguous, since P\(^2\)Est cannot tell whether the body or the device has rotated \(\pi\) through coordinate transformations. This ambiguity could be eliminated via machine learning, which is part of our future work.

9 Limitations and Future Work

In this article, we are mainly concerned with the selection (height and tilt angle) and description (coordinate system transformation) of data in fall detection, which will help extract features with physical meaning to build an interpretable detection model. Since we have not trained the detection model, we do not fully know the factors that affect detection accuracy. Our preliminary experiments have found that certain posture abnormalities (e.g., a crooked back) may increase the false positive rate. To develop a medical system suitable for specific situations (e.g., patients with movement disorder), we would like to conduct the follow-up studies in future: (1) collect more cases to build a data set; (2) use machine learning algorithms to build a model; (3) investigate the influence of model parameters on accuracy; (4) quantify the patient’s abnormality indicators and assess the risk of falling.

10 Conclusion

In this article, we have first identified and addressed the challenges in detecting human falls, and then developed a simple yet practical approach, P\(^2\)Est, for measuring body dynamics in terms of tilt angles and height changes. P\(^2\)Est is more efficient and scalable than the state-of-the-arts, as it leverages generic (widely deployed) mobile devices and can adapt itself to users’ habits. We have also implemented a prototype of P\(^2\)Est with Android phones and conducted extensive experiments. Our evaluation results have shown that P\(^2\)Est can detect falls in various complex environments. Moreover, the algorithms developed for P\(^2\)Est have the potential for various indoor applications.

Footnotes

https://www.who.int/news-room/fact-sheets/detail/falls.

https://developer.android.com//reference/android/hardware/Sensor.html.

References

[1]

B. Aguiar, T. Rocha, J. Silva, and I. Sousa. 2014. Accelerometer-based fall detection for smartphones. In Proceedings of the 2014 IEEE International Symposium on Medical Measurements and Applications. 1–6.

Abstract

1 Introduction

2 Related Work

3 Preliminaries

3.1 Definition of Fall

3.2 Pose-estimation Sensors

3.2.1 Orientation Determination.

3.2.2 Height Measurement.

3.2.3 State Detection.

3.3 Challenges in Pose Estimation

3.3.1 Tilt-angle Measurement.

3.3.2 Height Measurement.

4 Overview of P\(^2\)Est

4.1 Assumptions

4.2 Framework of P\(^2\)Est

5 Coordinate Transformation

5.1 Tilt Angle Calculation

5.2 Reference Coordinate System Update

6 Pressure Calibration

6.1 Floor Pressure Smoothing

6.2 Improvement of Fault-Tolerance

7 Experimental Evaluation

7.1 Down vs. Up Stairs

7.2 Ratio vs. Pressure

7.3 Indoor vs. Outdoor

7.4 Falling vs. Normal States

7.5 Threshold & Detection

8 Discussion

9 Limitations and Future Work

10 Conclusion

Footnotes

References

Cited By

Index Terms

Recommendations

IoT based fall detection and ambient assisted system for the elderly

A method with triaxial acceleration sensor for fall detection of the elderly in daily activities

Fall Detection Using Accelerometer, Gyroscope & Impact Force Calculation on Android Smartphones

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations