GB2523978A - User monitoring system and associated method - Google Patents

Abstract

A user monitoring system uses a reconfigurable sensing platform (20) capable of receiving and communicating with sensor devices (22) to exchange information for example about patient health and/or biometric information. A remote computing module receives, stores and transmits information wherein the reconfigurable sensing platform (20) is capable of simultaneously communicating with a plurality of sensor devices (22) and modifying information received from the sensor devices (22) using a fusion process to provide enhanced information about a user. Dynamically updated group data models may be provided using the linked data, so that information collected from a user may be used during training of the system to determine baseline background data models of health. The system is adaptable to interact automatically with a new sensor device, such as body and ambient temperature, fall snsors, PIR/Intruder detection, door, light, TV, brain, heart rate, blood pressure/oxygen sensors, or to allow an external service provider to modify information held.

Description

Title: User Monitoring System and Associated Method

Field of the invention

This invention relates to a user monitoring system and a method of processing user information.

Background to the invention

Personal healthcare devices associated with an individual are increasingly being used to monitor information about the user's own health and fitness. Such devices incorporate sensors to monitor heart rate and other indicators to allow individuals to monitor their health and/or fitness, allowing for health and fitness improvement. Such devices can also be used to allow healthcare professionals to undertake remote monitoring of the health of patients.

is Summary of the invention

In accordance with one aspect of the present invention, there is provided a user monitoring system or apparatus comprising: a user device incorporating a reconfigurable sensing platform capable of receiving and communicating with sensor devices to exchange information; a remote computing module in communication with the reconfigurable sensing platform to receive, recognise, store and transmit information between the remote computing module and the reconfigurable sensing platform; and wherein the reconfigurable sensing platform is capable of simultaneously communicating with a plurality of sensor devices and modifying information received from the sensor devices using a fusion process to provide enhanced information about a user, such as their status. The sensor devices typically provide information on health indicators including physiological statistics such as heartbeat, breathing rate, temperature, blood pressure, blood glucose, insulin levels and other indicators such as positional, speed and orientation indicators which, whilst not physiological signs, nevertheless indicate status relevant to a user's health, for example whether they are moving, lying down or sitting up. Thus the sensors may include biometric sensors, medical diagnostic sensors, movement and orientation sensors. Sensors relating to external factors, such as lighting or room temperature, may also be included. By

I

using a fusion process on information received from a plurality of sensors, the enhanced data thus obtained provides extra information about the user that is not immediately apparent from the unprocessed information received from the individual sensors themselves.

The user monitoring system may further comprise a plurality of user devices in communication with a common remote computing module, the common remote computing module adapted to store information from the plurality of user devices and dynamically update data models based on a group of all user devices, with this group a model used to update the ifision process carried out by each reconfigurable sensing platform. This allows information acquircd by a whole community of uscrs to be used to continually modify data models for the whole group, then using this information to improve data enhancement at each individual device. Thus by adaptively adjusting to the data as acquired by the entire group, the quality of enhanced data information is is continually improved.

The common remote computing module may be adapted to supply dynamically updated group data models to one or morc user devices to update the thsion process carried out by each reconfigurable sensing platform.

The user monitoring system may further comprise a plurality of user devices in communication with a common remote computing module, the common remote computing module adaptable in response to a new sensor device associated with a first user device so as to reconfigure the reconfigurable sensing platform of the first user device to operate with the new sensor device and to communicate with other user devices to reconfigure them to interact automatically with such a new, previously unknown, sensor device.

The remote computing module may be in communication with an external service provider to allow for external modification of information, such as additional programming, typically firmware updates, held on the remote computing module.

This feature ensures that where a user device is connected to a new or unknown sensor device, the external service provider, such as an internet or web-based service, can detect this and provide appropriate programming protocols to allow the new sensor device to be integrated into the system.

The remote computing module once programmed with appropriate software to allow communication with such a new sensor, can seed this information to all or selected user devices within the group of devices associated with the system. Thus the remote computing module acts like a hive-mind taking information from one user device, updating to accept such information, i.e. the new sensor, and then informing all other user devices of this information, either immediately or at such time as those other user io devices acquire an equivalent sensor device.

In accordance with another aspect of the invention, there is provided a method of processing user information comprising: i) defining a background data model by acquiring sensor information from sensor is devices from a plurality of users; ii) collecting user information from a specific user having a plurality of sensor devices; iii) comparing the collected user information relative to the background data model to define acceptable data values for the plurality of sensor devices associated with a particular user; and iv) supplying the acceptable data values to a reconfigurable sensing platform to provide enhanced data information concerning a user by analysing the sensor information using a fhsion process.

The method is particularly of use for monitoring the health status of a user.

Collecting user information, typically as data, from a specific user may take place during a training set of activities, so as to achieve a comparable data set for all users.

Acceptable data values are preferably determined by determining a baseline value of the collected user information relative to the background data model, with the background model giving confidence levels for such collected user information.

The fusion process is preferably based on the background model and baseline values determined from the collected user information.

In accordance with another aspect of the invention there is also provided a user device, typically for monitoring health status, comprising a reconfigurable sensing platform capable of receiving and communicating with sensor devices to exchange information, the reconfigurable sensing platform capable of simultaneously communicating with a plurality of sensor devices and modifying information received from the sensor devices using a fhsion process to provide enhanced information about a user, and typically enhanced information about a user's healthcare status.

The reconfigurable sensing platform may further comprise communication means adapted to communicate with a remote computing module so as to receive information from the remote computing module for modifying parameters used in the is fusion process.

The user monitoring system and method of processing information as aforesaid allow a user device to be dynamically updated on the basis of information acquired from a group of user devices allowing more reliable enhanced information to be generated.

The invention will now be described, by way of example, and with reference to the accompanying drawings in which: Figure 1 is a schematic diagram illustrating a user monitoring systcm in accordance with the present invention; Figure 2 is a schematic diagram of a reconfigurable sensing platform used in the invention; and Figure 3 shows is an explanatory diagram of functions within the reconfigurable sensing platform.

Description

In Figure 1, a user 10 wearing a personal monitoring device 12 attached to two or more sensors monitoring different health indicators such as blood-pressure, oxygen levels in the blood, heart rate, motion etc. or biometric indicators such as brain activity, brain mapping, body movements or facial recognition is in two-way wireless communication through the Internet 14 with a remote external computing module 1 6 accessible by a scrvicc provider 18. User 10 can bc undcr monitored carc by a healthcare professional and so require alerts to be generated when physiological signs are impaired. Alternatively the user can be interested in improving heahh indicators, such as blood pressure and heart rate in response to activity. For physically impaired users, the monitoring device can be used to aid assisted living, monitoring health and enabling gestures or voice activation to operate a sensor and cause the sensor in turn to operate a device within an enclosed environment, which can be electronic devices io such as a television, computer or even vehicles.

The user's personal healthcare device 12 is independently reconfigurable by service provider 18 so that additional sensors can be monitored by device 12, device 12 upgraded, or data acquired from a plurality of user devices combined to derive is additional information on the health and fitness of any individual user. Hcalthearc monitoring device 12 can be upgraded to incorporate additional sensors when user 10 wishes to monitor a new parameter of their body functions. For example, user 10 may initially start off with a heart rate sensor and a blood-pressure sensor and upgrade to incorporate a movement and location sensor. User 10 simply plugs the new sensor into personal healthcare device 12 and does not need to take any further action. The system will automatically configure to accept the new sensor, acquiring whatever relevant data and key codes it might require from external computing module 16 and addressing any software compatibility issues.

The system can adaptively learn in response to combinations of sensors and use these to provide enhanced information about a user's health status. For example, the system can have recorded data indicating that a user's heart rate and movement characteristics usually correspond in a certain way but if these change in an unexpected manner, the system can usc this recorded data to idcntitj possible health problems. Appropriate alerts can be provided to a user and!or an external party or object, such as a medical monitoring service associated with the service provider or computing device or smart television.

Within personal healthcare monitoring device 12, a reconfigurable sensing platform in the form of a standard on chip (S0C) module 20 is provided. Thus is typically a field programmable gate array (FPGA) configured to undergo simultaneous two-way communication with multiple user sensor devices 22, 22', 22", whether wirelessly or by hard-wired connection so as to store, modify and interpret data whether from user sensor devices or external module 16 and to provide communication ports for receiving and transmitting information to external module 16.

FPGA 20, such as a Xilinx Spartan or the like, can be considered as a top layer 24 of io fixed circuit programming, a middle layer 26 of rcconfigurablc programmable architecture and a bottom layer 28 for rcconfigurablc communications, sec Figure 3.

Top layer 24 allows for communication with remote external module 16, typically via industry standard Simple Object Access Protocol via extended mark-up language and is hypertext transfer protocol (XML plus HTTP access). A secure socket layer is used as an underlying transport mechanism to ensure that the exchange between remote module 16 and SoC module 20 incorporate industry standard encryption. Encryption may involve automated SIM-kcy exchange between external module 16 and SoC module 20, with security of data being frirther enhanced through a set of rolling key-encryption. This fixed layer 24 ensures that secure channel communication is possible between SoC module 20 and remote external module 16.

Middle layer 26 of rcconfigurablc architccturc allows for communication between SoC module 20 and attached sensor devices 22, 22', 22". Programming for this layer can be undertaken at the manufacturing stage or acquired from external module 16, with programming being modifiable from external module 16 throughout the life of user device 12 and also adaptable depending on acquired data from either sensors 22, 22', 22" or external module 16. This layer 26 is adaptable to new sensors as and when they are introduced, this layer automatically reconfiguring for compatibility with a new sensor and downloading appropriate software from external module 16 where required.

Bottom layer 28 is a communications layer allowing user device 12 to communicate with extemal module 16 via any appropriate communication standard such as Bluetooth, Wi-fl, Zigbcc, LAN, 3G. 4G or as desired by the customer.

Figure 2 shows in outline communication between SoC module 20, sensor devices 22, 22', 22" and external module 16. External module 16 provides a large but sealeable database which can store user profiles, device information and the like centrally and securely for all users of the system with appropriate encryption used for communication with SoC 20 or external service provider 18. *I0

External module 16 as shown in Figure 1 is accessible by external service provider 18 who can store device profiles, user profiles, communications profiles, operational programs, firmware updates and the like on the module for downloading by SoC module 20 as required. External module 16 also acquires data from all SoC modules is with which it communicates to update existing profiles and provide data for uploading to external service provider 18 who can monitor user data and if appropriate send this to healthcare practitioners.

External module 16 consists of a set of web-service application programming interfaces (API) which consist of a set of closed API to eater for secure channel communication between external module 16 and SoC module 20, a set of open API for developers to implement fitnetionality within SoC module 20, and which are thus adaptable by the developers, and a set of customised API which implement learning capabilities of the system overall and which will be discussed later.

The closed API ensures that the basic communications between extemal module 16 and SoC module 20 are well defined and ensure that there is always a fixed route to re-programme and monitor SoC 20 from a central point such as external module 16 whether the user device is new or an existing device requires updating.

The open API provide an open-platform for developers to create and implement functionality of current and new devices with SoC module 20. In particular inter-operability of new sensors can be customised and additional properties of new sensors implemented, typically through object-oriented programming.

The operation of the invention will now be described by way of example.

When a user 10 purchases a healthcare device 12 and inserts a subscriber identification module (SIM) card into the unit, software in SoC 20 utilises the unique identification from the SIM to form a set of encryption keys. During a one-off step/registration process, SoC module 20 and external module 16 establish a secure a and trusted communication path. The remainder of the communication (encrypted data) between dcvicc 12 and extcmal modulc 16 is performed via standard HTTP protocols.

Each user device 12 is able to communicate with extemal module 16 when it is first is initialised, allowing each user device 12 to be registered dynamically and securely.

Thus the operation of SoC module 20 can be described as follows: 1. On initial activation of a user device 12, SoC module 20 powers-up and a bootloader is used to program the "closed" API from EPROM to programmable logic blocks to ensure that SoC module 20 communicates with external module 16 as soon as it is switched on.

2. SoC module 20 self-registers its internal ID, which consists of a unique hardware vendor address and SIM ICCID with external module 16 and engages in a secure handshake mechanism to ensure that device 12 and external module 16 can now start to communicate securely via web protocols.

3. SoC module 20 can now be programmed remotely through external module 16 by a service engineer 18 who needs to ensure the validity and the authenticity of message exchange between SoC 20 and external module 16. The downloaded code will re-program a predefined set of logic blocks to ensure that SOC module 20 is configured.

4. Once configured, SoC module 20 downloads the "open" set of API from external module 16 which provides reconfiguration parameters and libraries to program SoC module 20 to be responsive to sensor devices. Typically SoC 20 is seeded with program routines which allow it immediately to communicate with, store data from, and be responsive to the most commonly used sensor devices. As new sensor devices are added and deleted, SoC 20 reprograms itself, either automatically or in response to external module 16 to communicate with any new sensor devices.

Thus for a new sensor device for which a profile is already stored on external module 16, SoC 20 will detect the sensor, download a sensor profile from extemal module 16, update its programmable logic block with the new sensor profile and the new sensor device can now connect and authenticate with SoC module 20 and interact with the a healthcarc monitoring system. If desired, new API can be developed when a new set of sensors arc introduced, and the revised configuration to incorporate the new sensor made available to existing and future installations of SoC modules 20 in the group of modules 20 associated with the system.

5. SoC module 20 monitors the interaction with its users, captures data from is sensors, and relays the information to external module 16.

SoC module 20 is particularly important to allow interaction between SoC module 20 and sensors, assessment of data obtained from sensors and intelligent adaptation of sensor software by SoC 20 to modi' and adapt ftmctionality of bio-sensors and by fusion learning to combine information from different sensors to deduce information beyond that collected.

Where a new sensor is added which is unrecognisable to SoC 20, SoC 20 will relay data information about the new device to external module 16 and if a suitable programming profile is not available on external module 16, external service provider 18 will be alerted so that appropriate software can be written and loaded into the system for downloading by SoC 20.

Where new sensor profiles are created in this way, the profile can then be seeded to other SoC modules 20 within the system. Typically manufacturers will provide a software development kit to enable a programmer to develop software to interact with new sensor devices, ideally prior to the sensor device launch so that an appropriate profile can be seeded onto external module 16 ready for download by SoC 20 as and when such sensors are acquired by users and added to their healthcare monitoring device 12.

The system can thus detect and reconfigure SoC 20 for any new device acquired without the user having to undertake any actions themselves. As long as a new sensor device has had its profile loaded into external module 16, then it is available for seeding to any and all of the groups of SoC modules associated with external module 16. Thus if required, all users can be updated at once to ensure their devices 12 are compatible with a new sensor or new sensor profiles can be downloaded as required a as users acquire such new sensors. Thus instead of a user needing to be actively involved in ensuring a new sensor is compatible with their healthcare monitoring device, the user uploads relevant software patches, identification codes and the like, the update is managed remotely and independently of user 10. Further the group of users subscribing to a system all benefit from an initial user adopting a new and is previously unknown sensor for which there are no appropriate programming routines as external module 16 will detect this, the external service provider 18 provide corrected programming and the system can be adapted to integrate the new sensor at a much earlier date than would otherwise be possible. Thus all SoC modules have continuous learning capabilities introduced through the growing set of all users within the system.

In addition to this feature, the present system also uses a fusion process to utilise acquired data in an adaptive manner so that acquired data can be used to provide enhanced information.

To ensure a user device is able to use a fusion process in accordance with the invention to deduce additional information, a user on acquiring a new device and setting it up will undergo an initial training session. This allows a user individual configuration range for each sensor to be collected and stored in external module 16 to give a baseline value for that user and ranges within which their sensor responses

are acceptable.

The fusion process is based on score normalisation. A set of scores, or confidence level, from a set of activities from different users and devices is used to define and generate a background model. The training of the background model can be achieved by utilising the set of training data collected via multiple users during the initial training routine via external module 16. Once the background model is trained (from about 10 users to N users, with the larger the background model, the more computing cycles is required), this model will be used as the basis for score normalisation and should not be changed. Any change in the background model will require all other SoC modules 20 to be updated with the new background model seeded from external io module 16.

When a new user is introduced to SoC module 20, the user will go through an initial training routine. This routine allows the user to train their model for each sensor device. The score obtained from each sensor device 22, 22', 22" will be re-biased is against a background model and a new score will be stored. The new score will be used as basis for acceptance during the operation of the system. Thus in the training process, a set of scores for user A (SeoreAA) is produced and a set of background scores for users already trained for device A on the system provides a BackgroundScoreAA. The normalised score "NormalisedScoreAA" for user A is calculated based on the ratio of ScoreAAiBackgroundScoreAA. A second device, device B is registered and again it goes through a training process for user A to generate a set of scores "ScoreAB" for user A. Again a set of background scores for users already trained for device B on the system is used to calculate a normalised score "NormalisedScoreAB" from the ratio of ScoreAB/BackgroundScoreAB.

A fusion method is then employed such that if (ScoreAA > Threshold/Bias Score) AND/OR/EXOR/NAND if (ScoreAB > Threshold/Bias Score), decisions are made using an appropriate look-up table. Thus by way of example, where sensor A is a fall sensor and sensor B is a heart sensor, the following look-up table may be used in response to sensor readings: A B Decision o o AIIOK o 1 Potential seizure, take action, investigate 1 0 Fall detected, patient OK 1 1 Fall detected, seizure, take drastic action Table 1: Example decision outcomes for two sensors The decision making mafrix can incorporate as many sensors as required. By using the fusion process to give a variety of outcomes depending on the combination of multiple sensor outputs obtained instead of acting on a single decision factor provided by an individual sensor, enhanced data accuracy is provided and more information derived from multiple sensors when assessed in combination than if assessed sequentially in turn. This can be used to trigger alerts about patient condition or if a desired, the device can monitor multiple activities by a user and use these to trigger to external devices. Thus a user with impaired mobility might have sensors detecting eye movement and finger movement to allow them to control remotely a number of different external devices such as a television, wheelchair or computer.

is Table 2 below illustrates a more complex arrangement where seven sensors are interpreted using a fusion process to determine the required action.

In Table 2, sensor A is a fall sensor where a value I indicates a fall having taken place, sensor B a heart sensor where a value of I indicates a disruption to the usual heart rhythm, sensor Cl is a temperature sensor (person) with readings in degrees Centigrade, sensor C2 a temperature sensor for the home with readings in degrees Centigrade. Sensors associated with external stimuli are also associated with the user device 12 and so sensor D is a movement detector such as a PIR!Intruder detector, sensor E a door sensor, and sensor F a TV/light sensor. For each of these three sensors, RT indicates the sensor has recently been triggered. Thus some of the sensors relate directly to the user and some of the sensors connected to device 12 detect external triggers such as room temperature and lighting.

Taking into account 27 degrees Centigrade as normal body temperature and 23 degrees Centigrade as the preferred ambient temperature, depending on the scenario readings could be obtained as follows with thc dcsircd action as detcrmincd from thc readings givell in the last column: A B Cl C2 D E F Outcome/Action o o 27 23 RT N/A RT AIIOK o 1 27 23 RT N/A RT Potcntial seizurc, take action, investigate 1 0 27 23 RT N/A RT Fall detected, patient OK 1 1 27 23 RT N/A RT Fall detected, seizure, take action 1 27 23 0 N/A 0 Fall detected, seizure, take drastic action, serious incident o o 27 23 RT RT RT All OK, front door entry/exit 1 0 27 23 RT RT 0 Fall detected, front door entry/exit (dependent on time of day, intrudcr?) 1 27 23 RT RT 0 Fall detected, front door entry/exit (dependent on time of day, intruder, startled, seizure?), investigate o o 27 23 RT N/A N/A All OK, patient moving around at homc o o 24 18 RI N/A RT Central heating turned off? o o 30 23 N/A N/A N/A Pationt ill? o o 30 30 N/A N/A N/A Heatwave, check with patient'? Table 2: Example dccision outcomes for scvcn sensors The flexibility and combination of external module 16 and SoC module 20 allows for the system's background model to be dynamically updated with new devices and new io users.

During operation, users will generate information such as heart rate, breathing rate, blood pressure, sounds from speech, brain activity, facial recognition, speed, location, and body movements such as eye movements, facial, hand and finger movements from their customised set of personal sensor devices. SoC module 20 uses the background model to perform score level normalisation in order to provide a higher rate of recognition. The proposed fusion process will introduce a series of bias scores (based on the success rate of recognition), to each device. The bias score for 0 to 1 (0 to 100%) provides the relevant bias to adjust the prominence of each device tailored to a particular set of users. Both the user module as well as the bias score is stored within the user profile. The profile is also stored within the external module 16 which allows users to utilise other SoC modules 20 elsewhere.

Claims (15)

1. Claims 1. A user monitoring system comprising: a user device incorporating a reconfigurable sensing platform capable of receiving and communicating with sensor devices to exchange information; a remote computing module in communication with the reconfigurable sensing platform to receive, store and transmit information between the remote computing module and the reconfigurable sensing platform; and wherein the reconfigurable sensing platform is capable of simultaneously io communicating with a plurality of sensor devices and modifying information received from the sensor devices using a frision process to provide enhanced information about a user.
2. 2. A user monitoring system according to claim 1, further comprising a plurality of is user devices in communication with a common remote computing module, the common remote computing module adapted to store information from the plurality of user devices and dynamically update data models based on a group of all user devices.
3. 3. A user monitoring system according to claim 2, wherein the common remote computing module is adapted to supply dynamically updated group data models to one or more user devices to update the ifision process carried out by each reconfigurable sensing platform.
4. 4. A user monitoring system according to claim 1, further comprising a plurality of user devices in communication with a common remote computing module, the common remote computing module adaptable in response to a new sensor device associated with a first user device so as to reconfigure the reconfigurable sensing platform of the first user device to operate with the new sensor device and to communicate with other user devices to reconfigure them to interact automatically with such a new sensor device.
5. 5. A user monitoring system according to any of the preceding claims, wherein the remote computing module is in communication with an external service provider to allow for external modification of information held on the remote computing module.
6. 6. A user monitoring system according to any of the preceding claims, wherein user devices detect at least health and/or biometric information from a user.
7. 7. A method of processing user information comprising: 1) defining a background data model by acquiring sensor information from sensor io devices from a plurality of users; ii) collecting uscr information from a spccific uscr having a plurality of sensor devices; iii) comparing the collected user information relative to the background data model to define acceptable data values for the plurality of sensor devices associated is with a particular user; and iv) supplying the acceptable data values to a reconfigurable sensing platform to provide enhanced data information concerning a user by analysing the sensor information using a fusion process.
8. 8. A method of processing user information according to claim 7, further comprising collecting user information from a specific user during a training set of activities.
9. 9. A method of processing uscr information according to claim 7 or claim 8, whcrcin acceptable data values are determined by calculating a baseline value of the collected user information relative to the background data model, with the background model giving confidence levels for said collected user information.
10. 10. A method of processing user information according to claim 9, wherein the fusion process is based on the background data model and baseline values determined from the collected user information.
11. 11. A method according to any of claims 7 to 10, wherein the user information relates to at least health and/or biomctric information.
12. 12. A user device comprising a reconfigurable sensing platform capable of receiving and communicating with sensor devices to exchange information, the reconfigurable sensing platform capable of simultaneously communicating with a plurality of sensor devices and modifying information received from the sensor devices using a fusion process to provide enhanced information about a user.
13. 13. A user device according to claim 12, wherein the rcconfigurable sensing platform a further comprises communication means adapted to communicate with a remote computing module so as to receive information from the remote computing module for modifying parameters used in the fusion process.
14. 14. A user device according to claim 12 or claim 13, wherein the sensor devices detect is at least health and/or biometric information from a user.
15. 15. A user monitoring system and method of processing user information substantially as herein described with reference to and as illustrated in the accompanying drawings.