JP6139148B2 - Autonomic function evaluation device, autonomic function evaluation system, autonomic function evaluation server, autonomic function evaluation device program, and autonomic function evaluation server program - Google Patents

Description

  The present invention relates to an autonomic nerve function evaluation device, an autonomic nerve function evaluation system, an autonomic nerve function evaluation server, an autonomic nerve function evaluation terminal device, an autonomic nerve function evaluation device program, an autonomic nerve function evaluation server program, and an autonomic nerve function evaluation terminal. Regarding the program.

  A technique for detecting autonomic nervous tone by measuring fluctuation for each heartbeat is known. Patent Document 1 discloses a high frequency component (HF) that is a parasympathetic nerve activity index and a sympathetic nerve activity index by performing frequency analysis on a heartbeat pattern obtained from a donor measured by an electrocardiograph. It is described that a low frequency component / high frequency component (LF / HF) ratio is calculated.

  In Patent Document 2, electrocardiogram data of a measurement subject is measured by an electrocardiogram monitor, and based on the measured electrocardiogram data, a high-frequency component HF, a low-frequency component LF, and a CVRR (electrocardiogram RR interval variation). Coefficient). Based on these calculated values, a sympathetic nerve activity index (LF / HF), a parasympathetic nerve activity index (CVRR × HF / (LF + HF)) are calculated, and the calculated sympathetic nerve activity index and It describes that the relationship with the accessory nerve activity index is two-dimensionally displayed on a display device. FIG. 16 is a diagram described in FIG. 16A and 16B show heart rate variability from electrocardiographic data input from an electrocardiogram monitor. 16C and 16D measure the RR interval by taking an interval between the R wave and the next R wave, and then plot the measured RR interval data at the time position of the rear R wave. It is described that the data is resampled at equal intervals (dotted lines in FIG. 16C) after interpolation.

  Patent Document 3 describes that a finger is brought into direct contact with a glass plate or a lens, and an apparatus user places the fingertip to measure a pulse wave. In the measurement of the pulse wave, the blood flow amount flowing through the blood vessel of the fingertip changes according to the pulse, so that the transmitted light amount of the fingertip due to external light entering from around the fingertip changes. The image of the fingertip is captured at a predetermined time interval by the CCD image sensor (the mobile phone camera (CCD image sensor) often has 7.5, 10, 15, 30 frames / second, etc.), and the luminance average for each frame Is calculated (see paragraphs “0014” to “0016”). In addition, the R component of the RGB element of the image data may be used instead of the luminance, or when the image data is in the YUV format or YIQ format generally used in the recording format of television broadcasting or digital video, It is described that the Y component can also be used (see paragraph “0076”).

JP 2005-261777 A JP 2011-120618 A JP 2009-72438 A

  The electrocardiogram monitor described in Patent Document 2 is: “The electrocardiogram monitor is equipped with a negative electrode at the throat of the subject, a positive electrode on the left flank, and a body ground on the right flank, and the heart movement as an electrical signal. It is obtained and recorded as electrocardiographic data "(paragraph" 0028 "), and the apparatus is complicated, and the use of the electrode requires labor. The same applies to the electrocardiograph described in Patent Document 1. Therefore, the subject cannot use such a device in daily life. Although the technique of the pulse wave detection using the camera of patent document 3 notifies a proper press state at the time of pressing a fingertip on a press surface to a test subject, it improves a pulse wave detection accuracy. There is a demand for a technique for detecting pulse waves with higher accuracy than that detected by this technique.

  This invention is made | formed in view of the problem which concerns, and provides the technique which a test subject can always carry and can manage his mental health easily.

  The autonomic nervous function evaluation device of the present invention detects a RR interval time-series data from a camera that obtains luminance data by photographing luminance of the skin that changes according to blood flow, a display that displays an image, and the luminance data. RR interval time series data detection processing unit, Fourier transform processing unit for Fourier transforming the RR interval time series data, and high frequency component / low frequency for separating the high frequency component and low frequency component of the Fourier transformed signal Component detection processing unit, and analysis result data output processing unit for outputting analysis result data for displaying on the display the activity of the parasympathetic nerve corresponding to the high frequency component and the activity of the sympathetic nerve corresponding to the low frequency component And comprising.

  An autonomic nervous function evaluation apparatus program according to the present invention is configured to capture, on a computer of an autonomic nervous function evaluation apparatus, imaging processing for imaging skin brightness data that changes according to blood flow with a camera, and RR interval time-series data from the brightness data. RR interval time-series data detection processing to detect, Fourier transform processing to Fourier transform the RR interval time-series data, detection processing to separate the high-frequency component and low-frequency component of the Fourier-transformed signal, An analysis result data output process for outputting analysis result data for displaying a parasympathetic activity corresponding to a high-frequency component and a sympathetic nerve activity corresponding to the low-frequency component on a display of the autonomic nerve function evaluation device; Let it run.

  The autonomic nerve function evaluation system of the present invention includes an autonomic nerve function evaluation terminal device and an autonomic nerve function evaluation server, and the autonomic nerve function evaluation terminal device captures the brightness of the skin that changes according to blood flow. A luminance data output processing unit to the server that outputs luminance data corresponding to the luminance of the skin to the autonomic nerve function evaluation server, and analysis data from the server that inputs analysis result data from the autonomic nerve function evaluation server A luminance data input processing unit for inputting the luminance data from the autonomic nerve function evaluation terminal; and an input processing unit, and a display that displays an image according to the analysis result data. An RR interval time series data detection processing unit for detecting RR interval time series data from the luminance data, and the RR interval time series data. A Fourier transform processing unit for Fourier transforming, a high frequency component / low frequency component detection processing unit for separating a high frequency component and a low frequency component of the signal subjected to the Fourier transform processing, and parasympathetic activity corresponding to the high frequency component And an analysis result data output processing unit for outputting the analysis result data for displaying the sympathetic nerve activity corresponding to the low frequency component on the display.

  The autonomic nerve function evaluation server of the present invention is an autonomic nerve function evaluation server that operates in cooperation with the autonomic nerve function evaluation terminal device in the autonomic nerve function evaluation system, from luminance data obtained by the autonomic nerve function evaluation terminal device RR interval time series data detection processing unit for detecting RR interval time series data, a Fourier transform processing unit for Fourier transforming the RR interval time series data, a high frequency component and a low frequency component of the Fourier transformed signal, A high-frequency component / low-frequency component detection processing unit, and parasympathetic activity corresponding to the high-frequency component and sympathetic activity corresponding to the low-frequency component are displayed on the display of the autonomic nerve function evaluation terminal device And an analysis result data output processing unit for outputting the analysis result data for the purpose.

  The autonomic nerve function evaluation server program of the present invention is a luminance data obtained by the autonomic nerve function evaluation terminal device in the computer of the autonomic nerve function evaluation server that operates in cooperation with the autonomic nerve function evaluation terminal device in the autonomic nerve function evaluation system. RR interval time-series data detecting process for detecting RR interval time-series data, Fourier transform process for Fourier-transforming the RR interval time-series data, and high-frequency component and low-frequency component of the Fourier-transformed signal The high-frequency component / low-frequency component detection processing to be separated, the parasympathetic nerve activity corresponding to the high-frequency component, and the sympathetic nerve activity corresponding to the low-frequency component are displayed on the display of the autonomic nerve function evaluation terminal device And an analysis result data output process for outputting the analysis result data for execution.

  An autonomic nerve function evaluation terminal device of the present invention is an autonomic nerve function evaluation terminal device that operates in cooperation with an autonomic nerve function evaluation server in an autonomic nerve function evaluation system, and shoots the brightness of skin that changes according to blood flow. A luminance data output processing unit to a server that outputs luminance data corresponding to the luminance of the skin to the autonomic nerve function evaluation server, and a server that inputs analysis result data from the autonomic nerve function evaluation server And an analysis data input processing unit, and a display for displaying an image corresponding to the analysis result data.

  The autonomic nerve function evaluation terminal device program of the present invention provides the computer with the autonomic nerve function evaluation terminal device that operates in cooperation with the autonomic nerve function evaluation server in the autonomic nerve function evaluation system, and changes the brightness of the skin according to the blood flow. Imaging processing for imaging with a camera, luminance data output processing to the server for outputting luminance data corresponding to the luminance of the skin to the autonomic nerve function evaluation server, and analysis result data from the autonomic nerve function evaluation server Data input processing to be input and analysis result image display processing for displaying an image corresponding to the analysis result data on the display of the autonomic nervous function evaluation terminal device are executed.

According to the technology of the present invention, an autonomic nerve function evaluation device and an autonomic nerve function evaluation terminal device that can be carried at all times are provided, and an autonomic nerve function evaluation server that operates in cooperation with the autonomic nerve function evaluation terminal device is provided. In addition, a program that operates in such an apparatus can be provided. The subject can manage his / her mental health by using such a device or a device and a program in daily life.

It is a figure which shows the autonomic nerve function evaluation system of embodiment. It is a flowchart of the process performed in a smart phone. It is a screen displayed on the display of a smart phone in pulse wave measurement mode start processing or luminance data detection processing. It is a screen which displays analysis result data on the display of a smart phone. It is a flowchart of the process performed in a server. It is a figure which shows an advance logic process typically. It is a figure which shows the brightness | luminance data after completion | finish of the process of each step in a server. It is a figure which contrasts the RR interval time series data obtained from a heart rate sensor, and the RR interval time series data which does not perform the low-pass filtering process obtained from a smart phone. It is a figure which shows how RR interval time series data from a smart phone changes when a smoothing parameter (spar) is changed. It is a figure which shows the two-dimensional display displayed on the display of a smart phone based on the analysis result data which the inside of a broken line generate | occur | produces from a server. It is a figure which shows distribution of the autonomic nerve activity parameter | index LnLF of the 60 second measurement according to the age of the male obtained from a heart rate sensor. It is a figure which shows distribution of the autonomic nerve activity parameter | index LnHF of the 60 second measurement according to the age of the male obtained from a heart rate sensor. It is a figure which shows distribution of the autonomic nerve activity parameter | index LnLF of the 60 second measurement according to the age of the woman obtained from a heart rate sensor. It is a figure which shows distribution of the autonomic nervous activity parameter | index LnHF of the 60 second measurement according to the age of the woman obtained from a heart rate sensor. It is a flowchart which shows the process in the autonomic nerve function evaluation apparatus of embodiment. It is a figure which shows background art.

  The autonomic nervous function evaluation apparatus according to the embodiment detects a RR interval time-series data from luminance data, a camera that captures luminance of the skin that changes in accordance with blood flow, obtains luminance data, a display that displays an image, and the luminance data. Detection processing unit for interval time series data (R-R interval time series data), Fourier transform processing unit for Fourier transforming RR interval time series data, and separating high frequency components and low frequency components of the Fourier transformed signal High-frequency component / low-frequency component detection processing unit and output of analysis result data for displaying parasympathetic nerve activity corresponding to high-frequency component and sympathetic nerve activity corresponding to low-frequency component on the display A processing unit.

  The autonomic nervous function evaluation apparatus according to the embodiment may include a low-pass filtering processing unit for removing noise included in the RR interval time-series data.

  The autonomic nervous function evaluation apparatus according to the embodiment may include a Savitzky-Golay filter (Sabitki-Golay filter) for removing noise included in luminance data and performing peak detection.

  The autonomic nervous function evaluation apparatus according to the embodiment may include a functionalization processing unit for performing continuous interpolation of luminance data.

  The autonomic nervous function evaluation apparatus according to the embodiment may include a forward logic processing unit that excludes a predetermined number of times of luminance data from a processing target when luminance data values are continuously the same.

  The autonomic nervous function evaluation apparatus of the embodiment may display a waveform of luminance data on a display when detecting luminance data.

  The autonomic nervous function evaluation apparatus according to the embodiment may display a finger image on the display and display a waveform of the luminance data before detecting the luminance data.

  The autonomic nervous function evaluation apparatus of the embodiment obtains an instantaneous heart rate that is inversely proportional to the value of the RR interval time series data, and the difference between the current instantaneous heart rate and the immediately preceding instantaneous heart rate is greater than the first predetermined value. In some cases, an abnormal value removal processing unit for RR interval time series data that removes the current RR interval time series data from the processing target may be provided.

  The autonomic nervous function evaluation apparatus of the embodiment obtains an instantaneous heart rate that is inversely proportional to the value of the RR interval time-series data, and the difference between the current instantaneous heart rate and the moving average value of the instantaneous heart rate until immediately before is second. When larger than a predetermined value, you may provide the abnormal value removal process part of the RR interval time series data which removes the present RR interval time series data from the object of processing.

  The autonomic nervous function evaluation apparatus program according to the embodiment detects, on the computer of the autonomic nervous function evaluation apparatus, photographing processing for capturing skin brightness data that changes according to blood flow by a camera, and RR interval time-series data from the brightness data. RR interval time-series data detection processing, Fourier transform processing for Fourier transforming RR interval time-series data, detection processing for separating high-frequency components and low-frequency components of Fourier-transformed signals, and high-frequency components An analysis result data output process for outputting analysis result data for displaying the activity of the parasympathetic nerve and the activity of the sympathetic nerve corresponding to the low-frequency component on the display of the autonomic nerve function evaluation apparatus.

  The autonomic nerve function evaluation system of the embodiment includes an autonomic nerve function evaluation terminal device and an autonomic nerve function evaluation server, and the autonomic nerve function evaluation terminal device has a camera that captures the brightness of the skin that changes according to blood flow, , Brightness data output processing unit to the server that outputs brightness data according to the skin brightness to the autonomic nervous function evaluation server, and analysis data input processing unit from the server that inputs analysis result data from the autonomic function evaluation server A display that displays an image according to the analysis result data, the autonomic nervous function evaluation server includes a luminance data input processing unit that inputs luminance data from the autonomic nervous function evaluation terminal, and an RR interval from the luminance data. RR interval time series data detection processing unit for detecting time series data, and Fourier transform processing for Fourier transform of RR interval time series data A high-frequency component / low-frequency component detection processing unit that separates a high-frequency component and a low-frequency component of a signal subjected to Fourier transform processing, and parasympathetic activity corresponding to the high-frequency component and sympathetic nerve corresponding to the low-frequency component And an analysis result data output processing unit for outputting analysis result data for displaying the activity on the display.

  The autonomic nerve function evaluation server of the embodiment is an autonomic nerve function evaluation server that operates in cooperation with the autonomic nerve function evaluation terminal device in the autonomic nerve function evaluation system, and RR is obtained from luminance data obtained by the autonomic nerve function evaluation terminal device. RR interval time series data detection processing unit for detecting interval time series data, Fourier transform processing unit for Fourier transform of RR interval time series data, and a high frequency component and a low frequency component of the Fourier transformed signal are separated. Analysis result data for displaying high-frequency / low-frequency component detection processing unit and parasympathetic nerve activity corresponding to the high-frequency component and sympathetic nerve activity corresponding to the low-frequency component on the display of the autonomic nerve function evaluation terminal device And an analysis result data output processing unit for outputting.

  The autonomic nerve function evaluation server program according to the embodiment is obtained from luminance data obtained by the autonomic nerve function evaluation terminal device in the computer of the autonomic nerve function evaluation server that operates in cooperation with the autonomic nerve function evaluation terminal device in the autonomic nerve function evaluation system. RR interval time series data detection processing for detecting RR interval time series data, Fourier transform processing for Fourier transform of RR interval time series data, and separating high frequency components and low frequency components of the Fourier transformed signals, Analytical result data for displaying high-frequency / low-frequency component detection processing and parasympathetic nerve activity corresponding to the high-frequency component and sympathetic nerve activity corresponding to the low-frequency component on the display of the autonomic nerve function evaluation terminal device The analysis result data output process to be output is executed.

  The autonomic nerve function evaluation terminal device of the embodiment is an autonomic nerve function evaluation terminal device that operates in cooperation with the autonomic nerve function evaluation server in the autonomic nerve function evaluation system, and captures the brightness of the skin that changes according to blood flow. Analysis data from the camera, the luminance data output processing unit to the server that outputs luminance data corresponding to the luminance of the skin to the autonomic nervous function evaluation server, and the server that inputs the analysis result data from the autonomic nervous function evaluation server An input processing unit, and a display for displaying an image corresponding to the analysis result data.

  The autonomic nerve function evaluation terminal device program according to the embodiment provides the brightness of the skin that changes according to blood flow to the computer of the autonomic nerve function evaluation terminal device that operates in cooperation with the autonomic nerve function evaluation server in the autonomic nerve function evaluation system. Data to be input for analysis processing data from the autonomic nerve function evaluation server, shooting data to be shot with the camera, luminance data output processing to the server that outputs luminance data according to the skin brightness to the autonomic nerve function evaluation server An input process and an analysis result image display process for displaying an image corresponding to the analysis result data on the display of the autonomic nervous function evaluation terminal device are executed.

“First Embodiment”
(Outline of the autonomic nervous function evaluation system)

  FIG. 1 is a diagram illustrating an autonomic nervous function evaluation system according to an embodiment.

  The autonomic nerve function evaluation system 1 of the embodiment constitutes a client-server system, and a smartphone (high-function mobile phone) 10 and an autonomic nerve function evaluation server as an embodiment of an autonomic nerve function evaluation terminal device As a server 20. The smartphone 10 and the server 20 are connected via the Internet line 30 and operate in cooperation.

  The smartphone 10 is a kind of mobile phone made based on a personal computer having high affinity with the Internet, and is widely used all over the world. For example, the smartphone 10 uses Android (registered trademark). Android is a software collective package composed of an open source operating system based on Linux (registered trademark), middleware, and various applications. Android runs on an MPU (microprocessor).

  The smartphone 10 generally includes a display function, a voice transmission function, a voice input function, a camera function, an Internet connection function, and a GPS (Global Positioning System) function in addition to the telephone function. As in a general personal computer, peripheral devices such as a ram, a ROM, and an interface device are connected by a bus line with the MPU as a center. An android is executed in an MPU (microprocessor) and controls peripheral devices to perform the desired functions described above. The smartphone 10 is one embodiment of an autonomic nerve function evaluation terminal device. As another embodiment, the tablet (multifunctional mobile terminal) having the various functions described above, an autonomic nerve function evaluation terminal specialized for use in this embodiment. It may be a device.

  The server 20 is always connected to the Internet line 30 and the installed server software (autonomic nerve function evaluation server program) is running, so that the server software can be executed immediately in response to a request from the client. Computer. The server 20 includes a high-speed, high-performance CPU (Central Processing Unit), a large-capacity ram, a large-capacity rom, and the like, and can perform complicated processing at a high speed. Therefore, even when a large number of clients access the server 20 via the Internet line 30, requests from the clients are received at high speed, information is processed instantaneously, and the result is sent via the Internet line 30. To the client.

  The smartphone 10 is connected to the Internet line 30 using radio. The server 20 is connected to the Internet line 30 using a high-speed line.

  Luminance data is transmitted from the smartphone 10 to the server 20, and after the arithmetic processing in the server 20, analysis result data is transmitted from the server 20 to the smartphone 10. Thus, in the embodiment, the smartphone 10 and the server 20 share processing. The reason is that there is a limit to the allowable size and processing speed of applications that can be installed on the smartphone 10, so when processing all the processing inside the smartphone 10, it takes a long time to obtain analysis result data. is there. Such a problem can be solved by adopting a client-server system. In addition, a large amount of data obtained from the client can be accumulated in the server and used as a common asset of the client.

"Processing of autonomic nervous function evaluation on smartphones"
(About luminance data detection in smartphones)
FIG. 2 is a flowchart of processing executed by the MPU in the smartphone 10.

  Software (autonomous nerve function evaluation terminal device program) for performing the processing of the flowchart of FIG. 2 is installed in the smartphone 10 as one of the applications. The installation may be performed by connecting a personal computer (not shown) and the smartphone 10 with a cable, or may be performed by downloading from the download site to the smartphone 10 via the Internet line 30.

  Prior to the description of FIG. 2, a role of the camera function of the smartphone 10 in the embodiment will be briefly described.

  The camera has a condenser lens and a CMOS image sensor (Seamos image sensor), both not shown. A camera provided in the smartphone 10 is used for photographing a landscape, a person, and the like in the same manner as a normal camera. In the embodiment, the camera is operated as if it is a part of the electrocardiograph described in Patent Document 1 and the electrocardiogram monitor described in Patent Document 2.

  For example, if the camera is brought close to the index finger of the hand, a change in the brightness of the skin according to the blood flow flowing in the blood vessel under the skin can be photographed. The blood flow that flows through the blood vessels is synchronized with the movement of the heart, and the change in skin brightness is also synchronized with the movement of the heart. Therefore, luminance data, which is information from a CMOS image sensor in which changes in luminance of the skin imaged by the camera are converted into electric signals, includes information on the movement of the heart in the same manner as electric signals in electrocardiographs and electrocardiogram monitors. .

  The signal obtained by detecting the change in luminance of the fingertip skin is a signal for indirectly detecting a pulse wave that reflects the motion of the heart. Therefore, in order to detect pulse wave data (heart rate data) with good quality from the luminance change obtained by averaging the outputs from all the pixels of the CMOS image sensor or the luminance change obtained from the R pixel, Filtering process is required. Therefore, the smartphone 10 transmits luminance data in order to perform high-level filtering processing that requires complicated and high speed in the server 20.

  This will be described below with reference to FIG. FIG. 2A is a flowchart of each process centering on the luminance data detection process executed in the smartphone 10.

The MPU performs a pulse wave measurement mode start process (step ST10).
The “pulse wave measurement mode” which is one of the operation modes is selected by the menu selection button of the smartphone 10. Thereby, the MPU of the smartphone 10 activates an application for pulse wave measurement, and performs the pulse wave measurement mode start process in step ST10. Details of the processing in step ST10 will be described later with reference to FIG. Note that switching the operation mode by the menu selection button of the smartphone 10 is a well-known technique.

The MPU performs measurement start notification processing (step ST11).
In the measurement start notification process, the MPU may operate the screen display function of the smartphone 10 to display an image “currently being measured ...” on the display (see FIG. 3G). The MPU may operate the voice guidance function of the smartphone 10 and send a voice “currently being measured” through a speaker.

The MPU controls the hardware and performs a photographing process of the image of the finger of the hand close to the camera (step ST12).
The MPU continues to shoot an image of the finger of the hand from the CMOS image sensor as a moving image, for example, for 60 seconds. It is also possible to improve the measurement accuracy by performing a rolling process after measuring for 60 seconds or more, for example, 70 seconds.

The MPU controls the hardware to perform luminance data detection processing from the captured image (step ST13).
The MPU acquires signals from all cells of the CMOS image sensor imaged in step ST12 in the YUV format, and calculates the luminance average from the sum of the luminance components (Y) of the image. The result of calculating the luminance average is used as luminance data, and the amplitude of the luminance data is sequentially stored in the memory. The luminance data is discrete time data obtained for each time corresponding to one frame, and corresponds to the ECG (Electrocardiogram) described in FIGS. 16A and 16B.

  In the smartphone 10, the processes in steps ST11 to ST13 proceed simultaneously by the MPU controlling the hardware. However, the process in step ST11 does not necessarily have to be performed at the same time. For example, the MPU may operate the voice guidance function and send a voice “currently being measured” by the speaker once or several times.

The MPU performs measurement end notification processing (step ST14).
In the end notification process, the MPU may operate the screen display function of the smartphone 10 to display an image message “Pulse wave measurement has been completed. The voice guidance function may be operated to send out a voice message “Pulse wave measurement has been completed. Thank you for your work.” As shown in Fig. 3 (g), It is also possible to simply delete the image display "".

The MPU performs luminance data output processing to the server (step ST15).
The MPU sends the URL (Uniform Resource Locator) of the server 20 and connects the server 20 and the smartphone 10 via the Internet line 30. When the connection is completed, the MPU transmits a unique identification code and luminance data allocated to each smartphone 10 (terminal) to the server 20.

The MPU performs luminance data transmission completion processing (step ST16).
The MPU disconnects the smartphone 10 from the Internet line 30 and automatically returns the operation mode of the smartphone 10 from the “pulse wave measurement mode” to the previous mode.

(Image displayed on the display in the luminance data detection process)
A touch sensor is stacked on the display of the smartphone 10, and a software button can be formed by an image displayed on the display and the touch sensor. The software button is a button formed on the screen of the display, and the function represented by the software button can be activated by touching. Hereinafter, the software button is simply referred to as a button.

  FIG. 3 is a screen displayed on the display of the smartphone 10 in the pulse wave measurement mode start process or the luminance data detection process.

(Display screen in the process of step ST10)
The display screen in the process of step ST10 will be described with reference to FIGS.

<Display consent screen>
The process proceeds to step ST10, and the “agreement screen” shown in FIG. If the “agree” button on the “consent screen” is pressed, a “registration screen” shown in FIG. 3B is displayed and the process continues. If the “agree” button is not pressed even after a predetermined time has elapsed, the process is automatically terminated, and the “pulse wave measurement mode” is automatically returned to the previous mode.

<Display registration screen>
Press "Men" or "Women" on the "Registration screen" shown in Fig. 3 (b) and enter "Gender", and enter "Birth date", "Occupation", "Prefecture" If each field is entered and the “register” button is pressed, a “registration completion screen” shown in FIG. 3C is displayed and the process continues. If the “Register” button is not pressed even after a predetermined time has elapsed, the process is automatically terminated, and the “pulse wave measurement mode” is automatically returned to the previous mode.

<Display registration completion screen>
If the “start” button on the “registration completion screen” shown in FIG. 3C is pressed, the “measurement / batch action ranking screen” shown in FIG. 3D is displayed and the processing continues. If the “Start” button is not pressed even after a predetermined time has elapsed, the process is automatically terminated and the “pulse wave measurement mode” is automatically returned to the previous mode.

  The images shown in FIG. 3A, FIG. 3B, and FIG. 3C described above are images that are displayed only at the first registration. When the “pulse wave measurement mode” is executed for the second time and thereafter, the screens of FIG. 3A, FIG. 3B, and FIG. 3C are not displayed, and after the “pulse wave measurement mode” starts, Immediately, the screen of FIG. 3D is displayed.

<Indication of selection of “Measurement” and “Batch Ranking”>
If the “Measure” button on the “Measure / Batch action ranking screen” shown in FIG. 3D is pressed, the question screen shown in FIG. 3E is displayed and the process continues. On the other hand, if the “batch action ranking” button on the “measurement / batch action ranking screen” shown in FIG. 3D is pressed, the processing of the “pulse wave measurement mode” is completed, as shown in FIG. 2B described later. Moving to the analysis result data display process, the image shown in FIG. 4B is displayed. If either the “measurement” or “batch action ranking” button is not pressed even after a lapse of a predetermined time, the processing is automatically terminated and the mode is automatically returned from the “pulse wave measurement mode” to the previous mode. The contents of the “batch action ranking” will be described later. Various settings can be made by pressing the setting help button.

<Display question screen>
If the test subject answers “What are you doing now?” In Q1 and “What are you feeling now” in Q2 of “Question screen” shown in FIG. 3 (e) and press “Next”, FIG. 3 (f) The “Measurement start preparation screen” is displayed and the process continues. By clicking the down arrow next to “Deskwork” that is the answer to Q1, the subject can select the content of the answer to Q1, and click the down arrow next to “Rose” that is the answer to Q2. The subject can select the content of the answer to Q2. If the “next” button is not pressed even after a predetermined time has elapsed, the processing is automatically terminated and the mode is automatically returned from the “pulse wave measurement mode” to the previous mode.

  Answers to Q1's “What are you doing now” include, for example, “desk work”, “meeting / customer service”, “other household chores”, “care for children”, “commuting / moving”, “study”, “ “Sleep / Nap”, “Hobby”, “Bathing”, “Commuter”, “Shopping”, “Drinking party”, etc. are prepared in advance. As answers to Q2's “How do you feel now?”, “Stress”, “Loose”, “Leisurely”, and “Ideal” are prepared in advance. “Stress” may be replaced with various other words having the same meaning, for example, “tension”, “spacious”, for example, “fatigue”, “relaxing”, for example, “relaxed”, “ideal”, for example, “ideal”, etc. .

<Display of measurement start preparation screen>
At the top of the “Measurement start preparation screen” shown in Fig. 3 (f), “Place your finger on the camera. Do not move your finger during measurement. Please press. Is displayed. A schematic diagram displays how to place a finger on the camera on the left side of the second stage of the “measurement start preparation screen”. An image of a finger detected by the camera is displayed on the right side of the second stage of the “measurement start preparation screen”. Luminance data detected by the camera is displayed in the third row of the “measurement start preparation screen”.

  The subject determines the position of the finger with respect to the camera while viewing the image of the finger on the right side of the second stage of the “measurement start preparation screen” shown in FIG. It is possible to start detection of stable luminance data with good accuracy by adjusting. The image of the finger is displayed on the right side of the second stage of the “Measurement Preparation Screen”, and the luminance data is displayed on the third stage of the “Measurement Preparation Screen”. In order to obtain good luminance data.

(Display screen in the process of step ST11)
After confirming that the finger is properly in contact with the camera, the subject presses the “measurement start” button displayed in the fourth row of the “measurement start preparation screen” shown in FIG. As a result, the process moves to “measurement start notification process” (step ST12), and a “measuring screen” shown in FIG. 3G is displayed. The tip of the bar graph and the heart mark displayed in the middle of FIG. 3G are displayed at the leftmost position immediately after the “measurement start” button is pressed. If the “measurement start” button is not pressed even after a predetermined time has elapsed, the process automatically ends and automatically returns from the “pulse wave measurement mode” to the previous mode.

(Display screen in step ST12 and step ST13)
<Display during measurement screen>
The “measuring screen” shown in FIG. 3G is displayed while the process of step ST12 and the process of step ST13 are continued. “Currently measuring ...” is displayed at the top of the display screen. The bar graph and heart symbol in the middle of the display screen move from the left side to the right side, indicating how much progress has been made in the total scheduled measurement time. In addition, luminance data is displayed in the lower part of the display screen so that it can be confirmed that the amplitude of the waveform is stable, that is, that the finger is properly in contact with the camera.

(Display screen in the process of step ST14)
When the luminance data for the scheduled time is acquired, the process of step ST13 ends. The screen during measurement shown in FIG. 3 (g) disappears, and the screen corresponding to the mode immediately before shifting to the “pulse wave measurement mode” (screen is not shown) or the default screen (screen is not shown) is automatically displayed. Changes.

(About display of analysis result data on smartphone)
Although processing in the server 20 will be described later, the server 20 finally sends analysis result data to the smartphone 10. With reference to FIG.2 (b), the process which the smart phone which received analysis result data performs is demonstrated. FIG. 2B is a flowchart of the analysis result data display process executed in the smartphone 10.

The MPU performs input processing of analysis result data from the server 20 (step ST17).
In response to the connection request from the server 20, the smartphone 10 receives the analysis result data.

The MPU performs image display processing of the analysis result data (step ST18).
The MPU performs control to display an image corresponding to the analysis result data from the server 20 on the display of the smartphone 10. The contents of the display image will be described later.

The MPU performs an analysis result data display end process (step ST19).
The MPU ends the image display of the analysis result data by operating the touch panel stacked on the display or when a predetermined time elapses.

<Display screen of analysis result data on smartphone>
FIG. 4 is a screen for displaying the analysis result data on the display of the smartphone.

<Display of measurement results>
FIG. 4A is a screen displayed on the display of the smartphone 10 based on the analysis result data transmitted by the server 20. In the uppermost part of the screen, areas classified into four areas of “ideal zone”, “stress”, “slow” and “slow” are displayed, and the position of the heart indicates the state of the subject.

  There is an area for displaying “your feeling” on the right side of the second row of the screen of FIG. 4A, and the answer of the subject who answered “What is your feeling?” In Q2 in FIG. 3E is displayed. . When the analysis result data agrees with the test subject's answer, a “target” is displayed.

  In the third row of the screen in Fig. 4 (a), "You seem to feel" stress "now. Looks like you think so too. XX is good for stress countermeasures. Is displayed. The ○○ portion is appropriately described, for example, “sports”.

  There are two buttons, “Exit” and “Share”, on the 4th row of the screen in FIG. 4 (a). When “Share” is pressed, Gmail (group email), Facebook (Facebook), etc. screens are activated. Thus, the measurement result image can be transmitted. When “END” is pressed, the display in FIG. 4A disappears after the above sharing process is completed. When “end” is pressed without pressing “share”, the display in FIG. 4A disappears without performing the above-described sharing process.

<Display of batch action ranking>
FIG. 4B shows how well the subject's own feelings (subjective evaluation) and the analysis result data (objective evaluation) have matched. In FIG. 4 (b), the hit rates, which are the degree of coincidence between (subjective evaluation) and (objective evaluation), are “stress”, “slow”, “slow”, “ideal” (“ideal zone”). ). For example, the hit rate of “stress” is 87%, and “desk work”, “meeting / customer service” are ordered in descending order of actions that are objectively evaluated from analysis result data as “stress”. , Ranked as “Other Housework”. As shown in FIG. 4B, the hit rate of “slow” is 55%, the hit rate of “relaxing” is 40%, and the hit rate of “ideal” is 80%. As for “Right”, “Leisurely”, and “Ideal” behavior rankings, the top three are ranked in order of ranking, as with “Stress”. Then, after the predetermined time has elapsed, the screen display shown in FIG. 4B is automatically deleted, and the display corresponding to the mode before displaying the analysis result data or the default mode is displayed. The

  Explaining the hit rate with "stress" as an example. Of the numbers that the server 20 objectively determines as “stress”, the ratio of the number that the subject subjectively determines as “stress” is defined as the hit rate of “stress”. The hit rate is a number of 100% or less. The arithmetic expression is expressed as follows.

  “Stress” hit rate (%) = 100 × (number of times subject entered “stress” and subjectivity) / (number of times server 20 determined “stress”)

  It is possible to calculate the hit rate of “slow”, the hit rate of “ideal zone”, and the hit rate of “slow” in the same way.

  In the above-described embodiment, the smartphone 10 and the server 20 are connected, and after the luminance data is transmitted from the smartphone 10 to the server 20, the connection between the smartphone 10 and the server 20 is disconnected. Then, when transmitting the analysis result data from the server 20 to the smartphone 10, the server 20 and the smartphone 10 are connected again, and after the analysis result data is transmitted from the server 20 to the smartphone 10, the server 20 and the smartphone 10 are connected. Was disconnected. However, the connection between the smartphone 10 and the server 20 may be maintained until the smartphone 10 transmits luminance data to the server 20 and transmits analysis result data from the server 20 to the smartphone 10.

  The connection between the smartphone 10 and the server 20 is from “pulse wave measurement mode start processing” (step ST10) in FIG. 2A to “analysis result data display processing” (step ST19) in FIG. 2B. In the case of maintaining, the flowchart shown in FIG. 2 is read as follows. The “pulse wave measurement mode end process” in step ST16 is not necessary. Then, subsequent to “brightness data output processing to server” in step ST15, “analysis result data input processing from server” in step ST17 is executed. Also, “analysis result data display processing end” in step ST19 is read as “pulse wave measurement mode end processing”. Furthermore, during the period from the end of “brightness data output processing to server” in step ST15 to the time before “analysis result data input processing from server” in step ST17, the image display surface of the display is “under analysis”. "Is displayed.

“Processing of autonomic nervous function evaluation on servers”
(About the overall process flow on the server)
FIG. 5 is a flowchart of processing executed by the CPU in the server 20.

  This will be described below with reference to FIG. Details of each process will be described later.

The CPU of the server 20 performs luminance data input processing from the terminal device (step ST20).
In the luminance data input process, the identification code and luminance data of the smartphone sent from the client and the gender, date of birth, occupation, and prefecture data input from the screen shown in FIG. Is stored in the process. After the process of step ST20, the processes from step ST21 to step ST30 are sequentially performed.

The CPU performs logic processing ahead (step ST21).
Advance logic processing is processing that makes it easier to detect the peak of the luminance data signal. When the luminance data values are continuously the same, the continuous luminance of the same value more than a predetermined predetermined number of times. This is a process of removing data from the processing target.

The CPU performs functionalization processing of luminance data (step ST22).
The luminance data is discrete time sampling data, and the sampling time is synchronized with the time in the smartphone 10. In the functionalization process, in order to synchronize with the time in the server 20, the luminance data at the time when the luminance data detected by the smartphone 10 does not exist is converted into a function and continuously interpolated to enable resampling. By enabling resampling, the Fourier transform process in step ST27 can be executed with high accuracy.

The CPU performs noise removal and peak detection processing using a Savitzky-Golay filter (step ST23).
The Savitzky-Golay filter itself is a known filtering technique. The waveform of the luminance data signal is less accurate than the pulse wave waveform detected by ECG, and it is difficult to reproduce the pulse wave peak detected by ECG by simple filtering. Here, when a simple differentiation is used when reproducing the peak of the pulse wave, high-frequency noise increases and the purpose cannot be achieved. Therefore, the Savitzky-Golay filter is used as a filter that clearly shows a peak without increasing noise.

The CPU performs a process of detecting RR interval time series data (step ST24).
The detection processing of RR interval time series data (RFR interval time series data) is processing for converting a luminance data signal corresponding to an ECG signal (ECG signal) into RR interval time series data. The RR interval time-series data detection process includes a process of detecting a peak of a pulse wave included in the luminance data signal clearly reproduced by the Savitzky-Golay filter in step ST23, and an adjacent detected by the differential operation. And a process for detecting a time between two peaks.

The CPU performs a low-pass filtering process (step ST25).
The low-pass filtering process is a process of converting the RR interval time series data that would be originally obtained if the luminance data accurately detected the pulse wave after detecting the RR interval time series data from the luminance data.

The CPU performs an abnormal value removal process of the RR interval time series data (step ST26).
The abnormal value removal process of the RR interval time series data is a process of removing the abnormal value included in the RR interval time series data. Here, the abnormal value is a value of RR interval time series data that would not normally be detected. For detecting the abnormal value, an instantaneous heart rate (HR) obtained by dividing 60 by the value of the RR interval time series data and inversely proportional to the value of the RR interval time series data is used. When an abnormal value is detected, it is assumed that noise mixed in the RR interval time-series data (that is, noise mixed in the luminance data signal) can be generated, and the RR interval indicating the abnormal value in the processing after step ST27. Do not use time series data. It may be an abnormal value when any one of the following first to third rules is established, and abnormal when any two or more combinations of the first to third rules are established. It may be a value.

<First rule>
n is a positive integer, and t (n) is RR interval time-series data detected at the nth time. If 60 / t (n) -60 / t (n-1) | ≦ the first predetermined value, it is abnormal Be a value. That is, if the difference between the current instantaneous heart rate and the immediately preceding instantaneous heart rate is greater than the first predetermined value, the RR interval time series data at that time is removed.

<Second rule>
(MeanH.R (N)) is a moving average value, and N is a positive integer (for example, 5, 8), and when | 60 / t-meanH.R (N) | ≧ second predetermined value range Is an abnormal value. That is, when the difference between the current instantaneous heart rate and the moving average value of the instantaneous heart rate until immediately before is larger than the second predetermined value, the RR interval time series data at that time is removed.

<Third rule>
As a result of applying the second rule, if it is an abnormal value continuously for a predetermined number of times (for example, 8 times), meanH.R (8) is reset and meanH.R (8) is calculated again. . That is, when applying the second rule, when the difference between the current instantaneous heart rate and the moving average value is larger than the second predetermined value for a predetermined number of times, the third rule sets the moving average value. A new moving average value is obtained after resetting.

The CPU performs a Fourier transform process (step ST27).
Using a known Hamming window, the RR interval time series data after the abnormal value removal processing obtained in step ST26 is subjected to Fourier transformation processing by known fast Fourier transformation. Since the luminance data has been functionalized in step ST22, the fast Fourier transform can be resampled at a timing different from the data sampling timing in the smartphone 10. Further, since the RR interval time-series data after the abnormal value removal process is Fourier-transformed, it is possible to obtain a Fourier-transformed power spectrum that would be originally obtained when noise is not mixed in the luminance data signal.

The CPU performs detection processing of the high frequency component HF and the low frequency component LF (step ST28).
After Fourier transform, the high frequency component HF and the low frequency component LF are detected. The high frequency component HF is an integral value of a high frequency power spectrum in a predetermined frequency range obtained by Fourier transform. The low frequency component LF is an integral value of a low frequency power spectrum in a predetermined frequency range obtained by Fourier transform.

The CPU performs the plotting process after adjusting the sex / age (step ST29).
In short, the plot processing is generation of analysis result data to be transmitted to the smartphone 10.
The high frequency component HF and the low frequency component LF values obtained in step ST28 are values specific to the subject at the time of measurement. The high-frequency component HF value and the low-frequency component LF value are different values depending on gender and age. In advance, the high frequency component HF value and the low frequency component LF value are stored in the server 20 as reference data for each identity and the same age for many subjects with different sex / age.
The high-frequency component HF value and the low-frequency component LF value of the subject of the smartphone 10 are compared with the reference data, and analysis result data for two-dimensionally displaying the subject's state is generated.
In the two-dimensional display, in order to approximate the spectrum to a normal distribution, LnHF that logarithmically displays the high frequency component HF also used in known papers, and LnLF that logarithmically displays the low frequency component LF are used.
Hereinafter, LnHF is referred to as an autonomic nerve activity index LnHF, and LnLF is referred to as an autonomic nerve activity index LnLF.
In the two-dimensional display, orthogonal coordinates are used, the first coordinate axis (X axis) represents the level of the autonomic nervous activity index LnHF, and the second coordinate axis (Y axis) represents the level of the autonomic nervous activity index LnLF. The average value of the distribution of the autonomic nervous activity index LnHF in the reference data is set as the origin of the first coordinate axis, and the average value of the distribution of the autonomic nervous activity index LnLF is set as the origin of the second coordinate axis orthogonal to the first coordinate axis.
The intersection of the first coordinate axis and the second coordinate axis is set as the origin. The first and second coordinate axes are divided into four quadrants. The first quadrant is an “ideal zone”, the second quadrant is “stress”, the third quadrant is “slow”, and the fourth quadrant is “slow”.
In the two-dimensional display, the region corresponding to the autonomic nerve activity index LnHF and the autonomic nerve activity index LnLF of the subject of the smartphone 10 is plotted.
Image data for plotting in which region the intersection of the subject's autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF is located is displayed as the analysis result data.

The CPU performs analysis result data output processing to the terminal device (step ST30).
The CPU transmits analysis result data to the smartphone 10. As described above, an image display corresponding to the analysis result data can be displayed on the display of the smartphone 10 using the application installed in the smartphone 10.

"Details of each process in the server"
Details of the processes described in steps ST21 to ST30 will be described in order.

(Advance logic processing of step ST21)
FIG. 6 is a diagram schematically illustrating the forward logic processing.

  The advance logic processing is a logic that reduces the number of luminance data to two consecutive times when the numerical value of the luminance data is the same three or more times. That is, continuous luminance data having the same value equal to or more than a predetermined predetermined number of times (for example, three times) is excluded from the processing target. Here, the same value may be given a width, and a small variation in the value within a predetermined range may be included in the target of the forward logic processing as the same value. By this process, the peak can be detected more easily in the detection process of the RR interval time series data in step ST24. FIG. 6A shows luminance data before forward logic processing. In FIG. 6A, the same data continues four times. FIG. 6B shows luminance data after the advance logic processing. In FIG. 6B, four consecutive times of the same data are processed twice.

(Functionization of luminance data in step ST22)
The luminance data is functionalized by a B-spline function that passes through all luminance data sample points. By using the B-spline function, the luminance data at the time when there is no luminance data detected by the smartphone 10 can be smoothly interpolated as a continuous function, so the processing after step ST23 is performed with the clock signal of the CPU of the server 20 as a reference. Can do. Also, luminance data for every fixed sample period (10 msec (milliseconds)) for performing FFT (Fast Fourier Transform) can be obtained as new time series data. The B-spline function itself is a known technique. As a data interpolation method, linear interpolation is also known. However, by using a B-spline function, the data interpolation accuracy is further improved as compared with linear interpolation.

(Noise removal processing and peak detection processing by Savitzky-Golay filter in step ST23)
The Savitzky-Golay filter is also referred to as a digital smoothing polynomial filter and a least square smoothing filter. It is a smoothing filter designed in the time domain and a differential filter. In the Savitzky-Golay filter of the embodiment, for example, the window is 0.9 (seconds) and approximated by a cubic polynomial to perform second order differentiation.

(Detection processing of RR interval time series data in step ST24)
After detecting the peak of the luminance data signal, RR interval time series data, which is the time between adjacent peaks, is detected. The logic of peak detection is a simple sign inversion which is a known technique. Detection of time between adjacent peaks is performed by a counter that operates in synchronization with a clock signal. Here, the peak detection is possible with simple sign reversal logic, and the RR interval time-series data can be detected because of the effect of the Savitzky-Golay filter in step ST23.

  (List of time series data by processing from step ST21 to step ST24)

  FIG. 7 is a diagram illustrating the luminance data after the processing of each step in the server 20 is completed.

  FIG. 7A is a diagram illustrating the luminance data after the luminance data input process of step ST20 is completed. FIG. 7B is a diagram illustrating the luminance data after the luminance data functionalization processing in step ST22 is completed. FIG. 7C is a diagram showing luminance data after the noise removal processing and peak detection processing by the Savitzky-Golay filter in step ST23.

* RR interval time series data (I'm sorry, my understanding was wrong.)
(Low-pass filtering process in step ST25)
Ambient light incident on the camera is a source of noise as stray light. The camera of the smartphone 10 is designed to exhibit the function of a camera that captures normal scenery and people. Therefore, the level of stray light that becomes noise with respect to a very small change in the brightness of the skin captured by the camera is relatively large, and the RR interval time series data (RRI) detected from the brightness data in step ST24 includes noise. Therefore, the RR interval time series data cannot be accurately separated unless noise is appropriately removed from the RR interval time series data obtained in step ST24.

  Therefore, an optimum filter characteristic for removing noise is obtained in advance by experiments, and a low-pass filter having such a characteristic is used in the process of step ST25. Hereinafter, how the optimum filter characteristic is obtained will be described.

  RR interval time-series data obtained after processing of step ST25 on the reference waveform using the waveform of RR interval (RRI) time-series data obtained from a commercially available heart rate sensor (MyBeat (registered trademark) of Union Tool Co., Ltd.) The characteristics of a low-pass filter that approximates the waveform of were obtained. The noise is removed from the RR interval time-series data detected from the luminance signal obtained from the camera of the smartphone 10 by the low-pass filtering process in step ST25, and the RR interval closer to the RR interval time-series data that should be detected originally. Time series data was obtained.

  8 and 9 are explanatory diagrams on how to determine the constants of the low-pass filter.

  FIG. 8 is a diagram comparing the RR interval time series data obtained from a commercially available heart rate sensor and the RR interval time series data obtained from the smartphone 10 without performing the low-pass filtering process in step ST25.

  FIG. 8A is RR interval time series data obtained from a commercially available heart rate sensor. FIG. 8B is RR interval time-series data that is obtained from the smartphone 10 and does not perform the low-pass filtering process of step ST25. FIG. 8A and FIG. 8B are data obtained simultaneously from the same person.

  The data of the commercially available heart rate sensor in FIG. 8A and the RR interval time series data before the low-pass filtering process in step ST25 from the smartphone 10 in FIG. 8B are considerably different. Therefore, an attempt was made to filter the RR interval time-series data from the smartphone 10 with a low-pass filter using a smoothing spline.

  FIG. 9 is a diagram illustrating how the RR interval time-series data after the low-pass filtering process changes when the smoothing parameter (spar) is changed.

  The smoothing parameter (spar) in FIG. 9A is 0.7. The smoothing parameter (spar) in FIG. 9B is 0.6. The smoothing parameter (spar) in FIG. 9C is 0.5. The smoothing parameter (spar) in FIG. 9D is 0.4. The smoothing parameter (spar) in FIG. 9 (e) is 0.3. The smoothing parameter (spar) in FIG. 9F is 0.2.

  The RR interval time series data after the low-pass filtering process shown in FIGS. 9A to 9F is compared with the RR interval time series data obtained from a commercially available heart rate sensor shown in FIG. A suitable result is obtained in which the smoothing parameter (spar) approximates in the range of 0.2 to 0.4. When the smoothing parameter (spar) is in the range of 0.3 to 0.4, a more favorable result is obtained in which both are more approximate. Therefore, in the present embodiment, the range of 0.3 to 0.4 as the smoothing parameter (spar) value closest to the RR interval time series data obtained from a commercially available heart rate sensor and the RR interval time series data obtained from the smartphone 10. Is used.

(Abnormal value removal processing of RR interval time series data in step ST26)
Table 1 is a table for explaining the abnormal value removal processing of the RR interval time series data. This will be described below with reference to Table 1.

  The first column of Table 1 is a number corresponding to the order in which the RR interval time series data is detected. Table 1 shows the first to the 14th time when the RR interval time series data is first detected.

  The second column of Table 1 is the value of the RR interval time series data, the value of the RR interval time series data detected for the first time is the time t1, the value of the RR interval time series data detected for the second time is The value of the RR interval time series data detected at time t2... 13th time is time t13, and the value of the RR interval time series data detected at time 14 is time t14.

  The third column of Table 1 is a value obtained by dividing 60 by the value of RR interval time series data to obtain an instantaneous heart rate (HR) that is inversely proportional to the value of RR interval time series data.

  The fourth column of Table 1 shows the result of calculating the absolute value | 60 / tn -60 / tn-1 | of the increase / decrease amount for each time of the instantaneous heart rate (HR). This is an operation corresponding to rule 1.

  The fifth column in Table 1 moves the instantaneous heart rate (HR) and the instantaneous heart rate of the predetermined heart rate (eight beats in the example of Table 1) before the instantaneous heart rate (HR). It is the absolute value of the difference from the value averaged by averaging. This is an operation corresponding to rule 2. That is, the ninth instantaneous heart rate (HR) is 60 / t9 and the moving average corresponding to the ninth time is meanH.R (8 (t1, t2, t3, t4, t5, t6, t7, t8). )) The absolute value of the difference from (the moving average from the first to the eighth) is the ninth HR difference. Similarly, the 10th instantaneous heart rate (HR) is 60 / t10 and the moving average corresponding to the 10th measurement is meanH.R (8 (t2, t3, t4, t5, t6, t7, t8, t9)) (the moving average from the second time to the ninth time) is the absolute value of the tenth HR difference. In the same manner, the calculation is performed for the eleventh and subsequent times.

  The fourth column of the 12th time in Table 1 is described as “error” because 60 / t12, which is the 12th instantaneous heart rate (HR), is an error value, and the HR difference is calculated. It means that t12 which is the RR interval time series data is not used and is not subjected to data processing.

  When removing the RR interval time-series data, the above-described three rules of the first rule, the second rule, and the third rule are applied. If any one of the first rule to the third rule is not cleared, the RR interval time-series data may be removed and not subject to data processing. The first rule to the third rule If any two or more combinations of (including all combinations of the first rule to the third rule) are not cleared, the RR interval time-series data is removed and not subjected to data processing. It is good.

  In the present embodiment, the first rule (rule 1), the second rule (rule 2), and the third rule (rule 3) are used in combination as follows.

  The first rule is used as follows. Since there is no data before the RR interval time series data for the first time, the first rule is not applied. From the second time onward, when the calculation result of | 60 / t (n) -60 / t (n-1) |, which is the difference between the instantaneous heart rate and the previous instantaneous heart rate, is less than or equal to the first predetermined value RR interval time-series data is used only for, and at the same time, in calculating meanH.R (8), which is the instantaneous heart rate (HR) of 8 beats, | 60 / t (n) -60 / t ( n-1) The instantaneous heart rate (60 / t (n)) is used only when the calculation result of | is equal to or less than the first predetermined value.

  In the first to eleventh times in Table 1, in accordance with this rule 1, processing is performed when the calculation result of | 60 / t (n) -60 / t (n-1) |

  When the calculation result of | 60 / t (n) -60 / t (n-1) | is larger than the first predetermined value, an instantaneous heart rate (HR) different from the previous one is suddenly generated due to noise or the like. (That is, it is determined that the RR interval time-series data different from the previous one is generated due to noise or the like), the RR interval time-series data is removed and is not subjected to data processing. At the same time, in the present embodiment, when calculating meanH.R (8), which is the instantaneous heart rate (HR) of 8 beats, the instantaneous heart rate that has become error (error) by applying the first rule. (HR) is not used for moving average calculation. Here, the notation of meanH.R (8) means a moving average value for an instantaneous heart rate (HR) of 8 beats, and if it is a moving average value of N beats, it is written as meanH.R (N). To do. The value of N can be determined as appropriate.

  The calculation of meanH.R (8) in the 13th time in Table 1 is based on this rule. As meanH.R (8), | 60 / t13-meanH.R (8 (t4, t5, t6, t7, t8, t9, t10, t11)) | (moving average from the fourth time to the eleventh time). | 60 / t13-meanH.R (8 (t5, t6, t7, t8, t9, t10, t11, t12)) |, which is the 13th meanH.R (8) that should be originally used, is not used. The calculation of meanH.R (8) for the 14th time also follows this rule. As meanH.R (8), | 60 / t14-meanH.R (8 (t5, t6, t7, t8, t9, t10 , t11, t13)) | (moving average from the 5th to the 13th time, excluding the 12th time).

  Further, although not shown in Table 1, the second rule is also used in the present embodiment. The second rule is that when the calculation result of | 60 / t-meanH.R (8) | is equal to or greater than the second predetermined value, an abnormal instantaneous heart rate (HR) that is different from the average value of the previous eight beats. Therefore, the RR interval time-series data is removed and is not subjected to data processing. Depending on the magnitude relationship between the first predetermined value and the second predetermined value, | 60 / t (n) -60 / t (n-1) | ≦ first predetermined value is not satisfied (first rule) or | 60 / t-meanH.R (N) | ≧ second predetermined value (second rule) is met, or both are judged to be abnormal values.

  Further, although not shown in Table 1, the third rule is also used in the present embodiment. The third rule is that the calculation result of the moving average value (meanH.R (8)) for a predetermined number of beats (for example, 8 beats) is continuously within a predetermined number of times (for example, 8 times) within a range of the second predetermined value. If not, reset meanH.R (8) and recalculate meanH.R (8). This calculation corresponds to the first to eighth processing of Table 1, but unlike Table 1, in this case the first value is detected, so the first rule is also applied to the first time, The moving average value is obtained in the eighth time of Table 1 one time earlier.

(Fourier transform process in step ST27)
In the Fourier transform process, three types of windows, known as a rectangular window (no window), a Hanning window, and a Hamming window, were tried. As a result, the hamming window is used because the best result is obtained in the separation of the high frequency component HF and the low frequency component LF, which will be described later. Since the RR interval time-series data has been functionalized, fast Fourier transform is possible in which resampling is performed at a constant period.

(Detection processing of high frequency component HF / low frequency component LF in step ST28)
The high frequency component HF represents the activity of the parasympathetic nerve, and the low frequency component LF is defined as an index representing the activity of the sympathetic nerve and is used in existing papers and documents. The high frequency component HF is an integral value of a power spectrum of 0.15 to 0.4 Hz obtained by a filter that passes a band of 0.15 to 0.4 Hz after Fourier transform. The low frequency component LF is an integral value of a power spectrum obtained by a filter that passes a band of 0.04 to 0.15 Hz.

(Plot processing after adjusting sex and age in step ST29)
In this process, the distribution of the high-frequency component HF and the distribution of the low-frequency component LF are measured in advance according to the sex and age of the subject, and the activity of the parasympathetic nerve of the subject measured by the smartphone 10 measured in step ST28 ( The state of the high-frequency component HF) and the activity of the sympathetic nerve (low-frequency component LF) is displayed two-dimensionally. The accuracy of the two-dimensionally displayed result depends on how many subjects' data is collected, and how it is evaluated and used, which will be described in detail.

  FIG. 10 is a diagram illustrating a two-dimensional display displayed on the display of the smartphone 10 based on the analysis result data from the server within the broken line. FIG. 10 is a part of the display screen displayed in FIG.

  As shown in FIG. 10, the region (first quadrant) that is higher than the origin of the first coordinate axis (X axis) and higher than the origin of the second coordinate axis (Y axis) is the “ideal zone”, which is higher than the origin of the first coordinate axis. Is lower, the region higher than the origin of the second coordinate axis (second quadrant) is “stress”, the region lower than the origin of the first coordinate axis and lower than the origin of the second coordinate axis (third quadrant) is “sag”, A region (fourth quadrant) that is higher than the origin of the first coordinate axis and lower than the origin of the second coordinate axis is “loosely”. The heart mark is the subject's data. None of LnLF, LnHF, distribution graph, X axis, Y axis, black circle, and alternate long and short dash line outside the broken line is displayed on the display of the smartphone 10.

  The autonomic nerve activity index LnLF is a logarithmic display of the low frequency component LF. The autonomic nervous activity index LnLF approaches a normal distribution by logarithmic display. The autonomic nerve activity index LnHF is a logarithmic display of the high frequency component HF. The autonomic nerve activity index LnHF approaches a normal distribution by logarithmic display.

  The higher the value of the autonomic nerve activity index LnLF, the higher the activity of the sympathetic nerve, and the lower the value of the autonomic nerve activity index LnLF, the lower the activity of the sympathetic nerve. Further, the higher the value of the autonomic nerve activity index LnHF, the higher the activity of the parasympathetic nerve, and the lower the value of the autonomic nerve activity index LnHF, the lower the activity of the parasympathetic nerve.

  Mental activities are classified according to the value of the autonomic nervous activity index LnHF and the value of the autonomic nervous activity index LnLF. An area where the value of the autonomic nervous activity index LnLF is higher than the average value and the value of the autonomic nervous activity index LnHF is higher than the average value is an “ideal zone”. A region where the value of the autonomic nervous activity index LnLF is lower than the average value and the value of the autonomic nervous activity index LnHF is higher than the average value is “loosely”. A region where the value of the autonomic nervous activity index LnLF is higher than the average value and the value of the autonomic nervous activity index LnHF is lower than the average value is “stress”. A region where the value of the autonomic nervous activity index LnLF is lower than the average value and the value of the autonomic nervous activity index LnHF is lower than the average value is “slow”.

  The boundary line divided into four quadrants according to the values of the autonomic nervous activity index LnLF and the autonomic nervous activity index LnHF (the line passing through the average value of the X axis and the line passing through the average value of the Y axis) is statistically processed in advance by sex and age Stored in the server 20. The server 20 extracts the positional information of the boundary line divided into the four quadrants based on the sex and age information of the subject of the smartphone 10, and sends the smartphone 10 information enabling the quadrant display by sex and age Include in data.

  On the other hand, after the low frequency component LF and the high frequency component HF of the subject of the smartphone 10 are obtained in step ST27, the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF are calculated. It is determined. The server 20 also includes the information on the position on the quadrant display of the heart mark in the analysis result data transmitted to the smartphone 10.

(How to determine the distribution of autonomic nerve activity index LnHF and autonomic nerve activity index LnLF)
In order to obtain the distribution of the autonomic nerve activity index LnHF and the autonomic nerve activity index LnLF, data on a large number of subjects by sex and age are required. First, after briefly summarizing the data collection technique, details will be described later.

  Since it is inefficient to collect data with the smartphone 10, a commercially available heart rate sensor (MyBeat), which is a dedicated system with higher accuracy than the smartphone 10, was used. A vast amount of data on the distribution of autonomic nerve activity index LnHF and autonomic nerve activity index LnLF accumulated in the past when using a commercially available heart rate sensor has already been accumulated. In order to use this enormous amount of data, the following three-step procedure was performed to make these data usable in this embodiment.

  As a first step, compare the autonomic nerve activity index LnHF and autonomic nerve activity index LnLF detected by the smartphone 10 of the embodiment with the autonomic nerve activity index LnHF and autonomic nerve activity index LnLF detected by a commercially available heart rate sensor, Confirm that the data collected by the smartphone 10 has sufficient accuracy. As a second step, a conversion formula between data obtained by the smartphone 10 and data obtained by a commercially available heart rate sensor is obtained. As a third step, normal distribution according to gender and age that can be applied to the present embodiment by applying the conversion formula obtained in the second step so that a huge amount of data obtained by a commercially available heart rate sensor can be used in the smartphone 10 of the embodiment. Obtain the mean and standard deviation of.

  In order to confirm the accuracy of the autonomic nerve activity index LnHF and the autonomic nerve activity index LnLF obtained from the smartphone 10, the measurement according to the embodiment and the measurement using a commercially available heart rate sensor were performed simultaneously. A total of 24 subjects were compared with 12 subjects. The inventor (hereinafter, abbreviated as the inventor) described in the application of the present application confirms that the accuracy of measurement by the smartphone 10 of the embodiment is generally good, and ends the procedure of the first step.

  Next, the second step for obtaining the conversion formula will be described. The average value of the normal distribution of the autonomic nerve activity index LnLF obtained from a commercially available heart rate sensor is E (X), the variance is V (X), and Equation 1 is obtained. Each of E (Y), V (Y), and SD (Y) indicates the average value, variance, and standard deviation of the autonomic nerve activity index LnLF obtained from the smartphone 10.

Equation 2 is obtained by assuming that the average value of the normal distribution of the autonomic nerve activity index LnHF obtained from a commercially available heart rate sensor is E (X) and the variance is V (X). Each of E (Y), V (Y), and SD (Y) indicates the average value, variance, and standard deviation of the autonomic nerve activity index LnHF obtained from the smartphone 10.

  Using Equation 1, each value of the autonomic nerve activity index LnLF obtained from a commercially available heart rate sensor can be converted into an average value, variance, and standard deviation of the corresponding autonomic nerve activity index LnLF of the smartphone 10. Also, using Equation 2, each value of the autonomic nerve activity index LnHF obtained from a commercially available heart rate sensor can be converted into the average value, variance, and standard deviation of the autonomic nerve activity index LnHF corresponding to the smartphone 10.

  Next, the third step will be described. In the third step, the average value of the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF accumulated using a commercially available heart rate sensor, and the gender and age distribution of the standard deviation are calculated using the formulas 1 and 2. A number specific to 10 is estimated.

  (Distribution of autonomic nervous activity index (LnLF, LnHF) measured by sex and age for 60 seconds)

  The distribution of autonomic nerve activity index of 60 seconds measurement of males and females by age group accumulated by a commercially available heart rate sensor is shown in FIGS. The measurement for 60 seconds uses only the time when it was stationary.

  FIG. 11 is a diagram showing the distribution of the autonomic nerve activity index LnLF measured for 60 seconds for each male age obtained from the heart rate sensor.

  FIG. 12 is a diagram showing the distribution of the autonomic nerve activity index LnHF obtained from the heart rate sensor and measured for 60 seconds for each male age group.

  FIG. 13 is a diagram showing the distribution of the autonomic nervous activity index LnLF obtained from the heart rate sensor and measured for 60 seconds for each female age group.

  FIG. 14 is a diagram showing the distribution of the autonomic nervous activity index LnHF obtained from the heart rate sensor and measured for 60 seconds for each female age group.

  The above-described results shown in FIGS. 11 to 14 are used to estimate the autonomic nerve activity index LnHF and the autonomic nerve activity index LnLF when using the smartphone 10 by using the equations 1 and 2 in the third step as described above. . Then, as shown in FIG. 10, the estimated values are used to separate the zones into four quadrants, and the luminance data detected by the smartphone 10 is processed to which zone corresponds to the current mental state of the subject. Can be displayed.

"Modification of the first embodiment"
(Modification of hardware)
In the above-described embodiment, the connection between the client and the server of the autonomic nervous function evaluation system has been described as using the Internet line. However, the connection between the both is not limited to this, and it is possible to use a dedicated line or a public line. Any line can be used regardless of a wired communication line or a wireless line. For example, a LAN, a dedicated line using a dedicated weak radio wave, a telephone line, or the like can be used.

  As the autonomic nervous function evaluation terminal device as a client, various devices such as a smartphone, a tablet, and a dedicated terminal can be used. In addition, the autonomic nervous function evaluation terminal apparatus is configured not only by the MPU and programs executed in the MPU, but also by performing a part or all of the processing of step ST10 to step ST16 with a random logic circuit. Also good. In this case, each process performed by hardware such as a random logic circuit is not performed by a computer and a program, but is performed by each processing unit configured by hardware.

  For example, the pulse wave measurement mode start processing unit may execute the pulse wave measurement mode start processing, the measurement start notification processing unit may execute the measurement start notification processing, or the finger of the hand near the camera The image capture processing unit may perform image capture processing of the finger of the hand close to the camera, or the captured image brightness data detection processing unit may execute brightness data detection processing from the captured image. The end notification processing unit may execute measurement end notification processing, the luminance data output transmission processing unit to the server may execute luminance data output processing to the server, and the pulse wave measurement mode end processing unit performs measurement. The pulse wave measurement mode end process may be executed, the analysis result data input processing unit from the server may execute the analysis result data input process from the server, and the analysis result data display end processing unit may execute the analysis result data It may perform the shows end processing. Furthermore, some or all of the hardware components described above may be configured by an integrated circuit.

  The autonomic nervous function evaluation server as a server is configured not only by a combination of a computer and software, but also by configuring a part or all of the processing of step ST20 to step ST30 with hardware such as an integrated circuit and a random logic circuit. You may do it. In this case, each process performed by hardware such as a random logic circuit is not performed by a computer and a program, but is performed by each processing unit configured by hardware.

  In addition, advance logic processing, noise removal processing and peak detection processing using a Savitzky-Golay filter, low pass filtering processing, and abnormal value removal processing of RR interval time series data are all included in noise included in luminance data or at RR intervals. This is a process for removing noise included in the series data, and any one of these or a combination of two or more of these (including all of these combinations) can be used as appropriate.

  In the hardware configuration, the luminance data input processing unit from the terminal device may execute luminance data input processing from the terminal device, the forward logic processing unit may execute forward logic processing, and the luminance data function The luminance data function processing may be executed by the conversion processing unit, the noise removal and peak detection processing may be executed by the Savitzky-Golay filter unit, and the RR interval time series is detected by the detection processing unit of the RR interval time series data. Data detection processing may be performed, low-pass filtering processing may be performed by the low-pass filtering processing unit, and abnormal value removal processing of the RR interval time-series data is performed by the RR interval time-series data abnormal value removal processing unit Alternatively, the Fourier transform processing unit may execute the Fourier transform process, and the high frequency component HF / low frequency component LF detection processing unit may perform the high frequency component. F / Low-frequency component LF detection processing may be executed, and after adjustment of gender / age, plot processing may be executed after adjustment of gender / age, and analysis result data output processing to the terminal device The analysis result data output process to the terminal device may be executed by the unit. Furthermore, some or all of the hardware components described above may be configured by an integrated circuit.

  How to divide the processing in the client-side autonomic nervous function evaluation terminal device from the processing in the server-side autonomic nervous function evaluation server can be determined as appropriate. For example, in order to correct different performances depending on the camera in the autonomic nerve function evaluation terminal device in the autonomic nerve function evaluation terminal device, advance logic processing (step ST21), luminance data functionalization processing (step ST22), Savitzky- Noise removal and peak detection processing by the Golay filter (step ST23), RR interval time series data detection processing (step ST24), low pass filtering processing (step ST25), RR interval time series data abnormal value removal processing (step ST26) Some or all of them may be performed in the autonomic nerve function evaluation terminal device. By doing so, it is possible to perform appropriate processing in accordance with the characteristics of the camera in the autonomic nervous function evaluation terminal device.

  The client-side autonomic nervous function evaluation terminal apparatus can perform the process of step ST29 in addition to the processes of step ST18 and step ST19. In this case, the server-side autonomic function evaluation server does not perform the process of step ST29. That is, it is possible to set what kind of image display the autonomic nerve function evaluation terminal device displays on the side of the autonomic nerve function evaluation terminal device according to the preference of the subject.

(Two-dimensional display specialized for a specific subject)
In the above-described embodiment, for a large number of unspecified subjects, a person who transmits luminance data by the smartphone 10 based on the distribution of the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF obtained from a commercially available heart rate sensor is used. Determined mental activity. That is, the mental state of the person was compared with a general person of the same age and same sex, and the state of the person's mental activity was displayed on the smartphone 10 as an image.

  However, the mental state of the self does not necessarily match the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF of the general person. Therefore, the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF calculated in the server 20 are accumulated based on the luminance data of a specific person acquired by the smartphone 10, and the autonomic nerve activity index LnLF and the autonomic nerve activity index LnHF are calculated. You may make it display the image of the state of self mental activity on the smart phone 10 on the basis of distribution.

  If the autonomic nerve function evaluation terminal system of the embodiment is used, it is possible not only to know one's own mental activity but also to actively manage mental activity. Some specific embodiments will be briefly described.

(Management of mental activities using music)
While listening to the music downloaded to the smartphone 10, the autonomic nervous function evaluation process in the smartphone 10 is executed. When analyzing the luminance data captured by the camera at the server 20, the results of the analysis of the luminance data (heart rate variability analysis results) for the first 30 seconds and the latter 30 seconds are compared to the first 30 seconds. If the autonomic nerve activity index LnLF in the latter half 30 seconds becomes higher (if it becomes “low frequency dominant (ie, sympathetic dominant)”), the server 20 determines that it is “live music”. Conversely, if the autonomic nervous activity index LnHF in the second half 30 seconds is higher than the first half 30 seconds (if it becomes “high frequency superiority (parasympathetic superiority)”), the server 20 will be “healing music”. to decide. What kind of music becomes “live-up music” and “healing music” depends on the past experience and sense of the person to be measured, and a specific classification is made for that person.

  When a subject using the smartphone 10 measures luminance data and looks at the display of the analysis result data, if a result of “stress” appears, the subject presses a predetermined button of the smartphone 10 and “healing music” is displayed on the display. A list can be displayed. Then, the subject can select and listen to desired music from the list. In this way, “stress” can be eliminated.

(Management of mental activities using photographs)
As in the case of using the music described above, the autonomic function evaluation process in the smartphone 10 is executed while viewing the photos stored in the smartphone 10. When analyzing the luminance data captured by the camera in the server 20, for example, comparing the result of analyzing luminance data (heart rate variability analysis result) of about 30 seconds in the first half and about 30 seconds in the second half, about 30 seconds in the first half If the autonomic nerve activity index LnLF in the latter half 30 seconds is higher than that (if it becomes “low-frequency dominant (that is, sympathetic dominant)”), the server 20 determines that it is a “live-up photograph”. Conversely, if the autonomic nerve activity index LnHF in the second half 30 seconds is higher than the first half 30 seconds (if it becomes “high-frequency dominant (parasympathetic dominant))”, the server 20 will be a “healing photo”. to decide. The kind of photo that becomes a “live-up photo” and a “healing photo” depends on the past experience and sense of the person to be measured, and is uniquely classified.

  When a subject using the smartphone 10 measures luminance data and looks at the display of the analysis result data, if a result of “stress” is obtained, the subject presses a predetermined button on the smartphone 10 and “healing photo” is displayed on the display. A list can be displayed. Then, the subject can select and view a desired photo from the list. In this way, “stress” can be eliminated.

(Management of mental activity using colors displayed on the display)
Similar processing can be performed for “color” displayed on the display in the same manner as the above-described music and photos. When a result of “stress” is obtained, the subject can press a predetermined button on the smartphone 10, select a desired color from the “healing color” list, and display the color on the display. From the combination of RGB colors, it is possible to create original “live-up colors” and “healing colors” that are optimal for the person.

(Management of mental activities using travel spots)
When you go to a place called “Healing Spot” or “Power Spot” in the travel course, the GPS function built into the smartphone 10 detects the location and the message “Let ’s measure!” Appears on the smartphone screen. Pop up. Therefore, by measuring the luminance data, it is possible to know one's state and whether or not the place is really a “healing spot” for the person being measured. These personal measurement data are anonymously stored in the server 20, and the more the data is stored, the more useful it is as the effect measurement data of the “healing spot”, and the more effective use of many people is possible.

(Management of mental activity using weather)
By synchronizing the daily weather data downloaded by the smartphone 10 or downloaded by the server 20 with the analysis result data calculated by the server 20, the weather and the autonomous state (mental condition) of the person as the data accumulates. The correlation becomes clear. When used for a long time, when the weather is cloudy or rainy, it becomes possible to understand not only the tendency of ordinary people such as “stress” but also the specific tendency of the person.

(Management of mental activities using GPS position information)
When individual subjects of the smartphone 10 upload luminance data to the server 20, many people are stressed at any location (prefecture, municipality unit, etc.) by uploading the location information using the GPS function. Or, it can be mapped on the map whether there are many people who are relaxed, and by making it public, it can be a means of knowing regional characteristics. This allows many people to use this result.

“Second Embodiment”
In the first embodiment, an autonomic nerve function evaluation system is configured as a client-server system. However, all processing on the server side of the first embodiment can be performed on the client side. The second embodiment relates to an autonomic nerve function evaluation apparatus that is such an embodiment. Such an autonomic nervous function evaluation apparatus corresponds to, for example, the case where the smartphone 10 described in FIG. 1 performs all the processing independently without receiving support from the server. When all of the processes described above are performed by a single smartphone, a slightly longer processing time is required than in the first embodiment, but a server need not be used. In addition, the processing capacity of smartphones is improving year by year, and the processing time continues to be shortened year by year.

  FIG. 15 is a flowchart illustrating processing in the autonomic nervous function evaluation apparatus according to the embodiment.

  The second embodiment will be described with reference to FIG. The correspondence relationship between the second embodiment and the first embodiment will be described below, and the description of the processing contents of each unit in the second embodiment will be omitted.

  The process of step ST300 corresponds to the process of step ST10. The process of step ST301 corresponds to the process of step ST11. The process of step ST302 corresponds to the process of step ST12. The process of step ST303 corresponds to the process of step ST13. The process of step ST304 corresponds to the process of step ST14. Here, the process of step ST14 is “measurement end notification process”, whereas the process of step ST304 is “measurement end notification process / display process during computation”. This is because a display that informs the subject that the processing from step ST305 onward is performed inside (for example, a smartphone) is performed.

  The process of step ST305 corresponds to the process of step ST21. The process of step ST306 corresponds to the process of step ST22. The process of step ST307 corresponds to the process of step ST23. The process of step ST308 corresponds to the process of step ST24. The process of step ST309 corresponds to the process of step ST25. The process of step ST310 corresponds to the process of step ST26. The process of step ST311 corresponds to the process of step ST27. The process of step ST312 corresponds to the process of step ST28. The process of step ST313 corresponds to the process of step ST29. The process of step ST314 corresponds to the process of step ST18. The process of step ST315 is a process of exiting the pulse wave measurement mode and exiting the pulse wave measurement mode, and corresponds to the “analysis result data display end process” of step ST19.

  In the second embodiment, the process performed by the CPU of the server is performed in the MPU. However, the MPU and the CPU are not essentially different in function, and the MPU can perform the same process as the CPU. . In this way, in the second embodiment, all processes can be performed only by the autonomic nervous function evaluation apparatus without using a server as in the first embodiment. In the second embodiment, it is not necessary to send personal information to the server, so that the personal secret is maintained.

"Other variations"
In both the first embodiment and the second embodiment, not all the processes of step ST21 (step ST305) to step ST23 (step ST307), step ST25 (step ST309), and step ST26 (step ST310) are performed. It is also possible to perform an appropriately selected process or an appropriately selected combination of processes. Each process from step ST21 (step ST305) to step ST23 (step ST307) is a process for improving the detection accuracy of the pulse wave included in the luminance data detected together with noise from the camera. Each process of step ST25 (step ST309) and step ST26 (step ST310) is a process of removing noise (the cause is luminance data detected together with noise) included in the RR interval time-series data. Therefore, the effects of each process may vary depending on the performance of the camera, and selecting the use of each process in consideration of the overall effect and the overall processing speed contributes to the improvement of the overall performance. Because.

  Embodiments combining some or all of the above-described embodiments can also be implemented. Moreover, it goes without saying that the present invention is not limited to the above-described embodiments, but covers the scope of the same technical idea.

1 Autonomic nerve function evaluation system 10 Smart phone 20 Server HF High frequency component LF Low frequency component LnHF Autonomic nerve activity index LnLF Autonomic nerve activity index

Claims (6)

A camera that obtains luminance data by photographing the luminance of the skin that changes according to blood flow;
A display for displaying images,
When the value of the brightness data is continuously the same value, a forward logic processing unit that removes the brightness data of a predetermined predetermined number of times or more from a processing target; and
RR interval time series data detection processing unit for detecting RR interval time series data from the luminance data;
A Fourier transform processing unit for Fourier transforming the RR interval time-series data;
A high-frequency component / low-frequency component detection processing unit that separates a high-frequency component and a low-frequency component of the Fourier-transformed signal;
An analysis result data output processing unit that outputs analysis result data for displaying on the display the activity of the parasympathetic nerve corresponding to the high frequency component and the activity of the sympathetic nerve corresponding to the low frequency component;
Autonomic nerve function evaluation device. A processing unit for displaying the waveform of the luminance data on the display when the luminance data is detected, or
A functionalization processing unit for continuous interpolation of the luminance data, or
Savitzky-Golay filter for removing the noise contained in the luminance data and performing peak detection, or
A low-pass filtering processing unit for removing noise included in the RR interval time-series data, or
An instantaneous heart rate inversely proportional to the value of the RR interval time series data is obtained, and if the difference between the current instantaneous heart rate and the immediately preceding instantaneous heart rate is greater than the first predetermined value, the current RR interval RR interval time-series data abnormal value removal processing unit for removing series data from the processing target, or
When an instantaneous heart rate that is inversely proportional to the value of the RR interval time series data is obtained, and the difference between the current instantaneous heart rate and the moving average value of the instantaneous heart rate until immediately before is greater than a second predetermined value, An RR interval time-series data abnormal value removal processing unit that removes current RR interval time-series data from a processing target;
The autonomic nervous function evaluation apparatus according to claim 1. In the computer of the autonomic nervous function evaluation device,
A photographing process for obtaining luminance data by photographing the luminance of the skin that changes according to blood flow with a camera;
When the value of the brightness data is continuously the same value, a forward logic process for excluding a predetermined number of times of the brightness data from a processing target;
A detection process of the RR interval time series data to detect the RR interval time series data from the luminance data,
Fourier transform processing for Fourier transforming the RR interval time series data;
A detection process for separating a high-frequency component and a low-frequency component of the Fourier-transformed signal;
Analysis result data output processing for outputting analysis result data for displaying the activity of the parasympathetic nerve corresponding to the high frequency component and the activity of the sympathetic nerve corresponding to the low frequency component on the display of the autonomic nerve function evaluation device; To execute,
Autonomic function evaluation device program. It has an autonomic nerve function evaluation terminal device and an autonomic nerve function evaluation server,
The autonomic nerve function evaluation terminal device
A camera that shoots the brightness of the skin that changes according to blood flow;
A luminance data output processing unit to the server that outputs luminance data according to the luminance of the skin to the autonomic nerve function evaluation server;
An analysis data input processing unit from a server that inputs analysis result data from the autonomic nerve function evaluation server;
A display for displaying an image according to the analysis result data,
The autonomic nervous function evaluation server is
A luminance data input processing unit for inputting the luminance data from the autonomic nervous function evaluation terminal;
When the value of the brightness data is continuously the same value, a forward logic processing unit that removes the brightness data of a predetermined predetermined number of times or more from a processing target; and
A detection processing section of the RR interval time series data to detect the RR interval time series data from the luminance data,
A Fourier transform processing unit for Fourier transforming the RR interval time-series data;
A high-frequency component / low-frequency component detection processing unit that separates a high-frequency component and a low-frequency component of the Fourier-transformed signal;
An analysis result data output processing unit for outputting the analysis result data for displaying the parasympathetic nerve activity corresponding to the high frequency component and the sympathetic nerve activity corresponding to the low frequency component on the display;
Autonomic function evaluation system. An autonomic function evaluation server that operates in cooperation with an autonomic function evaluation terminal device in an autonomic function evaluation system,
When the value of the luminance data obtained by the autonomic nervous function evaluation terminal device is continuously the same value, a forward logic processing unit that excludes the luminance data of a predetermined predetermined number of times or more from a processing target;
A detection processing section of the RR interval time series data to detect the RR interval time series data from the luminance data,
A Fourier transform processing unit for Fourier transforming the RR interval time-series data;
A high-frequency component / low-frequency component detection processing unit that separates a high-frequency component and a low-frequency component of the Fourier-transformed signal;
Analysis result data output processing for outputting the analysis result data for displaying the parasympathetic nerve activity corresponding to the high frequency component and the sympathetic nerve activity corresponding to the low frequency component on the display of the autonomic nerve function evaluation terminal device And comprising
Autonomic nerve function evaluation server. In the autonomic nerve function evaluation server computer that operates in cooperation with the autonomic nerve function evaluation terminal device,
When the value of the luminance data obtained by the autonomic nervous function evaluation terminal device is the same value continuously, a forward logic process for excluding a predetermined number of times the luminance data from the processing target;
A detection process of the RR interval time series data to detect the RR interval time series data from the luminance data,
Fourier transform processing for Fourier transforming the RR interval time series data;
A high-frequency component / low-frequency component detection process for separating a high-frequency component and a low-frequency component of the Fourier-transformed signal;
Analysis result data output processing for outputting the analysis result data for displaying the parasympathetic nerve activity corresponding to the high frequency component and the sympathetic nerve activity corresponding to the low frequency component on the display of the autonomic nerve function evaluation terminal device And to execute,
Autonomic function evaluation server program.

Images