CN111403030B - Mental health monitoring method, device, computer equipment and storage medium - Google Patents
Mental health monitoring method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111403030B CN111403030B CN202010124396.6A CN202010124396A CN111403030B CN 111403030 B CN111403030 B CN 111403030B CN 202010124396 A CN202010124396 A CN 202010124396A CN 111403030 B CN111403030 B CN 111403030B
- Authority
- CN
- China
- Prior art keywords
- head
- brain
- driver
- brain tissue
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000004630 mental health Effects 0.000 title claims abstract description 28
- 210000005013 brain tissue Anatomy 0.000 claims abstract description 188
- 210000004556 brain Anatomy 0.000 claims abstract description 145
- 238000012806 monitoring device Methods 0.000 claims abstract description 7
- 210000003128 head Anatomy 0.000 claims description 291
- 230000036541 health Effects 0.000 claims description 69
- 238000002329 infrared spectrum Methods 0.000 claims description 32
- 239000000523 sample Substances 0.000 claims description 32
- 238000003384 imaging method Methods 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 27
- 210000001061 forehead Anatomy 0.000 claims description 26
- 230000008451 emotion Effects 0.000 claims description 23
- 230000003993 interaction Effects 0.000 claims description 22
- 108010064719 Oxyhemoglobins Proteins 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 17
- 108010054147 Hemoglobins Proteins 0.000 claims description 10
- 102000001554 Hemoglobins Human genes 0.000 claims description 10
- 206010039203 Road traffic accident Diseases 0.000 claims description 5
- 230000002490 cerebral effect Effects 0.000 claims description 5
- 210000002442 prefrontal cortex Anatomy 0.000 claims description 4
- 230000009323 psychological health Effects 0.000 abstract description 34
- 238000010586 diagram Methods 0.000 description 15
- 230000002159 abnormal effect Effects 0.000 description 12
- 230000005856 abnormality Effects 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical group [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 3
- 239000008280 blood Substances 0.000 description 3
- 210000004369 blood Anatomy 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000002610 neuroimaging Methods 0.000 description 3
- 229910052760 oxygen Inorganic materials 0.000 description 3
- 239000001301 oxygen Substances 0.000 description 3
- 230000005477 standard model Effects 0.000 description 3
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 description 2
- 230000007177 brain activity Effects 0.000 description 2
- 230000003925 brain function Effects 0.000 description 2
- 230000019771 cognition Effects 0.000 description 2
- 108010002255 deoxyhemoglobin Proteins 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000005802 health problem Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000003333 near-infrared imaging Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 108091026922 FnrS RNA Proteins 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 230000008309 brain mechanism Effects 0.000 description 1
- 230000005189 cardiac health Effects 0.000 description 1
- 210000003710 cerebral cortex Anatomy 0.000 description 1
- 238000000701 chemical imaging Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000003988 neural development Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 208000020016 psychiatric disease Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
- A61B5/004—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
- A61B5/0042—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0075—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/14542—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring blood gases
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/1455—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters
- A61B5/14551—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters for measuring blood gases
- A61B5/14553—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters for measuring blood gases specially adapted for cerebral tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/20—Workers
- A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Heart & Thoracic Surgery (AREA)
- Biophysics (AREA)
- Psychiatry (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Developmental Disabilities (AREA)
- Optics & Photonics (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Radiology & Medical Imaging (AREA)
- Educational Technology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Child & Adolescent Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Neurology (AREA)
- Data Mining & Analysis (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application relates to a mental health monitoring method, a mental health monitoring device, computer equipment and a storage medium. The method comprises the following steps: controlling a camera to shoot a head image of a driver; determining a head position of the driver from the head image; according to the head position, obtaining a head coordinate positioning point of the driver; obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points; determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure; and according to the brain tissue type, the psychological health of the driver is monitored. By adopting the method, the psychological health of the driver can be timely and accurately monitored.
Description
Technical Field
The present application relates to the field of automotive technologies, and in particular, to a mental health monitoring method, a mental health monitoring device, a computer device, and a storage medium.
Background
Along with the development of automobile technology and the transformation of people's life style, the automobile gradually evolves from a tool of riding instead of walk into one of people's place of work and life, along with the time in the car increases progressively, monitors and manages the physical and mental health of driver, not only can in time discover the health problem of driver, avoids the state of an illness to delay, can remind when the health problem appears in the driver moreover, reduces the emergence of traffic accident.
At present, in-car health monitoring is generally carried out by measuring physiological characteristics such as heartbeat and blood pressure of a driver, judging whether the body health condition of the driver is abnormal or not by analyzing the measurement result, and warning when the abnormality occurs. However, the current health monitoring method is difficult to monitor the psychological health condition of the driver, and cannot accurately judge the negative emotions of depression, sadness and the like of the driver in time, so that driving safety is not facilitated.
Therefore, the current health monitoring method has the problem that the psychological health monitoring of the driver cannot be timely and accurately performed.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a mental health monitoring method, device, computer apparatus, and storage medium that can timely and accurately monitor the mental health of a driver.
A mental health monitoring method is applied to a vehicle, wherein the vehicle is provided with a camera and a near infrared spectrum imaging device, and the near infrared spectrum imaging device is used for shooting near infrared images of the brain of a driver; the method comprises the following steps:
controlling the camera to shoot a head image of the driver;
determining a head position of the driver from the head image;
Obtaining a head coordinate positioning point of the driver according to the head position;
obtaining coordinate points of brain tissue structures in the near-infrared brain image according to the head coordinate positioning points;
determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure;
and according to the brain tissue type, carrying out psychological health monitoring on the driver.
In one embodiment, the head images have a plurality of different shooting angles; the determining the head position of the driver from the head image includes:
determining head contour points of each of a plurality of the head images;
determining two-dimensional coordinates of the contour points of the head contour points;
fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points;
and determining the head position according to the three-dimensional coordinates of the contour points.
In one embodiment, the obtaining the head coordinate positioning point of the driver according to the head position includes:
determining a head region of the driver from the head position;
calculating a center point of the head region;
And obtaining the head coordinate positioning point of the driver according to the center point.
In one embodiment, there is a preset relative position between the head image and the near infrared brain image; the obtaining the coordinate point of the brain tissue structure in the near-infrared brain image according to the head coordinate positioning point comprises the following steps:
according to the head coordinate positioning points, carrying out coordinate transformation on the head image to obtain a transformed head image;
according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image;
and determining coordinate points of brain tissue structures in the transformed brain near-infrared image.
In one embodiment, the determining the brain tissue type of the brain tissue structure according to the coordinate point of the brain tissue structure includes:
respectively matching coordinate points of the brain tissue structure with coordinate point areas corresponding to a plurality of candidate brain tissue types;
and if the coordinate point is matched with the coordinate point region, judging that the brain tissue structure is a candidate brain tissue type corresponding to the coordinate point region.
In one embodiment, the psychological health monitoring of the driver according to the brain tissue type comprises:
acquiring a health monitoring value of the brain tissue structure in the near infrared brain image, and acquiring a health monitoring reference value corresponding to the brain tissue type;
comparing the health monitoring value with the health monitoring reference value;
and when the health monitoring value is not matched with the health monitoring reference value, judging that the psychological health of the driver is abnormal, and executing man-machine interaction operation for adjusting the psychological health of the driver.
In one embodiment, the brain tissue type comprises the prefrontal cortex of the brain; the health monitoring value includes an oxygenated hemoglobin content; the psychological health monitoring of the driver according to the brain tissue type further comprises:
obtaining the oxyhemoglobin content of the cerebral forehead cortex according to the received light intensity of the cerebral forehead cortex in the near infrared brain image;
comparing the oxyhemoglobin content with a preset content reference value;
and when the oxyhemoglobin content exceeds the content reference value, judging that the driver has negative emotion, and executing man-machine interaction operation for adjusting the negative emotion of the driver.
A mental health monitoring device is applied to a vehicle, wherein the vehicle is provided with a camera and a near infrared spectrum imaging device, and the near infrared spectrum imaging device is used for shooting near infrared images of the brain of a driver; the device comprises:
the image acquisition module is used for controlling the camera to shoot the head image of the driver;
an image recognition module for determining a head position of the driver from the head image;
the positioning point generation module is used for obtaining the head coordinate positioning point of the driver according to the head position;
the coordinate point generation module is used for obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points;
the brain tissue type determining module is used for determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure;
and the monitoring module is used for monitoring the psychological health of the driver according to the brain tissue type.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
controlling the camera to shoot a head image of the driver;
Determining a head position of the driver from the head image;
obtaining a head coordinate positioning point of the driver according to the head position;
obtaining coordinate points of brain tissue structures in the near-infrared brain image according to the head coordinate positioning points;
determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure;
and according to the brain tissue type, carrying out psychological health monitoring on the driver.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
controlling the camera to shoot a head image of the driver;
determining a head position of the driver from the head image;
obtaining a head coordinate positioning point of the driver according to the head position;
obtaining coordinate points of brain tissue structures in the near-infrared brain image according to the head coordinate positioning points;
determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure;
and according to the brain tissue type, carrying out psychological health monitoring on the driver.
According to the psychological health monitoring method, the device, the computer equipment and the storage medium, the head image of the driver is shot through the control camera, the head position of the driver is determined according to the head image, the head position of the driver can be accurately positioned through the head image, the head coordinate positioning point of the driver is obtained according to the head position, the coordinate point of the brain tissue structure in the near-infrared brain image is obtained according to the head coordinate positioning point, the coordinate point of the brain tissue structure in the near-infrared brain image can be accurately calibrated by taking the head position as a reference, the corresponding brain tissue type is accurately judged according to the coordinate point of the brain tissue structure, the psychological health monitoring is carried out on the driver based on the accurately judged brain tissue type, and the psychological health condition of the driver can be timely and accurately monitored.
Drawings
FIG. 1 is a diagram of an application environment for a method of central health monitoring in one embodiment;
FIG. 2 is a schematic diagram of the position of a near infrared imaging device in one embodiment;
FIG. 3 is a schematic diagram illustrating the operation of a near infrared imaging device according to one embodiment;
FIG. 4 is a schematic diagram of the positions of a transmitting probe and a receiving probe of a near infrared spectrum imaging device in one embodiment;
FIG. 5 is a schematic diagram of the positions of a transmitting probe and a receiving probe of a near infrared spectrum imaging device in another embodiment;
FIG. 6 is a flow chart of a method for central health monitoring according to one embodiment;
FIG. 7 is a schematic diagram of determining brain tissue structure coordinate points in a near-infrared image of a brain based on head coordinate locating points in one embodiment;
FIG. 8 is a block diagram of a central health monitoring device according to one embodiment;
fig. 9 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The mental health monitoring method provided by the application can be applied to an application environment shown in figure 1. Wherein the near infrared spectrum imaging device 102 and the camera 104 are connected with a central controller 106 of the automobile. Wherein near infrared spectrum imaging device 102 may be, but is not limited to, a fNIRS (functional near infrared spectrum) imaging device, and camera 104 may be, but is not limited to, a binocular camera.
In order to facilitate understanding of the present application by those skilled in the art, a camera and a near infrared spectrum imaging apparatus according to an embodiment of the present application will be described below with reference to fig. 1 to 5.
As shown in fig. 1, an application environment diagram of a mental health monitoring method is provided. The camera 104 may be a binocular camera, and is disposed in the steering wheel and opposite to the driver, and the binocular camera may use two cameras to simultaneously shoot two digital images of the target object from different angles, recover three-dimensional geometric information of the object based on parallax principle, and reconstruct three-dimensional of the object according to the three-dimensional geometric information, so as to obtain three-dimensional contour and position information of the object. Through binocular camera, on the one hand can detect facial expression, facial feature and people's eye feature to according to this analysis driver's psychological health condition, on the other hand can construct the 3D model of driver's head, the brain tissue structure in the auxiliary positioning near infrared spectrum image improves the accuracy of brain function imaging model.
As shown in fig. 2, a schematic position diagram of a near infrared spectrum imaging device 102 is provided. As can be seen from the figure, the near infrared spectrum imaging device may be two near infrared light source probes 102 located on the brain and behind the brain of the driver, where the near infrared light source probes 102 on the brain may be disposed in the ceiling of the automobile, the near infrared light source probes 102 behind the brain may be disposed in the seat headrest, and the two near infrared light source probes 102 enable the emitted and received near infrared light to cover the brain of the human, so as to obtain a comprehensive brain blood oxygen signal, and according to the signal, complete, accurate and real-time brain function imaging may be obtained.
As shown in fig. 3, a schematic diagram of the operating principle of a near infrared spectrum imaging device 102 is provided. As can be seen from the figure, the near-infrared light source probe 102 may be composed of a series of near-infrared light source emitting probes 302 and near-infrared light source receiving probes 304, wherein near-infrared light emitted by the near-infrared light source emitting probes 302 is received by different near-infrared light source receiving probes 304 through refraction, scattering and reflection, and meanwhile, each near-infrared light source receiving probe 304 also receives light emitted by different near-infrared light source emitting probes 302, and all emitted light intensities and received light intensities are comprehensively analyzed by using the modified Beer-Lamber law, so that a 3D real-time near-infrared spectrum image of brain activity can be established. Further, the near infrared spectrum imaging device is further provided with a near infrared light generating device, the device comprises a power supply module and a light source driving module, the power supply module supplies stable power for each module, and the light source driving module drives the laser diode to emit near infrared light.
As shown in fig. 4, a schematic diagram of the positions of the transmitting probe 302 and the receiving probe 304 of the near infrared spectrum imaging device 102 is provided. As can be seen from the figure, the transmitting probes 302 and the receiving probes 304 may be arranged in a matrix form, wherein the transmitting probes 302 may be arranged as shown by black dots in the figure and the receiving probes 304 may be arranged as shown by gray origin in the figure.
As shown in fig. 5, a schematic diagram of the positions of the transmitting probe 302 and the receiving probe 304 of another near infrared spectral imaging device 102 is provided. Wherein the transmitting probe 302 may be disposed at "1" and the receiving probe 304 may be disposed at "0".
In one embodiment, as shown in fig. 6, a mental health monitoring method is provided, and is illustrated by taking the application of the method to the central controller 106 in fig. 1 as an example, and includes the following steps:
in step S610, the camera is controlled to capture an image of the head of the driver.
The head image is an image containing the head of the driver and taking the interior of the automobile as the background.
In a specific implementation, a near infrared spectrum imaging device 102 is used for shooting a near infrared brain image of a driver, and the identification of the brain tissue structure type in the near infrared brain image can be realized based on the determination of the brain tissue structure position, unlike near infrared brain imaging in an indoor environment, the head position of the driver can be continuously changed due to jolt and shake of a vehicle in the driving process, and the position of the brain tissue structure is usually not fixed, so that the tissue structures such as forehead, top and the like of the brain are difficult to accurately position from the obtained near infrared brain image. Particularly, when the driver shakes a large extent, the near infrared spectrum imaging device 102 may not even be able to photograph the whole brain, so that the obtained near infrared brain image only contains partial brain tissue structures, and the recognition of the brain tissue structures is more difficult. In order to solve the above problem, the central controller 106 may send a shooting instruction to the camera 104, and after receiving the shooting instruction, the camera 104 shoots the head of the driver to obtain a partial image, and positions the head of the driver based on the head image, determines the head position, and further positions and identifies brain tissue in the near infrared image of the brain according to the head position.
For example, the camera 104 may be a binocular camera mounted on the steering wheel of the automobile and facing the driver, for example, the binocular camera may be mounted at the center of the steering wheel, the binocular camera may be capable of completely photographing the head of the driver, the obtained head image may further include the background in the automobile such as a seat, a window, a ceiling, etc., and then the camera 104 transmits the head image of the driver to the central controller 106 of the automobile.
Step S620, determining the head position of the driver from the head image.
The head position of the driver is the head coordinate of the driver.
In a specific implementation, the binocular camera can be used for shooting the head of the driver from different angles at the same time to obtain two head images, and after receiving the two head images, the central controller 106 extracts the head outlines of the driver in the two head images respectively to obtain respective head outline points and identifies the two-dimensional coordinates of the head outline points. Based on the parallax principle, the three-dimensional coordinates of the head contour point can be calculated by using the two sets of two-dimensional coordinates, and the calculated three-dimensional coordinates can be used as the head position of the driver.
For example, a point on the outline of the head of the driver is marked as a, and the head of the driver is photographed using a binocular camera, wherein the coordinates of the point a in the image photographed by the left camera are (10, 20), and the coordinates of the point a in the image photographed by the right camera are (15, 20). Based on the principle of human eye parallax, coordinates (10, 20) in the image shot by the left camera and coordinates (15, 20) in the image shot by the right camera are fitted, and the three-dimensional coordinates of the point A are obtained to be (12, 8, 20), (12, 8, 20) which are the head positions of the driver.
Step S630, according to the head position, a head coordinate positioning point of the driver is obtained.
The head coordinate positioning point is a point of the head of the driver and is used for positioning the head of the driver.
In a specific implementation, in order to accurately calibrate a brain tissue structure in a near-infrared brain image, a head coordinate positioning point can be selected in advance from the head of a driver, and the near-infrared brain image is calibrated by taking the head coordinate positioning point as a reference. The head coordinate positioning point may be selected as a center point of the head region of the driver, the central controller 106 determines the region corresponding to the head of the driver according to the position of the head outline of the driver, and obtains the head region, and the head coordinate positioning point may be obtained by calculating the center of the head region.
For example, a coordinate set in which the outline of the head of the driver can be obtained from the head image of the driver is { (x) 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),...,(x n ,y n ,z n ) By calculating
The center coordinates (x 0 ,y 0 ,z 0 )，(x 0 ,y 0 ,z 0 ) Can be used as a head coordinate positioning point.
And step S640, obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points.
The coordinate points of the brain tissue structures in the near-infrared brain image are determined by taking the head coordinate positioning points as reference points. The brain tissue structure is the tissue structure of the forehead, the top and the like of the brain on the cerebral cortex.
In a specific implementation, in order to accurately position a brain tissue structure in a near-infrared brain image and accurately identify the type of the brain tissue structure, a 3D model of the head of a driver can be established according to head coordinate positioning points, the brain of the driver and the coordinates of each brain tissue structure are determined in the 3D model, and the coordinates of the brain tissue structure in the near-infrared brain image can be determined by matching the near-infrared brain image with the brain of the driver in the 3D model.
For example, the point (x) is located according to the head coordinates 0 ,y 0 ,z 0 ) Three-dimensional coordinate set of head contour { (x) 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),...,(x n ,y n ,z n ) A 3D model of the driver's head may be built, and a brain model matching the driver's head position and size may be determined from the 3D model, the brain model still being represented by (x 0 ,y 0 ,z 0 ) For locating points, the standard model of each brain tissue structure is contained, and coordinate points corresponding to the standard model of each brain tissue structure are recorded, for example, the brain model can contain the forehead She Moxing, and the coordinate point set corresponding to the forehead She Moxing is calculated according to the head 3D model to be { (x) 11 ,y 11 ,z 11 ),(x 12 ,y 12 ,z 12 ),...,(x 20 ,y 20 ,z 20 ) }. After obtaining near infrared brain image, matching near infrared brain image with brain model to make them have same position and size, and making them pass through standard model of brain tissue structure In comparison, coordinate points of corresponding brain tissue structures in the near-infrared brain image can be determined, for example, after the near-infrared brain image is adjusted to have the same position and size as the forehead She Moxing of the brain, the brain tissue structures thereon can be marked with coordinate points { (x) 11 ,y 11 ,z 11 ),(x 12 ,y 12 ,z 12 ),...,(x 20 ,y 20 ,z 20 )}。
Step S650, determining a brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure.
Wherein the brain tissue type is forehead leaf, top leaf, etc. of brain tissue structure.
In a specific implementation, coordinate point sets corresponding to a plurality of candidate brain tissue types can be preset, coordinate points of the brain tissue structure are compared with each coordinate point set, and when the coordinate points of the brain tissue structure fall inside a certain coordinate point set, the brain tissue structure is judged to be the candidate brain tissue type corresponding to the coordinate point set.
For example, a coordinate point set of the forehead She Duiying is set in advance to { (x) 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),...,(x 10 ,y 10 ,z 10 ) The coordinate point set corresponding to the forehead leaf is { (x) 11 ,y 11 ,z 11 ),(x 12 ,y 12 ,z 12 ),...,(x 20 ,y 20 ,z 20 ) The coordinate point set corresponding to the top leaf is { (x) 21 ,y 21 ,z 21 ),(x 22 ,y 22 ,z 22 ),...,(x 30 ,y 30 ,z 30 ) When a coordinate point of a brain tissue structure is obtained, is { (x) 3 ,y 3 ,z 3 ),(x 4 ,y 4 ,z 4 ),...,(x 10 ,y 10 ,z 10 ) When the brain tissue type is determined to be the forehead lobe of the brain.
Step S660, performing psychological health monitoring on the driver according to the brain tissue type.
Wherein the mental health monitoring includes monitoring abnormal moods such as depression, sadness and the like of the driver.
In a specific implementation, for a specified brain tissue type, the central controller 106 may store a corresponding health monitoring reference value in advance, after the central controller 106 identifies the brain tissue type from the near infrared brain image, obtain a health monitoring value corresponding to the brain tissue type, compare the health monitoring value with a pre-stored health monitoring reference value, if the health monitoring value does not match with the health monitoring reference value, determine that the driver is psychological abnormal, the central controller 106 may send a psychological abnormality warning signal, and may send an adjusting signal to the man-machine interaction device on the vehicle for the man-machine interaction device to adjust the psychological health of the driver.
In practical applications, the central controller 106 may include a data acquisition module, a data analysis module, a health management module, etc., where the data acquisition module is composed of a photoelectric converter, an amplifier, a filter, etc., and is capable of collecting fnrs data and face images acquired by the near infrared spectrum imaging device 102 and the camera 104, recording the acquisition time of the data, and uploading the acquisition time to the data analysis module for processing. The data analysis module is used for carrying out real-time processing on the data acquired by the data acquisition module, so that fNIRS brain imaging and face representation restoration can be constructed. The health management module stores the fNIRS data and the face image output by the data analysis module according to the driver ID (Identity Document, identity mark), and judges the mental activities of the tested person such as neural development, perception and cognition, motor control, mental diseases, emotion and the like according to the brain activity area displayed by the fNIRS brain imaging, so that the mental health state of the driver is judged, and the change of the heart health state of the driver can be further analyzed by combining time dimension analysis. The near infrared spectrum imaging device 102fNIRS has the characteristics of low cost, good portability, non-contact performance, real-time monitoring and brain image reconstruction, suitability for brain mechanism identification of advanced cognition and interaction behavior under natural situations, convenience for mental health management, time saving, simplicity, no damage, high analysis speed, low cost, good result reproducibility and the like in the near infrared spectrum detection process.
For example, the fNIRS device may emit near infrared light with wavelengths of 850nm and 760nm through the emitting end, and may detect the content of oxyhemoglobin and deoxyhemoglobin in blood according to the near infrared wavelength received by the receiving end, and the fNIRS receiving end may detect near infrared light scattered, reflected and refracted at different frequencies using a photodiode or a photocell with the same frequency characteristics as a probe, and may display the change of light intensity on detection channels at different parts of the brain surface in real time by amplifying and filtering the received signal, thereby reflecting the change of the content of oxyhemoglobin and deoxyhemoglobin at different brain parts, further analyzing the cerebral blood oxygen signal, and may establish a real-time moving image of the brain region according to the greater oxygen supply amount of the brain region.
According to the mental health monitoring method, the camera is controlled to shoot the head image of the driver, the head position of the driver is determined according to the head image, the head position of the driver can be accurately positioned through the head image, the head coordinate positioning point of the driver is obtained according to the head position, the coordinate point of the brain tissue structure in the near-infrared brain image is obtained according to the head coordinate positioning point, the coordinate point of the brain tissue structure in the near-infrared brain image can be accurately calibrated by taking the head position as a reference, the corresponding brain tissue type is accurately judged according to the coordinate point of the brain tissue structure, mental health monitoring is carried out on the driver based on the accurately judged brain tissue type, and the mental health condition of the driver can be timely and accurately monitored.
In one embodiment, the head images have a plurality of different shooting angles; the step S620 may specifically include: determining head contour points of each of the plurality of head images; determining two-dimensional coordinates of contour points of the head contour points; fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points; and determining the head position according to the three-dimensional coordinates of the contour points.
The head contour points are coordinate points of the head contour in the head image.
In specific implementation, the binocular camera can be used for shooting the head of the driver from different angles at the same time to obtain two head images, and the central controller is used for respectively extracting the head outline of the driver in the two head images after receiving the two head images to obtain respective head outline points and identifying the two-dimensional coordinates of the head outline points. Based on the parallax principle, the three-dimensional coordinates of the head contour point can be calculated by using the two sets of two-dimensional coordinates, and the calculated three-dimensional coordinates can be used as the head position of the driver.
For example, a point on the outline of the head of the driver is marked as a, and the head of the driver is photographed using a binocular camera, wherein the coordinates of the point a in the image photographed by the left camera are (10, 20), and the coordinates of the point a in the image photographed by the right camera are (15, 20). Based on the principle of human eye parallax, coordinates (10, 20) in the image shot by the left camera and coordinates (15, 20) in the image shot by the right camera are fitted, and the three-dimensional coordinates of the point A are obtained to be (12, 8, 20), (12, 8, 20) which are the head positions of the driver.
In this embodiment, the two-dimensional coordinates of the head contour of the driver can be obtained from multiple perspectives by determining the head contour points of the respective head images and determining the two-dimensional coordinates of the contour points of the head contour points, the three-dimensional coordinates of the head contour of the driver can be obtained by fitting the two-dimensional coordinates of the contour points of the respective head images to the three-dimensional coordinates of the contour points, and the head position can be obtained by determining the head position from the three-dimensional coordinates of the contour points.
In one embodiment, the step S630 may specifically include: determining a head region of the driver based on the head position; calculating a center point of the head region; and obtaining a head coordinate positioning point of the driver according to the center point.
In a specific implementation, in order to accurately calibrate a brain tissue structure in a near-infrared brain image, a head coordinate positioning point can be selected in advance from the head of a driver, and the near-infrared brain image is calibrated by taking the head coordinate positioning point as a reference. The head coordinate positioning point may be selected as a center point of the head region of the driver, the central controller 106 determines the region corresponding to the head of the driver according to the position of the head outline of the driver, and obtains the head region, and the head coordinate positioning point may be obtained by calculating the center of the head region.
For example, a coordinate set in which the outline of the head of the driver can be obtained from the head image of the driver is { (x) 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),...,(x n ,y n ,z n ) By calculating
The center coordinates (x 0 ,y 0 ,z 0 )，(x 0 ,y 0 ,z 0 ) Can be used as a head coordinate positioning point.
In this embodiment, the head region of the driver is determined according to the head position, and the head of the driver can be accurately positioned by calculating the center point of the head region and obtaining the head coordinate positioning point of the driver according to the center point.
In one embodiment, there is a preset relative position between the head image and the near infrared brain image; the step S640 may specifically include: according to the head coordinate positioning points, carrying out coordinate transformation on the head image to obtain a transformed head image; according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image; and determining coordinate points of brain tissue structures in the transformed near infrared brain image.
In the specific implementation, the near infrared spectrum imaging device and the camera can be arranged at a fixed position in the vehicle, so that the relative positions of the near infrared spectrum imaging device and the camera are unchanged, a certain relative position exists between the shot near infrared brain image and the shot head image, and the coordinates of the near infrared brain image can be determined according to the head image. After the head coordinate positioning point is obtained, the head coordinate positioning point can be used as a reference point, the head image is subjected to coordinate transformation to obtain a transformed head image, the brain near infrared image is subjected to coordinate transformation according to the transformed head image and the relative position between the brain near infrared image and the head image to obtain a transformed brain near infrared image, and the coordinate point of the brain tissue structure in the transformed brain near infrared image can be determined.
For example, as shown in fig. 7, a schematic diagram is provided for determining brain tissue structure coordinate points in a near-infrared image of the brain from head coordinate locating points, including a head image 702 and a near-infrared image 704 of the brain. The coordinates of the point a may be (0, 0), the point a may be a reference point, the coordinate of the head image 702 may be transformed, for example, the point B at the lower left in the head image 702 may be transformed to (-8, -12, 0), the coordinate point in the near-infrared brain image 704 may be determined according to the relative position between the near-infrared brain image 704 and the head image 702, for example, the relative coordinates between the point C at the lower left in the near-infrared brain image 704 and the point B in the head image 702 are known, the coordinates of the point B may be (-8, -12, 0), the coordinates of the point C may be obtained to (-6, 0), and the coordinate points of the brain tissue structures in the near-infrared brain image may be determined according to the same processing method as the point C.
In this embodiment, the coordinate transformation is performed on the head image according to the head coordinate positioning point to obtain a transformed head image, the coordinates of the head image may be redetermined by taking the head coordinate positioning point as a reference point, the coordinate transformation is performed on the near-infrared brain image according to the transformed head image and the relative position to obtain a transformed near-infrared brain image, the coordinates of the near-infrared brain image may be determined, the coordinate points of the brain tissue structure in the transformed near-infrared brain image may be determined, and the brain tissue structure type may be determined according to the coordinate points of the brain tissue structure.
In one embodiment, the step S650 may specifically include: respectively matching coordinate points of the brain tissue structure with coordinate point areas corresponding to a plurality of candidate brain tissue types; and if the coordinate points are matched with the coordinate point areas, judging that the brain tissue structure is the candidate brain tissue type corresponding to the coordinate point areas.
The coordinate point area is a coordinate point set formed by a plurality of coordinate points.
In a specific implementation, coordinate point sets corresponding to a plurality of candidate brain tissue types can be preset, coordinate points of the brain tissue structure are compared with each coordinate point set, and when the coordinate points of the brain tissue structure fall inside a certain coordinate point set, the brain tissue structure is judged to be the candidate brain tissue type corresponding to the coordinate point set.
For example, a coordinate point set of the forehead She Duiying is set in advance to { (x) 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),...,(x 10 ,y 10 ,z 10 ) The coordinate point set corresponding to the forehead leaf is { (x) 11 ,y 11 ,z 11 ),(x 12 ,y 12 ,z 12 ),...,(x 20 ,y 20 ,z 20 ) The coordinate point set corresponding to the top leaf is { (x) 21 ,y 21 ,z 21 ),(x 22 ,y 22 ,z 22 ),...,(x 30 ,y 30 ,z 30 ) When a coordinate point of a brain tissue structure is obtained, is { (x) 3 ,y 3 ,z 3 ),(x 4 ,y 4 ,z 4 ),...,(x 10 ,y 10 ,z 10 ) When the brain tissue type is determined to be the forehead lobe of the brain.
In this embodiment, coordinate points of the brain tissue structure are respectively matched with coordinate point areas corresponding to a plurality of candidate brain tissue types, if the coordinate points are matched with the coordinate point areas, the brain tissue structure is determined to be the candidate brain tissue type corresponding to the coordinate point areas, and the type of the brain tissue structure in the near-infrared brain image can be determined by comparing the candidate brain tissue types with the plurality of candidate brain tissue types.
In one embodiment, the step S660 may specifically include: acquiring a health monitoring value of a brain tissue structure in a near infrared image of a brain, and acquiring a health monitoring reference value corresponding to the brain tissue type; comparing the health monitoring value with a health monitoring reference value; and when the health monitoring value is not matched with the health monitoring reference value, judging that the psychological health of the driver is abnormal, and executing man-machine interaction operation for adjusting the psychological health of the driver.
The health monitoring value is near infrared light intensity received by the near infrared spectrum imaging device receiving end, and the health monitoring reference value is standard near infrared light intensity received by the near infrared spectrum imaging device receiving end aiming at the specified brain tissue type.
In specific implementation, for a specified brain tissue type, the central controller may store a corresponding health monitoring reference value in advance, after the central controller identifies the brain tissue type from the near infrared brain image, obtain a health monitoring value corresponding to the brain tissue type, compare the health monitoring value with a pre-stored health monitoring reference value, if the health monitoring value is not matched with the health monitoring reference value, determine that the driver is psychological abnormal, the central controller may send a psychological abnormality alarm signal, and may send an adjusting signal to the man-machine interaction device on the vehicle for the man-machine interaction device to adjust the psychological health of the driver.
In this embodiment, a health monitoring value of a brain tissue structure in a near-infrared image of the brain is obtained, and a health monitoring reference value corresponding to a brain tissue type is obtained, and the health monitoring value can be compared with the health monitoring reference value; when the health monitoring value is not matched with the health monitoring reference value, the psychological health abnormality of the driver is judged, the man-machine interaction operation for adjusting the psychological health of the driver is executed, the psychological health abnormality of the driver can be timely and accurately found, the illness delay is avoided, the abnormal condition is processed, and the traffic accident is reduced.
In one embodiment, the brain tissue type includes the prefrontal cortex of the brain; health monitoring values include oxygenated hemoglobin content; the step S660 may further specifically include: obtaining the oxyhemoglobin content of the forehead cortex of the brain according to the received light intensity of the forehead cortex of the brain in the near infrared brain image; comparing the oxyhemoglobin content with a preset content reference value; and when the oxyhemoglobin content exceeds the content reference value, judging that the driver has negative emotion, and executing man-machine interaction operation for adjusting the negative emotion of the driver.
In a specific implementation, the fnigs device can emit near infrared light at an emitting end, the near infrared light is scattered, reflected and refracted, a receiving end of the fnigs device receives the near infrared light, the received near infrared light intensity can reflect the content of oxygenated hemoglobin of the brain, the central controller recognizes the content of oxygenated hemoglobin corresponding to brain tissue structure threo in the near infrared image of the brain after receiving the near infrared image of the brain, compares the content of oxygenated hemoglobin with a preset content reference value, and when the content of oxygenated hemoglobin exceeds the content reference value, the central controller can analyze the emotional negativity of a driver and send a signal to the man-machine interaction device to inform the man-machine interaction device to execute man-machine interaction operation for adjusting the negative emotion of the driver.
In this embodiment, according to the received light intensity of the brain forehead cortex in the brain near-infrared image, the oxygenated hemoglobin content of the brain forehead cortex is obtained, and the oxygenated hemoglobin content can be compared with a preset content reference value; when the content of the oxyhemoglobin exceeds the content reference value, the passive emotion of the driver is judged to appear, and the man-machine interaction operation for adjusting the passive emotion of the driver is executed, so that the psychological health abnormal condition of the driver can be timely and accurately found, the illness delay is avoided, the abnormal condition is processed, and the traffic accident is reduced.
It should be understood that, although the steps in the flowchart of fig. 6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least a portion of the steps in fig. 6 may include a plurality of steps or stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the execution of the steps or stages is not necessarily sequential, but may be performed in rotation or alternately with at least a portion of the steps or stages in other steps or stages.
In one embodiment, as shown in fig. 8, there is provided a mental health monitoring apparatus 800 comprising: an image acquisition module 801, an image recognition module 802, a localization point generation module 803, a coordinate point generation module 804, a brain tissue type determination module 805, a monitoring module 806, wherein:
an image acquisition module 801, configured to control a camera to capture an image of a head of a driver;
an image recognition module 802 for determining a head position of the driver from the head image;
The positioning point generating module 803 is configured to obtain a head coordinate positioning point of the driver according to the head position;
the coordinate point generating module 804 is configured to obtain coordinate points of a brain tissue structure in the near-infrared brain image according to the head coordinate positioning points;
a brain tissue type determining module 805, configured to determine a brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure;
the monitoring module 806 is configured to monitor psychological health of the driver according to the brain tissue type.
In one embodiment, the head images have a plurality of different shooting angles; the image recognition module 802 is further configured to determine head contour points of each of the plurality of head images; determining two-dimensional coordinates of contour points of the head contour points; fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points; and determining the head position according to the three-dimensional coordinates of the contour points.
In one embodiment, the localization point generation module 803 is further configured to determine a head region of the driver according to the head position; calculating a center point of the head region; and obtaining a head coordinate positioning point of the driver according to the center point.
In one embodiment, there is a preset relative position between the head image and the near infrared brain image; the coordinate point generating module 804 is further configured to perform coordinate transformation on the head image according to the head coordinate positioning point, so as to obtain a transformed head image; according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image; and determining coordinate points of brain tissue structures in the transformed near infrared brain image.
In one embodiment, the brain tissue type determining module 805 is further configured to match coordinate points of the brain tissue structure with coordinate point areas corresponding to the plurality of candidate brain tissue types, respectively; and if the coordinate points are matched with the coordinate point areas, judging that the brain tissue structure is the candidate brain tissue type corresponding to the coordinate point areas.
In one embodiment, the monitoring module 806 is further configured to obtain a health monitoring value of a brain tissue structure in the near infrared image of the brain, and obtain a health monitoring reference value corresponding to the brain tissue type; comparing the health monitoring value with a health monitoring reference value; and when the health monitoring value is not matched with the health monitoring reference value, judging that the psychological health of the driver is abnormal, and executing man-machine interaction operation for adjusting the psychological health of the driver.
In one embodiment, the brain tissue type includes the prefrontal cortex of the brain; health monitoring values include oxygenated hemoglobin content; the monitoring module 806 is further configured to obtain an oxyhemoglobin content of the brain forehead cortex according to the received light intensity of the brain forehead cortex in the brain near infrared image; comparing the oxyhemoglobin content with a preset content reference value; and when the oxyhemoglobin content exceeds the content reference value, judging that the driver has negative emotion, and executing man-machine interaction operation for adjusting the negative emotion of the driver.
For specific limitations of the mental health monitoring device, reference may be made to the above description of the mental health monitoring method, and no further description is given here. The above-described modules in the mental health monitoring device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 9. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing mental health monitoring data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a mental health monitoring method.
It will be appreciated by those skilled in the art that the structure shown in fig. 9 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of: controlling a camera to shoot a head image of a driver; determining a head position of the driver from the head image; according to the head position, obtaining a head coordinate positioning point of the driver; obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points; determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure; and according to the brain tissue type, the psychological health of the driver is monitored.
In one embodiment, the processor when executing the computer program further performs the steps of: determining head contour points of each of the plurality of head images; determining two-dimensional coordinates of contour points of the head contour points; fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points; and determining the head position according to the three-dimensional coordinates of the contour points.
In one embodiment, the processor when executing the computer program further performs the steps of: determining a head region of the driver based on the head position; calculating a center point of the head region; and obtaining a head coordinate positioning point of the driver according to the center point.
In one embodiment, the processor when executing the computer program further performs the steps of: according to the head coordinate positioning points, carrying out coordinate transformation on the head image to obtain a transformed head image; according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image; and determining coordinate points of brain tissue structures in the transformed near infrared brain image.
In one embodiment, the processor when executing the computer program further performs the steps of: respectively matching coordinate points of the brain tissue structure with coordinate point areas corresponding to a plurality of candidate brain tissue types; and if the coordinate points are matched with the coordinate point areas, judging that the brain tissue structure is the candidate brain tissue type corresponding to the coordinate point areas.
In one embodiment, the processor when executing the computer program further performs the steps of: acquiring a health monitoring value of a brain tissue structure in a near infrared image of a brain, and acquiring a health monitoring reference value corresponding to the brain tissue type; comparing the health monitoring value with a health monitoring reference value; and when the health monitoring value is not matched with the health monitoring reference value, judging that the psychological health of the driver is abnormal, and executing man-machine interaction operation for adjusting the psychological health of the driver.
In one embodiment, the processor when executing the computer program further performs the steps of: obtaining the oxyhemoglobin content of the forehead cortex of the brain according to the received light intensity of the forehead cortex of the brain in the near infrared brain image; comparing the oxyhemoglobin content with a preset content reference value; and when the oxyhemoglobin content exceeds the content reference value, judging that the driver has negative emotion, and executing man-machine interaction operation for adjusting the negative emotion of the driver.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of: controlling a camera to shoot a head image of a driver; determining a head position of the driver from the head image; according to the head position, obtaining a head coordinate positioning point of the driver; obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points; determining the brain tissue type of the brain tissue structure according to the coordinate points of the brain tissue structure; and according to the brain tissue type, the psychological health of the driver is monitored.
In one embodiment, the computer program when executed by the processor further performs the steps of: determining head contour points of each of the plurality of head images; determining two-dimensional coordinates of contour points of the head contour points; fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points; and determining the head position according to the three-dimensional coordinates of the contour points.
In one embodiment, the computer program when executed by the processor further performs the steps of: determining head contour points of each of the plurality of head images; determining two-dimensional coordinates of contour points of the head contour points; fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points; and determining the head position according to the three-dimensional coordinates of the contour points.
In one embodiment, the computer program when executed by the processor further performs the steps of: determining a head region of the driver based on the head position; calculating a center point of the head region; and obtaining a head coordinate positioning point of the driver according to the center point.
In one embodiment, the computer program when executed by the processor further performs the steps of: according to the head coordinate positioning points, carrying out coordinate transformation on the head image to obtain a transformed head image; according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image; and determining coordinate points of brain tissue structures in the transformed near infrared brain image.
In one embodiment, the computer program when executed by the processor further performs the steps of: respectively matching coordinate points of the brain tissue structure with coordinate point areas corresponding to a plurality of candidate brain tissue types; and if the coordinate points are matched with the coordinate point areas, judging that the brain tissue structure is the candidate brain tissue type corresponding to the coordinate point areas.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a health monitoring value of a brain tissue structure in a near infrared image of a brain, and acquiring a health monitoring reference value corresponding to the brain tissue type; comparing the health monitoring value with a health monitoring reference value; and when the health monitoring value is not matched with the health monitoring reference value, judging that the psychological health of the driver is abnormal, and executing man-machine interaction operation for adjusting the psychological health of the driver.
In one embodiment, the computer program when executed by the processor further performs the steps of: obtaining the oxyhemoglobin content of the forehead cortex of the brain according to the received light intensity of the forehead cortex of the brain in the near infrared brain image; comparing the oxyhemoglobin content with a preset content reference value; and when the oxyhemoglobin content exceeds the content reference value, judging that the driver has negative emotion, and executing man-machine interaction operation for adjusting the negative emotion of the driver.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. A mental health monitoring method for non-diagnostic purposes, characterized by being applied to a vehicle equipped with a camera and a near infrared spectrum imaging device; the camera is a binocular camera and is arranged in the steering wheel and is opposite to the driver; the near infrared spectrum imaging equipment is two near infrared light source probes positioned on the brain and behind the brain of the driver, wherein the near infrared light source probes on the brain are arranged in the ceiling of the vehicle, the near infrared light source probes behind the brain are arranged in a seat headrest of the vehicle, and the near infrared spectrum imaging equipment is used for shooting near infrared images of the brain of the driver; the method comprises the following steps:
Controlling the camera to shoot a head image of the driver;
determining a head position of the driver from the head image;
obtaining a head coordinate positioning point of the driver according to the head position;
obtaining coordinate points of brain tissue structures in the near-infrared brain image according to the head coordinate positioning points;
determining the brain tissue type of the brain tissue structure in the near-infrared brain image according to the coordinate points of the brain tissue structure;
and monitoring the negative emotion of the driver according to the brain tissue type so as to reduce traffic accidents.
2. The method according to claim 1, wherein the head images have a plurality of, a plurality of which are different in shooting angle; the determining the head position of the driver from the head image includes:
determining head contour points of each of a plurality of the head images;
determining two-dimensional coordinates of the contour points of the head contour points;
fitting the two-dimensional coordinates of the contour points of each of the plurality of head images to three-dimensional coordinates of the contour points;
and determining the head position according to the three-dimensional coordinates of the contour points.
3. The method of claim 2, wherein the obtaining the head coordinate positioning point of the driver based on the head position comprises:
Determining a head region of the driver from the head position;
calculating a center point of the head region;
and obtaining the head coordinate positioning point of the driver according to the center point.
4. The method of claim 1, wherein there is a preset relative position between the head image and the near infrared brain image; the obtaining the coordinate point of the brain tissue structure in the near-infrared brain image according to the head coordinate positioning point comprises the following steps:
according to the head coordinate positioning points, carrying out coordinate transformation on the head image to obtain a transformed head image;
according to the transformed head image and the relative position, carrying out coordinate transformation on the near-infrared brain image to obtain a transformed near-infrared brain image;
and determining coordinate points of brain tissue structures in the transformed brain near-infrared image.
5. The method of claim 1, wherein determining the brain tissue type of the brain tissue structure in the near-infrared image of the brain from the coordinate points of the brain tissue structure comprises:
respectively matching coordinate points of the brain tissue structure with coordinate point areas corresponding to a plurality of candidate brain tissue types;
And if the coordinate point is matched with the coordinate point region, judging that the brain tissue structure is a candidate brain tissue type corresponding to the coordinate point region.
6. The method of claim 1, wherein the monitoring the driver's negative emotion according to the brain tissue type comprises:
acquiring a health monitoring value of the brain tissue structure in the near infrared brain image, and acquiring a health monitoring reference value corresponding to the brain tissue type;
comparing the health monitoring value with the health monitoring reference value;
and when the health monitoring value is not matched with the health monitoring reference value, judging that the negative emotion occurs to the driver, and executing man-machine interaction operation for adjusting the negative emotion.
7. The method of claim 6, wherein the brain tissue type comprises a prefrontal cortex of the brain; the health monitoring value includes an oxygenated hemoglobin content; the monitoring of the negative emotion of the driver according to the brain tissue type further comprises:
obtaining the oxyhemoglobin content of the cerebral forehead cortex according to the received light intensity of the cerebral forehead cortex in the near infrared brain image;
Comparing the oxyhemoglobin content with a preset content reference value;
and when the oxyhemoglobin content exceeds the content reference value, judging that the negative emotion appears in the driver, and executing man-machine interaction operation for adjusting the negative emotion.
8. A mental health monitoring device for non-diagnostic purposes, characterized by being applied to a vehicle equipped with a camera and a near infrared spectrum imaging apparatus; the camera is a binocular camera and is arranged in the steering wheel and is opposite to the driver; the near infrared spectrum imaging equipment is two near infrared light source probes positioned on the brain and behind the brain of the driver, wherein the near infrared light source probes on the brain are arranged in the ceiling of the vehicle, the near infrared light source probes behind the brain are arranged in a seat headrest of the vehicle, and the near infrared spectrum imaging equipment is used for shooting near infrared images of the brain of the driver; the device comprises:
the image acquisition module is used for controlling the camera to shoot the head image of the driver;
an image recognition module for determining a head position of the driver from the head image;
The positioning point generation module is used for obtaining the head coordinate positioning point of the driver according to the head position;
the coordinate point generation module is used for obtaining coordinate points of brain tissue structures in the near-infrared brain images according to the head coordinate positioning points;
the brain tissue type determining module is used for determining the brain tissue type of the brain tissue structure in the near infrared brain image according to the coordinate points of the brain tissue structure;
and the monitoring module is used for monitoring the negative emotion of the driver according to the brain tissue type so as to reduce traffic accidents.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010124396.6A CN111403030B (en) | 2020-02-27 | 2020-02-27 | Mental health monitoring method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010124396.6A CN111403030B (en) | 2020-02-27 | 2020-02-27 | Mental health monitoring method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111403030A CN111403030A (en) | 2020-07-10 |
| CN111403030B true CN111403030B (en) | 2024-02-02 |
Family
ID=71435973
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010124396.6A Active CN111403030B (en) | 2020-02-27 | 2020-02-27 | Mental health monitoring method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111403030B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113080852B (en) * | 2021-03-26 | 2022-04-22 | 南京蔚来思创科技有限公司 | Brain biofeedback diagnosis and treatment instrument |
| CN115211850A (en) * | 2022-02-22 | 2022-10-21 | 武汉资联虹康科技股份有限公司 | Data 3D visual display method based on individual head model |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107179494A (en) * | 2017-05-27 | 2017-09-19 | 浙江万里学院 | Device fault detection method and system on a kind of general-purpose circuit board plate |
| WO2018042122A1 (en) * | 2016-08-29 | 2018-03-08 | Renault S.A.S. | Method for determining the placement of the head of a vehicle driver |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
| JP2018139070A (en) * | 2017-02-24 | 2018-09-06 | 株式会社デンソー | Vehicle display control device |
| CN109146931A (en) * | 2018-11-12 | 2019-01-04 | 深圳安科高技术股份有限公司 | A kind of three dimensional image processing method, system, device and storage medium |
| CN109394181A (en) * | 2018-12-05 | 2019-03-01 | 吉林大学 | A kind of brain functional area positioning system, method and movable equipment |
| CN110060272A (en) * | 2018-01-18 | 2019-07-26 | 杭州海康威视数字技术股份有限公司 | Determination method, apparatus, electronic equipment and the storage medium of human face region |
| CN110726695A (en) * | 2019-11-19 | 2020-01-24 | 湖北医药学院 | Method for identifying gastrodin near infrared spectrum specificity of different parts of brain tissue of mouse |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003223638A (en) * | 2002-01-31 | 2003-08-08 | Electric Power Dev Co Ltd | Registration correcting method in image processing and vegetation display method using the same |
| JP3928051B2 (en) * | 2004-01-14 | 2007-06-13 | 独立行政法人情報通信研究機構 | Psychological state evaluation device |
| JP4590555B2 (en) * | 2004-09-02 | 2010-12-01 | 国立大学法人長岡技術科学大学 | Sensitive state discrimination method and apparatus |
| JP5486182B2 (en) * | 2008-12-05 | 2014-05-07 | キヤノン株式会社 | Information processing apparatus and information processing method |
| GB201305171D0 (en) * | 2013-03-21 | 2013-05-01 | Univ Nottingham | Measurement of tissue structures |
| US9946953B2 (en) * | 2013-05-10 | 2018-04-17 | Koninklijke Philips N.V. | Apparatus and method for processing images of tissue samples |
| DE102014205313B4 (en) * | 2014-03-21 | 2015-10-15 | Siemens Aktiengesellschaft | Method for registering a near-infrared spectroscopy map and an anatomy image data set and x-ray device |
| CN107567637B (en) * | 2015-04-30 | 2022-01-25 | 皇家飞利浦有限公司 | Brain tissue classification |
| US10349032B2 (en) * | 2016-09-30 | 2019-07-09 | Veoneer Us, Inc. | Vehicle occupant head positioning system |
-
2020
- 2020-02-27 CN CN202010124396.6A patent/CN111403030B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018042122A1 (en) * | 2016-08-29 | 2018-03-08 | Renault S.A.S. | Method for determining the placement of the head of a vehicle driver |
| JP2018139070A (en) * | 2017-02-24 | 2018-09-06 | 株式会社デンソー | Vehicle display control device |
| CN107179494A (en) * | 2017-05-27 | 2017-09-19 | 浙江万里学院 | Device fault detection method and system on a kind of general-purpose circuit board plate |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
| CN110060272A (en) * | 2018-01-18 | 2019-07-26 | 杭州海康威视数字技术股份有限公司 | Determination method, apparatus, electronic equipment and the storage medium of human face region |
| CN109146931A (en) * | 2018-11-12 | 2019-01-04 | 深圳安科高技术股份有限公司 | A kind of three dimensional image processing method, system, device and storage medium |
| CN109394181A (en) * | 2018-12-05 | 2019-03-01 | 吉林大学 | A kind of brain functional area positioning system, method and movable equipment |
| CN110726695A (en) * | 2019-11-19 | 2020-01-24 | 湖北医药学院 | Method for identifying gastrodin near infrared spectrum specificity of different parts of brain tissue of mouse |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111403030A (en) | 2020-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210264598A1 (en) | Human detection device equipped with light source projecting at least one dot onto living body | |
| US8369608B2 (en) | System and method for detecting drowsy facial expressions of vehicle drivers under changing illumination conditions | |
| EP3652703B1 (en) | Visual, depth and micro-vibration data extraction using a unified imaging device | |
| US10696305B2 (en) | Apparatus and method for measuring physiological information of living subject in vehicle | |
| JP4403453B2 (en) | Method for converting head surface coordinates into brain surface coordinates, and transcranial brain function measuring device using the converted data | |
| US20150078642A1 (en) | Method and system for non-invasive quantification of biologial sample physiology using a series of images | |
| WO1990013091A1 (en) | Apparatus and method for imaging and classifying lesions | |
| CA2934659A1 (en) | System and methods for measuring physiological parameters | |
| CN111403030B (en) | Mental health monitoring method, device, computer equipment and storage medium | |
| CN111231970A (en) | Health regulation method and device, computer equipment and storage medium | |
| KR102031958B1 (en) | Prefrontal based cognitive brain-machine interfacing apparatus and method thereof | |
| CN111248890A (en) | Non-contact newborn heart rate monitoring method and system based on facial video | |
| CN112244765B (en) | Method, device and system for detecting brain temporary abnormal state | |
| KR102356465B1 (en) | Method and server for face registration and face analysis | |
| CN112507772A (en) | Face recognition security system and suspicious person detection and early warning method | |
| Paquit et al. | Combining near-infrared illuminants to optimize venous imaging | |
| CN212353957U (en) | Health regulating system | |
| Saikia et al. | K-means clustering for unsupervised participant grouping from fNIRS brain signal in working memory task | |
| CN116092157A (en) | Intelligent facial tongue diagnosis method, system and intelligent equipment | |
| Firdaus et al. | An Embedded Computer Vision Method to Extract Percentage of Eye Close for Detecting Drowsiness of a Safety Driving System | |
| Sacramento et al. | A real-time software to the acquisition of heart rate and photoplethysmography signal using two region of interest simultaneously via webcam | |
| CN114863399A (en) | Physiological state detection method and device, electronic equipment and storage medium | |
| KR102699187B1 (en) | System for the diagnosis of skin diseases and the analysis of skin states using multiband imagery information based on visual artificial intelligence and method thereof | |
| Sarkar et al. | Video magnification to detect heart rate for drivers | |
| Rundo et al. | Advanced Intelligent deep learning-based system for Robust Driving Assistance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 511458 room 1201, No. 37, Jinlong, Nansha street, Xiangjiang financial and business center, Nansha District, Guangzhou City, Guangdong Province (office only) Applicant after: Hechuang Automotive Technology Co.,Ltd. Address before: 511458 room 1201, No. 37, Jinlong, Nansha street, Xiangjiang financial and business center, Nansha District, Guangzhou City, Guangdong Province (office only) Applicant before: Hechuang Smart Technology Co.,Ltd. Address after: 511458 room 1201, No. 37, Jinlong, Nansha street, Xiangjiang financial and business center, Nansha District, Guangzhou City, Guangdong Province (office only) Applicant after: Hechuang Smart Technology Co.,Ltd. Address before: Room 1201, No. 37, Jinsha street, Nansha street, Xiangjiang financial and business center, Nansha District, Guangzhou City, Guangdong Province Applicant before: Guangzhou Auto Weilai New Energy Automotive Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |