JP7508958B2 - Information processing device, information processing system, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing system, and a program.

In facilities such as medical facilities and nursing homes, there are information processing systems that provide information about users, indicating the status of users using beds installed in the facilities (e.g., whether they are lying in bed, getting up, leaving the bed, etc.).

For example, there is known an information processing system that uses temperature image data of a user using a bed provided in a facility to detect the condition of the user and notifies a predetermined notification destination of information indicating the user's condition (see, for example, Patent Document 1).

The technology disclosed in Patent Document 1 acquires a temperature image of the bed used by the user and the area around the bed, and displays the acquired temperature image in a specified display area on the display screen. However, since it is difficult to grasp the position of a specified piece of equipment (e.g., a bed or pillow) from the temperature image, it is difficult for an administrator, for example, to set up or change the system while looking at the display screen.

The embodiment of the present invention has been made in consideration of the above problems, and in an information processing system that displays on a display screen temperature images of users who use a specified facility, it makes it easy for an administrator or the like to set up or change the system while looking at the display screen.

In order to solve the above problem, an information processing device according to one embodiment of the present invention has an acquisition unit that acquires a first image captured of a specified facility used by a user and a second image having a different resolution from the first image and representing the temperature of the specified facility; a conversion unit that converts the resolution of the first image and the resolution of the second image into the resolution of a display area that displays the first image or the second image; a display control unit that displays a setting screen including the display area that selectively displays the first image or the second image converted to the resolution of the display area, the setting screen setting the range of a detection area that detects a specified temperature and the temperature range of the detection area; and a setting reception unit that receives setting operations for the detection area and the temperature range setting from a user via the setting screen.

According to one embodiment of the present invention, in an information processing system that displays on a display screen temperature images of users who use a specified facility, it becomes easy for an administrator or the like to set up or change the system while viewing the display screen.

FIG. 1 is a diagram showing an example of a system configuration of an information processing system according to an embodiment; FIG. 2 is a diagram showing an example of a system configuration of an information processing system according to an embodiment; FIG. 2 is a diagram illustrating an example of an arrangement of imaging devices according to an embodiment. FIG. 4 is a diagram illustrating an example of a detection region according to an embodiment. 11A and 11B are diagrams for explaining examples of judgment conditions and judgment contents according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus according to an embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating an example of information managed by an information processing device according to an embodiment. 10 is a flowchart illustrating an example of a selection process of a display image according to the first embodiment. 5A to 5C are diagrams for explaining enlargement processing of a display image according to the first embodiment; FIG. 2 is a diagram showing an example of a display screen according to the first embodiment; FIG. 11 is a diagram showing an example of a setting screen according to the second embodiment; 13 is a flowchart illustrating an example of a process for setting a detection area according to the second embodiment. FIG. 11 is a diagram showing an image of a detection area on a captured image according to the second embodiment. 13 is a flowchart showing an example of a display process of a display image and a detection area according to a second embodiment.

The following describes an embodiment of the present invention with reference to the attached drawings.

<System Configuration>
1 and 2 are diagrams showing an example of a system configuration of an information processing system according to an embodiment. In the example of Fig. 1, the information processing system 100 includes, for example, a bed installed in a medical facility, a nursing facility, or the like, one or more imaging devices 102 that capture images of the surroundings of the bed, and an information processing device 101 communicably connected to the one or more imaging devices via a communication network 10. The bed is an example of a predetermined facility used by a user.

The imaging device 102 includes, for example, a thermal camera 112 that captures an image showing the temperature of the bed and the area around the bed (hereinafter referred to as a temperature image), and a camera 122 that captures a normal image of the bed and the area around the bed (hereinafter referred to as a captured image).

Thermal camera 112 may be, for example, a general infrared thermography camera. Thermal camera 112 is a device that images infrared radiation from a measurement object, converts it into temperature, and visualizes the temperature distribution using colors or the like. Thermal camera 112 may also be called thermography, thermovision, thermoviewer, etc.

Camera 122 is a camera that captures general images and is equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, etc.

The imaging device 102 is installed on a wall or ceiling, for example, in a medical facility or a care facility, so as to capture temperature images and images of the bed used by the user (patient or person requiring care, etc.) and the area around the bed.

The imaging device 102 also transmits temperature images of the bed and the area around the bed captured at a predetermined frame rate and the captured images to the information processing device 101 via the communication network 10. At this time, the imaging device 102 may transmit either the temperature image or the captured image to the information processing device 101, for example, in accordance with control information from the information processing device 101.

At this time, the images (temperature images and captured images) sent by the image capture device 102 include identification information (e.g., IP address, camera ID, bed ID, etc.) for identifying the image capture device 102, the bed, or the user, or information such as a display name.

The information processing device 101 is, for example, a computer such as a PC (Personal Computer) or a system including multiple computers. The information processing device 101 acquires temperature images and captured images transmitted from the imaging device 102, and detects a predetermined temperature in the bed and one or more detection areas set in advance around the bed using the acquired temperature images. The information processing device 101 also detects the condition of a user using the bed based on the predetermined temperatures detected in one or more detection areas.

Here, the user includes, for example, a patient hospitalized in a medical facility, a resident of a nursing home, etc. Here, as an example, the following explanation will be given assuming that the user is a patient hospitalized in a medical facility.

The information processing system 100 may also have multiple information processing devices 101, for example, as shown in FIG. 2. For example, the information processing system 100 may have a different information processing device 101 for each floor, department, or ward of a medical facility. In this case, the information processing system 100 may have an information processing server 201 communicatively connected to the multiple information processing devices 101 via the communication network 10, and the information processing server 201 may manage information on users who use all or some of the beds in the medical facility.

Furthermore, the information processing server 201 may be communicably connected to a nurse call system 202 provided in a medical facility via the communication network 10. For example, the information processing server 201 may notify the nurse call system 202 of the detection result of a user using a bed, and display the detection result on a display device 203 provided in a nurse's station and an information terminal 204 carried by a medical staff member, etc.

Note that the system configuration of the information processing system 100 shown in Figures 1 and 2 is an example. For example, in the information processing system 100 shown in Figure 2, the functions of the information processing server 201 may be possessed by one of the multiple information processing devices 101. In addition, some or all of the functions of the information processing server 201 may be possessed by the nurse call system 202.

(Example of arrangement of imaging device)
3 is a diagram showing an example of the arrangement of the imaging device according to an embodiment. For example, as shown in Fig. 3(A), the imaging device 102 is installed on a wall surface or the like of a hospital room 300 so as to be able to capture images (temperature images and captured images) of a bed 301 and the surroundings of the bed 301. The imaging device 102 may be installed on the ceiling of the hospital room 300.

Figure 3 (B) shows the hospital room 300 as seen from above. The image capturing device 102 is installed so as to capture an image of the bed 301 corresponding to the image capturing device 102 and a predetermined range including the periphery of the bed 301 (hereinafter referred to as the detection range 302).

(Example of detection area)
Fig. 4 is a diagram showing an example of a detection area according to an embodiment. The information processing device 101 manages area information, which is information on one or more detection areas set in advance within the detection range 302 of the image capture device 102. For example, as shown in Fig. 4, a plurality of detection areas are set within the detection range 302 of the image capture device 102. In the example of Fig. 4, a detection area A401, a detection area B402, a detection area C403, a detection area D404, a detection area E405, and a detection area F406 are set within the detection range 302.

In the example of FIG. 4, detection area A401 is set to include the area where the pillow used by the user is placed. Furthermore, detection area B402 is set near the center of bed 301, and detection area C403 and detection area D404 are set on both sides of bed 301. Furthermore, detection area E405 and detection area F406 are set in the area adjacent to the side of bed 301 (floor, etc.). As an example, information processing device 101 manages the ranges of the multiple detection areas using coordinate information indicating four points, the upper left, upper right, lower left, and lower right of the multiple detection areas. However, this is just an example, and information processing device 101 may manage an area surrounded by any number of points greater than or equal to three, as a detection area.

(About the criteria and content of the judgment)
The information processing device 101 has a function of detecting a specified temperature in multiple detection areas using temperature images acquired from the imaging device 102, and determining the condition of the user based on the detection area in which the specified temperature is detected or a change in the detection area.

For example, the information processing device 101 analyzes the acquired temperature image, and if the detection area contains a threshold or more of pixels within a preset temperature range, it determines that a specified temperature has been detected in the detection area.

In addition, the information processing device 101 prestores in a storage unit a correspondence between one or more detection areas where a predetermined temperature is detected and the state of the user, and determines the state of the user using the one or more detection areas where a predetermined temperature is detected and the above-mentioned correspondence.

Figure 5 is a diagram for explaining an example of the judgment conditions and judgment contents according to one embodiment. Figure 5 (A) shows an image of a temperature image of a user 501 when the user 501 is lying on a bed 301. In the example of Figure 5 (A), the user 501 is wearing clothes such as pajamas or loungewear, and the head 502, hands 503, feet 504, etc. are displayed in colors in the temperature range corresponding to the body temperature of the user 501.

In this way, when a predetermined temperature (e.g., 35°C to 39°C, etc.) equivalent to the body temperature of the user 501 is detected in the detection area A401, the information processing device 101 determines, for example, that the state of the user 501 is "asleep," which means that the user is lying in bed. This is because it is generally considered that there are few cases in which only the feet 504 and hands 503 of the user 501 are at the position of the pillow of the user 501.

With this method, for example, even if the user 501 is covered with a futon or blanket and the temperature of the user's 501's hands 503, feet 504, etc. cannot be detected, the information processing device 101 can determine that the user 501 is in the "asleep" state. Note that in this embodiment, "asleep" also includes a state in which the user 501 is awake and lying in bed 301.

Figure 5 (B) shows an image of a temperature image of user 501 when user 501 gets up from a "sleeping" state. In this way, when a predetermined temperature corresponding to the body temperature of user 501 is no longer detected in detection area A401 and a predetermined temperature is detected in detection area B402, the information processing device 101 determines that the state of user 501 is "getting up."

Figure 5 (C) shows an image of a temperature image of user 501 when the user 501 is detected to be in an "edge sitting" state, where the user 501 is sitting on the edge of the bed 301. For example, as shown in Figure 5 (C), when a predetermined temperature corresponding to user 501 is detected in detection area C403 for a predetermined period of time or longer, the information processing device 101 determines that the state of user 501 is "edge sitting".

In this way, the information processing device 101 determines the state of the user 501 based on the detection area among the multiple detection areas where a predetermined temperature corresponding to the user is detected, or based on a change in the detection area.

Note that the detection area shown in FIG. 4 and the judgment conditions and judgment contents described in FIG. 5 are merely examples. The information processing device 101 according to this embodiment may be configured to judge the state of a user using an acquired temperature image and one or more preset detection areas.

<Hardware Configuration>
(Hardware configuration of information processing device and information processing server)
Next, a description will be given of an example of the hardware configuration of the information processing device 101 and the information processing server 201. The information processing device 101 and the information processing server 201 have, for example, the hardware configuration of a computer 600 as shown in Fig. 6. Alternatively, the information processing device 101 and the information processing server 201 are configured by a plurality of computers 600.

Figure 6 is a diagram showing an example of the hardware configuration of a computer according to one embodiment. The computer 600 includes, for example, a CPU (Central Processing Unit) 601, a ROM (Read Only Memory) 602, a RAM (Random Access Memory) 603, a HD (Hard Disk) 604, a HDD (Hard Disk Drive) controller 605, a display 606, an external device connection I/F (Interface) 607, a network I/F 608, a keyboard 609, a pointing device 610, a DVD-RW (Digital Versatile Disk Rewritable) drive 612, a media I/F 614, and a bus line 615.

Of these, the CPU 601 controls the overall operation of the computer 600. The ROM 602 stores programs used to start up the computer 600, such as an IPL (Initial Program Loader). The RAM 603 is used, for example, as a work area for the CPU 601. The HD 604 stores various data, such as programs. The HDD controller 605 controls the reading and writing of various data from and to the HD 604 under the control of the CPU 601.

The display 606 displays various information such as a cursor, a menu, a window, text, or an image. The external device connection I/F 607 is an interface for connecting various external devices. The network I/F 608 is an interface for data communication using the communication network 10.

The keyboard 609 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 610 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 612 controls the reading and writing of various data from the DVD-RW 611, which is an example of a removable recording medium. Note that the DVD-RW 611 is not limited to the DVD-RW 611, and may be other recording media. The media I/F 614 controls the reading and writing (storing) of data from the media 613, such as a flash memory or a memory card. The bus line 615 includes an address bus, a data bus, various control signals, etc. for electrically connecting the above components.

(Hardware configuration of the imaging device)
7 is a diagram showing an example of a hardware configuration of an image capturing apparatus according to an embodiment. The image capturing apparatus 102 includes, for example, a control unit 710, a communication I/F 720, a thermal camera 112, a camera 122, and a bus 730.

The control unit 710 includes, for example, a CPU 711, a memory 712, and a storage device 713, and executes a predetermined program to realize each function of the imaging device 102. The CPU 711 is a calculation device that executes the predetermined program. The memory 712 includes, for example, a RAM (Random Access Memory), which is a volatile memory used as a work area for the CPU 711, and a ROM (Read Only Memory) that stores a program for starting the CPU 711 in advance. The storage device 713 is a large-capacity storage device that stores the predetermined program executed by the CPU 711 and various data, and is realized by, for example, an SSD (Solid State Drive).

The communication I/F 720 is a communication interface such as a wired LAN or a wireless LAN for connecting the image capturing device 102 to the communication network 10. The thermal camera 112 and the camera 122 correspond to the thermal camera 112 and the camera 122 described in FIG. 1. The bus 730 is commonly connected to each of the above components and transmits, for example, address signals, data signals, and various control signals.

<Functional configuration>
FIG. 8 is a diagram illustrating an example of a functional configuration of an information processing system according to an embodiment.

(Functional configuration of information processing device)
6 executes a predetermined program, the information processing device 101 realizes a communication unit 801, an acquisition unit 802, a conversion unit 803, a display control unit 804, a detection unit 805, a determination unit 806, a setting reception unit 807, a storage unit 808, etc. Note that at least a part of the above functional configurations may be realized by hardware.

The communication unit 801, for example, uses the network I/F 608 in FIG. 6 to connect the information processing device 101 to the communication network 10 and executes communication processing to communicate with the image capture device 102 and the information processing server 201, etc.

The acquisition unit 802 executes an acquisition process to acquire, via the communication unit 801, a captured image (first image) of the detection range 302 and a temperature image (second image) showing the temperature of the detection range 302 and having a different resolution from that of the captured image, both of which are transmitted by the imaging device 102.

The conversion unit 803 executes a conversion process to convert the resolution of the captured image and temperature image acquired by the acquisition unit 802 into the resolution of the display area in which the captured image or temperature image is displayed. For example, the conversion unit 803 enlarges the display image (hereinafter referred to as the display image) to be displayed in the display area, among the captured image and temperature image acquired by the acquisition unit 802, while maintaining the aspect ratio of the image so that the entire display image is displayed in the display area.

The display control unit 804 executes a display control process to selectively display in the display area the captured image or temperature image acquired by the acquisition unit 802 at the resolution of the display area. The display control unit 804 also displays various display screens including the display area on a display unit such as the display 606 in FIG. 6.

The detection unit 805 executes a detection process to detect a predetermined temperature in each detection area in the temperature image acquired by the acquisition unit 802, for example, as shown in FIG. 4.

For example, the memory unit 808 of the information processing device 101 stores area information 811 as shown in FIG. 9(A). In the example of FIG. 9(A), the area information 811 includes information such as the "detection area," "coordinate range," "temperature range," and threshold value (number of pixels) as items.

The "detection areas" correspond to the detection areas A401, B402, C403, etc. described in FIG. 4. For example, the detection area "A" corresponds to the detection area A401. The "coordinate range" is information indicating the coordinate range of each detection area, and is represented, for example, by the coordinates of the top left, top right, bottom left, and bottom right points of each detection area. The "temperature range" is information indicating the temperature range of a predetermined temperature detected in each detection area. The "threshold (number of pixels)" is a threshold for determining whether or not a temperature within the "temperature range" has been detected within each "detection area", and is represented, for example, by the number of pixels.

For example, when the "detection area" in the temperature image acquired by the acquisition unit 802 contains more than a "threshold (number of pixels)" of pixels within a pre-set "temperature range," the detection unit 805 outputs a detection result indicating that a predetermined temperature has been detected within the "detection area."

The determination unit 806 executes a determination process to determine the state of the user using the bed based on the detection result by the detection unit 805.

For example, the storage unit 808 of the information processing device 101 stores the judgment information 812 as shown in FIG. 9B. In the example of FIG. 9B, the judgment information 812 includes information such as "judgment content", "judgment condition", and "notification content" as items. The "judgment content" indicates the judgment content indicating the state of the user when the detection result by the detection unit 805 satisfies the "judgment condition". The "judgment condition" indicates the judgment condition for each "judgment content", and is represented, for example, by one or more detection areas where a predetermined temperature is detected, or a change in the detection area. The "notification content" indicates the notification content to be notified to a predetermined notification destination (for example, the information processing server 201, the nurse call system 202, etc.) when the detection result by the detection unit 805 satisfies the "judgment condition". In this way, the judgment information 812 stores the correspondence between the "judgment condition" (one or more detection results by the detection unit 805) and the "judgment content" indicating the state of the user.

The determination unit 806 determines the state of the user using the bed 301, for example, based on one or more detection results by the detection unit 805 and the above-mentioned correspondence relationship.

The setting reception unit 807 receives setting operations such as area information 811, judgment information 812, and various setting information 813 by, for example, an administrator, etc.

The storage unit 808 is realized, for example, by the program executed by the CPU 601 in FIG. 6, the HD 604, and the HDD controller 605, and stores various information and data, such as area information 811, judgment information 812, and setting information 813.

(Functional configuration of the imaging device)
7 executes a predetermined program, the image capturing device 102 realizes a communication unit 821, a temperature image transmission unit 822, a captured image transmission unit 823, a storage unit 824, etc. Note that at least a part of the above-mentioned functional configurations may be realized by hardware.

The communication unit 821, for example, uses the communication I/F 720 in FIG. 7 to connect the image capture device 102 to the communication network 10 and executes communication processing to communicate with the information processing device 101, etc.

The temperature image transmission unit 822, for example, uses the thermal camera 112 in FIG. 7 to capture temperature images of the bed 301 and the area within the detection range 302 around the bed 301, and transmits the captured temperature images to the information processing device 101 at a predetermined frame rate.

The captured image transmission unit 823, for example, uses the camera 122 in FIG. 7 to capture images of the bed 301 and the area within the detection range 302 around the bed 301, and transmits the captured images to the information processing device 101 at a predetermined frame rate.

The memory unit 824 is realized, for example, by a program executed by the CPU 711 in FIG. 7 and the storage device 713, and stores various information and data, for example, the identification information or display name of the imaging device 102 and the IP address of the information processing device 101.

Note that the functional configuration of the information processing system 100 shown in FIG. 8 is an example. For example, some or all of the functional configurations of the information processing device 101 may be possessed by the information processing server 201. Also, some of the functional configurations of the information processing device 101 may be possessed by the imaging device 102.

<Processing flow>
Next, the process flow of the information processing method according to this embodiment will be described.

[First embodiment]
(Display image selection process)
10 is a flowchart showing an example of a display image selection process according to the first embodiment. This process shows an example of a process in which the information processing device 101 selectively displays a temperature image or a captured image in a display area included in a display screen displayed on a display unit such as the display 606.

Note that at the start of the process shown in FIG. 10, the acquisition unit 802 of the information processing device 101 acquires the temperature image and the captured image transmitted from the imaging device 102, and the display control unit 804 causes the display unit to display, for example, a display screen 1200 as shown in FIG. 12.

In step S1001, the setting reception unit 807 of the information processing device 101 receives a selection operation to select whether to display a temperature image or a captured image in a specified display area on the display screen 1200 displayed by the information processing device 101.

In step S1002, the display control unit 804 of the information processing device 101 acquires the height and width of the display area 1205 displayed on the display screen 1200.

In step S1003, the conversion unit 803 of the information processing device 101 branches the process depending on whether a temperature image or a captured image was selected in step S1001. If a temperature image is selected, the conversion unit 803 transitions the process to step S1004. On the other hand, if a captured image is selected (if a temperature image is not selected), the conversion unit 803 transitions the process to step S1005.

When the process proceeds to step S1004, the conversion unit 803 obtains the height and width of the temperature image, for example, from the setting information 813, and enlarges the temperature image to fit the display area 1205.

On the other hand, when the process proceeds to step S1005, the conversion unit 803 obtains the height and width of the captured image, for example, from the setting information 813, and enlarges the captured image to fit the display area 1205.

In step S1006, the display control unit 804 of the information processing device 101 displays the image enlarged by the conversion unit 803 in step S1004 or step S1005 in the display area 1205.

Figure 11 is a diagram for explaining the enlargement process of the display image according to the first embodiment. For example, as shown in Figure 11, the resolution (width and height) of the display area 1205 is 640px (pixels) x 480px, the resolution of the temperature image 1101 is 160px x 120px, and the resolution of the captured image 1102 is 640px x 480px.

In this case, the horizontal ratio between the resolution of the display area 1205 and the resolution of the temperature image 1101 is 640px/160px=4 (times), and the vertical ratio is 480px×120px=4 (times). In this way, when the aspect ratio of the display area and the aspect ratio of the display image are equal, the conversion unit 803 enlarges the temperature image 1101 by four times in the horizontal direction and four times in the vertical direction. This allows the display control unit 804 to enlarge the temperature image 1101 so that the entire temperature image 1101 is displayed in the display area 1205 while maintaining the aspect ratio of the temperature image 1101.

The horizontal ratio between the resolution of the display area 1205 and the resolution of the captured image 1102 is 640px/640px=1 (times), and the vertical ratio is 480px/480px=1 (times). In this way, when the resolutions of the captured image 1102 and the display area 1205 are equal, the conversion unit 803 enlarges the captured image 1102 by 1 time, or does not enlarge (or reduce) the captured image 1102.

When the aspect ratio of the display area 1205 differs from the aspect ratio of the captured image 1102, the conversion unit 803 uses the smaller ratio to enlarge (or reduce) the captured image 1102 to the size of the display area 1205 while maintaining the aspect ratio of the captured image 1102.

For example, the resolution of the captured image 1102 is 280px x 160px. In this case, the horizontal ratio of the resolution of the captured image 1102 to the resolution of the display area 1205 is 640px/280px = 2.28 (times), and the vertical ratio is 480px. 160px = 3 (times). In this way, when the aspect ratio of the display area and the aspect ratio of the display image are different, the conversion unit 803 uses the smaller ratio (2.28 times) to enlarge the captured image 1102 by 2.28 times in the vertical direction and 2.28 times in the horizontal direction while maintaining the aspect ratio of the captured image 1102. As a result, the display control unit 804 can enlarge the captured image 1102 so that the entire captured image 1102 is displayed in the display area 1205 while maintaining the aspect ratio of the captured image 1102.

(Example of display screen)
12 is a diagram showing an example of a display screen according to the first embodiment. This diagram shows an example of a display screen 1200 that the display control unit 804 of the information processing device 101 causes to be displayed on a display unit such as the display 606.

In the example of FIG. 12, the display screen 1200 displays the judgment content 1201 indicating the user's condition judged by the judgment unit 806, and the user's information 1202 including the user's name, hospital room, bed name, etc., and a temperature image is displayed in the display area 1205.

As an example, the user information 1202 displays information such as the hospital room, bed, and name used by the user 501. This information may be acquired by the display control unit 804 from the setting information 813 stored in the storage unit 808, or may be acquired from the information processing server 201 or the nurse call system 202, for example.

When the "Temperature Image" button 1203 is selected, the display control unit 804, for example, by the process of FIG. 10, displays in the display area 1205 the temperature image of the user 501 acquired from the image capturing device 102, enlarged to the size of the display area 1205. At this time, the display control unit 804 may display a display element 1207 indicating the position of the bed 301, etc., in the temperature image displayed in the display area 1205 to make it easier to grasp the positional relationship of the user 501. Furthermore, the display control unit 804 may display a display element 1206 indicating the detection area where a predetermined temperature is detected, etc., in the temperature image displayed in the display area 1205.

When the "Photo taken" button 1204 is selected on the display screen 1200, the display control unit 804, for example, by the processing of FIG. 10, displays in the display area 1205 the photo taken of the user 501 acquired from the photographing device 102, enlarged to the size of the display area 1205.

Second Embodiment
For example, in the display screen 1200 shown in FIG. 12, an administrator or the like using the information processing device 101 can select the “Advanced Settings” button 1208 to display a setting screen for setting one or more detection areas, area information 811, judgment information 812, etc.

Fig. 13 is a diagram showing an example of a setting screen according to the second embodiment. In the example of Fig. 13, a setting screen 1300 displays a setting field 1310 for setting the range of one or more detection areas, a setting field 1320 for setting area information 811, and a setting field 1330 for setting judgment information.

When the "Captured Image" button 1311 is selected in the setting field 1310, the display control unit 804 of the information processing device 101 displays the captured image in the display area 1313, for example, by the processing of FIG. 10. This allows the administrator or the like to create, move, and change the size of a detection area, for example, by a drag operation, on the captured image displayed in the display area 1313.

When the "Temperature Image" button 1312 is selected in the setting field 1310, the display control unit 804 of the information processing device 101 displays a temperature image in the display area 1313, for example, by the process of FIG. 10. This allows the administrator or the like to set the temperature range and threshold (number of pixels) of the area information 811 while viewing the temperature image of the user 501 displayed in the display area 1313.

Thus, according to the second embodiment, in the information processing system 100 that detects information about a user based on a temperature image of the user using a specific piece of equipment (bed 301), it becomes easy for an administrator or the like to set up or change the system while looking at the display screen.

<Processing flow>
(Detection area setting process)
14 is a flowchart showing an example of a process for setting a detection area according to the second embodiment. In this example, it is assumed that the aspect ratio of the temperature image captured by the thermal camera 112 and the aspect ratio of the captured image captured by the camera 122 are the same, and that the image capturing device 102 is set in advance to capture the same detection range 302.

In step S1401, the information processing system 100 is assumed to display the captured image acquired by the acquisition unit 802 in the display area 1313 of the setting screen 1300, for example, by the processing of FIG. 10.

In step S1402, the setting reception unit 807 of the information processing device 101 receives a setting operation for the detection area on the shooting screen displayed in the display area 1313.

In step S1403, the setting reception unit 807 obtains the width and height of the display area 1313, for example, from the display control unit 804.

In step S1404, the setting reception unit 807 obtains the width and height of the captured image, for example, from the setting information 813 stored in the memory unit 808.

In step S1405, the conversion unit 803 calculates the magnification of the resolution between the display area 1313 and the captured image. For example, as shown in FIG. 15, if the resolution of the display area 1313 is 640px x 480px and the resolution of the captured image 1510 is 160px x 120px, the magnification of the resolution between the display area 1313 and the captured image is 4 times.

In step S1406, the conversion unit 803 calculates the coordinates of the detection area on the captured image from the coordinates of the detection area set in step S1402 and the magnification calculated in step S1405. For example, as shown in FIG. 15, the detection area on the display area 1313 is represented by four coordinates (0,0), (0,Y), (X,0), (X,Y), and the magnification of the resolution of the display area 1313 and the captured image is 4 times. In this case, the setting reception unit 807 multiplies each of the four coordinates by 1/4 to calculate the coordinates (0,0), (0,Y/4), (X/4,0), (X/4,Y/4) of the detection area on the captured image 1510.

In step S1407, the conversion unit 803 stores the calculated coordinates of the detection area on the captured image 1510 in, for example, the "coordinate range" of the area information 811 as shown in FIG. 9(A).

(Display processing of display image and detection area)
FIG. 16 is a flowchart illustrating an example of a display process of a display image and a detection area according to the second embodiment.

In step S1601, the setting reception unit 807 of the information processing device 101 receives a selection operation of the "Captured Image" button 1311 or the "Temperature Image" button 1312 on the setting screen 1300 as shown in FIG. 13.

In step S1602, the display control unit 804 of the information processing device 101 acquires, for example, the coordinates of the detection area on the captured image stored in the area information 811 in step S1407 of FIG. 14.

In step S1603, the display control unit 804 acquires the width and height of the display area 1313 that the display control unit 804 is displaying on the setting screen 1300.

In step S1604, the display control unit 804 obtains the width and height of the captured image, for example, from the setting information 813 stored in the memory unit 808.

In step S1605, the conversion unit 803 calculates the resolution ratio between the display area 1313 and the captured image.

In step S1606, the conversion unit 803 calculates the coordinates of the detection area on the display area 1313 based on the calculated magnification. For example, the conversion unit 803 multiplies the coordinates of the detection area on the captured image by the calculated magnification to calculate the coordinates of the detection area on the display area 1313.

In step S1607, the display control unit 804 uses the calculated coordinates to draw the detection area in the display area 1313 of the setting screen 1300.

In step S1608, the display control unit 804 displays the display image (temperature image or captured image) selected in step S1601 in the display area 1313, for example, by the processing of FIG. 10.

In the above process, the setting reception unit 807 stores the coordinates of the "detection area on the captured image" in the "coordinate range" of the area information 811, but the setting reception unit 807 may also store the "coordinates of the detection area on the temperature image" in the "coordinate range" of the area information 811.

In this case, the setting reception unit 807, for example, in steps S1404 and S1405 of FIG. 14, acquires the width and height of the temperature image and calculates the resolution magnification of the display area 1313 and the temperature image. Also, in steps 1406 and S1407, the setting reception unit 807 calculates the coordinates of the detection area on the temperature image and stores the coordinates of the detection area on the temperature image in the "coordinate range" of the area information 811.

As described above, according to each embodiment of the present invention, in an information processing system that displays a temperature image of a user using a bed 301 on a display screen, it becomes easy for an administrator or the like to set up or change the system while viewing the display screen.

For example, an administrator or the like can display the captured image in the display area 1313 of the setting screen 1300 as shown in FIG. 13, thereby enabling the administrator or the like to set the position of the detection area while checking the actual position of the bed 301 or the position of the pillow, etc.

In addition, by displaying a temperature image in the display area 1313 of the setting screen 1300 as shown in FIG. 13, the administrator can set the "temperature range" or "threshold (number of pixels)" of the area information 811 while checking the body temperature of the user using the bed 301.

The bed 301 in each of the above embodiments is an example of a specified facility. The bed 301 may be, for example, a stretcher, an operating table, an examination table, or any other facility on which a user 501 lies, similar to a bed, provided in a facility such as a medical facility or a care facility.

<Additional Information>
Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), and a conventional circuit module designed to execute each function described above.

Furthermore, the group of devices described in the examples represents only one of multiple computing environments for implementing the embodiments disclosed herein. In one embodiment, the information processing device 101 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and perform the processes disclosed herein. Similarly, the information processing server 201 can include multiple computing devices configured to communicate with each other.

Furthermore, the information processing device 101 and the information processing server 201 can be configured to share the disclosed processing steps, such as those shown in Figures 10, 14, 16, etc., in various combinations. For example, a process performed by a specific unit can be performed by the information processing device 101. Similarly, the function of a specific unit can be performed by the information processing server 201. Furthermore, the functions of the information processing device 101 and the information processing server 201 may be integrated into one server device or may be divided among multiple devices.

REFERENCE SIGNS LIST 100 Information processing system 101 Information processing device 201 Information processing server 802 Acquisition unit 803 Conversion unit 804 Display control unit 805 Detection unit 806 Determination unit 807 Setting reception unit

JP 2019-030628 A

Claims (10)

an acquisition unit that acquires a first image of a specific facility used by a user and a second image having a different resolution from the first image and showing a temperature of the specific facility;
a conversion unit that converts a resolution of the first image and a resolution of the second image into a resolution of a display area in which the first image or the second image is displayed;
a display control unit that displays a setting screen including the display area for selectively displaying the first image or the second image converted to a resolution of the display area, the setting screen receiving a setting of a range of a detection area for detecting a predetermined temperature and a setting of a temperature range of the detection area;
a setting reception unit that receives a setting operation for setting the detection area and the temperature range from a user via the setting screen;
An information processing device having the above configuration. The information processing device according to claim 1, wherein the conversion unit enlarges the display image, of the first image and the second image, so that the entirety of the display image is displayed in the display area while maintaining the aspect ratio of the display image to be displayed in the display area. The information processing device according to claim 2, wherein, when the aspect ratio of the display image differs from the aspect ratio of the display area, the conversion unit enlarges the display image using the smaller of the vertical ratio and horizontal ratio between the resolution of the display image and the resolution of the display area. The information processing device according to any one of claims 1 to 3, wherein the setting reception unit receives the setting of a detection area for detecting the predetermined temperature on the first image when the display control unit is causing the first image to be displayed in the display area. The information processing device according to claim 4, wherein the display control unit displays the detection area set on the first image on the second image when the second image is displayed in the display area. a detection unit that detects whether or not a threshold or more of pixels within a preset temperature range are included in the detection area in the second image;
a determination unit that determines a state of the user based on a detection result by the detection unit;
The information processing device according to claim 5 . a storage unit that stores in advance a correspondence relationship between one or more detection results by the detection unit and a state of the user corresponding to the detection result;
The information processing apparatus according to claim 6 , wherein the determination unit determines the state of the user by using one or more detection results by the detection unit and the correspondence relationship. The information processing device according to any one of claims 5 to 7, wherein the display control unit causes the display unit to display a display screen that displays information indicating the user's state and the second image. an acquisition unit that acquires a first image of a specific facility used by a user and a second image having a different resolution from the first image and showing a temperature of the specific facility;
a conversion unit that converts a resolution of the first image and a resolution of the second image into a resolution of a display area in which the first image or the second image is displayed;
a display control unit that displays a setting screen including the display area for selectively displaying the first image or the second image converted to a resolution of the display area, the setting screen being used to set a range of a detection area for detecting a predetermined temperature and a temperature range of the detection area;
a setting reception unit that receives a setting operation for setting the detection area and the temperature range from a user via the setting screen;
An information processing system having the above configuration. On the computer,
A process of acquiring a first image of a specific facility used by a user and a second image having a different resolution from the first image and showing a temperature of the specific facility;
A process of converting a resolution of the first image and a resolution of the second image into a resolution of a display area in which the first image or the second image is displayed;
a process of displaying a setting screen including the display area for selectively displaying the first image or the second image converted to a resolution of the display area, the setting screen being used to set a range of a detection area for detecting a predetermined temperature and a temperature range of the detection area;
A process of receiving a setting operation for setting the detection area and the temperature range from a user via the setting screen;
A program to execute.

Images