JP7332310B2 - Information processing method, information processing apparatus, and information processing program - Google Patents

Description

The present disclosure relates to an information processing method, an information processing apparatus, and an information processing program that cause a device to execute a predetermined operation.

Conventionally, during a cleaning operation, image recognition is performed by comparing an image captured by an imaging unit with an image of a registered foreign object stored in a storage unit, and the registered foreign object stored in the storage unit is recognized. is known (see, for example, Patent Document 1). When the registered foreign object is recognized, the electric cleaning device controls the suction drive unit based on the control mode stored in the storage unit corresponding to the recognized foreign object, and the recognized foreign object is detected. An indication identifying what it is is displayed on the display screen.

Patent No. 5771885

However, in the conventional technology described above, the situation in which the article is damaged is not estimated, and further improvement is required.

The present disclosure has been made to solve the above problems, and an information processing method, an information processing device, and an information processing method that can cause a self-propelled device to perform a predetermined operation according to the situation in which damage to an article has occurred. It aims at providing an information processing program.

An information processing method according to an aspect of the present disclosure is an information processing method in an information processing apparatus, in which first information acquired by at least one of one or more sensors installed in a space is acquired, and Based on the first information, damage to an article existing in the space is detected. Based on the first information, the situation in which the article is damaged is estimated. Outputting second information for causing the running device to perform a predetermined operation.

ADVANTAGE OF THE INVENTION According to this indication, when the article|item which exists in space is damaged, a self-propelled apparatus can be made to perform predetermined|prescribed operation|movement according to the condition which damage|damage of the article|item generate|occur|produced.

1 is a diagram showing a configuration of a device control system according to Embodiment 1 of the present disclosure; FIG. 1 is a diagram illustrating a configuration of a server device according to Embodiment 1 of the present disclosure; FIG. 4 is a diagram showing an example of device operation information stored in a device operation information storage unit in Embodiment 1. FIG. 4 is a diagram showing an example of article information stored in an article information storage unit in Embodiment 1. FIG. 4 is a diagram showing an example of movie information stored in a service information storage unit in Embodiment 1. FIG. 4 is a diagram showing an example of restaurant information stored in a service information storage unit in Embodiment 1. FIG. 4 is a first flowchart for explaining the operation of the server device according to Embodiment 1 of the present disclosure; 7 is a second flowchart for explaining the operation of the server device according to Embodiment 1 of the present disclosure; FIG. 4 is a diagram for explaining the operation of the device in the first situation in which an article slips out of a person's hand during daily operation in the first embodiment; FIG. 10 is a diagram for explaining the operation of the device in the second situation in which a plurality of persons are fighting in the first embodiment; FIG. 10 is a diagram for explaining the operation of the device in the third situation in which a suspicious person is intruding in the first embodiment; FIG. 7 is a diagram showing the configuration of a first server device according to Embodiment 2 of the present disclosure; FIG. 10 is a diagram showing a configuration of a second server device according to Embodiment 2 of the present disclosure; FIG.

(Findings on which this disclosure is based)
In the conventional technology described above, images of foreign matter other than objects to be cleaned are registered in advance in a storage unit, and the captured image captured during the cleaning operation is compared with the image of the foreign matter to recognize the registered foreign matter. An indication specifying what the recognized foreign object is is displayed on the display screen. From the viewpoint of avoiding malfunction or damage to the vacuum cleaner, the foreign matter includes "plastic bags", "documents", "cords", "screws" and the like. From the viewpoint of securing and avoiding stains and damages, "documents", "micro SD cards", "banknotes" and "jewels" are mentioned. In other words, in the prior art, foreign matter other than the cleaning target is recognized, but the cleaning target such as a broken mug, for example, is not recognized.

Moreover, in the conventional technology, when an article is damaged, the situation in which the article is damaged cannot be estimated. Therefore, the prior art does not disclose or suggest making the vacuum cleaner perform a predetermined operation according to the situation in which the article is damaged.

In order to solve the above problems, an information processing method according to one aspect of the present disclosure is an information processing method in an information processing device, in which at least one of one or more sensors installed in a space acquires Acquiring first information, detecting damage to an article existing in the space based on the first information, estimating a situation in which the article is damaged based on the first information, and estimating the situation , outputs the second information for causing the self-propelled device to perform a predetermined operation in the space.

According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation in which the article has been damaged is estimated, and the space is detected according to the estimated situation. 2nd information for making a self-propelled device perform predetermined operation|movement is output inside. Therefore, when an article existing in the space is damaged, the self-propelled device can be caused to perform a predetermined operation according to the situation in which the article is damaged.

Moreover, in said information processing method, the said 2nd information may be the information for making the said self-propelled apparatus output a predetermined|prescribed sound in the said space according to the said presumed situation.

According to this configuration, it is possible to cause the self-propelled device to output a predetermined sound in the space depending on the situation in which the article is damaged. For example, if the article is damaged in a situation where a plurality of people are fighting, the self-propelled device can be caused to move while outputting a soothing voice to the people who are arguing. can.

Further, in the above information processing method, third information may be output for causing the presentation device to present information for changing the estimated situation.

According to this configuration, the third information for causing the presentation device to present information for changing the estimated situation is output, so it is possible to change the situation in which the article is damaged. For example, if the article is damaged in a situation where multiple people are arguing, it is possible to present information suitable for reconciling the quarreling people.

Further, in the above information processing method, the self-propelled device is a self-propelled cleaner, and the second information is transmitted to the self-propelled cleaner in the space according to the estimated situation. It may be information for cleaning damaged articles.

According to this configuration, the self-propelled device is a self-propelled cleaner, and the damaged article can be cleaned by the self-propelled cleaner in the space according to the situation in which the article is damaged. For example, when an article is damaged in a situation where the article slips out of a person's hand during daily operation, the self-propelled cleaner can be caused to clean the damaged article.

In the above information processing method, the estimated situation is a situation in which a suspicious person has entered the space, and the second information causes the self-propelled device to disturb the suspicious person in the space. It may be information for executing an operation.

According to this configuration, when a situation in which damage to an article occurs is a situation in which a suspicious person has entered the space, the self-propelled device can be caused to perform an operation to disturb the suspicious person in the space.

Further, in the above information processing method, image data obtained by imaging the suspicious person is acquired from an imaging device arranged in the space, and further, the acquired image data and the presence of the suspicious person are detected. Notification information for notification may be transmitted.

According to this configuration, the image data of the suspicious person is acquired from the imaging device arranged in the space, and the acquired image data and the notification information for notifying the presence of the suspicious person are transmitted. Therefore, it is possible to notify others that a suspicious person exists.

Further, in the above information processing method, when damage information indicating that the self-propelled device is damaged is acquired, fourth information for requesting repair of the self-propelled device may be output. good.

According to this configuration, when the self-propelled device is damaged, repair of the self-propelled device can be automatically requested.

Further, in the information processing method described above, fifth information may be output for causing the presentation device to present information about an article for suppressing occurrence of the estimated situation.

According to this configuration, the presentation device presents the information about the article for suppressing the occurrence of the situation in which the article is damaged, so it is possible to suppress the occurrence of the situation in which the article is damaged. For example, when the article is damaged in a situation where a suspicious person has entered the space, presenting information about the crime prevention item as an article for suppressing the occurrence of the situation where the suspicious person has entered the space. can be done.

Further, in the above information processing method, the one or more sensors include at least one of a microphone device and an imaging device installed in the space, and the first information includes sound data acquired by the microphone device and At least one of the image data acquired by the imaging device may be included, and the estimation of the situation may be based on at least one of the sound data and the image data to estimate the situation in which the article is damaged.

According to this configuration, based on at least one of the sound data acquired by the microphone device installed in the space and the image data acquired by the imaging device installed in the space, the situation in which the article has been damaged is highly evaluated. can be estimated with precision.

In the information processing method described above, the first information is obtained at predetermined time intervals, and the situation is estimated based on a predetermined time when the article is damaged. The situation in which the article is damaged may be estimated based on a plurality of pieces of first information acquired within a period of time.

According to this configuration, the first information is acquired at predetermined time intervals, and damage to the article occurs based on a plurality of pieces of first information acquired within a predetermined period of time based on the time at which the damage to the article occurred. Since the state of damage is estimated, for example, it is possible to estimate the state of damage to the article with higher accuracy using past image data for a predetermined period from the time when the damage to the article occurred.

An information processing apparatus according to another aspect of the present disclosure includes an acquisition unit that acquires first information acquired by at least one of one or more sensors installed in a space, and based on the first information, the space an estimating unit for estimating a situation in which the article is damaged based on the first information; and a self-propelled robot in the space according to the estimated situation. an output unit for outputting second information for causing the formula device to perform a predetermined operation.

According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation in which the article has been damaged is estimated, and the space is detected according to the estimated situation. 2nd information for making a self-propelled device perform predetermined operation|movement is output inside. Therefore, when an article existing in the space is damaged, the self-propelled device can be caused to perform a predetermined operation according to the situation in which the article is damaged.

An information processing program according to another aspect of the present disclosure acquires first information acquired by at least one of one or more sensors installed in a space, and based on the first information, exists in the space damage to the article is detected, based on the first information, the situation in which the article is damaged is estimated, and the self-propelled device is caused to perform a predetermined operation in the space according to the estimated situation. The computer is operated to output second information for.

According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation in which the article has been damaged is estimated, and the space is detected according to the estimated situation. 2nd information for making a self-propelled device perform predetermined operation|movement is output inside. Therefore, when an article existing in the space is damaged, the self-propelled device can be caused to perform a predetermined operation according to the situation in which the article is damaged.

Embodiments of the present disclosure will be described below with reference to the accompanying drawings. It should be noted that the following embodiments are examples that embody the present disclosure, and do not limit the technical scope of the present disclosure.

(Embodiment 1)
FIG. 1 is a diagram showing the configuration of a device control system according to Embodiment 1 of the present disclosure. As shown in FIG. 1 , the device control system includes a server device 3 , gateway (GW) 5 , first sensor 11 , second sensor 12 , self-propelled cleaner 21 and display device 22 .

Gateway 5 , first sensor 11 , second sensor 12 , self-propelled cleaner 21 and display device 22 are arranged inside house 10 . The gateway 5 is wirelessly communicatively connected to the first sensor 11, the second sensor 12, the self-propelled cleaner 21, and the display device 22. FIG. Also, the gateway 5 is connected to the server device 3 via the network 4 so as to be able to communicate with each other. Network 4 is, for example, the Internet.

The first sensor 11 , the second sensor 12 , the self-propelled cleaner 21 and the display device 22 are connected to the server device 3 via the gateway 5 so as to be able to communicate with each other. The first sensor 11 , the second sensor 12 , the self-propelled cleaner 21 and the display device 22 may be directly connected to the server device 3 without going through the gateway 5 so as to be able to communicate therewith.

FIG. 2 is a diagram showing a configuration of a server device according to Embodiment 1 of the present disclosure. The server device 3 is communicably connected to a sensor group 1 including a plurality of sensors placed inside the house 10 and a device group 2 including a plurality of devices placed inside the house 10 . The sensor group 1 includes various sensors such as a first sensor 11 and a second sensor 12 . The device group 2 includes various devices such as a self-propelled cleaner 21, a display device 22, an information device 23, and the like. Note that the gateway 5 is omitted in FIG.

The first sensor 11 is, for example, a microphone device, collects sounds inside the house 10 , and transmits the sound data to the server device 3 . The second sensor 12 is, for example, an imaging device, images the inside of the house 10 , and transmits the image data to the server device 3 . Note that the sensor group 1 may include a thermal image sensor and a vibration sensor. The sensors that constitute the sensor group 1 may be installed on the walls, floors, and furniture of the house 10 , or may be mounted on any device of the device group 2 .

The self-propelled cleaner 21 is an example of a self-propelled device, and performs suction cleaning while moving autonomously. The self-propelled cleaner 21 cleans the floor surface in the house 10 while autonomously moving on the floor surface. Normally, the self-propelled cleaner 21 is connected to a charging device (not shown) installed at a predetermined place in the house 10, and a cleaning start button provided on the main body of the self-propelled cleaner 21 is pressed. When pressed by the user, or when cleaning instruction information is received from the server device 3, it moves away from the charging device and starts cleaning. The self-propelled cleaner 21 includes a control section, a camera, a speaker, a drive section, a cleaning section, and a communication section (not shown).

The controller controls the cleaning operation by the self-propelled cleaner 21 . The drive unit moves the self-propelled cleaner 21 . The drive unit includes drive wheels for moving the self-propelled cleaner 21 and motors that drive the drive wheels. A driving wheel is installed at the bottom of the self-propelled cleaner 21 . The cleaning unit is installed at the bottom of the self-propelled cleaner 21 and sucks the object to be sucked.

The camera captures the traveling direction of the self-propelled cleaner 21 . The communication unit transmits image data captured by the camera to the server device 3 . The communication unit also receives cleaning instruction information for starting cleaning from the server device 3 . A control part starts cleaning, when cleaning instruction|indication information is received by the communication part. In addition, the cleaning instruction information includes a damaged position where the article 6 is damaged within the house 10 . A damaged location is a location where an article 6, such as a mug and a plate, is damaged. After moving to the damaged position, the self-propelled cleaner 21 captures an image of the object to be sucked existing at the damaged position, and transmits the captured image data to the server device 3 . Then, the self-propelled cleaner 21 cleans the damaged position and returns to the charging device.

The speaker outputs a predetermined sound depending on the situation in which the article has been damaged. For example, when the article is damaged in a situation in which a plurality of people are arguing, the speaker outputs a sound that soothes the quarreling persons.

In Embodiment 1, the device control system includes the self-propelled cleaner 21 as an example of a self-propelled device, but the present disclosure is not particularly limited to this, and a self-propelled device such as a pet robot can be used. A type robot may be provided as an example of a self-propelled device. The self-propelled robot has functions other than the cleaning function of the self-propelled cleaner 21 .

The display device 22 is arranged on the wall of a predetermined room inside the house 10 . Further, the equipment control system in Embodiment 1 may include a plurality of display devices 22 . A plurality of display devices 22 may be arranged, for example, on the wall of each room such as a living room, kitchen, bedroom, bath, toilet, and entrance. Also, the display device 22 may be an information terminal such as a smart phone or a tablet computer. The display device 22 includes a communication section, a display section, and an input section (not shown).

The communication unit receives information indicating the state of the equipment from the equipment placed in the house 10 . The communication unit also receives presentation information from the server device 3 .

The display unit is, for example, a liquid crystal display device, and displays various information. The display unit displays information about the equipment arranged in the house 10 . The display unit displays, for example, the current state of the washing machine or the current state of the air conditioner. Also, the display unit displays the presentation information received by the communication unit.

The input unit is, for example, a touch panel, and receives input operations by the user. The input unit receives an input of an operation instruction for the equipment arranged in the house 10 . The input unit receives, for example, an input of an operation instruction for an air conditioner or an input of an operation instruction for a lighting device. The communication unit transmits the operation instruction input by the input unit to the device.

The information device 23 is, for example, a smart phone, a tablet computer, a personal computer, or a mobile phone, and has a function of communicating with the outside. The information equipment 23 includes a communication unit (not shown). The communication unit receives image data of an image of a suspicious person who has entered the house 10 and notification information for notifying the presence of the suspicious person from the server device 3, and sends the received image data and notification information to the police. to the server device managed by The server device 3 transmits the image data and the notification information to the server device managed by the police via the information device 23 in the house 10, so that the police can identify the source of the image data and the notification information. .

In the first embodiment, the server device 3 transmits the image data and the notification information to the server device managed by the police via the information device 23, but the present disclosure is not particularly limited to this, and the server device 3 The device 3 may directly transmit the image data and notification information to the server device managed by the police without going through the information device 23 .

The device group 2 includes, in addition to the self-propelled cleaner 21, the display device 22 and the information device 23, a washing machine, a lighting device, an air conditioner, an electric shutter, an electric lock, an air purifier, and the like. Devices that make up the device group 2 include, for example, home appliances, information devices, and home equipment.

The server device 3 includes a communication section 31 , a processor 32 and a memory 33 .

The communication unit 31 includes a sensor data reception unit 311 , a control information transmission unit 312 , a presentation information transmission unit 313 and a notification information transmission unit 314 . The processor 32 includes a damage detection unit 321, a situation estimation unit 322, a damage position identification unit 323, an equipment operation determination unit 324, a damaged item identification unit 325, a substitute item identification unit 326, a presentation information generation unit 327, a service identification unit 328, and a notification An information generator 329 is provided. The memory 33 includes a device operation information storage section 331 , an article information storage section 332 and a service information storage section 333 .

The sensor data receiving unit 311 acquires sensor data (first information) acquired by at least one of the one or more sensors installed in the house 10 (space). The sensor data receiving unit 311 receives sensor data from each sensor of the sensor group 1 . The sensor data (first information) includes sound data acquired by the first sensor 11 (microphone device) and image data acquired by the second sensor 12 (imaging device). The sensor data receiving unit 311 receives sound data as sensor data from the first sensor 11 and receives image data as sensor data from the second sensor 12 .

Also, the sensor data receiving unit 311 receives sensor data from each device of the device group 2 . Devices in the device group 2 also include devices equipped with sensors. A device equipped with a sensor transmits sensor data to the server device 3 . As described above, self-propelled cleaner 21 is equipped with a camera. Therefore, the sensor data receiving unit 311 receives image data from the self-propelled cleaner 21 as sensor data. Also, the display device 22 may include a microphone and a camera, and the sensor data receiving unit 311 may receive sound data and image data from the display device 22 as sensor data.

The damage detection unit 321 detects damage to articles existing in the house 10 based on the sensor data (first information) received by the sensor data reception unit 311 . If the sound data received from the first sensor 11 includes the characteristics of the sound generated when the article is damaged, the damage detection unit 321 detects that the article is damaged. The memory 33 may pre-store, for example, frequency components of a plurality of breaking sounds such as the sound of breaking pottery or glass. The damage detection unit 321 compares the frequency components of the sound data received from the first sensor 11 with the frequency components of a plurality of damage sounds stored in the memory 33, and if the two frequency components match, the article is damaged. Detect damage.

Note that the damage detection unit 321 detects the sound received from the first sensor 11 using a prediction model machine-learned using the sound data at the time when the article was damaged and the fact that the article was damaged as teacher data. From the data, it may be inferred that damage to the item has occurred. In this case, the prediction model is stored in the memory 33 in advance.

Further, the damage detection unit 321 may detect that the article is damaged from the image data captured by the second sensor 12 . For example, the sensor data receiving unit 311 may acquire temporally continuous image data from the second sensor 12 . The damage detection unit 321 analyzes the acquired image data, and if the image data includes a state in which an article is dropped from the hand of a person in the house 10 and the article is broken on the floor surface, the article is damaged. You can detect what you have done.

Moreover, the damage detection unit 321 may detect damage to the article using sensor data from another sensor such as a vibration sensor. Further, the damage detection unit 321 may detect damage to the article using sensor data from a plurality of sensors of the sensor group 1 .

The situation estimation unit 322 estimates the situation in which the article is damaged based on the sensor data (first information). The situation estimation unit 322 estimates the situation in which the article is damaged based on at least one of the sound data and the image data. That is, the sensor data receiving unit 311 acquires image data at predetermined time intervals. The situation estimating unit 322 estimates the situation in which the article is damaged based on a plurality of image data acquired within a predetermined period starting from the time when the article was damaged. The damage detection unit 321 can identify the point in time when the article is damaged by recognizing the characteristic component of the sound of the article being damaged from the sound data. The situation estimating unit 322 estimates the situation in which the damage to the article occurred based on a plurality of image data acquired within a predetermined period in the past from the point in time when the specified damage to the article occurred.

For example, if an item slips out of a person's hand during the person's routine movement, the item will be damaged. Also, for example, when a plurality of persons are arguing, one of the plurality of persons throws the article, thereby damaging the article. Also, for example, when a suspicious person intrudes into the house 10, the suspicious person destroys the article, thereby damaging the article. For this reason, the situations in which an article is damaged include, for example, a first situation in which an article slips out of a person's hand during daily activities, a second situation in which a plurality of people are arguing, and a suspicious person breaking in. There is a third situation where The situation estimating unit 322 determines that the situation in which the damage to the article has occurred includes a first situation in which the article slips out of the hand of a person during daily activities, a second situation in which a plurality of people are fighting, and a suspicious person. Estimate which of the third situations you are intruding on.

The situation estimation unit 322 analyzes a plurality of image data immediately before the damage to the article occurred, and recognizes the person's hand and the article included in the plurality of image data. Then, when the article falls from the person's hand, the situation estimation unit 322 estimates that the situation in which the article is damaged is the first situation in which the article slips out of the person's hand during daily activities. do. Note that the situation estimation unit 322 acquires the sound data immediately before the damage to the article occurred. A first situation may be assumed in which the item has slipped out of the person's hand while in motion.

In addition, the situation estimation unit 322 analyzes a plurality of image data immediately before the damage to the article occurred, and recognizes the hands of the plurality of persons and the article included in the plurality of image data. Then, when one of the plurality of persons throws the article, the situation estimation unit 322 estimates that the situation in which the article is damaged is the second situation in which the plurality of persons are fighting. Note that the situation estimation unit 322 acquires the sound data immediately before the damage to the article occurred. may be estimated to be a second situation in which two persons are quarreling.

In addition, the situation estimation unit 322 acquires the sound data immediately before the damage to the article occurred. , a second situation in which a plurality of people are fighting. In addition, the situation estimation unit 322 may acquire a plurality of image data immediately before the damage to the article occurs, and recognize the movement of a fight between a plurality of persons included in the plurality of image data. Moreover, the situation estimation unit 322 may detect vibration generated when a plurality of people are arguing with a vibration sensor.

Furthermore, the situation estimation unit 322 analyzes a plurality of image data immediately before the damage of the article occurred, and recognizes the person included in the plurality of image data. Then, when a person who is not a pre-registered resident of the house 10 is recognized, the situation estimating unit 322 estimates that the situation in which the article is damaged is the third situation in which a suspicious person has entered. . If a person who is not the pre-registered resident of the house 10 is recognized and the pre-registered resident of the house 10 is not recognized, the situation estimation unit 322 determines that the situation in which the article has been damaged is You may presume that it is the 3rd situation where a suspicious person intrudes.

The situation estimating unit 322 uses a prediction model that is machine-learned using the image data immediately before the damage to the article occurred and the situation in which the damage to the article occurred as teacher data. From the image data immediately before the point in time, the situation in which the article was damaged may be estimated. Note that the prediction model is stored in the memory 33 in advance.

It should be noted that the situation in which the article is damaged in Embodiment 1 is not limited to the first to third situations described above.

The damaged position specifying unit 323 specifies the damaged position of the article in the house 10 . The memory 33 may pre-store, for example, a floor plan of the house 10 expressed in a two-dimensional coordinate space. Self-propelled cleaner 21 may create a floor plan by moving inside house 10 and transmit the created floor plan to server device 3 . The damage position specifying unit 323 specifies the coordinates on the floor plan of the source of the sound caused by the damage of the article as the damage position.

It should be noted that the damage location identification unit 323 can more accurately identify the source of the sound of the damage of the article by collecting the sound of the damage of the article with a plurality of microphones. Further, the damaged position specifying section 323 may specify the position where the article is damaged from the image data captured by the second sensor 12 .

The appliance operation determination unit 324 determines a predetermined operation to be executed by the self-propelled cleaner 21 (self-propelled device) in the house 10 (space) according to the situation estimated by the situation estimation unit 322 . Further, the device operation determination unit 324 may determine a predetermined operation to be executed by a device other than the self-propelled cleaner 21 according to the situation estimated by the situation estimation unit 322 . In addition, the device operation determination unit 324 determines a predetermined operation to be executed by the sensors forming the sensor group 1 according to the situation estimated by the situation estimation unit 322 as well as the devices forming the device group 2. good too.

The device operation information storage unit 331 stores in advance device operation information that associates a situation in which an article has been damaged and an operation to be executed by the device.

FIG. 3 is a diagram showing an example of device operation information stored in the device operation information storage unit in the first embodiment.

As shown in FIG. 3, an action to be executed by the device is associated with a situation in which an article has been damaged. In a first situation in which an article slips out of a person's hand during a routine operation, an action of causing the self-propelled cleaner 21 to suck up the damaged article and an action of causing the display device 22 to present a substitute for the damaged article are performed. are associated. In a second situation in which a plurality of people are arguing, the operation of moving the self-propelled cleaner 21 while outputting a soothing voice to the plurality of people who are arguing and displaying a restaurant suitable for reconciliation on the display device 22 is performed. Or an action of presenting a movie is associated. In addition, in the third situation where a suspicious person has entered, the operation of moving the self-propelled cleaner 21 while disturbing the feet of the suspicious person and the operation of causing the imaging device (second sensor 12) to image the suspicious person. and the operation of causing the information device 23 to transmit image data of an image of a suspicious person and notification information for notifying the presence of the suspicious person to the police.

Note that the operation of the device associated with the situation in which the article is damaged is not limited to the above.

The device operation determination unit 324 refers to the device operation information storage unit 331 and determines a predetermined operation to be executed by the device associated with the situation estimated by the situation estimation unit 322 .

When a first situation is estimated in which an article slips out of a person's hand during daily operation, the device operation determination unit 324 determines the operation of the self-propelled cleaner 21 to suck up the damaged article and the replacement of the damaged article. It determines the operation of the display device 22 that causes the item to be presented. Further, when a second situation in which a plurality of people are arguing is estimated, the device operation determination unit 324 operates the self-propelled cleaner 21 to move while outputting a soothing voice to the plurality of people who are arguing. , and the operation of the display device 22 to present restaurants or movies suitable for reconciliation. In addition, when the third situation in which a suspicious person has entered is estimated, the device operation determination unit 324 causes the self-propelled cleaner 21 to move while disturbing the feet of the suspicious person and to image the suspicious person. The operation of the imaging device (second sensor 12) and the operation of the information device 23 for transmitting the image data of the suspicious person and the notification information for notifying the presence of the suspicious person to the police are determined.

Also, the device operation determination unit 324 controls the operation of the self-propelled cleaner 21 . The appliance operation determination unit 324 generates control information (second information) for causing the self-propelled cleaner 21 (self-propelled device) to perform a predetermined operation in the space according to the estimated situation. Here, the control information is cleaning instruction information for causing the self-propelled cleaner 21 to clean damaged articles in the house 10 (space) according to the estimated situation. When the device operation determination unit 324 determines the operation of the self-propelled cleaner 21 to suck the damaged article, the device operation determination unit 324 moves the self-propelled cleaner 21 to the damage position identified by the damage position identification unit 323, Cleaning instruction information is generated for causing the self-propelled cleaner 21 to clean the damaged article at the position.

The control information transmitting unit 312 transmits control information (second information) for causing the self-propelled cleaner 21 (self-propelled device) to perform a predetermined operation in the house 10 (space) according to the estimated situation. to output The control information transmission unit 312 transmits the cleaning instruction information generated by the appliance operation determination unit 324 to the self-propelled cleaner 21 . Upon receiving the cleaning instruction information, the self-propelled cleaner 21 moves to the damaged position, captures an image of the suction target, which is a damaged article, at the damaged position, transmits the captured image data to the server device 3, and removes the suction target. suck things. The sensor data receiving unit 311 acquires information about the object to be sucked by the self-propelled cleaner 21 . For example, the information about the object to be sucked is information about the appearance of the object to be sucked. Information about the appearance includes an image captured by a camera provided in self-propelled cleaner 21 . The image includes an object to be sucked. The sensor data receiving unit 311 receives image data of an object to be sucked transmitted from the self-propelled cleaner 21 .

When the equipment operation determination unit 324 identifies the operation of the display device 22 for presenting a substitute for the damaged item, the damaged item identification unit 325 determines based on the image data including the object to be sucked received from the self-propelled cleaner 21. , to identify damaged articles constituted by objects to be aspirated. The object to be sucked is a damaged article. The damaged article identification unit 325 identifies damaged articles based on the appearance of the suction target. The damaged article identification unit 325 recognizes an image including an object to be sucked, and identifies a damaged article from the recognized image. The memory 33 may store in advance a table in which images of a plurality of articles are associated with respective names (product names) of the plurality of articles. The damaged article specifying unit 325 compares the image data of the object to be sucked with the images of a plurality of articles stored in the memory 33, and corresponds to the image of the article that partially matches the image of the object to be sucked. Identify the name of the item to which it is attached as the name of the damaged item.

The article information storage unit 332 stores article information about articles.

FIG. 4 is a diagram showing an example of article information stored in an article information storage unit in the first embodiment.

As shown in FIG. 4, the article information includes the article number for identifying the article, the article name, the article type, the article category, the article color, the article size, the article weight, the article material, and the article information. Includes price, manufacturer of the item and store where the item is sold. The product information storage unit 332 stores product numbers, product names, product types, product categories, product colors, product sizes, product weights, product materials, product prices, product manufacturers and product sales. Store article information associated with stores.

Note that the item information shown in FIG. 4 is an example, and may include other information such as an image of the item. Also, all the item information may be managed in one table, or may be managed in a distributed manner in a plurality of tables.

A substitute item identification unit 326 identifies a substitute item related to the damaged item based on the item information regarding the damaged item. The substitute item identification unit 326 acquires the item information of the damaged item from the item information storage unit 332 .

For example, the replacement item may be the same item as the damaged item. In this case, the substitute item identification unit 326 identifies the same item as the damaged item as the substitute item.

Also, for example, the substitute item may be an item with the same attributes as the damaged item. In this case, the substitute item identification unit 326 identifies an item having the same attributes as the damaged item as a substitute item. Attributes are, for example, the color, size, weight or material of the article. The substitute item identification unit 326 identifies an item having the same color, size, weight and material as the damaged item as a substitute item. Note that the substitute item identification unit 326 may identify an item that is identical to the damaged item in at least one of color, size, weight, and material as the substitute item.

Also, for example, the substitute item may be an item with attributes similar to those of the damaged item. In this case, the substitute item identification unit 326 identifies an item having attributes similar to those of the damaged item as a substitute item. Attributes are, for example, the color, size or weight of the item. The replacement item identifier 326 identifies items similar to at least one of the color, size and weight of the damaged item as replacement items. For example, a color similar to blue is bluish purple or the like, and a similar color is associated with each color and stored in advance. Also, an article having a size similar to the size of the damaged article is, for example, an article having a size within -1 cm to +1 cm of the width, depth and height of the damaged article. It should be noted that an article having a size similar to that of the damaged article is not limited to an article whose size is within a predetermined range as described above. It may be an article whose size is within a predetermined percentage range, such as an article whose size is within a percentage range. Further, an item of weight similar to the weight of the damaged item is, for example, an item of weight within the range of -10 grams to +10 grams of the weight of the damaged item. Note that an item having a weight similar to the weight of the damaged item is not limited to an item whose size is within a predetermined range as described above, but is within a range of -10% to +10% of the weight of the damaged item. It may be an article within a predetermined ratio of sizes, such as a heavy article.

Also, for example, the substitute item may be an item that has the same attributes as the damaged item and is made of a material that is stronger than the damaged item. In this case, the substitute item identification unit 326 identifies, as a substitute item, an item that has the same attributes as the damaged item and is made of a material that is stronger than the damaged item. An attribute is, for example, the color of the item. When the material of the damaged article is ceramics, the substitute article identification unit 326 identifies a metal article having the same color as the damaged article and having higher strength than the damaged article as a substitute article.

Note that the memory 33 may include a user information storage unit that stores user information about users. The user information includes a user ID for identifying the user, the user's name, the user's address, the user's date of birth, the user's blood type, the user's family structure, and the user's belongings. The substitute item identification unit 326 may identify a user who owns the damaged item identified by the damaged item identification unit 325 and acquire user information of the identified user from the user information storage unit. The substitute item identification unit 326 may then identify a substitute item related to the damaged item based on the item information regarding the damaged item and the user information regarding the owner of the damaged item.

For example, the user information includes owned item information indicating multiple owned items owned by the owner. The substitute item identification unit 326 identifies an item different in type from the damaged item from among the plurality of owned items indicated by the owned item information, and identifies at least the identified item from among the plurality of sales items being sold. An item that has the same one attribute and is of the same type as the damaged item may be identified as a substitute item. The user information may also include residence information indicating the location of the owner's residence. The substitute item identification unit 326 may identify substitute items that can be purchased at stores within a predetermined range from the location of the residence indicated by the residence information.

The presentation information generating section 327 generates presentation information regarding the substitute item identified by the substitute item identifying section 326 . The presentation information includes an image showing the appearance of the substitute item. The presentation information may also include an object image for ordering the substitute item along with an image showing the appearance of the substitute item.

The presentation information transmission unit 313 outputs presentation information regarding the substitute item identified by the substitute item identification unit 326 . That is, the presentation information transmission section 313 transmits the presentation information generated by the presentation information generation section 327 to the display device 22 . The display device 22 displays the received presentation information in a predetermined manner.

Further, when a second situation in which a plurality of people are arguing is estimated, the control information generated by the appliance operation determination unit 324 is determined according to the estimated situation. This is voice output instruction information for causing the machine 21 (self-propelled device) to output a predetermined voice. When the device operation determination unit 324 determines the operation of the self-propelled cleaner 21 to move while outputting a voice to soothe the quarreling people, the device operation determination unit 324 moves the self-propelled cleaner 21 to the damage position identified by the damage position identification unit 323 . While moving the cleaner 21, voice output instruction information is generated for moving the self-propelled cleaner 21 at the damaged position while outputting voices to soothe the quarreling people.

The control information transmission unit 312 transmits the audio output instruction information generated by the device operation determination unit 324 to the self-propelled cleaner 21 . When the self-propelled cleaner 21 receives the voice output instruction information, it moves from the charging position to the damaged position, and outputs a voice to soothe the quarreling persons at the damaged position. At this time, the self-propelled cleaner 21 may output a sound to soothe the quarreling persons and suck the object to be sucked (broken article). Then, after the suction of the object to be sucked is completed, or after a predetermined period of time has passed since the voice output was started, the self-propelled cleaner 21 returns to the charging position.

When the device operation determination unit 324 identifies the operation of the display device 22 for presenting a restaurant or a movie suitable for reconciliation, the service identification unit 328 refers to the service information storage unit 333 and presents the service to the quarreling persons. Identify a restaurant or movie to go to.

The service information storage unit 333 pre-stores service information including movie information about movies being shown and restaurant information about restaurants.

FIG. 5 is a diagram showing an example of movie information stored in the service information storage unit in the first embodiment.

As shown in FIG. 5, the movie information includes movie titles, movie genres, screening locations, screening schedules, and vacant seat information. The service information storage unit 333 stores movie information in which movie titles, movie genres, screening locations, screening schedules, and vacant seat information are associated with each other.

Note that the movie information shown in FIG. 5 is an example, and may include other information such as actors appearing in the movie. Also, all of the movie information may be managed in one table, or may be distributed and managed in a plurality of tables.

The service identifier 328 identifies movies to present to the quarreling persons. The service identifying unit 328 identifies, for example, movies that are of a genre suitable for reconciliation and that can be viewed. It is preferable that whether or not the movie is of a genre suitable for reconciliation is included in advance in the movie information. Also, a movie that can be viewed is a movie that starts after the current time and has vacant seats. In addition, if the memory 33 pre-stores user information about each of a plurality of people who are arguing, and the user information includes information about a favorite movie genre, the service identifying unit 328 refers to the user information. , may identify movies that correspond to preferred movie genres that are common to multiple persons in a fight.

FIG. 6 is a diagram showing an example of restaurant information stored in the service information storage unit in the first embodiment.

As shown in FIG. 6, the restaurant information includes restaurant name, cuisine genre, restaurant location, restaurant business hours, and vacant seat information. The service information storage unit 333 stores restaurant information in which restaurant names, cuisine genres, restaurant locations, restaurant business hours, and vacant seat information are associated with each other.

Note that the restaurant information shown in FIG. 6 is an example, and may include other information such as menus and images of the restaurant. Further, all of the restaurant information may be managed in one table, or may be distributed and managed in a plurality of tables.

The service identification unit 328 identifies restaurants to present to the quarreling persons. The service identification unit 328 identifies, for example, a restaurant that is suitable for reconciliation and where meals are available. It is preferable that whether or not the restaurant is of a genre suitable for reconciliation is included in the restaurant information in advance. A restaurant that is open for dining is a restaurant that is open at the current time and has vacant seats. In addition, if the memory 33 preliminarily stores user information regarding each of a plurality of persons who are quarreling, and the user information includes information regarding a favorite genre of cuisine, the service identifying unit 328 refers to the user information. , may identify a restaurant that corresponds to a favorite genre of cuisine that is common to multiple persons in a fight.

In the first embodiment, the service specifying unit 328 may specify either a restaurant or a movie to be presented to the quarreling persons, or may specify a restaurant or a movie to be presented to the quarreling persons. may identify both restaurants and movies in

Further, in Embodiment 1, when the second situation in which a plurality of people are fighting is estimated, the device operation determination unit 324 determines the operation of the display device 22 to present a restaurant or movie suitable for reconciliation. However, the present disclosure is not particularly limited to this. may determine the behavior of When the device operation determination unit 324 identifies the operation of the display device 22 for presenting a service suitable for reconciliation, the service identification unit 328 refers to the service information storage unit 333 and presents the service to the quarreling persons. identify the services of

The presentation information generation unit 327 generates presentation information (third information) for causing the display device 22 (presentation device) to present information for changing the estimated situation. The presentation information generation unit 327 generates presentation information regarding restaurants or movies suitable for reconciliation identified by the service identification unit 328 .

The presentation information transmission unit 313 outputs presentation information for causing the display device 22 (presentation device) to present information for changing the estimated situation. Here, the presentation information transmission unit 313 transmits the presentation information regarding the restaurant or movie identified by the service identification unit 328 to the display device 22 . That is, the presentation information transmission section 313 transmits the presentation information generated by the presentation information generation section 327 to the display device 22 . The display device 22 displays the received presentation information in a predetermined manner.

Further, when a third situation in which a suspicious person has entered is estimated, the control information generated by the appliance operation determination unit 324 is sent to the self-propelled cleaner 21 (self-propelled device) in the house 10 (space). Disturbing action instruction information for causing a suspicious person to perform a disturbing action. When the device operation determination unit 324 determines that the self-propelled cleaner 21 moves while disturbing the feet of the suspicious person, the device operation determination unit 324 moves the self-propelled cleaner 21 to the damage position identified by the damage position identification unit 323 . Disturbance operation instruction information is generated for causing the self-propelled cleaner 21 to move while disturbing the feet of the suspicious person at the damaged position.

The control information transmission unit 312 transmits the disturbance operation instruction information generated by the device operation determination unit 324 to the self-propelled cleaner 21 . When the self-propelled cleaner 21 receives the disturbing operation instruction information, it moves from the charging position to the damaged position, and moves while disturbing the feet of the suspicious person at the damaged position. At this time, the self-propelled cleaner 21 measures the distance to the suspicious person with a distance sensor and moves while maintaining a predetermined distance from the suspicious person. After the suspicious person has left the house 10 (space), the self-propelled cleaner 21 sucks the suction target (broken article) and returns to the charging position. Further, the self-propelled cleaner 21 may move to block the doorway of the house 10 in order to confine the suspicious person in the house 10 until the police arrive.

Further, when the action of causing the imaging device (second sensor 12) to capture an image of a suspicious person is specified, the sensor data receiving unit 311 captures an image of the suspicious person from the second sensor 12 arranged in the house 10 (space). Get image data. Sensor data receiving section 311 outputs the image data to notification information generating section 329 .

The notification information generation unit 329 generates notification information for notifying that a suspicious person exists. The notification information includes, for example, information indicating the existence of a suspicious person and the address of the residence 10 . The notification information generation unit 329 outputs the image data acquired by the sensor data reception unit 311 and notification information to the notification information transmission unit 314 .

When it is determined that the information device 23 should operate to transmit the image data of the suspicious person and the notification information for notifying the presence of the suspicious person to the police, the notification information transmission unit 314 sends the sensor data reception unit 311 and the notification information for notifying the existence of the suspicious person to the information device 23 . The information equipment 23 receives the image data and notification information from the server device 3, and transmits the received image data and notification information to the server device managed by the police.

Self-propelled cleaner 21 may transmit damage information indicating that self-propelled cleaner 21 has been damaged to server device 3 when detecting that self-propelled cleaner 21 has been damaged. The sensor data receiver 311 may receive damage information transmitted by the self-propelled cleaner 21 . The notification information generator 329 may generate notification information (fourth information) for requesting the manufacturer to repair the self-propelled cleaner 21 when the damage information is received. The notification information transmission unit 314 may transmit notification information for requesting the manufacturer to repair the self-propelled cleaner 21 to the information device 23 . The information equipment 23 may receive notification information for requesting the manufacturer to repair the self-propelled cleaner 21 from the server device 3 and transmit the received notification information to the server device managed by the manufacturer.

FIG. 7 is a first flowchart for explaining the operation of the server device according to the first embodiment of the present disclosure, and FIG. 8 is a first flowchart for explaining the operation of the server device according to the first embodiment of the present disclosure. 2 is a flowchart. Note that FIG. 7 illustrates an example of detecting damage based on sound data, but detection of damage may be performed based on other sensor data such as image data as described above.

First, the sensor data receiving unit 311 receives sound data as sensor data from the first sensor 11 (step S1).

Next, the damage detection unit 321 uses the sound data received by the sensor data reception unit 311 to determine whether or not damage to an article in the house 10 has been detected (step S2). At this time, if the frequency component of the sound data received from the sensor data receiving unit 311 matches the frequency component of the pre-stored sound data indicating damage to the article, the damage detection unit 321 detects that the article has been damaged. detect. Here, if it is determined that damage to the article has not been detected (NO in step S2), the process returns to step S1.

On the other hand, if it is determined that damage to the article has been detected (YES in step S2), the sensor data receiving unit 311 receives A plurality of image data are received as sensor data from the second sensor 12 (step S3). Note that the sensor data receiving unit 311 requests the second sensor 12 to transmit a plurality of pieces of image data acquired within a predetermined period in the past from the time when the occurrence of damage to the article was detected, and responds to the request. may receive a plurality of image data transmitted by the second sensor 12 . Further, the sensor data receiving unit 311 may periodically receive image data from the second sensor 12 and store the received image data in the memory 33 . Then, when it is detected that damage to the article has occurred, the situation estimation unit 322 stores a plurality of image data acquired within a predetermined period in the past from the time when the damage to the article was detected. 33 may be read.

Next, the situation estimating unit 322 estimates the situation in which the damage to the article occurred based on a plurality of image data acquired within a predetermined period in the past from the time when the damage to the article was detected ( step S4). In the first embodiment, the situation estimating unit 322 determines whether the situation in which the article has been damaged is a first situation in which the article slips out of a person's hand during a routine action, a second situation in which a plurality of people are fighting, and a second situation in which a plurality of people are fighting. It is estimated whether it is the 2nd situation or the 3rd situation where a suspicious person is intruding.

Next, the situation estimation unit 322 determines whether or not the estimation result is the first situation in which the article has slipped out of the person's hand during daily activities (step S5).

Here, when it is determined that the estimation result is the first situation in which the article slipped out of the person's hand (YES in step S5), the damaged position identifying unit 323 identifies the damaged position of the article in the house 10. (step S6).

Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the first situation estimated by the situation estimation unit 322 (step S7). Here, when the estimation result is the first situation in which the article slipped out of the hand of the person, the device operation determination unit 324 determines the operation of the self-propelled cleaner 21 to suck the damaged article and the replacement article for the damaged article. and the operation of the display device 22 to present the

Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position identified by the damaged position identifying unit 323, and causes the self-propelled cleaner 21 to clean the damaged article at the damaged position. cleaning instruction information is generated (step S8).

Next, the control information transmission unit 312 transmits the cleaning instruction information generated by the appliance operation determination unit 324 to the self-propelled cleaner 21 (step S9). The self-propelled cleaner 21 receives cleaning instruction information from the server device 3 and moves toward the damaged position included in the cleaning instruction information. When the self-propelled cleaner 21 reaches the damaged position, the camera captures an image of the object to be sucked, and transmits the captured image data to the server device 3 . The self-propelled cleaner 21 sucks the object to be sucked when image data including the object to be sucked is transmitted to the server device 3 .

Next, the sensor data receiving unit 311 receives image data including the object to be sucked from the self-propelled cleaner 21 as sensor data (step S10).

Next, the damaged article identification unit 325 identifies a damaged article composed of the suction target based on the image data including the suction target received from the self-propelled cleaner 21 (step S11). The damaged article identification unit 325 compares the images of a plurality of articles stored in advance with the image of the suction target included in the image data, and recognizes the damaged article composed of the suction target. For example, if the object to be sucked is a fragment of a ceramic mug, the damaged article identification unit 325 recognizes an image of the article that partially matches the image of the fragment included in the image data, and corresponds to the image of the recognized article. Identify damaged items as damaged items.

Next, the substitute item identification unit 326 acquires item information of the damaged item from the item information storage unit 332 (step S12).

Next, the substitute item identification unit 326 identifies a substitute item related to the damaged item based on the item information regarding the damaged item (step S13). For example, the substitute item identification unit 326 identifies the same item as the damaged item as the substitute item.

Next, the presentation information generation unit 327 generates presentation information regarding the substitute item identified by the substitute item identification unit 326 (step S14).

Next, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22 (step S15). The display device 22 receives the presentation information transmitted by the server device 3 and displays the received presentation information. The display device 22 displays presentation information while the object to be sucked is being sucked by the self-propelled cleaner 21 . Note that the display device 22 may display the presentation information when the self-propelled cleaner 21 starts sucking the object to be sucked. Further, the display device 22 may continue to display the presentation information even after the self-propelled cleaner 21 has finished sucking the object to be sucked.

FIG. 9 is a diagram for explaining the operation of the device in the first situation in which an article slips out of a person's hand during daily operation in the first embodiment.

When the article 6 slips out of the hand of the person 61 during daily operation and the article 6 is damaged, the self-propelled cleaner 21 moves to the damage position and sucks the damaged article 6 . In addition, the display device 22 installed in the room displays presentation information 221 including an image 222 for confirming whether or not to purchase a replacement item that is the same as the damaged item 6 .

As shown in FIG. 9, the presentation information 221 includes, for example, a sentence "Would you buy a new mug?" and an image 222 containing.

Note that the appliance operation determination unit 324 may generate presentation information for notifying the user of starting cleaning when generating the cleaning instruction information. In this case, the control information transmission unit 312 transmits the cleaning instruction information generated by the equipment operation determination unit 324 to the self-propelled cleaner 21, and transmits the presentation information generated by the equipment operation determination unit 324 to the display device 22. You may send. The display device 22 may display presentation information for notifying the user of starting cleaning. In this case, the presentation information includes, for example, a sentence "Are you okay? Not injured? I'm going to clean now."

On the other hand, if it is determined in step S5 of FIG. 7 that the estimation result is not the first situation in which the article slipped out of the person's hand (NO in step S5), the situation estimation unit 322 It is determined whether or not the second situation is one in which two persons are quarreling (step S16).

Here, when the estimation result is determined to be the second situation in which a plurality of people are fighting (YES in step S16), the damaged position specifying unit 323 specifies the damaged position of the article in the house 10. (Step S17).

Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the second situation estimated by the situation estimation unit 322 (step S18). Here, when the estimation result is the second situation in which a plurality of people are fighting, the device operation determination unit 324 causes the self-propelled cleaner 21 to move while outputting a soothing voice to the plurality of people who are fighting. and the operation of the display device 22 to present restaurants or movies suitable for reconciliation.

Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and moves the self-propelled cleaner 21 to the damaged position. A sound output instruction information for moving while outputting a sound soothing is generated (step S19).

Next, the control information transmission section 312 transmits the voice output instruction information generated by the device operation determination section 324 to the self-propelled cleaner 21 (step S20). When the self-propelled cleaner 21 receives the voice output instruction information, it moves from the charging position to the damaged position, and outputs a voice to soothe the quarreling persons at the damaged position. Then, the self-propelled cleaner 21 sucks the object to be sucked (broken article). After the suction of the object to be sucked is completed, or after a predetermined period of time has elapsed since the sound output was started, the self-propelled cleaner 21 returns to the charging position.

Next, the service identification unit 328 refers to the service information storage unit 333 and identifies restaurants to be presented to the quarreling persons (step S21). A service specifying unit 328 refers to pre-stored user information, specifies a genre of favorite food common to a plurality of people who are quarreling, and specifies restaurants corresponding to the specified genre and at which meals are available.

In the process of FIG. 8, the service identifying unit 328 identifies a restaurant to be presented to a plurality of people who are arguing, but the present disclosure is not particularly limited to this. A movie may be specified for presentation.

Next, the presentation information generation unit 327 generates presentation information regarding the restaurant identified by the service identification unit 328 as suitable for reconciliation (step S22).

Next, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22 (step S23). The display device 22 receives the presentation information transmitted by the server device 3 and displays the received presentation information. The display device 22 displays presentation information while the self-propelled cleaner 21 is outputting a sound. Note that the display device 22 may display the presentation information when the self-propelled cleaner 21 starts outputting the sound. Further, the display device 22 may continue to display the presentation information even after the self-propelled cleaner 21 finishes outputting the sound.

FIG. 10 is a diagram for explaining the operation of the device in the second situation in which a plurality of persons are fighting in the first embodiment.

When an article 6 is damaged while a plurality of persons 62 and 63 are arguing, the self-propelled cleaner 21 moves to the damaged position and outputs a voice soothing the plurality of persons 62 and 63 who are arguing. . In FIG. 10, for example, the self-propelled cleaner 21 outputs a voice saying "Okay, calm down". In addition, the display device 22 installed in the room displays presentation information 223 for presenting a restaurant suitable for reconciling the quarreling persons 62 and 63 .

As shown in FIG. 10, the presentation information 223 includes, for example, a sentence "Would you like to eat Italian food at M restaurant?" and a reservation button for reserving a restaurant. When the reservation button is pressed, the screen shifts to a reservation screen for reserving a restaurant.

On the other hand, in step S16 of FIG. 8, when it is determined that the estimation result is not the second situation in which a plurality of people are arguing (NO in step S16), the situation estimation unit 322 determines that the estimation result is a suspicious person. (step S24).

Here, if it is determined that the estimation result is not the third situation in which a suspicious person is intruding (NO in step S24), that is, if the situation in which the article is damaged cannot be estimated, the process is executed. finish. If it is determined that the estimation result is not the third situation in which a suspicious person is intruding, the device operation determination unit 324 may determine the operation of the self-propelled cleaner 21 to suck up the damaged article. .

On the other hand, when it is determined that the estimated result is the third situation in which a suspicious person has entered (YES in step S24), the damaged position specifying unit 323 specifies the damaged position of the article in the house 10 (step S25).

Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the third situation estimated by the situation estimation unit 322 (step S26). Here, if the estimation result is the third situation in which a suspicious person has intruded, the device operation determination unit 324 determines the operation of the self-propelled cleaner 21 to move while disturbing the feet of the suspicious person, and the operation of the information device 23 for transmitting the image data of the suspicious person and the notification information for notifying the existence of the suspicious person to the police. do.

Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and causes the self-propelled cleaner 21 to disturb the feet of the suspicious person at the damaged position. Disturbing action instruction information for moving while moving is generated (step S27).

Next, the control information transmission section 312 transmits the disturbance operation instruction information generated by the device operation determination section 324 to the self-propelled cleaner 21 (step S28). When the self-propelled cleaner 21 receives the disturbing operation instruction information, it moves from the charging position to the damaged position, and moves while disturbing the feet of the suspicious person at the damaged position. After the suspicious person has left the house 10 (space), the self-propelled cleaner 21 sucks the suction target (broken article) and returns to the charging position.

Next, the sensor data receiving unit 311 receives image data of an image of a suspicious person from the second sensor 12 arranged in the house 10 (space) (step S29).

Next, the notification information generator 329 generates notification information for notifying that a suspicious person exists (step S30).

Next, the notification information transmission unit 314 transmits the image data acquired by the sensor data reception unit 311 and the notification information generated by the notification information generation unit 329 to the information device 23 (step S31). The information equipment 23 receives the image data and notification information from the server device 3, and transmits the received image data and notification information to the server device managed by the police.

FIG. 11 is a diagram for explaining the operation of the device in the third situation in which a suspicious person is intruding in the first embodiment.

When the article 6 is damaged when the suspicious person 64 enters, the self-propelled cleaner 21 moves to the damage position and disturbs the foot of the suspicious person 64. - 特許庁In FIG. 11, for example, self-propelled cleaner 21 moves around suspicious person 64 while maintaining a predetermined distance from suspicious person 64 . The second sensor 12 also transmits image data of the suspicious person 64 to the server device 3 . Further, the information device 23 receives image data of the suspicious person 64 and notification information for notifying the presence of the suspicious person 64 from the server device 3, and the received image data and notification information are managed by the police. Send to the server device.

Further, after the information device 23 has transmitted the image data and the notification information to the server device managed by the police, the display device 22 installed in the room displays presentation information 224 for presenting that the police have been notified. good too. As shown in FIG. 11, the presentation information 224 includes, for example, the text "I reported to the police."

In this way, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation in which the damage to the article occurred is estimated, and depending on the estimated situation, in the space Second information is output for causing self-propelled cleaner 21 to perform a predetermined operation. Therefore, when an article existing in the space is damaged, the self-propelled cleaner 21 can be caused to perform a predetermined operation according to the situation in which the article is damaged.

Note that the presentation information generation unit 327 may generate presentation information (fifth information) for causing the display device 22 (presentation device) to present information about an article for suppressing the occurrence of the estimated situation. In addition, the presentation information transmission unit 313 transmits (outputs) presentation information (fifth information) for causing the display device 22 (presentation device) to present information about an article for suppressing the occurrence of the estimated situation. good too. For example, when a third situation in which a suspicious person is intruding is estimated, the presentation information generation unit 327 may generate presentation information regarding security supplies and transmit the information to the display device 22 .

(Embodiment 2)
The equipment control system in Embodiment 1 includes one server device, but the equipment control system in Embodiment 2 includes two server devices.

FIG. 12 is a diagram showing the configuration of a first server device according to Embodiment 2 of the present disclosure, and FIG. 13 is a diagram showing the configuration of a second server device according to Embodiment 2 of the present disclosure.

The device control system in Embodiment 2 includes a first server device 3A, a second server device 3B, a gateway 5 (not shown), a first sensor 11, a second sensor 12, a self-propelled cleaner 21, a display device 22 and An information device 23 is provided. In the second embodiment, the same reference numerals are assigned to the same configurations as in the first embodiment, and the description thereof is omitted. The sensor group 1 includes various sensors such as a first sensor 11 and a second sensor 12 . The device group 2 includes various devices such as a self-propelled cleaner 21, a display device 22, an information device 23, and the like. 12 and 13, the gateway 5 is omitted.

The first server device 3A is communicably connected to the sensor group 1, the device group 2, and the second server device 3B via a network. Also, the second server device 3B is communicably connected to the device group 2 and the first server device 3A via a network.

The first server device 3A is operated by, for example, a platformer. The second server device 3B is operated by, for example, a third party.

The first server device 3A includes a communication section 31A, a processor 32A and a memory 33A.

The communication unit 31A includes a sensor data reception unit 311, a control information transmission unit 312, a notification information transmission unit 314, a damaged article information transmission unit 315, and an equipment operation information transmission unit 316. The processor 32A includes a damage detection unit 321, a situation estimation unit 322, a damage position identification unit 323, an appliance operation determination unit 324, a damaged article identification unit 325, and a notification information generation unit 329. The memory 33A includes a device operation information storage section 331. FIG.

The damaged item information transmission unit 315 transmits damaged item information indicating the damaged item identified by the damaged item identification unit 325 to the second server device 3B.

The device operation information transmission section 316 transmits to the second server device 3B device operation information indicating the operation of the display device 22 for presenting the restaurant or movie suitable for reconciliation determined by the device operation determination section 324 .

The second server device 3B has a communication section 31B, a processor 32B and a memory 33B.

The communication unit 31B includes a presentation information transmission unit 313, a damaged article information reception unit 317, and an equipment operation information reception unit 318. The processor 32B includes a substitute item identification unit 326 and a presentation information generation unit 327. FIG. The memory 33B includes an article information storage section 332. FIG.

The damaged article information receiving section 317 receives the damaged article information transmitted by the first server device 3A. The substitute item identification unit 326 identifies substitute items related to the damaged item based on the damaged item information received by the damaged item information receiving unit 317 .

The device operation information receiving section 318 receives the device operation information transmitted by the first server device 3A. The service identifying unit 328 refers to the service information storage unit 333 when the equipment operation information indicating the operation of the display device 22 for presenting restaurants or movies suitable for reconciliation is received by the equipment operation information receiving unit 318, and determines whether the service identifying unit 328 is fighting. identify a restaurant or movie to present to multiple persons in the

In the second embodiment, the first server device 3A transmits the damaged article information and the device operation information to the second server device 3B, but the present disclosure is not particularly limited to this, and the second server device 3B sends a request for damaged article information and device operation information to the first server device 3A, and the first server device 3A sends the damaged article information and device operation information to the second server device 3B in response to the request. good.

Further, in the second embodiment, the device control system may include a plurality of second server devices 3B.

In each of the above-described embodiments, each component may be configured by dedicated hardware, or realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Some or all of the functions of the device according to the embodiment of the present disclosure are typically implemented as an LSI (Large Scale Integration), which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them. Further, circuit integration is not limited to LSIs, and may be realized by dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connections and settings of circuit cells inside the LSI may be used.

Also, some or all of the functions of the device according to the embodiment of the present disclosure may be implemented by a processor such as a CPU executing a program.

In addition, the numbers used above are all examples for specifically describing the present disclosure, and the present disclosure is not limited to the illustrated numbers.

In addition, the order in which each step shown in the above flowchart is executed is for illustrative purposes in order to specifically describe the present disclosure, and may be an order other than the above as long as the same effect can be obtained. . Also, some of the above steps may be executed concurrently (in parallel) with other steps.

The information processing method, information processing apparatus, and information processing program according to the present disclosure can cause the self-propelled device to perform a predetermined operation according to the situation in which the article is damaged. It is useful as an information processing method, an information processing apparatus, and an information processing program to be executed.

1 sensor group 2 device group 3 server device 3A first server device 3B second server device 4 network 5 gateway 6 article 10 house 11 first sensor 12 second sensor 21 self-propelled cleaner 22 display device 23 information device 31, 31A , 31B communication unit 32, 32A, 32B processor 33, 33A, 33B memory 311 sensor data reception unit 312 control information transmission unit 313 presentation information transmission unit 314 notification information transmission unit 315 damaged article information transmission unit 316 equipment operation information transmission unit 317 damage Item information receiving unit 318 Equipment operation information receiving unit 321 Damage detection unit 322 Situation estimation unit 323 Damage position identification unit 324 Equipment operation determination unit 325 Damaged item identification unit 326 Substitute item identification unit 327 Presentation information generation unit 328 Service identification unit 329 Notification information Generation unit 331 Device operation information storage unit 332 Goods information storage unit 333 Service information storage unit

Claims (12)

An information processing method in an information processing device,
Acquiring first information acquired by at least one of the one or more sensors installed in the space;
Detecting damage to an article existing in the space based on the first information,
Based on the first information, estimate the situation in which the damage to the article occurred,
Output second information for causing the self-propelled device to perform a predetermined operation in the space according to the estimated situation,
The second information includes information for moving the self-propelled device to the damage position of the article, causing the self-propelled device to image the damaged article, and transmitting the image data taken,
Acquiring the image data of the damaged article transmitted by the self-propelled device,
Identifying the damaged article based on the image data,
identifying a replacement item that has the same or similar attributes as the identified damaged item;
Outputting sixth information for causing a presentation device to present information about the identified alternative product;
Information processing methods. The second information is information for causing the self-propelled device to output a predetermined sound in the space according to the estimated situation,
The information processing method according to claim 1. Furthermore, outputting third information for causing a presentation device to present information for changing the estimated situation;
3. The information processing method according to claim 1 or 2. The self-propelled device is a self-propelled vacuum cleaner,
The second information is information for causing the self-propelled cleaner to clean the damaged article in the space according to the estimated situation.
The information processing method according to any one of claims 1 to 3. the estimated situation is a situation in which a suspicious person has entered the space;
The second information is information for causing the self-propelled device to perform an action to disturb the suspicious person in the space.
The information processing method according to any one of claims 1 to 4. Further, acquiring image data of the suspicious person from an imaging device arranged in the space,
Further, transmitting the obtained image data and notification information for notifying that the suspicious person exists.
6. The information processing method according to claim 5. Furthermore, when damage information indicating that the self-propelled device is damaged is obtained, outputting fourth information for requesting repair of the self-propelled device,
7. The information processing method according to claim 5 or 6. Furthermore, outputting fifth information for causing a presentation device to present information about an article for suppressing the occurrence of the estimated situation,
The information processing method according to any one of claims 1 to 7. The one or more sensors include at least one of a microphone device and an imaging device installed in the space,
The first information includes at least one of sound data acquired by the microphone device and image data acquired by the imaging device,
estimating the situation is based on at least one of the sound data and the image data, estimating the situation in which the article has been damaged;
The information processing method according to any one of claims 1 to 8. Acquisition of the first information includes acquiring the first information at predetermined time intervals,
estimating the situation based on a plurality of pieces of first information acquired within a predetermined period of time based on the point in time when the damage to the article occurred, estimating the situation in which the damage to the article occurred;
The information processing method according to any one of claims 1 to 9. a first acquisition unit that acquires first information acquired by at least one of the one or more sensors installed in the space;
a detection unit that detects damage to an article existing in the space based on the first information;
an estimating unit for estimating a situation in which damage to the article has occurred based on the first information;
A first output unit that outputs second information for causing the self-propelled device to perform a predetermined operation in the space according to the estimated situation;
The second information includes information for moving the self-propelled device to the damage position of the article, causing the self-propelled device to image the damaged article, and transmitting the image data taken,
a second acquisition unit that acquires the image data of the damaged article transmitted by the self-propelled device;
a damaged article identifying unit that identifies the damaged article based on the image data;
a substitute item identification unit that identifies a substitute item having the same or similar attributes as the identified damaged item;
a second output unit that outputs sixth information for causing a presentation device to present information about the identified alternative product;
Information processing device. Acquiring first information acquired by at least one of the one or more sensors installed in the space;
Detecting damage to an article existing in the space based on the first information,
Based on the first information, estimate the situation in which the damage to the article occurred,
Output second information for causing the self-propelled device to perform a predetermined operation in the space according to the estimated situation ,
The second information includes information for moving the self-propelled device to the damage position of the article, causing the self-propelled device to image the damaged article, and transmitting the image data taken,
Acquiring the image data of the damaged article transmitted by the self-propelled device,
Identifying the damaged article based on the image data,
identifying a replacement item that has the same or similar attributes as the identified damaged item;
An information processing program that causes a computer to output sixth information for causing a presentation device to present information about the identified alternative article .

Images