JP5771885B2 - Electric vacuum cleaner - Google Patents

Description

  The present invention relates to a vacuum cleaner.

  In recent years, very small-sized cameras such as CCD (Charge Coupled Device) image sensors and CMOS (Complementary Metal Oxide Semiconductor) image sensors have become widespread, and this small camera is placed near the suction port and sucked in. There has been proposed a vacuum cleaner that can check things.

  For example, Patent Document 1 (Japanese Patent Application Laid-Open No. 2003-164401) describes a vacuum cleaner in which a CCD camera is arranged near the suction port and a display unit for displaying a captured image of the CCD camera is provided near the handle. Yes. In the electric vacuum cleaner described in Patent Document 1, the user can perform cleaning while looking at the captured image of the CCD camera displayed on the display screen of the display unit, and works while confirming the dust to be sucked in. be able to.

  Further, the electric vacuum cleaner described in Patent Document 1 has a function of detecting and recording (storing) an object presumed to be a noble metal by measuring the brightness of an image captured by a CCD camera. Yes. The user can confirm the precious metal that has been accidentally sucked in by performing a reproduction instruction operation after cleaning and reproducing the recorded image and gazing at the reproduced image.

  In Patent Document 2 (Japanese Patent Laid-Open No. 2006-141836), captured image data output from an imaging unit such as a CCD camera that captures an object sucked from a suction head is updated and recorded at regular intervals. In addition, there is described a vacuum cleaner that records captured image data when it is detected that an object of a certain size or larger is sucked.

  In the electric vacuum cleaner of Patent Document 2, when the user feels that an object other than dust is sucked during the suction operation in the vacuum cleaner, the suction operation is stopped by operating the suction stop button. And in the vacuum cleaner of this patent document 2, in response to a user's operation for image reproduction, captured image data is reproduced and a captured image is displayed on a display monitor. The user can check what is accidentally sucked into the vacuum cleaner by gazing at the captured image displayed on the display monitor.

JP 2003-164401 A JP 2006-141836 A

  However, in the case of the electric vacuum cleaner of Patent Document 1, it is possible to confirm the presence of dust by imaging the top of the bag with a camera. However, during the cleaning operation such as floor cleaning, the user looks at the display screen. However, the mode of cleaning is unnatural, and it is normal to clean while looking at the vicinity of the suction portion where the suction port is located. For this reason, foreign substances other than cleaning objects, such as precious metals, cannot be recognized in advance and cannot be sucked, and cannot be denied.

  And in the case of the vacuum cleaner of patent document 1, like the vacuum cleaner of patent document 2, when a user feels that the object other than dust was sucked accidentally during the suction operation in the vacuum cleaner, Operate the suction stop button to stop the suction operation, and then perform an image playback operation to display the captured image on the display screen, and gaze at the reproduced captured image. It is necessary to confirm what is.

  That is, in the case of the electric vacuum cleaners of Patent Document 1 and Patent Document 2 described above, when the user feels that he / she has sucked an object other than dust during the cleaning operation, the user presses a stop button or the like. It is necessary to temporarily stop the cleaning operation of the electric vacuum cleaner, and then perform a reproduction operation of the captured image such as a playback button so that the captured image is displayed on the display screen. It takes a lot of time to check for foreign matter.

  In addition, the user needs to look at the captured image by sequentially searching the display screen being played, find out the captured image of the foreign object, and determine what the foreign object is. The confirmation work is troublesome.

  An object of this invention is to provide the electric cleaning apparatus which can improve the above problem.

In order to solve the above problems, the present invention provides:
A suction part having a suction port for sucking a cleaning object;
A suction drive unit for sucking the cleaning object from the suction port and taking it into a dust collection chamber;
An imaging unit for imaging foreign objects other than the cleaning object and the cleaning object;
A storage unit;
A display unit comprising a display screen;
The foreign matter image, foreign matter specifying information for displaying what the foreign matter is, and a control mode for determining whether or not the foreign matter is sucked by the suction driving unit are stored in the storage unit in association with each other. Registration means for registering ,
During the cleaning operation, the captured image captured by the imaging unit, performs image recognition by comparing the image of the registered the foreign substance stored in the storage unit, stored in the storage unit the An image recognition unit for recognizing the registered foreign matter,
When the image recognition unit recognizes the registered foreign object, the suction drive unit is controlled based on the control mode stored in the storage unit corresponding to the recognized foreign object. Control means for displaying on the display screen a display for identifying what the recognized foreign matter is ;
An electric cleaning device is provided.

  According to the present invention having the above configuration, the image recognition unit recognizes the cleaning object and the foreign object other than the cleaning object separately. Then, an image of the foreign matter recognized by the image recognition unit, for example, a thumbnail image of a captured image of the foreign matter is displayed on the display screen of the display unit.

  Therefore, the user can display the image automatically displayed on the display screen of the display unit without performing a reproduction operation of the captured image or without looking at the reproduced captured image. A foreign object recognized by the vacuum cleaner can be confirmed by just looking.

In the above vacuum cleaner,
It is preferable to include a drive control unit that supplies a control signal to the suction drive unit in accordance with the image recognition result in the image recognition unit during a cleaning operation.

  According to this invention, based on the image recognition result, a drive control signal such as whether to stop the suction drive unit or to continuously perform a suction operation is supplied. That is, even if it is a foreign object, a drive control signal corresponding to the recognized foreign object can be generated and the suction drive unit can be appropriately driven and controlled.

In the above vacuum cleaner,
The display control means displays an image of the foreign matter on the display screen when the image recognition unit recognizes the foreign matter during a cleaning operation.
It is preferable to further comprise means for notifying an alarm in conjunction with the display of the foreign object image.

  According to this invention, when a foreign object is recognized by image recognition during the cleaning operation, an alarm is notified in synchronization with the image of the recognized foreign object being displayed on the display screen.

  Therefore, the user knows that the electric vacuum cleaner has recognized the foreign object by the alarm, and can recognize what the foreign object is by viewing the display image on the display screen. For this reason, the user can appropriately determine whether the cleaning can be continued or should be stopped, and can respond quickly.

In the above vacuum cleaner,
As the imaging unit, comprising a first imaging unit disposed in the vicinity of the suction port,
The foreign matter corresponding to the image displayed on the display screen may be configured to be sucked from the suction port.

  According to the present invention, the user can confirm the foreign matter sucked from the suction port and existing in the dust collecting chamber without performing the reproduction operation only by looking at the image displayed on the display screen.

In the above vacuum cleaner,
The image of the foreign matter displayed on the display screen is preferably accompanied by an additional display indicating whether the foreign matter is sucked from the suction port or not sucked from the suction port.

  Further, according to the present invention, for example, the foreign matter that has not been sucked in because it is large from the suction port, and the foreign matter that is sucked from the suction port and present in the dust collection chamber, can be seen by simply viewing the additional display of the image displayed on the display screen. You can confirm it.

  According to the electric vacuum cleaner of the present invention, the image is automatically displayed on the display screen of the display unit without performing a reproduction operation of the captured image or without looking at the reproduced captured image. The foreign object recognized by the vacuum cleaner can be confirmed simply by looking at the image.

It is a figure which shows the structural example of 1st Embodiment of the vacuum cleaner by this invention. It is a block diagram which shows the electrical structural example in 1st Embodiment of the vacuum cleaner by this invention. It is a figure used in order to demonstrate the principal part of the structural example of 1st Embodiment of the vacuum cleaner by this invention. It is a figure for demonstrating the example of a display of the display screen in the structural example of 1st Embodiment of the vacuum cleaner by this invention. It is a figure which shows the flowchart for demonstrating the process operation example in 1st Embodiment of the vacuum cleaner by this invention. It is a figure which shows the structural example of the self-propelled type vacuum cleaner which comprises 2nd Embodiment of the vacuum cleaner by this invention. It is a block diagram which shows the electrical structural example of 2nd Embodiment of the vacuum cleaner by this invention. It is a figure which shows the flowchart for demonstrating the processing operation example of the self-propelled vacuum cleaner which comprises 2nd Embodiment of the vacuum cleaner by this invention. It is a figure which shows the flowchart for demonstrating the processing operation example of the mobile telephone terminal as an example of the portable apparatus which comprises 2nd Embodiment of the vacuum cleaner by this invention. It is a block diagram which shows the electrical structural example of 3rd Embodiment of the vacuum cleaner by this invention. It is a figure which shows a part of flowchart for demonstrating the processing operation example of the self-propelled vacuum cleaner which comprises 3rd Embodiment of the vacuum cleaner by this invention. It is a figure which shows a part of flowchart for demonstrating the processing operation example of the self-propelled vacuum cleaner which comprises 3rd Embodiment of the vacuum cleaner by this invention. It is a figure which shows the flowchart for demonstrating the processing operation example of the server apparatus which comprises 3rd Embodiment of the vacuum cleaner by this invention.

[First Embodiment]
A case where the first embodiment of the electric vacuum cleaner according to the present invention is applied to an electric vacuum cleaner that performs cleaning while being operated by a user will be described with reference to the drawings.

  Drawing 1 is a figure for explaining the example of whole composition of electric vacuum cleaner 1 of a 1st embodiment. The vacuum cleaner 1 of this embodiment includes a vacuum cleaner body 10, a suction hose 11, a handle 12, a joint pipe 13, a suction part 14, an operation part 15, a display part 16, a control circuit part 20, Consists of.

  The vacuum cleaner body 10 includes a dust collection chamber 101 and a suction drive unit 102, and the suction drive unit 102 drives the suction drive unit 102 to suck and collect foreign objects other than the cleaning target such as dust and garbage from the suction unit 14. It has a function of collecting in the dust chamber 101. The cleaner body 10 includes a roller 103 that can move autonomously when the user performs cleaning.

  The dust collection chamber 101 of the cleaner body 10 is connected to the suction hose 11. And the handle 12 is couple | bonded with the opposite side to the connection side with the cleaner body 10 of the suction hose 11. Furthermore, the side of the handle 12 opposite to the side connected to the suction hose 11 is coupled to the joint pipe 13. A suction portion 14 is provided on the distal end side of the joint pipe 13.

  The handle 12 is provided with an operation unit 15, a display unit 16, and a control circuit unit 20. The user performs cleaning with the handle 12. The operation unit 15 includes a plurality of key buttons. The display unit 16 includes, for example, an LCD (Liquid Crystal Display) and includes a display screen 16D that can display an image. In addition, although the illustration is abbreviate | omitted in this embodiment, the operation part 15 contains the touchscreen arrange | positioned on the display screen 16D of the display part 16 so that it may overlap.

  The user can operate the key buttons of the operation unit 15 when necessary while cleaning the handle 12. Further, the display screen 16D of the display unit 16 can be viewed while holding the handle 12 and cleaning. The control circuit unit 20 is electrically connected to the operation unit 15 and the display unit 16 and is also electrically connected to the suction drive unit 102 of the cleaner body 10.

  The suction part 14 has a suction port 141 having an area (cross-sectional area) of a predetermined size. The suction port 141 is provided so as to have an opening on the surface side of the housing 142 of the suction portion 14 that faces the cleaning surface such as the floor surface. When the suction drive unit 102 of the cleaner body 10 is driven, a cleaning object smaller than the cross-sectional area and foreign objects other than the cleaning object are sucked from the suction port 141, and through the joint pipe 13, the handle 12, and the suction hose 11, It is sucked into the dust collection chamber 101. Note that long items such as plastic bags, plastic bags, cords, and the like may not be able to be sucked entirely even if they are sucked into the suction port 141.

  The electric vacuum cleaner 1 of this embodiment is provided with a plurality of camera units. That is, the housing 142 of the suction portion 14 is a surface facing a cleaning surface such as a floor surface, and two cameras 171 and 172 are provided in the vicinity of the suction port 141. When the user performs the cleaning operation, the suction unit 14 is moved in the front-rear direction as viewed from the user. However, the camera 171 is provided at a position in the front direction of the suction port 141 in the moving direction of the suction unit 14. 172 is provided at a rearward position of the suction port 141. Cameras 171 and 172 photograph a cleaning object on a cleaning surface such as a floor and foreign objects other than the cleaning object. In the vicinity of the cameras 171 and 172, although not shown in FIG. 1, illumination including, for example, an LED (Light Emitting Diode) is provided to illuminate the photographing position brightly.

  Further, in this embodiment, in order to be able to photograph a cleaning object and a foreign object other than the cleaning object on the cleaning surface such as the floor in front of the front-rear movement direction when the user performs the cleaning operation, The suction unit 14 is provided with a camera 173 other than the surface facing the cleaning surface such as the floor surface. In the example of FIG. 1, the camera 173 is a surface of the housing 142 of the suction portion 14 that intersects with a surface facing the cleaning surface such as the floor surface during the cleaning operation, and is forward of the front-rear movement direction during the cleaning operation. It is attached to the side surface.

  Further, in the vicinity of the coupling portion between the suction hose 11 and the dust collection chamber 101, the camera can be photographed so that a cleaning object sucked and collected in the dust collection chamber 101 and foreign objects other than the cleaning object can be photographed. 174 is attached.

  The four cameras 171 to 174 described above include, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and an imaging lens, and are electrically connected to the control circuit unit 20. It is connected.

[Description of Configuration Example of Control Circuit Section 20]
In this embodiment, the control circuit unit 20 performs image recognition on the captured images from the four cameras 171 to 174 during the cleaning operation, and at least removes the cleaning object and foreign objects other than the cleaning object. An image recognition unit that recognizes separately is provided. When the image recognition unit recognizes the foreign object, the control circuit unit 20 has a function of notifying the user and what the foreign object is. Furthermore, in this embodiment, the control circuit unit 20 has a function of forcibly stopping the cleaning operation according to the recognized foreign matter.

  FIG. 2 is a block diagram illustrating an internal configuration example of the control circuit unit 20 and a connection relationship between each unit electrically connected to the control circuit unit 20.

  As shown in FIG. 2, the control circuit unit 20 sends a drive control unit 202, a display control unit 203, an operation unit interface 204, camera interfaces 205, 206, and the like to the control unit 201 formed of a microcomputer via a system bus 200. 207 and 208, a captured image memory 209, an image recognition unit 210, an image registration memory 211, an external interface 212, a display image generation unit 213, and an alarm driving unit 214 are connected to each other.

  The suction controller 102 disposed in the cleaner body 10 is electrically connected to the drive controller 202. The control circuit unit 201 supplies a drive start control signal to the suction drive unit 102 in response to a cleaning start operation through the operation unit 15 by the user, and supplies a drive stop control signal in response to the cleaning stop operation.

  The display control unit 203 includes a video RAM (Random Access Memory), and has a display controller function for displaying an image on the display screen 16D of the display unit 16.

  The operation unit 15 is connected to the operation unit interface 204. As described above, in this embodiment, the operation unit 15 includes the above-described plurality of key buttons and also includes a touch panel provided to overlap the display screen 16D. In addition to the interface function for the buttons, a touch panel interface function for detecting a touch operation position on the display screen 16D is also included.

  The cameras 171, 172, 173, and 174 are connected to the camera interfaces 205, 206, 207, and 208, respectively. In this embodiment, as described above, illumination units 171L, 172L, 173L, and 174L made of LEDs, for example, are added to the cameras 171, 172, 173, and 174, respectively.

  The control circuit unit 20 supplies camera control signals to the cameras 171, 172, 173, and 174 through the camera interfaces 205, 206, 207, and 208, respectively, and captures images of the cameras 171, 172, 173, and 174, respectively. Controls start and stop of imaging. Each of the cameras 171, 172, 173, and 174 receives imaging start control, and supplies the captured image to the system bus 200 through each of the camera interfaces 205, 206, 207, and 208 in the imaging operation state.

  Lighting of the illumination units 171L, 172L, 173L, and 174L is controlled by the control unit 201. In this case, the control unit 201 of the control circuit unit 20 measures the brightness of the captured images (such as the average value of the brightness of the captured images) from the cameras 171, 172, 173, and 174 by software processing, and captures the captured images. The control circuit unit 20 is controlled to light up through the camera interfaces 205, 206, 207, and 208 when the brightness of the light is below a predetermined value. Note that the control unit 201 may control so that the illumination units 171L, 172L, 173L, and 174L are always turned on when the cameras 171, 172, 173, and 174 are in the imaging operation state.

  The captured image memory 209 temporarily stores captured image information transmitted from each of the cameras 171, 172, 173, and 174 through the camera interfaces 205, 206, 207, and 208, and temporarily stores the captured image information. The image is supplied to the image recognition unit 210 for image recognition. Further, as described later, the captured image memory 209 stores and holds captured image information based on the image recognition result when the image recognition unit 210 recognizes a foreign object other than the cleaning target.

  In this case, the control circuit unit 20 stores the captured image information by separately storing the memory area of the captured image memory 209 for each of the cameras 171, 172, 173, and 174. The captured image information is stored in association with the respective identification information, and the captured images from the cameras 171, 172, 173, and 174 are stored in a state where they can be identified separately.

  In this embodiment, the captured image information temporarily stored in the captured image memory 209 is used for image recognition processing by the image recognition unit 210, and a frame obtained by adding a margin to the number of frames for the processing time from which an image recognition result is obtained. A few minutes. The captured image information whose foreign matter is not recognized by the image recognition unit 210 is discarded and replaced with new captured image information. When the image recognition unit 210 recognizes a foreign object, the captured image information in which the foreign object is recognized is transferred from the captured image memory 209 to the display image generation unit 213 based on the image recognition result of the image recognition unit 210. The display image generation unit 213 holds the display image. At this time, captured image information in which a foreign object is recognized is held in the display image generation unit 213 in association with foreign object identification information (foreign object ID) described later included in the image recognition result from the image recognition unit 210.

  When the capacity of the captured image memory 209 is large, all of the captured image information during one cleaning operation is stored in the captured image memory 209, and the temporarily stored captured image is stored at the end of the cleaning operation. May be deleted. Of course, all of the picked-up images during a plurality of cleaning operations may be stored and held in the picked-up image memory 209. Further, the captured image stored and held in the captured image memory 209 can be appropriately recorded on a storage medium such as a USB memory or an SD card (SD memory card) through the external interface 212 as appropriate.

  In this embodiment, the image registration memory 211 stores in advance image information of an object to be recognized as a foreign object other than the object to be cleaned by a method described later. The image recognition unit 210 recognizes and detects a foreign object other than the cleaning object from the captured image information from each of the cameras 171, 172, 173, and 174. For the captured image information from each of 172 and 173, pattern matching is performed in comparison with the image stored in the image registration memory 211, and a match between the two or a similarity at a predetermined similarity level or higher is detected. Thus, foreign objects other than the object to be cleaned are recognized.

  The captured image information from the camera 174 may also be recognized by comparison by pattern matching with the registered captured image. However, the camera 174 includes the suction hose 11, the cleaner body 10, and the dust collection chamber. Since it is a captured image at a joint with 101, image recognition may be difficult. Therefore, in this embodiment, for the captured image information from the camera 174, the image recognition unit 210 determines the size of the cleaning object and the foreign objects other than the cleaning object that are sucked into the dust collection chamber 101 through the suction hose 11. It is determined whether or not the size is larger than a preset size, and it is determined whether or not the intensity of the reflected light when the light is applied like a precious metal is greater than or equal to a predetermined value. It is made to recognize whether it is a foreign substance other than the cleaning object.

  In the image registration memory 211, captured image information of an object assumed as a foreign object other than the object to be cleaned is registered and stored in advance before shipment from the manufacturer of the electric vacuum cleaner 1 to the factory. In this case, the image registration memory 211 stores a plurality of pieces of image information taken by the cameras 171, 172, and 173 as registered image information for one foreign object. For each piece of image information captured by each of the cameras 171, 172, and 173, image information from a plurality of imaging directions such as from the front, from the back, from the left side, from the right side, and the like is stored in the image registration memory 211. Is remembered. The registered image information of these foreign matters is stored in the image registration memory 211 in association with identification information (hereinafter referred to as foreign matter ID) given to each foreign matter.

  In this embodiment, the text registration information for displaying the name of the foreign object together with the image of the foreign object on the display screen 16D is stored in the image registration memory 211 in association with the foreign object ID. Furthermore, in this embodiment, as the aspect of the suction drive control for the suction drive unit 102 of the electric vacuum cleaner 1 when the registered foreign matter is recognized in the image registration memory 211, in this embodiment, “suction drive stop” or “suction drive continuation” (No control required) "is stored in association with the foreign object ID of the registered foreign object. Note that various modes such as “suction turbo drive”, “strong suction drive”, “weak suction drive”, and “weakest suction drive” may be set as the mode of suction drive control. Further, for example, instead of “stop suction driving” when a foreign object is recognized, “suction drive weak” or “suction drive weakest” is set, and then the user recognizes the foreign object recognized by looking at the image on the display screen 16D. When it is determined that the suction can be sucked in, the foreign matter is sucked by controlling to “suction turbo drive” or “strong suction drive” while the recognized foreign matter cannot be sucked by looking at the image on the display screen 16D. In this case, the suction drive can be controlled to stop.

  In this embodiment, not only image information prepared in advance by the manufacturer, but also an image of an object that the user wants the electric cleaning device 1 to recognize as a foreign object other than the cleaning object is registered. The registration memory 211 can be stored.

  In other words, the electric vacuum cleaner 1 according to this embodiment has a function of registering a captured image of a foreign object other than the object to be cleaned by the user. When the user selects the registration function of the captured image of the foreign object, the registration procedure of the registered image of the foreign object is displayed on the display screen 16D of the display unit 16 by a message or animation. The user can perform registration processing of a foreign object registration image while watching the registration procedure message and animation on the display screen 16D.

  For example, on the display screen 16D, first, a screen that prompts the camera 171, 172, and 173 to capture an image of a foreign object to be registered is displayed. Then, the camera 171, 172, and 173 are each instructed about the positional relationship between the camera and the subject and prompted to take an image with each camera. When the user takes an image with the cameras 171, 172, 173 in accordance with the registration procedure displayed on the display screen 16D, they are stored in the image registration memory 211 in association with the assigned foreign object ID. Finally, on the display screen 16D, an instruction for prompting the user to input the name of the foreign object is displayed. When the user inputs the name of the foreign object to be registered, the input text character information is registered. It is stored in the image registration memory 211 in association with the foreign object ID of the foreign object.

  Finally, an instruction prompting the user to make a setting input for the suction drive control for the suction drive unit 102 of the vacuum cleaner 1 when the registered foreign object is recognized is displayed on the display screen 16D. When “stop” or “continue suction drive (no control required)” is input and instructed, the suction drive control mode is stored in the image registration memory 211 in association with the registered image of the foreign matter.

  The above description is a foreign object image registration process using the cameras 171 to 173 included in the electric vacuum cleaner 1. In this embodiment, the foreign object image registration process is not limited to the method of imaging the foreign object using the cameras 171 to 173 provided in the electric vacuum cleaner 1 as described above, but also via an external interface 212, such as a USB memory or an SD card. It is also possible to register through the storage medium. In that case, a storage medium such as a USB memory or an SD card stores the registered image, the name of the foreign object, and the suction drive control mode that are captured in the same manner as the images captured by the cameras 171 to 173. Information is transferred to and stored in the image registration memory 211.

  Note that captured image information of various foreign objects, names of the foreign objects, and suction drive control modes thereof can be prepared on a website on the Internet operated by the manufacturer of the vacuum cleaner 1. In this case, the user accesses the website using, for example, a personal computer, obtains information on the foreign object that the user wants to register, stores the information in a USB memory or an SD card, and registers an image through the external interface 212. The image of the foreign object can be registered by transferring it to the memory 211. Of course, if the vacuum cleaner 1 has a network connection function, the Web site can be directly accessed without a personal computer.

  FIG. 3 shows an example of the foreign matter stored in the image registration memory 211. In the example of FIG. 3, the image registration memory 211 has a foreign object name, registered images V01, V02, V03,..., Thumbnail images SM01, SM02, SM03,. Is stored in association with foreign substance identification information (foreign substance ID) assigned to each foreign object. In this embodiment, the mode of suction drive control for the suction drive unit 102 is “stop suction drive” or “continue suction drive (control unnecessary)”.

  As described above, each of the registered images V01, V02, V03,... Includes image information of a plurality of captured images and is held as one set in association with the foreign object ID. Thumbnail images SM01, SM02, SM03,... Are used to display an image on the display screen 16D indicating what the recognized foreign object is when the image recognition unit 210 recognizes the foreign object. Of the captured images 171 to 173, the image includes a reduced image of the captured image that can be easily recognized by the user as representing the foreign matter.

  The thumbnail image is not registered in the image registration memory 211 in advance. When the image recognition unit 210 recognizes a foreign object, the display image generation unit 213 recognizes the thumbnail image based on the recognition result. A captured image registered for a foreign object may be acquired from the image registration memory 211 and generated.

  In the example of FIG. 3, the aspect of suction drive control when each foreign object is recognized is from the viewpoint of avoiding failure or damage of the vacuum cleaner 1, ensuring the cleanliness of the sucked foreign object or avoiding damage. From both viewpoints, “(suction operation) stop” and “(suction drive control) unnecessary” are stored.

  That is, in the example of FIG. 3, “plastic bag”, “plastic bag”, “document”, “code”, “pin, screw, nail” stop at the suction port 141, or the joint pipe 13 From the viewpoint of avoiding failure or breakage of the vacuum cleaner 1, the suction drive control mode is “(suction operation) because the joint pipe 13 or the suction hose 11 may be damaged in the middle of the suction hose 11 or the suction hose 11. ) Stopped ". Although not illustrated in FIG. 3, the same applies to “toothpick, needle” and the like.

  In the example of FIG. 3, “document”, “micro SD card”, and “USB memory” are driven by suction from the viewpoint of ensuring the cleanliness of foreign matter that is sucked in and avoiding contamination and breakage. The mode of control is “(suction operation) stop”. Although not illustrated in FIG. 3, “SD card, SIM card” and the like are the same. In the example of FIG. 3, “banknotes”, “gold coins, coins”, and “jewels” have the suction drive control mode “(suction drive control) unnecessary”. (Operation) stop ”. Although not illustrated in FIG. 3, “check, stamp” and the like are the same. Note that the suction drive control mode setting may be configured to be changeable by the user.

  In this embodiment, the image recognizing unit 210 includes a plurality of registered images of foreign substances stored and held in the image registration memory 211 as described above, and the cameras 171, 172, and 173 received through the captured image memory 209. Image recognition is performed by comparing the picked-up images picked up by the pattern matching processing. Then, the image recognizing unit 210 indicates that the degree of similarity is equal to or higher than a predetermined similarity level with the captured images from the cameras 171, 172, and 173 even in one of the plurality of registered images for one foreign object. When a foreign object is recognized, it is determined that the foreign object in the registered image has been recognized, and the foreign object ID of the registered image is output to the display image generation unit 213 as an image recognition result. At this time, the captured image of the camera that has recognized the foreign object in the image recognition unit 210 is supplied from the captured image memory 209 to the display image generation unit 213 and held in association with the foreign object ID of the image recognition result.

  Further, when the image recognition unit 210 determines that a foreign object has been recognized, the image recognition unit 210 further detects which of the cameras 171, 172, and 173 determines that the foreign object has been recognized. That is, the image recognizing unit 210 determines the image-recognized foreign object and which of the cameras 171, 172, and 173 has recognized the foreign object. Then, based on which camera the captured image that recognized the foreign object is, the image recognition unit 210 determines whether the recognized foreign object is sucked into the dust collecting chamber 101 of the cleaner body 10 or not. It is determined whether it exists outside the vacuum cleaner 1.

  That is, when the image recognizing unit 210 determines that a foreign object has been recognized in the captured image from the camera 171 or 172 disposed in the vicinity of the suction port 141, the determined foreign object is sucked into the dust collection chamber 101. It is recognized that there is a high possibility. In this embodiment, the image recognition unit 210 further refers to the image recognition result for the captured image of the camera 174, determines whether or not a foreign object is detected for the captured image of the camera 174, and is recognized. It is determined whether or not foreign matter has actually been sucked into the dust collection chamber 101. When it is determined that a foreign object has been recognized in the captured image from the camera 171 or 172 disposed in the vicinity of the suction port 141, the determined foreign object does not refer to the recognition result of the camera 174, and the dust collection chamber Although it may be determined that the air has been sucked into 101, by referring to the recognition result of the captured image from the camera 174, whether the air is sucked into the dust collection chamber 101 can be determined with higher accuracy.

  In addition, when the image recognition unit 210 determines that a foreign object has been recognized for a captured image from the camera 173 that captures an image in front of the suction unit 14, the image recognition unit 210 further always refers to the image recognition result for the captured image of the camera 174. Then, it is determined whether or not a foreign object has been detected, and it is determined whether or not the foreign object has been sucked into the dust collection chamber 101.

  As described above, when the image recognition unit 210 recognizes a foreign object by image recognition, in addition to the foreign object ID of the foreign object recognized by the image recognition result, the image recognition unit 210 determines whether the position of the foreign object is inside or outside, that is, the dust collection chamber 101. It is determined whether it is sucked into (inside) or is not sucked in and exists outside the electric vacuum cleaner 1 (outside). Then, the image recognition unit 210 supplies the recognized foreign object ID added with the foreign object location information (inside or outside information) to the display image generation unit 213 as an image recognition result.

  Further, in this embodiment, as described above, when a foreign object is recognized by the image recognition unit 210, the captured image of the camera 173 at the timing is supplied from the captured image memory 209 to the display image generation unit 213. To do. The display image generation unit 213 stores and holds the captured image of the camera 173 in association with the foreign object ID of the image recognition result sent from the image recognition unit 210 at this time.

  The captured image from the camera 173 stored and held is a captured image in front of the suction unit 14 at the position where the foreign object is image-recognized, and a peripheral image for determining where the foreign object is image-recognized. Become. By viewing the peripheral image at the position where the foreign object imaged by the camera 173 is recognized, the user can roughly determine where the foreign object is recognized in the cleaned room.

  The display image generation unit 213 generates display image information to be displayed on the display unit 16 through the display control unit 203. In this embodiment, in addition to the function of generating the guide image for the foreign substance registration process described above, the foreign substance recognized during the cleaning operation is used based on the information of the image recognition result sent from the image recognition unit 210. An image to be notified to the person is generated. In this embodiment, a thumbnail image that is a reduced image of the captured image stored in the image registration memory 211 is used as the foreign object image recognized during the cleaning operation.

  That is, the display image generation unit 213 receives the foreign object ID as the foreign object recognition result from the image recognition unit 210, the location information thereof, and the captured image of the camera 173 at the time of foreign object recognition. Then, the display image generation unit 213 receives thumbnail image information SMi (i = 01, 02,...) Associated with the foreign object ID from the image registration memory 211 based on the foreign object ID acquired from the image recognition unit 210. 4), and a thumbnail image 221 based on the thumbnail image information is displayed on the display screen 16D through the display control unit 203 as shown in FIG.

  Further, as shown in FIG. 4A, the display image generation unit 213 provides a foreign substance name column 222 in the display column of each thumbnail image 221. The display image generation unit 213 receives the name of the foreign object from the image registration memory 211 based on the foreign object ID acquired from the image recognition unit 210 and displays the name of the foreign object in the foreign object name column 222.

  Further, as shown in FIG. 4A, the display image generation unit 213 provides a display field 223 for the location of a foreign object in the display field of each thumbnail image 221. Then, the display image generation unit 213 generates additional display information indicating whether the location of the foreign object is inside or outside from the location information sent from the image recognition unit 210 together with the foreign object ID. As shown in FIG. 4 (A), it is displayed in the display field 223. In FIG. 4A, the foreign matter displayed as “inside” in the display field 223 indicates that the foreign matter is present in the dust collection chamber 101 of the cleaner body 10, and the foreign matter displayed as “outside” It shows that it exists outside without being sucked.

  The display image generation unit 213 performs the above operation every time the foreign object recognition result is received from the image recognition unit 210 during the cleaning operation, and the recognized foreign object is displayed on the display screen 16D as shown in FIG. The thumbnail image 221 is additionally displayed. Accordingly, as shown in FIG. 4A, at the end of the cleaning operation, a list of foreign substances recognized during the cleaning operation is displayed on the display screen 16D as a list of thumbnail images.

  As described above, the suction drive control mode is registered and stored in the image registration memory 211 for each foreign object, and the foreign matter suction drive control mode recognized by the image recognition unit 210 is “(suction operation) stop”. When this is set, the suction drive unit 102 is forcibly stopped. Therefore, in this embodiment, the display image generation unit 213 displays a thumbnail when the foreign matter suction drive control mode corresponding to the foreign matter ID of the image recognition result from the image recognition unit 210 is “(suction operation) stop”. The display column 224 of the suction drive stop control is provided in the display column of the image 221, and the character “stop” indicating that the suction drive stop control is performed is displayed on the display column 224, and the user is forcibly stopped. The notification is made so that the foreign matter at the time can be understood.

  In this embodiment, when the user clicks one of the thumbnail images displayed on the display screen 16 </ b> D, the display image generation unit 213 controls the clicked thumbnail image based on the control of the control unit 201. 4B, a captured image 241 obtained by recognizing the foreign object ID recognized by the image recognition unit 210 corresponding to the recognized foreign object ID and stored and held in correspondence with the recognized foreign object ID is displayed as shown in FIG. The image is displayed in a half area of 16D. Further, the display image generation unit 213 stores the captured image (peripheral image) 242 of the camera 173 stored and held corresponding to the recognized foreign object ID, as shown in FIG. 4B, on the remaining display screen 16D. Display in the area.

  The example in FIG. 4B shows a case where a thumbnail image of the ring is clicked by the user, and the captured image 241 is a captured image of the ring captured by either the camera 171 or 172. Moreover, the captured image 242 is a peripheral image in front of the position where the electric vacuum cleaner 1 sucked the ring, and in the example of FIG. 4B, a state in which a gap between two pieces of furniture is photographed is shown. .

  By viewing the captured image 241, the user can accurately know the image-recognized foreign matter, and the captured image 242 indicates where the foreign matter was being cleaned when the image was recognized. I can know.

  Next, an alarm generator 215 is connected to the alarm driver 214. When the image recognition unit 210 recognizes a foreign object, the alarm driving unit 214 controls the alarm generation unit 215 based on the image recognition result to generate an alarm and confirm that the user has recognized the foreign object. Let them know. As described above, when a foreign object is recognized by the image recognition unit 210, a thumbnail image of the recognized foreign object is displayed on the display screen 16D based on the image recognition result. In conjunction (synchronized), an alarm is generated from the alarm generation unit 215. The user knows the recognition of the foreign matter in the image recognition unit 210 by the occurrence of the alarm, and knows what the recognized foreign matter is by looking at the thumbnail image displayed on the display screen 16D.

  In this example, the alarm generation unit 215 includes a buzzer sound generation unit. The alarm generation unit 215 is not limited to a buzzer sound generation unit, and may be a vibration generation unit such as a vibrator, or may be a speaker that emits alarm sound or alarm music. Further, alarm sound and vibration may be combined.

  In this embodiment, the alarm generation unit 215 flushes the thumbnail image of the image-recognized foreign matter displayed on the display screen 16D. As a result, as will be described later, even if thumbnail images of a plurality of foreign objects are displayed on the display screen 16D, the user can easily recognize what the foreign objects are recognized at the timing. It is configured as follows.

  In the configuration of the control circuit unit 20 shown in FIG. 2, the image recognition unit 210, the display image generation unit 213, and the alarm driving unit 214 are realized by the microcomputer constituting the control unit 201 as the software functions. You can also

[Description of processing operation in control circuit unit 20 of electric vacuum cleaner 1]
FIG. 5 is a flowchart for explaining the flow of the processing operation in the control circuit unit 20 of the electric vacuum cleaner 1 of the first embodiment. In the description of the flowchart of FIG. 5 below, it is assumed that the control unit 201 realizes the functions of the image recognition unit 210, the display image generation unit 213, and the alarm driving unit 214 as software functions.

  First, the control unit 201 monitors the user's operation input through the operation unit 15 through the operation unit interface 204 and waits for the suction drive start button to be pressed (step S101). When it is determined in step S101 that the suction drive start button has been pressed, the control unit 201 supplies a suction drive start control signal to the suction drive unit 102 through the drive control unit 202 to start suction drive and the camera. A camera shooting start control signal is supplied to each of 171 to 174 to start camera shooting (step S102).

  And the control part 201 performs the image recognition process as mentioned above with respect to the captured image information from the cameras 171-174 (step S103), and discriminate | determines whether the foreign material was recognized (step S104). If it is determined in step S104 that no foreign matter has been recognized, the control unit 201 monitors the user's operation input through the operation unit 15 through the operation unit interface 204, and whether or not the suction drive stop button has been pressed. (Step S105).

  If it is determined in step S105 that the suction drive stop button has not been pressed, the control unit 201 returns the process to step S103, and repeats the processes after step S103. When it is determined in step S105 that the suction drive stop button has been pressed, the control unit 201 supplies a suction drive stop control signal to the suction drive unit 102 through the drive control unit 202 to stop the suction drive. A camera shooting stop control signal is supplied to each of the cameras 171 to 174 to end the camera shooting (step S106). And the control part 201 returns a process to step S101, and repeats the process after this step S101.

  When it is determined in step S104 that a foreign object has been recognized, the control unit 201 sounds an alarm by the alarm generation unit 215 and displays a thumbnail image of the recognized foreign object on the display screen 16D. At this time, as described above, the control unit 201 determines whether the foreign matter is in the dust collection chamber 101 or outside without being sucked, and the determination result is as shown in FIG. The thumbnail image is additionally displayed (step S107).

  Next, the control unit 201 refers to the stored contents of the image registration memory 211, acquires information on the suction drive control mode stored corresponding to the foreign matter ID of the recognized foreign matter, and forcibly stops the suction operation. It is determined whether or not it is necessary (step S108).

  When it is determined in step S108 that the forced stop of the suction operation is not necessary, the control unit 201 continues the suction operation of the suction drive unit 102 (step S109), returns the process to step S105, and the steps after step S105 Repeat the process.

  At this time, the vacuum cleaner 1 is controlled so as to continue the suction operation, but the user sees the occurrence of the alarm in step S107 and the thumbnail image on the display screen 16D as necessary. Then, the drive stop button can be pressed. Then, the control unit 201 determines that the drive stop button is pressed in step S105, and stops the suction drive.

  Next, when it is determined in step S108 that the suction operation must be forcibly stopped, the control unit 201 supplies a control signal for stopping the suction operation of the suction drive unit 102 to the suction drive unit 102 to perform the suction operation. Stop forcibly. Then, in the forced suction operation stop state, a lock state is set so that an operation such as restart of the suction operation through the operation unit 15 cannot be performed (step S110).

  This locked state is released when the user inputs an unlocking operation through the operation unit 15 after removing the foreign matter in the vicinity of the suction port 141 or opening the dust collecting chamber 101 to remove the foreign matter. .

  Next to step S110, the control unit 201 determines whether or not the lock has been released (step S111). When it is determined that the lock has not been released, the control unit 201 ignores an operation input such as a restart. Is performed (step S112). After step S112, the control unit 201 returns the process to step S111, and repeats the processes after step S111.

  If it is determined in step S111 that the lock has been released, the control unit 201 returns the process to step S101 and repeats the processes after step S101.

  In the above description, when the suction drive is stopped in accordance with the recognition of the foreign object, the locked state is set to be insensitive to the normal start button operation. I was allowed to. However, depending on the recognized foreign matter, there is no need to remove the foreign matter near the suction port 141 or to open the dust collecting chamber 101 to remove the foreign matter. In this embodiment, the user can recognize the foreign matter recognized by the electric vacuum cleaner 1 on the display screen 16D. Considering the above, it is possible to allow the user confirmed on the display screen 16D to restart the suction drive without performing the unlocking operation as described above on the operation unit 15 of the vacuum cleaner 1. A one-time cleaning continuation button may be provided.

[Effect of the first embodiment]
As described above, the electric vacuum cleaner 1 of the first embodiment described above detects foreign objects other than the assumed cleaning object by image recognition during the cleaning operation, and displays a thumbnail image of the detected foreign objects. While displaying on the screen 16D, the detection of a foreign object is notified by an alarm. Therefore, the user can immediately know what the foreign material is by knowing the detection of the foreign material by the alarm and viewing the thumbnail image on the display screen 16D. In this case, the user does not need any reproduction operation as in the case of the prior document described at the beginning, which is very convenient.

  The user can confirm whether or not the suction operation should be stopped after confirming the foreign matter detected in the thumbnail image, which is convenient. In addition, in the first embodiment, for a foreign object that should be forcibly stopped in advance in advance, it is possible to automatically stop the suction operation by registering it. It is possible to prevent the electric vacuum cleaner 1 from being damaged, and to prevent foreign matter desired to be kept clean from being sucked into the dust collection chamber 101.

  In addition, the thumbnail image of the recognized foreign matter has an additional display indicating whether the foreign matter is sucked into the dust collection chamber 101 or is outside without being sucked. Can easily know the location of foreign matter.

  Further, in this embodiment, when a thumbnail image displayed on the display screen 16D is clicked by the user, an actual captured image of the detected foreign object is displayed, and the foreign object is image-recognized. A peripheral captured image in front of the position suction unit 14 is displayed. For this reason, the user can recognize exactly what the detected foreign matter is, and can conveniently find out where the foreign matter was.

  In particular, when a foreign object hidden behind a furniture gap or under a bed is recognized, even if the foreign object is not sucked in from the peripheral captured image in front of the suction portion 14 at the cleaning position where the foreign object is image-recognized, The location of the foreign object can be recognized. And the vacuum cleaner 1 of this embodiment can also recognize the foreign material in the place which the vacuum cleaner 1 is not made into the cleaning object, or the place where cleaning is impossible from the recognition result of the captured image of the camera 173. Therefore, the electric vacuum cleaner 1 of this embodiment recognizes what the user thinks has been lost as a foreign object regardless of whether it is on the floor surface to be cleaned, etc. It can be informed along with its location, which is very convenient. Needless to say, the furniture is brightly illuminated by the illumination unit attached to the camera and captured by the camera, such as in a gap in the furniture or under the bed.

  Moreover, although the vacuum cleaner of patent document 1 demonstrated at the beginning can stop suction operation | movement with respect to the thing judged to be larger than the area of a suction inlet, a magnitude | size is compared with the area of a suction mouth. If it is difficult to determine whether it is large or not, string-like objects such as electrical outlet cords and AC adapter cables, shopping bags, etc. may not stop the suction. Also, even if it is smaller than the area of the suction port, foreign matter that should not be sucked in, such as small media such as micro SD card, SD card or SIM card, sharp screws or nails, etc. are sucked in. There is a problem that it cannot be avoided.

  On the other hand, according to the electric vacuum cleaner of the first embodiment described above, a string-like object such as an electric outlet cord or an AC adapter cable, or a small size such as a shopping bag, a micro SD card, an SD card or a SIM card. Media, screws, nails, etc. are registered in the image registration memory and defined so that the suction operation is controlled to stop. There is an effect that it is possible to recognize and reliably stop by the recognition.

  Further, according to the electric vacuum cleaner of the first embodiment described above, when the cleaning operation is completed, a list of thumbnail images of foreign objects whose images have been recognized during the cleaning operation is displayed on the display screen. By looking at the list image, it is possible to easily know whether there is a foreign object that needs to be taken out from the dust collection chamber. In addition, when the thumbnail image is clicked, the image of the actually captured foreign object is enlarged and displayed on the display screen. Therefore, even if the thumbnail image is an unknown or uncertain foreign object, Can be surely confirmed.

[Second Embodiment]
The vacuum cleaner according to the second embodiment includes a self-propelled vacuum cleaner and a portable device including a display unit, for example, a mobile phone terminal. In the second embodiment, the self-propelled vacuum cleaner and the mobile phone terminal are connected through a wireless communication path. As the wireless communication path, a Bluetooth (registered trademark) standard short-range wireless communication path is used in the following example. The wireless communication path is not limited to this, and a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark) or other wireless communication means may be used.

[Outline of electric vacuum cleaner 30]
FIG. 6 is a diagram for explaining a self-propelled electric vacuum cleaner 30 that constitutes the electric vacuum cleaner according to the second embodiment. FIG. 6A is a diagram illustrating the self-propelled electric vacuum cleaner 30 of this example on the side of the bottom surface 30a including the suction port 31, and FIG. 6B is a self-propelled electric vacuum cleaner of this example. It is the figure which looked at 30 from the advancing direction at the time of cleaning operation. In FIG. 6A, an arrow AR indicates the traveling direction (forward direction) during the cleaning operation of the self-propelled electric vacuum cleaner 30.

  As shown in FIG. 6A, the self-propelled electric vacuum cleaner 30 includes a substantially disc-shaped housing, and a suction port 31 is provided on the bottom surface 30a. And in the housing | casing of the vacuum cleaner 30, as shown with a dotted line in FIG.6 (B), it connects to the suction inlet 31, and collects the foreign material other than the cleaning target and the cleaning target which were sucked in. A chamber 32 is provided. Although not shown in FIG. 6, in the housing of the electric vacuum cleaner 30, a suction drive unit is provided in the dust collection chamber 32 to suck in the cleaning object and foreign matters other than the cleaning object. ing.

  The vacuum cleaner 30 according to this embodiment includes a flexible brush 33 and a rolling brush 34 in the suction port 31 as shown in FIG. And the roller 35 is provided in the approximate center of the front of the advancing direction at the time of cleaning operation | movement with respect to the suction inlet 31 position of the bottom face 30a of the housing | casing of the vacuum cleaner 30. FIG. The roller 35 is attached so as to be rotatable in a plane in which the axial direction of the rotation shaft is parallel to the bottom surface 30a. For this reason, the axial center direction of the rotating shaft of the roller 35 can be directed not only in a direction orthogonal to the arrow AR but also in an arbitrary direction. Further, on both sides of the suction port 31 of the bottom surface 30a, wheels (wheels) 36 and 37 that are rotationally driven by a travel drive unit (not shown in FIG. 6) are provided.

  When the wheels 36 and 37 are driven and rotated by the travel drive unit, the vacuum cleaner 30 travels by itself. As described above, the roller 35 can rotate with any direction as the rotation axis direction. The vacuum cleaner 30 not only travels straight in the direction of the arrow AR, but can also rotate and travel and move in an arbitrary direction intersecting the arrow AR. Further, the electric vacuum cleaner 30 can travel backward in the direction opposite to the arrow AR by controlling the rotation direction of the wheels by the travel drive unit.

  In this embodiment, cameras 381 and 382 are provided in the vicinity of the suction port 31 on the bottom surface 30a so that the direction perpendicular to the bottom surface 30a is the photographing direction (optical axis direction). These cameras 381 and 382 correspond to the cameras 171 and 172 in the electric vacuum cleaner 1 of the first embodiment.

  A camera 383 is provided on the side surface of the housing in front of the roller 35 of the vacuum cleaner 30 so that the shooting direction (optical axis direction) is substantially parallel to the bottom surface 30a. Further, a camera 384 is provided on the side surface of the housing behind the roller 35 of the vacuum cleaner 30 so that the photographing direction (optical axis direction) is substantially parallel to the bottom surface 30a. The camera 383 images the front of the self-propelled electric vacuum cleaner 30 in the traveling direction at a predetermined angle of view. In addition, the camera 384 captures the rear of the self-propelled vacuum cleaner 30 in the traveling direction with a predetermined angle of view. These cameras 383 and 384 correspond to the camera 173 in the electric vacuum cleaner 1 of the first embodiment.

  That is, in the second embodiment, the peripheral image at the position where the foreign object is recognized is not only the front captured image in the traveling direction of the vacuum cleaner 30 but also the rear captured image (foreign object ID). Are stored in association with each other.

  Since the cameras 383 and 384 have a predetermined angle of view, the photographing direction (optical axis direction) is parallel to the bottom surface 30a if at least the floor surface to be cleaned in the traveling direction of the vacuum cleaner 30 can be imaged. Need not be.

  In the second embodiment, the vacuum cleaner 30 does not have an image recognition function and does not have a display unit. The image recognition function and the display unit are provided in a mobile phone terminal connected to the vacuum cleaner 30 through a wireless communication path.

  The four cameras 381 to 384 described above are configured to include, for example, a CCD image sensor, a CMOS image sensor, and an imaging lens, as in the first embodiment.

[Circuit Configuration of the Vacuum Cleaning Device of the Second Embodiment]
FIG. 7 is a diagram illustrating an electrical circuit configuration of the vacuum cleaner 30 of the second embodiment and the mobile phone terminal 40 wirelessly connected to the vacuum cleaner 30.

  That is, as shown in FIG. 7, the electric vacuum cleaner 30 has a drive control unit 302 and a transmission image information generation unit 303 through a system bus 300 with respect to a control unit 301 on which a microcomputer is mounted. The operation unit 304, the wireless communication unit 305, and the cameras 381, 382, 383, and 384 are connected. In FIG. 7, the camera interface is not shown for the sake of simplicity.

  A suction drive unit 306 and a travel drive unit 307 are connected to the drive control unit 302. The suction drive unit 306 is for sucking foreign objects other than the cleaning object and the suctionable cleaning object from the suction port 31. The traveling drive unit 307 drives the wheels 36 and 37 to rotate so that the electric vacuum cleaner 30 travels (self-runs) by itself.

  When the control unit 301 receives a pressing operation input of a cleaning start button provided in the operation unit 304, the control unit 301 supplies a drive start control signal to the suction drive unit 306 and the travel drive unit 307 through the drive control unit 302. To start the automatic cleaning operation while self-propelled by the electric vacuum cleaner 30. Further, the control unit 301 completes cleaning when receiving an input of pressing a cleaning stop button provided on the operation unit 304 or based on a cleaning support program stored in a memory built in the control unit 301. When it is determined that the drive has been performed, a drive stop control signal is supplied to the suction drive unit 306 and the travel drive unit 307 through the drive control unit 302 to stop the driving of the suction drive unit 306 and the travel drive unit 307, and the vacuum cleaner 30 Stop the cleaning operation.

  Further, in the second embodiment, when the control unit 301 receives a forced stop control signal from the mobile phone terminal 40 through the wireless communication unit 305 based on the cleaning support program stored in the built-in memory, The forced stop control signal is supplied to the suction drive unit 306 and the travel drive unit 307 through the drive control unit 302, and the drive is forcibly stopped, and the cleaning operation by the vacuum cleaner 30 is forcibly stopped.

  Each of the cameras 381 to 384 is controlled by the control unit 301 to perform an imaging operation during the cleaning operation in the same manner as in the first embodiment. In the second embodiment, lighting units 381L, 382L, 383L, and 384L made of LEDs, for example, are added to the cameras 381, 382, 383, and 384, respectively, and the same as in the first embodiment. Thus, lighting control is performed by the control unit 301.

  The transmission image information generation unit 303 generates transmission image information of captured image information from each of the cameras 381 to 384 while the electric vacuum cleaner 30 is performing a cleaning operation. In this case, the transmission image information generation unit 303 adds camera identification information (camera ID) as to which of the cameras 381 to 384 is captured image information. . In addition, the transmission image information generation unit 303 performs encoding processing such as data compression as appropriate on the captured image information from each of the cameras 381 to 384 to generate transmission image information, and transfers the transmission image information to the wireless communication unit 305.

  As described above, the wireless communication unit 305 has a Bluetooth (registered trademark) standard short-range wireless communication function in this embodiment. The wireless communication unit 305 wirelessly transmits the transmission image information from the transmission image information generation unit 303 to the mobile phone terminal 40. The wireless communication unit 305 also receives a forced stop control signal based on the image recognition result sent from the mobile phone terminal 40 and passes it to the control unit 301 through the system bus 300.

  As described above, when the control unit 301 receives the forced stop control signal through the wireless communication unit 305, the control unit 301 supplies the forced stop control signal to the suction drive unit 306 and the travel drive unit 307 through the drive control unit 302. The driving is forcibly stopped, and the cleaning operation by the electric vacuum cleaner 30 is forcibly stopped.

  In the configuration of the vacuum cleaner 30 in FIG. 7, the functions of the drive control unit 302 and the transmission image information generation unit 303 can be configured as software functions by the control unit 301.

  Next, a configuration example of the mobile phone terminal 40 used in the second embodiment will be described. The mobile phone terminal 40 in this embodiment has a configuration of a high-function mobile phone terminal called a so-called smartphone (abbreviated as a smartphone in FIG. 8). That is, the mobile phone terminal 40 of this example has a configuration in which each unit described later is controlled by a control unit 401 including a microcomputer.

  The control unit 401 includes a wireless communication unit 402, a display control unit 403, a touch panel interface 404, a captured image storage unit 405, an image recognition unit 406, an image registration memory 407, and an external device via the system bus 400. An interface 408, a display image generation unit 409, a mobile phone function unit 410, and a camera unit 411 are connected.

  As described above, in this embodiment, the wireless communication unit 402 has a Bluetooth (registered trademark) standard short-range wireless communication function, and wirelessly communicates with the vacuum cleaner 30. The wireless communication unit 402 performs wireless communication that is completely independent of the mobile phone communication performed by the mobile phone terminal 40 via the mobile phone base station.

  A display unit 412 made of, for example, an LCD is connected to the display control unit 403. A touch panel 413 is connected to the touch panel interface 404. The touch panel 413 receives a touch operation input by a finger or a pen-type position indicator, and outputs position information where the touch operation input is made. The touch panel 413 is transparent, and is disposed so as to be superimposed on the display screen of the display unit 412, in this example, the LCD screen. The user can view the display image displayed on the display screen of the display unit 412 through the touch panel 413, and instruct the touch panel 413 to input a touch operation on the display image. Operation input can be performed. The position information of the touch operation input from the touch panel 413 corresponds to the position information on the display image displayed on the display screen.

  The control unit 401 receives the position information of the touch operation input from the touch panel 413 through the touch panel interface 404, discriminates the instruction operation input on the touch panel 413 by the user and the control processing set according to the operation position, Control processing corresponding to the determination result is executed. A processing program for the control unit 401 to execute a processing function corresponding to the position information of the touch operation input from the touch panel 413 is stored in a memory built in the control unit 401.

  The display control unit 403 has a function similar to that of the display control unit 203 in the first embodiment, includes a video RAM that stores display image data to be displayed on the display screen of the display unit 412, and controls the control unit 401. Based on this, the mobile phone terminal 40 displays an image, and the above-described thumbnail image and the like are displayed when a cleaning support program described later is started.

  The captured image storage unit 405 decodes and separates each of the captured image information of the cameras 381 to 384 from the transmission image information sent from the vacuum cleaner 30 through the wireless communication unit 402, and the image recognition unit 406 Temporary storage for image recognition. The captured image storage unit 406 has the same function as the captured image memory 209 of the first embodiment, except for the function of decoding and separating each of the captured image information of the cameras 381 to 384 from the transmission image information. .

  The image recognition unit 406 is activated when a program that supports the cleaning operation in the electric vacuum cleaner 30 is activated, and performs the same processing operation as the image recognition unit 210 of the first embodiment described above. However, in the second embodiment, when a foreign object is recognized by the image recognition unit 406, the control unit 301 performs the suction drive stored in the image registration memory 407 corresponding to the recognized foreign object ID. Although the control mode is referred to, when the suction drive control mode is stopped (forced stop), the forced stop control signal is wirelessly transmitted to the vacuum cleaner 30 through the wireless communication unit 402.

  The image registration memory 407 also has the same function as the image registration memory 211 of the first embodiment described above. That is, in the second embodiment, by using the display unit 412 of the mobile phone terminal 40 instead of the display unit 16 in the first embodiment, the same as described in the first embodiment, A captured image of a foreign object captured by the cameras 381 to 384 can be registered and stored in the image registration memory 407. In the second embodiment, similarly to the first embodiment, a captured image of a foreign object from a USB memory or an SD card can be registered and stored in the image registration memory 407 through the external interface 408. . In addition, the camera unit 411 included in the mobile phone terminal 40 can capture a foreign object, register it in the image registration memory 407, and store it. Further, the cellular phone function unit 410 accesses a server of a website holding a captured image of a foreign object via the Internet, acquires a captured image of the necessary foreign object from the server, and stores it in the image registration memory 407. You can also register and memorize it.

  The program that supports the processing for registering the image of the foreign object in the image registration memory 407 is downloaded in advance from the server of the website that supports the function of the vacuum cleaner 30 through the Internet, and is stored in the mobile phone terminal 40. It is stored in a memory built in the control unit 401. Similarly, a cleaning support program for performing foreign object recognition during the cleaning operation of the electric vacuum cleaner 30 to be described later is downloaded from the server and stored in a memory built in the control unit 401 of the mobile phone terminal 40. .

  The display image generation unit 409 is activated when a program that supports the cleaning operation in the electric vacuum cleaner 30 is activated, and performs the same processing operation as the display image generation unit 213 of the first embodiment.

  The cellular phone function unit 410 includes a transmitter unit and a receiver unit, and is a processing function unit that executes functions of a cellular phone such as a call through a cellular phone network and e-mail. In this embodiment, since the function of the mobile phone is not directly related to the present invention, detailed description thereof is omitted in this specification.

  In the configuration of the mobile phone terminal 40 of FIG. 7, the processing unit excluding the memory of the image recognition unit 406 and the captured image storage unit 405 and the function of the display image generation unit 409 are configured by the control unit 401 as software functions. It is something that can be done.

[Description of Processing During Cleaning Operation in Second Embodiment]
In the second embodiment, the user activates a program that supports the cleaning operation in the electric vacuum cleaner 30 in the mobile phone terminal 40 before pressing the cleaning start button of the operation unit 304 of the electric vacuum cleaner 30. I will let you. Then, when the user presses the cleaning start button of the operation unit 304 of the vacuum cleaner 30, processing operations described below are executed in the vacuum cleaner 30 and the mobile phone terminal 40.

  FIG. 8 is a flowchart for explaining an example of the flow of processing operations in the electric vacuum cleaner 30 at this time. FIG. 9 is a flowchart for explaining an example of the flow of processing operations of the mobile phone terminal 40 when the cleaning support program is activated.

  First, the processing operation of the vacuum cleaner 30 will be described with reference to FIG. In the following description, the control unit 301 executes the functions of the drive control unit 302 and the transmission image generation unit 303 as software functions.

  The control unit 301 determines whether or not the cleaning start button has been pressed, and waits for the cleaning to start (step S301). When it is determined in step S301 that the cleaning start button has been pressed, the control unit 301 starts driving the suction drive unit 306 and the travel drive unit 307 and instructs each of the cameras 381 to 384 to start capturing images. To do. Further, the wireless communication unit 305 generates a wireless communication path with the mobile phone terminal 40 and wirelessly connects to the mobile phone terminal 40 (step S302).

  And the control part 301 collects the captured image from each of the cameras 381-384 in cleaning operation, produces | generates transmission image information (step S303), and transmits to the mobile telephone terminal 40 through the wireless communication part 305 (step S303). Step S304).

  Then, the control unit 301 monitors the reception signal of the wireless communication unit 305 and determines whether or not a forced stop control signal is received from the mobile phone terminal 40 (step S305). When it is determined in step S305 that the forced stop control signal has not been received, the control unit 301 determines whether or not the end of cleaning has been detected based on whether or not all cleaning in the cleaning range has been completed. (Step S306) When the end of the cleaning is not detected, the process returns to Step S303, the transmission image information is generated, and the cleaning operation is continued while being wirelessly transmitted to the mobile phone terminal 40.

  When it is determined in step S305 that the forced stop control signal has been received from the mobile phone terminal 40, the control unit 301 stops driving the suction drive unit 306 and the travel drive unit 307 and each of the cameras 381 to 384. Is instructed to stop image capturing. Further, the wireless communication unit 305 disconnects the wireless communication path with the mobile phone terminal 40 (step S307).

  Even when the end of cleaning is detected in step S306, the control unit 301 advances the process to step S307 to stop driving the suction drive unit 306 and the travel drive unit 307, and for each of the cameras 381 to 384. The radio communication unit 305 disconnects the wireless communication path with the mobile phone terminal 40.

  With the above, the process at the time of the cleaning operation in the electric vacuum cleaner 30 is completed.

  Next, with reference to FIG. 9, the processing operation in the mobile phone terminal 40 will be described. In the following description, it is assumed that the control unit 401 executes the processing function part excluding the memory of the image recognition unit 406 and the captured image storage unit 405 and the function of the display image generation unit 409 as software functions.

  First, the control unit 401 determines whether or not the cleaning support program has been activated (step S401). When it is determined in step S401 that the cleaning support program has not been activated, the control unit 401 executes another process (step S402), and after the other process ends, returns the process to step S401. .

  And when it determines with the cleaning assistance program having been started by step S401, the control part 401 produces | generates the wireless communication path with the vacuum cleaner 30 by the wireless communication part 402 (step S403). Thereafter, the control unit 401 receives the transmission image information sent from the vacuum cleaner 30 through the generated wireless communication path, separates and acquires the captured image information from each of the cameras 381 to 384, and In the same manner as in the first embodiment, the image recognition processing is executed by comparison by pattern matching between the captured images from the cameras 381 to 384 and the foreign object images stored in the image registration memory 407 (step S404). .

  Then, the control unit 401 determines whether or not a foreign object has been recognized as a result of the image recognition in step S404 (step S405). If it is determined in step S405 that no foreign object is recognized, the control unit 401 determines whether or not the wireless communication path has been disconnected by the vacuum cleaner 30 (step S409). When it is determined in step S409 that the wireless communication path is disconnected, the control unit 401 determines that the cleaning operation in the electric vacuum cleaner 30 has been stopped, and ends this processing routine.

  If it is determined in step S409 that the wireless communication path is not disconnected, the control unit 401 determines that the cleaning operation in the electric vacuum cleaner 30 is continued, and returns the process to step S404. The processes after step S404 are repeated.

  If it is determined in step S405 that the foreign object has been recognized as a result of the image recognition, the control unit 401 sounds an alarm and displays a thumbnail image of the recognized foreign object on the display screen of the display unit 412. The alarm can be performed by using any of the ring tones provided in the mobile phone function unit 410 or by using the vibration by the vibrator together. At this time, as described in the first embodiment, the control unit 401 determines whether the foreign matter is in the dust collection chamber 32 or outside without being sucked, and the determination result is shown in FIG. As shown in A), the thumbnail image is additionally displayed (step S406).

  Next, the control unit 401 refers to the stored contents of the image registration memory 407, obtains information on the suction drive control mode stored corresponding to the recognized foreign object ID, and the cleaning operation is forcibly stopped. It is determined whether or not it is necessary (step S407).

  When it is determined in step S407 that the forced stop of the cleaning operation is not necessary, the control unit 401 continues the cleaning operation in the electric vacuum cleaner 30 without generating the forced stop control signal, and the process proceeds to step S404. The process after step S404 is repeated.

  Next, when it is determined in step S407 that the forced stop of the cleaning operation is necessary, the control unit 401 generates a forced stop control signal for stopping the driving of the suction drive unit 306 and the travel drive unit, and the wireless communication unit It transmits to the vacuum cleaner 30 through 402 (step S408). Thereafter, the control unit 401 advances the processing to step S409, waits for determination that the wireless communication path is disconnected in step S409, and ends this processing routine.

[Effects of Second Embodiment]
As described above, according to the vacuum cleaning device of the second embodiment, during the cleaning operation by the self-propelled vacuum cleaner 30, foreign substances other than the assumed cleaning object are detected by image recognition, The thumbnail image of the detected foreign object can be displayed on the display screen of the display unit 412 of the mobile phone terminal 40, and the detection of the foreign object can be notified by an alarm.

  Therefore, the user knows that the self-propelled electric vacuum cleaner 30 has detected a foreign object by an alarm, and immediately knows what the foreign object is by looking at the thumbnail image on the display screen of the mobile phone terminal 40. be able to. In this case, the user does not need any reproduction operation as in the case of the prior document described at the beginning, which is very convenient.

  And in the case of this 2nd Embodiment, the display part is not provided in the self-propelled electric vacuum cleaner 30, but the portable phone which is in a position separated from the electric vacuum cleaner 30 by wireless connection. Since the display unit 412 of the telephone terminal 40 is used, there is no need to go to the self-propelled electric vacuum cleaner 30 during cleaning and view the display screen, which is convenient.

  Also in the second embodiment, for foreign matters that should be forcibly stopped in advance, it is possible to automatically stop the suction operation by registering the foreign matter. It is possible to prevent the vacuum cleaner 30 from being damaged or broken, and to prevent foreign matter desired to be kept clean from being sucked into the dust collection chamber 32.

  In addition, the thumbnail image of the recognized foreign matter is additionally displayed to indicate whether the foreign matter is sucked into the dust collection chamber 32 or is outside without being sucked. Can easily know the location of foreign matter.

  Also in the second embodiment, when the thumbnail image displayed on the display screen of the display unit 412 is clicked by the user, an actual captured image of the detected foreign matter is displayed and the foreign matter is displayed. The peripheral captured images in the front and rear of the suction portion at the cleaning position where the image is recognized are displayed. For this reason, the user can accurately recognize what the detected foreign matter is, and can easily know where the foreign matter is located, which is convenient.

[Modification of Second Embodiment]
In the above description of the second embodiment, the cleaning start and stop of the vacuum cleaner 30 are performed by operating the operation buttons provided on the operation unit 304 provided in the vacuum cleaner 30. When the cleaning support program is activated on the mobile phone terminal 40, the cleaning start and stop of the vacuum cleaner 30 can be instructed through the touch panel 413 of the mobile phone terminal 40.

  Moreover, in the above-mentioned 2nd Embodiment, although the vacuum cleaner 30 was not provided with the image recognition part, you may provide an image recognition part also in the vacuum cleaner 30. FIG. In that case, the user registers an image of the foreign object in the image registration memory of the vacuum cleaner 30, and the image registration memory 407 of the mobile phone terminal 40 stores the foreign object stored in the server of the website. Images may be acquired through the Internet and registered and held. In this way, the user only has to register in the vacuum cleaner 30 only the image of the object that he / she wants to recognize as a foreign object, and other general foreign objects are registered by the user. Even if not, it can be recognized by the mobile phone terminal 40.

  Furthermore, when the vacuum cleaner 30 is provided with an image recognition unit, the mobile phone terminal 40 may use only the display function without providing the image recognition unit.

  In the above description of the second embodiment, the mobile device is an example using the mobile phone terminal 40. However, any mobile device including a display unit may be used, for example, It may be a pad type portable terminal, a tablet type terminal, a wearable terminal, or a personal computer. Further, the present invention can be applied not only to portable devices but also to all devices including a display unit, such as desktop devices and stationary devices.

  Also in the first embodiment, a mobile phone terminal can be used as a separate display unit as in the second embodiment. On the other hand, also in 2nd Embodiment, it can also be set as the structure similar to 1st Embodiment without giving a display part to the vacuum cleaner 30 and using a separate mobile telephone terminal.

  In the second embodiment, the suction drive control mode of the suction drive unit 306 is stopped (forced stop). However, as in the first embodiment, “suction turbo drive” and “suction drive stronger” are shown. ”,“ Weak suction drive ”,“ weakest suction drive ”, and the like.

  In the second embodiment, not only the suction drive control mode but also the travel drive control mode of the travel drive unit 307 can be considered as a control mode independently or in parallel with the suction drive control. In addition, control modes such as “travel drive high speed”, “travel drive low speed”, and “travel drive ultra-low speed” can be used. Further, as a travel drive control mode of the travel drive unit 307, for example, according to the foreign object detected by the image recognition from the image captured by the camera 383, the “running drive right turn” so as to avoid the foreign object recognized by the vacuum cleaner, "Travel drive left turn", or "Travel drive forward", "Travel drive backward", "Running drive stop and turn right", "Running drive stop and turn left", etc. It may be. When traveling control is performed so as to avoid this foreign matter, the suction drive control of the suction drive unit 306 may or may not be performed together.

[Third Embodiment]
In the second embodiment, since the image recognition process is performed by a portable device connected to the vacuum cleaner 30 through a wireless communication path, an advanced image recognition process may be performed depending on the capability of the connected portable device. The case where it is not possible is assumed. The third embodiment is an example in which the problem can be improved using a server (cloud) on the Internet.

  FIG. 10 shows an example of the electrical configuration of the electric vacuum cleaner according to the third embodiment. The electric vacuum cleaner according to the third embodiment includes a self-propelled electric vacuum cleaner 30 </ b> A and a server device 60 connected to the electric vacuum cleaner 30 </ b> A via a communication network 50. In this example, the communication network 50 includes the Internet and a home wireless LAN that can be connected to the Internet. The server device 60 is a server device operated by, for example, a sales company of the vacuum cleaner 30A that provides a cleaning support program to the vacuum cleaner 30A. In the third embodiment, as will be described later, the vacuum cleaner A function for image recognition processing of a foreign object is provided for captured image information from each of the cameras 381 to 384 sent from 30A.

  The self-propelled electric vacuum cleaner 30 </ b> A has the same configuration as the self-propelled electric vacuum cleaner 30 of the second embodiment regarding the suction cleaning function and the traveling function. However, the vacuum cleaner 30A according to the third embodiment performs a suction control and a travel control after waiting for an image recognition result from the server device 60 and a point having a display unit for displaying the image-recognized foreign matter. The point comprised in this way differs from the vacuum cleaner 30 in 2nd Embodiment.

  In the configuration of the vacuum cleaner 30A shown in FIG. 10, the same reference numerals are given to the same portions as those of the vacuum cleaner 30 of the second embodiment, and the description thereof is omitted. That is, in the vacuum cleaner 30A, the drive control unit 302, the transmission image generation unit 303, and the operation unit 304 are provided to the control unit 301A via the system bus 300, as in the second embodiment. In addition to this, in addition to this, a display control unit 311 to which the display unit 312 is connected, a wireless communication unit 313, a non-cleaning object detection unit 314, and a current position detection unit 315 are connected. The

  The display unit 312 is made of an LCD, for example, and is not shown, but is provided on the upper surface of the housing of the electric vacuum cleaner 30A opposite to the bottom surface 30a (see FIG. 6) so that the display screen is exposed. ing.

  The wireless communication unit 313 is connected to a wireless LAN provided in the home where the electric vacuum cleaner 30A included in the communication network 50 is used, and the image data is transmitted to and from the server device 60 through the communication network 50 through the Internet. And a communication function for exchanging data of image recognition results. In addition, 30 A of vacuum cleaners of this 3rd Embodiment hold | maintain the address information of the server apparatus 60, etc., in order to connect the communication path with the server apparatus 60 memorize | stored, when cleaning operation | movement is started. .

  The non-cleaning object detection unit 314 does not have a function for recognizing an image in comparison with a registered image, but the size of a captured image from each of the cameras 381 to 384 is set in advance. Non-cleaning object other than the object to be cleaned by judging whether it is above or not, and whether the intensity of the reflected light when shining on the light is higher than a predetermined value. It has a function of recognizing whether it is an object, that is, a foreign object. In addition, the non-cleaning target object detection unit 314 is configured so that only the objects existing on the floor surface to be cleaned are set as detection targets for the captured images from the cameras 383 and 384, respectively. This is because an object other than the floor does not affect the suction drive and the travel drive in the cleaning operation.

  In this embodiment, the current position detection unit 315 includes a GPS (Global Positioning System) positioning function unit, and detects the current position of the electric vacuum cleaner 30A. In this embodiment, the detected current position information is added to captured image information from each of the cameras 381 to 384 transmitted from the electric vacuum cleaner 30A to the server device 60, as will be described later. This current position information is used in consideration of the presence of a specific item corresponding to the country or region, such as banknotes or personal items, in the image recognition in the server device 60.

  As described above, the server device 60 has a function of performing foreign object image recognition processing on captured image information from each of the cameras 381 to 384 sent from the vacuum cleaner 30A. The configuration of the server device 60 shown in FIG. 10 shows the configuration of only the foreign object image recognition processing function unit for convenience.

  That is, the server device 60 is connected to the communication network 50 to communicate with the vacuum cleaner 30A, a captured image storage unit 602, an image recognition unit 603, an image registration memory 604, and a recognized foreign object. An information generation unit 605 is provided.

  The captured image storage unit 602 has the same function as the captured image storage unit 405 in the mobile phone terminal 40 of the second embodiment described above, and transmission image information transmitted from the vacuum cleaner 30A received through the communication interface 601. Then, each of the captured image information of the cameras 381 to 384 is decoded and separated and temporarily stored for image recognition in the image recognition unit 603. The captured image storage unit 602 has the same functions as the captured image memory 209 of the first embodiment, except for the function of decoding and separating each of the captured image information of the cameras 381 to 384 from the transmission image information. .

  The image recognition unit 603 has the same function as the image recognition unit 406 in the mobile phone terminal 40 of the second embodiment described above. The image registration memory 604 also has a function similar to that of the image registration memory 211 of the first embodiment described above, but the image registration memory 604 of the third embodiment registers various foreign object images. And keep it in memory. The image registration memory 604 also stores information on foreign matters that differ depending on the country and region, such as banknotes and coins, such as the country in which the foreign matter is issued.

  Note that the captured image information sent from the vacuum cleaner 30A to the server device 60 includes information on the current position of the vacuum cleaner 30A. The server device 60 recognizes the country or region where the electric vacuum cleaner 30A is cleaning from the information on the current position, and from the image registration memory 604, the foreign matter that differs depending on the country or region, such as banknotes or coins, is read from the image registration memory 604. The banknotes and coins of the country specified by the position are read out for comparison of image recognition. For example, in Japan, 10,000 yen bill, 5,000 yen bill, 2,000 yen bill, thousand yen bill, 500 yen coin, 100 yen coin, 50 yen coin, 10 yen coin, 5 yen coin, 1 yen coin, In the US, a comparison image for each country is used, such as dollar bills, dollars, cents, euro in the EU, and yuan in China. Of course, banknotes and coins of all ages, including old bills and old coins, can be read out for comparison in image recognition.

  When the image recognition unit 603 recognizes a foreign object based on the image recognition result from the image recognition unit 603, the recognized foreign object information generation unit 605 receives the recognized foreign object ID and thumbnail from the image registration memory 604. The image and the information of the suction drive control mode are acquired and transmitted to the electric vacuum cleaner 30A through the communication interface 601 as information on the foreign matter that has recognized them. The recognized foreign matter information generation unit 605 also includes location information on whether the foreign matter recognized from the image recognition unit 603 is present in the dust collection chamber of the vacuum cleaner 30A or outside without being sucked in the recognized foreign matter information. To be sent to the electric vacuum cleaner 30A.

  The recognized foreign object information generation unit 605 stores the captured image storage unit 602 in the captured image storage unit 602 when receiving a request for detailed image information from the vacuum cleaner 30A when the user clicks on a thumbnail image displayed on the display unit 312. The captured image information that has been processed is converted into information corresponding to the request and transmitted to the vacuum cleaner 30A through the communication interface 601.

  Although not shown in the image registration memory 604 in the third embodiment, the registered foreign matters such as high-value antiques, high-value old coins, premier expensive stamps, etc. Premier information is also stored. The server device 60 transmits the recognized foreign object information including the premium information.

[Description of Processing During Cleaning Operation in Third Embodiment]
In the third embodiment, the user presses the cleaning start button of the operation unit 304 of the electric vacuum cleaner 30A. Then, in the vacuum cleaner 30 </ b> A and the server device 60, processing operations described below are executed.

  FIG.11 and FIG.12 is a flowchart for demonstrating the example of the flow of processing operation in the vacuum cleaner 30A at this time. FIG. 13 is a flowchart for explaining an example of the flow of processing operations of the server device 60.

  First, the processing operation of the vacuum cleaner 30A will be described with reference to FIGS. In the following description, the control unit 301A executes the functions of the drive control unit 302, the transmission image generation unit 303, and the non-cleaning object detection unit 314 as software functions.

  The controller 301A determines whether or not the cleaning start button has been pressed, and waits for the cleaning to start (step S311). When it is determined in step S311 that the cleaning start button has been pressed, the control unit 301A starts driving the suction drive unit 306 and the travel drive unit 307 and instructs each of the cameras 381 to 384 to start imaging. To do. Further, the wireless communication unit 313 generates a communication path with the server device 60 (step S312).

  Then, the control unit 301A collects captured images from the cameras 381 to 384 during the cleaning operation, generates transmission image information, and together with the current position information detected by the current position detection unit 315, the wireless communication unit 313. Is transmitted to the server device 60 (step S313).

  Next, as described above, the control unit 301A performs processing for detecting foreign objects other than the cleaning target on the captured images from the cameras 381 to 384 based on the size and the luminance (step S314). Then, the control unit 301A determines whether or not the presence of foreign matter other than the cleaning target has been detected (step S315). When it is determined that the presence of foreign matter has been detected, the control unit 301A and the driving unit 306 and the traveling through the drive control unit 302 are determined. A temporary stop instruction signal is supplied to the drive unit 307 to temporarily stop the cleaning operation (step S316). Then, the control unit 301A determines whether or not the recognized foreign object information from the server device 60 has been received (step S317).

  Even when it is determined in step S315 that the presence of a foreign object has not been detected, the control unit 301A jumps the process to step S317 to determine whether or not the recognized foreign object information from the server device 60 has been received. .

  When it is determined in step S317 that the recognized foreign substance information from the server device 60 has not been received, the control unit 301A determines whether or not a predetermined time has elapsed since the cleaning operation was temporarily stopped (step S317). S318) When it is determined that the predetermined time or more has not elapsed, the process returns to step S317, and the processes after step S317 are repeated.

  In step S318, when it is determined that a predetermined time or more has elapsed since the cleaning operation was temporarily stopped, the control unit 301A supplies a drive start signal to the suction drive unit 306 and the travel drive unit 307 through the drive control unit 302. Then, the pause state is canceled and the cleaning operation is resumed (step S319). And control part 301A returns a process to step S313, and repeats the process after this step S313.

  When it is determined in step S317 that the recognized foreign matter information from the server device 60 has been received, the control unit 301A displays the received thumbnail image on the display screen of the display unit 312 with the location information (whether it is in the dust collection chamber, (External)) (step S320). In the third embodiment, when the vacuum cleaner 30A displays the thumbnail image of the foreign matter on the display screen of the display unit 312, the vacuum cleaner 30A does not perform alarm notification, but the first embodiment and the first embodiment described above. Of course, as in the second embodiment, alarm notification may also be performed.

  Next, the control unit 301A determines whether or not to stop the cleaning operation based on whether or not the recognized foreign object information includes a forced stop control signal (step S321 in FIG. 12). When it is determined in step S321 that the forced stop control signal is not included, the control unit 301A determines whether or not the end of the cleaning is detected depending on whether or not all the cleaning in the cleaning range is completed ( In step S322), when the end of cleaning is not detected, the process returns to step S313, the transmission image information is generated, and the cleaning operation is continued while being wirelessly transmitted to the server device 60.

  When it is determined in step S321 that the forced stop control signal is included, the control unit 301A stops driving the suction drive unit 306 and the travel drive unit 307, and for each of the cameras 381 to 384. Instructs to stop taking images. Further, the communication path with the server device 60 is disconnected (step S323).

  Even when the end of cleaning is detected in step S322, the control unit 301A advances the process to step S323 to stop the driving of the suction drive unit 306 and the travel drive unit 307, and for each of the cameras 381 to 384. Then, the stop of image capturing is instructed, and the communication path with the server device 60 is disconnected.

  Thus, the process during the cleaning operation in the electric vacuum cleaner 30A is completed.

Next, the processing operation of the server device 60 will be described with reference to FIG.
First, the server device 60 determines whether or not a communication path generation request has been received from the electric vacuum cleaner 30A (step S601). When it is determined that a communication path generation request has been received, A communication path with the cleaner 30A is generated (step S602).

  Next, the server device 60 receives the transmission image information transmitted from the vacuum cleaner 30A through the generated wireless communication path, and separately acquires the captured image information from the cameras 381 to 384, Similarly to the first embodiment, image recognition processing is executed by comparison by pattern matching between the captured images from the cameras 381 to 384 and the image of the foreign object stored in the image registration memory 407 (step S603). ).

  Then, the server device 60 determines whether or not a foreign object has been recognized as a result of the image recognition in step S603 (step S604). When it is determined in step S604 that no foreign object is recognized, the server device 60 determines whether or not the communication path is cut by the electric vacuum cleaner 30A (step S608). When it is determined in step S608 that the communication path is disconnected, the server device 60 determines that the cleaning operation in the electric vacuum cleaner 30A has been stopped, and ends this processing routine.

  If it is determined in step S608 that the communication path is not disconnected, the server device 60 determines that the cleaning operation in the electric vacuum cleaner 30A is continued, and returns the process to step S603. The processing after S603 is repeated.

  If it is determined in step S604 that the foreign object has been recognized as a result of the image recognition, the recognized foreign object information including the thumbnail image of the recognized foreign object, the foreign object ID, the location information, and the premier information is generated. It transmits to the cleaner 30A (step S605).

  Further, the server device 60 refers to the stored contents of the image registration memory 604, acquires information on the suction drive control mode stored corresponding to the foreign matter ID of the recognized foreign matter, and needs to forcibly stop the cleaning operation. Whether or not (step S606).

  When it is determined in step S606 that the forced stop of the cleaning operation is not necessary, the server device 60 continues the cleaning operation in the electric vacuum cleaner 30A without generating the forced stop control signal, and the process proceeds to step S603. The process after step S603 is repeated.

  Next, when it is determined in step S606 that the forced stop of the cleaning operation is necessary, the server device 60 generates a forced stop control signal for stopping the driving of the suction drive unit 306 and the travel drive unit 307, and the electric cleaning is performed. It transmits to the machine 30A (step S607). Thereafter, the server device 60 proceeds to step S608, waits for determination that the communication path is disconnected in step S608, and ends this processing routine.

  In the third embodiment, when a thumbnail image displayed on the display screen of the display unit 312 is clicked by the user, a detailed captured image corresponding to the thumbnail image is displayed together with the foreign matter ID of the thumbnail image. The vacuum cleaner 30 </ b> A sends an information acquisition request to the server device 60. In response to this acquisition request, the server device 60 reads actual captured image information and peripheral image information of the foreign substance specified by the foreign object ID from the captured image storage unit 602, and transmits the captured image information to the electric vacuum cleaner 30A. To do. The vacuum cleaner 30 </ b> A displays the received captured image information on the display screen of the display unit 312, and displays peripheral captured images in front and behind the suction unit at the cleaning position where the foreign object has been image-recognized. Is done. For this reason, the user can accurately recognize what the detected foreign matter is, and can easily know where the foreign matter is located, which is convenient. Further, when the recognized foreign object information received in step S605 includes premier information, the vacuum cleaner 30A changes the background color of the thumbnail image of the foreign object, blinks the display screen, or notifies an alarm. Can do. This prevents the user from overlooking high-value foreign objects and expensive foreign objects.

[Effect of the third embodiment]
As described above, in the electric vacuum cleaner of the third embodiment, when the self-propelled electric vacuum cleaner 30A recognizes the presence of a foreign object, the temporary cleaning operation is stopped and the recognition from the server device 60 is performed. Wait for foreign material information to arrive. When the vacuum cleaner 30A does not receive the recognized foreign object information from the server device 60 even after waiting for a certain period of time or longer, the server device 60 does not recognize the foreign object, so that the cleaning operation can be continued. It is judged that there is no problem, and the cleaning operation is continued as it is. When the foreign object is recognized by the server device 60, the recognized foreign object information is sent within a certain time, so the vacuum cleaner 30A waits for the reception, and the recognized foreign object information includes a forced stop control signal. The cleaning operation can be stopped according to the forced stop control signal.

  As described above, the self-propelled electric vacuum cleaner 30A temporarily stops the cleaning operation in consideration of the delay of the image recognition process in the server device 60. Therefore, regardless of the delay of the image recognition process in the server device 60, Control as expected. For this reason, the occurrence of a failure due to the self-propelled electric vacuum cleaner 30 </ b> A sucking in a foreign substance that should not be sucked in can be prevented.

  In addition, after the temporary stop, the vacuum cleaner 30A itself makes a zoom image of the foreign object or the vacuum cleaner 30A moves and images the foreign object from another angle according to the determination by the electric vacuum cleaner 30A or an instruction from the server device 60. It may be. Further, the electric vacuum cleaner 30A may be configured to automatically zoom and image a foreign object when temporarily stopped. By transmitting the picked-up image picked up by this zoom image pickup from the vacuum cleaner 30A to the server device 60 again, the server device 60 can perform image recognition with higher accuracy.

  In addition to the above effects, it goes without saying that the same effects as those of the first and second embodiments described above can also be obtained in the third embodiment.

[Modification of Third Embodiment]
In the third embodiment described above, the current position detection unit 315 including the GPS positioning function unit is provided. In short, the country and the region where the electric vacuum cleaner 30A is used can be transmitted to the server device 60. As long as it can be performed, for example, the user may input the address of the place where the vacuum cleaner 30 </ b> A is used, and the input address information may be transmitted to the server device 60.

  Moreover, in the above-mentioned 3rd Embodiment, although the vacuum cleaner 30A was not provided with the image recognition part, you may provide an image recognition part also in the vacuum cleaner 30A. In that case, the user registers an image of the foreign object in the image registration memory of the vacuum cleaner 30A, and the image recognition unit of the vacuum cleaner 30A recognizes the foreign object registered in the image registration memory. The server device 60 also recognizes foreign objects that are not registered in the image registration memory of the electric vacuum cleaner 30A. Note that the number of server devices is not limited to one, and a plurality of server devices can be used in cooperation.

  Also in the third embodiment, the display unit 312 may be provided in a portable device such as a mobile phone terminal wirelessly connected to the electric vacuum cleaner 30A, a tablet terminal, a wearable terminal, or a personal computer. . Moreover, you may make it provide in apparatus provided with various display parts, such as a desktop apparatus and a stationary apparatus.

[Other Embodiments or Modifications]
In addition, although the above description was mainly demonstrated supposing the household vacuum cleaner, it can be applied also to the business vacuum cleaner used in a factory, a store, a restaurant, etc. Needless to say. Of course, the present invention can be applied to various types of vacuum cleaners such as a desktop vacuum cleaner such as a small vacuum cleaner dedicated to the desk, and a futon cleaner.

  In the above embodiment, camera shooting is performed by continuous shooting (moving image shooting), but intermittent shooting by still images may be used. Moreover, both may be mixed.

  Also, the camera shooting method and resolution are not the same, and may be different depending on the mode and situation. For example, a lost article mode (searchable object mode) is provided, and the flash shooting is performed continuously when the camera shooting is started, or when a foreign object other than the object to be cleaned (lost object candidate, searchable object candidate) is detected after the camera shooting starts. When shooting in zoomed-out mode, shooting from multiple angles, increasing the resolution, using multiple cameras, etc., when using the lost item mode, you can use a different camera shooting mode from normal shooting to find lost items (search items). can do. Further, the processing method and accuracy of image recognition of a captured image taken by the camera may be changed according to the mode and the situation.

  Of the images registered in the image registration memory, a specific foreign object designated to be searched in the lost article mode may be recognized in multiple stages. For example, to reduce the probability of oversight as much as possible, first use a coarse image to search for a range of similarities, and then find a high-definition image to narrow down the found objects. Utilizing this, image recognition from a plurality of angles is performed. Also, special processing for image recognition may be added. For example, various feature extraction and various filtering such as contour (edge) extraction and enhancement, luminance extraction and enhancement, hue extraction and enhancement, and various enhancement processes may be added and used for image recognition. Furthermore, it is possible to perform image recognition without compressing the captured image without changing the captured image, or to increase the image recognition processing time more than usual, for example, several times as long.

  If the lost item is a ring, such as a ruby platinum ring, the color and luster (light reflection) can be selectively discriminated to detect the red color of the ruby and the silver color (glossy grayish white) of the platinum ring. Try to increase. Further, the shape discrimination ability can be selectively enhanced so as to increase the detection accuracy by the shape of the ruby stone or the size of the ring defined by the inner diameter size.

  Moreover, the case where a lost article is a product with which the size etc. are normalized can be considered. For example, the SD card has a width of 24 mm, a length of 32 mm, and a thickness of 2.1 mm, and the micro SD card has a width of 11 mm, a length of 15 mm, and a thickness of 1.0 mm, and the shape is precisely defined by the SD standard. Therefore, when the lost item is an SD card or a micro SD card, first search for a rectangular object based on the shape, then filter by whether the aspect ratio (length to width ratio) is appropriate, then precisely What is necessary is image recognition. Of course, if a distance meter is used, absolute values such as the inner diameter and length of the object can be measured, and therefore identification can be made using the measured values. Even if there is no distance meter, if there is an object to be compared which is a reference for length and size, the absolute value of the length and size can be grasped by relative comparison with the comparison object. Furthermore, the logo and characters on the surface of the SD card or micro SD card can be recognized and identified.

  In the case of a contact lens with a transparent fallen object, when shooting with a camera, the background color is taken through, so the difference in brightness and color is detected accurately, or light such as a flash is applied to make it clear. The shape of the thin aspheric lens, which is peculiar to the contact lens, may be identified after detecting the reflected light that shines to improve the detection accuracy. Normally, hard contact lenses have a diameter of around 9 mm, and soft contact lenses have a diameter of around 13 mm. Therefore, by specifying whether the type of contact lens dropped is hard or soft, the identification ability for the size is enhanced and the detection is easy. be able to.

  Furthermore, even if the captured image of a foreign object has a low similarity to a lost object and there is a high possibility that it is not a lost object, the similarity level threshold is made variable to detect the foreign object as a lost object in order to reduce the risk of missing the lost object. It is also possible to perform.

  Needless to say, various methods such as template matching, cluster analysis, and neural network can be applied to image recognition and similarity level calculation.

  In the lost item mode, the suction drive control may be set to “weak suction drive” or “weakest suction drive” until a lost item is found. Furthermore, “suction drive stop” can be set. In addition, in the case of a self-propelled vacuum cleaner, not only the mode of suction drive control but also the mode of travel drive control, such as “running drive low speed” and “running drive ultra-low speed” until a lost item is discovered Control aspects can be used. Further, “travel drive stop (temporary stop)” may be performed every predetermined time or every predetermined distance. In this case, in a paused state, the camera can be carefully photographed using not only normal photographing but also flash photographing or zoom photographing. Furthermore, panoramic photography or 360-degree photography may be performed while searching for lost items while being rotated.

  Note that foreign objects to be searched in the lost article mode, for example, images of a ring, an SD card, a micro SD card, a contact lens, etc., can be read from the image registration memory, displayed on the display screen, and designated.

  In the above embodiment, the recognition (detection) of the foreign matter during the cleaning operation is performed by recognizing the captured image information from the camera. For example, an infrared sensor, a microphone, an ultrasonic sensor, a tactile sensor The foreign matter may be recognized (detected) by using another sensor such as a biosensor, an odor sensor, or a metal detection sensor. Of course, you may use these multiple types of sensors together. For example, when a microphone is used as the sensor, it is possible to recognize that the object is a foreign object other than the object to be cleaned if sound or sound is collected by the microphone and an object that emits sound or sound is detected. In this case, “foreign matter detection based on the recognition result of the image picked up by the microphone” in the flowchart of FIG. If it replaces by, the vacuum cleaner which can detect the foreign material by voice recognition will be realizable. As a mode of performing foreign object detection by voice recognition, in addition to a mode of collecting sound near the suction part by the sound collecting part during the cleaning operation and detecting it before the foreign object is sucked into the vacuum cleaner, metal or bags It is conceivable to detect the foreign matter by collecting the abnormal sound generated after the foreign matter is sucked into the vacuum cleaner. When foreign matter detection by voice recognition is performed, a mode of suction drive control such as suction drive stop can be considered as in the case of foreign matter detection by image recognition. Moreover, the aspect which performs alarm alerting | reporting by display control, audio | voice control, vibration control, etc. can be considered. Note that noise generated by the vacuum cleaner itself can be removed by the sound source separation process.

  Note that an infrared camera or a video camera may be used as the cameras 171 to 174 and the cameras 381 to 384. Moreover, you may use them together. In the above embodiment, the case where the number of cameras is four has been described, but it goes without saying that the number of cameras is not limited. Furthermore, the installation location of the camera is not limited to this embodiment, and various aspects are possible.

  In the above embodiment, a stationary object is exemplified as a cleaning object or a foreign object other than the cleaning object. However, a moving object such as a creature such as an insect or a motor-driven toy may be used. .

  The preferred embodiments of the present invention have been described in detail above with reference to the drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present invention can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present invention.

1: Vacuum cleaner 10: Vacuum cleaner body 11: Suction hose 12: Handle 13: Fitting pipe 14: Suction part 141: Suction port 15: Operation part 16: Display part 16D: Display screen 101: Dust collection chamber 102: Suction drive Units 171 to 174: Camera 20: Control circuit unit 201: Control unit 202: Drive control unit 210: Image recognition unit 211: Image registration memory 213: Display image generation unit 214: Alarm generation unit 30: Self-propelled electric vacuum cleaner 31: Suction unit 32: Dust collection chambers 381-384: Camera 406: Image recognition unit 407: Image registration memory 409: Display image generation unit 412: Display unit

Claims (14)

A suction part having a suction port for sucking a cleaning object;
A suction drive unit for sucking the cleaning object from the suction port and taking it into a dust collection chamber;
An imaging unit for imaging foreign objects other than the cleaning object and the cleaning object;
A storage unit;
A display unit comprising a display screen;
The foreign matter image, foreign matter specifying information for displaying what the foreign matter is, and a control mode for determining whether or not the foreign matter is sucked by the suction driving unit are stored in the storage unit in association with each other. Registration means for registering ,
During the cleaning operation, the captured image captured by the imaging unit, performs image recognition by comparing the image of the registered the foreign substance stored in the storage unit, stored in the storage unit the An image recognition unit for recognizing the registered foreign matter,
When the image recognition unit recognizes the registered foreign object, the suction drive unit is controlled based on the control mode stored in the storage unit corresponding to the recognized foreign object. Control means for displaying on the display screen a display for identifying what the recognized foreign matter is ;
An electric cleaning device comprising: A suction part having a suction port for sucking a cleaning object;
A suction drive unit for sucking the cleaning object from the suction port and taking it into a dust collection chamber;
A travel drive for self-propelling the device;
An imaging unit for imaging foreign objects other than the cleaning object and the cleaning object;
A storage unit;
A display unit comprising a display screen;
The storage unit in association with the image of the foreign object, information for specifying a foreign object for displaying what the foreign object is, and the control mode of the suction drive unit and / or the travel drive unit corresponding to the foreign object Registration means for storing and registering in
During the cleaning operation, image recognition is performed by comparing the captured image captured by the imaging unit with the registered image of the foreign object stored in the storage unit, and the image stored in the storage unit is stored. An image recognition unit for recognizing the registered foreign matter,
When the image recognition unit recognizes the registered foreign object, based on the control mode and the foreign object specifying information stored in the storage unit corresponding to the recognized foreign object, Control means for controlling the suction drive unit and / or the travel drive unit and displaying on the display screen a display for identifying what the recognized foreign matter is;
An electric cleaning device comprising: A non-cleaning object detection unit that detects foreign objects other than the cleaning object from a captured image during the cleaning operation of the imaging unit;
When the non-cleaning object detection unit detects a foreign object other than the cleaning object, the control unit temporarily stops the operations of the suction driving unit and the travel driving unit.
The electric vacuum cleaner according to claim 2. The control unit waits for a determination result of whether or not the registered foreign object is recognized by the image recognition unit after the temporary stop, and when the registered foreign object is recognized by the image recognition unit, The control according to the control mode stored in the storage unit is performed, and when the registered foreign object is not recognized by the image recognition unit, the pause is canceled.
The electric cleaning device according to claim 3. The imaging unit is composed of a plurality of imaging units provided at different sites,
The control means displays whether or not the recognized foreign matter has been taken into the dust collection chamber based on which of the plurality of imaging portions is the imaging unit that has recognized the foreign matter by the image recognition unit. Do more
The electric vacuum cleaner according to any one of claims 1 to 4, wherein the electric vacuum cleaner is provided. Foreign substance specific information for display the whether the foreign matter is what has electric vacuum according to any one of claims 1 to 5, characterized in that it comprises a thumbnail image of the name or the foreign substance of the foreign substance apparatus. Electric vacuum device according to any one of claims 1 to 6, wherein the means for notifying an alarm in conjunction with the display in which the foreign matter to identify what is, further comprising. When the foreign object is recognized by the image recognition unit, a captured image of the foreign object imaged by the imaging unit is displayed on the display screen based on a predetermined operation by the user.
The electric vacuum cleaner according to any one of claims 1 to 7, wherein The imaging unit includes a first imaging unit provided at a site that images a subject in the moving direction of the suction port during the cleaning operation, and the first imaging unit causes the image recognition unit to perform the foreign object. when recognized, the foreign substance is captured surrounding image for image recognition position is determined somewhere on the display screen of the display unit, the display means displays the foreign matter together with the peripheral image electric vacuum device according to any one of claims 1 to 8. Wherein the display screen of the display unit, electric according to any one of claims 1 to 9, characterized in that the list of the cleaning operation of the foreign matter that the recognized by the image recognition unit in the image is displayed Cleaning device. The display unit comprises a separate device having a wireless communication function,
Electric vacuum device according to any one of claims 1 to 10, characterized in that it comprises the display unit and a wireless communication unit for wireless communication separately. The storage unit and the image recognition unit are provided in separate devices having a wireless communication function,
Electric vacuum device according to any one of claims 1 to 11, characterized in that it comprises a wireless communication unit for apparatus and radio communication of the further body. The vacuum cleaner according to any one of claims 1 to 12 , further comprising an imaging control unit that supplies a control signal to the imaging unit according to the foreign matter recognized by the image recognition unit. The image recognition unit includes a first image recognition unit built in the electric vacuum cleaner and a second image recognition unit including an image recognition processing device connected through a communication network. A communication unit for performing communication is further provided,
Means for transmitting the captured image from the imaging unit to the image recognition processing device through the communication unit, and receiving information of an image recognition result from the image recognition processing device through the communication unit;
The first image recognition unit recognizes the foreign object by comparing the captured image with an image of the foreign object stored in a storage unit included in the electric vacuum cleaner,
The control means performs control according to the foreign matter recognized by the first image recognition unit, and performs control based on the image recognition result received from the second image recognition unit. The electric vacuum cleaner of Claim 1 or Claim 2 .

Images