JP5008728B2 - User detection method and apparatus, and control method - Google Patents

Description

  The present invention relates to a detection method and apparatus for a user in the vicinity of a door (door), for example, a detection method and apparatus for a passenger in an elevator car. The present invention also relates to a control method for performing opening / closing control of a car and a landing door, and operation control of the car.

  Due to the recent increase in crime prevention awareness, surveillance cameras are also installed in the car in elevator systems, and monitoring images are generally displayed and recorded in a manager's room or security company. The camera is installed in the elevator car. Installation is becoming common.

As described above, in an elevator in which a surveillance camera is installed, for opening / closing control of a door (camera image), passengers photographed with the same camera as the image photographed by the surveillance camera and set in advance The presence / absence of a passenger is determined by comparing with a background image without a door, and door opening / closing control is performed. Furthermore, the photographed image is divided into several areas, the direction of the passenger moving in the area is determined, and in conjunction with this, safer door opening / closing control is performed (for example, see Patent Document 1).
Japanese Patent Laying-Open No. 2005-179030 (first page, FIG. 2)

  However, in determining whether there is a passenger near the door, the image is divided into regions, and each region is determined by pattern recognition, etc., so processing for image recognition is required, and further, the movement of the passenger is determined. Therefore, there is a problem that a high-performance arithmetic device and a large memory capacity are required. In addition, the cost can be reduced by lowering the performance of the computing device, but the image recognition processing capability is reduced, so the passenger movement determination process is slowed down, and the movement of the passenger and the movement of the door opening / closing control are linked. There was a problem of disappearing. There were similar problems in vehicles other than elevators. In addition, there are similar problems in doors other than vehicle doors.

The user detection method of the present invention is:
An extraction step of extracting the edge of the doorway by comparing the image data in the door-closed state obtained from a camera that shoots to include the doorway with the door, and the image data in the door-open state,
An inspection area determining step for calculating the coordinates of the center point of the edge and the length of the edge side from the extracted information on the edge of the entrance and exit, and determining one or more inspection areas centered on the center point ; ,
A user detection step of detecting a user around the door based on a change in the image data of the inspection region determined in the inspection region determination step, and the inspection region has a length of the calculated side It is determined based on the above.
In addition, the user detection device of the present invention is
Extraction means for extracting the edge of the doorway by comparing the image data in the door-closed state and the image data in the door-open state obtained from a camera that shoots to include the doorway with the door,
An inspection area determination means for calculating the coordinates of the center point of the edge and the length of the edge side from the extracted information on the edge of the doorway, and determining one or more inspection areas centered on the center point; ,
User detection means for detecting a user around the door based on a change in the image data of the inspection area determined by the inspection area determination means;
And the inspection area is determined based on the calculated length of the side.

According to the present invention, since the image recognition process is performed from the image near the entrance / exit, the movement of the user near the door can be linked to the control of the door, and the safety can be improved.
In addition, since the camera image automatically determines where the door is in the image, there is no need to adjust the image based on the camera installation, improving the degree of freedom of the camera installation position, The trouble of installing the camera can be reduced.

It is a figure which shows the elevator control system containing the passenger detection apparatus in an elevator car of Embodiment 1 of this invention. It is a block diagram which shows an example of the image process part 106 of FIG. It is a figure which shows an example of the surveillance camera image | video in the elevator car of a door closed state in Embodiment 1 of this invention. It is a figure which shows an example in the elevator car monitoring camera image | video of the door open state in Embodiment 1 of this invention. It is a flowchart which shows the operation | movement which extracts the information showing the position and magnitude | size of a test | inspection area | region in Embodiment 1 of this invention. It is a figure which shows the example of the test | inspection area | region determined by the operation | movement of FIG. It is a figure which shows the relationship between the pixel of the starting point of the test | inspection area | region shown by FIG. 6, and the memory location on the video memory map of the data of the said pixel. It is a figure which shows the information memorize | stored in the test | inspection area | region information memory 204 of FIG. It is a flowchart which shows the procedure of preparation of the image for a reference in Embodiment 1 of this invention. It is a flowchart which shows the procedure of the comparison process in Embodiment 1 of this invention. It is a flowchart which shows the procedure of the comparison process in Embodiment 1 of this invention. (A)-(c) is a figure explaining the judgment method of the direction of movement in Embodiment 1 of this invention.

Explanation of symbols

  106 image processing unit, 202 video memory, 203 determination processing unit, 204 inspection area information memory, 205 reference memory

  Hereinafter, the case where the present invention is applied to an elevator door opening / closing control system equipped with a surveillance camera will be described.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating an elevator door control system according to Embodiment 1 of the present invention. In the figure, reference numeral 100 denotes a “car” that carries people and goods by an elevator, and 101 denotes a surveillance camera installed in the car 100. Reference numeral 102 denotes an information display device installed in the elevator car, information on which floor the elevator car 100 is currently located, driving direction (information on going up or down), or Then, an image in the car photographed by the monitoring camera 101 is displayed.

Reference numeral 103 denotes an operation panel installed in the car 100, on which a button on each floor, a door control button, an emergency button, and the like for instructing a floor (floor) desired to be stopped are installed. Passengers in the car 100 operate the operation panel 103 to instruct a floor to go to.
An image processing unit 106 inputs a monitoring camera video from the monitoring camera 101, analyzes the presence / absence of a passenger and the direction of movement from the monitoring camera video, and transmits the presence / absence of a passenger's movement to the door control unit 107. The surveillance camera video is transmitted to the information display device 102.

Reference numeral 104 denotes a door in the car 100, which is called a car door. The car door 104 moves together with the car 100 and is always closed during the movement of the car 100 to ensure the safety of passengers in the car 100. Reference numeral 105 denotes a door installed on each floor and is referred to as a floor door. The floor door 105 is normally closed. When the car 100 arrives on the floor, the floor door 105 and the car door 104 are simultaneously controlled to open and close, and passengers get on and off.
The operation control of the elevator is controlled by the elevator operation control unit 108, and is controlled by the motor 109 connected to the car 100 and the rope 110 for performing movement control and the door control unit 107 for controlling the door. The door control unit 107 controls the car door 104 and the floor door 105 based on an instruction from the elevator operation control unit 108 and an instruction from the image processing unit 106.

Next, the image processing unit 106 will be described with reference to FIG. The image processing unit 106 shown in FIG.
An AD converter 201, a video memory 202, a determination processing unit 203, an inspection area information memory 204, and a reference memory 205 are included.

  The AD converter 201 converts an analog signal representing a monitoring camera image input from the monitoring camera 101 into a digital signal. The digital video signal output from the AD converter 201 is transmitted to the information display device 102 and input to the determination processing unit 203.

The video memory 202 temporarily stores digital video signals.
The inspection area information memory 204 stores information indicating the position and size of an inspection area used for image determination, which will be described later, and the direction of an entrance / exit with respect to the inspection area.
The reference memory 205 stores a reference image used for image determination described later, and includes a first temporary storage area 205b and a second temporary storage area 205c in addition to the reference image area 205a.

  The determination processing unit 203 includes first and second counters 203a and 203b, receives the digital video signal input from the AD converter 201, stores video data stored in the video memory 202, and references the reference memory 205. Reference image stored in the image area 205a, temporary storage data stored in the temporary storage areas 205b and 205c, information indicating the position and size of the inspection area stored in the inspection area information memory 204, and the door Image determination is performed with reference to the door information DS transmitted from the control unit 107, and door control information DC is generated based on the determination result. The image processing unit 106 or a part thereof, for example, the determination processing unit 203 can be configured by a programmed computer, for example.

  A control signal DC indicating the determination result is transmitted to the door control unit 107, and the door control unit 107 controls the car door 104 and the floor door 105. The door control will be described later.

  Next, an image determination procedure for determining the presence or absence of a passenger and movement performed in the determination processing unit 203 will be described. The determination processing unit 203 inputs a digital video signal obtained by converting the video from the monitoring camera 101 by the AD converter 201. At this time, the video of the monitoring camera 101 is transmitted as a moving image, but since it can be recognized as one still image by the vertical synchronization signal of the monitoring camera 101, the determination processing unit 203 receives the input digital video signal. Treat as a still image.

  The determination processing unit 203 performs three processes: (1) determination of an inspection area, (2) preparation of a reference image, and (3) comparison processing, for the presence / absence of passengers and movement determination. These will be described in order.

First, (1) determination of the inspection area will be described.
FIG. 3 is an example of an image taken by the monitoring camera 101 installed in the upper part of the car. Since the inside of the car is photographed at an angle, the image becomes distorted, and FIG. 3 imitates it. In the figure, 103 is an operation panel, 102 is an information display device, and 104 is a double door. Further, 301 is the left side of the elevator car from the inside of the car toward the car door 104, 302 is the face opposite to the car door 104, 303 is the bottom face of the car, and 304 is from the car to the car door 104. On the right side of the elevator car on the right side, 305 is the same side as the car door 104, the surface on which the operation panel 103 and the video display device 102 are installed, and 306 is the same side as the car door 104, 305 and a surface at a substantially symmetrical position around the car door. In addition, this invention is applicable not only to the elevator provided with the double door as shown in FIG. 3, but a single door elevator.

  4 is an example of an image when the car 100 stops on an arbitrary floor and both the car door 104 and the floor door 105 are opened. The difference from FIG. 3 is whether the door is open. There is no difference. A hatched line 330 indicates the other side of the door when the door is opened.

A method for determining the inspection area will be described with reference to FIGS. 3 and 4 and the flowchart of FIG.
The inspection area is determined in the absence of passengers. (Note that the items that are always installed in the elevator car are kept installed.) For example, it may be executed at midnight or early morning when there are few elevator users. In this case, it may be estimated that there is no passenger in the elevator on the condition that it has been stopped on the same floor for a long time and the door is not opened or closed. You may use the image imaged in the state confirmed that there was no passenger by methods other than determination as a reference image.

In FIG. 5, first, a signal DC for controlling closing of the car door 104 and the floor door 105 is transmitted from the determination processing unit 203 to the door control unit 107 in order to determine the inspection area, and as a response, the door is closed. A signal DS (indicating that the door is closed in this case) DS is transmitted from the door control unit 107 to the determination processing unit 203 (S502).
With the door closed, a digital video signal (digital video obtained from the surveillance camera video) obtained by AD conversion of the video signal output from the surveillance camera 101 is converted into the reference memory 205. The reference image area 205a is copied (S503). An example of the digital video obtained from the surveillance camera video saved in the reference image area 205a at this time is shown in FIG.
Next, a signal DC for controlling to open the car door 104 and the floor door 105 is transmitted from the determination processing unit 203 to the door control unit 107, and a signal DS indicating that the door is open is received as a response. To the determination processing unit 203 (S504).

  With this door open, a digital image obtained from the surveillance camera image is copied to the video memory 202 (S505). FIG. 4 shows an example of a digital image obtained from the surveillance camera image stored in the video memory 202 at this time.

  In step S506 of FIG. 5, the data stored in the reference image area 205a is compared with the data stored in the video memory 202. The change at this time is that the door is closed in FIG. 3, whereas the door is open in FIG. 4, and the hatched portion indicated by reference numeral 330 in FIG. Next, the edges 340, 341, and 342 of the changed portion are extracted (S507). Since the edges 340, 341, and 342 of the changed portion correspond to the edges of the entrance / exit (entrance / exit opening), they are extracted linearly.

  Next, the extracted edge of the changed portion (entrance / exit edge) 340, 341, 342 is converted into coordinates, and the length of each side of the changed portion of the edge 340, 341, 342 (entrance / exit) is determined from the coordinate information. The length of the part within the imaging range (indicated by the frame line AV) of each side constituting the edge is obtained, and the center of each side (the center of the part within the imaging range AV of each side constituting the edge of the entrance / exit) ) The positions of 340c, 341c, and 342c are calculated, set to the coordinates of the reference point of the inspection area, and the size (length of one side) of the inspection area is determined. For example, the center of the inspection region coincides with the reference points 340c, 341c, and 342c of the inspection region, and each side constitutes the side that forms the edge 340, 341, 342 of the entrance / exit (the center is determined as the reference point). A square equal to half the length of the side. Then, from the reference point of each inspection area and the length of each side, the coordinates of the top left vertices 310b, 311b, 312b of the inspection area are obtained as start point coordinates. Further, the relative position and direction of the entrance / exit are determined for each inspection area (S508).

  In this way, for example, the first, second and third inspection areas 310, 311 and 312 shown in FIG. 6 are set, and for each of these, the coordinates of the start point, the size of the area (one side Length), the direction of the doorway is determined. The direction of the doorway is “right” in the first inspection region 310, “left” in the second inspection region 311, and “up” in the third inspection region 312.

In step S509, the coordinates of the starting point of the inspection area at this time are converted into address information of digital video data stored in the video memory 202. Since the surveillance camera is a camera that is installed and fixed in the car 100, the digital video obtained from the video of the surveillance camera to be photographed has almost the same configuration, and is overwritten on the memory when it is taken into the video memory 202. If configured so, the digital video of each pixel obtained from the surveillance camera video is written to the same memory address in the video memory 202. That is, the digital video obtained from the surveillance camera video is supplied in the order of the lines in the screen from top to bottom and the pixels in each line from left to right, and is also supplied to the video memory 202 in that order. Written in. Therefore, the data of each of the inspection areas 310, 311 and 312 is not stored for each inspection area but is stored discontinuously. However, the data of each pixel in each inspection area is written at the same address in the video memory.
For example, the data of the start point (upper left point) of the inspection area 310 is stored at a specific address 351 in the video memory 202, for example, an address represented by “0x01005000”, as shown in FIG. The same applies to the inspection areas 311 and 321.

Next, memory address information for specifying the coordinates of the start point of the inspection area, information indicating the size of the inspection area, and the direction of the entrance / exit are stored in the inspection area information memory 204 in the image processing unit 106 (S510). . Information representing the length of one side is stored as information representing the size of the inspection area. An example of storage at this time is shown in FIG. In FIG. 8, three columns 601, 602, and 603 are provided for the first, second, and third inspection areas 310, 311, and 312, and each correspond to the coordinates of the start point of the inspection area. The address, the size of the inspection area, and the direction of the entrance / exit are indicated.
The image data of the inspection area can be read sequentially by designating the address 351 storing the data of the starting point of the inspection area and the size of the inspection area (the number of pixels in the horizontal direction and the number of lines in the vertical direction). it can.

Next, (2) a reference image is prepared.
In FIG. 9, first, a signal DC for controlling the closing of the door for obtaining a reference image is transmitted from the determination processing unit 203 to the door control unit 107, and as a response, a signal DS indicating that the door is closed. Is transmitted from the door control unit 107 to the determination processing unit 203 (S531).

  Next, the digital video obtained from the surveillance camera image is copied to the video memory 202 (S532), and the address corresponding to the coordinates of the start point of the inspection area and the information indicating the size of the inspection area are referred to, Is copied to the reference image area 205a (S533). That is, only the image of the inspection area necessary for reference is stored in the reference image area 205a.

Next, (3) the comparison process will be described. The comparison process is always performed during elevator operation.
In FIG. 10, first, the count value i of the first counter 203a in the determination processing unit 203 is set to zero (S551).
Then, the digital video obtained from the surveillance camera image is copied to the video memory 202 for one screen (S552), and information on the inspection area stored in the inspection area information memory 204 (information indicating the starting point and size of the inspection area) ) Based on the reference image data stored in the reference image area 205a (S553).

  If it is determined by comparison that there is no change (No in S554), it can be determined that the passenger is not moving, the count value i of the counter 203a is incremented by 1 (S556), and the count value i of the counter 203a is 5 If this is the case, that is, if there is no change of 5 frames or more, the image just acquired at that time is updated as a new reference image (overwritten on the reference image area 205a), and the first and second temporary storage areas 205b are updated. And the first and second temporarily stored data (described later) stored in 205c are deleted (S558). In this case, since the passenger is not moving, a signal DC indicating a determination result that there is no problem even if the door is closed is transmitted to the door control unit 107 (S559).

Next, if it is determined by the comparison in step S554 that there is a change (Yes in S554), the process proceeds to step S571 in FIG. 11 and data determined to have changed (stored in the video memory 202 at the time of this determination). Data) is copied to the first temporary storage area 205b as the first temporary storage data (S571). Then, the count j of the second counter 203b in the determination processing unit 203 is reset, and the next one screen of the digital video obtained from the surveillance camera image is copied to the video memory 202 (S573). The first temporary storage stored in the first temporary storage area 205b based on the address indicating the coordinates of the start point of the inspection area stored in the inspection area information memory 204 and the information indicating the size of the inspection area The data is compared (S574).
If the comparison shows no change (No in S575), the count value j of the counter 203b is incremented by 1 (S576), and if the count value j of the counter 203b becomes 5 or more, that is, if there is no change of 5 frames or more ( (Yes in S577), the first temporarily stored data is copied as reference image data to the reference image area 205a (S578).

  In this case, since the passenger is not moving, a signal DC indicating a determination result that there is no problem even if the door is closed is transmitted to the door control unit 107 (S579).

If it is determined by the comparison in step S575 that there is a change (Yes in S575), the direction of movement of the passenger is determined.
First, in order to determine the direction of movement, another digital video obtained from the monitoring camera image is continuously acquired and stored in the second temporary storage area 205c as second temporary storage data (S580).

  In the reference image data 205, information indicating the direction of the entrance / exit is stored as related information, as shown in FIG. 8, and based on the direction of the entrance / exit, the movement of the changed portion in the comparison process is stored. The direction is determined, and the direction of passenger movement is determined (S581). In the detection of passenger movement, movement of luggage, wheelchairs, strollers, etc. that passengers carry is handled in the same manner as passenger movement. Furthermore, the “passenger” mentioned here includes an elevator crew.

A method for determining passenger movement will be described with reference to FIGS. 12 (a), 12 (b), and 12 (c). 12A, 12B, and 12C show reference image data 701, first temporary storage data 702, and second temporary storage data 704, respectively, and there is a change between them. Yes. The doorway direction is the right side on the drawing.
A portion where the first temporary storage data 702 has changed is a hatched portion 703, and a portion where the second temporary storage data has changed is a hatched portion 705.
The first temporary storage data 702 and the second temporary storage data 704 are compared for each dot (pixel) of the image, and the change in the data is analyzed. That is, the difference (change) between the changed portion 703 of the first temporary storage data 702 and the changed portion 705 of the second temporary storage data 704 is obtained. In this case, for example, motion detection may be performed based on the data of each dot (pixel) of the image, and it may be determined whether the direction of motion is the direction toward the center of the entrance or the reverse direction. . In the illustrated example, since the change from the first temporary storage data 702 to the second temporary storage data 704 changes to the opposite side to the entrance / exit direction, it is determined to move to the opposite side to the entrance / exit direction. .

In this way, it is determined whether or not the movement is in the direction of the doorway (S582), and if the movement is in the direction of the doorway, the determination result “open the door” or “maintain the open state” is shown. The signal DC is transmitted to the door control unit 107 (S583), and if the movement is in the opposite direction to the door, the signal DC indicating the determination result that “the door may be closed” is transmitted to the door control unit 107. (S584).
Further, the second temporary storage data is copied as new first temporary storage data, that is, written (overwritten) in the first temporary storage area 205b (S585), the process proceeds to step S572, and the above-described processing is performed. Repeat the same process.

As described above, the determination processing unit 203 determines in S559 in FIG. 10 and S579, S583, and S584 in FIG. 11 that “the door may be closed” or “the door is kept open”. Is transmitted to the door control unit 107.
In addition to the opening / closing control signal DC from the image processing unit 106 described above, the door control unit 107 is based on instructions from the elevator operation control unit 108 and information such as an optical sensor (not shown) installed in the door. Take control.

  Note that the door opening and closing based on the control described with reference to FIGS. 10 and 11 described above is performed only after the elevator stops on any floor and the door opens, so the control itself Alternatively, the opening and closing of the door based on the control described with reference to FIGS. 10 and 11 may be started when the elevator stops on any floor and the door opens.

Embodiment 2. FIG.
In the second embodiment (2) the process S532 of the flowchart of FIG. 9 in the preparation of the reference image, and (3) the process S552 of the flowchart of FIG. 10 and the processes S571 and S580 of FIG. 11 in the comparison process. The digital video obtained from the surveillance camera image is copied to the video memory 202. However, when copying, only the data in the inspection target area is stored in the video memory based on the information indicating the position and size of the inspection area. It may be configured to copy to 202.

Embodiment 3 FIG.
In the first embodiment (1) in the determination of the inspection area, the processes S503 and S505 in the flowchart of FIG. 5 copy the digital video obtained from the monitoring camera image to the video memory 202 and the reference memory 205, respectively. However, the digital video obtained from the surveillance camera image may be reduced to half or a quarter by copying data and copied. By doing so, the capacity of the video memory 202 can be reduced to half.
In this case, the capacity of the video memory 202 and the reference memory 205 is further reduced by combining (2) preparation of the reference image and (3) the comparison processing with the method of copying only the data in the inspection area. be able to.
In this case, in the process of obtaining the coordinates of the reference point (or start point) of the inspection area (S508 in FIG. 5), a process for converting the coordinate value of the area in accordance with the thinning rate is required. In other words, when the coordinate value of each pixel or each position on the screen is represented by the number of lines or the number of pixels before thinning with a predetermined point on the screen, for example, the point at the upper left corner as the origin, the number of lines by thinning Or a value representing the size of the reference area (or start point) of the inspection area or the inspection area obtained based on the data with the reduced number of pixels, and a magnification corresponding to the thinning rate (for example, by thinning, When the number of pixels is 1 / N, it is necessary to multiply the number by N).

  As mentioned above, although the case where this invention was applied to the detection of the passenger in the elevator car was demonstrated, this invention is applicable also to the detection of the passenger (waiting person) in the exterior of an elevator, for example, a landing. The present invention can also be applied to detection of passengers (passengers and passengers or users) of vehicles other than elevators. For example, the present invention can be applied to detection of a passenger who tries to pass through a train door. The present invention can also be applied to detection of a person (user) who is going to pass through a door other than a vehicle door, for example, a doorway of a building. “User” in the claims includes passengers of vehicles, and from another angle, includes all persons who are going through the door, persons who pass near the door, and the like.

Claims (9)

An extraction step of extracting the edge of the doorway by comparing the image data in the door-closed state obtained from a camera that shoots to include the doorway with the door, and the image data in the door-open state,
An inspection area determining step for calculating the coordinates of the center point of the edge and the length of the edge side from the extracted information on the edge of the entrance and exit, and determining one or more inspection areas centered on the center point ; ,
A user detection step of detecting a user around the door based on a change in the image data of the inspection region determined in the inspection region determination step ,
The inspection area is determined based on the calculated side length . The use according to claim 1, wherein, in the inspection area determination step , the inspection area is determined as an area surrounded by a square having a side length of half of the calculated side length. Person detection method. The user detecting step includes
In a state where it is estimated that there is no user around the door, a portion in the inspection area of the reference image obtained from the camera,
In a state where there is a possibility that there is a user around the door, by comparing the portion in the inspection area of the image obtained from the camera,
The user detection method according to claim 1, wherein user detection is performed. A direction information adding step of adding information indicating the direction of the entrance to the inspection area;
The method further comprises the step of detecting the direction of movement of the user using the first image data in the inspection area, the second image data in the inspection area, and the orientation information. The user detection method according to claim 1.   5. The user detection method according to claim 1, wherein image data of only a portion within the inspection area among images obtained by the camera is written in a video memory.   The user detection method according to claim 1, wherein the user is a passenger in an elevator car, and the door is an elevator door.   The user detection method according to claim 1, wherein the camera is installed in an elevator car.   The control method of performing the door opening / closing control based on the detection result of the user's movement direction by the user detection method according to claim 4. Extraction means for extracting the edge of the doorway by comparing the image data in the door-closed state and the image data in the door-open state obtained from a camera that shoots to include the doorway with the door,
An inspection area determination means for calculating the coordinates of the center point of the edge and the length of the edge side from the extracted information on the edge of the doorway, and determining one or more inspection areas centered on the center point ; ,
User detection means for detecting a user around the door based on a change in the image data of the inspection area determined by the inspection area determination means ,
The user detection apparatus according to claim 1, wherein the inspection area is determined based on the calculated side length .

Images