JP4825350B2 - Passing object counting device and counting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device and a method for counting passing bodies which can perform fast processing and sequential processing by decreasing a computation quantity. SOLUTION: The passing body counting device is equipped with image pickup devices 11 and 12 which pickup images of a passing body 5 passing a reference line 1 on a plane 2 along the plane 2 and are provided opposite the plane 2, a comparing operation part 20 which compares the images of the passing body 5 picked up by the image pickup devices 11 and 12 with each other to obtain in time series depth information of the depth of the passing body 5 in a depth direction crossing the plane 2 at multiple positions on the passing body 5 along the reference line 1 and speed information of a traveling direction perpendicular to the reference line 1 and parallel to the plane 2 at at least one of the positions, and an integral operation 21 which temporally integrates the value obtained by adding a value weighted with the speed information to the obtained depth information by directions of the speed along the reference value 1.

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a passing object counting device and a counting method, and more particularly to a passing object counting device and a counting method for counting passing objects from a stereo image.
[0002]
[Prior art]
From the stereo image of the passing object obtained by the two imaging devices, height information and velocity information on the counting line are calculated, and three-dimensional information is obtained. Next, the obtained three-dimensional information is cut into circles at a certain height, and a planar spatio-temporal image is created when passing objects are arranged in the time direction for a certain time. Then, using the created planar spatio-temporal image, passing objects are counted depending on whether or not the object has an area of a certain width or more and a time at a certain height.
[0003]
[Problems to be solved by the invention]
The conventional passing object counting apparatus as described above has the following problems. In a method that counts whether or not an object has a certain width and time area at a certain height in a planar spatiotemporal image for a certain time, it takes time to calculate and sequential processing cannot be performed. It was.
[0004]
Accordingly, an object of the present invention is to provide a passing object counting device and a counting method capable of reducing the amount of calculation and performing high-speed processing and sequentially processing.
[0005]
[Means for Solving the Problems]
  To achieve the above objective,Of the first aspectFor example, as shown in FIG. 1, the passing object counting device counts the passing object 5 from the depth information and the velocity information of the passing object 5 passing through the reference line 1 on the plane 2 along the plane 2. 10; an integration operation unit 21 (FIG. 2) for time-integrating a value obtained by adding a value weighted by the speed information to the acquired depth information along the reference line 1 for each speed direction; and obtained by the time integration. Passing count calculation units 22 and 23 (FIG. 2) that count the passing object 5 that has passed the reference line 1 based on the set value and a preset threshold value.
[0006]
If comprised in this way, since the integration calculating part 21 and the passage count calculating parts 22 and 23 are provided, the value which added the value weighted by speed information to the acquired depth information according to the direction of speed along the reference line 1 The time-integrated value can be obtained, and the passage counting calculation units 22 and 23 are provided, so that the passing object 5 that has passed through the reference line 1 based on the value obtained by the time integration and a preset threshold value can be obtained. Can be counted. The direction of speed includes a positive direction and a negative direction for the same speed direction.
[0007]
  To achieve the above purpose,Of the second aspectFor example, as shown in FIG. 1, the passing object counting device captures a passing object 5 passing through a reference line 1 on the plane 2 along the plane 2, and a plurality of imaging devices 11 provided facing the plane 2. , 12; the images of the passing object 5 captured by the plurality of imaging devices 11, 12 are compared with each other, and the plane 2 of the passing object 5 intersects with respect to a plurality of parts along the reference line 1 on the passing object 5. A comparison operation unit 20 that acquires, in time series, velocity information in a traveling direction perpendicular to the reference line 1 and parallel to the plane 2 with respect to depth information in the depth direction and at least one of the plurality of parts. And an integration calculation unit 21 (FIG. 2) for time-integrating a value obtained by adding a value weighted by the speed information to the acquired depth information along the reference line 1 for each speed direction. For the same traveling direction, the traveling direction has a positive direction and a negative direction.
[0008]
If comprised in this way, since it comprises the some imaging devices 11 and 12 and the comparison calculating part 20, depth information and speed information can be acquired in time series, and since the integral calculating part 21 is provided, it is acquired. A value obtained by time-integrating a value obtained by adding a value obtained by weighting the depth information to the speed information along the reference line 1 for each speed direction can be acquired. It is the volume of the passing object 5 that is calculated by the integration calculation unit 21 after all.
[0009]
  To achieve the above purpose,Of the third aspectAs shown in FIG. 1, for example, the passing object counting device captures a passing object 5 that passes through a reference line 1 on the plane 2 along the plane 2, and is a first imaging device provided facing the plane 2. 11; the passing object 5 that is provided in parallel to the first imaging device 11 and opposed to the plane 2 and that passes through the reference line 1 is delayed by a predetermined time from the imaging time of the first imaging device 11. A second imaging device 12 for imaging; an image A of the passing object 5 imaged by the first imaging device 11 and an image of the passing object imaged by the second imaging device 12 delayed by the predetermined time Compared with B, with respect to a plurality of parts along the reference line 1 on the passing object 5, the depth information in the depth direction intersecting the plane 2 of the passing object 5 and at least one part of the plurality of parts, Progression perpendicular to reference line 1 and parallel to plane 2 A comparison operation unit 20 (FIG. 2) that acquires speed information in a direction in time series; a value obtained by adding a value weighted by the speed information to the acquired depth information along the reference line 1 for each speed direction Is integrated with time integration unit 21 (FIG. 2).
[0010]
For example, as shown in FIG. 12, the surface on which the passing object 5 moves is not limited to a horizontal plane, but may be a surface 2 'inclined like a slope or a staircase. However, if necessary, for example, as shown in FIG. 12A, the depth information and the speed information are corrected by the inclination angle.
[0011]
The reference line 1 that intersects the traveling direction of the passing object 5 is preferably a line perpendicular to the traveling direction of the passing object 5. The first and second imaging devices are preferably arranged in parallel in a direction parallel to the reference line 1. The depth direction is typically a direction perpendicular to a plane or a vertical direction.
[0012]
With this configuration, the image A of the passing object 5 captured by the first imaging device 11 is compared with the image B of the passing object 5 captured by the second imaging device 12 after being delayed by a predetermined time. The depth information in the depth direction intersecting the plane 2 of the passing object 5 with respect to a plurality of parts along the reference line 1 on the passing object 5 and at least one of the plurality of parts is perpendicular to the reference line 1. A comparison calculation unit 20 that acquires speed information in a traveling direction parallel to the plane 2 in time series, and a value obtained by weighting the acquired depth information with the speed information is added along the reference line 1 for each speed direction. Since the integration calculation unit 21 for time integration of the obtained values is provided, the position information and speed information of the passing object 5 can be obtained from the images A and B.
[0013]
  AlsoThe fourth aspect is,Second aspectOrThird aspectPassing object counting device 10InIt is preferable to provide a difference passage count calculation unit 22 that counts that the passing object 5 has passed by 1 when a value obtained by time integration by the integration calculation unit 21 exceeds a preset threshold value.
[0014]
If comprised in this way, the count of the passing object 5 will be grasped | ascertained with the time with respect to the passing number (natural number) of a passing object. Or grasp by the number of passing in a certain time.
[0015]
  AlsoThe fifth aspect is,Second aspectOrThird aspectPassing object counting device 10InIt is preferable to provide a division passage count calculation unit 23 that counts a value obtained by dividing the value obtained by time integration by the integration calculation unit 21 by a preset threshold as the number of passing objects 5 passing through.
[0016]
If comprised in this way, the count of the passing object 5 will divide | segment at a time, and will calculate | require the number of passing objects which passed in the meantime to a decimal point.
[0017]
The preset threshold value is typically a volume value for one passing object. The volume value may be determined appropriately depending on conditions such as the passing object, location, time, season, and the like. For example, if the passing object is a person and the place is an amusement park, the child is the center, and the volume value is small although it is determined in consideration of the average ratio of children to adults. If it is a rush hour station, adult men are the center. In winter, an overcoat is worn. This should be obtained statistically for each case.
[0018]
  To achieve the above purpose,Of the sixth aspectIn the passing object counting method, for example, as shown in FIG. 4, an imaging step S <b> 1 for capturing a plurality of images by imaging a passing object passing through a reference line on the plane along a plane from a direction facing the plane. And comparing the plurality of images acquired in the imaging step S1, and regarding the plurality of portions along the reference line on the passing object, the depth information in the depth direction intersecting the plane of the passing object, and the Comparison operation steps S9 and S12 for acquiring, in time series, velocity information in a traveling direction perpendicular to the reference line and parallel to the plane with respect to at least one of the plurality of portions; An integration operation step S13 for time-integrating a value obtained by adding the value weighted by the speed information along the reference line for each speed direction.
[0019]
  To achieve the above purpose,Of the seventh aspectIn the passing object counting method, for example, as shown in FIG. 5, a passing object passing through a reference line on a plane along a plane is imaged from a first position facing the plane at a first time point. A first imaging step S32 for acquiring an image A; the passing object passing through the reference line at a second time point after a predetermined time from the first time point, in parallel with the first position; And a second imaging step S33 in which an image B is obtained by imaging from a second position facing the plane; the image A and the image B are compared, and the reference line on the passing object is compared with the reference line. The depth information in the depth direction intersecting the plane of the passing object and the velocity information in the traveling direction perpendicular to the reference line and parallel to the plane with respect to the plurality of portions along the depth direction intersecting the plane of the passing object Are calculated in a time series. 8, S39 and; and an integral calculation step (S40) integrating the value of the weighted values obtained by adding to the speed of the orientation by along the reference line by the velocity information on the acquired depth information time.
[0020]
When configured in this manner, the image A and the image B are compared, the information on the depth direction intersecting the plane of the passing object and the reference line (typically perpendicularly intersecting) and the traveling direction parallel to the plane Since the comparison calculation steps S38 and S39 for calculating the speed of the moving object 5 are provided, position information and speed information of the passing object 5 can be obtained.
[0021]
  AlsoThe eighth aspect,Sixth aspectOrSeventh aspectPassing object counting methodInA step S0 in which a volume of one average passing object is set as a threshold value in advance; and when the value obtained by time integration by the integration calculation unit exceeds the threshold value, the passing object is 1 It is preferable to provide a passage counting calculation step S15 that counts as only passing.
[0022]
  AlsoThe ninth aspect,Sixth aspectOrSeventh aspectPassing object counting methodInA step S0 in which an average volume of one passing object is set in advance as a threshold; and the passing object has passed a value obtained by dividing the value obtained by time integration by the integration calculation unit by the threshold. A division passage count calculation step S20 for counting as a number may be provided.
[0023]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol or a similar code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted.
[0024]
FIG. 1 is a schematic perspective view of a passing object counting device 10 (hereinafter referred to as “counting device 10” as appropriate) according to an embodiment of the present invention. In the figure, the passing object 5 moves in the traveling direction on a horizontal passage 2 which is a plane. An orthogonal coordinate system XYZ is taken so that the XY axis is placed in the passage 2, and the passage 2 extends in parallel with the Y axis. Further, a counting line 1 which is a reference line is taken perpendicular to the extending direction of the passage 2, that is, parallel to the X axis. The traveling direction of the passing object 5 is typically parallel to the extending direction of the passage 2, that is, the Y axis. The passing object 5 is a person in the present embodiment. In the present embodiment, the counting line 1 is perpendicular to the extending direction of the passage 2, but it does not have to be particularly perpendicular. If it is not vertical, the parallax and the speed may be corrected by the angle with the X axis. The counting line 1 is a reference line for counting the passing objects 5 that have passed through this line, and may be a virtual line, and does not need to be physically drawn in the path 2.
[0025]
On the other hand, in order to image the passing object 5 passing through the counting line 1 of the passage 2 above the counting line 1 of the passage 2 in the figure in a vertical direction above the counting line 1 of the passage 2 at a height sufficient to pass the person 5 as a passing object. The first imaging device 11 and the second imaging device 12 are installed. The first imaging device 11 and the second imaging device 12 are installed in parallel at an appropriate interval so that they are parallel to the counting line 1 and the optical axis is parallel to the Z axis. The height sufficient for the person 5 to pass here is, for example, about 3000 mm. Further, the appropriate interval is, for example, about 50 mm.
[0026]
An example of the configuration of the passing object counting device 10 will be described with reference to FIG. The counting device 10 includes a first CCD camera 11 as a first imaging device, a second CCD camera 12 as a second imaging device, and a personal computer 13 as an arithmetic device.
[0027]
The first CCD camera 11 and the second CCD camera 12 are connected to a personal computer 13 so that the personal computer 13 can acquire images captured by the first CCD camera 11 and the second CCD camera 12, respectively. It is configured.
[0028]
The personal computer 13 includes a control unit 14 and controls the counting device 10 as a whole. An imaging device interface 15 is connected to the control unit 14, and the first CCD camera 11 and the second CCD camera 12 are connected to the control unit 14 via the imaging device interface 15 and controlled. Yes.
[0029]
Further, in the control unit 14, from the stereo images acquired by the first CCD camera 11 and the second CCD camera 12, the height of the passing object 5 as depth information of the passing object 5 and the speed information of the passing object 5 are obtained. A comparison operation unit 20 that acquires the speed of a certain passing object 5 in time series, and time-integrates a value obtained by adding the value obtained by weighting the acquired height of the passing object 5 by speed along the reference line 1 according to the speed direction. Integral calculation unit 21, a difference passage count that counts that the passing object 5 has passed by 1 when the volume value obtained by time integration by the integral calculation unit 21 exceeds a preset threshold value The calculating part 22 and the division | segmentation passage count calculating part 23 which counts the value which divided the volume value with the preset threshold value as the number which the passing object passed are provided. The comparison calculation unit 20 may weight the value obtained by adding the height along the reference line 1 according to the direction of the speed by the speed.
[0030]
In the present embodiment, the difference passage count calculation unit 22 and the division passage count calculation unit 23 are both provided, but either one may be provided. At this time, when the difference passage count calculation unit 22 is provided, for example, a time when the passing object 5 has passed by an arbitrary natural number can be obtained. When the division passage count calculation unit 23 is provided, for example, at a certain time Since the passing number of passing objects 5 in an arbitrary time interval divided by (2) can be sequentially obtained as a decimal number (for example, 10.6), a difference passage count calculation unit 22 and a division passage count calculation unit according to the purpose and application. One of 23 may be selected.
[0031]
A storage unit 18 is connected to the control unit 14 and can store data such as images captured by the first CCD camera 11 and the second CCD camera 12 and calculated information.
[0032]
Further, an input device 16 for inputting information for operating the counting device 10 and an output device 17 for outputting a result processed by the counting device 10 are connected to the control unit 14. The input device 16 is, for example, a keyboard or a mouse, and the output device 17 is, for example, a display or a printer. Although the input device 16 and the output device 17 are illustrated as externally attached to the personal computer 13 in this figure, they may be built in.
[0033]
Further, as shown in FIG. 3, an image processing unit 25 that processes an image captured by the CCD camera, an image captured by the first CCD camera, and an image captured by the second CCD camera are included in the comparison operation unit 20. Corresponding point search unit 26 that obtains corresponding points from the captured image, depth information calculation unit 27 that calculates the height that is the depth information of the passing object 5, and speed information calculation unit 28 that calculates the speed that is the speed information of the passing object 5. Is provided. To process the image, for example, the first image as the image A, the second image, and the third image as the image B are acquired from the imaging signals from the first CCD camera 11 and the second CCD camera 12. That means.
[0034]
With reference to the flow of FIG. 4, an example of the effect | action of the counting apparatus 10 as 1st Embodiment is demonstrated. Here, an image captured by the first CCD camera 11 is described as a left image, and an image captured by the second CCD camera is described as a right image. First, the user of the counting device 10 sets a threshold value in the personal computer 13 (step 0 (displayed as “S0” in the figure).
same as below)).
[0035]
Next, the control unit 14 of the personal computer 13 acquires the left image and the right image captured by the first CCD camera 11 and the second CCD camera 12 as a stereo image via the imaging device interface 15 (step 1). .
[0036]
The control unit 14 acquires the left image of the t frame, which is the first image captured at a certain time t, by the image processing unit 19 in the control unit 14 from the acquired stereo image (step 2). The right image of t frame, which is the second image, is acquired (step 3). Further, a right image of the t + Δt frame, which is a third image captured at time t + Δt after Δt has elapsed from time t, is acquired (step 3).
[0037]
In practice, images are taken simultaneously at a fixed interval, and the image at time t is used as the first image and the second image, and the image at time t + Δt shifted by one frame or more is used as the third image. That's fine. At this time, Δt may be appropriately determined according to the setting conditions of the counting device 10, but it is desirable that Δt be the time from when a certain frame N is imaged until the next frame N + 1 is imaged. That is, a frame shifted by one frame is employed. At this time, when Δt is set to 1/30 seconds, for example, when a person walking at 80 m / min is imaged, the frame N + 1 is an image in which the imaged person moves about 4.4 cm in comparison with the frame N. .
[0038]
Next, edge extraction of the right image of t frame is performed (step 5), and it thins (step 6). Then, the first corresponding point search of the left image of the t frame is performed using the edge points on the counting line 1 of the right image of the t frame (step 7). Further, edge extraction, thinning, and corresponding point search will be described in detail later with reference to FIG.
[0039]
Depth information is calculated from the corresponding points obtained by the first corresponding point search (step 8). Here, a non-background portion is defined in advance by background removal, and linear interpolation is performed on the portion to reflect the value at the edge point (step 9). Here, the linear interpolation is, for example, using the three-dimensional coordinates of one edge point and the edge points adjacent to the edge point to connect the two edge points with a straight line, thereby obtaining the three-dimensional coordinates of the point between the two edge points. It is what you want to do.
[0040]
At the same time, the second corresponding point search of the right image of the t + Δt frame is performed using the edge points on the counting line 1 of the right image of the t frame (step 10), and the corresponding points obtained by the second corresponding point search are performed. Speed information is calculated from (step 11). Similarly, linear interpolation is performed (step 12). A method for calculating the depth information and the speed information will be described later with reference to FIGS.
[0041]
Then, the calculated depth information multiplied by the speed information is added in the time direction, and the volume value is added (step 13). A method for calculating the volume value will be described later with reference to FIGS. The reason why the depth information is weighted by multiplying the depth information is that a passing object with a high passing speed is inversely proportional to the passing speed and the number of passing frames. Further, since the volume of the passing object is calculated, the shadow of the passing object having no volume is not erroneously counted.
[0042]
The volume value obtained in this way is compared with a preset threshold value (step 15). If the volume value is larger than the threshold value (step 15 is Y), one person who is the passing object 5 has passed. Judging and counting (step 16), the threshold value is subtracted from the volume value (step S18). At this time, the velocity information calculated above is reflected as the direction of travel of the passing object 5 that has passed (step 17). If the volume value is smaller than the threshold value (step 15 is N), the next frame is acquired and the volume value is added. The preset threshold is typically the minimum volume value of the passing object.
[0043]
In the above, the case where the difference passage count calculation unit 22 is used for counting has been described. However, when the division passage count calculation unit 23 is used (indicated by a broken line in the figure), the speed information is first added to the similarly calculated depth information. The product multiplied by the information is added in the time direction and the volume value is added (step 13). And it is judged whether it counts with the volume value obtained by this (step 19). This determination may be made by, for example, setting the number of frames of the acquired image in advance and automatically counting when the set number of frames is reached, or determining by sequentially counting by the user. You may make it do.
[0044]
When counting (step 19 is Y), the acquired volume value is divided by a preset threshold value (step 20), and the value obtained thereby is counted as the number of passing persons of the passing object 5 (step 20). 21). At this time, the velocity information calculated above is reflected as the direction of travel of the passing object 5 that has passed (step 17). When not counting (N in step 19), the next frame is acquired and the volume value is added.
[0045]
With reference to the flow of FIG. 5, an example of the operation of the counting device 10 as the second embodiment will be described. As in the first embodiment, first, the user of the counting device 10 sets a threshold value in the personal computer 13 (step 30) and acquires a stereo image (step 31).
[0046]
The control unit 14 acquires, from the acquired stereo image, the left image of the t frame that is the first image as the image A captured at a certain time t by the image processing unit 19 in the control unit 14 (step 32). ). Further, a right image of a t + Δt frame, which is a third image as an image B taken at time t + Δt after Δt has elapsed from time t, is acquired (step 33).
[0047]
Actually, the first CCD camera and the second CCD camera pick up images at each time point simultaneously and in parallel, and the time point t as the first image is shifted by one frame or more as the third image. You may make it employ | adopt the image of t + (DELTA) t. At this time, Δt may be appropriately determined according to the setting conditions of the counting device 10, but it is desirable that Δt be the time from when a certain frame N is imaged until the next frame N + 1 is imaged.
[0048]
Next, edge extraction of the left image of t frame is performed (step 34), and it thins (step 35). Then, the third corresponding point search of the right image of the t + Δt frame is performed using the edge points on the counting line 1 of the left image of the t frame (step 36).
[0049]
Next, depth information is calculated from the corresponding points obtained by the third corresponding point search (step 37). Here, a non-background portion is defined in advance by background removal, and linear interpolation is performed on the portion to reflect the value at the edge point (step 38). At the same time, speed information is calculated (step 39), and linear interpolation is also performed (step 40). A method for calculating the depth information and the speed information will be described later with reference to FIGS. Then, the calculated depth information multiplied by speed information is added in the time direction and the volume value is added (step 41). Hereinafter, counting is performed in the same manner as in the first embodiment.
[0050]
The third corresponding point search will be described in more detail with reference to FIG. Here, although a person is expressed by hatching as a passing object in each illustrated frame, this does not indicate a cross section but merely represents a person imaged from above.
[0051]
Here, in order to obtain depth information and speed information, corresponding points of the two images are obtained. First, the correspondence between the left image of t frame and the right image of t + Δt frame is obtained. At this time, the correspondence relationship is not clearly determined for pixels having no feature in the neighboring region.
Therefore, the search is performed according to the following flow.
[0052]
First, edge extraction of the left image of t frame is performed to obtain a thinned image, and corresponding point search is performed only at the edge point. Edge extraction refers to extracting a contour, which is a set of edge points of a target passing object, from an image acquired with shading and specifying the coordinates, for example, with coordinates. Thinning refers to, for example, converting an image obtained by edge extraction into an image in which only the outline of a passing object obtained by edge extraction is indicated by a thin line.
[0053]
In order to perform the corresponding point search, first, a point on the specified outline of the left image of the t frame is selected as an edge point, and a window is set in a neighborhood region including the edge point on the grayscale image. Typically, an edge point on the counting line is selected. An edge point is typically an intersection of the fine line and the count line, and usually there are a plurality of edge points, typically 10 to 20 points. For example, the leftmost point among the plurality of edge points is first selected. The window may be set so that the selected edge point is the center. Then, the window is set to the right image of the t + Δt frame with reference to the set window.
[0054]
With respect to these windows, the correlation between the gray values of the left image of t frame and the right image of t + Δt frame is obtained using Equation 1 shown in FIG. Where I_L(I, j) and I_R(I, j) are the gray values in the windows on the left and right images, and μ_{L ,} μ_{R ,} σ_{L ,} σ_RAre the mean and variance of the gray values in the left and right windows, respectively. C takes a value within a range from −1 to +1. The C value is calculated in each window while moving the window of the right image little by little with respect to the selected edge point of the left image, for example, parallel to the count line and moving in the direction orthogonal to the count line. The window of the right image that gives the maximum value among C obtained in this way is regarded as a local region corresponding to the selected window of the left image.
[0055]
That is, the left and right image windows are correlated as described above, and the right image window corresponding to the window including the edge point selected in the left image (C is maximized) is searched. The center of the window of the right image obtained by this search becomes the edge point of the right image corresponding to the edge point of the left image selected previously.
[0056]
The search for the window is performed, for example, in a direction parallel to the count line (X-axis direction) and a vertical direction (Y-axis direction). For example, when the passing object is a person, the direction parallel to the counting line is regarded as a parallax between the left and right images as will be described later, so the height is calculated to be 1 m to 2 m, for example, with reference to the window of the left image that has already been set. What is necessary is just to search the width | variety of the parallax to be performed. Since the vertical direction is regarded as the movement amount of the person as will be described later, the width of the movement amount that the person is expected to move during the time Δt may be searched with reference to the window of the left image that has already been set. If the value of C is increasing in the direction of search, the value of C is further maximized, and the search is continued in the same search direction until the downward trend is reached. You may make it select the window where the value of C becomes the maximum in the already searched range.
[0057]
If the window search is performed in this way, it is not necessary to correlate the windows of the left and right images of the part that clearly does not need to be searched, so that the amount of calculation can be further reduced. As described above, the corresponding points are searched for all the edge points existing in the left image, and the corresponding edge points of the left and right images are obtained.
[0058]
Although the third corresponding point search of the second embodiment has been described above, the same corresponding point search is performed for the first corresponding point search and the second corresponding point search of the first embodiment. .
[0059]
The calculation of depth information and speed information will be described in more detail with reference to FIG. First, a vector connecting corresponding edge points of the left and right images obtained above is considered, and this is separated into a horizontal component (X-axis direction component) and a vertical component (Y-axis direction component). The horizontal component can be viewed as an amount representing the parallax between the left and right images, and the distance to the target passing object can be calculated using this parallax. On the other hand, the vertical component can be regarded as the amount of movement of the attention point in the direction along the path, and based on this, the speed information of the passing object can be obtained. In this way, the three-dimensional height that is the depth information at the edge point and the speed that is the speed information can be collectively obtained in pairs.
[0060]
Since the speeds obtained for one passing object should normally be equal, it is not necessary to obtain the speeds for all the edge points. For example, the speed of the point closest to the center may be adopted. Alternatively, the speed of each edge point is obtained, and an edge point having a protruding value may be re-searched as a search error, or the point may be rejected. Or you may obtain | require all the speed | velocity | rate of each edge point, calculate those average values, and may employ | adopt as the speed | velocity | rate of the object.
[0061]
Further, for the weighting based on the speed described later, the speed for each point may be used, or the speed of the passing object obtained by the average value may be used in common for each point.
[0062]
Here, the left and right images are images that are shifted in time by Δt. However, if Δt is a sufficiently small value, the left and right images do not have a difference enough to support parallax. Conversely, Δt may be determined as such. That is, the length is sufficient to obtain the speed information without any problem for obtaining the parallax. For example, when the target passing object is a pedestrian, it is set to about 1/60 to 1/10 seconds, particularly 1/40 to 1/20 seconds, and more preferably about 1/30 seconds.
[0063]
As described above, according to the second embodiment, the height and speed are obtained using three images in the first embodiment, but only processing of two images is required. The amount of calculation can be greatly reduced, and the processing speed can be increased.
[0064]
Here, with reference to FIG. 8, the calculation method of the height z from the plane as the depth information of the target passing object will be described. The distance a to the target passing object 5 can be calculated using Expression 2 shown in FIG. Here, w is the distance between the CCD cameras (baseline length), f is the focal length of the CCD camera when the imaging lens of the CCD camera is a single lens, and d is the parallax on the imaging surface of the CCD camera. The focal length here is the focal length of the combination lens when a commonly used combination lens is used. Further, when the distance from the imaging lens of the CCD camera to the counting line 1 is h, the height z of the target passing object 5 is calculated by Expression 3 shown in FIG.
[0065]
With reference to FIG. 9 and FIG. 10, a method of counting using the height and speed of the passing object calculated in each frame by the above-described method will be described. Assume that the height of an object at a certain position x in a certain t frame is z (x, t), and the velocity at the same position and the same frame is v (x, t). According to the corresponding point search described above, z (x, t) corresponds to a value of 0 to 255 corresponding to a height of about 0 to 185 cm, and v (x, t) corresponds to a speed of about −12 to 12 km / h. A value of -5 to 5 is obtained (FIG. 9A).
[0066]
Next, the product zv of the frames is added in the X-axis direction. This sum, that is, Equation 4 shown in FIG. 11, means a value obtained by multiplying the depth information of a certain frame by the weight of the speed (the volume that passes through the unit time at a certain point in time). Intuitively, it can be viewed as a cross-sectional area in a person's space-time when passing through the counting line, so this value will be referred to herein as an “area value” (FIG. 9B). Since it is necessary to measure separately according to the direction of the speed later, this addition work is also performed in accordance with the sign of v (x, t), that is, the variable to be added as shown in Equation 5 shown in FIG. I'll keep it separate.
[0067]
Next, the area values calculated for each line are added in the time direction (FIG. 10- (a)). This sum, that is, Equation 6 shown in FIG. 11, represents the volume of the person who passed. This is referred to herein as “volume value” (FIG. 10- (b)). As long as the frame interval is kept constant, the passing speed and the number of passing frames are inversely proportional. Therefore, the volume value calculated from the passing of each person is normalized to some extent by this addition in consideration of the weight of the speed. That is, the true volume value in the three-dimensional space is obtained.
[0068]
A case where the passing object 5 moves on the slope will be described with reference to FIG. For example, when the passing object 5 is moving on an inclined surface 2 ′ such as a staircase or a slope, the plane 2 is a virtual plane 2 perpendicular to the line connecting the reference line 1 and the first and second imaging devices 11 and 12. It is good to do.
[0069]
If the target passing object 5 is a person, it stands in the vertical direction even if it is a slope, so the optical axis of the CCD camera is directed in the vertical direction (FIG. 12A). In this case, the virtual plane 2 is a horizontal plane. In this example, for example, when the angle formed between the inclined surface 2 ′ and the virtual plane 2 is θ, the time of the passing object 5 moving on the inclined surface 2 ′ is obtained in order to obtain the height of the passing object 5 from the virtual plane 2. A value obtained by multiplying the amount of movement of Δt by tan θ may be subtracted from the calculated height of the passing object 5 moving on the slope 2 ′.
[0070]
If the target passing object 5 is, for example, a car, the slope may be used as it is as the plane 2 of the present invention. In this case, the optical axis of the CCD camera is directed in a direction perpendicular to the inclined surface (FIG. 12B).
[0071]
Next, the results of a counting experiment actually performed using the embodiment of the present invention will be described. The experiment was conducted in a temporary passage in a university building under the following conditions in the daytime in fine weather.
Experiment 1: Passing about 40 people per minute
Experiment 2: Passing about 80 people per minute
Experiment 3: Passing about 120 people per minute
The setting conditions of the image pickup apparatus are a CCD camera as the image pickup apparatus, a camera height of 295 cm, a passage width of 200 cm, a camera base line length (inter-camera distance) of 47 mm, and a focal length of 4.8 mm. Further, in actual counting, the number of counting persons is calculated at the time when the processing of 30 lines is completed using the above-described division passage counting calculation unit 23. A value obtained by dividing the total volume value for 30 lines by the volume value for one person determined in advance by another input image is defined as the number of passing people. Here, a line is a frame processing unit.
[0072]
FIG. 13 shows the results of Experiment 1, FIG. 14 shows the results of Experiment 2, and FIG. 15 shows the results of Experiment 3 as the count results every 20 seconds in 600 frames. The positive and negative signs given to the passing number of people shown in each figure indicate that the person has progressed in opposite directions. The counting results in each figure are displayed as natural numbers. However, since the division passage counting calculation unit 23 is used, it is possible to display up to the decimal point.
[0073]
The accuracy of each experiment was 87% in Experiment 1, 89% in Experiment 2, and 90% in Experiment 3. However, this accuracy is an average value calculated every 20 seconds. When the accuracy was calculated using data obtained by continuously counting for a long time, each experiment showed a high accuracy exceeding 95%.
[0074]
FIG. 16 shows a comparison between a counting result obtained by manual counting performed by a person with a counter and a counting result counted using the present invention. In manual counting, when the crowding is low, the counting can be performed accurately, but the accuracy decreases extremely as the crowding becomes intense.
[0075]
On the other hand, in the counting using the present invention, it can be seen that a stable accuracy is maintained regardless of the congestion.
[0076]
As described above, according to the embodiment of the present invention, the height and speed of the passing object can be acquired from the stereo images acquired from the two CCD cameras, and the volume of the passing object is determined from the acquired height and speed. Since it can be obtained separately, it is possible to sequentially count passing objects more accurately than in the past.
[0077]
【The invention's effect】
As described above, according to the present invention, the depth of the crossing of the plane of the passing object is compared with respect to the plurality of parts along the reference line on the passing object by comparing the images of the passing object captured by the plurality of imaging devices. A comparison operation unit for acquiring speed information in a traveling direction perpendicular to a reference line and parallel to a plane in time series with respect to direction depth information and at least one of the plurality of portions, and the acquired depth information And an integration calculation unit that integrates a value obtained by adding the value weighted by the speed information along the reference line according to the direction of the speed with respect to time, so that the depth information and the speed information of the passing object captured by the plurality of imaging devices are obtained. It is possible to acquire a value obtained by integrating the value obtained by weighting the acquired depth information with speed information along the reference line according to the speed direction, time integration, reducing the amount of calculation, enabling high-speed processing, and enabling sequential processing It is possible to provide an over-object counting device and counting method.
[Brief description of the drawings]
FIG. 1 is a schematic perspective block diagram showing an outline of a passing object counting apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration example of a passing object counting apparatus according to an embodiment of the present invention.
FIG. 3 is a block diagram showing a configuration example of a comparison operation unit used in the embodiment of the present invention.
FIG. 4 is a flowchart showing an example of the operation of the passing object counting apparatus according to the first embodiment of the present invention or the passing object counting method according to the embodiment of the present invention.
FIG. 5 is a flowchart showing an example of the operation of a passing object counting device according to a second embodiment of the present invention or a passing object counting method according to an embodiment of the present invention.
FIG. 6 is an image diagram for explaining a processing example when searching for a third corresponding point between the first image and the third image in the embodiment of the present invention.
FIG. 7 is an image diagram for explaining vectors connecting corresponding points of the first image and the third image in the case of FIG. 6;
FIG. 8 is a stylized front view illustrating a method for calculating the height of a passing object from the parallax between the first imaging device and the second imaging device in the embodiment of the present invention.
FIG. 9 is a stylized perspective view illustrating a method of performing counting in the embodiment of the present invention.
FIG. 10 is a stylized perspective view for explaining a counting method in the case of FIG. 9;
FIG. 11 is a diagram showing a calculation formula used in the description of the embodiment of the present invention.
FIGS. 12A and 12B are diagrams illustrating a case of counting by a virtual plane (a) and a case of using a slope as a plane of the present invention (b) as an example of a method of counting passing objects moving on an inclined surface.
FIG. 13 is a diagram showing a result of a counting experiment of Experiment 1 according to the embodiment of the present invention.
FIG. 14 is a diagram showing a result of a counting experiment of Experiment 2 according to the embodiment of the present invention.
FIG. 15 is a diagram illustrating a result of a counting experiment of Experiment 3 according to the embodiment of the present invention.
FIG. 16 is a diagram showing a comparison between a counting result of manual counting and a counting result of counting using the passing object counting device according to the embodiment of the present invention.
[Explanation of symbols]
1 Count line
2 passage
5 passing objects
10 Passing object counting device
11 First CCD camera
12 Second CCD camera
13 Arithmetic unit
14 Control unit
15 Imaging device interface
16 input devices
17 Output device
18 Storage unit
20 Comparison operation part
21 Integral calculation unit
22 Subtraction passage count calculator
23 Division Passing Count Calculation Unit

Claims (9)

A passing object counting device for counting passing objects from the height and velocity from the plane of passing objects passing through a reference line on the plane along a plane;
An image A of the passing object captured by an imaging device that captures the passing object and is provided opposite to the plane and acquires a stereo image, and is delayed by a predetermined time from the time when the image A was captured. An image processing unit for acquiring an image B of the passing object imaged by the imaging device;
A corresponding point search unit for obtaining corresponding points from the image A and the image B;
A speed information calculation unit that calculates the speed of the passing object using the corresponding points;
A depth information calculation unit that calculates the height of the passing object from the stereo image of the image A or the stereo images of the image A and the image B;
An integration operation unit for time-integrating a value obtained by multiplying the acquired height by the weight of the speed for each direction of speed along the reference line;
A passage counting calculation unit that counts the passing object that has passed through the reference line based on a value obtained by the time integration and a preset threshold;
Passing object counting device. A first imaging device provided to face the plane, which images a passing object passing through a reference line on the plane along the plane;
A second imaging device provided in parallel with the first imaging device and facing the plane, and imaging the passing object passing through the reference line;
A first image of the passing object imaged by the first imaging device at a first time point, a second image of the passing object imaged by the second imaging device, and the first time point An image processing unit that obtains a third image of the passing object imaged by the second imaging device with a delay of a predetermined time;
A corresponding point search unit for obtaining corresponding points from the first image and the second image and obtaining corresponding points from the second image and the third image;
Using the corresponding points obtained from the first image and the second image , the height of the passing object from the plane is time-sequentially related to a plurality of parts along the reference line on the passing object. A depth information calculation unit to be acquired;
Using the corresponding points obtained from the second image and the third image , the velocity in the traveling direction perpendicular to the reference line and parallel to the plane is time- sequentially related to at least one of the plurality of portions. A speed information calculation unit to be acquired automatically ;
An integration operation unit for time-integrating a value obtained by multiplying the acquired height by the weight of the speed for each direction of speed along the reference line;
Passing object counting device. A first imaging device provided to face the plane, which images a passing object passing through a reference line on the plane along the plane;
In parallel with the first imaging device, and provided opposite to said plane, a second imaging device for an image shooting the passage object passing through the reference line;
A first image of the passing object imaged by the first imaging device, the first of the said passage object imaged by a predetermined time by delay in the second imaging device from the imaging point of the first image An image acquisition unit for acquiring two images;
A corresponding point search unit for determining corresponding points from the first image and the second image;
Time using the corresponding points, for a plurality of sites along the reference line on the passage body, the height from the first and the plane of the passage body from the quantity representing the parallax between the second image A depth information calculation unit acquired in series;
A speed information calculation unit that acquires, in a time series, a speed in a traveling direction that is perpendicular to the reference line and parallel to the plane with respect to at least one part of the plurality of parts using the corresponding points ;
An integration operation unit for time-integrating a value obtained by multiplying the acquired height by the weight of the speed for each direction of speed along the reference line;
Passing object counting device. When a value obtained by time integration in the integration calculation unit exceeds a preset threshold value, a difference passage count calculation unit that counts that the passing object has passed by 1 is provided.
The passing object counting device according to claim 2 or 3. A division passage count calculation unit that counts a value obtained by dividing the value obtained by time integration in the integration calculation unit by a preset threshold as the number of passing objects;
The passing object counting device according to claim 2 or 3. A first imaging step of obtaining a first image by imaging a passing object passing through a reference line on the plane along a plane from a first position facing the plane at a first time point; ;
The passing object passing through the reference line is opposed to the plane in parallel with the first position at the first time point and a second time point after a predetermined time from the first time point. A second imaging step of imaging from the second position to obtain a second image and a third image;
An image processing step of obtaining the first image, the second image, and the third image;
A corresponding point search step of obtaining corresponding points from the first image and the second image and obtaining corresponding points from the second image and the third image;
Using the corresponding points obtained from the first image and the second image , the height of the passing object from the plane is time-sequentially related to a plurality of parts along the reference line on the passing object. Depth information calculation step to be acquired;
Using the corresponding points obtained from the second image and the third image , the velocity in the traveling direction perpendicular to the reference line and parallel to the plane is time- sequentially related to at least one of the plurality of portions. Speed information calculation step to be acquired automatically ;
An integration operation step of time-integrating a value obtained by multiplying a value obtained by multiplying the acquired height by the weight of the speed for each direction of speed along the reference line;
Passing object counting method. A first imaging step of obtaining a first image by imaging a passing object passing through a reference line on the plane along a plane from a first position facing the plane at a first time point; ;
The passing object passing through the reference line is imaged from a second position parallel to the first position and facing the plane at a second time point after a lapse of a predetermined time from the first time point. to a second imaging step of obtaining a second image;
An image acquisition step of acquiring the first image and the second image;
A corresponding point search step of obtaining corresponding points from the first image and the second image;
Time using the corresponding points, for a plurality of sites along the reference line on the passage body, the height from the first and the plane of the passage body from the quantity representing the parallax between the second image Depth information calculation step acquired in series;
A velocity information calculation step of acquiring, in a time series, a velocity in a traveling direction that is perpendicular to the reference line and parallel to the plane with respect to at least one portion of the plurality of portions using the corresponding points ;
An integration operation step of time-integrating a value obtained by multiplying a value obtained by multiplying the acquired height by the weight of the speed for each direction of speed along the reference line;
Passing object counting method. Presetting a volume of one average passing object as a threshold;
When the value obtained by time integration in the integration calculation unit exceeds the threshold value, a passing count calculation step for counting that the passing object has passed by 1 is provided.
The passing object counting method according to claim 6 or 7. Presetting a volume of one average passing object as a threshold;
A division passage counting calculation step of counting a value obtained by dividing the value obtained by time integration in the integration calculation unit by the threshold as the number of passing objects;
The passing object counting method according to claim 6 or 7.

Images