JP2018136895A - Image processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To meet the need for the provision of a technology that makes it possible to determine with higher accuracy whether or not there is occurrence of a prescribed event in a moving image captured by an imaging device.SOLUTION: Provided is an image processing device comprising: a data acquisition unit for acquiring multiple combinations of candidate parameters arranged in an ascending or descending order from beginning to tail, and an input moving image; and a parameter specification unit for determining, using the multiple combinations of candidate parameters in order from the beginning, whether or not there is occurrence of a prescribed event in the input moving image, repeatedly until at least the determination result changes, and specifying a parameter for determination for determining, on the basis of a candidate parameter with which the determination result changes, whether or not there is occurrence of the event in the captured moving image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and a program.

  2. Description of the Related Art Conventionally, a technique for determining whether or not a predetermined event has occurred in a captured moving image (captured moving image) based on a moving image captured by an imaging apparatus is known. For example, a technique is known in which whether or not a predetermined event has occurred in a captured moving image is detected by detecting the motion of the subject in the captured moving image and comparing the motion of the subject with a predetermined parameter. (For example, refer to Patent Document 1). In such a technique, parameters are selected according to the installation location of the imaging device and the number of imaging frames per second.

JP 2004-248090 A

  However, it is desirable to provide a technique that can determine with high accuracy whether or not a predetermined event has occurred in a moving image captured by an imaging device. For example, a technique is provided that makes it possible to more accurately determine whether or not a predetermined event has occurred in a moving image based on a more appropriately specified parameter by specifying the parameter more appropriately. It is hoped that

  In order to solve the above problem, according to an aspect of the present invention, a data acquisition unit that acquires a plurality of candidate parameters and input moving images arranged in ascending or descending order from the beginning to the end, and the plurality of types Using the candidate parameters in order from the beginning, it is repeatedly determined whether or not a predetermined event has occurred in the input moving image at least until the determination result changes, and based on the candidate parameter in which the determination result changes, the captured moving image There is provided an image processing apparatus comprising: a parameter specifying unit that specifies a determination parameter for determining whether or not the event has occurred in an image.

  The parameter specifying unit, when the input moving image in which the event has not occurred is acquired by the data acquisition unit, candidate parameters that are no longer determined that the event has occurred in the input moving image, or The candidate parameter immediately before the candidate parameter that has started to be determined that the event has occurred in the input moving image is specified as the determination parameter, and the input in which the event has occurred by the data acquisition unit When a moving image is acquired, a candidate parameter immediately before a candidate parameter that is no longer determined that the event has occurred in the input moving image, or the event has occurred in the input moving image The candidate parameter that has begun to be determined may be specified as the determination parameter.

  The parameter specifying unit, when the input moving image in which the event has not occurred is acquired by the data acquisition unit, candidate parameters that are no longer determined that the event has occurred in the input moving image, or A candidate parameter immediately before a candidate parameter that has started to be determined that the event has occurred in the input moving image may be specified as the determination parameter.

  When the input moving image in which the event has not occurred is acquired by the data acquisition unit, the parameter specifying unit determines candidate parameters that are no longer determined that the event has occurred in the input moving image, You may specify as a parameter for determination.

  The parameter specifying unit is one of the candidate parameters that has started to determine that the event has occurred in the input moving image when the input moving image in which the event has not occurred is acquired by the data acquisition unit The previous candidate parameter may be specified as the determination parameter.

  The image processing apparatus may include a state determination unit that determines whether the event has occurred in the captured moving image using the determination parameter.

  The image processing apparatus may include a detection result display control unit that controls display of the detection result of the event when it is determined that the event has occurred in the captured moving image.

  The parameter specifying unit calculates, as a first average feature amount, an average value for each corresponding position of the optical flow in the first time range based on the input moving image, and is shorter than the first time range. An average value for each corresponding position of the optical flow in the two time ranges is calculated as a second average feature amount, and a predetermined calculation result and a threshold value according to the first average feature amount and the second average feature amount Whether or not the event has occurred in the input moving image may be determined based on the relationship between

  The determination parameter may include at least one of the first time range, the second time range, and the threshold value.

  The determination parameter may include at least one of a temporal density at which the optical flow is calculated and a spatial density at which the optical flow is calculated.

  The image processing device, when an optical flow display control unit that controls display according to an optical flow detected based on the captured moving image and an operation for changing the determination parameter are input, the determination parameter A parameter changing unit to be changed.

  When the parameter used in the past is recorded, the parameter specifying unit uses the parameter used in the past to determine whether an event corresponding to the parameter used in the past occurs in the input moving image. The image processing apparatus determines whether the event has occurred in the input moving image by the parameter specifying unit when the input moving image in which the event has not occurred is acquired by the data acquisition unit. When it is determined that there is no event, or when the input moving image in which the event has occurred is acquired by the data acquisition unit, and it is determined that the event has occurred in the input moving image by the parameter specifying unit If it is, at least one of the table used in the past and the information related to the parameter used in the past It may comprise parameter information display control unit that controls.

  The information related to the parameters used in the past includes the types of events determined using the parameters used in the past, the imaging distance by the imaging device that captured the moving image used for the determination, and It may include at least one of the angles of view of the imaging device.

  The parameter specifying unit may specify the parameter used in the past as the determination parameter when a selection operation of the parameter used in the past is input.

  The parameter specifying unit may specify the determination parameter for each region based on the event for each region provided on the input moving image.

  Further, according to another aspect of the present invention, the computer is configured to acquire a plurality of candidate parameters and input moving images arranged in ascending or descending order from the beginning to the end, and the plurality of candidate parameters. Are used in order from the beginning, repeatedly determining whether or not a predetermined event has occurred in the input moving image until at least the determination result changes, and based on the candidate parameter that changes the determination result, A program for functioning as an image processing apparatus is provided, comprising: a parameter specifying unit that specifies a determination parameter for determining whether or not the event has occurred.

  As described above, according to the present invention, there is provided a technique capable of determining with high accuracy whether or not a predetermined event has occurred in a moving image captured by an imaging device.

1 is a diagram illustrating a configuration example of an image processing system according to a first embodiment. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 1st Embodiment. It is a figure which shows the example of the some image by which the change of the vehicle travel route was imaged with the imaging device along the time series. It is a figure for demonstrating the function of a state determination part. It is a figure showing an example of a parameter for judgment. It is a figure for demonstrating the 1st example of the method of specifying the parameter for determination. It is a figure for demonstrating the 2nd example of the method of specifying the parameter for determination. It is a figure for demonstrating the 3rd example of the method of specifying the parameter for determination. It is a figure for demonstrating the 4th example of the method of specifying the parameter for determination. It is a flowchart which shows the operation example of the parameter for determination in the image processing system which concerns on 1st Embodiment. It is a figure which shows the example of a display of an abnormality detection area | region. 5 is a flowchart illustrating an example of an operation for determining a state in the image processing system according to the first embodiment. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 2nd Embodiment. It is a figure which shows the example of a display of an optical flow. 10 is a flowchart illustrating an operation example of determination parameter change in the image processing system according to the second embodiment. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 3rd Embodiment. It is a figure which shows the example of the information relevant to the parameter used in the past, and the parameter used in the past. It is a figure which shows the example of the determination result using the parameter used in the past. It is a figure which shows the example of a display of the information relevant to the parameter used in the past, and the parameter used in the past. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 4th Embodiment. It is a figure which shows the example of the correspondence of the coordinate for every area | region, and the kind of event. 14 is a flowchart illustrating an example of an operation for specifying a determination parameter in the image processing system according to the fourth embodiment. It is a figure which shows the hardware constitutions of the image processing apparatus which concerns on embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different numerals to the same reference numerals. However, when there is no need to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given. In addition, similar components in different embodiments may be distinguished by attaching different alphabets after the same reference numerals. However, in the case where it is not necessary to particularly distinguish each of similar components of different embodiments, only the same reference numerals are given.

(0. Overview)
First, an outline of the present embodiment will be described. In general, a technique for determining whether or not a predetermined event has occurred in a captured moving image (captured moving image) based on a moving image captured by an imaging apparatus is known. For example, a technique is known in which whether or not a predetermined event has occurred in a captured moving image is detected by detecting the motion of the subject in the captured moving image and comparing the motion of the subject with a predetermined parameter. (For example, refer to Patent Document 1). In such a technique, parameters are selected according to the installation location of the imaging device and the number of imaging frames per second.

  However, in the present specification, a technique that makes it possible to determine with high accuracy whether or not a predetermined event has occurred in a moving image captured by an imaging apparatus will be mainly described. For example, in the technique described in Patent Document 1, it is necessary to specify a parameter through trial and error, and it takes time to specify the parameter. In this specification, it is possible to determine with high accuracy whether or not a predetermined event has occurred in a moving image based on a more appropriately specified parameter by specifying the parameter more appropriately. The technology to do is mainly explained.

  The background of the present embodiment has been described above.

(1. First embodiment)
Next, the first embodiment will be described.

[1-1. System overview]
First, a configuration example of the image processing system according to the first embodiment will be described.

  FIG. 1 is a diagram illustrating a configuration example of an image processing system according to the first embodiment. As illustrated in FIG. 1, an image processing system 1A according to the first embodiment includes an image processing device 10A and an imaging device 30. An image processing system 1B according to the second embodiment described later includes an image processing apparatus 10B instead of the image processing apparatus 10A, and an image processing system 1C according to the third embodiment includes an image processing apparatus 10A. Instead, an image processing apparatus 10C is provided, and an image processing system 1D according to the fourth embodiment includes an image processing apparatus 10D instead of the image processing apparatus 10A.

  As illustrated in FIG. 1, the image processing apparatus 10 </ b> A and the imaging apparatus 30 can perform communication by wire or wirelessly. For example, the image processing device 10A and the imaging device 30 may be directly connected or may be connected via a network. When the image processing apparatus 10A and the imaging apparatus 30 are connected via a network, the type of the network is not particularly limited. For example, such a network may include the Internet. Alternatively, the imaging device 30 may be integrated with the image processing device 10A instead of a separate body.

  The imaging device 30 continuously captures an imaging range including a detection target area to obtain a moving image. Here, the position where the imaging device 30 is installed is not limited. Further, the detection target area may be the entire imaging range or a part thereof. In the present specification, it is assumed that the imaging device 30 is a fixed camera. However, the imaging device 30 may be a movable camera. For example, the imaging device 30 may be a PTZ (pan / tilt / zoom) camera. The moving image captured by the imaging device 30 is provided to the image processing device 10A.

  The image processing device 10 </ b> A acquires the moving image provided from the imaging device 30. Then, the image processing apparatus 10 </ b> A determines whether or not a predetermined event has occurred in the moving image based on the moving image provided from the imaging device 30. The predetermined event is not particularly limited. In this specification, as an example of the predetermined event, a change in the vehicle travel route (for example, avoidance) is mainly assumed. However, the predetermined event may be a change in another vehicle travel route (for example, reverse travel).

  Alternatively, the predetermined event may be any change other than a change in the vehicle travel route (for example, a movement that shows a regular movement when normal but a movement that violates the rule shown when abnormal). For example, a change in the vehicle travel route may be replaced with a change in the movement of an article conveyed on the belt conveyor, or a change in a flow of people (for example, a flow of people who visit the first class). Also good. Alternatively, the predetermined event may be some natural phenomenon (for example, debris flow, falling object, landslide, tsunami, eruption, etc.). In the following description, a state in which a predetermined event occurs in a moving image is also referred to as “the moving image is abnormal” or “the moving image is abnormal”, and the predetermined event occurs. The moving image in the state is also referred to as “moving image in which an abnormality has occurred” or the like. A state in which a predetermined event has not occurred in a moving image is also referred to as “no abnormality in the moving image” or “a moving image is normal”, and a moving image in which a predetermined event has not occurred It is also referred to as “a moving image in which no occurrence has occurred (normal moving image)”.

  At this time, the image processing apparatus 10A specifies a determination parameter for determining whether or not a predetermined event has occurred in the moving image. The image processing apparatus 10A determines whether a predetermined event has occurred in the moving image using the determination parameter. Further, the image processing apparatus 10 </ b> A can output a region where a predetermined event occurs as a detection result. Details of the determination by the image processing apparatus 10A will be described later in detail.

  In the example illustrated in FIG. 1, there is one image processing apparatus 10 </ b> A and one imaging apparatus 30. However, the image processing apparatus 10A and the imaging apparatus 30 may include a plurality of any one of them, or a plurality of both of them. That is, the numbers of the image processing device 10A and the imaging device 30 are not particularly limited.

  The configuration example of the image processing system 1A according to the first embodiment has been described above.

[1-2. Functional configuration example of image processing apparatus]
Next, a functional configuration example of the image processing apparatus 10A according to the first embodiment will be described. FIG. 2 is a block diagram illustrating a functional configuration example of the image processing apparatus 10A according to the first embodiment. As illustrated in FIG. 2, the image processing apparatus 10A according to the first embodiment includes a control unit 110A, an operation unit 120, a communication unit 130, a storage unit 140, and a display unit 150.

  The control unit 110A includes a CPU (Central Processing Unit) and the like, and the function can be realized by the program stored in the storage unit 140 being expanded and executed by the CPU in a RAM (Random Access Memory). At this time, a computer-readable recording medium that records the program can also be provided. Or control part 110A may be constituted by exclusive hardware, and may be constituted by combination of a plurality of hardware.

  The operation unit 120 receives an operation input from a user. The operation unit 120 can provide the operation received from the user to the control unit 110A. In this specification, the case where the operation unit 120 is a touch panel is mainly assumed. However, the operation unit 120 may be an input device (for example, a button) other than the touch panel.

  The communication unit 130 is a communication interface for performing communication with the imaging device 30. As described above, in this specification, it is mainly assumed that the communication unit 130 performs wired communication, but the communication unit 130 may perform wireless communication.

  Storage unit 140 is a storage device capable of storing a program and data for operating control unit 110A. In addition, the storage unit 140 can temporarily store various data required in the course of the operation of the control unit 110A. For example, the storage device may be a non-volatile storage device.

  The display unit 150 has a function of performing output according to control by the control unit 110A. For example, the display unit 150 can output a detection result by the control unit 110A. Here, the form of the display unit 150 is not particularly limited. For example, the display unit 150 may be a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp.

  In the example shown in FIG. 2, the operation unit 120, the communication unit 130, the storage unit 140, and the display unit 150 are provided inside the image processing apparatus 10A. However, some or all of these functional units may be provided outside the image processing apparatus 10A. Here, the control unit 110A includes a data acquisition unit 111, a parameter identification unit 112A, a state determination unit 113, and a detection result display control unit 114. Details of these functional units included in the control unit 110A will be described later.

  The functional configuration example of the image processing apparatus 10A according to the first embodiment has been described above.

[1-3. detail of function]
Next, details of functions of the image processing system 1A will be described. First, as an example of a method for determining whether or not a predetermined event has occurred in a moving image (captured moving image) captured by the imaging device 30, a change in a vehicle traveling route (for example, avoidance) is included in the moving image. An example of determining whether or not it has occurred will be described. However, as described above, the determination of whether or not a predetermined event has occurred in the moving image is not limited to the determination of whether or not a change in the vehicle travel route (for example, avoidance) has occurred in the moving image.

  FIG. 3 is a diagram illustrating an example of a plurality of images in which changes in the vehicle travel route are captured by the imaging device 30 in time series. With reference to FIG. 3, an example of a plurality of images captured in time series by the imaging device 30 is shown. More specifically, in order from the oldest time, the image G21 is captured at time T21, the image G22 is captured at time T22, the image G23 is captured at time T23, and the image G24 is captured at time T24. .

  Here, the case where the detection target area is the entire area of the image will be mainly described as an example. However, the detection target area is not limited to the entire area of the image. For example, the detection target area may be a partial area of the image. Referring to FIG. 3, the image G21 shows a vehicle showing regular movement. On the other hand, since a part of the road is blocked by the earth and sand K2, the image G22 shows that the movement of the vehicle traveling so as to avoid the earth and sand K2 is irregular. The vehicle which shows irregular motion is reflected also in the image G23 and the image G24.

  Note that the images G <b> 21 to G <b> 24 illustrated in FIG. 3 are only part of the moving image captured by the imaging device 30. Therefore, an image is captured by the imaging device 30 between the images G21 to G24 (specifically, between the image G21 and the image G22, between the image G22 and the image G23, and between the image G23 and the image G24). There are other images. When a moving image is captured by the imaging device 30, the moving image captured by the imaging device 30 is provided to the image processing device 10A as a captured moving image.

  In the image processing apparatus 10A, when the communication unit 130 receives a captured moving image, the data acquisition unit 111 acquires the captured moving image. The state determination unit 113 determines whether there is an abnormality in the captured moving image acquired by the data acquisition unit 111. In this specification, it is assumed that the state determination unit 113 determines whether or not an abnormality has occurred in the captured moving image using the determination parameter specified by the parameter specification unit 112A. The determination of the determination parameter by the parameter specifying unit 112A will be described in detail later.

  FIG. 4 is a diagram for explaining the function of the state determination unit 113. Referring to FIG. 4, an example of a captured moving image G2 received by the communication unit 130 is illustrated. The captured moving image G2 also includes the images G21 to G24 illustrated in FIG. In the example illustrated in FIG. 4, among the plurality of images included in the captured moving image G <b> 2, the image whose imaging time is newer is shown on the right side. As shown in FIG. 4, the state determination unit 113 detects an optical flow by calculating, for each pixel, a motion vector between two adjacent images in the captured moving image G2. In FIG. 4, a set along a time series of motion vectors (optical flow) for each pixel is shown as a motion information group M2.

  In the image processing apparatus 10A, the state determination unit 113 calculates an average value for each corresponding position of the motion vector (optical flow) in the first time range t21 as a first average feature amount. Hereinafter, the first average feature amount is also referred to as a “long-time motion average vector”. Note that the motion average vector can be expressed as (Formula 2) below using the motion vector (Formula 1).

  Referring to FIG. 4, a long-time motion average vector V21 is shown. Here, for the sake of simplicity of the drawing, FIG. 4 shows only the motion average vector for some pixels as the long-time motion average vector V21. However, in this specification, it is assumed that the long-time motion average vector V21 is an average value of motion vectors calculated for each corresponding pixel in the first time range t21.

  On the other hand, the state determination unit 113 calculates an average value for each corresponding position of the motion vector (optical flow) in the second time range t23 shorter than the first time range t21 as the second average feature amount. Hereinafter, the second average feature amount is also referred to as a “short-time motion average vector”.

  Referring to FIG. 4, a short-time motion average vector V23 is shown. Here, for the sake of simplicity of the drawing, FIG. 4 shows only the motion average vector for some pixels as the short-time motion average vector V23. However, in this specification, it is assumed that the short-time motion average vector V23 is an average value of motion vectors calculated for each corresponding pixel in the second time range t23.

  Referring to FIG. 4, since the first time range t21 and the second time range t23 are different, there is a difference between the long-time motion average vector V21 and the short-time motion average vector V23. Furthermore, even if some noise is included in the image, most of the noise is canceled out by averaging. Therefore, the short-time motion average vector V23 contains almost no noise, and it is determined with higher accuracy whether or not an abnormality has occurred in the captured moving image G2.

  FIG. 4 shows an example in which the end time points of the first time range t21 and the second time range t23 are the current time (that is, the latest image is captured). However, the end time points of the first time range t21 and the second time range t23 may not be strictly current (that is, the time point when the image before the latest image is captured). . Further, the first time range t21 and the second time range t23 do not have to coincide strictly. That is, the end points of the first time range t21 and the second time range t23 may be shifted.

  The state determination unit 113 determines whether or not an abnormality has occurred in the captured moving image G2 based on a relationship between a predetermined calculation result corresponding to the long-time motion average vector V21 and the short-time motion average vector V23 and a threshold value. To do. For example, the state determination unit 113 determines that an abnormality has occurred in the captured moving image G2 when there is a pixel whose predetermined calculation result exceeds a threshold value. On the other hand, the state determination unit 113 determines that there is no abnormality in the captured moving image G2 when there is no pixel whose predetermined calculation result exceeds the threshold.

  Here, the predetermined calculation according to the long-time motion average vector V21 and the short-time motion average vector V23 is not particularly limited. For example, the predetermined calculation includes an amount corresponding to an angle between the long-time motion average vector V21 and the short-time motion average vector V23 (hereinafter, also referred to as “an angle between both vectors”), and a long-time motion. It may include at least one calculation among amounts corresponding to the magnitude of the difference between the average vector V21 and the short-time motion average vector V23 (hereinafter also referred to as “difference vector”).

  Here, the amount corresponding to the angle between the two vectors may be an average value of the angles or an angular dispersion. The amount corresponding to the magnitude of the difference vector may be an average value (or total value) of the magnitudes of the difference vectors, or a change amount of the magnitude of the difference vector (for example, the magnitude of the magnitude of the difference vector). It may be an average value (or total value) of first-order derivatives according to time) or an area of a region where the magnitude of the difference vector exceeds a predetermined size (hereinafter also referred to as “motion region”). May be.

  As described above, in this example, it is assumed that a change in the vehicle travel route is determined as abnormal. Here, it is assumed that the direction of movement changes when a movement that violates a rule such as a change in the vehicle travel route occurs. Therefore, when it is determined that the occurrence of a motion that violates the rule is abnormal, the predetermined calculation by the state determination unit 113 is an amount corresponding to the angle between the long-time motion average vector V21 and the short-time motion average vector V23. It should be an operation.

  In this example, the case where the predetermined calculation by the state determination unit 113 is calculation of the variance of the angle between both vectors will be mainly described as an example. Here, the motion vector is expressed as in (Equation 3), and the motion average vector (each of the long-time motion average vector V21 and the short-time motion average vector V23) is expressed as (Equation 4). The average value of the angle between the motion average vector V21 and the short-time motion average vector V23 is expressed as (Equation 5), and the variance of the angle between both vectors is expressed as (Equation 6). The

  When there is a pixel whose result (in this example, the angle dispersion between both vectors) obtained based on a predetermined calculation exceeds the threshold, the state determination unit 113 has an abnormality in the captured moving image G2. It is determined that On the other hand, when there is no pixel in which the result obtained based on a predetermined calculation (in this example, the variance of the angle between both vectors) exceeds the threshold, the state determination unit 113 has an abnormality in the captured moving image G2. It is determined that it has not occurred.

  In the above, the case where the predetermined calculation by the state determination unit 113 is the calculation of the amount corresponding to the angle between the long-time motion average vector V21 and the short-time motion average vector V23 has been mainly described. However, as described above, it may be assumed that the occurrence of debris flow is determined to be abnormal. Here, when a natural phenomenon such as debris flow occurs, it is assumed that the magnitude of movement in the captured moving image G2 changes. Therefore, when the occurrence of a natural phenomenon is determined as abnormal, the predetermined calculation by the state determination unit 113 is preferably an amount calculation according to the magnitude of the difference vector.

  Here, the case where the calculation of the amount according to the magnitude of the difference vector is the calculation of the average value of the magnitudes of the difference vectors will be mainly described as an example. Here, the average value of the magnitudes of the long-time motion average vector V21, the short-time motion average vector V23, and the difference vector is expressed as (Equation 7).

  The example in which the state determination unit 113 determines whether or not an abnormality has occurred in the captured moving image has been described above. As described above, the state determination unit 113 determines whether an abnormality has occurred in the captured moving image using the determination parameter specified by the parameter specification unit 112A. Here, various parameters are assumed as the determination parameters. Below, the example of the parameter for determination is demonstrated.

  FIG. 5 is a diagram illustrating examples of determination parameters. As shown in FIG. 5, the types of determination parameters are assumed to be the first time range t21 described above, the second time range t23 described above, a threshold value (compared with the calculation result described above), and the like. Further, as shown in FIG. 5, as the type of determination parameter, density or the like is assumed. The density may be a temporal density at which the optical flow is calculated (for example, every frame, every 3 frames, etc.), or a spatial density at which the optical flow is calculated (for example, every pixel, 3 For each pixel).

  When the value of the determination parameter changes, it is assumed that the ease of determination that an abnormality has occurred changes. Hereinafter, the fact that it is easy to determine that an abnormality has occurred is also referred to as “relaxation condition” being “loose”. On the other hand, the fact that it is difficult to determine that an abnormality has occurred is also said that the “abnormality determination condition” is “strict”.

  For example, as shown in FIG. 5, when the determination parameter is in the first time range t21, the larger the value of the determination parameter (the difference between the first time range t21 and the second time range t23 is. It is assumed that the condition for abnormality determination becomes severe (because the calculation result tends to exceed the threshold value). On the other hand, as shown in FIG. 5, when the determination parameter is the second time range t23, the smaller the determination parameter value is, the smaller the difference between the first time range t21 and the second time range t23 is. It is assumed that the condition for abnormality determination becomes severe (because the calculation result tends to exceed the threshold value).

  Further, as shown in FIG. 5, when the determination parameter is a threshold value, it is assumed that the smaller the determination parameter value, the more severe the condition for abnormality determination (because the above calculation result easily exceeds the threshold value). Is done. On the other hand, as shown in FIG. 5, when the determination parameter is density, the larger the determination parameter value, the greater the difference between the above long-time motion average vector and the above short-time motion average vector. Therefore, it is assumed that the condition for abnormality determination becomes severe (because the above calculation result easily exceeds the threshold).

  Hereinafter, an example in which the parameter specifying unit 112A specifies one of these determination parameters will be described. However, the parameter specifying unit 112A may specify any two or more of these determination parameters. In such a case, the parameter specifying unit 112A may specify any two or more of these determination parameters. As an example, the parameter specifying unit 112A may specify any two or more of these determination parameters independently.

  FIG. 6 is a diagram for explaining a first example of a method for specifying a determination parameter. First, the storage unit 140 stores in advance a plurality of candidate parameters arranged in order of severe abnormality determination conditions from the beginning to the end. For example, the order in which the conditions for abnormality determination are strict corresponds to the first time range t21 and density in the descending order (that is, descending order), and the second time range t23 and threshold value in the ascending order ( That is, it corresponds to ascending order). The candidate parameter corresponds to a determination parameter candidate. Here, it is assumed that candidate parameters P1 to P4 arranged in order of severe abnormality determination conditions are used as examples of a plurality of candidate parameters. However, the number of candidate parameters is not limited to four, and may be plural.

  The data acquisition unit 111 acquires candidate parameters P1 to P4 stored in the storage unit 140. Further, the data acquisition unit 111 acquires an input moving image for specifying a determination parameter. Note that in this specification, the imaging target region is the same in the input moving image for specifying the determination parameter and the captured moving image in which it is determined whether or not an abnormality has occurred by the state determination unit 113. The case is mainly assumed. That is, the case where the input moving image for specifying the determination parameter is also captured by the imaging device 30 and provided to the image processing device 10A is mainly assumed. However, the imaging target region may be different between the input moving image for specifying the determination parameter and the captured moving image in which it is determined whether or not an abnormality has occurred by the state determination unit 113. At this time, the input moving image for specifying the determination parameter may be stored in advance by the storage unit 140.

  In the example shown in FIG. 6, it is confirmed that there is no abnormality (normal) in the input moving image. It may be confirmed in any way that no abnormality has occurred in the input moving image. For example, the fact that no abnormality has occurred in the input moving image may be confirmed by a person when the input moving image is captured. Alternatively, the fact that no abnormality has occurred in the input moving image may be confirmed by a person when the input moving image stored in advance by the storage unit 140 is reproduced and displayed.

  The parameter specifying unit 112A uses the candidate parameters P1 to P4 in order from the head, and repeatedly determines whether or not an abnormality has occurred in the input moving image until at least the determination result changes. Then, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter whose determination result changes. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113. Therefore, according to such a configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred in the captured moving image.

  In the example illustrated in FIG. 6, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the leading candidate parameter P1, and obtains a determination result “abnormal”. Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P2, and obtains a determination result “abnormal”. Therefore, in the candidate parameter P2, the determination result is “no change”.

  Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P3, and obtains a determination result “normal”. Accordingly, in the candidate parameter P3, the determination result is “changed”. Therefore, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter P3 whose determination result changes.

  In the example illustrated in FIG. 6, since the input moving image in which no abnormality has occurred is acquired by the data acquisition unit 111, the parameter specifying unit 112 </ b> A determines the candidate parameter (that is, the input moving image with respect to the input moving image). The candidate parameter (P3) that is no longer determined to be abnormal is specified as a determination parameter. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113.

  Further, it may be difficult to prepare an input moving image in which an abnormality has occurred in advance. However, as in the example shown in FIG. 6, if the determination parameter is specified using an input moving image in which no abnormality has occurred, it is not necessary to prepare an input moving image in which an abnormality has occurred in advance. Further, as in the example shown in FIG. 6, the determination using the candidate parameter is performed in the order in which the abnormality determination condition is strict, so that the determination using the next candidate parameter is performed when the pixel in which the abnormality occurs is detected. Therefore, the processing time is reduced.

  FIG. 7 is a diagram for explaining a second example of a method for specifying a determination parameter. In this case, the storage unit 140 stores in advance a plurality of candidate parameters arranged in order of loose abnormality determination conditions from the beginning to the end. For example, the order in which the condition of abnormality determination is gradual corresponds to the first time range t21 and the density in ascending order (that is, ascending order), and the second time range t23 and the threshold value in the descending order (threshold). That is, it corresponds to descending order). Here, as an example of a plurality of candidate parameters, a case is assumed where candidate parameters P4 to P1 arranged in order of loose abnormality determination conditions are used.

  The data acquisition unit 111 acquires candidate parameters P4 to P1 stored in the storage unit 140. Further, the data acquisition unit 111 acquires an input moving image for specifying a determination parameter. In the example shown in FIG. 7, it is confirmed that there is no abnormality (normality) in the input moving image.

  The parameter specifying unit 112A uses the candidate parameters P4 to P1 in order from the head, and repeatedly determines whether an abnormality has occurred in the input moving image until at least the determination result changes. Then, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter whose determination result changes. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113. Therefore, according to such a configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred in the captured moving image.

  In the example illustrated in FIG. 7, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the leading candidate parameter P4, and obtains a determination result “normal”. Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P3, and obtains a determination result “normal”. Therefore, in the candidate parameter P3, the determination result is “no change”.

  Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P2, and obtains a determination result “abnormal”. Therefore, in the candidate parameter P2, the determination result is “changed”. Therefore, the parameter specifying unit 112A specifies a determination parameter based on the candidate parameter P2 whose determination result changes.

  In the example illustrated in FIG. 7, since the input moving image in which no abnormality has occurred is acquired by the data acquiring unit 111, the parameter specifying unit 112A performs the candidate parameter (that is, the input moving image with respect to the input moving image). The candidate parameter P3 immediately before the candidate parameter P2 that has started to be determined as abnormal is specified as the determination parameter. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113.

  Further, it may be difficult to prepare an input moving image in which an abnormality has occurred in advance. However, as in the example shown in FIG. 7, if the determination parameter is specified using an input moving image in which no abnormality has occurred, it is not necessary to prepare an input moving image in which an abnormality has occurred in advance.

  FIG. 8 is a diagram for explaining a third example of a method for specifying a determination parameter. First, the storage unit 140 stores in advance a plurality of candidate parameters arranged in order of severe abnormality determination conditions from the beginning to the end. For example, the order in which the conditions for abnormality determination are strict corresponds to the first time range t21 and density in the descending order (that is, descending order), and the second time range t23 and threshold value in the ascending order ( That is, it corresponds to ascending order). Here, as an example of a plurality of candidate parameters, it is assumed that candidate parameters P1 to P4 arranged in order of severe abnormality determination conditions are used. However, the number of candidate parameters is not limited to four, and may be plural.

  The data acquisition unit 111 acquires candidate parameters P1 to P4 stored in the storage unit 140. Further, the data acquisition unit 111 acquires an input moving image for specifying a determination parameter.

  In the example shown in FIG. 8, it is confirmed that an abnormality has occurred in the input moving image. It may be confirmed in any way that an abnormality has occurred in the input moving image. For example, the occurrence of an abnormality in the input moving image may be confirmed by a person when the input moving image is captured. Alternatively, the occurrence of an abnormality in the input moving image may be confirmed by a person when the input moving image stored in advance by the storage unit 140 is reproduced and displayed.

  The parameter specifying unit 112A uses the candidate parameters P1 to P4 in order from the head, and repeatedly determines whether or not an abnormality has occurred in the input moving image until at least the determination result changes. Then, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter whose determination result changes. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113. Therefore, according to such a configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred in the captured moving image.

  In the example illustrated in FIG. 8, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the leading candidate parameter P1, and obtains a determination result “abnormal”. Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P2, and obtains a determination result “abnormal”. Therefore, in the candidate parameter P2, the determination result is “no change”. Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P3, and obtains a determination result “abnormal”. Therefore, in the candidate parameter P3, the determination result is “no change”.

  Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P4, and obtains a determination result “normal”. Accordingly, in the candidate parameter P4, the determination result is “changed”. Therefore, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter P4 whose determination result changes.

  In the example illustrated in FIG. 8, since the input moving image in which an abnormality has occurred is acquired by the data acquisition unit 111, the parameter specifying unit 112A determines whether the candidate parameter whose determination result changes (that is, the input moving image The candidate parameter P4 immediately before the candidate parameter P4 that is no longer determined to be abnormal is specified as the determination parameter. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113.

  Further, as in the example shown in FIG. 8, the determination using the candidate parameter is performed in the order in which the abnormality determination condition is strict, and the determination using the next candidate parameter is performed when the pixel in which the abnormality occurs is detected. Therefore, the processing time is reduced.

  FIG. 9 is a diagram for explaining a fourth example of a method for specifying a determination parameter. In this case, the storage unit 140 stores in advance a plurality of candidate parameters arranged in order of loose abnormality determination conditions from the beginning to the end. For example, the order in which the condition of abnormality determination is gradual corresponds to the first time range t21 and the density in ascending order (that is, ascending order), and the second time range t23 and the threshold value in the descending order (threshold). That is, it corresponds to descending order). Here, as an example of a plurality of candidate parameters, a case is assumed where candidate parameters P4 to P1 arranged in order of loose abnormality determination conditions are used.

  The data acquisition unit 111 acquires candidate parameters P4 to P1 stored in the storage unit 140. Further, the data acquisition unit 111 acquires an input moving image for specifying a determination parameter. In the example shown in FIG. 9, it is confirmed that an abnormality has occurred in the input moving image.

  The parameter specifying unit 112A uses the candidate parameters P4 to P1 in order from the head, and repeatedly determines whether an abnormality has occurred in the input moving image until at least the determination result changes. Then, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter whose determination result changes. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113. Therefore, according to such a configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred in the captured moving image.

  In the example illustrated in FIG. 9, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the leading candidate parameter P4, and obtains a determination result “normal”. Subsequently, the parameter specifying unit 112A determines whether an abnormality has occurred in the input moving image using the subsequent candidate parameter P3, and obtains a determination result “abnormal”. Accordingly, in the candidate parameter P3, the determination result is “changed”. Therefore, the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter P3 whose determination result changes.

  In the example illustrated in FIG. 9, since the input moving image in which an abnormality has occurred is acquired by the data acquisition unit 111, the parameter specifying unit 112A determines whether the candidate parameter whose determination result changes (that is, the input moving image The candidate parameter (P3) that has started to be determined as abnormal is specified as a determination parameter. According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113.

  Subsequently, an operation example for determining a determination parameter in the image processing system 1A according to the first embodiment will be described. FIG. 10 is a flowchart illustrating an example of an operation for specifying a determination parameter in the image processing system 1A according to the first embodiment. Note that the determination parameter specifying operation example described below is merely an example of the determination parameter specifying operation in the image processing system 1A. Therefore, the determination parameter specifying operation in the image processing system 1A is not limited to the determination parameter specifying operation example described below.

  First, in the image processing apparatus 10A, the input moving image is acquired by the data acquisition unit 111 (S11). Subsequently, the parameter specifying unit 112A acquires the first candidate parameter from the storage unit 140 (S12). The parameter specifying unit 112A obtains a determination result by determining whether an abnormality has occurred in the input moving image using the acquired candidate parameter.

  Subsequently, as long as the determination result does not change (“No” in S13), the parameter specifying unit 112A acquires the next candidate parameter (S14), and whether an abnormality has occurred in the input moving image using the acquired candidate parameter. By determining whether or not, the determination result is obtained, and the operation is shifted to S13. On the other hand, when the determination result changes (“Yes” in S13), the parameter specifying unit 112A specifies the determination parameter based on the candidate parameter when the determination result changes (S15), and the determination parameter specifying End the operation.

  Using the determination parameters specified in this manner, the state determination unit 113 determines whether or not an abnormality has occurred in the captured moving image. When the state determination unit 113 determines that an abnormality has occurred in the captured moving image, the detection result display control unit 114 controls display of the abnormality detection result. As an example, when the state determination unit 113 determines that an abnormality has occurred in the captured moving image, the detection result display control unit 114 displays an area in which an abnormality has occurred (an abnormality detection area) as an example of the detection result. The display unit 150 is controlled as described above.

  FIG. 11 is a diagram illustrating a display example of the abnormality detection area. Referring to FIG. 11, the detection result display control unit 114 controls the display unit 150 so that the captured moving image G25 is displayed on the display unit 150. In addition, the detection result display control unit 114 controls the display unit 150 so that the abnormality detection region D25 (for example, the region where the vehicle is prevented from running) in the captured moving image G25 is displayed in a different manner from the other regions. doing. For example, as in the example illustrated in FIG. 11, the detection result display control unit 114 displays a predetermined display object (for example, shading) on the abnormality detection area D25 in the captured moving image G25. The unit 150 may be controlled.

  Subsequently, an operation example of state determination in the image processing system 1A according to the first embodiment will be described. FIG. 12 is a flowchart illustrating an operation example of state determination in the image processing system 1A according to the first embodiment. Note that the state determination operation example described below is merely an example of the state determination operation in the image processing system 1A. Accordingly, the state determination operation in the image processing system 1A is not limited to the state determination operation example described below.

  First, in the image processing apparatus 10A, the state determination unit 113 acquires the determination parameter specified by the parameter specification unit 112A (S21). Subsequently, the imaging device 30 captures a moving image and provides the captured moving image to the image processing device 10A. The moving image provided by the imaging device 30 is acquired as a captured moving image by the data acquisition unit 111 (S22). Subsequently, the state determination unit 113 performs state determination (determination as to whether or not an abnormality has occurred in the captured moving image) based on the determination parameter and the captured moving image, and obtains a determination result (S23).

  When the determination result is abnormal (“Yes” in S24), the detection result display control unit 114 controls the display unit 150 so that a region where an abnormality has occurred (abnormality detection region) is displayed as an example of the detection result. (S25), and the process proceeds to S26. On the other hand, when the determination result is normal (“No” in S24), the state determination unit 113 shifts the operation to S26. When the operation is continued (“No” in S26), the data acquisition unit 111 shifts the operation to S22. On the other hand, when the data acquisition unit 111 ends the operation (“Yes” in S26), the data acquisition unit 111 ends the operation.

  The first embodiment has been described above.

(2. Second Embodiment)
Next, the second embodiment will be described. As shown in FIG. 1, the image processing system 1B according to the second embodiment includes an image processing device 10B instead of the image processing device 10A, as compared with the image processing system 1A according to the first embodiment. . Therefore, in the following, description of the imaging device 30 is omitted, and the image processing device 10B is mainly described.

[2-1. Functional configuration example of image processing apparatus]
First, a functional configuration example of the image processing apparatus 10B according to the second embodiment will be described. FIG. 13 is a block diagram illustrating a functional configuration example of the image processing apparatus 10B according to the second embodiment. As illustrated in FIG. 13, the image processing apparatus 10B according to the second embodiment includes a control unit 110B instead of the control unit 110A, as compared with the image processing apparatus 10A according to the first embodiment. Therefore, below, description about the operation part 120, the communication part 130, the memory | storage part 140, and the display part 150 is abbreviate | omitted, and mainly demonstrates the difference with 110 A of control parts among the control parts 110B.

  As illustrated in FIG. 13, the control unit 110B includes a data acquisition unit 111, a parameter identification unit 112A, a state determination unit 113, and a detection result display control unit 114, similarly to the control unit 110A. In addition, the control unit 110B includes an optical flow detection unit 115, an optical flow display control unit 116, and a parameter change unit 117. Details of these functional units included in the control unit 110B will be described later.

[2-2. detail of function]
Next, details of functions of the image processing system 1B will be described. Also in the second embodiment, as in the first embodiment, the data acquisition unit 111 acquires a captured moving image. In the second embodiment, the optical flow detection unit 115 detects the optical flow based on the captured moving image acquired by the data acquisition unit 111. The optical flow display control unit 116 controls display according to the optical flow.

  FIG. 14 is a diagram illustrating a display example of the optical flow. Referring to FIG. 14, the optical flow display control unit 116 is configured so that the display unit 150 displays a set (motion information group M2) along the time series of motion vectors (optical flow) for each pixel. Is controlling. In the example shown in FIG. 14, the direction of the motion vector is indicated by the direction of the arrow, and the size of the motion vector is indicated by the size of the arrow. However, the direction and magnitude of the motion vector may be indicated by other modes. For example, the direction of the motion vector may be indicated by a difference in color type (for example, yellow, red, etc.), and the magnitude of the motion vector may be indicated by color shading.

  Further, as shown in FIG. 14, the optical flow display control unit 116 controls display of a display object (for example, a slider) for changing the determination parameter. The user can input a determination parameter changing operation by operating the display object. Note that the display object is not limited to the slider. For example, the display object may be an input field for directly inputting a determination parameter. In the example shown in FIG. 14, the first time range t21, the second time range t23, the threshold value, and the density are displayed as determination parameters.

  In the second embodiment, the user can change the determination parameter by changing the determination parameter while viewing the optical flow displayed in this manner. The second embodiment is useful when the user changes (adjusts) the determination parameter specified by the parameter specifying unit 112A. When the determination parameter changing operation is input by the user, the parameter changing unit 117 changes the determination parameter based on the changing operation. For example, as shown in FIG. 14, when the display of the OK button 151 is controlled by the optical flow display control unit 116, the determination parameter changing operation may be confirmed by an operation of selecting the OK button 151. .

  When the determination parameter is not changed, the state determination unit 113 may perform state determination (determination as to whether an abnormality has occurred in the captured moving image) using the determination parameter specified by the parameter specifying unit 112A. . On the other hand, when the parameter for determination is changed by the parameter changing unit 117, the state determination unit 113 uses the determination parameter after the change to perform state determination (determination as to whether an abnormality has occurred in the captured moving image). Just do it.

  The optical flow detected by the optical flow detection unit 115 may be recorded in the storage unit 140 and used later. Here, the optical flow recorded in the storage unit 140 may be used in any way. As an example, when an erroneous determination is made by the state determination unit 113, the parameter specifying unit 112A obtains a past optical flow recorded in the storage unit 140, and again determines a determination parameter based on the past optical flow. You may specify.

  Next, an operation example of changing the determination parameter in the image processing system 1B according to the second embodiment will be described. FIG. 15 is a flowchart illustrating an example of an operation for changing a determination parameter in the image processing system 1B according to the second embodiment. Note that the state determination operation example described below is merely an example of a determination parameter change operation in the image processing system 1B. Therefore, the determination parameter changing operation in the image processing system 1B is not limited to the determination parameter changing operation example described below.

  First, in the image processing apparatus 10B, the optical flow display control unit 116 acquires the determination parameter specified by the parameter specifying unit 112A (S31). Subsequently, the imaging device 30 captures a moving image and provides the captured moving image to the image processing device 10B. The moving image provided by the imaging device 30 is acquired as a captured moving image by the data acquisition unit 111 (S32). Subsequently, the optical flow detection unit 115 detects an optical flow from the captured moving image (S33).

  The optical flow display control unit 116 controls the display of the optical flow (S34). At this time, the optical flow display control unit 116 may control the display of the determination parameter specified by the parameter specifying unit 112A. Then, it is possible to make the user grasp the current determination parameter. When the determination parameter changing operation is detected by the operation unit 120, the parameter changing unit 117 changes the determination parameter based on the changing operation (S36).

  When the data acquisition unit 111 continues the operation (“No” in S37), the data acquisition unit 111 shifts the operation to S32. On the other hand, when the data acquisition unit 111 ends the operation (“Yes” in S37), the data acquisition unit 111 ends the operation.

  The second embodiment has been described above.

(3. Third embodiment)
Subsequently, a third embodiment will be described. As shown in FIG. 1, the image processing system 1C according to the third embodiment includes an image processing device 10C instead of the image processing device 10A, as compared with the image processing system 1A according to the first embodiment. . Therefore, in the following, description of the imaging device 30 is omitted, and the image processing device 10C is mainly described.

[3-1. Functional configuration example of image processing apparatus]
First, a functional configuration example of an image processing apparatus 10C according to the third embodiment will be described. FIG. 16 is a block diagram illustrating a functional configuration example of an image processing apparatus 10C according to the third embodiment. As illustrated in FIG. 16, the image processing apparatus 10C according to the third embodiment includes a control unit 110C instead of the control unit 110A, as compared with the image processing apparatus 10A according to the first embodiment. Therefore, below, description about the operation part 120, the communication part 130, the memory | storage part 140, and the display part 150 is abbreviate | omitted, and mainly demonstrates the difference with 110 A of control parts among 110 C of control parts.

  As illustrated in FIG. 16, the control unit 110C includes a data acquisition unit 111, a state determination unit 113, and a detection result display control unit 114, similarly to the control unit 110A. In addition, the control unit 110C includes a parameter specifying unit 112B, a parameter information recording control unit 118, and a parameter information display control unit 119. Details of these functional units included in the control unit 110C will be described later.

[3-2. detail of function]
Next, details of functions of the image processing system 1C will be described. Also in the third embodiment, as in the first embodiment, the data acquisition unit 111 acquires an input moving image. In the third embodiment, the parameter specifying unit 112B makes a determination based on the input moving image as in the first embodiment, when the parameters used in the past are not recorded in the storage unit 140. Specify the parameters. When the determination parameter specified by the parameter specifying unit 112B is used by the state determination unit 113, the parameter information recording control unit 118 uses the determination parameter used by the state determination unit 113 as a parameter used in the past. Control is performed so as to be recorded in the storage unit 140.

  The parameter information recording control unit 118 also controls information related to the determination parameters used by the state determination unit 113 to be recorded in the storage unit 140 as information related to parameters used in the past. It's okay.

  FIG. 17 is a diagram illustrating an example of parameters used in the past and information related to parameters used in the past. As shown in FIG. 17, information related to parameters used in the past may include the types of events that have been determined using parameters used in the past. The type of event may be the various events described above that can be determined by the state determination unit 113. In the example shown in FIG. 17, escape, reverse run, fallen object, and collapse are shown as the types of events.

  Further, as illustrated in FIG. 17, the information related to the parameter used in the past may include the angle of view of the imaging device 30 that captured the captured moving image used for the determination by the state determination unit 113. Further, as illustrated in FIG. 17, the information related to the parameters used in the past may include an imaging distance by the imaging device 30 (for example, the shortest distance from the imaging device 30 to the road).

  Here, the information related to the parameters used in the past may be set in any way. For example, the type of event may be set by the user, or may be automatically determined from the captured moving image by the state determination unit 113. The imaging distance may be set by the user, or may be automatically measured from the captured moving image by the state determination unit 113. Further, the angle of view may be acquired from the imaging device 30.

  When parameters used in the past are recorded in the storage unit 140, the parameter specifying unit 112B uses the parameters used in the past to generate an event corresponding to the parameters used in the past in the input moving image. It is determined whether or not.

  Then, when an input moving image (a normal input moving image) in which it has been confirmed that such an event has not occurred is acquired by the data acquisition unit 111, and parameters used in the past by the parameter specifying unit 112B When it is determined that an event has not occurred in the input moving image (the input moving image is normal), the parameters used in the past are considered to be parameters that are not erroneously determined. Therefore, in such a case, the parameter information display control unit 119 may control the display of at least one of the parameters used in the past and the information related to the parameters used in the past. As a result, the image processing system 1C according to the third embodiment can appropriately determine the occurrence of such an event even if parameters previously used for the input moving image acquired by the data acquisition unit 111 are used. Can be presented to the user.

  FIG. 18 is a diagram illustrating an example of determination results using parameters used in the past. In the example shown in FIG. 18, it is confirmed that each event does not occur in the input moving image acquired by the data acquisition unit 111 (the input moving image is normal). Further, the parameter specifying unit 112B determines that the event “evasion” has not occurred in the input moving image (the input moving image is normal) using the parameter P5 used in the past. Further, the parameter specifying unit 112B determines that the event “reverse running” has not occurred in the input moving image (the input moving image is normal) using the parameter P6 used in the past.

  On the other hand, the parameter specifying unit 112B determines that an event “falling object” has occurred in the input moving image (the input moving image is abnormal) using the parameter P7 used in the past. Further, it is determined by the parameter specifying unit 112B that the event “breakdown” has occurred in the input moving image (the input moving image is abnormal) using the parameter P8 used in the past.

  In such an example, the parameter P5 used in the past and the parameter P6 used in the past are considered to be parameters that are not erroneously determined. Therefore, in such a case, the parameter information display control unit 119 may control the display of at least one of the parameter P5 used in the past and the information related to the parameter P5 used in the past. The parameter information display control unit 119 may control display of at least one of the parameter P6 used in the past and the information related to the parameter P6 used in the past. As a result, the image processing system 1C according to the third embodiment can appropriately apply the event (runaway) even if the parameter P5 or P6 used in the past is used for the input moving image acquired by the data acquisition unit 111. Alternatively, it can be shown to the user that the occurrence of reverse running) can be determined.

  FIG. 19 is a diagram illustrating a display example of parameters used in the past and information related to the parameters used in the past. As shown in FIG. 19, the parameter information display control unit 119 controls the display of the parameter P5 used in the past and information related to the parameter P5 used in the past (type of event, imaging distance, angle of view). You can do it. Further, the parameter information display control unit 119 may control display of the parameter P6 used in the past and information related to the parameter P6 used in the past (type of event, imaging distance, angle of view).

  When a parameter selection operation used in the past is input, the parameter specifying unit 112B specifies a parameter used in the past as a determination parameter. For example, as shown in FIG. 19, when the display of the OK button 151 is controlled by the parameter information display control unit 119, the parameter selection operation used in the past is determined by the operation of selecting the OK button 151. May be. When the display of the cancel button 152 is controlled by the parameter information display control unit 119, the parameter selection operation used in the past may be canceled by an operation of selecting the cancel button 152.

  In addition, when an input moving image in which an event has been confirmed by the data acquisition unit 111 (the input moving image is abnormal) is acquired, and an event has occurred in the input moving image by the parameter specifying unit 112B If it is determined that the input moving image is abnormal (ie, the input moving image is abnormal), the parameters used in the past are considered to be parameters that are not erroneously determined. Therefore, in such a case, the parameter information display control unit 119 may control the display of at least one of the parameters used in the past and the information related to the parameters used in the past. As a result, the image processing system 1C according to the third embodiment can appropriately determine the occurrence of such an event even if parameters previously used for the input moving image acquired by the data acquisition unit 111 are used. Can be presented to the user. Then, when a parameter selection operation used in the past is input, the parameter specifying unit 112B may specify a parameter used in the past as a determination parameter.

  The third embodiment has been described above.

(4. Fourth embodiment)
Subsequently, a fourth embodiment will be described. As shown in FIG. 1, the image processing system 1D according to the fourth embodiment includes an image processing device 10D instead of the image processing device 10A, as compared with the image processing system 1A according to the first embodiment. . Therefore, in the following, description of the imaging device 30 is omitted, and the image processing device 10D is mainly described.

[4-1. Functional configuration example of image processing apparatus]
First, a functional configuration example of an image processing apparatus 10D according to the fourth embodiment will be described. FIG. 20 is a block diagram illustrating a functional configuration example of an image processing apparatus 10D according to the fourth embodiment. As illustrated in FIG. 20, the image processing device 10D according to the fourth embodiment includes a control unit 110D instead of the control unit 110A, as compared with the image processing device 10A according to the first embodiment. Therefore, below, description about the operation part 120, the communication part 130, the memory | storage part 140, and the display part 150 is abbreviate | omitted, and mainly demonstrates the difference with control part 110A among control part 110D.

  As illustrated in FIG. 20, the control unit 110D includes a data acquisition unit 111, a state determination unit 113, and a detection result display control unit 114, similarly to the control unit 110A. In addition, the control unit 110D includes a parameter specifying unit 112C and a region information acquisition unit 121. Details of these functional units included in the control unit 110D will be described later.

[4-2. detail of function]
Next, details of functions of the image processing system 1D will be described. In the fourth embodiment, the state determination unit 113 performs different state determination for each of the plurality of detection target regions in the input moving image (determines whether or not a different event occurs for each of the plurality of detection target regions). . In the following description, each of the plurality of detection target areas is simply referred to as “area”. In the storage unit 140, the correspondence between coordinates and event types is recorded for each region. Therefore, the area information acquisition unit 121 acquires the correspondence (area information) between the coordinates for each area and the type of event, and the parameter identification unit 112C determines the area based on the correspondence recorded in the storage unit 140. A different determination parameter is specified for each.

  FIG. 21 is a diagram illustrating an example of a correspondence relationship between coordinates for each region and the type of event. As shown in FIG. 21, the event type “reverse run” for which it is determined whether or not it occurs in the region R1 is associated with the coordinate “r1” of the region R1. In addition, the event type “runaway” in which it is determined whether or not it occurs in the region R2 is associated with the coordinate “r2” of the region R2. In the fourth embodiment, such a correspondence relationship is recorded in advance in the storage unit 140. In the example shown in FIG. 21, the region R1 and the region R2 are adjacent to each other, but the region R1 and the region R2 may be separated from each other or may partially overlap.

  As in the first embodiment, the data acquisition unit 111 acquires an input moving image. In the fourth embodiment, the parameter specifying unit 112C acquires the correspondence between the coordinates for each region and the event type. The parameter specifying unit 112C specifies a determination parameter for each region based on the coordinates and event types for each region. The determination parameter for each region may be specified in the same manner as the determination parameter according to the first embodiment. According to such a configuration, since determination parameters corresponding to different events for each region are specified, it is possible to determine with high accuracy whether or not a different event has occurred for each region.

  Subsequently, an operation example for determining a determination parameter in the image processing system 1D according to the fourth embodiment will be described. FIG. 22 is a flowchart illustrating an operation example of determination parameter identification in the image processing system 1D according to the fourth embodiment. Note that the determination parameter specifying operation example described below is merely an example of the determination parameter specifying operation in the image processing system 1D. Therefore, the determination parameter specifying operation in the image processing system 1D is not limited to the determination parameter specifying operation example described below.

  First, in the image processing apparatus 10D, an input moving image is acquired by the data acquisition unit 111 (S11). Subsequently, the parameter specifying unit 112C acquires the coordinates of the first region (S16). Then, the parameter specifying unit 112C acquires the first candidate parameter from the storage unit 140 (S12). The parameter specifying unit 112C obtains a determination result by determining whether or not an event in the region has occurred in the input moving image using the acquired candidate parameter.

  Subsequently, as long as the determination result does not change (“No” in S13), the parameter specifying unit 112C acquires the next candidate parameter (S14), and using the acquired candidate parameter, the event in the region is included in the input moving image. By determining whether or not it has occurred, a determination result is obtained, and the operation proceeds to S13. On the other hand, when the determination result changes (“Yes” in S13), the parameter specifying unit 112C specifies the determination parameter for the region based on the candidate parameter when the determination result changes (S17).

  The parameter specifying unit 112C acquires the coordinates of the next region (S19), unless specifying the determination parameters for the entire region (“No” in S18), and uses the acquired candidate parameters for the input moving image. By determining whether or not an event has occurred, a determination result is obtained, and the operation proceeds to S13. On the other hand, when the parameter specifying unit 112C specifies the determination parameter for the entire region (“Yes” in S18), the parameter specifying unit 112C ends the determination parameter specifying operation.

  The fourth embodiment has been described above.

(5. Hardware configuration example)
Next, a hardware configuration example of the image processing apparatus 10 according to the embodiment of the present invention will be described.

  FIG. 23 is a diagram illustrating a hardware configuration of the image processing apparatus 10 according to the embodiment of the present invention. The image processing apparatus 10 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, , An input device 908, an output device 909, a storage device 910, and a communication device 911.

  The CPU 901 functions as an arithmetic processing device and a control device, and controls the overall operation in the image processing device 10 according to various programs. Further, the CPU 901 may be a microprocessor. The ROM 902 stores programs used by the CPU 901, calculation parameters, and the like. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 904 including a CPU bus or the like.

  The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905. Note that the host bus 904, the bridge 905, and the external bus 906 are not necessarily configured separately, and these functions may be mounted on one bus.

  The input device 908 includes an input means for inputting information such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever, and an input control circuit that generates an input signal based on the input by the user and outputs the input signal to the CPU 901. Etc. A user who operates the image processing apparatus 10 can input various data and instruct a processing operation to the image processing apparatus 10 by operating the input device 908.

  The output device 909 includes, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a display device such as a lamp, and an audio output device such as a speaker.

  The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 910 is configured with, for example, an HDD (Hard Disk Drive). The storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data.

  The communication device 911 is a communication interface configured with, for example, a communication device for connecting to a network. Further, the communication device 911 may support either wireless communication or wired communication.

  The details of the embodiment of the present invention have been described above.

(5. Effects of the present embodiment)
As described above, according to the present embodiment, the data acquisition unit 111 that acquires a plurality of candidate parameters and input moving images arranged in ascending or descending order from the beginning to the end, and the plurality of candidate parameter Using in order from the beginning, it repeatedly determines whether or not a predetermined event has occurred in the input moving image until at least the determination result changes, and an event has occurred in the captured moving image based on the candidate parameter whose determination result changes An image processing apparatus 10A is provided that includes a parameter specifying unit 112A that specifies a determination parameter for determining whether or not the image is determined.

  According to such a configuration, it is possible to specify a determination parameter such that the condition for abnormality determination is not too loose while reducing the possibility of erroneous determination by the state determination unit 113. Therefore, according to such a configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred in the captured moving image.

(6. Description of modification)
The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the above description, the imaging apparatus 30 may be a PTZ (pan / tilt / zoom) camera. However, if the imaging range of the imaging device 30 changes due to PTZ, it may be determined that an abnormal state has occurred in the detection target region at an unintended timing. Therefore, when the PTZ is executed, it is preferable that the determination by the state determination unit 113 is not executed. Alternatively, when the PTZ is executed, display control by the detection result display control unit 114 is preferably not performed.

1 (1A, 1B, 1C, 1D) Image processing system 10 (10A, 10B, 10C, 10D) Image processing device 30 Imaging device 110 (110A, 110B, 110C, 110D) Control unit 111 Data acquisition unit 112 (112A, 112B) 112C) Parameter identification unit 113 State determination unit 114 Detection result display control unit 115 Optical flow detection unit 116 Optical flow display control unit 117 Parameter change unit 118 Parameter information recording control unit 119 Parameter information display control unit 121 Area information acquisition unit 120 Operation Unit 130 communication unit 140 storage unit 150 display unit

Claims (16)

A data acquisition unit for acquiring a plurality of candidate parameters and input moving images arranged in ascending or descending order from the beginning to the end;
Using the plurality of candidate parameters in order from the top, it is repeatedly determined whether or not a predetermined event has occurred in the input moving image until at least the determination result changes, and based on the candidate parameter that changes the determination result A parameter specifying unit for specifying a determination parameter for determining whether or not the event has occurred in the captured moving image;
An image processing apparatus comprising: The parameter specifying unit includes:
When the input moving image in which the event has not occurred is acquired by the data acquisition unit,
Candidate parameters that are no longer determined that the event has occurred in the input moving image, or candidate parameters immediately before the candidate parameters that have started to be determined that the event has occurred in the input moving image, Specified as the determination parameter,
When the input moving image in which the event occurs is acquired by the data acquisition unit,
Candidate parameters immediately before a candidate parameter that is no longer determined that the event has occurred in the input moving image, or candidate parameters that have started to be determined that the event has occurred in the input moving image, Specify as the determination parameter,
The image processing apparatus according to claim 1. The parameter specifying unit includes:
When the input moving image in which the event has not occurred is acquired by the data acquisition unit,
Candidate parameters that are no longer determined that the event has occurred in the input moving image, or candidate parameters immediately before the candidate parameters that have started to be determined that the event has occurred in the input moving image, Specify as the determination parameter,
The image processing apparatus according to claim 1. The parameter specifying unit includes:
When the input moving image in which the event has not occurred is acquired by the data acquisition unit,
A candidate parameter that is no longer determined to be the event in the input moving image is specified as the determination parameter.
The image processing apparatus according to claim 3. The parameter specifying unit includes:
When the input moving image in which the event has not occurred is acquired by the data acquisition unit,
The candidate parameter immediately before the candidate parameter that has started to be determined that the event has occurred in the input moving image is specified as the determination parameter.
The image processing apparatus according to claim 3. The image processing apparatus includes:
A state determination unit that determines whether or not the event has occurred in the captured moving image using the determination parameter;
The image processing apparatus according to claim 1. The image processing apparatus includes:
When it is determined that the event has occurred in the captured moving image, a detection result display control unit that controls display of the detection result of the event is provided.
The image processing apparatus according to claim 1. The parameter specifying unit includes:
Based on the input moving image, an average value for each corresponding position of the optical flow in the first time range is calculated as the first average feature amount, and the optical value in the second time range shorter than the first time range is calculated. An average value for each corresponding position of the flow is calculated as a second average feature quantity, and based on a relationship between a predetermined calculation result corresponding to the first average feature quantity and the second average feature quantity and a threshold value. Determining whether the event has occurred in the input moving image;
The image processing apparatus according to claim 1. The determination parameter includes at least one of the first time range, the second time range, and the threshold value.
The image processing apparatus according to claim 8. The determination parameter includes at least one of a temporal density at which the optical flow is calculated and a spatial density at which the optical flow is calculated.
The image processing apparatus according to claim 8. The image processing apparatus includes:
An optical flow display control unit for controlling display according to the optical flow detected based on the captured moving image;
When the determination parameter changing operation is input, a parameter changing unit that changes the determination parameter;
The image processing apparatus according to claim 1, comprising: The parameter specifying unit includes:
When parameters used in the past are recorded, it is determined whether an event corresponding to the parameters used in the past has occurred in the input moving image using the parameters used in the past,
The image processing apparatus includes:
When the input moving image in which the event has not occurred is acquired by the data acquisition unit and the event is determined by the parameter specifying unit that the event has not occurred, or
When the input moving image in which the event occurs is acquired by the data acquisition unit and when the event is determined in the input moving image by the parameter specifying unit,
A parameter information display control unit configured to control display of at least one of the parameters used in the past and the information related to the parameters used in the past;
The image processing apparatus according to claim 1. The information related to the parameters used in the past includes the types of events determined using the parameters used in the past, the imaging distance by the imaging device that captured the moving image used for the determination, and Including at least one of the angles of view of the imaging device,
The image processing apparatus according to claim 12. The parameter specifying unit includes:
When the selection operation of the parameter used in the past is input, the parameter used in the past is specified as the determination parameter.
The image processing apparatus according to claim 12. The parameter specifying unit includes:
Based on the event for each area provided on the input moving image, the determination parameter is specified for each area;
The image processing apparatus according to claim 1. Computer
A data acquisition unit for acquiring a plurality of candidate parameters and input moving images arranged in ascending or descending order from the beginning to the end;
Using the plurality of candidate parameters in order from the top, it is repeatedly determined whether or not a predetermined event has occurred in the input moving image until at least the determination result changes, and based on the candidate parameter that changes the determination result A parameter specifying unit for specifying a determination parameter for determining whether or not the event has occurred in the captured moving image;
A program for causing an image processing apparatus to function.

Images