JP7084795B2 - Image processing equipment, image providing equipment, their control methods and programs - Google Patents

Description

The present invention relates to an image processing device, an image providing device, a control method and a program thereof.

In recent years, surveillance cameras have been widely installed, and techniques for protecting the privacy of individuals reflected in images taken by surveillance cameras have been proposed.

For example, Patent Document 1 discloses a technique of extracting a human body region in an image from a background image and performing a blurring process. Further, Patent Document 2 discloses a technique of extracting a face region in an image by a face detection process and displaying only a specific person by collating it with a face image registered in advance.

Japanese Unexamined Patent Publication No. 2008-191884 Japanese Unexamined Patent Publication No. 2004-06256

In a recording management system equipped with a function to record images from a surveillance camera, there is a need to protect privacy by individually setting whether or not mask processing is applied according to the individual objects reflected in the recorded images and the user who views them. There is. In the above-mentioned conventional technique, although it is possible to apply privacy protection processing to a photographed image for a predetermined target, it is difficult to flexibly set privacy protection for each object of the recorded image.

An object of the present invention is to make it possible to flexibly perform mask processing on a captured image.

The image processing apparatus according to one aspect of the present invention is
Image acquisition means to acquire captured images and
An information acquisition means for acquiring analysis data including flag information indicating whether or not an object existing in the captured image is a mask target, which is recorded corresponding to the captured image.
A detection means for detecting an object from the captured image and
Among the objects detected from the captured image, the mask processing means for generating an image in which the object indicated to be the mask target by the flag information is masked is provided.

According to the present invention, it is possible to flexibly perform mask processing on a recorded photographed image.

The block diagram which showed the surveillance camera system 10. The block diagram which showed the network camera 100 and the network storage apparatus 300 in the surveillance camera system 10. The figure which shows the relationship between an image file and an analysis data file. The block diagram which showed the image display apparatus 400 of the surveillance camera system 10. The flowchart which shows the operation of the image display device 400. The figure which showed the correspondence between the image and the metadata in a certain frame. It is a figure explaining the generation of a privacy protection image. The figure which showed the operation state in a client terminal. The figure which showed the correspondence between the image and the metadata in a certain frame. The figure which showed the operation state in a client terminal. The block diagram which shows an example of the hardware composition of the image display device 400.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown.

<Embodiment 1>
FIG. 1 is a configuration diagram showing a surveillance camera system 10. The surveillance camera system 10 is an example of an image processing system, and includes a network camera 100, a management server 200, a network storage device 300, an image display device 400, and a LAN 500 which is a network line.

The network camera 100 is a photographing device that photographs a predetermined area, and is installed in, for example, a retail store to photograph the inside of the store. Further, the network camera 100 has a function of performing image analysis processing on captured images (moving images and still images) and generating analysis data, for example, moving object detection, moving object tracking, human body detection, face recognition, and an object. Perform detection and so on.

The management server 200 collects images and analysis data taken by a plurality of network cameras 100 installed in the entire predetermined area. The management server 200 also collects past images and analysis data recorded in the network storage device 300, which will be described later. The management server 200 manages the image information over the entire predetermined area by using the data collected in this way. The management server 200 may be equipped with an image analysis processing function. In this case, the function of generating analysis data in the network camera 100 may be omitted. Further, for example, the management server 200 may perform more advanced image analysis processing on an image that could not be analyzed by the image analysis processing by the network camera 100, and generate analysis data.

The network storage device 300 is a recording device. The image taken by the network camera 100 and the analysis data processed by the image analysis are recorded in the network storage device 300 via the LAN 500. The management server 200 functions as an image providing device that provides image data and analysis data to the image display device 400 in cooperation with the network camera 100 and the network storage device 300.

The image display device 400 as an image processing device displays images based on the images and analysis data recorded in the network storage device 300. The image display device 400 also displays image information managed by the management server 200 over the entire predetermined area. Further, the image display device 400 has a function of an input device for performing an image search operation such as an event scene from the analysis data obtained by the image analysis process. As the image display device 400, for example, a personal computer (PC) or the like is assumed. Further, the physical connection form of each device to the LAN 500 may be wired or wireless, and there is no limitation on the physical form as long as it is connected by protocol. For example, a tablet terminal wirelessly connected to the LAN 500 may be used as the image display device 400.

Further, the number of network cameras 100 may be one or more, and there is no upper limit. Further, the number of the management server 200, the network storage device 300, and the image display device 400 connected to the LAN 500 is not limited to the example shown in FIG. 1, and there are a plurality of each if they can be identified by an address or the like. It doesn't matter.

FIG. 2 is a block diagram showing a functional configuration example of the network camera 100 and the network storage device 300 in the surveillance camera system 10. The network camera 100 includes an image pickup sensor unit 101, a development processing unit 102, an image coding unit 103, an image analysis unit 104, and a LAN I / F unit 105.

The image pickup sensor unit 101 is an image pickup device such as CMOS that converts an optical image formed on an image pickup surface into a digital electric signal by photoelectric conversion. The development processing unit 102 performs predetermined pixel interpolation and color conversion processing on the digital electric signal obtained by photoelectric conversion from the image pickup sensor unit 101, and a digital image such as RGB or YUV (hereinafter, “image” is referred to as “image”). Also includes the concept of digital images). Further, the development processing unit 102 performs predetermined arithmetic processing using the developed image, and performs image processing such as white balance, sharpness, contrast, and color conversion based on the obtained arithmetic result. ..

The image coding unit 103 applies compression coding, frame rate setting, and the like to the image from the development processing unit 102 to generate an image for distribution. The compression method here is, for example, MPEG4, H.M. Based on standards such as 264, MJPEG or JPEG. Furthermore, it also files images such as mp4 and move formats.

The image analysis unit 104 performs object detection and recognition processing on the image from the development processing unit 102, and assigns a unique ID to the object identified from the positional relationship between the frames of the detected and recognized object. By doing so, the tracking process of the object is performed. Hereinafter, the object is also referred to as an object. The image analysis unit 104 discriminates attributes such as a person (male, female), an animal, an immovable body, and the like from the pattern of the shape feature and the behavioral feature, respectively, for the object. Furthermore, for each object, flag information as to whether or not it is a mask target (object for anonymization), that is, whether or not to protect privacy is also given. Then, metadata in which position information, attributes, flag information, etc. for each object are described is created as analysis data for each frame. In the present embodiment, the mask target is a target for processing so that a specific individual or object cannot be identified on the image.

Further, the position information of the object is information that can specify the position where the object exists. For example, the position information of an object includes information on the center coordinates of the object in the image. Further, the method for discriminating the attribute of an object is not limited to the above, and any method can be used as long as the attribute can be discriminated, such as discriminating the attribute by a specific object detection process using an edge feature. May be good.

Further, in the image analysis unit 104, the flag information for privacy protection may be set to either ON or OFF by default. Alternatively, for example, the flag information is automatically set according to the attribute of the object, for example, the flag information is set to ON if the attribute is "person", and the flag information is set to OFF if the attribute is other attributes. It may be configured to be. In either case, the state of the flag information can be updated by the user's instruction on the management server 200. For example, by playing back an image on the management server 200 and individually specifying whether or not the user is subject to privacy protection for the displayed object, the flag information of the object is subsequently turned OFF or ON. It is configured to be in a state. Alternatively, the state of the flag information may be changed from the specific image display device 400. As described above, according to the present embodiment, since the target of privacy protection can be set by managing the flag information, flexible privacy protection for the captured image can be realized.

The LAN I / F unit 105 cooperates with the LAN I / F unit 301 of the network storage device 300 to construct a network file system such as NFS or CIFS, for example.

The network storage device 300 includes a LAN I / F unit 301, an image storage unit 302, and an analysis data storage unit 303.

The LAN I / F unit 301 constructs a network file system in cooperation with the LAN I / F unit 105 of the network camera 100 or the LAN I / F unit 401 of the image display device 400.

The image storage unit 302 records an image file encoded and filed by the image coding unit 103 of the network camera 100 by the network file system. The analysis data storage unit 303 records the analysis data file filed by the image analysis process performed by the image analysis unit 104 of the network camera 100 by the network file system.

FIG. 3 is a diagram showing the relationship between the image file and the analysis data file. The image file is a file in which the image is encoded by the image coding unit 103 of the network camera 100. The analysis data file is a file of analysis data obtained by image analysis processing in the image analysis unit 104 of the network camera 100. As described above, for each frame, analysis data including the position information, attributes, and flag information about privacy protection of the detected and recognized objects is generated. The analysis data is metadata in a description format such as XML, and is held as an analysis data file.

In FIG. 3, for example, the metadata (Metadata [M]) corresponding to the image (Frame [M]) in which the objects ID1 (person) and ID2 (car) to which unique IDs are assigned are shown is each object. The position information (for example, center coordinates) and attributes (Human, car) of are described. Further, in the metadata, flag information about privacy protection of each object (in the illustrated example, "Privacy (ON)" which means privacy protection for ID1 and "Privacy" which means that privacy protection is not performed for ID2. (OFF) ") is described.

FIG. 4 is a block diagram showing a functional configuration example of the image display device 400 in the surveillance camera system 10. The image display device 400 includes a LAN I / F unit 401, an image decoding unit 402, an analysis data acquisition unit 403, an object extraction unit 404, a background image update unit 405, an image comparison unit 406, and an image composition unit. 407 and an output unit 408 are included.

The LAN I / F unit 401 constructs a network file system in cooperation with the LAN I / F unit 301 of the network storage device 300.

The image decoding unit 402 decompresses and decodes the image file acquired by the network file system from the image storage unit 302 of the network storage device 300, and outputs the image file to the object extraction unit 404, the background image update unit 405, and the image comparison unit 406. do. As described above, the LAN I / F unit 401 and the image decoding unit 402 of the present embodiment are one configuration example of the captured image acquisition unit that acquires the captured image by the network camera 100.

The analysis data acquisition unit 403 acquires the analysis data file recorded in the analysis data storage unit 303 of the network storage device 300 by the network file system and outputs it to the object extraction unit 404. As described above, the analysis data includes flag information, and the analysis data acquisition unit 403 of the present embodiment acquires flag information indicating whether or not an object existing in the captured image is a mask target. This is an example of the configuration of the information acquisition unit.

The object extraction unit 404 extracts an object from the image (also referred to as a decoded image) generated by the image decoding unit 402. The object extraction unit 404 extracts an object existing in the decoded image by detecting a moving object using the background subtraction method and detecting a person or the like by pattern matching. Then, the object extraction unit 404 identifies whether or not the detected object is an object for privacy protection based on the analysis data input from the analysis data acquisition unit 403. The identification of the object subject to privacy protection is performed based on the position information and the flag information of the object described in the metadata corresponding to the image (frame). The object extraction unit 404 outputs object information to the background image update unit 405 each time the image is processed. The object information includes the position information and the shape information of the extracted object.

In the background image updating unit 405, the object subject to privacy protection is obtained from the decoded image from the image decoding unit 402, the object information from the object extraction unit 404, and the protected area ratio information (described later) from the image comparison unit 406. Create an erased background image. Then, the created background image is output to the image comparison unit 406 and the image composition unit 407.

The image comparison unit 406 compares the decoded image from the image decoding unit 402 with the background image output from the background image update unit 405 to obtain a difference image, and creates a protected image based on the difference image. For example, the luminance difference (or color difference) is obtained for each pixel of the decoded image and the background image, the luminance difference is binarized with a specific threshold value, and the extracted region is used as a protected region. The protected area is an area where it is assumed that an object whose privacy is to be protected exists. Then, a protected image such as an image (silhouette image) in which the protected area is filled with a predetermined color (for example, black) is created. Instead of the silhouette image, an image may be created in which the extrinsic rectangular area of the object is filled with a predetermined color (for example, black) so as to hide the object whose privacy is to be protected, so that the object cannot be identified. You may create a mosaic-processed image.

Further, the image comparison unit 406 creates the protection area ratio information indicating the ratio of the area of the protection area to the area of the image and outputs it to the background image update unit 405. Based on this information, the background image updating unit 405 determines when the lighting conditions suddenly change, or when the shooting range of the imaging unit changes due to control of pan, tilt, zoom, etc., and efficiently backgrounds. The image can be updated.

The image composition unit 407 synthesizes the protected image created by the image comparison unit 406 and the background image output from the background image update unit 405, and provides the obtained composite image to the output unit 408. As described above, the object extraction unit 404, the background image update unit 405, the image comparison unit 406, and the image composition unit 407 of the present embodiment perform mask processing for masking the object to be masked in the captured image based on the flag information. Functions as an image processing unit. The output unit 408 displays the composite image created by the image composition unit 407 on the display unit. Thereby, the masked photographed image can be provided to the user.

FIG. 10 is a block diagram showing an example of the hardware configuration of the image display device 400. Devices such as the network camera 100, the management server 200, and the network storage device 300 may also have the hardware configuration shown in FIG.

The image display device 400 includes a CPU 1001, a ROM 1002, a RAM 1003, an auxiliary storage device 1004, a display unit 1005, an operation unit 1006, a communication unit 1007, and a bus 1008.

The CPU 1001 controls the entire image display device 400 by using computer programs and data stored in the ROM 1002 and the RAM 1003. That is, when the CPU 1001 operates according to the program stored in the ROM 1002 or the like, the function of the image display device 400 as described in FIG. 4 is realized. Similarly, the functions of the network camera 100, the management server 200, the network storage device 300, and the like are also realized by the CPU that operates according to the program.

The ROM 1002 stores programs and parameters that do not need to be changed. The RAM 1003 temporarily stores programs and data supplied from the auxiliary storage device 1004, data supplied from the outside via the communication unit 1007, and the like. The auxiliary storage device 1004 is composed of, for example, a hard disk drive or the like.

The display unit 1005 is composed of, for example, a liquid crystal display or the like, and displays a GUI (Graphical User Interface) for operating the image display device 400 and the like. The operation unit 1006 is composed of, for example, a keyboard, a mouse, or the like, and inputs various instructions to the CPU 1001 in response to an operation by the user. The communication unit 1007 communicates with an external device such as the network camera 100 and the management server 200. The bus 1008 connects each component of the image display device 400 to transmit information.

Next, the details of the operation of the surveillance camera system 10 according to the first embodiment will be described with the above configuration.

FIG. 5 is a flowchart showing the operation of the image display device 400. In the surveillance camera system 10, the image taken by the network camera 100 is subjected to image analysis processing and recorded in the network storage device 300 as the image file and the analysis data file described with reference to FIG. The image display device 400 reads an image file and an analysis data file from the network storage device 300 and reproduces the image. In the present embodiment, the recording (accumulation) and reading of the image file and the analysis data file to the network storage device 300 is performed via the management server 200. The following operations are executed by the CPU 1001 according to the program stored in the ROM 1002 of the image display device 400.

In step S1, the image decoding unit 402 decodes the image read from the image storage unit 302 of the network storage device 300 to generate the decoded image 501. Further, the analysis data acquisition unit 403 reads the analysis data file from the analysis data storage unit 303 while synchronizing with the reading of the image file, and acquires the analysis data 502.

In step S2, the object extraction unit 404 detects an object from the decoded image 501 by using, for example, a background subtraction method and a pattern matching method. Based on the analysis data 502, the object extraction unit 404 identifies the object whose privacy is to be protected and the object which is not the protection target for each frame, and generates the object information 503. The object extraction unit 404 outputs the generated object information to the background image update unit 405.

In step S3, the background image updating unit 405 updates the background image based on the decoded image 501 and the object information 503, and sets the image in which the object whose privacy is to be protected is deleted from the input image as the background image 504. Generate.

In step S4, the image comparison unit 406 compares the decoded image 501 input from the image decoding unit 402 with the background image 504 updated by the background image update unit 405 to obtain a difference image 505. Then, a silhouette image 506 is created as a protective image based on the difference image 505. The silhouette image 506 shows an object in which the privacy protection flag information is ON.

In step S5, the image synthesizing unit 407 creates a composite image 507 by synthesizing the background image 504 generated by the background image updating unit 405 in step S3 and the silhouette image 506 generated by the image comparison unit 406 in step S4. , Output to the output unit 408. As described above, the protected image (silhouette image) is created by silhouettening the protected area determined based on the difference image 505 between the decoded image 501, which is a captured image, and the background image 504 in which the object to be protected for privacy is erased. 506) is generated. Then, the protected image and the background image are combined to obtain a composite image 507 in which the object subject to privacy protection is masked.

In step S6, the control unit (not shown) that controls the entire image display device 400 determines whether or not to end the above processing. If it is determined that the process does not end (NO in step S6), the process returns to step S1. When it is determined that the control unit ends (YES in step S6), such as when the processing has been completed for all the frames of the read image file, the processing ends.

FIG. 6A is a diagram showing the correspondence between the image (FIG. 6A (a)) and the metadata (FIG. 6A (b)) in a certain frame n. FIG. 6A is an image of the frame N, and the objects IDs 1 to 4 are shown. FIG. 6A (b) is metadata created by the image analysis unit 104 of the network camera 100 by analyzing the image of the frame n. Here, the flag information for privacy protection is ON (that is, the privacy is protected) for the object ID 1, but the flag information is OFF for the other objects.

FIG. 6B is a diagram illustrating a specific example of generation of a privacy protection image according to the present embodiment. FIG. 6B (a) is a decoded image 501 of the frame N decoded by the image decoding unit 402. The object extraction unit 404 detects an object from the decoded image 501 by a background subtraction method, a pattern matching method, or the like. Then, based on the position information included in the analysis data, the flag information is applied to the object detected from the decoded image 501, and the object information 503 is obtained. This situation is shown in FIG. 6B (b). In FIG. 6B (b), object information including identification information, position information, shape information, and flag information is obtained for the four extracted objects. The shape information is a rectangle that surrounds each object, but is not limited to this. For example, when an object is extracted by the background subtraction method, the shape information may be composed of a set of pixels in the extracted object.

FIG. 6B (c) is a background image generated by the background image updating unit 405 based on the decoded image sequentially acquired from the image decoding unit 402 in chronological order and the object detection result by the object extraction unit 404. For example, the background image updating unit 405 divides the image into a plurality of blocks, and measures the time (stable background time) in which no change occurs for each block. While measuring the stable background time for each block, the blocks containing the extracted object area are reset to zero stable background time. The background image updating unit 405 uses the image in the block whose stable background time exceeds a predetermined value as the image of the corresponding block of the decoded image 501 (FIG. 6B (a)) acquired from the image decoding unit 402. By updating, the background image is updated. For the update, the image of the block in the image background image may be replaced with the image of the corresponding block of the decoded image 501, and the added average or the weighted average of the images of both blocks may be used.

In the present embodiment, the background image update unit 405 further arranges the objects whose flag information is OFF among the objects extracted by the object extraction unit 404 on the background image of FIG. 6B (c). In the example of FIG. 6B (b), the flag information is OFF for ID2 to ID4, and these images are arranged in the background image shown in FIG. 6B (c). FIG. 6B (d) shows the background image 504 as a result. In this way, the background image updating unit 405 obtains a background image 504 (FIG. 6B (d)) in which the object whose privacy is to be protected is erased from the input image.

The image comparison unit 406 obtains a difference between the decoded image 501 (FIG. 6B (a)) generated by the image decoding unit 402 and the background image 504 (FIG. 6B (d)) generated by the background image updating unit 405. , The difference image 505 (FIG. 6B (e)) is acquired and silhouetted to obtain the silhouette image 506 (FIG. 6B (f)). The image composition unit 407 synthesizes the silhouette image 506 (FIG. 6B (f)) and the background image 504 (FIG. 6B (d)) generated by the background image update unit 405, and the composite image 507 (FIG. 6B (g)). To generate. In this way, an image in which an object in which the flag information indicating privacy protection is turned on (that is, an object to be masked) is silhouetted can be obtained.

FIG. 7 is a diagram showing an operating state of the surveillance camera system 10 when the metadata as shown in FIG. 6A (b) is added. In FIG. 6A (b), a flag is added to protect the privacy of only the person with the object ID 1. Therefore, when the recorded image is reproduced on the client terminal (image display device 400), the object ID 1 is displayed as a silhouette on all the client terminals. Since the flag information included in the analysis data can be set for each object, the privacy protection setting (mask setting) can be flexibly set.

In the first embodiment, the background image and the silhouette image are combined in the image composition unit 407, but the composition of the silhouette image is not limited to the background image. Any image may be used, such as a one-sided black image or a fixed background image prepared in advance. The output destination of the output unit 408 is not limited to the display device, and may be another device such as another network or a storage device.

<Embodiment 2>
In the first embodiment, object information including flag information indicating whether or not to protect privacy (whether or not to be a mask target) is set for each object existing in the captured image. In the first embodiment, the object information includes one flag information, but the present invention is not limited to this, and the object information may include two or more flag information. In the second embodiment, the privacy protection flag information described in the metadata of the analysis data file is set for each user group of users who perform the image reproduction (viewing) operation. This makes it possible to implement different privacy protection processing functions for each user group.

FIG. 8 is a diagram showing the correspondence between the image (FIG. 8 (a)) and the metadata (FIG. 8 (b)) in a certain frame N. The privacy protection flag information described in the metadata shown in FIG. 8 (b) is different from that in FIG. 6A (b).

Here, for each object, a flag is given to the user group A (Gr.A) and the user group B (Gr.B) as to whether or not to protect the privacy. For example, the person with the object ID 1 is flagged with no privacy protection (OFF) for the user group A and privacy protection (ON) for the user group B. On the other hand, the person with the object ID 3 is flagged as privacy-protected (ON) for both the user groups A and B.

FIG. 9 is a diagram showing an operating state corresponding to FIGS. 8 (a) and 8 (b). In the image display device 400a belonging to the user group A, the person with the object ID 3 is silhouetted to protect privacy, but the person with the object ID 1 is not silhouetted and is displayed as it is. On the other hand, in the image display device 400b belonging to the user group B, both the person of the object ID 1 and the person of the ID 3 are displayed in silhouette. As described above, in the second embodiment, the privacy protection target can be different for each user or user group, and the image can be reproduced.

In the above, the privacy protection flag is set for each user group, but the present invention is not limited to this. For example, the privacy protection flag may be set for each user, for each terminal that reproduces an image, or for each terminal location. This makes it possible to control the privacy protection display according to the installation location of the terminal, such as a clean room, police mechanism, security, etc., from a general location, etc., according to various installation locations of the terminal that reproduces the image. ..

<Other embodiments>
Although the embodiments have been described in detail above, the present invention can be implemented as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an image pickup device, a web application, etc.), or it may be applied to a device consisting of one device. good.

The present invention also supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads the program. It can also be realized by the processing to be executed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

10 Surveillance camera system, 100 network camera, 101 image sensor unit, 102 development processing unit, 103 image coding unit, 104 image analysis processing unit, 105, 301, 401 LAN I / F unit, 200 surveillance camera system management server, 300 Network storage device, 302 image storage unit, 303 analysis data storage unit, 400 image information display device, 402 image decoding unit, 403 analysis data acquisition unit, 404 object extraction unit, 405 background image update unit, 406 image comparison unit, 407 Image composition unit, 408 output unit, 500 LAN

Claims (16)

Image acquisition means to acquire captured images and
An information acquisition means for acquiring analysis data including flag information indicating whether or not an object existing in the captured image is a mask target, which is recorded corresponding to the captured image.
A detection means for detecting an object from the captured image and
An image processing apparatus comprising: a mask processing means for generating an image that masks an object that is indicated to be a mask target by the flag information among the objects detected from the captured image. The analysis data includes position information indicating the position of an object in the captured image.
The image processing apparatus according to claim 1, wherein the mask processing means applies the flag information to an object whose position is detected by the detection means and whose position corresponds to the position information. The mask processing means is
A generation means for generating a first background image in which an object to be masked does not exist among the objects detected by the detection means, and a generation means.
The image processing apparatus according to claim 1 or 2, further comprising a compositing means for synthesizing a masked image of the object to be masked with the first background image. The generation means uses the captured images sequentially acquired and the second background image generated based on the detection result by the detection means to display an object detected by the detection means that is not a mask target. The image processing apparatus according to claim 3, wherein the first background image is generated by arranging the image. The synthesizing means generates a protected image by masking a region of the object to be masked, which is determined based on a difference image between the captured image and the first background image, and the protected image and the first background image. The image processing apparatus according to claim 3 or 4, wherein the background image of the above is synthesized. The image processing apparatus according to claim 5, wherein the area of the object to be masked is silhouetted or mosaiced in the protected image. The image processing apparatus according to claim 5, wherein in the protected image, a rectangular area surrounding the area of the object to be masked is silhouetted or mosaiced. The image acquisition means acquires a captured image for each frame constituting the moving image, and obtains the captured image.
The image processing apparatus according to any one of claims 1 to 7, wherein the information acquisition means acquires the analysis data in synchronization with the frame. The image processing apparatus according to any one of claims 1 to 8, wherein the flag information is given to each object existing in the captured image. The flag information is used for each user who views the captured image, for each user group of users who view the captured image, for each image display device that displays the captured image, or for displaying the captured image. The image processing device according to claim 9, wherein the image processing device is provided for each installation location of the device. Acquisition means for acquiring captured images taken by the shooting means, and
A means for creating object information including flag information indicating whether or not an object is a mask target for an object detected from a captured image acquired by the acquisition means, and a means for creating the object.
An image providing device comprising: an image file in which the captured image is encoded and a providing means for providing an analysis data file including the object information to an external device. The image providing device according to claim 11, wherein the creating means determines whether or not the object is a mask target based on the attributes of the detected object. The image providing device according to claim 11, wherein the creating means determines whether or not the detected object is a mask target in response to an instruction from a user. The image acquisition process to acquire the captured image and
An information acquisition step of acquiring analysis data including flag information indicating whether or not an object existing in the captured image is a mask target, which is recorded corresponding to the captured image.
The detection step of detecting an object from the captured image and
A control of an image processing apparatus comprising: Method. The acquisition process to acquire the captured image taken by the photographing means,
A creation step of creating object information including flag information indicating whether or not the object is a mask target for the object detected from the captured image acquired in the acquisition step, and a creation step.
A control method for an image providing device, comprising: a providing step of providing an image file in which the captured image is encoded and an analysis data file including the object information to an external device. A program for causing a computer to execute the control method according to claim 14 or 15.

Images