JP6099026B1 - MONITORING SYSTEM, VIDEO DISPLAY METHOD, AND COMPUTER PROGRAM - Google Patents

Abstract

[PROBLEMS] To effectively perform mask processing by suppressing the non-display of a selected captured image even when the captured image of a monitoring camera to be displayed on a monitor is switched, and to improve the efficiency and marketing of a user. Improve analysis accuracy. An information processing apparatus generates a background image of each captured video distributed from each camera, stores the background image in a storage unit, and appears in a captured video or captured video of an imaging area distributed from a first camera. A first edited video obtained by performing mask processing for changing the moving object to a mask image is displayed on the monitor. The information processing apparatus acquires a background image of the captured video distributed from the selected second camera in response to switching from the first camera to the second camera, and captures the background image and the imaging distributed from the second camera. Using the video, the second edited video obtained by performing mask processing on the captured video delivered from the second camera is displayed on the monitor. [Selection] Figure 31

Description

The present invention relates to an edited video after mask processing for changing an image area of a moving body (for example, a person) detected in a captured image of a monitoring area to a mask image for making it impossible to specify who the moving body is. The present invention relates to a monitoring system , a video display method, and a computer program .

  In stores such as convenience stores, cameras that shoot inside the store are installed, and there are a wide variety of monitoring systems that monitor people in the store using video (hereinafter referred to as “captured video”) obtained by imaging with the camera. It is popular. In the operation of such a monitoring system, when using a photographed image for purposes other than monitoring (for example, the purpose of grasping the flow of people in a store and promoting marketing analysis and service improvement), the person shown by the camera ( For example, customers' privacy needs to be protected.

  Here, for example, a monitoring system disclosed in Patent Document 1 is known as a prior art for grasping a tendency of a person's action for each attribute of a person in a store. The monitoring system of Patent Literature 1 includes a monitoring device that generates and outputs a mask video in which a person's image area detected from a moving image captured in a monitoring area is changed to a mask image. The monitoring device acquires area information for each person detected from the captured image, determines an attribute for each person detected from the captured video, and sets a mask image for each attribute. The monitoring device generates a contour line for each person based on the region information and attributes for each person, and generates and outputs a mask video in which the inside of the contour line is changed to a mask image designated for each attribute.

Japanese Patent No. 5834193

  According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed. Many existing surveillance cameras do not have a function (mask function) capable of executing the mask processing of Patent Document 1.

  Here, for example, in the user's monitoring work, a plurality of monitoring cameras are installed in the monitoring area, and the monitor of the client PC that has received the latest captured video distributed from each monitoring camera, by user operation, It is assumed that the respective captured images are switched and displayed in order. Each time the client PC switches the display of the captured video of the surveillance camera, the client PC performs a mask process on the captured video of the selected surveillance camera. Therefore, in the initial learning (see below) in the mask process, the background image ( In other words, it is necessary to generate an image region (a still region having no motion in the captured image). However, during the initial learning process, since the generation of the background image in the captured video is not completed, the client PC cannot display the captured video and cannot display the captured video (for example, 30 seconds to 1 minute). There is a problem that the degree) occurs.

  If the initial learning process in the client PC is omitted every time the display of the captured video of the monitoring camera is switched, an appropriate background image is not generated in the client PC. For this reason, the client PC cannot correctly detect the moving object (for example, a person) in the mask process, and the person in the edited video after the mask process is displayed without being replaced with the mask image. There is a possibility that privacy is not protected properly.

The present invention has been devised in view of the above-described conventional situation, and mask processing is performed after suppressing the non-display of the selected captured image even when the captured image of the monitoring camera to be displayed on the monitor is switched. It is an object of the present invention to provide a monitoring system , a video display method, and a computer program that are appropriately applied to improve the efficiency of user monitoring operations and the accuracy of marketing analysis.

The present invention is to change a captured image of an imaging area distributed from a plurality of cameras that capture an imaging area and a first camera among the plurality of cameras, or a moving body appearing in the captured image to a mask image. An information processing device that displays on a monitor the first edited video obtained by performing the masking process, and the information processing device repeatedly generates a background image of each captured video distributed from each of the cameras. The information processing apparatus stores the background image of the captured video that is distributed from the first camera and stored in the storage unit in response to switching from the first camera to the second camera. allowed to held in the storage unit, further, the background image of the delivered video image from the second camera acquired from the storage unit, which is delivered as a background image acquired from the storage unit from the second camera imaging By using the image, and displays the second edited video obtained by performing the masking process on distributed video image from the second camera on the monitor, to provide a monitoring system.

  In addition, the present invention changes a plurality of cameras that capture an imaging area, a captured image of the imaging area distributed from a first camera of the plurality of cameras, or a moving body that appears in the captured image to a mask image. An information processing apparatus that displays on a monitor a first edited video obtained by performing a mask process for performing, and a server apparatus that generates a background image of each captured video distributed from each of the cameras and stores the background image in a storage unit; The server device updates the background images of the captured images acquired from the storage unit at regular intervals, stores the updated background images in the storage unit, and the information processing device In response to switching from the first camera to the second camera, a background image after generation or update corresponding to the selected second camera is acquired from the server device, and the background image acquired from the server device and the previous image are acquired. Provided is a monitoring system for displaying on a monitor a second edited video obtained by performing the mask processing on a captured video distributed from the second camera using a captured video distributed from a second camera. To do.

Further, the present invention provides a mask process for changing a captured image of the imaging area distributed from a first camera of a plurality of cameras that capture an imaging area or a moving body appearing in the captured image to a mask image. A video display method in a monitoring system for displaying on a monitor using an information processing device, the first edited video obtained by performing the processing , wherein the information processing device is a background image of each captured video distributed from each of the cameras Are repeatedly generated and stored in the storage unit, and the information processing apparatus is delivered from the first camera and stored in the storage unit in response to switching from the first camera to the second camera. the background image of the video allowed to hold in the storage unit, further, obtains the background image of captured image distributed from the second camera from the storage unit, a background image acquired from the storage unit And a second edited video obtained by performing the mask process on the captured video delivered from the second camera using the captured video delivered from the second camera and the video displayed on the monitor Provide a method.

Further, the present invention, the mask processing for changing video image of the imaging area which is distributed from the first camera of the plurality of cameras for imaging the imaging area, or the moving object appearing in the captured image in the mask image Monitoring apparatus comprising: an information processing apparatus that displays on a monitor the first edited video obtained by performing the above and a server device that generates a background image of each captured video distributed from each of the cameras and stores the background image in a storage unit In the video display method in the system, the server device updates a background image of each captured video acquired from the storage unit at regular intervals, stores the updated background images in the storage unit, and the information In response to switching from the first camera to the second camera, the processing device acquires a generated or updated background image corresponding to the selected second camera from the server device, and Using the background image acquired from the apparatus and the captured video distributed from the second camera, the second edited video obtained by performing the mask process on the captured video distributed from the second camera is monitored. A video display method is provided.

  According to the present invention, even if there is a switching of the captured video of the monitoring camera to be displayed on the monitor, the mask processing is appropriately performed after suppressing the non-display of the selected captured video, and the user's monitoring work Efficiency and marketing analysis accuracy can be improved.

The figure which shows an example of the system configuration | structure of the surveillance camera system of each embodiment Block diagram showing an example of the internal configuration of an omnidirectional camera or fixed camera in detail Block diagram showing in detail an example of the internal configuration of a PTZ camera Block diagram showing an example of the internal configuration of a client PC in detail Block diagram showing an example of the internal configuration of the recorder in detail The figure which shows an example of a camera setting screen Figure showing an example of the area setting screen Explanatory drawing which shows the 1st example of the time-series transition of the screen accompanying the user operation regarding the setting of a mask process (MOR process). Explanatory drawing which shows the 2nd example of the time-series transition of the screen accompanying user operation regarding the setting of a mask process (MOR process). The flowchart explaining in detail an example of the operation procedure of the client PC accompanying the user operation regarding the setting of the mask process (MOR process) in the first embodiment. The flowchart explaining in detail an example of the operation procedure of the mask process (MOR process) common to each embodiment FIG. 11 is a flowchart for explaining in detail an example of an operation procedure of background image update processing in steps S15 and S16 in FIG. Explanatory diagram schematically showing the background image generation procedure by initial learning Explanatory drawing which shows the concept of a mask process (MOR process) typically (A) And (B) The figure which shows the example of a display which imaged the imaging | video of several cameras into 3 parts, (C) The figure which shows the example of a display which divided | segmented the imaging | video images of several cameras into 4 parts The flowchart explaining in detail an example of the operation procedure performed in the client PC when displaying the captured images of a plurality of cameras in the second embodiment. The figure which shows the example of a display which divided the picked-up image of a camera, and the several different picked-up image of the past of the same camera into 4 parts The flowchart explaining in detail an example of the operation procedure performed in the client PC when displaying the captured video of the camera and the recorded video from the recorder in the second embodiment. FIG. 10 is a sequence diagram for explaining in detail an example of an operation procedure for setting a mask process (MOR process) of a client PC for a mask camera and a normal camera and displaying a captured image in the third embodiment. The figure which shows an example of the camera setting screen in 3rd, 4th each embodiment The figure which shows the 4-part dividing display example of the edit image | video with which the picked-up image displayed on the monitor of client PC was masked. Diagram showing an example of the recorder setting screen (A) Sequence diagram for explaining in detail a first example of an operation procedure for setting mask processing (MOR processing) in the fourth embodiment. (B) Mask processing (MOR processing) in the fourth embodiment. Sequence diagram for explaining in detail the second example of setting operation procedure The sequence diagram explaining in detail the 1st example of the operation | movement procedure regarding acquisition and the display of the captured image in the client PC after operation start in 4th Embodiment The sequence diagram explaining in detail the 2nd example of the operation | movement procedure regarding acquisition and a display of the captured image in the client PC after operation start in 4th Embodiment FIG. 24 and FIG. 25 are flowcharts for explaining in detail an example of the display procedure of the client PC in step S69. The figure which shows another example of the system configuration | structure of the surveillance camera system of each embodiment. Schematic diagram illustrating the sequence of displaying the edited video after mask processing on the monitor of the client PC when the fixed camera has a mask function in time series Schematic diagram for explaining the sequence of displaying the edited video after mask processing on the monitor of the client PC in a time series when the client PC has a mask function. Explanatory drawing of the problem at the time of display accompanying the change of the live image of the camera to be displayed in the mask camera method and the client PC method Explanatory drawing of an example of the operation | movement outline | summary of 5th Embodiment. The schematic diagram which shows the outline | summary of the 1st video display system in 5th Embodiment, and a 2nd video display system The flowchart explaining an example of the operation | movement procedure of the 1st video display system in 5th Embodiment. The flowchart explaining an example of the operation | movement procedure of the 2nd video display system in 5th Embodiment. FIG. 10 is a sequence diagram for explaining an example of the operation procedure of the first video display method and the second video display method in the fifth embodiment. Sequence diagram following FIG. The schematic diagram which shows the outline | summary of the 3rd video display system in 5th Embodiment. 37 is a flowchart for explaining an example of the operation procedure of the main process shown in FIG. 37 is a flowchart for explaining an example of the operation procedure of the background process shown in FIG. Block diagram showing an example of the internal configuration of the server PC in detail The schematic diagram which shows the outline | summary of the 4th video display system in 5th Embodiment. Explanatory drawing of the problem at the time of the display of the edited recording video of the camera at any specified date and time in the mask camera system and the client PC system The schematic diagram which shows the outline | summary of the 5th video display system in 5th Embodiment. The flowchart explaining an example of the operation | movement procedure of the main process shown in FIG. The flowchart explaining an example of the operation | movement procedure of the background process shown in FIG. The schematic diagram which shows the outline | summary of the 6th video display system in 5th Embodiment.

(Background to the contents of the first embodiment)
According to Patent Document 1 described above, a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area. Once secured, the behavior of the person in the store can be grasped.

  However, Patent Document 1 does not consider the idea of changing a person existing in an arbitrary area in a monitoring area directly designated by a user (for example, a monitor) to a captured image of a camera to a mask image. . Although it is important to protect the privacy of a person existing in the captured video, depending on the installation environment of the camera, when one or more persons existing in all areas in the captured video all become mask images. The visibility of the captured image deteriorates. For this reason, it is desired to achieve both the protection of the privacy of a person and the suppression of deterioration of the visibility of the captured image.

  Therefore, in the following first embodiment, the privacy of a person shown in the captured image of the camera is protected and the deterioration of the visibility of the captured image of the camera is suppressed so as to suit the environment of the place where the camera is installed. An example of a monitoring system and a mask processing setting method that improve the usability of the user will be described.

  Hereinafter, each embodiment specifically disclosing the monitoring system according to the present invention will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

  Hereinafter, a monitoring camera system according to the present invention, a monitoring camera system in which at least one surveillance camera is installed in a monitoring area (for example, a store, a factory, a sales office, a public facility such as a library or an event venue, an office, or a bank). An example of a manned surveillance camera system or an unmanned surveillance camera system is possible.

  The present invention can also be expressed as a mask processing setting method in the monitoring system. Further, the present invention executes a program for causing a client PC (Personal Computer), which is a computer constituting the monitoring system, to execute an operation specified by the mask processing setting method, or an operation specified by the mask processing setting method. It can also be expressed as a recording medium on which a program for causing the program to be recorded is recorded.

  Hereinafter, a user of a monitoring camera system as an example of a monitoring system (for example, a monitoring person browsing a captured image of a monitoring area) is simply referred to as a “user”. In the following description, “omnidirectional camera”, “PTZ camera”, and “fixed camera” are simply referred to as “camera”. Therefore, in the following description, when simply described as “camera”, the camera indicates at least one of an omnidirectional camera, a PTZ camera, and a fixed camera.

(First embodiment)
FIG. 1 is a diagram illustrating an example of a system configuration of a monitoring camera system 100 according to each embodiment. In the surveillance camera system 100, recorders REC1 and REC4 and routers RUT2, RUT3 and RUT4 are connected via a network NW. The surveillance camera system 100 includes a client PC 10 used by a user, a plurality of cameras (refer to an omnidirectional camera, a PTZ (Pan Tilt Zoom) camera, and a fixed camera described later), and a plurality of recorders (refer to a later description). It is a configuration.

  The recorder REC1 is connected to the omnidirectional camera CAM11, the PTZ cameras CAM12 and CAM13, and the fixed camera CAM14 via a router RUT1 having a known routing function. The captured images captured by these omnidirectional cameras CAM11, PTZ cameras CAM12, CAM13, and fixed camera CAM14 are stored as recorded images in the recorder REC1 via the router RUT1 in association with the corresponding camera identification information and imaging date / time. The The omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including this embodiment, a plurality of cameras (that is, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) are associated with the recorder REC1.

  The recorder REC4 has a function as an online storage, and stores data such as a captured video requested to be recorded from a connection destination device (for example, another recorder, the client PC 10) of the network NW.

  The recorder REC2 is connected to the network NW, the PTZ camera CAM21, and the fixed camera CAM22 via a router RUT2 having a known routing function. The captured images captured by the PTZ camera CAM21 and the fixed camera CAM22 are stored as recorded images in the recorder REC2 via the router RUT2 in association with the corresponding camera identification information and imaging date / time. The PTZ camera CAM21 and the fixed camera CAM22 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including the present embodiment, a plurality of cameras (that is, the PTZ camera CAM21 and the fixed camera CAM22) are associated with the recorder REC2.

  The recorder REC3 is connected to the network NW, the omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34 via a router RUT3 having a known routing function. The captured images respectively captured by these omnidirectional cameras CAM31, PTZ cameras CAM32, CAM33, and fixed camera CAM34 are stored as recorded images in the recorder REC3 via the router RUT3 in association with the corresponding camera identification information and imaging date / time. The The omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34 may each capture the same imaging area or different imaging areas. Thus, in each embodiment including this embodiment, a plurality of cameras (that is, the omnidirectional camera CAM31, the PTZ cameras CAM32, CAM33, and the fixed camera CAM34) are associated with the recorder REC3.

  The client PC 10 as an example of the information processing apparatus is imaged by the above-described cameras (specifically, omnidirectional cameras CAM11 and CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, and CAM34). Each of the captured images has a function of performing a mask process for changing at least a part or the whole of a moving body (for example, a person, the same applies hereinafter) appearing in the captured image to a mask image. The client PC 10 displays the captured video transmitted (distributed) from various cameras or recorders on the monitor DP, or monitors the edited video after masking the captured video transmitted (distributed) from various cameras or recorders. Or display on DP. In the following description, the moving body is not limited to a person, and may be an automobile or a robot.

  Here, the mask image is subjected to image processing (that is, mask processing) for making it difficult to specifically identify who the person is in order to protect the privacy of the person appearing in the captured video. It is an image. The mask image is, for example, an image in which the inside of a contour line of a person appearing in a captured video is processed to be translucent. In the following description, the mask process may be referred to as a MOR (Moving Object Remover) process. The MOR process, which will be described in detail later with reference to FIGS. 11 and 12, shows a process of replacing a moving object region where a moving object appears in a captured image with a mask image (for example, a translucent image). As a result, it is difficult to specifically identify who the person appears in the captured video, and the privacy of the person can be protected. For example, when a moving body (for example, a person) in a captured image is replaced with a mask image MSK1 of a translucent image, for example, by the mask processing unit 114 (see later), the inside of the outline of the person appearing in the captured image becomes translucent. You can see through the background image and see what is in the background. Thereby, even if a person is replaced with the mask image MSK1, it is possible to ensure visibility as a video while protecting the privacy of the person.

  In the following description, an image captured by the camera is referred to as “captured image”, and a mask image after the captured image is masked by, for example, the client PC 10 or a mask camera having a mask function (see later) is captured image. The video superimposed on is called “edited video”.

  The client PC 10 is connected to the network NW and the system controller 5 via a router RUT 4 having a known routing function. The system controller 5 is connected to a joystick J1 provided to support user operations. For example, the system controller 5 performs various settings related to the operation of the devices constituting the surveillance camera system 100.

  The network NW may be a wired network (for example, an intranet, the Internet, a wired LAN (Local Area Network), or a wireless network (for example, a wireless LAN), for example, the recorders REC1, REC2, REC3, and REC4 are connected via the network NW. The monitor DP may be directly connected to the client PC 10. In addition, the monitor DP is illustrated as being provided separately from the client PC in Fig. 1. However, the monitor DP is shown in Fig. 4. The system controller 5, the joystick J1, and the client PC 10 may be installed in, for example, a monitoring room where a user resides during monitoring.

  The network NW may be a wide area network (WAN), and in this case, for example, a recorder is provided for each of a plurality of bases or a plurality of stores in a company, The picked-up image picked up for each store is stored in each recorder.

  2 is a block diagram showing in detail an example of the internal configuration of the omnidirectional camera CAM11,... Or the fixed camera CAM14,. The internal configurations of the omnidirectional cameras CAM11 and CAM31 and the fixed cameras CAM14, CAM22, and CAM34 are the same. Here, for the sake of simplicity of description, for example, the internal configuration of the omnidirectional camera CAM11 will be described. In the description of FIG. 2, the omnidirectional camera CAM11 may be read as another omnidirectional camera (for example, the omnidirectional camera CAM31) or another fixed camera (for example, the fixed camera CAM14). However, in the description of FIG. 2, the contents of different configurations of the omnidirectional camera CAM11 and the fixed camera will be described. The omnidirectional camera CAM11 shown in FIG. 2 includes a processor C11, a communication unit C12, a lens C13, an image sensor C14, a memory C15, and a power management unit C16.

  The processor C11 performs signal processing for overall control of operations of each unit of the omnidirectional camera CAM11, data input / output processing with other units, data calculation processing, and data storage processing. The processor C11 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA).

  For example, the processor C11 acquires omnidirectional image data obtained by imaging in the image sensor C14 in accordance with a user operation using the system controller 5 or the client PC 10, and further, a specific range of the omnidirectional image data. Two-dimensional panoramic image data (that is, image data obtained by two-dimensional panorama conversion) obtained by cutting out an image of (direction) is generated and stored in the memory C15. Further, the processor C11 acquires captured image data having a fixed angle of view obtained by imaging by the image sensor C14 in accordance with a user operation using the system controller 5 or the client PC 10, and stores it in the memory C15.

  The communication unit C12 is a network interface (I / F) that controls data communication with the router RUT1, and transmits omnidirectional image data, two-dimensional panoramic image data, or captured image data with a fixed angle of view via the router RUT1. To the recorder REC1. Further, the communication unit C12 transmits the omnidirectional image data, the two-dimensional panoramic image data, or the captured image data with a fixed angle of view to the client PC 10 in association with the camera identification information and the imaging date and time via the router RUT1 and the recorder REC1. It doesn't matter.

  For example, in the omnidirectional camera CAM11, the lens C13 is configured by using a lens having a wide angle of view or a fisheye lens. An optical image of the subject light is formed on the light receiving surface (imaging surface) of the sensor C14. On the other hand, the lens C13 is configured by using, for example, a lens having a fixed angle of view in the fixed camera CAM14. The lens C13 receives and collects subject light from the angle of view of the imaging area to be imaged by the fixed camera CAM14, and collects it. An optical image of the subject light is formed on the light receiving surface (imaging surface) of C14.

  The image sensor C14 is configured by using, for example, a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, and receives an optical image of subject light from the imaging area collected by the lens C13 as a light receiving surface (imaging surface). ), The omnidirectional image data of the imaging area or the captured image data of a fixed angle of view is acquired. The omnidirectional image data or the captured image data having a fixed angle of view is acquired by the processor C11.

  The memory C15 includes a ROM 15z in which programs for setting the operation of the omnidirectional camera CAM11 or the fixed camera CAM14 and data of setting values are stored, captured image data having a fixed angle of view, omnidirectional image data, or a partial range thereof. A RAM 15y that stores cut-out image data and work data obtained by cutting out, and a memory card 15x that is detachably connected to the omnidirectional camera CAM11 or the fixed camera CAM14 and stores various data.

  The power management unit C16 supplies DC power to each unit of the omnidirectional camera CAM11 or the fixed camera CAM14. The power management unit C16 may supply DC power to other devices (for example, PTZ cameras CAM12 and CAM13) connected to the router RUT1.

  FIG. 3 is a block diagram showing in detail an example of the internal configuration of the PTZ cameras CAM12, CAM13,. The internal configurations of the PTZ cameras CAM12, CAM13, CAM21, CAM32, and CAM33 are the same. Here, for the sake of simplicity of description, for example, the internal configuration of the PTZ camera CAM12 will be described. In the description of FIG. 3, the PTZ camera CAM12 may be replaced with another PTZ camera (for example, the PTZ camera CAM13). Moreover, about each part similar to the omnidirectional camera CAM11 and the fixed camera CAM14, the code | symbol corresponding to each part of FIG. 2 is attached | subjected, and the description is abbreviate | omitted. The PTZ camera CAM 12 is a camera that can adjust the optical axis direction (imaging direction), for example, according to a view angle change instruction from the system controller 5 or the client PC 10 used by the user.

  The PTZ camera CAM12 includes, for example, a processor C21, a communication unit C22, a lens C23, an image sensor C24, and a memory C25 as well as the omnidirectional camera CAM11, and also includes an imaging direction control unit C27 and a lens drive motor C28. For example, when there is a view angle change instruction from the system controller 5 or the client PC 10, the processor C21 notifies the imaging direction control unit C27 of the view angle change instruction.

  The imaging direction control unit C27 controls the imaging direction of the PTZ camera CAM12 in at least one of the pan direction and the tilt direction according to the view angle change instruction notified from the processor C21, and further changes the zoom magnification as necessary. A control signal for this is output to the lens drive motor C28. The lens drive motor C28 drives the lens C23 in accordance with this control signal, changes the imaging direction (that is, the optical axis direction), and adjusts the focal length of the lens C23 to change the zoom magnification.

  The lens C23 is configured using one or two or more lenses. In the lens C23, the optical axis directions of pan rotation and tilt rotation are changed by driving the lens driving motor C28 according to the control signal from the imaging direction control unit C27.

  FIG. 4 is a block diagram showing an example of the internal configuration of the client PC 10 in detail. The client PC 10 illustrated in FIG. 4 includes a processor 11, a communication unit 12, an operation unit 13, a storage 15, a memory 16, and a power management unit 17.

  The processor 11 performs signal processing for overall control of operations of each unit of the client PC 10, data input / output processing with other units, data calculation processing, and data storage processing. The processor 11 is configured using, for example, a CPU, MPU, DSP, or FPGA. The processor 11 includes an image input unit 111, a background image processing unit 112, a mask processing setting unit 113, a mask processing unit 114, and a video output control unit 115.

  The image input unit 111 inputs captured images transmitted from various cameras and recorders and received by the communication unit 12.

  The background image processing unit 112 will be described in detail with reference to FIG. 11 and FIG. 12. The background image processing unit 112 uses the captured video acquired by the image input unit 111 to perform background image processing performed in mask processing (MOR processing) in the client PC 10. Perform various generation and update processes. Details of generation and update of the background image will be described later.

  The mask process setting unit 113 performs settings related to a mask process (MOR process) that can be executed by the client PC 10 in response to a user operation (described later).

  The mask processing setting unit 113, for example, in response to a user operation that specifies whether or not to perform mask processing on the captured video acquired by the image input unit 111, the distribution source (transmission source) of the captured video Is generated in association with a camera (that is, an omnidirectional camera, a PTZ camera, or a fixed camera), and mask setting information indicating execution or non-execution of mask processing is generated and stored in the storage 15 or the memory 16.

  Further, the mask processing setting unit 113, for example, position information (for example, coordinate information) of the designated area in response to a user operation that designates a non-target area for mask processing in the captured video acquired by the image input unit 111, for example. Is generated and stored in the storage 15 or the memory 16. Although details of generation of the mask setting information will be described later, the mask setting information holds at least the position information of the mask processing execution / non-execution area and the mask processing non-target area for each camera.

  Based on the mask setting information generated by the mask processing setting unit 113, the mask processing unit 114 selects a region not to be masked specified by the mask setting information from the captured video acquired by the image input unit 111. The excluded area is selected as a mask process target area. The mask processing unit 114 generates a mask image in which the outline of the moving body (for example, a person) is changed to a mask image (for example, a translucent image) in the selected mask processing target region. Details of the mask processing will be described later.

  The video output control unit 115 displays the captured video from the camera set not to execute the mask process or the edited video already masked (see later) by the camera as it is on the monitor DP. In addition, the video output control unit 115 uses the captured video from the camera set to execute the mask process in the client PC 10 and the mask image generated by the mask processing unit 114 for the captured video. An edited video in which a mask image is superimposed on the position of the moving body (for example, a person) inside is generated. The video output control unit 115 displays the edited video on the monitor DP.

  The communication unit 12 receives captured video or edited video (described later) transmitted from various cameras and recorders via the router RUT 4 and outputs the received video to the processor 11.

  The operation unit 13 is a user interface (UI) for notifying the processor 11 of the contents of a user input operation, and is configured by a pointing device such as a mouse or a keyboard. The operation unit 13 may be configured using, for example, a touch panel or a touch pad that is arranged corresponding to the display screen of the monitor DP and can be directly input by a user's finger or stylus pen.

  The monitor DP as an example of a display unit is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and is set by a mask processing setting unit 113 (see FIG. 6) or an area setting screen. (See FIG. 7) or a captured image or an edited video output by the video output control unit 115 is displayed.

  The storage 15 as an example of a storage unit stores, for example, mask setting information generated by the mask processing setting unit 113, and is further generated by a captured image distributed from (transmitted to) various cameras and recorders or the processor 11. Stores edited video data.

  The memory 16 includes a ROM 161 that stores a program for specifying the operation of the client PC 10 and data of setting values, and a RAM 162 that stores various data generated by the processor 11 and work data being processed. Although not shown in FIG. 4, the memory 16 may further include a memory card (not shown) that is connected to the client PC 10 so as to be freely inserted and removed and that stores various data.

  The power management unit 17 supplies DC power to each unit of the client PC 10.

  FIG. 5 is a block diagram showing in detail an example of the internal configuration of the recorders REC1,. The internal configuration of the recorders REC1, REC2, REC3, and REC4 is the same. For the sake of simplicity, the recorder REC1, REC2, REC3, and REC4 will be described as the internal configuration of the recorder REC1, for example. In the description of FIG. 5, the recorder REC1 may be replaced with another recorder (for example, the recorder REC2). The recorder REC1 shown in FIG. 5 includes a processor R11, a communication unit R12, a memory R13, a storage R14, and a power management unit R16.

  The processor R11 performs signal processing for overall control of operations of each unit of the recorder REC1, data input / output processing with other units, data calculation processing, and data storage processing. The processor R11 is configured using, for example, a CPU, MPU, DSP, or FPGA.

  For example, the processor R11 captures image data captured by various cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) in accordance with a user operation using the system controller 5 or the client PC 10. The captured video data is stored as a recorded video in the storage R14 in association with the imaging date and camera identification information.

  The communication unit R12 captures captured image data captured by various cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14) or an edited image obtained by masking the captured image (see below). ) Is received via the router RUT1 and output to the processor R11. Further, the communication unit R12 transmits the recorded video data read from the storage R14 by the processor R11 to the client PC 10 in response to an instruction from the client PC 10.

  The memory R13 includes a ROM 131 that stores a program for defining the operation of the recorder REC1 and data of set values, and a RAM 132 that stores various data generated by the processor R11 and work data being processed. Although not shown in FIG. 5, the memory R13 may further include a memory card (not shown) that is detachably connected to the recorder REC1 and stores various data.

  The storage R14 stores captured video data distributed (transmitted) from various cameras, for example. The storage R14 may store the edited video data when the edited video data generated by the client PC 10 is transmitted from the client PC 10 to the recorder REC1.

  The power management unit R15 supplies DC power to each unit of the recorder REC1.

  Next, the operation of the monitoring camera system 100 of the present embodiment will be described in detail with reference to FIGS.

  FIG. 6 is a diagram illustrating an example of the camera setting screen WD1. FIG. 7 is a diagram illustrating an example of the area setting screen WD2. FIG. 8 is an explanatory diagram showing a first example of time-series transition of a screen accompanying a user operation related to setting of mask processing (MOR processing). FIG. 9 is an explanatory diagram illustrating a first example of time-series transition of a screen accompanying a user operation regarding setting of mask processing (MOR processing). FIG. 10 is a flowchart for explaining in detail an example of an operation procedure of the client PC 10 in accordance with a user operation regarding the setting of the mask process (MOR process) in the first embodiment.

  First, the camera setting screen WD1 shown in FIG. 6 will be described.

  In FIG. 6, a part of the camera setting screen WD1 is extracted and shown. The mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP, for example, by a user operation. In this camera setting screen WD1, at least IP (Internet) is provided to individual cameras of, for example, a maximum of 64 cameras (that is, composed of an omnidirectional camera, a PTZ camera, and a fixed camera) by the user's operation using the operation unit 13. Protocol) address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be associated and set. In addition, by switching the four tabs (camera 1-16, camera 17-32, camera 33-48, camera 49-64) provided on the camera setting screen WD1, about the corresponding camera in units of 16 cameras, Information on at least an IP (Internet Protocol) address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be set.

  In the IP address column CLM1, the IP address of the corresponding camera is input.

  In the column CLM2 of mask processing (MOR processing), either of execution (ON) or non-execution (OFF) of mask processing (MOR processing) in the client PC 10 is selected (see frame WK2).

  In the column CLM3 for setting the mask process (MOR process), a setting button (see frame WK1) for displaying an area setting screen WD2 for setting a non-target area for the mask process (MOR process) is displayed. However, the setting button of the column CLM3 is displayed so that it can be pressed for a camera for which ON is selected in the column CLM2, but the setting button of the column CLM3 is not displayed so that it can be pressed for a camera for which OFF is selected in the column CLM2. (For example, refer to the dotted lines on the fifth, eighth, tenth, and fifteenth stages from the top of FIG. 6 in the row CLM3).

  In an initial learning period column CLM4, an initial learning period in a mask process (MOR process) described later is input (see frame WK3). Similarly, the initial learning period of the column CLM4 can be input for a camera whose ON is selected in the column CLM2, but the initial learning period of the column CLM4 cannot be input for a camera whose OFF is selected in the column CLM2. For example, see the fifth, eighth, tenth, and fifteenth dotted lines from the top of the page of FIG.

  In the normal learning period column CLM5, a normal learning period in a mask process (MOR process) described later is input (see frame WK3). Similarly, the normal learning period of the column CLM5 can be input for a camera whose ON is selected in the column CLM2, but the normal learning period of the column CLM5 cannot be input for a camera whose OFF is selected in the column CLM2. (For example, refer to the dotted lines on the fifth, eighth, tenth, and fifteenth tiers from the top of FIG. 6 of the row CLM5).

  Next, the area setting screen WD2 shown in FIG. 7 will be described.

  In FIG. 7, the entire area setting screen WD2 is shown. When the setting button of the column CLM3 is pressed for a camera whose ON is selected in the column CLM2 of the camera setting screen WD1, the mask processing setting unit 113 switches the region setting screen WD2 to the monitor DP and displays it. The mask processing setting unit 113 may display the camera setting screen WD1 and the region setting screen WD2 separately on the monitor DP. In this area setting screen WD2, a non-target area for mask processing (MOR processing) can be set in association with the camera while the captured video MV1 captured by the camera is displayed by the user's operation using the operation unit 13. .

  On the area setting screen WD2, items of device name TM1, visual area TM2, non-visual area TM3, deletion TM4, and image call TM5, a display area for video, and a set button STBT for setting completion are shown. . The display area for video is a display area of the captured video MV1 (that is, live video) currently captured by the camera (for example, PTZ camera CAM13) specified by the device name TM1. However, the video displayed in the video display area of the area setting screen WD2 is not limited to the captured video MV1 (that is, live video) currently captured by the camera (for example, the PTZ camera CAM13), and is already in the recorder REC1, for example. It may be a recorded video recorded in the past. In this case, the client PC 10 receives the past recorded video distributed (transmitted) from the recorder REC1, and displays it in the video display area of the area setting screen WD2.

  In the device name TM1, for example, a PTZ camera CAM13 is shown.

  In the visual area TM2, area selection buttons VA1, VA2, and VA3 for designating areas not to be masked are displayed. In other words, if a region that is not subject to mask processing is designated, even if there is a person who moves within that region, the mask processing is not performed and the image is displayed in the video. The area selection button VA1 is used to designate a non-target area for mask processing with a rectangle (for example, a rectangular shape) (see FIG. 8). The area selection button VA2 is used for designating a non-maskable area as a polygon (for example, a pentagon) (see FIG. 9). The area selection button VA3 is used to select a mask processing non-target area designated by operating the area selection button VA1 or the area selection button VA2.

  In the non-visual area TM3, area selection buttons IVA1, IVA2, and IVA3 for designating an area (hidden area) to be hidden from the captured image are displayed. In other words, when a hidden area is designated, the hidden area is filled with, for example, black, and is not displayed in the video. The area selection button IVA1 is used to designate a hidden area with a quadrangle (for example, a rectangular shape). The area selection button IVA2 is used to designate a hidden area with a polygon (for example, a pentagon). The area selection button IVA3 is used to select a hidden area designated by the operation of the area selection button IVA1 or the area selection button IVA2.

  The deletion TM4 includes a selection button DL1 for deleting one selected mask processing non-target area or one selected hidden area, and all selected mask processing non-target areas or all selected mask processing. A selection button DL2 for deleting the hidden area is displayed. Note that, when only one of the non-masking target areas or the hidden areas is selected, the corresponding area portion is indicated by a dotted line in the captured video MV1.

  The set button STBT is pressed by a user operation when a non-masking target area or a hidden area is designated and the designated mask processing non-target area or hidden area is determined. Accordingly, the mask processing setting unit 113 displays the position information (more specifically, the position information in the captured video MV1) of the specified non-target mask processing area and the hidden area by pressing the set button STBT. The data is stored in the storage 15 or the memory 16 in association with the camera specified by the above.

  Here, the setting of the non-target area for the mask process will be described with reference to FIGS. 8 and 10, for example. By a user operation for displaying the camera setting screen WD1 (see FIG. 6), the mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP (S1).

  When a user operation selects the column CLM2 of the record of the PTZ camera CAM13 on the camera setting screen WD1 (see FIG. 6) and the setting button (see the frame WK1) of the column CLM3 of the record is pressed (S2, YES), the mask processing setting unit 113 displays an area setting screen WD2 corresponding to the PTZ camera CAM13 on the monitor DP (S3). Unless the setting button of the column CLM3 is pressed (S2, NO), the processing of the processor 11 of the client PC 10 does not proceed to step S3.

  In the area setting screen WD2 of FIG. 8, the area selection button VA1 (for example, a rectangular area selection button VA1) of the visual area TM2 is pressed by a user operation (S4), and the persons JB1, JB2, and JB3 are shown. It is assumed that the television apparatus in the captured video MV1 is designated to be surrounded by a drag operation or the like (S5). The mask processing setting unit 113 draws a rectangular frame JGA1 around the television device designated by the region selection button VA1 on the region setting screen WD3. Further, the mask processing setting unit 113 uses the position information (for example, coordinate information) of the frame JGA1 as a region not to be masked when the set button STBT is pressed by a user operation on the region setting screen WD3 (S6). Then, camera setting information associated with the PTZ camera CAM 13 is generated and stored in the storage 15 or the memory 16 (S7). The mask processing unit 114 refers to the camera setting information generated by the mask processing setting unit 113 and masks moving bodies (for example, persons JB1, JB2, JB3) other than the inside of the frame JGA1 in the captured video MV1 as mask images MK1, MK2. , MK3 to perform mask processing. The video output control unit 115 generates an edited video using the mask images MK1, MK2, and MK3 generated by the mask processing unit 114 and the captured video MV1, and displays the edited video on the video display screen WD4 of the monitor DP.

  Thereby, since the inside of the frame JGA1 is a non-masking target area, the masking process is not performed. For example, even if a person moving to the television set in the frame NMK1 of the video display screen WD4 is displayed, It is possible to suppress the deterioration of the visibility of the edited video displayed on the video display screen WD4 without being changed to the mask image (for example, the mask image MK1).

  Note that the video output control unit 115 executes mask processing (more specifically, normal learning) when performing normal learning (see below) executed during mask processing in the mask processing unit 114. “MOR Running” indicator ID1 indicating that the image is displayed on the upper left of the video display screen WD4. Accordingly, the user can visually and easily grasp the state in which the mask process is being executed in the client PC 10 by browsing the video display screen WD4 of the monitor DP.

  Here, the setting of the non-target area for mask processing will be described with reference to FIGS. 9 and 10, for example. By a user operation for displaying the camera setting screen WD1 (see FIG. 6), the mask processing setting unit 113 displays the camera setting screen WD1 on the monitor DP (S1).

  When a user operation selects the column CLM2 of the record of the PTZ camera CAM13 on the camera setting screen WD1 (see FIG. 6) and the setting button (see the frame WK1) of the column CLM3 of the record is pressed (S2, YES), the mask processing setting unit 113 displays an area setting screen WD2 corresponding to the PTZ camera CAM13 on the monitor DP (S3). Unless the setting button of the column CLM3 is pressed (S2, NO), the processing of the processor 11 of the client PC 10 does not proceed to step S3.

  In the area setting screen WD2 of FIG. 9, the area selection button VA2 (for example, the pentagonal area selection button VA2) of the visual area TM2 is pressed by the user operation (S4), and the persons JB1, JB2, and JB3 are shown. It is assumed that the pentagonal vertices PT1, PT2, PT3 are selected and the sides between the vertices are designated by a drag operation or the like so that the head periphery of the person JB1 in the captured video MV1 is surrounded (S5). The mask processing setting unit 113 draws a pentagonal frame JGA2 around the head of the person JB1 designated by the region selection button VA2 on the region setting screen WD3. Furthermore, the mask processing setting unit 113 uses the position information (for example, coordinate information) of the frame JGA2 as an area not to be masked when the set button STBT is pressed by a user operation on the area setting screen WD3 (S6). Then, camera setting information associated with the PTZ camera CAM 13 is generated and stored in the storage 15 or the memory 16 (S7). The mask processing unit 114 refers to the camera setting information generated by the mask processing setting unit 113 and masks moving bodies (for example, persons JB2, JB3) other than the inside of the frame JGA2 in the captured video MV1 as mask images MK2, MK3 (FIG. 8)) is performed. The video output control unit 115 generates an edited video using the mask images MK2 and MK3 generated by the mask processing unit 114 and the captured video MV1 and displays it on a video display screen (not shown) of the monitor DP.

  Thereby, since the inside of the frame JGA2 is a non-masking target area, the masking process is not performed. For example, even if a person moving around the head of the person JB1 in the video display screen (not shown) is projected. The person is not changed to the mask image (for example, the mask image MK1), and the degradation of the visibility of the edited video displayed on the video display screen can be suppressed.

  Next, details of the operation contents of the mask process (MOR process) in the processor 11 of the client PC 10 will be described with reference to FIGS. 11 and 12. FIG. 11 is a flowchart illustrating in detail an example of an operation procedure of mask processing (MOR processing) common to the embodiments. FIG. 12 is a flowchart illustrating in detail an example of the operation procedure of the background image update process in steps S15 and S16 of FIG.

  In FIG. 11, the background image processing unit 112 of the processor 11 analyzes a captured video (for example, a captured video for a certain period of time after the start of imaging by the camera) acquired by the image input unit 111 and moves in the captured video. A background image made up of a region without an image (hereinafter referred to as “still region”) is generated (S11). Hereinafter, the process of step S11 is defined as “initial learning” in the mask process (MOR process).

  FIG. 13 is an explanatory diagram schematically showing a procedure for generating a background image by initial learning. The background image processing unit 112 acquires captured video for a certain period of time, and in a plurality of captured images constituting the captured video (see the left side of the paper in FIG. 13), by known image processing (for example, calculation of difference between frames), A still area and a moving area (hereinafter referred to as “moving body area”) are detected between frames of the captured image. In FIG. 13, the person JB4 is shown as a moving object. The background image processing unit 112 generates a background image BG1 including a region without motion (that is, a still region) by the above-described image processing.

  Note that during the initial learning process of step S11, the background image processing unit 112 cannot separate the still area and the moving object area in the captured video. For this reason, the captured image during the initial learning period is not displayed on the monitor DP. In other words, the video output control unit 115 does not display the captured video distributed (transmitted) from the camera on the monitor DP during the initial learning period.

  FIG. 14 is an explanatory diagram schematically showing the concept of mask processing (MOR processing). After the completion of the initial learning in step S11, the mask processing unit 114 of the processor 11 uses the pixel value (for example, luminance value or RGB (Red Green Blue) value, etc.) of the background image BG1 generated in step S11 and the camera. The pixel value of the current captured video AA1 distributed (transmitted) is determined in units of pixels constituting a background image or captured video or a unit of a predetermined pixel area composed of a plurality of pixels (for example, a unit of 4 pixels × 4 pixels). Compare.

  As a result of the comparison described above, the mask processing unit 114 determines that the region in the captured image having a pixel value larger than a certain threshold (known) is the moving object region OB1 (S12). The mask processing unit 114 changes the image of the corresponding part in the captured video determined to be the moving object region to the mask image MSK1 (for example, a translucent image), and the edited video BB1 in which the mask image MSK1 is superimposed on the moving object region in the captured video AA1. Is generated and displayed on the monitor DP (S13). The process of displaying the edited video BB1 on the monitor DP may be performed by the video output control unit 115. Further, the mask processing unit 114 may store the generated edited video BB1 in the storage 15 or the memory 16 in association with the camera identification number (for example, camera ID) or transmit it to the recorder (for example, the recorder REC1). You may memorize it. For example, when a moving body (for example, a person) in a captured image is replaced by a mask image MSK1 of a translucent image by the mask processing unit 114, for example, the outline of the person appearing in the captured image becomes translucent and the background image is transparent. You can see what is in the background. Thereby, even if a person is replaced with the mask image MSK1, it is possible to ensure visibility as a video while protecting the privacy of the person.

  On the other hand, as a result of the comparison described above, the mask processing unit 114 determines an area in the captured image in which the pixel value is smaller than a certain threshold (known) as a still area (S14). The mask processing unit 114 displays the image of the corresponding part in the captured image determined to be a still area as it is on the monitor DP as the current captured image or background image (S14). Note that the video output control unit 115 may perform the process of displaying the image of the corresponding part in the captured video determined to be a still region on the monitor DP. Further, the mask processing unit 114 may store the image of the corresponding part in the captured video determined to be a still area in the storage 15 or the memory 16 in association with the camera identification number (for example, camera ID) or the recorder ( For example, it may be transmitted and stored in the recorder REC1). Steps S12, S13, and S14 are not performed in the order of time series, but are performed substantially simultaneously or simultaneously in parallel. The same applies to steps S15 and S16.

  After step S14, the background image processing unit 112 updates the background image of the video area determined as the still area in step S14 using the video area and the background image generated in step S11 (S15). For example, when the background image is updated once per second (see FIG. 12) and 1% of the information of the new captured video is used and 99% of the information of the current background image is used, about 100 seconds later. The background image is completely updated. Details of the processing in step S15 will be described later with reference to FIG.

  After step S14, the background image processing unit 112 updates the background image of the video area determined as the moving object area in step S12 using the video area and the background image generated in step S11 (S16). Hereinafter, “normal learning” in the mask process (MOR process) is defined as the process in steps S15 and S16. For example, when updating the background image once a second (see FIG. 12) and using 0.1% of new captured video information and 99.9% of current background image information, The background image is completely updated after about 1000 seconds. In this case, the normal learning period is the longer period (ie, 1000 seconds = about 17 minutes) of the 100 seconds exemplified in step S15 and the 1000 seconds exemplified in step S16. Details of the processing in step S16 will be described later with reference to FIG.

  If the mask process (MOR process) ends after steps S15 and S16 (S17, YES), the process of the processor 11 shown in FIG. 11 ends. On the other hand, if the mask process (MOR process) does not end after steps S15 and S16 (S17, YES), the process of the processor 11 returns to step S12.

  Each process of step S15 and S16 is performed in the order of the process shown in FIG. The difference between the process of step S15 and the process of step S16 is a value of a background image update ratio X described later. In the description of FIG. 12, for example, the background image update ratio X = 0.01 (= 1%) for the process of step S15, and the background image update ratio X = 0.001 (= 0.1) for the process of step S16. %).

  More specifically, the background image update ratio X = 0.01 (= 1%) means that 1% of new captured video information is used and 99% of current background image information is used in the background image update. It means to do. Similarly, the background image update ratio X = 0.001 (= 0.1%) means that 0.1% of the information of the new captured video is used in the update of the background image, and the current background image is 99.9%. Means to use. In any case, updating the background image means taking in information of a new captured video in accordance with the value of the background image update ratio X.

  In the description of FIG. 12, the pixel value is an RGB value (that is, a value indicating the intensity of red, green, and blue in the pixel in the image). When the color table is 24-bit RGB, (R, G, B) = (255, 255, 255) indicates white, and (R, G, B) = (0, 0, 0) indicates black. , (R, G, B) = (255, 0, 0) indicates red, (R, G, B) = (0, 255, 0) indicates green, and (R, G, B) = ( 0, 0, 255) indicates blue.

  In FIG. 12, the background image processing unit 112 sets the coordinates (x, y) of the pixels constituting the background image to be updated to (0, 0) (S21). The background image processing unit 112 acquires the pixel value (R0, G0, B0) at the pixel at coordinates (x, y) from the current captured video (S22). The background image processing unit 112 acquires the pixel values (R1, G1, B1) at the pixel at coordinates (x, y) from the current background image (for example, the background image generated in the initial learning in step S11) (S23). .

  The background image processing unit 112 uses the pixel values (R0, G0, B0) acquired in step S22 and the pixel values (R1, G1, B1) acquired in step S23, and formulas (1) to (3). Accordingly, the pixel values (R2, G2, B2) at the coordinates (x, y) of the pixels constituting the updated background image are calculated (S24).

R2 = R0 × X + R1 × (1.0−X) (1)
G2 = G0 × X + G1 × (1.0−X) (2)
B2 = B0 × X + B1 × (1.0−X) (3)

  For example, in the case of R0 = 100, R1 = 50, and X = 0.1 (= 10%), R2 = 100 × 0.1 + 50 × 0.9 = 55 is calculated by Equation (1). Thereby, the background image processing unit 112 can update the R (element) value of the corresponding pixel from 50 (= R1) to 55 (= R2).

  After step S24, when the background image processing unit 112 updates the pixel values by calculation using the equations (1) to (3) for the coordinates (x, y) of all the pixels of the background image to be updated. (S25, YES), the processing of the background image processing unit 112 shown in FIG. 12 ends.

  On the other hand, after step S24, the pixel value is updated by calculation using Equations (1) to (3) for the coordinates (x, y) of all the pixels of the background image to be updated by the background image processing unit 112. If not (S25, NO), the background image processing unit 112 sets the coordinates (x, y) of the pixel to the next coordinates (S26). After step S26, the background image processing unit 112 returns to step S22.

  Accordingly, the processor 11 performs the mask process (MOR) shown in FIGS. 11 and 12 to change a moving area (for example, a person) in the captured image into a mask image, so that An edited video that makes it difficult to identify who the person is can be generated and displayed on the monitor DP, and the privacy of the person can be protected. Further, the processor 11 can update the background image to follow the environmental change even when an environmental change occurs, for example, when a new fixed object is installed in the imaging area, or the light is turned off or turned on. it can. For example, the processor 11 can update a newly-installed fixed object in the background image, and can also generate an appropriate edited video corresponding to the environmental change and display it on the monitor DP.

  In the process of step S16, after a sufficient time for the background image processing unit 112 to determine that the new fixed object is a static region has elapsed, it is captured as a background image. It is preferable that the background image is captured after the background follow-up time has elapsed. The background follow-up time corresponds to a normal learning period. For example, in an environment (for example, a place) where the possibility of stopping is low, such as a person on a road, it is preferable to set a normal learning period (for example, 5 minutes = 300 seconds) with a short background following time. On the other hand, in an environment (for example, a place) where there is a high possibility of stopping such as a person in a library, office, factory, etc., the background follow-up time is preferably set to a longer normal learning period (for example, 20 minutes = 1200 seconds). .

  As described above, the surveillance camera system 100 according to the first embodiment includes at least one camera (for example, the omnidirectional camera CAM11,...) That images the imaging area, and the captured image AA1 or the captured image of the imaging area captured by the camera. A client PC 10 that displays on the monitor DP an edited video BB1 obtained by performing a mask process for changing at least part or all of the moving object region OB1 of the person appearing in AA1 into a mask image. In addition, the client PC 10 displays an area setting screen WD2 for setting a non-masking target area including the display area of the captured video MV1 on the monitor DP (see FIG. 7). In response to a user operation, the client PC 10 performs camera setting in which an area specified in the display area of the captured video MV1 in the area setting screen WD2 (see, for example, the frame JGA1 illustrated in FIG. 8) is not subjected to mask processing. The information is stored in the storage 15 or the memory 16 in association with the camera corresponding to the captured video MV1 (for example, the camera that distributed the captured video).

  As a result, the client PC 10 individually sets the non-target area of the mask process (MOR process) performed in the client PC 10 for each camera by a simple operation of the user so as to suit the environment of the place where the various cameras are installed. Can be set to Therefore, in order to efficiently investigate the cause of the trouble when there is any trouble, the surveillance camera system 100 captures the captured video MV1 or recorder (for example, the recorder REC1) distributed from the camera (for example, the PTZ camera CAM13). ) Can be changed to a mask image so that the person's privacy can be protected, and the visibility of the camera's captured image displayed in an area not subject to mask processing (MOR processing) can be changed. Can be prevented. In other words, the surveillance camera system 100 achieves both the protection of the privacy of the person shown in the captured image of the camera and the suppression of the deterioration of the visibility of the captured image of the camera so as to suit the environment of the place where the camera is installed. User convenience can be improved.

  Further, the client PC 10 uses the camera setting information generated for each camera to set the mask processing target area excluding the non-mask processing target area (see, for example, the frame JGA1 shown in FIG. 8) from the entire area of the captured video MV1. The edited video obtained by performing the masking process is displayed on the monitor DP. As a result, the client PC 10 can output video after ensuring the visibility of the video area portion that the user is paying attention to in the captured video, for example, and the person appearing in the video of the mask processing target area is used as the mask image. Since the display can be replaced, it is difficult to specify who is in the mask processing target area, and privacy can be ensured.

  Further, the client PC 10 displays a camera setting screen WD1 for setting at least execution or non-execution of mask processing for each camera on the monitor DP, and any of the cameras in the camera setting screen WD1 according to a user operation. When the execution of the mask processing corresponding to is designated, the camera setting screen WD1 is switched to the region setting screen WD2 corresponding to the designated camera and displayed on the monitor DP. Thereby, the user can easily set execution or non-execution of the mask process (MOR process) on the captured image captured by any camera on the camera setting screen WD1. Furthermore, when the user sets execution of mask processing (MOR processing) on the camera setting screen WD1 and presses the setting button, the region setting screen WD2 is displayed on the monitor DP, so execution of mask processing (MOR processing) is performed. And the setting of the non-target region can be performed continuously.

  Further, the client PC 10 sets a rectangular area designated in the display area of the captured video MV1 in the area setting screen WD2 as a non-maskable area by a user operation. As a result, the user can select a portion of which the visibility is to be ensured by making it a non-target area of the mask process while viewing the captured video MV1 displayed on the area setting screen WD2.

  Further, the client PC 10 sets a polygonal area (for example, a pentagonal area) designated in the display area of the captured video MV1 in the area setting screen WD2 as a non-maskable area by a user operation. As a result, the user can select a portion whose visibility is to be ensured by making it a non-target area of the mask process while viewing the captured image MV1 displayed on the area setting screen WD2 in a polygonal shape (for example, a pentagonal area). Can do.

  In addition, the client PC 10 displays a camera setting screen WD1 for setting at least execution or non-execution of the mask processing for each camera on the monitor DP, and performs imaging in the mask processing in accordance with a user operation on the camera setting screen WD1. The initial learning period for generating the background image of the video MV1 and the normal learning period for updating the background image in the mask process are stored in the storage 15 or the memory 16 in association with each camera.

  As a result, the user can perform initial learning and normal learning of the background image of the mask processing (MOR processing) performed in the client PC 10 by a simple operation of the user so as to suit the environment of the place where the various cameras are installed. The period can be individually set for each camera, and user convenience can be improved. Therefore, the client PC 10 determines the behavioral characteristics of a person existing in the vicinity (for example, a place where the movement of the person is large, a place where the movement of the person is small, depending on the environment (for example, the place) where the camera (for example, the PTZ camera CAM 13) is set The background image can be generated and updated so as to conform to (2), and the edited video after the mask processing suitable for the environment can be generated and displayed.

(Background to the contents of the second embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  For this reason, while the client PC receives different captured images captured by a plurality of cameras and separately displays them on the monitor, the setting of whether or not to perform mask processing is individually set for each captured image. It was not possible to do this, and the user-friendliness was not good. For example, when the camera is set in advance to perform mask processing, while the video distributed from the camera is displayed on the monitor of the client PC, the camera imaging process is temporarily stopped and the camera Unless the mask processing is changed to the setting that does not execute, the client PC cannot be changed to the setting that disables the mask processing in the camera during the display on the monitor.

  Therefore, in the second embodiment, when a plurality of different types of captured images are divided and displayed on a monitor, the execution or non-execution of mask processing for each captured image is simply set for each captured image, An example of a monitoring system and a video display method for improving user convenience will be described.

(Second Embodiment)
As a monitoring system according to the second embodiment, the surveillance camera system 100 described in the first embodiment will be described as an example. Since the internal configuration of the surveillance camera system 100 of the second embodiment is the same as that of the surveillance camera system 100 of the first embodiment, description of the same contents is simplified or omitted, and different contents are described.

  FIG. 15A and FIG. 15B are diagrams illustrating a display example in which captured images of a plurality of cameras are divided into three. FIG. 15C is a diagram illustrating a display example in which captured images of a plurality of cameras are divided into four.

  Assume that the client PC 10 receives different captured images distributed (transmitted) from a plurality of cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, and CAM13). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask processing (MOR processing) is performed in each camera, the processor 11 performs processing shown in FIGS. 11 and 12 in the mask processing unit 114. Then, the edited video MV2, MV3, and MV4 after mask processing in which the person in each captured video is changed (replaced) to a mask image is generated. In the video output control unit 115, the processor 11 can display the video display screen WD5a or the video display screen 5b in which the edited videos MV2, MV3, and MV4 generated by the mask processing unit 114 are divided into three and displayed on the monitor DP. Yes (see FIG. 15A or FIG. 15B).

  In the video display screen WD5a of FIG. 15A, the display area of the edited video MV2 out of the entire display area 3DV1 of the edited video is larger than the display areas of the other edited videos MV3 and MV4. In the video display screen WD5b of FIG. 15B, the display area of the edited video MV4 out of the entire display area 3DV2 of the edited video is larger than the display areas of the other edited videos MV2 and MV3. The processor 11 is designated with the three-divided display form shown in FIG. 15A in response to a user operation on the display switching box CGB1 shown in FIG. 15A, FIG. 15B, or FIG. The display mode (for example, the 3-part display mode shown in FIG. 15B or the 4-part display mode shown in FIG. 15C) can be switched. However, in order to avoid complication of the drawings of FIGS. 15A and 15B, the display switching box CGB1 is not shown in FIGS. 15A and 15B. In addition, the processor 11 can set by user operation which edit video display area can be maximized in the three-divided display mode shown in FIGS. 15A and 15B.

  Assume that the client PC 10 receives different captured images distributed (transmitted) from a plurality of cameras (for example, the omnidirectional camera CAM11, the PTZ cameras CAM12, CAM13, and the fixed camera CAM14). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask processing (MOR processing) is performed in each camera, the processor 11 performs processing shown in FIGS. 11 and 12 in the mask processing unit 114. Then, the edited video MV2, MV3, MV4, and MV5 after mask processing in which the person in each captured video is changed (replaced) to a mask image are generated. In the video output control unit 115, the processor 11 can display on the monitor DP a video display screen WD6 in which the edited videos MV2, MV3, MV4, and MV5 generated by the mask processing unit 114 are divided into four (FIG. 5). 15 (C)).

  In the video display screen WD6 in FIG. 15C, the display areas of the four edited videos MV2, MV3, MV4, and MV5 are equal in the entire edited video display area 4DV1.

  In FIG. 15A, FIG. 15B, and FIG. 15C, all the images displayed on the image display screens WD5a, WD5b, and WD6 are the edited images after the processor 11 performs the masking process. It is shown. However, at least one video may be a captured video that has not been subjected to mask processing. This is caused by setting information on whether or not to perform mask processing (MOR processing) for each camera on the camera setting screen WD1. In other words, when the client PC 10 displays a plurality of videos in a divided manner, the client PC 10 executes an edited video or a mask process in which a mask process corresponding to each camera is executed according to the camera setting information set by the user on the camera setting screen WD1. It is only necessary to divide and display the captured video as it is received.

  For example, in the case of a four-divided display mode, one captured image distributed from a camera installed near the entrance of the user's company is not subjected to mask processing by the client PC 10 and is directly displayed as a video display screen (for example, a video display screen WD6). Is displayed. Furthermore, the three captured images distributed from other (three) cameras installed in the company building are each subjected to mask processing, and the plurality of edited videos obtained by the mask processing are the same image display screen. May be displayed in the remaining display area. Accordingly, a user (for example, a supervisor) can appropriately determine and set the specific necessity of the person appearing at each location according to the circumstances of the installation location of each camera. For example, the user can grasp the face of the person in the captured image of the place where the user wants to grasp the face. On the other hand, the person's face becomes a mask image in a place where the face does not need to be grasped or where the face should not be grasped. Therefore, the privacy protection of the person can be observed.

  Next, a first operation procedure of split display of the client PC 10 of this embodiment will be described with reference to FIG. FIG. 16 is a flowchart illustrating in detail an example of an operation procedure performed in the client PC 10 when displaying captured images of a plurality of cameras according to the second embodiment. The process illustrated in FIG. 16 is performed in the client PC 10 for each camera that distributes the captured video to the client PC 10.

  In FIG. 16, the mask processing setting unit 113 of the processor 11 determines whether or not the client PC 10 has acquired the captured video of the camera to be divided and displayed (S31). If the mask processing setting unit 113 determines that the captured video of the camera that is the target of split display has not been acquired (NO in S31), the mask processing setting unit 113 waits until the captured video of the camera that is the target of split display is acquired. (S32). In this case, the video output control unit 115 does not display the captured video from the camera that is the target of the split display, and therefore displays nothing on the monitor DP. That is, the display area in the video display screen of the monitor DP is blank. Alternatively, if there is a captured video that the client PC 10 has already received, the video output control unit 115 may display the captured video in a display area in the video display screen of the monitor DP.

  When the mask processing setting unit 113 determines that the captured video of the camera to be divided is acquired (S31, YES), the storage 15 or the memory 16 is based on the camera identification information included in the captured video. Is read with reference to the record of the corresponding camera in the camera setting information stored in (S33). The mask process setting unit 113 determines whether or not the setting to execute the mask process (MOR process) is performed on the captured video in the read record (S34).

  When it is determined that the mask record (MOR process) is set to be executed in the read record (S34, YES), the mask process setting unit 113 notifies the mask process unit 114 to that effect. In response to the notification from the mask processing setting unit 113, the mask processing unit 114 performs the mask processing (MOR processing) shown in FIGS. 11 and 12 on the captured image from the corresponding camera, and performs the mask processing after the mask processing. An edited video is generated (S35). The video output control unit 115 displays the edited video generated in step S35 in a corresponding display area (for example, the display area of the edited video MV2 in FIG. 15A) in the video display screen of the monitor DP (S35).

  On the other hand, when the mask processing setting unit 113 determines that the mask record (MOR process) is not set in the read record (S34, NO), the original captured image (that is, the mask process is performed). The image output control unit 115 is notified that an imaged image obtained by non-imaging imaging (the same applies hereinafter) is displayed (S36). In response to the notification from the mask processing setting unit 113, the video output control unit 115 converts the original captured video received from the camera by the client PC 10 from the corresponding display area (for example, FIG. 15A) in the video display screen of the monitor DP. It is displayed on the display area of the edited video MV2 (S36).

  If the divided display of the client PC 10 is finished after step S35 or step S36 (S37, YES), the processing of the processor 11 shown in FIG. 16 is finished. On the other hand, if the divided display of the client PC 10 does not end after steps S35 and S36 (S37, NO), the processing of the processor 11 returns to step S31.

  FIG. 17 is a diagram illustrating a display example in which a captured image of a camera and a plurality of different captured images of the same camera are divided into four. The client PC 10 receives the current captured video (live video) distributed (transmitted) from one camera (for example, the fixed camera CAM 14), and further, a plurality of different recordings distributed from one recorder (for example, the recorder REC1). Assume that video is received. The plurality of different recorded videos are recorded videos captured in the past by a camera (for example, the fixed camera CAM 14) that distributes the current captured video (live video). When it is set on the camera setting screen WD1 (see FIG. 6) that the mask process (MOR process) is performed in this camera, the processor 11 causes the mask processing unit 114 to follow the processes shown in FIGS. Edited video MV6, edited video MV7, MV8, and MV9 after mask processing in which a person in each captured video and recorded video is changed (replaced) to a mask image are generated. In the video output control unit 115, the processor 11 can display on the monitor DP a video display screen WD7 that is obtained by dividing the edited video MV6, edited video MV7, MV8, and MV9 generated by the mask processing unit 114 into four. Yes (see FIG. 17).

  In the video display screen WD7 of FIG. 17, the display area of four videos (specifically, the edited video MV6, the edited video MV7, MV8, and MV9) out of the entire display area 4DV2 of the edited video and the edited video. It is. The edited recording videos MV7, MV8, and MV9 are, for example, those having the oldest or newest shooting date in the order of the edited recording video MV7, the edited recording video MV8, and the edited recording video MV9. As a result, the client PC 10 can divide and display on the monitor DP the edited video and the edited recorded video obtained by masking (MOR processing) the current captured video captured by the same camera and the recorded video captured in the past. Therefore, it is possible to effectively support the tracking of the user's person and the grasping of the action.

  Next, a second operation procedure of split display of the client PC 10 of this embodiment will be described with reference to FIG. FIG. 18 is a flowchart illustrating in detail an example of an operation procedure performed in the client PC 10 when displaying a captured image of a camera and a recorded image from a recorder in the second embodiment. The process illustrated in FIG. 18 is performed in the client PC 10 for each camera that distributes the current captured video (live video) to the client PC 10 or for each recorder that distributes past recorded video captured by the camera.

  Further, when the process shown in FIG. 18 and the process shown in FIG. 16 are the same, the same step number as the step number of the process shown in FIG. explain.

  In FIG. 18, the mask processing setting unit 113 of the processor 11 determines whether the client PC 10 has acquired the captured video from the camera that is the target of split display or the recorded video from the recorder (S31A). When it is determined that the captured video from the camera or the recorded video from the recorder is not acquired (S31A, NO), the mask processing setting unit 113 acquires the captured video from the camera or the recorded video from the recorder. Wait (S32). The client PC 10 can receive and acquire the corresponding recorded video distributed from the recorder when requesting the corresponding recorded video from the recorder by the user's designation of the imaging date and time. The same applies to each embodiment. It is.

  When the mask processing setting unit 113 determines that the captured video from the camera that is the target of split display or the recorded video from the recorder is acquired (S31A, YES), identification of the camera included in the captured video or the recorded video Based on the information, the camera setting information stored in the storage 15 or the memory 16 is read with reference to the corresponding camera record (S33A). The mask process setting unit 113 determines whether or not the setting to execute the mask process (MOR process) is performed on the captured video or the recorded video in the read record (S34A).

  When it is determined that the mask record (MOR process) is set to be executed in the read record (S34A, YES), the mask process setting unit 113 notifies the mask process unit 114 to that effect. In response to the notification from the mask processing setting unit 113, the mask processing unit 114 performs the mask processing (MOR processing) illustrated in FIGS. 11 and 12 on the captured video from the corresponding camera or the recorded video from the recorder. Then, the edited video or edited video recorded after the mask process is generated (S35A). The video output control unit 115 displays the edited video or the edited recorded video generated in step S35 in a corresponding display area in the video display screen of the monitor DP (for example, the display area of the edited video MV6 or the edited recorded video MV7 in FIG. 17). (S35A).

  On the other hand, when it is determined that the mask record (MOR process) is not set to be executed in the read record (S34A, NO), the mask process setting unit 113 displays the original captured video or recorded video. Is notified to the video output control unit 115 (S36A). In response to the notification from the mask processing setting unit 113, the video output control unit 115 displays the original captured video or recorded video received by the client PC 10 from the camera in a corresponding display area (for example, FIG. 17) in the video display screen of the monitor DP. It is displayed in the display area of the edited video MV6 or edited video MV7 (S36A).

  As described above, the surveillance camera system 100 according to the second embodiment includes a plurality of cameras (for example, the omnidirectional cameras CAM11,...) That capture an imaging area and a moving body that appears in the captured images of the imaging areas captured by the individual cameras. A client PC 10 that holds setting information for execution or non-execution of mask processing for changing at least a part of (for example, a person) into a mask image. In addition, the client PC 10 uses an individual edited video obtained by performing mask processing on individual captured images captured by a plurality of cameras set to execute mask processing, or a single camera set to execute mask processing. The edited video obtained by masking the individual captured video and the individual captured video captured by at least one camera set not to execute the mask processing are divided and displayed on the monitor DP.

  Similarly to the first embodiment, the client PC 10 according to the second embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, when the client PC 10 divides and displays a plurality of different captured images on the monitor DP, the masking process is executed or not performed for each camera as a distribution source (transmission source) of each captured image received by the client PC 10. Since execution settings can be easily set, user convenience can be improved. Accordingly, the client PC 10 can change the setting for performing mask processing for each captured image to be divided and displayed so as to adapt to the circumstances of the camera installation location. It can contribute to business and high-precision marketing analysis.

  The surveillance camera system 100 according to the second embodiment further includes a recorder (for example, REC1) that stores a captured image of an imaging area captured by each camera in association with an imaging date and time as a recorded video. The client PC 10 captures an edited image obtained by masking the captured image captured by the camera set to execute the mask process and an image captured at different past imaging dates and times by the camera set to execute the mask process. Individual edited and recorded videos obtained by masking a plurality of recorded videos are divided and displayed on the monitor DP.

  As a result, the client PC 10 can divide and display on the monitor DP the edited video and the edited recorded video obtained by masking (MOR processing) the current captured video captured by the same camera and the recorded video captured in the past. Therefore, it is possible to effectively support, for example, tracking of a person appearing in the vicinity of the installation location of the camera or grasping an action in monitoring work or marketing analysis.

(Background to the contents of the third embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  When the mask processing cannot be performed in the client PC, the mask processing cannot be performed on a captured image captured by a camera without a function for executing the mask processing (hereinafter referred to as “mask function”). It is difficult to add a mask function to an existing camera without a mask function. For this reason, it was difficult to improve monitoring efficiency and marketing analysis accuracy.

  Therefore, in the third embodiment, mask processing is appropriately performed on a captured image captured by a camera that does not have a mask function without adding a mask function separately to a camera that does not have a mask function. An example of a monitoring system and a video display method for improving monitoring efficiency and marketing analysis accuracy will be described.

(Third embodiment)
The monitoring camera system 100 described in the first embodiment will be described as an example of a monitoring system according to the third embodiment. Since the internal configuration of the surveillance camera system 100 of the third embodiment is the same as that of the surveillance camera system 100 of the first embodiment, description of the same contents is simplified or omitted, and different contents are described.

  In the third and fourth embodiments, in the surveillance camera system 100 shown in FIG. 1, at least the fixed camera CAM 14 has the mask function described above. That is, the fixed camera CAM 14 can execute the mask process (MOR process) shown in FIGS. 11 and 12 on the captured image captured by itself. In other words, in each of the third and fourth embodiments, the processor C11 of the fixed camera CAM 14 includes the image input unit 111, the background image processing unit 112, and the mask processing setting unit 113 in the processor 11 of the client PC 10 illustrated in FIG. In addition, it is assumed that the same components as the mask processing unit 114 are included.

  Note that the fixed camera CAM 14 performs a mask process equivalent to the mask process in the client PC 10 (that is, the mask process performed in the region excluding the non-target region for the mask process based on the user operation described in the first embodiment). Even if it cannot be performed, for example, the mask processing described in Patent Document 1 may be performed. Further, in each of the third and fourth embodiments, the mask camera is described as one unit. However, the present invention is not limited to only one unit, and a plurality of cameras may operate as a mask camera.

  In each of the third and fourth embodiments, a camera having a mask function like the fixed camera CAM14 is referred to as a “mask camera” for convenience, and a mask function like other cameras excluding the fixed camera CAM14. For convenience, a camera that does not have a camera is called a “normal camera”.

  Next, an operation procedure related to setting of mask processing and display of captured video of the client PC 10 according to the present embodiment will be described with reference to FIGS. FIG. 19 is a sequence diagram illustrating in detail an example of an operation procedure for setting a mask process (MOR process) of the client PC 10 for the mask camera and the normal camera and displaying a captured image in the third embodiment. FIG. 20 is a diagram illustrating an example of a camera setting screen in the third and fourth embodiments. FIG. 21 is a diagram illustrating an example of four-division display of an edited video that is displayed on the monitor DP of the client PC 10 and in which the captured video is masked.

  In FIG. 19, when there is an operation for starting operation of the client PC 10 by a user operation (S41), the client PC 10 executes mask processing in the mask camera by a user operation on the camera setting screen WD1A (see FIG. 20), for example ( When ON or non-execution (OFF) is designated, the camera setting information in which the mask process is designated to be executed or non-execution in the mask camera and an instruction to set the camera setting information are transmitted to the mask camera ( S42).

  Here, the camera setting screen WD1A shown in FIG. 20 will be described. Descriptions of items overlapping or similar to the camera setting screen WD1 shown in FIG. 6 are simplified or omitted, and different contents will be described.

  In FIG. 20, a part of the camera setting screen WD1A is extracted and shown. The mask processing setting unit 113 displays the camera setting screen WD1A on the monitor DP, for example, by a user operation. In this camera setting screen WD1A, at least an IP address, for each individual camera of, for example, a maximum of 64 cameras (that is, an omnidirectional camera, a PTZ camera, and a fixed camera) is operated by the user using the operation unit 13. The mask process (MOR process), the setting of the mask process (MOR process), the initial learning period, and the normal learning period can be set in association with each other. Further, by switching the four tabs (camera 1-16, camera 17-32, camera 33-48, camera 49-64) provided on the camera setting screen WD1A, for the corresponding camera in units of 16 cameras, Information on at least an IP address, mask processing (MOR processing), setting of mask processing (MOR processing), initial learning period, and normal learning period can be set.

  In the mask process (MOR process) column CLM2A, the mask process is not executed (OFF), the mask process (CAM-MOR) is executed by the mask camera, or the mask process (PC-MOR) is executed by the client PC 10 (ON). ) Is selected (see frame WK2A). In other words, execution (ON) of the mask process on the camera setting screen WD1 in FIG. 6 is executed on the camera setting screen WD1 in FIG. 20 by executing the mask process (CAM-MOR) on the mask camera or on the client PC 10. This corresponds to execution (ON) of processing (PC-MOR).

  In the column CLM3 for setting the mask process (MOR process), when the mask process (CAM-MOR) in the mask camera or the mask process (PC-MOR) in the client PC 10 is selected in the column CLM2A, the mask process is performed. A setting button (see frame WK1) for displaying an area setting screen WD2 for setting the non-target area is displayed. However, the setting button of the column CLM3 is not displayed so as to be able to be pressed for a camera for which non-execution of mask processing (OFF) is selected in the column CLM2A (for example, the fifth column, the eighth column from the page of FIG. 20 in the column CLM3, (See the dotted lines on the 10th and 15th stages).

  In FIG. 19, the mask camera follows the instruction transmitted from the client PC 10 in step S42, and executes (ON) or disables mask processing specified by the camera setting information based on the camera setting information transmitted together with the instruction. Execution (OFF) is set, and a response to the effect is returned to the client PC 10 (S43).

  The client PC 10 instructs the normal camera and the mask camera to start imaging and distribute the captured video (S44). The normal camera starts imaging in accordance with an instruction from the client PC 10 (S45), and thereafter continues to distribute the captured video obtained by imaging (S47). The client PC 10 receives a captured image distributed from a normal camera.

  The mask camera starts imaging in accordance with an instruction from the client PC 10 (S46), and further determines whether the mask process set in step S43 is executed (ON) or not executed (OFF). That is, when the mask camera is set to execute (ON) in step S43, the mask camera displays the edited video obtained by performing mask processing (see FIGS. 11 and 12) on the captured video obtained by imaging. Distribution is performed toward the PC 10 and the distribution is continued (S48). On the other hand, if the mask process is set to non-execution (OFF) in step S43, the mask camera distributes the captured image obtained by the imaging as it is to the client PC 10, and continues the distribution (S48).

  The client PC 10 receives the received image when it is set in the camera setting information corresponding to the normal camera or the mask camera that the mask camera executes the mask process or the mask process itself is not executed. The video is displayed on the monitor DP as it is (S49, see FIG. 21).

  On the other hand, if the client PC 10 is set to execute the mask process in the camera setting information corresponding to the normal camera or the mask camera, the client PC 10 performs the mask process (see FIG. 11 and FIG. 12), and the edited video obtained by the mask process is displayed on the monitor DP (S49, see FIG. 21).

  In the video display screen WD7A of FIG. 21, the display areas of four videos (specifically, the edited videos MV6A, MV7A, MV8A, and MV9A) are equal in the entire display area of the edited video (see FIG. 17). The edited videos MV6A and MV9A are, for example, the edited videos obtained as a result of the masked image captured by the mask camera being displayed on the client PC 10 as they are. The edited video MV7A is, for example, displayed on the edited video obtained as a result of the captured video captured by the mask camera being masked by the client PC 10 without being masked by the mask camera. The edited video MV8A is displayed, for example, as an edited video obtained as a result of masking the captured video captured by the normal camera in the client PC 10. As a result, the client PC 10 can select the mask camera or the client PC 10 itself according to the execution (ON) or non-execution (OFF) setting of the mask process corresponding to the normal camera having no mask function or the mask camera having the mask function. Since the edited video that has undergone the mask processing can be divided and displayed on the monitor DP, it is possible to contribute to the efficiency of monitoring work and marketing analysis while ensuring the privacy of the person in the captured video captured by each camera.

  As described above, the surveillance camera system 100 according to the third embodiment captures an imaging area, and executes a mask process for changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area to a mask image. A normal camera (for example, an omnidirectional camera CAM11) that cannot be performed, and a client PC 10 that holds setting information on whether or not to perform mask processing on a captured image. When the client PC 10 is set to execute mask processing on the captured video, the client PC 10 performs mask processing on the captured video and displays the edited video obtained by the mask processing on the monitor DP. On the other hand, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution of the mask processing in the client PC 10 for the captured image is set.

  Similarly to the first embodiment, the client PC 10 of the third embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, the client PC 10 appropriately performs mask processing on the captured image captured by the normal camera not having the mask function without separately adding a mask function to the normal camera not having the mask function. , Monitoring efficiency and marketing analysis accuracy can be improved.

  Further, the surveillance camera system 100 according to the third embodiment can execute a mask process for capturing an imaging area and changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area into a mask image. A mask camera (for example, a fixed camera CAM14). The client PC 10 holds setting information indicating whether or not to execute mask processing in the mask camera, and when the execution of mask processing in the mask camera is set, the edited video based on the mask processing in the mask camera is directly displayed on the monitor DP. indicate.

  As a result, when the mask processing is performed by setting the execution of the mask processing for the mask camera that already has the mask function, the client PC 10 receives the edited video after the mask processing generated by the mask camera and displays it as it is. As a result, the execution of mask processing in the client PC 10 can be omitted, the processing load can be reduced, and load distribution can be achieved.

  Further, the surveillance camera system 100 according to the third embodiment can execute a mask process for capturing an imaging area and changing at least a part of a moving body (for example, a person) appearing in a captured image of the imaging area into a mask image. A mask camera (for example, a fixed camera CAM14). The client PC 10 holds setting information on whether or not to execute mask processing in the mask camera, and when the mask processing in the mask camera is set not to be executed, the captured image is masked and obtained by this mask processing. The edited video is displayed on the monitor DP.

  Thereby, the client PC 10 has a mask function superior to the mask processing in the mask camera, for example (for example, excluding the non-maskable area for the mask processing based on the user operation described in the first embodiment) When the mask processing performed in the area can be performed), the edited video after the mask processing according to the user's preference can be obtained rather than the edited video obtained by the mask processing performed by the mask camera, and user convenience can be improved. .

(Background to the contents of the fourth embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed.

  That is, in Patent Document 1, since the mask process can be performed only by the camera, the live image (captured image) currently being captured can be masked by the camera, but the past captured image (which is already stored in the recorder) Hereinafter, the mask process could not be performed on “recorded video”.

  For this reason, when a recorded video of an event is verified afterwards, for example, an authorized person (manager, etc.) and an unauthorized person (general user) access to protect the privacy of the subject person. It was necessary to prepare two types of recorded video. For example, an authorized person can browse a recorded video in which a person in the recorded video is not masked, and a general user browses a recorded video in which a person in the recorded video is masked. Under such an operation, the recording capacity of the recorder is inevitably increased in order to record the captured image of the same scene. Particularly in recent years, the image quality of captured images has been improved (for example, 4K), and the amount of captured image data stored in the recorder is enormous.

  In other words, in order to prepare the two types of recorded video described above, one camera is not sufficient, for example, two cameras (specifically, a function for executing mask processing (hereinafter referred to as “mask function”). ) And a camera that does not have a mask function. For this reason, since the number of parts increases in system construction, the cost increase cannot be avoided, and the system configuration is complicated. Further, if the above-described client PC cannot perform mask processing, the present invention is not limited to the case where two types of recorded video are required only by the presence / absence of authority (whether or not the authority is the same as that of the above-mentioned manager and general user). Depending on the content and type of the recorded video, two types of recorded video, that is, a recorded video that has been masked and a recorded video that has not been masked, may be required.

  Therefore, in the fourth embodiment, when browsing a recorded video of an event that has been captured by the camera and is already stored in the recorder in the past, a mask process is performed or masked using one data of the recorded video. An example of a monitoring system and an image display method for easily selecting whether to perform processing and improving the monitoring efficiency and marketing analysis accuracy will be described.

(Fourth embodiment)
The monitoring camera system 100 described in the third embodiment will be described as an example of a monitoring system according to the fourth embodiment. Since the internal configuration of the monitoring camera system 100 of the fourth embodiment is the same as that of the monitoring camera system 100 of the third embodiment, the description of the same contents will be simplified or omitted, and different contents will be described.

  In the fourth embodiment, in the surveillance camera system 100 shown in FIG. 1, each recorder to which at least one camera is connected (that is, the recorders REC1, REC2, and REC3) has at least one connected to each recorder. Paired (in other words, tied) with two cameras. More specifically, an omnidirectional camera CAM11, PTZ cameras CAM12, CAM13, and a fixed camera CAM14 are paired (linked) to the recorder REC1. A PTZ camera CAM21 and a fixed camera CAM22 are paired (linked) to the recorder REC2. The recorder REC3 is paired (linked) with an omnidirectional camera CAM31, a PTZ camera CAM32, a CAM33, and a fixed camera CAM34. The pairing relationship between the at least one camera and the recorder is not limited to the fourth embodiment, and may be similarly applied to the first, second, and third embodiments described above.

  FIG. 22 is a diagram showing an example of the recorder setting screen WD8. The processor 11 of the client PC 10 displays the recorder setting screen WD8 on the monitor DP in response to a user operation for displaying the recorder setting screen WD8.

  On the recorder setting screen WD8, the recorder is set by a user operation. For example, the IP address of the recorder, the HTTP (Hypertext Transfer Protocol) port number, the name, the FTP (File Transfer Protocol) port number, the time zone, and the setup of at least one paired camera are performed. FIG. 22 exemplarily shows respective devices, models, and IP addresses for four recorders, for example.

  In the recorder setting screen WD8 displayed on the monitor DP, a record (frame WK6) for specifying one of the recorders “Device mng.” (Device manager, see frame WK4), “Recorder” (see frame WK5), by user operation. When the selection is made by pressing in the order of “reference”, the processor 11 switches the display so that the Edit button (see the frame WK7) that cannot be pressed before the selection can be pressed.

  When the Edit button (see frame WK7) is pressed by a user operation, the processor 11 sets a camera setting screen (see FIG. 20) related to at least one camera paired with the recorder of the selected record (see frame WK6). ) Is displayed on the monitor DP. Items that can be set on the camera setting screen (see FIG. 20) are the same as those described in the third embodiment (see FIG. 20), and thus detailed description thereof is omitted.

  Next, an operation procedure related to the mask processing setting of the client PC 10 of the present embodiment will be described with reference to FIGS. 23 (A) and 23 (B). FIG. 23A is a sequence diagram illustrating in detail a first example of an operation procedure for setting mask processing (MOR processing) in the fourth embodiment. FIG. 23B is a sequence diagram illustrating in detail a second example of the operation procedure for setting mask processing (MOR processing) in the fourth embodiment.

  In FIG. 23A, when there is an operation for starting device setting of the client PC 10 by a user operation (S51), the client PC 10 is based on, for example, pressing the Edit button (see frame WK7) on the recorder setting screen WD8. The displayed camera setting screen (see FIG. 20) is displayed on the monitor DP. The client PC 10 executes (ON) or does not perform mask processing on the mask camera (that is, the fixed camera CAM 14 capable of executing the mask function) paired with the recorder (for example, the recorder REC1) by a user operation on the camera setting screen. The recorder is instructed to execute (OFF) (S52).

  The recorder (for example, the recorder REC1) executes (ON) or does not execute the mask processing for the mask camera (that is, the fixed camera CAM14) paired with itself (for example, the recorder REC1) in accordance with the instruction from the client PC 10 in step S52. The mask camera is instructed to set (OFF) (S53). The mask camera sets execution (ON) or non-execution (OFF) of the mask process according to the instruction from the recorder in step S53 (S54), and returns a response to the effect that the setting is completed to the recorder. When the recorder receives a response indicating the completion of setting from the mask camera, the recorder similarly returns a response indicating the completion of setting to the client PC 10.

  Thereby, for example, when the client PC 10 and the mask camera do not exist in the same base or store, the client PC 10 cannot set the mask processing directly for the mask camera, but the client PC 10 and the mask camera By interposing a recorder that mediates the data transmission / reception, the mask processing from the client PC 10 to the mask camera can be set smoothly.

  In FIG. 23B, when there is an operation for starting device setting of the client PC 10 by a user operation (S51), the client PC 10 is based on, for example, pressing the Edit button (see frame WK7) on the recorder setting screen WD8. The displayed camera setting screen (see FIG. 20) is displayed on the monitor DP. The client PC 10 executes (ON) or does not perform mask processing on the mask camera (that is, the fixed camera CAM 14 capable of executing the mask function) paired with the recorder (for example, the recorder REC1) by a user operation on the camera setting screen. The execution (OFF) setting is instructed directly to the mask camera (S52A).

  The mask camera sets execution (ON) or non-execution (OFF) of mask processing in accordance with the instruction from the client PC 10 in step S52A (S54A), and directly returns a response indicating that the setting has been completed to the client PC 10.

  Thereby, for example, when the client PC 10 and the mask camera exist in the same base or store, the client PC 10 smoothly and easily sets the mask processing directly on the mask camera without going through the recorder. be able to.

  Next, an operation procedure related to display of a captured image after setting of mask processing of the client PC 10 of the present embodiment will be described with reference to FIGS. FIG. 24 is a sequence diagram illustrating in detail a first example of an operation procedure regarding acquisition and display of a captured image in a client PC after the start of operation according to the fourth embodiment. FIG. 25 is a sequence diagram illustrating in detail a second example of an operation procedure regarding acquisition and display of a captured image in a client PC after the start of operation according to the fourth embodiment. FIG. 26 is a flowchart for explaining in detail an example of the operation procedure of the display of the client PC in step S69 of FIGS.

  In the description of FIG. 25, when the process shown in FIG. 25 and the process shown in FIG. 24 are the same, the same step number is assigned and the description is simplified or omitted, and different contents will be described.

  In FIG. 24, when there is an operation for starting the operation of the client PC 10 by a user operation (S61), the client PC 10 instructs the recorder to start distributing video (that is, to record (S62)).

  In accordance with the instruction of the client PC 10 in step S62, the recorder starts distributing the captured video or edited video, or instructs the mask camera and the normal camera to start distributing the captured video or edited video (S63). The mask camera starts imaging in accordance with the instruction of the recorder in step S63 (S64). Similarly, the normal camera starts imaging according to the instruction of the recorder in step S63 (S65).

  Since the normal camera cannot perform the masking process, after starting imaging, distribution of the captured video obtained by imaging to the recorder is continued (S66). When the recorder receives the captured video (live video) distributed from the normal camera in step S66, the recorder stores it in the storage R14 (S68).

  After the start of imaging, the mask camera masks the captured image obtained by the start of imaging in step S64 when the execution (ON) of the mask process is set in FIG. 23 (A) or FIG. 23 (B). Distribution of the edited video obtained by the processing (for example, see FIGS. 11 and 12) to the recorder is continued (S67). On the other hand, the mask camera, after starting imaging, when the non-execution (OFF) of the mask processing is set in FIG. 23A or FIG. 23B, imaging obtained by starting imaging in step S64. Distribution of the video to the recorder is continued (S67). Note that when the recorder receives the captured video (live video) or the edited video distributed from the mask camera in step S67, the recorder stores it in the storage R14 (S68).

  In step S62, for example, when a user operation for distributing a recorded video stored in the recorder in the past is performed, an instruction to start video distribution is given by the user operation (in other words, an imaging date and time desired by the user). And recording date and time) are included. In this case, the recorder reads from the storage R14 the recorded video imaged at the imaging date and time transmitted from the client PC 10 and distributes it to the client PC 10 (S68).

  In step S62, for example, when a user operation for distributing a captured image captured by the camera is performed, the captured image (live image) currently captured by the camera is distributed in accordance with an instruction to start distributing the image. The In this case, the recorder distributes the captured video distributed from the normal camera in step S66 and the captured video or edited video distributed from the mask camera in step S67 to the client PC 10, and continues the distribution (S68).

  The client PC 10 executes the mask processing set for each type of camera paired (linked) with the captured video (live video), recorded video, or edited video distributed from the recorder in step S68 ( The video is displayed on the monitor DP according to the setting of ON or non-execution (OFF) (S69). Details of step S69 will be described later with reference to FIG.

  In FIG. 25, since the normal camera cannot perform the masking process, after starting the imaging, the distribution of the captured video obtained by the imaging to the recorder and the client PC 10 is continued (S66A). When the recorder receives the captured video (live video) distributed from the normal camera in step S66A, the recorder stores it in the storage R14 (S68A).

  After the start of imaging, the mask camera masks the captured image obtained by the start of imaging in step S64 when the execution (ON) of the mask process is set in FIG. 23 (A) or FIG. 23 (B). The distribution of the edited video obtained by the processing (for example, see FIGS. 11 and 12) to the recorder and the client PC 10 is continued (S67A). On the other hand, the mask camera, after starting imaging, when the non-execution (OFF) of the mask processing is set in FIG. 23A or FIG. 23B, imaging obtained by starting imaging in step S64. The distribution of the video to the recorder and the client PC 10 is continued (S67A). When the recorder receives the captured video (live video) or edited video distributed from the mask camera in step S67A, the recorder stores it in the storage R14 (S68A).

  In step S62, for example, when a user operation for distributing a recorded video stored in the recorder in the past is performed, an instruction to start video distribution is given by the user operation (in other words, an imaging date and time desired by the user). And recording date and time) are included. In this case, the recorder reads out the recorded video imaged at the imaging date and time transmitted from the client PC 10 from the storage R14 and distributes it to the client PC 10 (S68A).

  The client PC 10 executes the mask processing set for each type of camera paired (linked) with the captured video (live video), recorded video, or edited video distributed from the recorder in step S68 ( The video is displayed on the monitor DP according to the setting of ON or non-execution (OFF) (S69). Details of step S69 will be described later with reference to FIG.

  In FIG. 26, the client PC 10 receives and acquires the video distributed from any of the recorder, the mask camera, and the normal camera in step S68 or step S68A (S71).

  The client PC 10 reads the camera setting information (see FIGS. 22 and 20) stored in the storage 15 or the memory 16, and the camera paired with the recorder of the video distribution source acquired in step S71 or the video of the video. Reference is made to the setting of execution or non-execution of mask processing for the distribution source camera (S72).

  If the client PC 10 determines that the mask processing non-execution (OFF) is set in the camera setting information, the video (specifically, the captured video or the recorded video) acquired in step S71 is directly monitored DP (S73).

  If the client PC 10 determines that the execution (ON) of the mask process in the corresponding camera is set in the camera setting information, the image acquired in step S71 (specifically, the mask process is performed by the mask camera). The edited video that has been applied is displayed on the monitor DP as it is (S73).

  If the client PC 10 determines that execution of mask processing in the client PC 10 is set for the corresponding camera in the camera setting information, the video (specifically, captured video or edited video) acquired in step S71. Is masked (FIGS. 11 and 12), and the edited video obtained by the masking is displayed on the monitor DP (S74). However, when the client PC 10 receives the edited video distributed from the recorder in step S71, if the edited video data includes information indicating that the mask processing has been performed by the mask camera, the client PC 10 May be preferentially displayed on the monitor DP without masking the received edited video without taking into account the contents of the current camera setting information. As a result, the client PC 10 can omit the execution of further mask processing on the edited video that has already been masked in another device (that is, the mask camera), and can reduce the processing load on the client PC 10. Degradation of the visibility of the edited video can be suppressed.

  As described above, the surveillance camera system 100 according to the fourth embodiment is associated with at least one camera (for example, the omnidirectional camera CAM11, the fixed camera CAM14,...) That images the imaging area, and captures the imaging area. Execution of mask processing for changing at least a part of a moving object (for example, a person) appearing in the captured image in the imaging area to a mask image, or a recorder (for example, the recorder REC1) that stores the video as a recorded video in association with the imaging date And a client PC 10 that holds non-executable setting information for each camera associated with the recorder. For example, when execution of mask processing is set in response to designation of an imaging date and time by a user operation, the client PC 10 displays an edited video image obtained by masking a recorded video imaged at the designated imaging date and time. Display on monitor DP. On the other hand, when the non-execution of the mask process is set, the client PC 10 displays the recorded video captured at the designated imaging date and time as it is on the monitor DP.

  Similarly to the first embodiment, the client PC 10 of the fourth embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, when the client PC 10 browses a recorded video of an event that has been captured by the camera in the past and is already stored in the recorder, the client PC 10 performs a mask process in the client PC 10 using one piece of the recorded video. It is possible to easily select whether or not to perform mask processing, and it is possible to improve monitoring efficiency and marketing analysis accuracy.

  In the surveillance camera system 100 of the fourth embodiment, at least one camera includes a normal camera (for example, an omnidirectional camera CAM11) that cannot perform mask processing. The client PC 10 displays the edited video obtained by masking the captured video on the monitor DP when the execution (ON) of the mask processing for the captured video captured by the normal camera is set. On the other hand, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution (OFF) of the mask processing for the captured image captured by the normal camera is set.

  Thereby, when the client PC 10 receives the current captured video (live video) captured by the normal camera, depending on the setting of execution or non-execution of the mask processing on the captured image by the normal camera, Captured video or edited video that matches the user's preference can be displayed on the monitor DP, which can contribute to improving the accuracy of user monitoring operations and marketing analysis.

  In the surveillance camera system 100 of the fourth embodiment, at least one camera includes a mask camera (for example, a fixed camera CAM 14) capable of performing mask processing. When execution of mask processing in the mask camera is set for the captured image captured by the mask camera, the client PC 10 directly displays the edited image in which the captured image is masked by the mask camera on the monitor DP. On the other hand, when execution of mask processing in the client PC 10 is set for the captured image captured by the mask camera, the client PC 10 masks the captured image and displays the resulting edited image on the monitor DP. .

  As a result, when the mask process is performed by setting the execution of the mask process for the mask camera that already has the mask function, the client PC 10 receives the edited video after the mask process generated by the mask camera and displays it as it is. The execution of the mask process in the client PC 10 can be omitted, the processing load can be reduced, and the load can be distributed. Further, when the mask processing is performed by setting the execution of the mask processing in the client PC 10, the client PC 10 has a mask function superior to, for example, the mask processing in the mask camera (for example, the user described in the first embodiment, The edited video obtained by performing mask processing in a region excluding the non-target region for mask processing based on the operation) can be obtained, and the video suitable for the user's needs can be confirmed.

  Further, the client PC 10 displays the captured image as it is on the monitor DP when the non-execution of the mask process for the current captured image (live image) captured by the mask camera is set. This allows the user to check the current live video when it is necessary to view the current live video as it is due to, for example, event conditions or restrictions, and to check the video that meets the user's needs. Can do.

(First process leading to the contents of the fifth embodiment)
According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed. Many existing surveillance cameras do not have a function (mask function) capable of executing the mask processing of Patent Document 1.

  Here, for example, in the user's monitoring work, a plurality of monitoring cameras are installed in the monitoring area, and the monitor of the client PC that has received the latest captured video distributed from each monitoring camera, by user operation, It is assumed that the respective captured images are switched and displayed in order. Each time the client PC switches the display of the captured video of the surveillance camera, the client PC performs a mask process on the captured video of the selected surveillance camera. Therefore, in the initial learning (see below) in the mask process, the background image ( In other words, it is necessary to generate an image region (a still region having no motion in the captured image). However, during the initial learning process, since the generation of the background image in the captured video is not completed, the client PC cannot display the captured video and cannot display the captured video (for example, 30 seconds to 1 minute). There is a problem that the degree of occurrence occurs (see FIG. 30). FIG. 30 is an explanatory diagram of a display problem associated with switching of a live video of a camera to be displayed in the mask camera system and the client PC system.

  Here, the above-described problem will be described more specifically with reference to FIG.

  In the mask camera system (see FIG. 28 for details), mask processing is performed on the captured image in each mask camera (that is, each camera A and B having a mask function). That is, the image part of the moving body (for example, a person) in the edited video BB1 generated by the camera A is replaced with the mask image, and the image part of the moving body (for example, the person) in the edited video CC1 generated by the camera B is It has been replaced with a mask image. Here, in a state where the edited video BB1 distributed from the camera A is displayed on the monitor, when the distribution source of the video displayed on the monitor is switched from the camera A to the camera B by the user operation of the client PC, The client PC can immediately display the edited video CC1 generated by the camera B.

  On the other hand, in the client PC system (refer to FIG. 29 for details), for example, the cameras A and B do not have a mask function, and the captured image distributed from the cameras A and B is masked in the client PC. That is, the captured video distributed from the camera A is masked by the client PC to generate the edited video BB1, and the captured video distributed from the camera B is masked by the client PC to generate the edited video CC1. Here, in a state where the edited video BB1 generated by the client PC based on the captured video distributed from the camera A is displayed on the monitor, the distribution source of the video displayed on the monitor by the user operation of the client PC is When the camera A is switched to the camera B, the client PC needs to perform initial learning to generate a background image when masking the captured video distributed from the camera B. This initial learning is, for example, about 30 seconds to 1 minute. During this initial learning period, the client PC cannot display the edited video CC1 as a result of masking the captured video distributed from the camera B. In other words, even if the camera PC is switched from the camera A to the camera B, the client PC cannot immediately display the edited video CC1 as a result of masking the captured video delivered from the camera B.

  If the initial learning process in the client PC is omitted every time the display of the captured video of the monitoring camera is switched, an appropriate background image is not generated in the client PC. For this reason, the client PC cannot correctly detect the moving object (for example, a person) in the mask process, and the person in the edited video after the mask process is displayed without being replaced with the mask image. There is a possibility that privacy is not protected properly.

  Therefore, in the first video display method to the fourth video display method (see later) of the fifth embodiment, even if there is a switching of the captured video of the surveillance camera to be displayed on the monitor, the selected captured video An example of a monitoring system and an image display method for suppressing masking and appropriately performing mask processing to improve the efficiency of user monitoring work and the accuracy of marketing analysis will be described.

(Fifth embodiment)
The monitoring camera system 100 described in the first embodiment will be described as an example of the monitoring system of the fifth embodiment. The internal configuration of the monitoring camera system 100 of the fifth embodiment is a configuration that further includes a server PC 30 with respect to the monitoring camera system 100 of the first embodiment (see FIG. 27). Since the configuration of each device other than the server PC 30 is the same as that of the surveillance camera system 100 of the first embodiment, the description of the same contents is simplified or omitted, and different contents will be described.

  FIG. 27 is a diagram illustrating another example of the system configuration of the monitoring camera system 100 of each embodiment. In the surveillance camera system 100 shown in FIG. 27, a server PC 30 is added to the surveillance camera system 100 shown in FIG. The server PC 30 is connected to the network NW. The server PC 30 is also connected to the client PC 10 and the recorders REC1 and REC4 via the network NW. The server PC 30 is also connected to various cameras via the network NW and each router. A specific internal configuration of the server PC 30 will be described later with reference to FIG.

  Here, details of the above-described mask camera system and client PC system will be described with reference to FIGS. FIG. 28 is a schematic diagram illustrating, in time series, a procedure in which the edited video BB1 after the mask process is displayed on the monitor DP of the client PC 10 when the fixed camera CAM 14 has a mask function. FIG. 29 is a schematic diagram illustrating, in time series, a procedure in which the edited video BB1 after the mask process is displayed on the monitor DP of the client PC 10 when the client PC 10 has a mask function. Here, the fixed camera CAM 14 is described as an example of the camera, but it goes without saying that the mask camera method and the client PC method are not limited to the fixed camera CAM 14 and can be applied to other cameras as well.

  In the mask camera system shown in FIG. 28, when the fixed camera CAM14 is a mask camera having a mask function and the execution (ON) of the mask process is set, the fixed camera CAM14 performs the captured video AA1 by the mask process. The moving body region OB1 of the moving body (for example, a person) appearing at is replaced with the mask image MSK1 to obtain the edited video BB1. The fixed camera CAM 14 distributes (transmits) the edited video BB1 to the client PC 10. The client PC 10 displays the edited video BB1 distributed from the fixed camera CAM14 on the monitor DP. That is, the person in the captured video AA1 is replaced with a mask image by mask processing of the fixed camera CAM14 as a mask camera before being distributed from the fixed camera CAM14 to the client PC 10.

  In the client PC system shown in FIG. 29, when the fixed camera CAM 14 does not have a mask function, or has a mask function but non-execution (OFF) of mask processing is set, the fixed camera CAM 14 captures the captured video AA1. The captured image AA1 is distributed (transmitted) to the client PC 10 as it is without performing the mask process for replacing the moving object region OB1 of the moving object (for example, a person) appearing in FIG. The client PC 10 performs a mask process on the captured video AA1 distributed from the fixed camera CAM14, and generates an edited video BB1 in which the moving object area OB1 of the moving object (for example, a person) appearing in the captured video AA1 is replaced with the mask image MSK1. Displayed on the monitor DP. That is, the person in the captured video AA1 is distributed from the fixed camera CAM14 to the client PC 10, and then replaced with a mask image by the masking process of the client PC 10.

  FIG. 31 is an explanatory diagram of an example of an operation outline of the fifth embodiment. In the following description, when two different cameras are illustrated in the omnidirectional cameras CAM11 and CAM31, PTZ cameras CAM12 and CAM13, CAM21, CAM32, and CAM33, and fixed cameras CAM14, CAM22, and CAM34 illustrated in FIG. A ”and“ Camera B ”. In the fifth embodiment, the client PC 10 distributes the video displayed on the monitor DP while displaying the edited video BB1 generated by masking the captured video distributed from the camera A on the monitor DP. Is switched from the camera A to the camera B, the background image BG1 of the edited video BB1 of the camera A is stored in the storage 15 as an example of the information storage unit, and the background of the edited video CC1 of the camera B at the previous use is stored. The image BG2 is acquired from the storage 15. The client PC 10 can display the edited video CC1 after the mask process without performing the initial learning (see step S11 in FIG. 11) by performing the mask process using the acquired background image BG2. The information storage unit stores a background image of the edited video BB1, and in addition to the storage 15 described above, for example, a cloud computer (not shown) connected via a server PC 30, RAM 162, an SD card, and a network NW, which will be described later. Etc.

(First video display method)
In the first video display method, the client PC 10 saves the captured image of the camera A or the background image of the edited video in the storage 15 when the distribution source of the video displayed on the monitor DP is switched from the camera A to the camera B ( The mask process is performed using the background image saved (saved) when the camera A is switched to the next time (see FIGS. 32 and 33).

  FIG. 32 is a schematic diagram illustrating an overview of the first video display method and the second video display method according to the fifth embodiment. As a main process executed by the processor 11, the client PC 10 receives, for example, the captured video AA1 distributed from the camera A, and displays the edited video BB1 obtained by performing mask processing on the captured video AA1 on the monitor DP (S81). ). When the distribution source of the video displayed on the monitor DP is switched from the camera A to the camera B, the client PC 10 saves the background image BG1 of the edited video BB1 of the camera A in the storage 15 (S82). The client PC 10 acquires the background image BG2 of the edited video CC1 of the camera B at the previous use from the storage 15 for the camera B to be displayed next (S83). The client PC 10 performs a mask process using the background image BG2 acquired in step S83, and displays the edited video CC1 after the mask process on the monitor DP.

  FIG. 33 is a flowchart for explaining an example of the operation procedure of the first video display method in the fifth embodiment. As a premise of the description of FIG. 33, it is assumed that a camera as a distribution source of a video displayed on the monitor DP is not selected.

  In FIG. 33, it is assumed that the camera A is selected as a distribution source of the video displayed on the monitor DP by a user operation (S91A). The processor 11 of the client PC 10 determines whether or not the background image of the image captured by the camera A or the edited image is stored in the storage 15 (S92A). When the processor 11 determines that neither the background image of the captured image of the camera A nor the background image of the edited image is stored in the storage 15 (S92A, NO), the processor 11 performs initial learning using the captured image of the camera A (FIG. 11 (see step S11) (S93A). Thereby, the processor 11 can generate a background image of the captured image of the camera A. After step S93A, the process of the processor 11 proceeds to step S95A.

  On the other hand, when the processor 11 determines that the background image of the camera A captured image or edited video is stored in the storage 15 (S92A, YES), the processor 11 stores the camera A captured image or edited video background image. 15 (S94A). After step S94A, the process of the processor 11 proceeds to step S95A.

  After step S93A or step S94A, the processor 11 performs mask processing using the captured video distributed from the camera A and the background image generated in step S93A or the background image acquired in step S94A. Is displayed on the monitor DP (S95A).

  Here, for example, it is assumed that the distribution source of the video displayed on the monitor DP is switched from the camera A to the camera B (S91B). The processor 11 stores the background image of the edited video of the camera A in the storage 15 (S96B).

  The processor 11 determines whether or not the background image of the captured video or edited video of the camera B is stored in the storage 15 (S92B). If the processor 11 determines that neither the background image of the captured image of the camera B nor the background image of the edited image is stored in the storage 15 (S92B, NO), the processor 11 performs initial learning using the captured image of the camera B (FIG. 11 (see step S11) (S93B). Thereby, the processor 11 can generate a background image of the captured image of the camera B. After step S93B, the process of the processor 11 proceeds to step S95B.

  On the other hand, when the processor 11 determines that the background image of the captured image or edited video of the camera B is stored in the storage 15 (S92B, YES), the processor 11 stores the background image of the captured image or edited video of the camera B. 15 (S94B). After step S94B, the process of the processor 11 proceeds to step S95B.

  After step S93B or step S94B, the processor 11 performs mask processing using the captured video delivered from the camera B and the background image generated in step S93B or the background image acquired in step S94B. Is displayed on the monitor DP (S95B).

  Here, for example, it is assumed that the distribution source of the video displayed on the monitor DP is switched from the camera B to the camera A again. Note that the camera A may be switched to another camera instead of the camera A. For example, when the camera A is switched again, the processor 11 stores the background image of the edited video of the camera B used before the switching in the storage 15 (S96A). After step S96A, the process of step S92A → step S94A → step S95A is similarly repeated. Further, each time there is a camera switching operation by a user operation, for example, “step S91B → step S96B (save background image used before switching to storage) for the camera selected as the switching target. → Step S92B → Step S94B or Step S93B → Step S95B ”is repeatedly executed as a unit.

  According to the first video display method, the client PC 10 acquires from the storage 15 the background image of the captured video or edited video that was previously used by the selected camera when the video of the camera is switched and displayed on the monitor DP. Therefore, it is not necessary to generate a background image of the captured video distributed from the camera selected by switching by initial learning. That is, when the camera PC 10 is switched from the camera A to the camera B, the client PC 10 can suppress the non-display of the video during the period of generating the background image of the captured video distributed from the camera B, thereby improving the efficiency of the user's monitoring work. And can improve the accuracy of marketing analysis. The first video display method is particularly effective when the content of the video displayed on the monitor DP is difficult to change depending on the date and time due to, for example, the indoor environment. When the content of the video is difficult to change depending on the date and time, for example, when the influence of sunlight from the outside is small, or when the movement of the device is difficult to occur such as in a factory. In other words, the first video display method is particularly effective in an environment where the background image (that is, the still region) of the captured video delivered from the camera does not change significantly.

(Second video display method)
In the second video display method, the client PC 10 is the time when the distribution source of the video displayed on the monitor DP has been switched from the camera A to the camera B, and when the captured image or edited video of the camera B was previously displayed (in other words, Only when the elapsed time from the time when the background image of the captured image or edited image is generated is shorter than the default value (set value), the background image used last time of the camera B is acquired from the storage 15 and mask processing is performed (FIG. 32 and FIG. 34).

  In FIG. 32, after step S82, the client PC 10 compares the elapsed time from the previous display time of the captured video or edited video of the camera (for example, camera B) selected by switching with a predetermined value (set value). (S84). As described above, the client PC 10 performs the camera B only when the elapsed time from the previous display of the captured image or edited image of the camera selected by switching (for example, the camera B) is shorter than the default value (set value). The previously used background image is acquired from the storage 15 and masked.

  FIG. 34 is a flowchart for explaining an example of the operation procedure of the second video display method in the fifth embodiment. In the description of FIG. 34, the same steps as those in FIG. 33 are assigned the same step numbers, and the description is simplified or omitted, and different contents will be described.

  In FIG. 34, when the processor 11 determines that the background image of the captured image or edited video of the camera A is stored in the storage 15 (S92A, YES), the processor 11 captures the previous captured image or edited video of the camera A. The elapsed time from the display time is compared with a predetermined value (set value) (S97A). When the processor 11 determines that the elapsed time from the previous display time of the captured image or edited image of the camera A is longer than the predetermined value (set value) (S97A, YES), the image captured from the camera A is captured. Initial learning (see step S11 in FIG. 11) is performed using the video (S93A).

  On the other hand, when the processor 11 determines that the elapsed time from the previous display time of the captured image of the camera A or the edited image is shorter than the predetermined value (set value) (S97A, NO), the captured image of the camera A or The background image of the edited video is acquired from the storage 15 (S94A). The case where the camera B is selected is the same as the process of step S97A, and thus the description regarding step S97B when the camera B is selected is omitted.

  According to the second video display method, for example, the client PC 10 does not significantly change the background image within a period of, for example, 30 minutes after the previously displayed captured video or edited video of the camera B, for example, within 30 minutes. Based on the premise of the above, a background image at the time of previous use is acquired from the storage 15 and used for mask processing. In a situation where many surveillance cameras are installed in the imaging area to be monitored, the client PC 10 often switches and displays the captured video or edited video of the camera at regular intervals. In addition, the background image may change in a short period in a place where there are many changes in the environment at the imaging position of the camera (for example, movement of a person, placement of a new article). In such a situation, the second video display method is particularly effective.

  FIG. 35 and FIG. 36 are sequence diagrams for explaining an example of the operation procedure of the first video display method and the second video display method in the fifth embodiment.

  In FIG. 35, it is assumed that a display request for a captured image of camera A is made by a user operation using the operation unit 13 (S101). The processor 11 requests the background image of the camera A from the storage 15 in accordance with this display request (S102), but receives a response from the storage 15 that there is no corresponding background image (S103). Note that the processing in steps S102 and S103 may be replaced with processing in which the processor 11 searches for the background image of the camera A in the storage 15 in accordance with the display request in step S101, but determines that it has not been found.

  The processor 11 transmits a distribution request for the captured image of the camera A to the camera A via the communication unit 12 (S104). Thereby, the camera A distributes (transmits) the captured video (live video) obtained by the imaging to the client PC 10 until a distribution stop request is received from the processor 11 in step S109 (S105). The processor 11 receives the captured video distributed from the camera A.

  In order to generate the background image of the camera A, the processor 11 performs initial learning (see step S11 in FIG. 11) using the captured video distributed from the camera A (S106). Furthermore, after executing the initial learning in step S106, the processor 11 performs mask processing on the captured image of the camera A, and displays the edited image obtained by the mask processing on the monitor DP (S107).

  Here, it is assumed that a display request for the captured image of the camera B is made by a user operation using the operation unit 13 (S108). In accordance with this display request, the processor 11 transmits a request to stop the distribution of the captured image of the camera A (S109). Thereby, the camera A stops the delivery of the captured video.

  The processor 11 saves (saves) the background image used when the edited video of the camera A is displayed in the storage 15 (S110). Furthermore, it is assumed that the processor 11 requests the background image of the camera B from the storage 15 (S111), but receives a response from the storage 15 that there is no corresponding background image (S112). Note that the processing in step S111 and step S112 may be replaced with processing in which the processor 11 searches for the background image of the camera B in the storage 15 according to the display request in step S108, but determines that it has not been found.

  The processor 11 transmits a distribution request for the captured image of the camera B to the camera B via the communication unit 12 (S113). Thereby, the camera B distributes (transmits) the captured video (live video) obtained by the imaging to the client PC 10 until a distribution stop request is received from the processor 11 in step S115 (S114). The processor 11 receives the captured video distributed from the camera A.

  In order to generate the background image of the camera B, the processor 11 performs initial learning (see step S11 in FIG. 11) using the captured video distributed from the camera B (S106). Further, after executing the initial learning in step S106, the processor 11 performs mask processing on the captured image of the camera B, and displays the edited image obtained by the mask processing on the monitor DP (S107).

  Here, it is assumed that the display request of the captured image of the camera A is made again by the user operation using the operation unit 13 (S101). In accordance with this display request, the processor 11 transmits a distribution stop request for the captured image of the camera B (S115). Thereby, the camera B stops the delivery of the captured video.

  The processor 11 saves (saves) the background image used when the edited video of the camera B is displayed in the storage 15 (S110). Furthermore, the processor 11 requests the background image of the camera A from the storage 15 (S102), and receives a response of the corresponding background image from the storage 15 (S103A). Note that the processing in step S102 and step S103A may be replaced with processing that the processor 11 searches for and acquires the background image of the camera A in the storage 15 in accordance with the display request in step S101.

  The processor 11 transmits a distribution request for the captured image of the camera A to the camera A via the communication unit 12 (S104). Thereby, the camera A distributes (transmits) the captured video (live video) obtained by the imaging to the client PC 10 until the distribution stop request is received from the processor 11 in step S115 described above (S105). The processor 11 receives the captured video distributed from the camera A.

  The processor 11 does not need to perform initial learning for generating the background image of the camera B, and uses the captured image of the camera B acquired in step S103A and the captured video distributed from the camera A to obtain the captured video. Mask processing is performed, and the edited video obtained by the mask processing is displayed on the monitor DP (S107A). In the processing after step S107A, every time the camera is switched, the same processing (that is, processing in which mask processing is performed using the background image used last time) is repeated.

  In the second video display method, after step S115, for example, the process of step S97A in FIG. When the processor 11 determines that the background image of the captured image or edited video of the camera A is stored in the storage 15, the elapsed time from the previous display of the captured image or edited video of the camera A and a predetermined value ( (Set value) is compared (S97A). Only when the processor 11 determines that the elapsed time from the previous display time of the captured image or edited image of the camera A is shorter than the predetermined value (set value) (S97A, NO), the captured image or edited image of the camera A The background image is acquired from the storage 15 (S103A).

(Third video display method)
In the third video display method, the client PC 10 constantly updates the background image of the captured video or edited video of the camera that is not currently displayed on the monitor DP by background processing. The client PC 10 displays the mask processing as the main processing and the edited video after the mask processing on the monitor DP using the latest background image updated by the background processing and the captured video distributed from the camera (FIG. 37-FIG. 39).

  While displaying the captured video or edited video distributed from the camera, for example, since it is necessary to grasp the status of the monitoring target, the captured video using a display frame rate (1 fps or higher, for example, 30 fps) of a certain value or higher. Display of edited video after mask processing is required. On the other hand, it is only necessary to update the background image for the captured image of the other camera that is not currently displayed on the monitor DP. For example, an extremely low-speed background image update process such as one frame (one frame) per 10 seconds may be used. Also, even if background image update processing is performed at such a low speed, the captured image or edited video of the camera is not displayed, so the effect on the screen display for the terminal operator (for example, the user) (for example, There is no display delay) and the influence on the performance (processing load, etc.) of the mask processing as the main processing is slight. Even when background image update processing (normal learning) is performed by background processing, the resources and performance required for the client PC 10 are very small. For example, the amount of information of “background image” × “number of cameras” It is sufficient if there are resources that can be processed and stored.

  FIG. 37 is a schematic diagram showing an outline of the third video display method in the fifth embodiment. As a main process executed by the processor 11, the client PC 10 receives, for example, the captured video AA1 distributed from the camera A, and displays the edited video BB1 obtained by performing mask processing on the captured video AA1 on the monitor DP (S81). ). Further, as background processing executed by the processor 11, the client PC 10 receives, for example, a captured video distributed from another camera (for example, camera B) that is not currently displayed on the monitor DP, and the received captured video and The background image of the camera B is updated using the captured image of the camera B or the background image of the edited video (S85).

  The background image update process in step S85 is performed on a captured image distributed from a camera that is currently communicable (connectable) with the client PC 10, and may be performed periodically, for example. It may be performed if at least a certain time elapses, or may be performed repeatedly. Note that details of step S85 have been described in, for example, step S14 and step S15 in FIG.

  The client PC 10 stores the background image generated or updated by the background processing in the storage 15 (S86). Here, when the distribution source of the video displayed on the monitor DP is switched from the camera A to the camera B, the client PC 10 stores the background image of the edited video of the camera A in the storage 15 and then displays the camera B to be displayed next. , The background image of the captured image or edited video of the camera B saved in the storage 15 by the background processing is acquired from the storage 15 (S83). As the main process, the client PC 10 performs a mask process using the background image acquired in step S83 and the captured video distributed from the camera B, and displays the edited video after the mask process on the monitor DP.

  FIG. 38 is a flowchart for explaining an example of the operation procedure of the main process shown in FIG. FIG. 39 is a flowchart for explaining an example of the operation procedure of the background processing shown in FIG. Assuming that the camera A is selected as the distribution source of the video displayed on the monitor DP as the premise of the explanation of FIG. 38, the background image of the camera B not currently being displayed on the monitor DP is assumed as the premise of the explanation of FIG. Assume that generation or update processing is performed.

  In FIG. 38, the processor 11 of the client PC 10 acquires the background image of the image captured by the camera A or the edited image from the storage 15 (S121). The processor 11 performs mask processing using the captured video currently distributed from the camera A and the background image acquired in step S121, and displays the edited video after the mask processing on the monitor DP (S122). Here, it is assumed that the camera as the distribution source of the video displayed on the monitor DP is changed by a user operation using the operation unit 13 (S123). For example, assume that an operation for switching from camera A to camera B is performed. When the camera is not changed (S123, NO), the processing of the processor 11 returns to step S122.

  In this case (S123, YES), the processor 11 acquires the background image of the captured video or edited video of the camera B from the storage 15 (S124). The processor 11 performs mask processing using the captured video currently distributed from the camera B and the background image acquired in step S124, and displays the edited video after the mask processing on the monitor DP (S125). Here, it is assumed that the camera as the distribution source of the video displayed on the monitor DP is changed by a user operation using the operation unit 13 (S126). For example, it is assumed that an operation for switching from the camera B to the camera C is performed. When the camera is not changed (S126, NO), the process of the processor 11 returns to step S125.

  Even when an operation for switching from the camera B to the camera C is performed, the subsequent processes are repeatedly executed for the selected camera in the same manner as in steps S121 and S122.

  In FIG. 39, the processor 11 acquires the captured video distributed from the camera B (S131). In the case where the background image of the captured image or edited video of the camera B is stored in the storage 15, the processor 11 acquires the background image of the captured image or edited video of the camera B from the storage 15 (S132). Note that if the background image of the image captured by the camera B or the edited image is not stored in the storage 15, the process of step S132 is omitted.

  The processor 11 updates the background image (see FIG. 12) using the captured image distributed from the camera B and the background image acquired in step S132 (S133). Alternatively, when the process of step S132 is omitted, the processor 11 generates a background image of the camera B using the captured video distributed from the camera B (see step S11 in FIG. 11) (S133). . The processor 11 stores the background image of the camera B generated in step S133 or the updated background image of the camera B in the storage 15 (S134).

  Thereafter, the processor 11 similarly performs each process (BGpc) for generating and updating the background image for the camera C in units of the processes (BGpb) in steps S131 to S134, and thereafter also for the camera A. Similarly, background image generation and update processing (BGpa) is performed. Thereafter, the process of updating the background image (BGpb) → the process of updating the background image (BGpc) → the process of updating the background image (BGpa) is repeated.

  According to the third video display method, the client PC 10 can update the background image of the camera that is not currently displayed on the monitor DP by background processing, so that the client PC 10 can compare with the first video display method and the second video display method described above. The latest background image can be updated without receiving an excessive processing load, and the accuracy of the background image to be used is improved. As a result, when the client PC 10 is switched to the camera whose background image is updated, the stationary object and the moving body (for example, a person) in the captured image captured by the camera are displayed in the first video display method described above. It becomes possible to recognize correctly compared with the second video display method, and as a result, the frequency of erroneously recognizing a stationary object as a moving object can be reduced.

(4th video display system)
In the fourth video display method, the client PC 10 and the server PC 30 share the main processing and background processing of the third video display method. Specifically, the client PC 10 executes main processing in the third video display method, and the server PC 30 executes background processing in the third video display method. The server PC 30 saves (saves) the background image obtained by generating or updating the background image as background processing in the storage SP 14 (see FIG. 40).

  FIG. 40 is a block diagram showing in detail an example of the internal configuration of the server PC. The server PC 30 shown in FIG. 40 includes a processor SP11, a communication unit SP12, a memory SP13, a storage SP14, and a power management unit SP15.

  The processor SP11 performs signal processing for overall control of operations of each unit of the server PC 30, data input / output processing with other units, data calculation processing, and data storage processing. The processor SP11 is configured using, for example, a CPU, MPU, DSP, or FPGA.

  For example, the processor SP11 obtains each captured video data distributed from various cameras (for example, omnidirectional cameras CAM11, CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, CAM34). Obtained via the communication unit SP12. The processor SP11 generates a background image by repeatedly performing, for example, the background processing shown in FIG. 39 using captured images distributed from various cameras (for example, periodically updates the background image). Update). Further, the processor SP11 may store the captured video distributed from various cameras as a recorded video in the storage SP14 in association with the imaging date and camera identification information.

  The communication unit SP12 receives captured image data distributed from various cameras (for example, omnidirectional cameras CAM11, CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, CAM34). Output to the processor SP11. Further, the communication unit SP12 transmits the recorded video data read from the storage SP14 by the processor SP11 to the client PC 10, or transmits the background image data read from the storage SP14 by the processor SP11 to the client PC 10.

  The memory SP13 includes a ROM 131SP that stores a program for defining the operation of the server PC 30 and set value data, and a RAM 132SP that stores various data generated by the processor SP11 and work data being processed. Although not shown in FIG. 40, the memory SP13 may further include a memory card (not shown) that is detachably connected to the server PC 30 and stores various data.

  The storage SP 14 stores, for example, captured video data distributed (transmitted) from various cameras, and background image data generated or updated by the processor SP. Further, the storage SP 14 may store the edited video or edited video data when the edited video or edited video data generated by the client PC 10 is received from the client PC 10.

  The power management unit SP15 supplies DC power to each unit of the server PC 30.

  FIG. 41 is a schematic diagram showing an outline of the fourth video display method in the fifth embodiment. As a main process executed by the processor 11, the client PC 10 receives, for example, the captured video AA1 distributed from the camera A, and displays the edited video BB1 obtained by performing mask processing on the captured video AA1 on the monitor DP (S81). ). The server PC 30 is distributed from various cameras (for example, omnidirectional cameras CAM11, CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, CAM34) as background processing executed by the processor SP11. The captured images are received, and the background images of the various cameras are updated using the received captured images and the captured images of the various cameras or the background images of the edited video (S85A).

  The background image update process in step S85A is performed on captured images distributed from all cameras that are currently communicable (connectable) with the server PC 30, and may be performed periodically, for example. However, it may be performed at least after a certain time has elapsed, or in short, it may be performed repeatedly. Note that details of step S85A have been described in, for example, step S14 and step S15 in FIG.

  The server PC 30 stores the background image generated or updated by the background processing in the storage SP 14 and sequentially updates it (S86A). Here, when the distribution source of the video displayed on the monitor DP is switched from the camera A to the camera B, the client PC 10 may or may not store the background image of the edited video of the camera A in the storage 15. It doesn't matter. Further, the client PC 10 receives and acquires the captured image of the camera B or the background image of the edited video stored in the storage SP14 by the background processing of the server PC 30 for the camera B to be displayed next from the server PC 30 (S83A). . As a main process, the client PC 10 performs a mask process using the background image acquired in step S83A and the captured video distributed from the camera B, and displays the edited video after the mask process on the monitor DP.

  As described above, in the fourth video display system, the client PC 10 performs the main process shown in FIG. 38, while the server PC 30 performs the background process shown in FIG. That is, the client PC 10 and the server PC 30 share the main process and the background process. Therefore, according to the fourth video display method, since the background images of various cameras are updated by the background processing of the server PC 30, the client PC 10 displays the captured video of one of the cameras on the monitor DP by a user operation. Each time the operation is performed, the latest background image of the selected camera can be acquired from the server PC 30 in real time, and the mask process can be executed as the main process after acquisition. Thereby, the client PC 10 can display the edited video appropriately masked on the monitor DP without generating a background image of the camera selected by the user operation (that is, without performing initial learning).

  As described above, the surveillance camera system 100 according to the fifth embodiment includes a plurality of cameras (e.g., the omnidirectional camera CAM11, the fixed camera CAM14,...) That capture an imaging area and the first camera (camera A) of the plurality of cameras. The client PC 10 displays on the monitor DP the captured video of the imaging area distributed from (1)) or the edited video obtained by performing mask processing for changing the moving body appearing in the captured video to a mask image. The client PC 10 generates a background image of each captured video distributed from each camera and stores it in the storage 15. In response to switching from the camera A to the second camera (camera B), the client PC 10 acquires the background image of the captured video distributed from the selected camera B from the storage 15 and the background image acquired from the storage 15 Using the captured video distributed from camera B, an edited video obtained by performing mask processing on the captured video distributed from camera B is displayed on monitor DP.

  Similarly to the first embodiment, the client PC 10 of the fifth embodiment can also perform mask processing that replaces at least a part or all of the moving person appearing in the captured image with a mask image. As a result, when the client PC 10 switches the video of the camera and displays it on the monitor DP, the client PC 10 can acquire the background image of the captured video or the edited video used by the selected camera last time from the storage 15 and perform mask processing. Therefore, it is not necessary to generate a background image of the captured video delivered from the camera selected by switching by initial learning. That is, when the camera PC 10 is switched from the camera A to the camera B, the client PC 10 can suppress the non-display of the video during the period of generating the background image of the captured video distributed from the camera B, thereby improving the efficiency of the user's monitoring work. And can improve the accuracy of marketing analysis.

  Further, the client PC 10 stores (saves) the background image of the captured video distributed from the camera A in the storage 15 in accordance with the switching from the camera A to the camera B. As a result, the client PC 10 can store the latest background image of the camera A that has been used until the switching is performed in the storage 15, and when the captured image of the camera A is displayed on the monitor DP next time, the latest background image is displayed. It is possible to perform mask processing quickly and appropriately.

  Further, in response to the switch from camera A to camera B, the client PC 10 starts from the selected camera B when the elapsed time from the previous display of the edited video of the camera B is shorter than a predetermined time (for example, 30 minutes). A background image of the delivered captured video is acquired from the storage 15. Thereby, for example, if the installation environment of the camera B is such that the background image does not change significantly within 30 minutes after displaying the captured image or edited video of the camera B last time, Since the background image can be acquired from the storage 15 and used in the mask process, the mask process can be executed quickly and appropriately without performing initial learning again on the captured image from the camera B.

  Further, the client PC 10 obtains a background image of the captured video delivered from at least one camera, which is not displayed on the monitor DP, from the storage 15 and updates it at regular intervals, and stores the updated background image in the storage 15. To do. As a result, the client PC 10 can update the background image of the camera that is not currently displayed on the monitor DP by the background processing, and therefore has an excessive processing load compared to the first video display method and the second video display method described above. Without being received, it can be updated to the latest background image, and the accuracy of the background image to be used is improved. As a result, when the client PC 10 is switched to the camera whose background image is updated, the stationary object and the moving body (for example, a person) in the captured image captured by the camera are displayed in the first video display method described above. It becomes possible to recognize correctly compared with the second video display method, and as a result, the frequency of erroneously recognizing a stationary object as a moving object can be reduced.

  Moreover, the surveillance camera system 100 of the fifth embodiment includes a plurality of cameras (for example, an omnidirectional camera CAM11, a fixed camera CAM14,...) That capture an imaging area, and a first camera (camera A) among the plurality of cameras. From the client PC 10 that displays on the monitor DP the captured video of the imaging area distributed from the above, or the edited video obtained by performing mask processing for changing the moving body appearing in the captured video to a mask image, and distributed from each camera. And a server PC 30 that generates a background image of each captured video and saves it in the storage SP14. As a background process, the server PC 30 updates the background image of each captured video acquired from the storage SP14 at regular intervals, and stores the updated background image in the storage SP14. The client PC 10 acquires a generated or updated background image corresponding to the selected camera B from the server PC 30 in response to switching from the camera A to the second camera (camera B), and the background acquired from the server PC 30 Using the image and the captured video distributed from camera B, an edited video obtained by performing mask processing on the captured video distributed from camera B is displayed on monitor DP.

  In other words, the background image of various cameras is updated by the server PC 30 as a background process for the main process of the client PC 10. Therefore, the client PC 10 can acquire the latest background image of the selected camera in real time from the server PC 30 every time an operation for displaying the captured image of any camera on the monitor DP is performed by a user operation. In addition, mask processing can be executed as main processing. Thereby, the client PC 10 can display the edited video appropriately masked on the monitor DP without generating a background image of the camera selected by the user operation (that is, without performing initial learning).

(Second process leading to the contents of the fifth embodiment)

  According to Patent Document 1, since a mask image is generated for each position information and person attribute (for example, gender, age) in a moving image of a person detected in a monitoring area, privacy is ensured by the mask image. The behavior of the person in the store can be grasped above.

  However, in Patent Document 1, it is a camera that changes the image area of a person detected in a moving image to a mask image, and the mask is set by a receiving device (for example, a client PC used by a user) that receives the moving image from the camera. It is not taken into account that processing is performed. Many existing surveillance cameras do not have a function (mask function) capable of executing the mask processing of Patent Document 1.

  Here, for example, in a user's monitoring work, when past captured images that are respectively captured from a monitoring camera installed in a monitoring area and recorded (recorded) on an external device such as a recorder are displayed on the monitor of the client PC Further, it is assumed that the recording date and time to be reproduced are sequentially switched and displayed by a user operation. Since the client PC performs a mask process on the recorded video at the designated recording date and time every time the recording date and time of the recorded video is switched, a background image (that is, a video image) in the recorded video in the initial learning (see later) in the mask process is performed. It is necessary to generate a still image area that does not move in the recorded video. However, during the initial learning process, since the generation of the background image in the recorded video is not completed, the client PC cannot display the recorded video and cannot display the recorded video (for example, 30 seconds to 1 minute). There is a problem that the degree is generated (see FIG. 42). FIG. 42 is an explanatory diagram of a problem when displaying the edited and recorded video of the camera at an arbitrary designated date and time in the mask camera system and the client PC system.

  Here, with reference to FIG. 42, the above-described problem will be described more specifically.

  In the mask camera system (see FIG. 28 for details), mask processing is performed on a captured image in a mask camera (that is, a camera having a mask function). That is, the image portion of the moving body (for example, a person) in the edited video generated by the mask camera is replaced with the mask image, and is stored in the recorder (for example, the recorder REC1) as the edited video. Here, when an instruction to reproduce a past captured video (for example, a recorded video stored in the recorder REC1) captured at an arbitrary designated date and time is given by the user operation, the client PC edits acquired from the recorder REC1. The recorded video DD1 can be displayed immediately.

  On the other hand, in the client PC method (see FIG. 29 for details), for example, the camera does not have a mask function, and the client PC performs a mask process on the captured video distributed from the camera. That is, the captured video delivered from the camera is masked by the client PC to generate an edited video. Here, when an instruction to reproduce a past captured video (for example, a recorded video stored in the recorder REC1) captured at an arbitrary designated date and time is given by the user operation, the client PC acquires the recording acquired from the recorder REC1. When masking a video, it is necessary to perform initial learning in order to generate a background image. This initial learning is, for example, about 30 seconds to 1 minute. During this initial learning, the client PC cannot display the edited recorded video DD1 as a result of masking the recorded video acquired from the recorder REC1. In other words, the client PC cannot immediately display the edited recorded video DD1 as a result of masking the recorded video acquired from the recorder REC1, even if there is an instruction to reproduce the recorded video captured at any specified date and time by a user operation. .

  When the recording date and time of the recorded video of the surveillance camera is designated by a user operation, if the initial learning process in the client PC is omitted, an appropriate background image is not generated in the client PC. For this reason, the client PC cannot correctly detect the moving object (for example, a person) in the mask process, and the person in the edited recording video after the mask process is displayed without being replaced with the mask image. May not be properly protected.

  Therefore, in the fifth video display method to the sixth video display method of the fifth embodiment, when the recording date / time of the recorded video of the surveillance camera to be displayed on the monitor is specified, the specified recording date / time is set. An example of a monitoring system and a video display method for suppressing the non-display of recorded video and appropriately performing mask processing to improve the efficiency of the user's monitoring work and the accuracy of marketing analysis will be described.

(5th video display system)
FIG. 43 is a schematic diagram showing an outline of the fifth video display method in the fifth embodiment. In the fifth video display method, the client PC 10 receives, for example, the captured video AA1 distributed from the camera A as the main process executed by the processor 11, and uses the edited video BB1 obtained by performing mask processing on the captured video AA1. The data is displayed on the monitor DP (S81). Note that, in step S81, the client PC 10 receives a recorded video distributed from a distribution device that holds a past recorded video, for example, in addition to performing mask processing on the captured video AA1 distributed from the camera A. The edited video obtained by performing the masking process may be displayed on the monitor DP. This process is the same in step S81 in FIG. However, the client PC 10 holds a background image of the recorded video distributed from the distribution device in advance before performing the masking process of step S81, or the background image is included in the recorded video distributed from the distribution device. In this case, it is assumed that the background image is acquired. The distribution device may be any of the above-described camera A, recorder (for example, recorders REC1, REC2, REC3, and REC4), PC recorder, and client PC 10 itself. The PC recorder is, for example, a general-purpose PC that holds a recorded video in a hard disk built in the PC recorder itself. Specifically, the PC recorder may be a client PC 10 or another PC ( (Not shown in FIG. 27). For example, the PC recorder records (records) captured images distributed from various cameras shown in FIG. 27 on the built-in hard disk, and distributes the recorded video (recorded video) to itself or another PC (for example, the client PC 10). This is a general-purpose PC in which an application to be installed is incorporated.

  Further, the client PC 10 performs background processing executed by the processor 11 from various cameras (for example, omnidirectional cameras CAM11, CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, CAM34). The distributed captured images are received, and background images of various cameras are generated repeatedly (for example, periodically) using each received captured image (S88).

  The background image repetitive generation process in step S88 is performed on captured video images distributed from all cameras that are currently communicable (connectable) with the client PC 10, and may be performed periodically, for example. It may be performed if at least a certain time has passed, or may be performed repeatedly. The details of step S88 have already been described in step S11 of FIG.

  The client PC 10 stores the background images generated by the background processing (in other words, sequentially added) in the storage 15 and sequentially accumulates them (S86B). Here, when the recording date and time of the recorded video that the user wants to play back to the camera A is designated by the user operation, the client PC 10 stores the background image of the camera A generated at the generation date and time closest to the recording date and time. Is read out and acquired (S87). As a main process, the client PC 10 performs a mask process using the background image acquired in step S87 and the recorded video at the above-described recording date and time of the camera A acquired from the storage 15 or the storage SP 14 of the server PC 30. The edited recording video is displayed on the monitor DP.

  FIG. 44 is a flowchart for explaining an example of the operation procedure of the main process shown in FIG. FIG. 45 is a flowchart for explaining an example of the operation procedure of the background processing shown in FIG. As a premise of the description of FIG. 44, a recorded video displayed on the monitor DP (that is, a captured video captured at a recording date and time designated by a user operation) is transmitted from the recorder (for example, the recorder REC1) to the client PC 10 for storage. 15 is stored. 45, the server PC 30 performs the same processing in the order of camera A → camera B → camera C →. The camera C is one of the cameras shown in FIG.

  In FIG. 44, a user operation using the operation unit 13 selects a camera (for example, camera A) as a distribution source of a recorded video displayed on the monitor DP (S141, YES), and the recording date and time that the user wants to play back is selected. It is assumed that it is designated (S142). In this case, the processor 11 of the client PC 10 acquires the captured image of the camera A (that is, the recorded image of the camera A) captured from the storage 15 (S143). The processor 11 stores the background image of the camera A generated at the generation date / time closest to the recording date / time (see FIG. 45) from the background images for each generation date / time of the camera A stored in the storage 15 by the background processing. 15 (S144).

  The processor 11 performs mask processing (see FIGS. 11 and 12) using the recorded video of the camera A acquired in step S143 and the background image of the camera A acquired in step S144 (S145). The processor 11 displays the edited recorded video after the mask processing on the monitor DP (S146). After step S146, the main process of the processor 11 returns to step S141. If no camera is selected by the user operation at this time (S141, NO), the main process of the processor 11 ends.

  In FIG. 45, the client PC 10 receives the captured video distributed from the camera A. The processor 11 acquires the captured video distributed from the camera A via the communication unit 12 (S151). The processor 11 generates a background image of the captured image of the camera A acquired in step S151 (see step S11 in FIG. 11, S152).

  The processor 11 obtains the background image of the captured image of the camera A generated in step S152, the identification information of the camera A (for example, the ID or IP address of the camera A, or a combination thereof) and the generation date and time of the background image generated in step S152. Correspondingly stored in the storage 15 (S153).

  After step S153, the processor 11 determines whether or not a predetermined time (for example, 30 minutes) has elapsed since the generation of the background image in step S152 (S154). If it is determined that a predetermined time (for example, 30 minutes) has passed since the generation of the background image in step S152 (S154, YES), the processing of the processor 11 returns to step S151, and the generation of the background image of the camera A is thereafter performed. Is repeated every predetermined time described above.

  The processor 11 also executes the same processing (BGqb, BGqc) as the generation processing (BGqa) of the background image of the camera A every predetermined time for the camera B, camera C, and other cameras.

  According to the fifth video display method, the client PC 10 repeatedly generates a background image of a captured video distributed from various cameras (see above) as background processing, and uses the background image as the camera identification information and the background. The image is stored and stored in the storage 15 in association with the image generation date and time. Therefore, when the recording date / time of the recorded video of the monitoring camera to be displayed on the monitor DP is specified by the user operation on the client PC 10, the client PC 10 is generated at the generation date / time closest to the specified recording date / time. A background image obtained from the storage 15 can be masked using the background image and the recorded video of the recording date and time. As a result, when the client PC 10 displays the recorded video that the user wants to reproduce on the monitor DP, the client PC 10 appropriately performs mask processing for replacing a moving body such as a person with a mask image while suppressing the non-display of the recorded video. It is possible to improve the efficiency of user monitoring operations and the accuracy of marketing analysis.

(Sixth video display method)
In the sixth video display method, as in the above-described fourth video display method, the client PC 10 and the server PC 30 share the main processing and background processing of the fifth video display method. Specifically, the client PC 10 executes main processing in the fifth video display method, and the server PC 30 executes background processing in the fifth video display method. The server PC 30 saves (saves) the background image obtained by repeated generation of the background image as background processing in the storage SP 14 in association with the camera identification information and the generation date and time of the background image.

  FIG. 46 is a schematic diagram showing an outline of the sixth video display method in the fifth embodiment. As a main process executed by the processor 11, the client PC 10 receives, for example, the captured video AA1 distributed from the camera A, and displays the edited video BB1 obtained by performing mask processing on the captured video AA1 on the monitor DP (S81). ). The server PC 30 is distributed from various cameras (for example, omnidirectional cameras CAM11, CAM31, PTZ cameras CAM12, CAM13, CAM21, CAM32, CAM33, fixed cameras CAM14, CAM22, CAM34) as background processing executed by the processor SP11. The captured image is received, and background images of various cameras are generated repeatedly (for example, periodically) using each received captured image (S88A).

  The background image repetitive generation process in step S88A is performed on captured video images distributed from all cameras that are currently communicable (connectable) with the server PC 30, and may be performed periodically, for example. It may be performed if at least a certain time has passed, or may be performed repeatedly. Note that details of step S88A have been described in, for example, step S11 of FIG.

  The server PC 30 stores the background images generated by the background processing (in other words, sequentially added) in the storage SP 14 and sequentially accumulates them (S86C). Here, when the recording date and time of the recorded video that the user wants to play back to the camera A is designated by the user operation, the client PC 10 displays the background image of the camera A generated on the generation date and time closest to the recording date and time on the server PC 30. (S87). As a main process, the client PC 10 performs a mask process using the background image acquired in step S87 and the recorded video at the above-described recording date and time of the camera A acquired from the storage 15 or the storage SP 14 of the server PC 30. The edited recording video is displayed on the monitor DP.

  According to the sixth video display method, the client PC 10 performs the main process shown in FIG. 44, while the server PC 30 performs the background process shown in FIG. That is, the client PC 10 and the server PC 30 share the main process and the background process. Therefore, according to the sixth video display method, the background images of various cameras are updated by the background processing of the server PC 30, so that the client PC 10 can select one of the cameras that the user wants to reproduce by user operation. Each time a recording video recording date and time designation operation is performed, a camera background image closest to the specified recording date and time can be quickly acquired from the server PC 30, and the main process is performed using the background image and the recording video at the recording date and time. As a result, the edited recorded video after the masking process can be displayed on the monitor DP. As a result, the client PC 10 can display the edited video appropriately masked on the monitor DP, regardless of the camera's desired recording date and time, so that, for example, investigation of the cause when an incident occurs Can be efficiently supported, improving the efficiency of user monitoring operations and improving the accuracy of marketing analysis.

  As described above, the surveillance camera system 100 according to the fifth embodiment stores a camera 15 that captures an imaging area (for example, the fixed camera CAM 14) and a captured image of a certain period of the imaging area distributed from the camera as a recorded video. And a client PC 10 for storing the data. The client PC 10 repeatedly generates a background image of the captured video for a certain period of the imaging area, associates the generated background image, the camera identification information, and the background image generation date with the storage image 15 and sequentially adds them. And accumulate. The client PC 10 acquires the background image generated at the generation date and time closest to the specified playback date and time from the storage 15 in accordance with the designation of the recording date and time (playback date and time) that the user desires to reproduce. An edited and recorded video obtained by performing mask processing for changing a moving body (for example, a person) appearing in the recorded video to a mask image using the background image and the captured video for a certain period of the imaging area as a recorded video. Display on monitor DP.

  Therefore, when the recording date / time of the recorded video of the monitoring camera to be displayed on the monitor DP is specified by the user operation on the client PC 10, the client PC 10 is generated at the generation date / time closest to the specified recording date / time. A background image obtained from the storage 15 can be masked using the background image and the recorded video of the recording date and time. As a result, when the client PC 10 displays the recorded video that the user wants to reproduce on the monitor DP, the client PC 10 appropriately performs mask processing for replacing a moving body such as a person with a mask image while suppressing the non-display of the recorded video. It is possible to improve the efficiency of user monitoring operations and the accuracy of marketing analysis.

  The surveillance camera system 100 according to the fifth embodiment also includes a camera (for example, a fixed camera CAM 14) that captures an image capturing area, and a captured image of a certain period of the image capturing area distributed from the camera as a recorded image in the storage 15. The client PC 10 to be stored and the background image of the captured video for a certain period of the imaging area distributed from the camera are repeatedly generated, and the generated background image, the camera identification information, and the generation date and time of the background image are associated with each other and stored. And a server PC 30 stored in the SP 14. The client PC 10 acquires the background image generated at the generation date and time closest to the specified playback date and time from the server PC 30 and acquires from the server PC 30 according to the designation of the recording date and time (playback date and time) that the user desires to play back. Obtained by performing a mask process for changing a moving body (for example, a person) appearing in the recorded video to a mask image using the background image and the captured video for a certain period of the imaging area as the recorded video stored in the storage 15. The edited recorded video is displayed on the monitor DP.

  Accordingly, since the background images of various cameras are updated by the background processing of the server PC 30, the client PC 10 performs an operation for specifying the recording date and time of the recorded video of any camera that the user wants to reproduce by the user operation. Each time it is done, the background image of the camera closest to the designated recording date and time can be quickly acquired from the server PC 30, and the mask processing as the main processing is executed using this background image and the recorded video at the recording date and time to perform masking. The edited and recorded video after processing can be displayed on the monitor DP. As a result, the client PC 10 can display the edited video appropriately masked on the monitor DP, regardless of the camera's desired recording date and time, so that, for example, investigation of the cause when an incident occurs Can be efficiently supported, improving the efficiency of user monitoring operations and improving the accuracy of marketing analysis.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope of the claims, and it is obvious that the present invention also applies to the combination of the above-described embodiments. It is understood that it belongs to the technical scope of

The present invention appropriately performs mask processing after suppressing the non-display of the selected captured image even when the captured image of the monitoring camera to be displayed on the monitor is switched, and improves the efficiency of the monitoring operation of the user. It is useful as a monitoring system , a video display method and a computer program for improving marketing analysis accuracy.

10 Client PC
11 Processor 12 Communication Unit 13 Operation Unit 15 Storage 16 Memory 17 Power Management Unit 30 Server PC
100 surveillance camera system (monitoring system)
111 Image Input Unit 112 Background Image Processing Unit 113 Mask Processing Setting Unit 114 Mask Processing Unit 115 Video Output Control Unit 161 ROM
162 RAM
CAM11, CAM31 Omnidirectional camera CAM12, CAM13, CAM21, CAM32, CAM33 PTZ camera CAM14, CAM22, CAM34 Fixed camera NW Network REC1, REC2, REC3, REC4 Recorder

Claims (8)

A plurality of cameras for imaging the imaging area;
Monitor the captured video of the imaging area distributed from the first camera of the plurality of cameras or the first edited video obtained by performing mask processing for changing a moving body appearing in the captured video to a mask image An information processing device to be displayed on
The information processing apparatus repeatedly generates a background image of each captured video distributed from each individual camera and stores it in a storage unit,
In response to switching from the first camera to the second camera, the information processing apparatus causes the storage unit to hold a background image of a captured video that is distributed from the first camera and stored in the storage unit. Every
Further, a background image of the captured video distributed from the second camera is acquired from the storage unit, and the second image is acquired using the background image acquired from the storage unit and the captured video distributed from the second camera. Displaying on the monitor a second edited video obtained by performing the mask processing on the captured video delivered from the camera;
Monitoring system. The monitoring system according to claim 1,
When the elapsed time from the previous display time of the second edited video is shorter than a predetermined time in response to switching from the first camera to the second camera, the information processing apparatus starts from the selected second camera. Obtaining a background image of the delivered captured video from the storage unit;
Monitoring system. A plurality of cameras for imaging the imaging area;
Monitor the captured video of the imaging area distributed from the first camera of the plurality of cameras or the first edited video obtained by performing mask processing for changing a moving body appearing in the captured video to a mask image An information processing device to be displayed on
The information processing device generates a background image of each captured video distributed from each of the cameras and stores it in a storage unit,
The information processing apparatus acquires, from the storage unit, a background image of a captured video distributed from the selected second camera in response to switching from the first camera to the second camera. The second edited video obtained by performing the mask processing on the captured video distributed from the second camera using the background image and the captured video distributed from the second camera are displayed on the monitor. ,
Further , the information processing apparatus acquires a background image of a captured video distributed from at least one of the cameras that is not displayed on the monitor, updates the background image at a predetermined interval, and updates the background image after the update. Storing in the storage unit;
Monitoring system. A plurality of cameras for imaging the imaging area;
Monitor the captured video of the imaging area distributed from the first camera of the plurality of cameras or the first edited video obtained by performing mask processing for changing a moving body appearing in the captured video to a mask image An information processing device to be displayed on
A server device that generates a background image of each captured video distributed from each of the cameras and stores the background image in a storage unit,
The server device updates the background image of each captured video acquired from the storage unit at regular intervals, stores each background image after update in the storage unit,
The information processing apparatus acquires a generated or updated background image corresponding to the selected second camera from the server apparatus in response to switching from the first camera to the second camera, and the server apparatus The second edited video obtained by performing the mask processing on the captured video distributed from the second camera using the background image acquired from the video and the captured video distributed from the second camera is used as the monitor. indicate,
Monitoring system. A first image obtained by performing mask processing for changing a captured image of the imaging area distributed from a first camera of a plurality of cameras that capture an imaging area or a moving body appearing in the captured image to a mask image. A video display method in a monitoring system for displaying an edited video on a monitor using an information processing device ,
The information processing apparatus repeatedly generates a background image of each captured video distributed from each of the cameras, stores the background image in a storage unit,
In response to switching from the first camera to the second camera, the information processing apparatus causes the storage unit to hold a background image of a captured video that is distributed from the first camera and stored in the storage unit. Every
Further, a background image of the captured video distributed from the second camera is acquired from the storage unit, and the second image is acquired using the background image acquired from the storage unit and the captured video distributed from the second camera. Displaying on the monitor a second edited video obtained by performing the mask processing on the captured video delivered from the camera;
Video display method. The obtained by performing a mask process for changing video image of the imaging area which is distributed from the first camera of the plurality of cameras for imaging the imaging area, or the moving object appearing in the captured image in the mask image An information processing apparatus for displaying one edited video on a monitor;
A server device that generates a background image of each captured video distributed from each individual camera and stores it in a storage unit, and a video display method in a monitoring system,
The server device updates the background image of each captured video acquired from the storage unit at regular intervals, stores each background image after update in the storage unit,
The information processing apparatus acquires a generated or updated background image corresponding to the selected second camera from the server apparatus in response to switching from the first camera to the second camera, and the server apparatus The second edited video obtained by performing the mask processing on the captured video distributed from the second camera using the background image acquired from the video and the captured video distributed from the second camera is used as the monitor. indicate,
Video display method. A first image obtained by performing mask processing for changing a captured image of the imaging area distributed from a first camera of a plurality of cameras that capture an imaging area or a moving body appearing in the captured image to a mask image. To an information processing device that is a computer that displays edited video on a monitor,
Repeatedly generating a background image of each captured video distributed from each individual camera and storing it in a storage unit;
In response to switching from the first camera to the second camera,
Holding a background image of a captured image distributed from the first camera and stored in the storage unit in the storage unit;
A background image of the captured video distributed from the second camera is acquired from the storage unit, and the background image acquired from the storage unit and the captured video distributed from the second camera are used to obtain a background image from the second camera. Displaying the second edited video obtained by performing the mask process on the delivered captured video on the monitor,
Computer program. A first image obtained by performing mask processing for changing a captured image of the imaging area distributed from a first camera of a plurality of cameras that capture an imaging area or a moving body appearing in the captured image to a mask image. To an information processing device that is a computer that displays edited video on a monitor,
In response to switching from the first camera to the second camera, each captured video that is generated from the background image of each captured video distributed from each camera, stored in the storage unit, and acquired from the storage unit A background image obtained by generating or updating a background image corresponding to the second camera is obtained from a server device that updates the background image of the camera at regular intervals and stores the updated background images in the storage unit. And displaying the second edited video obtained by performing the mask process on the captured video delivered from the second camera on the monitor using the captured video delivered from the second camera. To make it run
Computer program.