KR20230147360A - User service app execution by cloud surveillance camera system - Google Patents

Abstract

The present specification relates to a cloud monitoring camera system. The cloud monitoring camera system comprises: a monitoring camera that obtains user identification information by photographing a monitoring area and transmits the user identification information and monitoring camera identification information to a cloud server; and the cloud server that identifies a user photographed by the monitoring camera based on the user identification information and transmits a notification to a user terminal so that the notification confirming whether a user service app associated with a location where the monitoring camera is installed is executed is outputted to the user terminal of the identified user.

Description

User service app execution by cloud surveillance camera system {USER SERVICE APP EXECUTION BY CLOUD SURVEILLANCE CAMERA SYSTEM}

This specification relates to a system including a plurality of surveillance cameras connected to the cloud and the execution of a user service app using the same.

Recently, surveillance cameras are developing into intelligent surveillance camera systems that incorporate artificial intelligence, such as not only storing images captured in the surveillance area, but also recognizing people's faces in images captured in the surveillance area or detecting the occurrence of events. , It is widely used in various places such as shopping malls, gas stations, and public institutions.

Meanwhile, among various places where the above-mentioned surveillance camera system is installed, especially places such as stores, shopping malls, and gas stations, user services such as discount coupons and reward accumulation are provided in relation to services or products provided at those places, and these users In order to receive a service, the user installs a service provision application (hereinafter referred to as an app) on his or her portable terminal and then runs the app whenever necessary to receive the user service.

The content described above is only intended to help understand the background technology of the technical ideas of the present invention, and therefore, it cannot be understood as content corresponding to prior art known to those skilled in the art of the present invention.

However, in the case of the above-described user service, there is a problem that causes inconvenience to the user because each time the user visits a specific store, the user service app must be separately launched in order to receive the user service related to the store.

Meanwhile, there is a need for technology that automatically provides various services to people visiting a specific location by utilizing the object recognition technology of intelligent surveillance cameras.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

This specification presents a cloud surveillance camera system. The cloud surveillance camera system includes a surveillance camera that captures a surveillance area to obtain user identification information and transmits the user identification information and surveillance camera identification information to a cloud server; and identifying the user captured by the surveillance camera based on the user identification information, and outputting a notification to the user terminal of the identified user to determine whether a user service app associated with the location where the surveillance camera is installed is running. It may include a cloud server that transmits the notification to the user terminal as much as possible.

The cloud surveillance camera system and other embodiments may include the following features.

Depending on the embodiment, the surveillance camera may obtain the user identification information by analyzing captured image data for the surveillance area.

Depending on the embodiment, the user identification information includes the user's face image, the user's vehicle license plate number, and a user identification mark attached to the user, and the surveillance camera detects the user from captured image data for the surveillance area. The face image, the user's vehicle number, and the user identification mark attached to the user can be extracted.

Depending on the embodiment, the cloud server may register the user by receiving the user identification information from the user terminal.

Depending on the embodiment, the user service app associated with the location where the surveillance camera is installed may include a user app that provides service coupons provided at the location where the surveillance camera is installed, and a user app that accumulates rewards provided at the location where the surveillance camera is installed. It may be either a user app or a user app related to the use of a service provided at a location where the surveillance camera is installed.

Depending on the embodiment, the user service app may be executed according to a selection input for a notification confirming whether or not the user service app associated with the location where the surveillance camera is installed is executed.

Depending on the embodiment, the cloud server also associates the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed at a location providing the service provided by the user service app, and specifies the surveillance camera When a surveillance camera with identification information transmits user identification information, a control message can be sent to the user terminal of the identified user to output a notification confirming whether the user service app associated with the specific surveillance camera identification information is running. .

Meanwhile, this specification presents another cloud surveillance camera system. The cloud surveillance camera system includes a surveillance camera that captures a surveillance area to obtain user identification information and transmits the user identification information and surveillance camera identification information to a cloud server; And identifying the user captured by the surveillance camera based on the user identification information, and executing a user service app associated with the location where the surveillance camera is installed among the user apps installed on the user terminal of the identified user. It may include a cloud server that transmits an operation command for the installed user service management app to the user terminal.

The cloud surveillance camera system and other embodiments may include the following features.

Depending on the embodiment, the surveillance camera may obtain the user identification information by analyzing captured image data for the surveillance area.

Depending on the embodiment, the user identification information includes the user's face image, the user's vehicle license plate number, and a user identification mark attached to the user, and the surveillance camera detects the user from captured image data for the surveillance area. The face image, the user's vehicle number, and the user identification mark attached to the user can be extracted.

Depending on the embodiment, the cloud server receives the user identification information from the user terminal, registers a user corresponding to the user identification information, and associates the user service app with the registered user information. You can save it.

According to the embodiment, the user service management app is transmitted and installed on the user terminal before registration of the user identification information, transmits the user identification information to the cloud server, and outputs the monitoring to the user terminal. A pre-registered user service app can be executed by selecting a notification to check whether or not the user service app associated with the location where the camera is installed is selected.

Depending on the embodiment, the user service app associated with the location where the surveillance camera is installed may include a user app that provides service coupons provided at the location where the surveillance camera is installed, and a user app that accumulates rewards provided at the location where the surveillance camera is installed. It may be a user app, or a user app related to the use of services provided at a location where the surveillance camera is installed.

Depending on the embodiment, the cloud server also associates the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed at a location providing the service provided by the user service app, and specifies the surveillance camera When a surveillance camera with identification information transmits user identification information, the user service is managed by the user terminal to execute a user service app associated with the specific surveillance camera identification information among the user service apps installed on the user terminal of the identified user. You can send control messages to the app.

Meanwhile, this specification presents a method of executing a user service app by a cloud surveillance camera system including a surveillance camera and a cloud server. The method of executing the user service app includes obtaining user identification information by photographing a surveillance area by the surveillance camera; The surveillance camera transmitting the user identification information and surveillance camera identification information to the cloud server; The cloud server identifying a user based on the user identification information, and specifying a place where the identified user enters based on the user identification result and the surveillance camera identification information; and transmitting, by the cloud server, a notification confirming whether the user service app associated with the specified location is running to the user terminal of the identified user.

The method of executing a user service app by a cloud surveillance camera system including the surveillance camera and a cloud server and other embodiments may include the following features.

According to an embodiment, the method of executing the user service app includes receiving the user identification information and registering the user with the cloud server; And registering the surveillance camera identification information, the installation location information of the surveillance camera corresponding to the surveillance camera identification information, and the user service app associated with the user service provided at the installation location to the cloud server. It may further include; .

According to an embodiment, the method of executing the user service app includes the step of associating the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed at a location providing the service provided by the user service app. It may further include ;.

According to an embodiment, the user service app execution method further includes registering a user service app associated with a location where the surveillance camera is installed to the cloud server through a user service management app installed on the user terminal. And, when the user service management app receives the surveillance camera identification information and user identification information installed at a specific installation location from the cloud server, the user service management app installs the surveillance camera at the specific installation location among the user service apps installed on the user terminal of the identified user. A notification confirming whether the user service app associated with the camera identification information is executed is output, and the user service app can be executed according to the user's selection of the notification confirming whether the user service app is executed.

Depending on the embodiment, the user service app may be a user app that provides a discount coupon provided at a location where the surveillance camera is installed, a user app that accumulates rewards provided at a location where the surveillance camera is installed, or the surveillance camera. It may be a user app related to the use of services provided in a location where is installed.

Depending on the embodiment, the user identification information may include the user's face image, the user's vehicle license plate number, and an identification tag attached to the user.

According to embodiments disclosed herein, when a user visits a specific location where a surveillance camera is installed, the surveillance camera recognizes the user and automatically runs a user service app associated with the user service provided at the specific location. Zoom has the effect of making it easier for users to receive services.

Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention along with specific details for carrying out the invention. Therefore, the present invention is described in such drawings. It should not be interpreted as limited to the specific details.
1 is a conceptual diagram of a method of executing a user service app by a cloud surveillance camera system according to an embodiment.
Figure 2 is a schematic block diagram of a surveillance camera according to an embodiment.
Figure 3 is a schematic block diagram of a cloud server according to an embodiment.
Figure 4 is a diagram for explaining an AI processing unit applied to object recognition in a surveillance camera and a cloud server according to an embodiment.
Figure 5 is a flowchart showing a method of executing a user service app by a cloud surveillance camera system according to an embodiment.
Figure 6 is a flowchart showing a method of executing a user service app by a cloud surveillance camera system according to another embodiment.

The technology disclosed in this specification can be applied to surveillance cameras. However, the technology disclosed in this specification is not limited to this, and can be applied to all devices and methods to which the technical idea of the technology can be applied.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the spirit of the technology disclosed in this specification. In addition, the technical terms used in this specification, unless specifically defined in a different way in this specification, should be interpreted as meanings generally understood by those skilled in the art in the field to which the technology disclosed in this specification belongs. It should not be interpreted in a very comprehensive sense or in an excessively reduced sense. In addition, if the technical term used in this specification is an incorrect technical term that does not accurately express the idea of the technology disclosed in this specification, it is a technical term that can be correctly understood by a person with ordinary knowledge in the field to which the technology disclosed in this specification belongs. It should be understood and replaced with . Additionally, general terms used in this specification should be interpreted as defined in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Terms containing ordinal numbers, such as first, second, etc., used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component without departing from the scope of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numerals regardless of the reference numerals, and duplicate descriptions thereof will be omitted.

Additionally, when describing the technology disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the technology disclosed in this specification, and should not be construed as limiting the spirit of the technology by the attached drawings.

Videos or images according to an embodiment of the present invention include both still images and moving images, unless otherwise specified.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

1 is a conceptual diagram of a method of executing a user service app by a cloud surveillance camera system according to an embodiment.

Referring to FIG. 1, a cloud surveillance camera system 1000 according to an embodiment of the present specification may include a plurality of surveillance cameras 100 and a cloud server 200 installed in various locations.

The plurality of surveillance cameras 100 may be electronic devices for capturing images placed at fixed locations in various places, such as a super market 410, a gas station 420, a shopping mall 430, and a parking lot 440. The plurality of surveillance cameras 100 can each capture a monitored area as a video or still image.

The surveillance camera 100 may be a network camera used by connecting to the Internet by wire or wirelessly. Each of the surveillance cameras 100 may be a PTZ camera with pan, tilt, and zoom functions. The surveillance camera 100 may be a camera including a lens and an image sensor. The lens may be a lens group consisting of one or more lenses. An image sensor can convert an image input by a lens into an electrical signal.

The surveillance camera 100 may provide an image composed of RGB, an infrared image, or a distance image including distance information for the space to be captured, that is, the surveillance area. The surveillance camera 100 may have the function of recording sounds occurring in the monitored area. The surveillance camera 100 may have a function to generate a notification, record a video, or capture a still image when a change such as movement or sound occurs in the monitored area.

The cloud server 200 may be a device that receives the video itself captured through the surveillance camera 100 and/or the video obtained by editing the video and processes it for a specified purpose.

The cloud server 200 is connected to the surveillance camera 100 through a network and can transmit and receive video data, video data processing results, and various control data.

Each surveillance camera 100 transmits not only video data but also surveillance camera identification information, including information about the location where it is installed, such as a super market, gas station, shopping mall, parking lot, etc., to the cloud server 200. By registering itself, the cloud server 200 can distinguish which surveillance camera the received video data is from.

The communication network, which is a wired or wireless communication path between the cloud server 200 and the surveillance camera 100, is, for example, a wired network such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks). It may encompass wireless networks such as networks, wireless LANs, CDMA, Bluetooth, and satellite communications, but the scope of the present specification is not limited thereto.

Meanwhile, the cloud server 200 may store various learning models suitable for video analysis purposes. The cloud server 200 may store an artificial intelligence-based learning model that can identify objects such as a person's face or a car license plate number from image data received from the surveillance camera 100.

The cloud surveillance camera system 1000 or a portion thereof is connected to a video security solution such as a Digital Video Recorder (DVR), Central Management System (CMS), Network Video Recorder (NVR), and Video Management System (VMS) to collect video data. It may be delivered or included as part of these video security solutions.

The user 30 first downloads a user service app for user services provided in various places such as a super market, gas station, shopping mall, and parking lot from the app market or cloud server 200 and installs it on the user terminal 300.

The user 30 transmits his or her face image, vehicle number, or identification mark attached to the user, such as a QR code, to the cloud server 200 as user identification information that can identify the user through the user terminal 300 to the cloud server 200. It can be registered on the server 200. User identification information may be transmitted to the cloud server 200 through the user service app described above. The cloud server 200 stores the user service app installed on the user terminal 300 in association with surveillance camera identification information for the surveillance camera 100 installed at the location providing the service provided by the user service app.

The surveillance camera 100 photographs each surveillance area, and when an object such as a person or a car is photographed in the captured image, information that can identify the object, such as the person's face or the car number, is extracted. For example, when a surveillance camera installed at a gas station 420 among the surveillance cameras 100 captures a car in which a specific user 30 is riding, the vehicle is used as user identification information for the specific user 30 in the captured video. The number is extracted and transmitted to the cloud server (200). At this time, not only the car number but also the surveillance camera identification information is transmitted to the cloud server 200 as user identification information.

The cloud server 200 may identify the user captured by the surveillance camera based on user identification information received from the surveillance camera. The cloud server 200 can identify the user captured by the surveillance camera by comparing the user identification information with the user database, and the user identified based on the user identification result and the surveillance camera identification information is located at the location where the surveillance camera is installed. You can decide whether to enter or not. For example, when the cloud server 200 receives user identification information from a surveillance camera, the user service app can be used in a location where a surveillance camera is installed on the user terminal 300 of the user 30 corresponding to the user identification information. To execute, a notification is output to the user terminal, and the corresponding user service app can be executed according to the user's selection input. Reference numeral 350 shows an example of output from the user terminal 300 after user recognition in the cloud server 200. Since the image of the identified user was received from a surveillance camera installed at the gas station 420, the cloud server 200 determines that the user 30 visited the gas station 420 to refuel, and enters the gas station's payment details. A notification (310) confirming whether to print the reward accumulation barcode to the user terminal 300 so that the corresponding rewards can be accumulated is transmitted to the user terminal 300. Next, when the user 30 touches the notification 310 for barcode output, the user terminal 300 outputs the barcode 311 for reward accumulation to the user terminal 300. Accordingly, the user 30 can easily receive reward accumulation for purchases as a user service by touching the notification 310 automatically displayed on his terminal 300 without the need to run a separate app to accumulate rewards.

In addition, when the cloud server 200 determines that the user has entered the super market 410, it can print a discount coupon provided by the super market 410 on the user's terminal and allow the user to enter the shopping mall 430. If it is determined that the user has entered, a promotional coupon provided by the shopping mall 300 can be printed on the user's terminal. If it is determined that the user has entered the parking lot 440, parking information is provided to the user's terminal to indicate an empty parking space. You can also run apps. Although four cases are used as examples in this specification, any user service app related to the service provided to the customer at a specific location is registered in advance on the cloud server 200 and when the user visits the specific location, the service provided at the location is registered. User service apps related to the service can be run automatically.

According to another embodiment, user service apps for receiving user services provided at each place the user visits may be pre-installed on the user terminal 300, regardless of the system. In this case, the user can download the user service management app from the app market or cloud server 200 and install it on the user terminal 300. The user service management app can run a user service app related to a service provided at a specific location among the user service apps described above. The user service management app may transmit user identification information and a list of user service apps installed on the user terminal to the cloud server 200. The cloud server 200 may store each user service app in the database in the list of received user service apps in association with the identification information of the surveillance camera 100 installed at the location providing the service.

In this case, when the surveillance camera 100 acquires user identification information by photographing the surveillance area and transmits this along with the surveillance camera identification information to the cloud server 200, the cloud server 200 stores the received user identification information. Based on this, identify the user captured by the surveillance camera 100 and run the user service app associated with the location where the surveillance camera 100 that identified the user is installed among the user apps installed on the user terminal 300 of the identified user. An operation command for the user service management app installed on the user terminal 100 of the identified user is transmitted to the user terminal 100 of the identified user. The user service management app outputs a message to the user terminal 300 as to whether to run the user service app instructed to run in the notification, and when the user approves execution of the service app, it executes the user service app to allow the user to You can ensure that you receive services.

Figure 2 is a schematic block diagram of a surveillance camera according to an embodiment.

Referring to FIG. 2, the surveillance camera 100 may include an image sensor 110, an encoder 120, a memory 130, a processor 140, an AI processing unit 150, and a communication unit 160. .

The image sensor 110 performs the function of acquiring an image by photographing a surveillance area. For example, the image sensor converts optical signals into electrical signals (hereinafter referred to as CCD (Charge-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor)). It can be implemented with semiconductor devices that can be converted to (explained in the video).

The encoder 120 encodes the image acquired through the image sensor 110 into a digital signal, such as H.264, H.265, MPEG (Moving Picture Experts Group), M-JPEG (Motion Joint Photographic Experts Group) standards, etc.

The memory 130 can store video data, audio data, still images, metadata, etc. Metadata includes object detection information (movement, sound, intrusion into a designated area, etc.) captured in the surveillance area, object identification information (people, cars, faces, hats, clothes, etc.), and detected location information (coordinates, size, etc.). It may be data that it contains. Additionally, the still image is created with metadata and stored in the memory 130, and may be created by capturing image information for a specific analysis area among the image analysis information. As an example, a still image may be implemented as a JPEG image file. As an example, a still image may be created by cropping a specific area of the video data determined to be an identifiable object among the video data of the surveillance area detected in a specific area and for a specific period of time, which can be generated in real time along with metadata. can be transmitted.

The processor 140 controls all components constituting the surveillance camera 100 and can control all functions from image capture to extraction of object identification information and transmission of image data.

The AI processing unit 150 is for artificial intelligence image processing and extracts objects such as face images, vehicle license plates, and object identification marks attached to objects from images captured through surveillance cameras according to an embodiment of the present specification. Thus, identification information about the object can be obtained. The AI processing unit 150 may be implemented as a single module with the processor 140 that controls the overall system, or may be implemented as an independent module. Embodiments of the present specification may apply the YOLO (You Only Lock Once) algorithm in detecting and extracting identification information for objects. YOLO is an AI algorithm suitable for surveillance cameras that process real-time video because of its fast object detection speed. Unlike other object-based algorithms (Faster R-CNN, R_FCN, FPN-FRCN, etc.), the YOLO algorithm resizes one input image and passes it through a single neural network to indicate the location of each object. It outputs the object classification probability of the bounding box and the detected object. Finally, one object is recognized (detection) once through non-max suppression. Meanwhile, it should be noted that the object recognition algorithm disclosed in this specification is not limited to the above-described YOLO and can be implemented with various deep learning algorithms.

The communication unit 160 may transmit captured image data and acquired object identification information to the cloud server 200 through a network. Additionally, the communication unit 160 transmits video data, audio data, still images, and/or metadata to a video security solution or display device such as a DVR, CMS, NVR, or VMS. The communication unit 160 according to one embodiment may transmit video data, audio data, still images, and/or metadata to a video security solution in real time. The communication unit 160 may perform at least one communication function among wired and wireless LAN (Local Area Network), Wi-Fi, ZigBee, Bluetooth, and Near Field Communication.

Figure 3 is a schematic block diagram of a cloud server according to an embodiment.

Referring to FIG. 3, the cloud server 200 may include a communication unit 210, a decoder 220, a memory 230, a processor 240, an AI processing unit 250, and a database 260. .

The communication unit 210 receives video data, audio data, still images, and/or metadata from the surveillance camera 100, and transmits the received data to a video security solution or display device such as DVR, CMS, NVR, and VMS. You can. The communication unit 210 according to one embodiment may transmit video data, audio data, still images, and/or metadata to a video security solution in real time. The communication unit 210 communicates with the surveillance camera 100 using at least one communication function among wired and wireless LAN (Local Area Network), Wi-Fi, ZigBee, Bluetooth, and Near Field Communication. Communication can be performed.

The decoder 220 performs an operation of decoding the video received through the communication unit 160 of the surveillance camera 100 into a video signal, such as H.264, H.265, MPEG (Moving Picture Experts Group), It can follow M-JPEG (Motion Joint Photographic Experts Group) standards, etc.

The memory 230 can store video data, audio data, still images, metadata, etc. A still image is created with metadata and stored in the memory 230, and may be created by capturing image information for a specific analysis area among image analysis information. As an example, a still image may be implemented as a JPEG image file. As an example, a still image may be created by cropping a specific area of the video data determined to be an identifiable object among the video data of the surveillance area detected in a specific area and for a specific period of time, which can be generated in real time along with metadata. It can be saved as .

The processor 240 controls all components constituting the cloud server 200, and functions related to processing of image data, determination of installation location based on identification information of the surveillance camera 100, processing of object identification information, and object recognition. can be processed.

The AI processing unit 250 is for artificial intelligence image processing. According to an embodiment of the present specification, the AI processing unit 250 recognizes an object from the object identification information obtained through the surveillance camera 100 and determines who the person photographed in the surveillance area is. Or, you can determine what the car number is. The AI processing unit 250 may be implemented as one module with the processor 240 that controls the overall system, or may be implemented as an independent module. In embodiments for object recognition in the cloud server 200, an object recognition algorithm such as the You Only Lock Once (YOLO) algorithm may be applied in detecting and extracting identification information for an object.

The database 260 may include a user DB 261, a user service app DB 262, and a surveillance camera DB 263.

The user DB 261 is a database of user identification information about themselves provided by users who wish to be provided with the automatic execution service of the user service app by the cloud surveillance camera system 1000. User identification information may include a user's face image, a user's vehicle registration number, and a user identification mark attached to the user. This user identification information is received from the user terminal 300 and then matched to the corresponding user. and is registered in the cloud server 200 as user information.

The user service app DB 262 is a database that matches the user service app and user information associated with the user service provided for each location where the surveillance camera 100 is installed, and is a user service executable on the user terminal 300 for each registered user. It may be a database listing information about apps.

The surveillance camera DB 263 provides surveillance camera installation location information that matches surveillance camera identification information for the surveillance camera 100 connected to the cloud surveillance camera system 1000 and location information where the surveillance camera corresponding to the surveillance camera identification information is installed. It may be a database created to determine where the received video data was filmed based on specific surveillance camera identification information.

When the cloud server 200 receives user identification information and user service app information from the user terminal 300, the cloud server 200 provides a service provided by the user service app installed on the user terminal 300 of the user corresponding to the user identification information. It is stored in association with surveillance camera identification information for the surveillance camera 100 installed in a location that provides.

Therefore, the cloud server 200 can identify the person photographed by the specific surveillance camera based on the user identification information received from the specific surveillance camera and the user information data of the user DB 261, and can identify the person photographed by the specific surveillance camera. Based on the surveillance camera identification information received from the camera and the surveillance camera installation location information in the surveillance camera DB 263, it is possible to specify where the video for the identified user was filmed, that is, the location where the user entered. , Based on the user service app list information in the user service app DB 262, the identified user's terminal determines the user service app associated with the location where the user entered, and then transmits a control command to execute it to the user terminal. .

Figure 4 is a diagram for explaining an AI processing unit applied to object recognition in a surveillance camera and a cloud server according to an embodiment.

Looking at FIG. 4, the AI processing unit 500 may include an electronic device including an AI module capable of performing AI processing. Additionally, the AI processing unit 500 may be included as at least a portion of the surveillance camera 100 and the cloud server 200 and may be equipped to perform at least a portion of AI processing together. AI processing may include all operations related to the processor 140 of the surveillance camera 100 and the processor 240 of the cloud server 200. For example, the surveillance camera 100 and the cloud server 200 may perform processing/judgment and control signal generation operations by AI processing the acquired video signal.

The AI processing unit 500 may be a client device that directly uses AI processing results, or it may be a device in a cloud environment that provides AI processing results to other devices. The AI processing unit 500 is a computing device capable of learning a neural network, and may be implemented with various electronic devices such as a chip implemented as an integrated circuit, a server, a desktop PC, a laptop PC, and a tablet PC.

The AI processing unit 500 may include an AI processor 21, a memory 25, and/or a communication unit 27.

The AI processor 21 can learn a neural network using a program stored in the memory 25. In particular, the AI processor 21 can learn a neural network to recognize related data from surveillance cameras. Here, a neural network for recognizing related data from a surveillance camera may be designed to simulate the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. there is. Multiple network modes can exchange data according to each connection relationship to simulate the synaptic activity of neurons sending and receiving signals through synapses. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, multiple network nodes are located in different layers and can exchange data according to convolutional connection relationships. Examples of neural network models include deep neural networks (DNN), convolutional neural networks (CNN), Recurrent Networks (RNN), Restricted Boltzmann Machine (RBM), and deep trust neural networks ( It includes various deep learning techniques such as DBN (deep belief networks) and Deep Q-Network, and can be applied to fields such as computer vision, speech recognition, natural language processing, and voice/signal processing.

Meanwhile, the processor that performs the above-described functions may be a general-purpose processor (e.g., CPU), or may be an AI-specific processor (e.g., GPU) for artificial intelligence learning.

The memory 25 can store various programs and data necessary for the operation of the AI processing unit 500. The memory 25 can be implemented as non-volatile memory, volatile memory, flash-memory, hard disk drive (HDD), or solid state drive (SDD). The memory 25 is accessed by the AI processor 21, and reading/writing/modifying/deleting/updating data by the AI processor 21 can be performed. Additionally, the memory 25 may store a neural network model (eg, deep learning model 26) generated through a learning algorithm for data classification/recognition according to an embodiment of the present invention.

Meanwhile, the AI processor 21 may include a data learning unit 22 that learns a neural network for data classification/recognition. The data learning unit 22 can learn standards regarding what learning data to use to determine data classification/recognition and how to classify and recognize data using the learning data. The data learning unit 22 can learn a deep learning model by acquiring learning data to be used for learning and applying the acquired learning data to the deep learning model.

The data learning unit 22 may be manufactured in the form of at least one hardware chip and mounted on the AI processing unit 500. For example, the data learning unit 22 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (CPU) or a graphics processor (GPU) to form the AI processing unit 500. It may be mounted on . Additionally, the data learning unit 22 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable recording medium that can be read by a computer. In this case, at least one software module may be provided by an operating system (OS) or an application.

The data learning unit 22 may include a learning data acquisition unit 23 and a model learning unit 24.

The learning data acquisition unit 23 may acquire learning data required for a neural network model for classifying and recognizing data.

The model learning unit 24 can use the acquired training data to train the neural network model to have a judgment standard on how to classify certain data. At this time, the model learning unit 24 can learn a neural network model through supervised learning that uses at least some of the learning data as a judgment standard. Alternatively, the model learning unit 24 can learn a neural network model through unsupervised learning, which discovers a judgment standard by learning on its own using training data without guidance. Additionally, the model learning unit 24 can learn a neural network model through reinforcement learning using feedback on whether the result of the situational judgment based on learning is correct. Additionally, the model learning unit 24 may learn a neural network model using a learning algorithm including error back-propagation or gradient descent.

When the neural network model is learned, the model learning unit 24 may store the learned neural network model in memory. The model learning unit 24 may store the learned neural network model in the memory of a server connected to the AI processing unit 500 through a wired or wireless network.

The data learning unit 22 further includes a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or save the resources or time required for generating the recognition model. You may.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning to determine the situation. For example, the learning data preprocessor may process the acquired data into a preset format so that the model learning unit 24 can use the acquired learning data for training for image recognition.

Additionally, the learning data selection unit may select data required for learning from among the learning data acquired by the learning data acquisition unit 23 or the learning data pre-processed by the pre-processing unit. The selected learning data may be provided to the model learning unit 24.

Additionally, the data learning unit 22 may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

The model evaluation unit inputs evaluation data into the neural network model, and when the analysis result output from the evaluation data does not satisfy a predetermined standard, the model learning unit 22 can perform re-training. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluation unit may evaluate the evaluation data as not meeting a predetermined standard if the number or ratio of inaccurate evaluation data exceeds a preset threshold among the analysis results of the learned recognition model for the evaluation data. there is.

The communication unit 27 can transmit the results of AI processing by the AI processor 21 to an external electronic device. For example, external electronic devices may include other surveillance cameras, video security solutions such as DVR, CMS, NVR, and VMS, or display devices.

Meanwhile, the illustrated AI processing unit 500 is described as functionally divided into the AI processor 21, memory 25, communication unit 27, etc., but the above-mentioned components are integrated into one module and are called AI module. Let's make it clear that it can happen.

Hereinafter, various embodiments of a method of executing a user service app by the cloud surveillance camera system 1000 will be described with reference to the attached FIGS. 1 to 6.

Example 1: Running a user service-only app

Figure 5 is a flowchart showing a method of executing a user service app by a cloud surveillance camera system according to an embodiment.

Referring to FIGS. 2, 3, and 5, first, the surveillance camera 100 transmits surveillance camera identification information about itself and information about the installation location of the surveillance camera corresponding to the surveillance camera identification information to the cloud server 200 (S501) ), the cloud server 200 stores the surveillance camera 100 corresponding to the received surveillance camera identification information and the location where the surveillance camera 100 is installed in the surveillance camera DB 263 and registers the surveillance camera 100 ( S503).

Meanwhile, the user installs the user service app associated with the user service provided at the installation location of the surveillance camera 100 on the user terminal 300 (S505), and then sends the user identification information and the installed user service app information to the cloud server (200). ) (S507).

The cloud server 200 registers a user corresponding to the received user identification information, and records the user service app installed on the corresponding user terminal 300 in the user information. A surveillance camera installed at a location providing the service provided by the user service app ( 100) is stored in association with the surveillance camera identification information (S509). Here, the user service app is a user app that provides discount coupons provided at a location where a surveillance camera is installed, a user app that accumulates rewards provided at a location where a surveillance camera is installed, or a service provided at a location where the surveillance camera is installed. This could be a user app related to usage.

Next, the surveillance camera 100 captures the surveillance area and obtains user identification information (S511), and then transmits the acquired user identification information and surveillance camera identification information to the cloud server 200 (S513).

The cloud server 200 identifies the user based on the received user identification information and specifies the location where the identified user enters based on the user identification result and the surveillance camera identification information (S515).

Finally, the cloud server 200 transmits a notification confirming whether the user service app associated with the specified location is running to the user terminal 300 of the identified user (S517).

Afterwards, when the user selects to run the user service app in response to the notification, the user terminal 300 runs the user service app associated with the specified location and outputs a discount coupon or reward accumulation barcode provided at the specified location. Provides user services such as (S519).

Example 2: Running a user service app using a user service management app

Figure 6 is a flowchart showing a method of executing a user service app by a cloud surveillance camera system according to another embodiment.

Referring to FIGS. 2, 3, and 6, first, the surveillance camera 100 transmits surveillance camera identification information about itself and information about the installation location of the surveillance camera corresponding to the surveillance camera identification information to the cloud server 200 (S601) ), the cloud server 200 stores the surveillance camera 100 corresponding to the received surveillance camera identification information and the location where the surveillance camera 100 is installed in the surveillance camera DB 263 and registers the surveillance camera 100 ( S603).

Meanwhile, the user installs a user service management app that can run user service apps related to the user service provided at the installation location of the surveillance camera 100 on the user terminal 300, and installs the surveillance camera (100) in the user service management app. ) Register user service apps related to user services provided at the installation location (S605).

When the user terminal 300 transmits user identification information and information about the installed user service app to the cloud server 200 (S607), the cloud server 200 registers the user corresponding to the received user identification information, and the user The user service app that can be executed on the corresponding user terminal is matched to the information and registered (S609). Here, the user service app is a user app that provides discount coupons provided at a location where a surveillance camera is installed, a user app that accumulates rewards provided at a location where a surveillance camera is installed, or a service provided at a location where the surveillance camera is installed. This could be a user app related to usage.

Next, the surveillance camera 100 photographs the surveillance area to obtain user identification information (S611), and then transmits the obtained user identification information and surveillance camera identification information to the cloud server 200 (S613).

The cloud server 200 identifies the user based on the received user identification information, and specifies the place where the identified user enters, that is, the place where the surveillance camera captured the user, based on the user identification result and the surveillance camera identification information. (S615).

The cloud server 200 transmits the specified location information and a notification confirming whether the user service app associated with the specified location is executed to the user terminal 300 of the identified user (S617).

Afterwards, the user service management app of the user terminal 300 outputs a notification to the user terminal 300 to confirm whether to run the user service app associated with the location specified as entered by the user among the pre-registered user service apps. Do it (S619).

Finally, when the user selects to run the user service app in response to the notification, the user terminal 300 runs the user service app associated with the specified location and outputs a discount coupon or reward accumulation barcode provided at the specified location. Provides user services such as (S621).

In the above description, steps, processes or operations may be further divided into additional steps, processes or operations, or may be combined into fewer steps, processes or operations, depending on the implementation of the invention. Additionally, some steps, processes, or operations may be omitted, or the order between steps or operations may be switched, as needed. In addition, each step or operation included in the method for generating the panoramic surveillance image described above may be implemented as a computer program and stored in a computer-readable recording medium, and each step, process, or operation may be executed by a computer device. .

The term “unit” (e.g., control unit, etc.) used herein may mean, for example, a unit including one or a combination of two or more of hardware, software, or firmware. “Part” may be used interchangeably with terms such as unit, logic, logical block, component, or circuit, for example. A “part” may be the minimum unit of an integrated part or a part thereof. “Part” may be the minimum unit or part of one or more functions. The “part” may be implemented mechanically or electronically. For example, a “part” may be an Application-Specific Integrated Circuit (ASIC) chip, Field-Programmable Gate Arrays (FPGAs), or programmable-logic device, known or to be developed in the future, that performs certain operations. It can contain at least one.

At least a portion of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be stored in a computer-readable storage media, e.g., in the form of a program module. It can be implemented with instructions stored in . When the instruction is executed by a processor, the one or more processors may perform the function corresponding to the instruction. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Computer-readable storage media/computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), and optical media (e.g. CD-ROM (compact disc read only memory, digital versatile disc (DVD), magneto-optical media (e.g., floptical disk), hardware devices (e.g., read only memory (ROM), random disk (RAM)) access memory, or flash memory, etc.), and may also include those implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, program instructions include machine language such as that created by a compiler. In addition to code, it may include high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate as one or more software modules to perform the operations of various embodiments, and vice versa. Same thing.

A module or program module according to various embodiments may include at least one of the above-described components, some of them may be omitted, or may further include other additional components. Operations performed by modules, program modules, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic manner. Additionally, some operations may be executed in a different order, omitted, or other operations may be added.

As used herein, the term “one” is defined as one or more than one. Additionally, the use of introductory phrases such as “at least one” and “one or more” in a claim may mean that the same claim contains introductory phrases such as “at least one” and “one or more” and ambiguous phrases such as “an.” The introduction of another claim element by the ambiguous phrase "a", if any, shall be construed to mean that any particular claim containing the claim element so introduced is limited to an invention containing only one such element. It shouldn't be.

In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of the items listed together.

Unless otherwise specified, terms such as “first” and “second” are used to optionally distinguish between the elements described by such terms. Accordingly, these terms are not necessarily intended to indicate temporal or other priority of such elements, and the mere fact that particular means are recited in different claims does not indicate that a combination of such means cannot be advantageously used. . Accordingly, these terms are not necessarily intended to indicate temporal or other priority of such elements. The mere fact that a particular measure is cited in different claims does not indicate that a combination of these measures cannot be used usefully.

The phrase “may be X” indicates that condition X may be met. This phrase also indicates that condition X may not be met. For example, a reference to a system containing a specific component should also include scenarios in which the system does not contain the specific component. For example, a reference to a method that includes a specific behavior should also include scenarios in which the method does not include that specific component. However, as another example, a reference to a system configured to perform a specific action should also include scenarios in which the system is not configured to perform a specific task.

The terms “comprising,” “having,” “consisting of,” “consisting of,” and “consisting essentially of” are used interchangeably. For example, any method may include at least the operations included in the drawings and/or the specification, or may include only the operations included in the drawings and/or the specification. Alternatively, the word “comprising” does not exclude the presence of elements or acts listed in a claim.

Connections as described herein may be any type of connection suitable for transmitting signals to or from each node, unit or device, for example via an intermediate device. Accordingly, unless implied or otherwise stated, a connection may be, for example, a direct connection or an indirect connection. A connection may be described or depicted with reference to being a single connection, multiple connections, one-way connection, or two-way connection. However, different embodiments may vary the implementation of the connection. For example, you could use a separate one-way connection rather than a two-way connection, or vice versa. Additionally, multiple connections can be replaced with a single connection that transmits multiple signals sequentially or in a time-multiplexed manner. Likewise, a single connection carrying multiple signals may be split into various connections carrying subsets of those signals. Therefore, many options exist for transmitting signals.

In the above, preferred embodiments of the technology of this specification have been described with reference to the attached drawings. Here, the terms or words used in this specification and claims should not be construed as limited to their usual or dictionary meanings, but should be construed as meanings and concepts consistent with the technical idea of the present invention. The scope of the present invention is not limited to the embodiments disclosed in this specification, and the present invention may be modified, changed, or improved in various forms within the scope described in the spirit and claims of the present invention.

Claims (20)

A surveillance camera that captures user identification information by photographing the surveillance area and transmits the user identification information and surveillance camera identification information to a cloud server; and
Identify the user captured by the surveillance camera based on the user identification information, and output a notification to the user terminal of the identified user to determine whether a user service app associated with the location where the surveillance camera is installed is executed. A cloud server transmitting the notification to the user terminal; including
Cloud surveillance camera system. The method of claim 1, wherein the surveillance camera,
Obtaining the user identification information by analyzing captured video data for the surveillance area
A cloud surveillance camera system characterized in that. According to clause 2,
The user identification information includes the user's face image, the user's vehicle license plate number, and a user identification mark attached to the user,
The surveillance camera extracts the user's face image, the user's vehicle number, and the user identification mark attached to the user from the captured image data for the surveillance area.
A cloud surveillance camera system characterized in that. The method of claim 1, wherein the cloud server:
Registering a user by receiving the user identification information from the user terminal
A cloud surveillance camera system characterized in that. The user service app of claim 1, where the surveillance camera is installed,
A user app that provides coupons for services provided at a location where the surveillance camera is installed, a user app that accumulates rewards provided at a location where the surveillance camera is installed, or a user app related to the use of services provided at a location where the surveillance camera is installed. which one of
A cloud surveillance camera system characterized in that. The method of claim 5, wherein the user service app,
Executed according to the selection input for the notification confirming whether or not the user service app associated with the location where the surveillance camera is installed is running.
A cloud surveillance camera system characterized in that. The method of claim 6, wherein the cloud server:
Associating the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed in a location providing the service provided by the user service app,
When a surveillance camera with specific surveillance camera identification information transmits user identification information, a control message is sent to the user terminal of the identified user to output a notification confirming whether the user service app associated with the specific surveillance camera identification information is running. transmitting
A cloud surveillance camera system characterized in that. A surveillance camera that captures user identification information by photographing the surveillance area and transmits the user identification information and surveillance camera identification information to a cloud server; and
Based on the user identification information, the user captured by the surveillance camera is identified, and among the user apps installed on the user terminal of the identified user, a user service app associated with the location where the surveillance camera is installed is installed on the user terminal. A cloud server that transmits an operation command for the user service management app to the user terminal; including
Cloud surveillance camera system. The method of claim 8, wherein the surveillance camera,
Obtaining the user identification information by analyzing captured video data for the surveillance area
A cloud surveillance camera system characterized in that. According to clause 9,
The user identification information includes the user's face image, the user's vehicle license plate number, and a user identification mark attached to the user,
The surveillance camera extracts the user's face image, the user's vehicle number, and the user identification mark attached to the user from the captured image data for the surveillance area.
A cloud surveillance camera system characterized in that. The method of claim 8, wherein the cloud server:
After receiving the user identification information from the user terminal, registering a user corresponding to the user identification information,
Stores the user service app in association with the registered user information.
A cloud surveillance camera system characterized in that. The method of claim 11, wherein the user service management app,
is transmitted and installed on the user terminal before registration of the user identification information,
transmitting the user identification information to the cloud server,
Executing a pre-registered user service app according to a selection input for a notification output to the user terminal to check whether or not the user service app associated with the location where the surveillance camera is installed is executed.
A cloud surveillance camera system characterized in that. The method of claim 12, wherein the user service app associated with the location where the surveillance camera is installed,
A user app that provides coupons for services provided at a location where the surveillance camera is installed, a user app that accumulates rewards provided at a location where the surveillance camera is installed, or a user app related to the use of services provided at a location where the surveillance camera is installed.
A cloud surveillance camera system characterized in that. The method of claim 8, wherein the cloud server:
Associating the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed in a location providing the service provided by the user service app,
When a surveillance camera with specific surveillance camera identification information transmits user identification information, the user terminal is instructed to execute a user service app associated with the specific surveillance camera identification information among the user service apps installed on the user terminal of the identified user. Sending control messages to user service management apps
A cloud surveillance camera system characterized in that. In a method of executing a user service app by a cloud surveillance camera system including a surveillance camera and a cloud server,
Obtaining user identification information by having the surveillance camera capture a surveillance area;
The surveillance camera transmitting the user identification information and surveillance camera identification information to the cloud server;
The cloud server identifying a user based on the user identification information, and specifying a place where the identified user enters based on the user identification result and the surveillance camera identification information; and
Transmitting, by the cloud server, a notification confirming whether a user service app associated with the specified location is running to the user terminal of the identified user;
How to include it. According to claim 15,
receiving the user identification information and registering the user with the cloud server; and
Registering the surveillance camera identification information, installation location information of the surveillance camera corresponding to the surveillance camera identification information, and a user service app associated with a user service provided at the installation location to the cloud server;
How to include more. According to claim 15,
Associating the user service app installed on the user terminal with surveillance camera identification information for a surveillance camera installed at a location providing the service provided by the user service app;
How to include more. According to claim 17,
Registering a user service app associated with a location where the surveillance camera is installed on the cloud server through a user service management app installed on the user terminal, further comprising:
The user service management app is,
When receiving surveillance camera identification information and user identification information installed at a specific installation location from the cloud server, the user service app associated with the surveillance camera identification information installed at the specific installation location is selected from among the user service apps installed on the user terminal of the identified user. Prints a notification to confirm execution,
Running the user service app according to the user's selection in response to a notification confirming whether the user service app is running.
A method characterized by: The method of claim 15, wherein the user service app,
A user app that provides discount coupons provided at a location where the surveillance camera is installed, a user app that accumulates rewards provided at a location where the surveillance camera is installed, or a user app related to the use of services provided at a location where the surveillance camera is installed.
A method characterized by: The method of claim 15, wherein the user identification information is:
Contains the user's face image, the user's vehicle license plate number, and the identification tag attached to the user.
A method characterized by:

Images