WO2024164928A1

WO2024164928A1 - Early warning method and apparatus, and electronic device and storage medium

Info

Publication number: WO2024164928A1
Application number: PCT/CN2024/075218
Authority: WO
Inventors: 唐晨晨; 唐军; 陆振心; 莫传喜; 张泽蛟
Original assignee: 深圳市冠标科技发展有限公司
Priority date: 2023-02-08
Filing date: 2024-02-01
Publication date: 2024-08-15
Also published as: CN118696361A

Abstract

An early warning method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring, in real time, video data which is collected by a camera device; analyzing the video data to determine behavior features of each pedestrian in the video data; when the behavior features of the pedestrians comprise a dangerous behavior, sending prompt information and the video data to a user equipment; receiving warning information from the user equipment, wherein the warning information is determined by the user equipment according to the prompt information and the video data; performing analog-to-digital conversion processing on the warning information, so as to obtain first digital information; performing compression processing on the first digital information, so as to obtain second digital information; performing channel coding processing on the second digital information, so as to obtain third digital information; and sending third digital information to an audio playing device, so as to play the warning information by means of the audio playing device and warn the pedestrian whose behavior feature is a dangerous behavior.

Description

Early warning method, device, electronic device and storage medium

This application claims the priority of the Chinese patent application filed with the China Patent Office on February 8, 2023, with application number 202310084476.7 and application name “Pedestrian Warning Method, Device, Electronic Device and Storage Medium”, all contents of which are incorporated by reference in this application.

Technical Field

The present invention relates to the field of artificial intelligence technology, and in particular to an early warning method, device, electronic equipment and storage medium.

Background Art

Traffic lights are used as signal lights at intersections to provide traffic signals to passers-by. Existing traffic light equipment basically only serves as a signal light. For pedestrians who run red lights, they can only be warned by nearby traffic police. However, whether it is processed through monitoring records or handled by traffic police on the scene, there is a certain delay. As a result, relevant warning and education can only be carried out after pedestrians have completed the dangerous behavior of running red lights, and it is impossible to effectively prevent the occurrence of such behavior.

Summary of the invention

In order to solve the above-mentioned problems existing in the prior art, the embodiments of the present application provide an early warning method, device, electronic device and storage medium, which can promptly issue warnings to pedestrians who engage in dangerous behaviors such as running red lights, prevent pedestrians from continuing their current dangerous behaviors, have higher timeliness, and can effectively prevent the occurrence of dangerous behaviors.

In a first aspect, an embodiment of the present application provides an early warning method, which is applied to a cloud server in an early warning system, wherein the early warning system further includes a user device, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud service and the audio playback device is greater than a first threshold, and the method specifically includes:

Obtain video data collected by camera equipment in real time;

Analyze the video data to determine the behavioral characteristics of each pedestrian in the video data;

When the behavior characteristics of each pedestrian include dangerous behavior, prompt information and video data are sent to the user device;

Receiving warning information from a user device, wherein the warning information is determined by the user device according to the prompt information and the video data;

Performing analog-to-digital conversion on the warning information to obtain first digital information;

compressing the first digital information to obtain second digital information;

performing channel coding processing on the second digital information to obtain third digital information;

The third digital information is sent to the audio playback device so as to play the warning information through the audio playback device to warn the pedestrians whose behavior characteristics are dangerous behaviors.

In a second aspect, an embodiment of the present application provides an information sending method, which is applied to a user device in an early warning system, wherein the early warning system further includes a cloud server, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud server and the audio playback device is greater than a first threshold, and the method specifically includes:

receiving prompt information and video data from a cloud server, wherein the video data includes a marker;

Display prompt information and video data to users;

Receive warning information input by the user;

Send warning information to the cloud server.

In a third aspect, an embodiment of the present application provides an early warning device, which is applied to a cloud server in an early warning system. The early warning system also includes a user device, and a camera device and an audio playback device arranged at both ends of the pedestrian passage, and The distance between the cloud service and the audio playback device is greater than a first threshold, and the device specifically includes:

A first receiving module, used for acquiring video data collected by a camera device in real time;

An analysis module, used to analyze the video data and determine the behavior characteristics of each pedestrian in the video data;

A first sending module, used for sending prompt information and video data to a user device when the behavior characteristics of each pedestrian include dangerous behavior;

The first receiving module is further used to receive warning information from the user equipment, wherein the warning information is determined by the user equipment according to the prompt information and the video data;

The analysis module is further used to perform analog-to-digital conversion processing on the warning information to obtain first digital information, perform compression processing on the first digital information to obtain second digital information, and perform channel coding processing on the second digital information to obtain third digital information;

The first sending module is further used to send third digital information to the audio playback device so as to play the warning information through the audio playback device to warn pedestrians whose behavior characteristics are dangerous behaviors.

In a fourth aspect, an embodiment of the present application provides an information sending device, which is applied to a user device in an early warning system, wherein the early warning system also includes a cloud server, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud service and the audio playback device is greater than a first threshold, and the device specifically includes:

A second receiving module is used to receive prompt information and video data from a cloud server, wherein the video data includes a mark;

A display module is used to display prompt information and video data to users;

The second receiving module is further used to receive warning information input by the user;

The second sending module is used to send warning information to the cloud server.

In a fifth aspect, an embodiment of the present application provides an early warning device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method of the first aspect.

In a sixth aspect, an embodiment of the present application provides an information sending device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method of the second aspect.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute a method as described in the first aspect or the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to cause the computer to execute the method of the first aspect or the second aspect.

The implementation of the present application has the following beneficial effects:

In the implementation mode of the present application, the video data of the pedestrian passage is obtained in real time by the camera equipment arranged at both ends of the pedestrian passage, and then the obtained video data is analyzed to determine the behavior characteristics of each pedestrian in the video data. Then, the action of each pedestrian is predicted according to the behavior characteristics of each pedestrian. When the probability of the pedestrian making dangerous behaviors such as running a red light is predicted to be higher than the preset value, the behavior characteristics can be determined to be dangerous behaviors. When the behavior characteristics of each pedestrian include dangerous behaviors, prompt information and video data are sent to the user device, prompting the backstage staff to pay attention to the pedestrian through the user device, and then when the pedestrian makes dangerous behaviors such as running a red light, a warning is issued to the pedestrian in time to prevent the pedestrian from continuing the current dangerous behavior. Compared with the traditional warning method, it has higher timeliness and can effectively prevent the occurrence of dangerous behaviors. At the same time, the warning information is converted by analog to digital, compressed and channel coded, and then the processed first warning voice is sent to the audio playback device for playback through the long-range radio (Long Range Radio, LoRa) protocol, which overcomes the problem of easy packet loss in the LoRa protocol and realizes low-cost and long-distance signal transmission.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the implementation modes of the present application, the drawings required for use in the description of the implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of the hardware structure of an early warning device provided in an embodiment of the present application;

FIG2 is a framework diagram of an early warning system provided in an embodiment of the present application;

FIG3 is a schematic diagram of a flow chart of an early warning method provided in an embodiment of the present application;

FIG4 is a schematic diagram of a video data screen with a mark provided in an embodiment of the present application;

FIG5 is a block diagram of the functional modules of an early warning device provided in an embodiment of the present application;

FIG6 is a block diagram of the functional modules of an information sending device provided in an embodiment of the present application;

FIG7 is a schematic diagram of the structure of an early warning device provided in an embodiment of the present application;

FIG8 is a schematic diagram of the structure of an information sending device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

Reference to "embodiment" herein means that a particular feature, result, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

First, refer to FIG1 , which is a schematic diagram of the hardware structure of an early warning device provided in an embodiment of the present application. The early warning device 100 includes at least one processor 101 , a communication line 102 , a memory 103 and at least one communication interface 104 .

In this embodiment, the processor 101 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

The communication link 102 may include a path to transmit information between the above components.

The communication interface 104 can be any transceiver-like device (such as an antenna, etc.) used to communicate with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

The memory 103 may be a read-only memory (ROM) or other static storage device that can store static information and instructions, a random access memory (RAM) or other dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EPROM). programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

In this embodiment, the memory 103 can exist independently and be connected to the processor 101 via the communication line 102. The memory 103 can also be integrated with the processor 101. The memory 103 provided in the embodiment of the present application can generally be non-volatile. Among them, the memory 103 is used to store computer-executable instructions for executing the scheme of the present application, and the execution is controlled by the processor 101. The processor 101 is used to execute the computer-executable instructions stored in the memory 103, thereby implementing the method provided in the following embodiment of the present application.

In an optional implementation, the computer-executable instructions may also be referred to as application code, which is not specifically limited in this application.

In an optional implementation, the processor 101 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 1 .

In an optional embodiment, the early warning device 100 may include multiple processors, such as the processor 101 and the processor 107 in FIG. 1. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In an optional embodiment, if the early warning device 100 is a server, for example, it can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. Then the early warning device 100 can also include an output device 105 and an input device 106. The output device 105 communicates with the processor 101 and can display information in a variety of ways. For example, the output device 105 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 106 communicates with the processor 101 and can receive user input in a variety of ways. For example, the input device 106 can be a mouse, a keyboard, a touch screen device, or a sensor device.

The above-mentioned early warning device 100 can be a general device or a special device. The embodiment of the present application does not limit the type of the early warning device 100.

Secondly, Figure 2 is a framework diagram of an early warning system provided by an embodiment of the present application. Specifically, the system may include: a cloud server 201, a camera device 202, an audio playback device 203, a user device 204 and a display device 205, wherein the distance between the cloud server 201 and the audio playback device 203 is greater than a first threshold.

In this embodiment, the cloud server 201 can be a smart phone (such as an Android phone, an iOS phone, a Windows Phone, etc.), a tablet computer, a PDA, a laptop computer, a mobile Internet device MID (Mobile Internet Devices, referred to as: MID), a server, a chip, a chipset, etc., which is used to receive data transmitted by other devices, and process these data to generate new data and send it to the corresponding device.

In this embodiment, the camera devices 202 can be grouped in two and respectively arranged at the two ends of the pedestrian passage to fully capture the pedestrians at both ends of the pedestrian passage, and accurately locate the pedestrians in the shooting area based on the principle of binocular visual positioning. Specifically, the camera device 202 can be a mobile Internet device MID with a camera function, such as various types of cameras, video cameras, etc., for acquiring the target area including the pedestrian passage, that is, the real-time video stream data of the common shooting area of the two camera devices 202, and sending it to the cloud server 201 for analysis.

In this embodiment, the audio playing devices 203 can also be grouped in two and respectively disposed at the two ends of the pedestrian passage. Specifically, it can be a device with voice playback and voice collection functions, used to receive voice information sent by the cloud server 201 for playback, or obtain voice information at the device and send it to the cloud server 201 for processing.

In this embodiment, the user device 204 can be a personal mobile device such as a smart phone, a tablet computer, a PDA, a laptop computer, or a fixed device such as a server. The user device 204 is provided with an alarm device, a display device, and an input device, wherein the alarm device is used to generate a corresponding alarm according to the received prompt information to remind the background user to pay attention to the display device; the display device is used to display the received video data; and the input device is used to receive the input information of the background user and convert it into a warning message and send it to the cloud server 201.

In this embodiment, the display devices 205 can also be arranged in groups of two, respectively, at the two ends of the pedestrian passage. Specifically, it can be a device with a display function, such as a light-emitting diode (LED) display screen, a liquid crystal display screen, etc., for receiving text, video or image information sent by the cloud server 201 for display.

Finally, the early warning method proposed in this application can also be applied to other access restriction scenarios, such as highly confidential scientific research sites, factory workshops, etc., and then in different scenarios, the management personnel can also change accordingly. For example, for scientific research scenarios, the early warning method proposed in this application can be changed into an early warning method for scientific researchers, which is used to warn scientific researchers who have insufficient access rights to enter places; similarly, for factory scenarios, the early warning method proposed in this application can be changed into a workshop worker method, which is used to warn workshop workers who enter the wrong place. In this embodiment, the pedestrian passage scenario is taken as an example, and pedestrians are taken as the execution object to illustrate the early warning method proposed in this application. The early warning methods in other scenarios are similar to the early warning methods in the pedestrian passage scenario, and will not be repeated here.

The following will take the traffic scenario as an example to illustrate the warning method disclosed in this application:

Refer to Figure 3, which is a schematic diagram of a flow chart of an early warning method provided in an embodiment of the present application. The early warning method can be applied to the cloud server 201 in the early warning system shown in Figure 2, and can specifically include the following steps:

301: The cloud server obtains the video data collected by the camera device in real time.

In this embodiment, a camera device 202 is respectively provided at both ends of the pedestrian passage, for real-time acquisition of video stream data in the shooting area, and synchronization of the acquired video data to the cloud server 201 in real time. Specifically, each camera device 202 can capture the entire pedestrian passage, and the intersection of the shooting areas of the two camera devices 202 includes the entire pedestrian passage. Thus, the images of pedestrians at both ends of the pedestrian passage can be better acquired, and the setting of dual camera devices is used to simulate binocular vision, and then the principle of binocular vision positioning can be used to more accurately identify the position of each pedestrian in the video data.

302: The cloud server analyzes the video data to determine the behavior characteristics of each pedestrian in the video data.

In this embodiment, the video data may be framed to obtain at least one video image. Specifically, the framed images may be screened to select images of a series of different actions and arrange them in the order in the video data to obtain at least one video image.

Then, the motion features of each pedestrian in at least one video image can be extracted to obtain the motion feature sequence of each pedestrian. Specifically, the features of the postures of each pedestrian in the above-mentioned series of different motion images arranged in sequence can be extracted respectively, and the extracted features can be horizontally spliced according to the order of the corresponding images to obtain the motion feature sequence.

Finally, the action feature sequence of each pedestrian can be compared with the preset dangerous behavior feature library to determine the behavior feature of each pedestrian. In this embodiment, the historical data can be analyzed to extract the behavior features of pedestrians who have performed dangerous behaviors such as running red lights in the historical data center before performing dangerous behaviors such as running red lights. Through machine learning, the probability of performing dangerous behaviors such as running red lights after each behavior feature is determined, and each behavior feature is associated with its corresponding probability and stored in the dangerous behavior feature library. Therefore, after obtaining the action feature sequence of each pedestrian, It is compared with the behavior features stored in the dangerous behavior feature library, and then the probability of each pedestrian performing dangerous behaviors such as running a red light is predicted, and then the behavior features of the pedestrian are determined based on the probability.

Specifically, in this embodiment, when the probability of performing dangerous behaviors such as running a red light is greater than or equal to a preset threshold, the behavior characteristics of the pedestrian can be determined as dangerous behaviors. When the probability of performing dangerous behaviors such as running a red light is less than a preset threshold, the behavior characteristics of the pedestrian can be determined as normal behaviors.

303: When the behavior characteristics of each pedestrian include dangerous behavior, the cloud server sends prompt information and video data to the user device.

In this embodiment, the probability of each dangerous behavior that the pedestrian may perform can be determined in the dangerous behavior feature library according to the action feature sequence in step 302, for example: [running a red light, 50%]; [crossing the road in a non-pedestrian passage, 20%]; [crossing the road on a bicycle, 30%], as prompt information. At the same time, in the video data, the pedestrian whose behavior characteristics are dangerous behaviors is marked. For example: use a red box to frame its image in the video, and the box can move synchronously with the movement of the pedestrian in the video, and the above prompt information can be marked next to the box, as shown in Figure 4.

In an optional implementation, the prompt information may also include audio prompt information. Specifically, after obtaining the probability of each dangerous behavior that the pedestrian may perform and marking the pedestrian in the video, a corresponding probability broadcast voice and prompt voice may be generated according to the probability of each dangerous behavior to broadcast to the user of the user device, so as to prevent the user from being away from the user device and not being able to notice the synchronized real-time video in the first place.

Then, the video data marked with prompt information is converted into digital form, compressed and channel encoded, and then sent to the user device through the Long Range Radio (LoRa) protocol, thereby overcoming the problem of easy packet loss in the LoRa protocol and achieving low-cost, long-distance signal transmission.

In an optional implementation, information can also be transmitted through other transmission protocols, such as the Narrow Band Internet of Things (NB-IoT) protocol, the fifth generation mobile communication technology (5th Generation Mobile Communication Technology, referred to as 5G) protocol, and other transmission protocols, and this application does not impose any restrictions on this.

304: The user device displays prompt information and video data to the user.

In this embodiment, after receiving the prompt information and video data, the user device can determine the prompt level according to the prompt information, and then call the corresponding level of prompt sound to play through the alarm device of the user device to attract the attention of the backstage personnel. At the same time, the video data is played through the display device, so that the backstage personnel can observe the real-time situation of the pedestrian in the pedestrian passage through the display device.

305: The user device receives the warning information input by the user.

In this embodiment, the backstage personnel can input warning information through the input device of the user device. Specifically, the input device can be a microphone, a keyboard, a virtual keyboard, a touch screen, etc. Based on this, when the backstage personnel see the marked pedestrian or other pedestrians performing dangerous behaviors through the display device, they can shout through the microphone, or input text through the keyboard or virtual keyboard, or select the edited text or voice through the touch screen, and input it into the user device as warning information.

306: The user device sends a warning message to the cloud server.

In this embodiment, the method in which the user device sends the warning information to the cloud server is similar to the method in which the cloud server sends the prompt information and video data to the user device in step 303, and will not be repeated here.

307: The cloud server performs analog-to-digital conversion on the warning information to obtain first digital information.

In this embodiment, before the warning information is converted into analogue digital form, the warning information may be first extracted to obtain key information; the key information may then be reorganized and converted into voice to obtain a warning voice; the warning voice may then be converted into analogue digital form to obtain the first digital information. Thus, it may be determined whether there is any missing information in the warning information, for example, the appearance of the pedestrian is not described, resulting in an unclear target of the call. At this point, the cloud server may The system extracts the appearance features of pedestrians, such as age, gender, clothing, etc., and then reorganizes these features together with key information into a new warning voice for conversion, ensuring that the warning voice can be correctly conveyed to pedestrians with dangerous behavior characteristics.

308: The cloud server compresses the first digital information to obtain second digital information.

In this embodiment, the compression process may be compression coding.

309: The cloud server performs channel coding processing on the second digital information to obtain third digital information.

In this embodiment, the channel coding process may be forward error correction coding.

310: The cloud server sends third digital information to the audio playback device.

In this embodiment, the method in which the cloud server sends the third digital information to the audio playback device is similar to the method in which the cloud server sends the prompt information and video data to the user device in step 303, and will not be repeated here.

In this embodiment, the nearest audio playback device can be determined as a receiving device based on the location of the pedestrian to receive the third digital information and play the warning voice, or the third digital information can be sent to multiple audio playback devices near the pedestrian at the same time to ensure that the pedestrian can hear the warning voice.

311: The audio playback device plays a warning message.

In addition, in this embodiment, the direction of the pedestrian can be determined based on the video data of the pedestrian, and then the video data of the pedestrian can be sent to a display device in the direction of the pedestrian, so that the display device can display the video data to further warn the pedestrian and improve the warning effect.

In this embodiment, the warning information can be audio information, text information, or instruction information. It should be noted that when the warning information is audio information, the audio playback device can directly play the audio information when receiving the warning information; when the warning information is text information, the audio playback device first converts the text information into corresponding audio through voice-to-text technology, and then the audio playback device plays the audio; when the warning information is instruction information, the audio playback device first parses the instruction to obtain the voice template corresponding to the instruction (multiple voice templates are pre-recorded in the audio playback device), and then plays the voice template.

In summary, in the early warning method provided by the present invention, the video data of the pedestrian passage is obtained in real time by the camera equipment set at both ends of the pedestrian passage, and then the obtained video data is analyzed to determine the behavior characteristics of each pedestrian in the video data. Then, the action of each pedestrian is predicted according to the behavior characteristics of each pedestrian. When the probability of the pedestrian making dangerous behaviors such as running a red light is higher than the preset value, the behavior characteristics can be determined as dangerous behaviors. When the behavior characteristics of each pedestrian include dangerous behaviors, prompt information and video data are sent to the user device to prompt the backstage staff to pay attention to the pedestrian through the user device, and then when the pedestrian makes dangerous behaviors such as running a red light, a warning is issued to the pedestrian in time to prevent the pedestrian from continuing the current dangerous behavior. Compared with the traditional warning method, it has higher timeliness and can effectively prevent the occurrence of dangerous behaviors. At the same time, the warning information is converted by analog-to-digital conversion, compression and channel coding, and then the processed first warning voice is sent to the audio playback device for playback through the Long Range Radio (LoRa) protocol, which overcomes the problem of easy packet loss in the LoRa protocol and realizes low-cost and long-distance signal transmission.

Refer to FIG5 , which is a functional module block diagram of an early warning device provided in an embodiment of the present application. The early warning device can be applied to the cloud server 201 in the early warning system shown in FIG2 . Specifically, as shown in FIG5 , the early warning device 500 includes:

The first receiving module 501 is used to obtain video data collected by a camera device in real time;

An analysis module 502 is used to analyze the video data and determine the behavior characteristics of each pedestrian in the video data;

A first sending module 503, configured to send prompt information and video data to a user device when the behavior characteristics of each pedestrian include dangerous behavior;

The first receiving module 501 is further configured to receive warning information from a user device, wherein the warning information is a signal received by the user device according to Prompt information and video data determined;

The analysis module 502 is further used to perform analog-to-digital conversion processing on the warning information to obtain first digital information, perform compression processing on the first digital information to obtain second digital information, and perform channel coding processing on the second digital information to obtain third digital information;

The first sending module 503 is further used to send third digital information to the audio playback device, so as to play warning information through the audio playback device to warn pedestrians whose behavior characteristics are dangerous behaviors.

In the embodiment of the present invention, in analyzing the video data to determine the behavior characteristics of each pedestrian in the video data, the analysis module 502 is specifically used to:

Performing frame processing on the video data to obtain at least one video image;

Extracting the motion features of each pedestrian in at least one video image to obtain a motion feature sequence of each pedestrian;

According to the action feature sequence of each pedestrian, a comparison is made in the preset dangerous behavior feature library to determine the behavior characteristics of each pedestrian.

In an embodiment of the present invention, before sending the prompt information and the video data to the user equipment, the analysis module 502 is further used to:

Generate behavior prompt information according to the behavior characteristics of pedestrians with dangerous behavior characteristics;

In the video data, pedestrians whose behavior characteristics are dangerous behaviors are marked, wherein the marking includes prompt information.

In an embodiment of the present invention, before performing analog-to-digital conversion on the warning information to obtain the first digital information, the analysis module 502 is further used to:

Extract the warning information to obtain key information;

Reorganize and convert key information into voice to obtain warning voice;

Perform analog-to-digital conversion on the warning information to obtain first digital information, including:

Perform analog-to-digital conversion on the warning voice to obtain first digital information.

Refer to FIG6 , which is a functional module block diagram of an information sending device provided in an embodiment of the present application. The information sending device can be applied to the user equipment 204 in the early warning system shown in FIG2 . Specifically, as shown in FIG6 , the information sending device 600 includes:

The second receiving module 601 is used to receive prompt information and video data from the cloud server, wherein the video data includes a mark;

A display module 602 is used to display prompt information and video data to the user;

The second receiving module 601 is further used to receive warning information input by the user;

The second sending module 603 is used to send warning information to the cloud server.

Refer to FIG. 7 , which is a schematic diagram of the structure of an early warning device provided in an embodiment of the present application. As shown in FIG. 7 , the early warning device 700 includes a first transceiver 701, a first processor 702, and a first memory 703. They are connected via a first bus 704. The first memory 703 is used to store computer programs and data, and can transmit the data stored in the first memory 703 to the first processor 702. In a possible implementation, the early warning device can be the cloud server 201 in the early warning system shown in FIG. 2 .

The first processor 702 is used to read the computer program in the first memory 703 and perform the following operations:

Obtain video data collected by camera equipment in real time;

compressing the first digital information to obtain second digital information;

In the embodiment of the present invention, in analyzing the video data to determine the behavior characteristics of each pedestrian in the video data, the first processor 702 is specifically configured to perform the following operations:

In an embodiment of the present invention, before sending the prompt information and the video data to the user equipment, the first processor 702 is further configured to perform the following operations:

In an embodiment of the present invention, before performing analog-to-digital conversion on the warning information to obtain the first digital information, the first processor 702 is further configured to perform the following operations:

Extract the warning information to obtain key information;

Reorganize and convert key information into voice to obtain warning voice;

Refer to FIG8 , which is a schematic diagram of the structure of an information sending device provided in an embodiment of the present application. As shown in FIG8 , the information sending device 800 includes a second transceiver 801, a second processor 802, and a second memory 803. They are connected via a second bus 804. The second memory 803 is used to store computer programs and data, and can transmit the data stored in the second memory 803 to the second processor 802. In a possible implementation, the information sending device can be the user device 204 in the early warning system shown in FIG2 .

The second processor 802 is used to read the computer program in the second memory 803 and perform the following operations:

Display prompt information and video data to users;

Receive warning information input by the user;

Send warning information to the cloud server.

It should be understood that the warning device and information sending device in the present application may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices MID (Mobile Internet Devices, referred to as: MID), robots or wearable devices, etc. The above-mentioned warning devices and information sending devices are only examples, not exhaustive, and include but are not limited to the above-mentioned warning devices and information sending devices. In actual applications, the above-mentioned warning devices and information sending devices may also include: smart vehicle terminals, computer equipment, etc.

Through the description of the above implementation modes, those skilled in the art can clearly understand that the present invention can be implemented by combining software with a hardware platform. Based on such an understanding, all or part of the contribution of the technical solution of the present invention to the background technology can be embodied in the form of a software product, which can be stored in a storage medium such as ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation modes of the present invention or some parts of the implementation modes.

Therefore, the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement some or all of the steps of any early warning method and/or information sending method described in the above method implementation. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a tape, a disk, a USB flash drive, a flash memory, etc.

An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any one of the early warning methods and/or information sending methods recorded in the above method implementation.

It should be noted that, for the above-mentioned method implementations, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the implementations described in the specification are all optional implementations, and the actions and modules involved are not necessarily required by this application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device implementation described above is only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software program module.

If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

A person skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable memory, which can include a flash drive, a read-only memory (ROM), a random access memory (RAM), and a memory. Access Memory, referred to as RAM), disk or CD, etc.

The above is a detailed introduction to the implementation methods of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above implementation methods is only used to help understand the method and core idea of the present application. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

An early warning method, characterized in that the method is applied to a cloud server in an early warning system, the early warning system also includes a user device, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud service and the audio playback device is greater than a first threshold, and the method includes:

Acquire video data collected by the camera device in real time;

Analyzing the video data to determine the behavior characteristics of each pedestrian in the video data;

When the behavior characteristics of each pedestrian include dangerous behavior, sending prompt information and the video data to the user equipment;

Receiving warning information from the user equipment, wherein the warning information is determined by the user equipment according to the prompt information and the video data;

Performing analog-to-digital conversion on the warning information to obtain first digital information;

compressing the first digital information to obtain second digital information;

performing channel coding processing on the second digital information to obtain third digital information;

The third digital information is sent to the audio playback device so that the warning information is played through the audio playback device to warn pedestrians whose behavior characteristics are dangerous behaviors.
The method according to claim 1, characterized in that the step of analyzing the video data to determine the behavior characteristics of each pedestrian in the video data comprises:

Performing frame processing on the video data to obtain at least one video image;

Extracting the motion features of each of the pedestrians in at least one video image to obtain a motion feature sequence of each of the pedestrians;

According to the action feature sequence of each pedestrian, a comparison is performed in a preset dangerous behavior feature library to determine the behavior feature of each pedestrian.
The method according to claim 1, characterized in that before sending the prompt information and the video data to the user equipment, the method further comprises:

generating behavior prompt information according to the behavior characteristics of the pedestrian whose behavior characteristics are dangerous behaviors;

In the video data, pedestrians whose behavior characteristics are dangerous behaviors are marked, wherein the mark includes the prompt information.
The method according to any one of claims 1 to 3, characterized in that before performing analog-to-digital conversion on the warning information to obtain the first digital information, the method further comprises:

Extracting information from the warning information to obtain key information;

Reorganizing and converting the key information into voice to obtain a warning voice;

The step of performing analog-to-digital conversion on the warning information to obtain first digital information includes:

The warning voice is subjected to analog-to-digital conversion processing to obtain first digital information.
An information sending method, characterized in that the method is applied to a user device in an early warning system, the early warning system also includes a cloud server, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud server and the audio playback device is greater than a first threshold, and the method includes:

Receiving prompt information and video data from the cloud server, wherein the video data includes a mark;

Displaying the prompt information and the video data to the user;

Receive warning information input by the user;

The warning information is sent to the cloud server.
An early warning device, characterized in that the device is applied to a cloud server in an early warning system, the early warning system also includes a user device, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud service and the audio playback device is greater than a first threshold, and the device includes:

A first receiving module, used for acquiring the video data collected by the camera device in real time;

An analysis module, used to analyze the video data to determine the behavior characteristics of each pedestrian in the video data;

A first sending module, configured to send prompt information and the video data to the user equipment when the behavior characteristics of each pedestrian include dangerous behavior;

The first receiving module is further configured to receive warning information from the user equipment, wherein the warning information is determined by the user equipment according to the prompt information and the video data;

The analysis module is further configured to perform analog-to-digital conversion processing on the warning information to obtain first digital information, perform compression processing on the first digital information to obtain second digital information, and perform channel coding processing on the second digital information to obtain third digital information;

The first sending module is further used to send the third digital information to the audio playback device, so that the warning information is played through the audio playback device to warn pedestrians whose behavior characteristics are dangerous behaviors.
An information sending device, characterized in that the device is applied to a user device in an early warning system, the early warning system also includes a cloud server, and a camera device and an audio playback device arranged at both ends of a pedestrian passage, and the distance between the cloud server and the audio playback device is greater than a first threshold, and the device includes:

A second receiving module, configured to receive prompt information and video data from the cloud server, wherein the video data includes a mark;

A display module, used to display the prompt information and the video data to the user;

The second receiving module is further used to receive warning information input by the user;

The second sending module is used to send the warning information to the cloud server.
An early warning device, characterized in that it comprises a first processor, a first memory, a first communication interface and one or more programs, wherein the one or more programs are stored in the first memory and are configured to be executed by the first processor, and the one or more programs include instructions for executing the steps described in any one of the methods of claims 1-4.
An information sending device, characterized in that it includes a second processor, a second memory, a second communication interface and one or more programs, wherein the one or more programs are stored in the second memory and are configured to be executed by the second processor, and the one or more programs include instructions for executing the steps described in the method described in claim 5.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 5.
A computer program product, characterized in that the computer program product comprises a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method according to any one of claims 1 to 5.