CN111263097B

CN111263097B - Media data transmission method and related equipment

Info

Publication number: CN111263097B
Application number: CN202010051951.7A
Authority: CN
Inventors: 刘俊; 杨胜凯
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-09-19
Filing date: 2020-01-16
Publication date: 2024-01-02
Anticipated expiration: 2040-01-16
Also published as: WO2021051912A1; CN111263097A

Abstract

The application discloses a media data transmission method and related equipment, wherein the media data transmission method comprises the following steps: the camera generates a plurality of original video frames; the camera generates video using a plurality of original video frames; the method comprises the steps that a camera obtains position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video; the video and the position information are sent to the storage device by the video camera, so that the position information of the video and the target image can be transmitted, and the consumption of transmission bandwidth is effectively reduced.

Description

Media data transmission method and related equipment

Technical Field

The present disclosure relates to the field of data processing, and in particular, to a media data transmission method and related devices.

Background

The video camera captures a large amount of video data and then transmits the video to the storage device for storage.

With the push of the intelligence of the video camera, the video camera needs to transmit pictures to the storage device in addition to the video, so that the storage device can perform operations such as image recognition. Additional picture transmission increases the network bandwidth usage and the space usage of the storage device.

Disclosure of Invention

The embodiment of the application provides a media data transmission method and related equipment, which can transmit the position information of video and images, and effectively reduce the consumption of transmission bandwidth.

In a first aspect, an embodiment of the present application provides a method for transmitting media data, where the method includes:

the camera generates a plurality of original video frames;

the camera generates video using a plurality of original video frames;

the method comprises the steps that a camera obtains position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

the camera sends the video and the location information to the storage device.

In this example, by transmitting the position information of the video and the target image, since the size of the position information is far smaller than the size of the image, the transmission bandwidth occupied by the position information is also far smaller than the transmission bandwidth occupied by the image, and the consumption of bandwidth resources is reduced.

With reference to the first aspect, in one possible embodiment of the first aspect,

in the case where the category of the target image is a large map, the position information includes a first absolute position including one or more of a frame number and a time stamp of the target image in the video and/or a first relative position including an offset of the target image with respect to a particular video frame.

in the case that the category of the target image is a small image, the position information comprises a second absolute position and/or a second relative position, wherein the second absolute position comprises one or more of a frame number and a time stamp of a video frame corresponding to the target image in the video, and the second absolute position also comprises the position of the target image in the corresponding video frame; the second relative position includes an offset of the target image relative to the particular video frame, a position of the target image in the corresponding video frame.

In this example, the location information includes an absolute location and a relative location, and the absolute location and the relative location may be transmitted simultaneously to verify each other, so as to avoid an error caused by a loss of the video frame during transmission.

With reference to the first aspect, in one possible embodiment of the first aspect, the camera does not generate the target image, and the target image is not sent to the storage device.

In this example, the camera does not generate the target image and does not send the target image to the storage device, so that the target image does not need to be transmitted separately, and consumption of bandwidth resources is reduced.

With reference to the first aspect, in a possible embodiment of the first aspect, the method further includes:

the camera selects a target frame where a target image is located from the video;

and the camera acquires the position information of the target image in the video according to the target frame and the video.

the target picture is located in an I frame in a group of pictures GOP of the video.

the camera acquires a target image from a plurality of original video frames, wherein the image quality of the target image is an image including target features in the plurality of original video frames.

With reference to the first aspect, in a possible embodiment of the first aspect, the method further includes: the storage device receives the video sent by the camera and the position information of the target image;

the storage device acquires the target image from the corresponding video frame of the video according to the position information.

the storage device stores the position information of the video and the target image sent by the camera;

when the storage life cycle of the video is finished, the storage device acquires a target image from a corresponding video frame of the video according to the position information;

The storage device stores the target image;

the storage device deletes the video.

In this example, at the end of the storage life cycle of the video, the target image is acquired from the corresponding video frame of the video according to the location information, and is stored, so that the target image can be stored after the subsequent video is deleted, and the target image can be provided for the subsequent image searching.

the storage device receives a video stream sent by a camera and position information of a target image, wherein the target image is a video frame in the video stream or a part of the video frame in the video stream;

the storage device acquires the target image from the corresponding video frame of the video stream according to the position information.

In a second aspect, an embodiment of the present application provides a method for transmitting media data, where the method includes:

the storage device receives the video sent by the camera and the position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

With reference to the second aspect, in one possible embodiment of the second aspect,

With reference to the second aspect, in one possible embodiment of the second aspect, in a storage life cycle of the video, when the storage device receives a request for reading the target image, the storage device acquires the target image from a corresponding video frame in the video according to the location information.

With reference to the second aspect, in a possible embodiment of the second aspect, the method further includes: when the storage life cycle of the video is finished, the storage device acquires a target image from a corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

In this example, in the storage life cycle of the video, the target image is acquired from the corresponding video frame in the video, so that in the storage life cycle of the video, the storage device only needs to store the video and the position information, and the memory space occupied by the position information is far smaller than the memory space occupied by the target image, so that the consumption of storage resources can be reduced when the position information is stored relative to the storage of the target image.

With reference to the second aspect, in a possible embodiment of the second aspect, the storage device does not store the target image during a storage lifecycle of the video; after the storage life cycle of the video is finished, the storage device stores the target image.

In this example, in the storage life cycle of the video, the target image does not need to be stored, so that the consumption of storage resources can be reduced.

In a third aspect, an embodiment of the present application provides a media data transmission apparatus, including:

A first generation unit configured to generate a plurality of original video frames;

a second generation unit for generating a video using a plurality of original video frames;

an acquisition unit configured to acquire position information of a target image in a video, where the target image is a video frame in the video or a portion of the video frame in the video;

and the transmitting unit is used for transmitting the video and the position information to the storage device.

With reference to the third aspect, in one possible embodiment of the third aspect,

With reference to the third aspect, in a possible embodiment of the third aspect, the media data transmission device does not generate the target image, and does not send the target image to the storage device.

With reference to the third aspect, in a possible embodiment of the third aspect, the method further includes:

With reference to the third aspect, in a possible embodiment of the third aspect, the method is further used for:

a target image is acquired from a plurality of original video frames, and the image quality of the target image is an image including target features in the plurality of original video frames.

In a fourth aspect, an embodiment of the present application provides a media data transmission apparatus, including:

a receiving unit, configured to receive a video transmitted by a camera and position information of a target image, where the target image is a video frame in the video or a portion of the video frame in the video;

and the acquisition unit is used for acquiring the target image from the corresponding video frame of the video according to the position information.

With reference to the fourth aspect, in a possible embodiment of the fourth aspect,

With reference to the fourth aspect, in one possible embodiment of the fourth aspect, during a storage life cycle of the video, when the storage device receives a request for reading the target image, the storage device acquires the target image from a corresponding video frame in the video according to the location information.

With reference to the fourth aspect, in a possible embodiment of the fourth aspect, the method is further used for: when the storage life cycle of the video is finished, the storage device acquires a target image from a corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

With reference to the fourth aspect, in a possible embodiment of the fourth aspect, the method is further used for:

receiving a video stream sent by a camera and position information of a target image, wherein the target image is a video frame in the video stream or a part of the video frame in the video stream;

and acquiring a target image from the corresponding video frame of the video stream according to the position information.

In a fifth aspect, embodiments of the present application provide a camera, including:

a processor for generating a plurality of original video frames, the video being generated using the plurality of original video frames; acquiring position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the receiving and transmitting module is used for transmitting the video and the position information to the storage device.

With reference to the fifth aspect, in one possible embodiment of the fifth aspect,

With reference to the fifth aspect, in a possible embodiment of the fifth aspect, the camera does not generate the target image, and the transceiver module does not send the target image to the storage device.

With reference to the fifth aspect, in a possible embodiment of the fifth aspect, the method is further used for:

selecting a target frame in which a target image is positioned from the video;

and acquiring the position information of the target image in the video according to the target frame and the video.

In a sixth aspect, embodiments of the present application provide a storage device, including:

the receiving and transmitting module is used for receiving the video sent by the camera and the position information of the target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the processor is used for acquiring the target image from the corresponding video frame of the video according to the position information.

With reference to the sixth aspect, in one possible embodiment of the sixth aspect,

With reference to the sixth aspect, in one possible embodiment of the sixth aspect, during a storage life cycle of the video, when the storage device receives a request for reading the target image, the storage device acquires the target image from a corresponding video frame in the video according to the location information.

With reference to the sixth aspect, in a possible embodiment of the sixth aspect, the method is further used for: when the storage life cycle of the video is finished, acquiring a target image from a corresponding video frame of the video according to the position information; save the target image, and delete the video.

With reference to the sixth aspect, in a possible embodiment of the sixth aspect, the method is further used for:

In a seventh aspect, embodiments of the present application provide a camera, including: a processor, a transceiver and a memory, the processor executing code in the memory to perform the method as in the first aspect.

In an eighth aspect, embodiments of the present application provide a storage device, including: a processor, a transceiver and a memory, the processor executing code in the memory to perform the method as in the second aspect.

In a ninth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any one of the first and second aspects.

In a tenth aspect, there is provided a computer program product which, when read and executed by a computer, performs the method of any of the first and second aspects.

These and other aspects of the present application will be more readily apparent from the following description of the embodiments.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required for the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application.

Fig. 1 is a schematic diagram of an image captured by a camera according to an embodiment of the present application;

FIG. 2 provides a schematic diagram of a large and small diagram according to an embodiment of the present application;

fig. 3A is a schematic diagram of a video frame transmission according to an embodiment of the present application;

FIG. 3B is a schematic diagram of a camera transmitting video and position information according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a storage device storing video and location information according to an embodiment of the present application;

FIG. 5A is a schematic diagram of an index table of a large graph according to an embodiment of the present application;

FIG. 5B is a schematic diagram of an index table of a small graph according to an embodiment of the present application;

fig. 6A is a schematic diagram of a camera extracting a target image according to an embodiment of the present application;

FIG. 6B is a schematic diagram of an index table for multiplexing and storing small images according to an embodiment of the present application;

FIG. 6C is a schematic diagram of an index table for multiplexing storage of a small image according to an embodiment of the present application;

FIG. 6D is a schematic diagram of an index table providing another large map according to an embodiment of the present application;

fig. 7 is an interaction schematic diagram of a media data transmission method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a media data transmission device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a camera according to an embodiment of the present application;

FIG. 10 is a schematic view of another camera according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a media data transmission device according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a memory device according to an embodiment of the present application;

Fig. 13 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the accompanying drawings.

The video transmission process to which the present application relates will first be described.

As shown in fig. 1, the video camera acquires a video according to a time axis t, wherein the video comprises n frames of images I ₁ ，I ₂ ，…，I _n Image I ₁ Is the camera at t ₁ Image acquired at moment, image I ₂ Is the camera at t ₂ Image, …, image I, acquired at the moment _n Is the camera at t _n Images acquired at the moment. Here, t ₁ ，t ₂ ，…，t _n The time intervals therebetween may be equal or unequal, that is, t _n -t _n-1 ，t _n-1 -t _n-2 ，…，t ₂ -t ₁ May be equal or unequal, and is not particularly limited herein.

The camera needs to select a target image from the video to send to the storage device in addition to sending the video to the storage device. The target image may be a large image or a small image, and the storage device may be a storage server, a video stream management platform, or the like. As shown in fig. 2, the large map may be a complete image of a certain video frame, or an image occupying an area of a certain video frame exceeding a preset threshold, or the like. In a particular embodiment, the large map may contain multiple target subjects (subjects may be understood as target features), for example, the large map may include a scene where a vehicle bumps into a pedestrian, and thus the large map may be used to analyze relationships and behaviors between different target subjects. The small picture may be a partial region of a certain video frame. In a particular embodiment, the thumbnail may include only a single target subject, or only a partial region of a single target subject, e.g., the thumbnail may include a face portion of a pedestrian, and thus the thumbnail may be used to analyze the details and structure of a single target subject. Here, the target subject may be a pedestrian, an animal, a vehicle, a license plate, a road sign, a traffic light, or the like, which is not particularly limited herein. The small image may be obtained by extracting a region image block having the target subject from the large image, and the extraction method may be an image feature extraction algorithm, specifically, HOG (histogram of Oriented Gradient, directional gradient histogram), SIFT (Scale-invariant features transform, scale invariant feature transform), and the like, which are not limited herein.

In order to reduce the bandwidth required by the video camera to send the video and the target image to the storage device, the video camera may further compress the video by using a video compression algorithm and compress the target image by using a picture compression algorithm, where the video compression algorithm may be h.264, h.265, h.266, and the like, and is not specifically limited herein. The picture compression algorithm may be JPEG, HEIF, or the like, and is not particularly limited herein.

When the bandwidth occupied by the video camera for sending data to the storage device is reduced, the video camera compresses the video by adopting a video compression algorithm and compresses the target image by adopting a picture compression algorithm, and the bandwidth occupied after compression is reduced. However, bandwidth still consumes transmission resources during transmission, especially when a large amount of target images are transmitted, the consumption of bandwidth resources still can be huge, and after the target images are transmitted to the storage device, the storage device needs to consume memory resources to store the target images, so that the consumption of memory resources is large. Therefore, how to reduce the transmission of the target image is a problem to be solved.

The embodiment of the application aims to solve the problems that the large bandwidth consumption is caused when the target image is transmitted and the memory resource and the hard disk resource are greatly consumed when the storage device stores the target image, the video (the video can be transmitted in a streaming mode and is called as video streaming) and the position information of the target image are transmitted, and the bandwidth occupied by the position information is far smaller than the bandwidth occupied by the image because the size of the position information is far smaller than the size of the image, so that the consumption of bandwidth resource is reduced, the storage device stores the video and the position information after receiving the position information of the video and the target image, and the consumption of the storage resource can be reduced relative to the storage of the target image.

The application provides a media data transmission method and related equipment, which can effectively reduce consumption of bandwidth resources and storage resources.

Transmitting video including image I to a storage device at a camera ₁ ，I ₂ ，…，I _n When the camera needs to send the video, the camera also needs to send the position information of the target image, so as to mark the position of the target image in the video. Here, the video and the location information may be transmitted simultaneously or may be transmitted separately; the same channel may be used for transmission, or different channels may be used (e.g., both use a data channel, or a video stream uses a data channel, and location information uses a management channel), which is not particularly limited herein. The positional information when the target image is a large image and a small image will be described in detail below.

When the category of the target image is a large graph, the position information of the target image may be the first absolute position or the first relative position, or may include both the first absolute position and the first relative position. Wherein the first absolute position may include one or more of a frame number, a time stamp, etc. of the target image in the video. For example, a video includes n frames of image I ₁ ，I ₂ ，…，I _n The target image may be a 5 th frame image and then the first absolute position may be a frame number 5 of the target image in the video. First phaseThe pair position may be an offset relative to a particular video frame, etc. For example, a video includes n frames of image I ₁ ，I ₂ ，…，I _n The target image may be a 5 th frame image. 5-1=4, then the first relative position may be an offset 4 of the target image with respect to the first frame image, or may be an offset of the target image with respect to the I frame when the specific video frame is, for example, an I frame after the video frame is compressed. Of course, when the video frame corresponding to the target image is an I frame, the position information may be described by using a first relative position, specifically, the position information may be a frame number of the I frame, etc., and when the target image frame is a non-I frame, the position information may be described by using a first relative position, specifically, an offset of the target image with respect to the I frame, etc. Supplementary explanation: when the large image is not a complete frame image, the position information of the target image may also include the position of the target image in the video frame, the position including coordinates and size (see more particularly below).

When the category of the target image is a small image, the position information of the target image may be the second absolute position or the second relative position, or include both the second absolute position and the second relative position. The second absolute position may include one or more of a frame number, a time stamp, and the like of a video frame corresponding to the target image in the video, and a position of the target image in the video frame, where the position includes coordinates and a size, the coordinates of the target image in the video frame may be expressed as (x, y), x is a transverse coordinate, y is a longitudinal coordinate, the size of the target image in the video frame may be expressed as mXn, m is a transverse size, and n is a longitudinal size. The second relative position includes an absolute position of a video frame corresponding to the target image and a relative position of the target image in the corresponding video frame, a relative position of a video frame corresponding to the target image and an absolute position of the target image in the corresponding video frame, a relative position of a video frame corresponding to the target image, and a relative position of the target image in the corresponding video frame. The relative position of the target image in the corresponding video frame may be an offset relative to the position of a specific mark, for example, the video frame is an image of the target pedestrian visiting the Tiananmen, and the target image is the target pedestrian, then the relative position of the target image in the video frame may be the direction and distance of the target pedestrian relative to the Tiananmen, for example, the position of the target pedestrian on the eastern side of the Tiananmen, 100 meters away, and is denoted as (east, 100).

The location information of the target image may further include a frame type of the video frame in which the target image is located when the video frame is transmitted after being compressed, where the frame type includes an I frame and a P frame. Transmitting video at a camera to a storage device, e.g. comprising image I ₁ ，I ₂ ，…，I _n When video frames need to be encoded and transmitted, as shown in fig. 3A, fig. 3A shows a schematic diagram of the transmission of video frames. After the video frame is encoded, an I frame and a P frame are obtained, where the I frame may be a complete video frame, the I frame may be understood as a key frame, and the P frame is a difference between the I frame and a previous key frame, for example, the I frame is an image of a car at a certain moment when the car is running, and the P frame may be a position offset of the car at a next moment when the car is corresponding to the previous moment, and so on. The target image may be an I frame or a P frame.

When the video and the position information are sent to the storage device by the video camera, the associated information of the target image can be sent, wherein the associated information comprises the acquisition time of the video, the acquisition time of the video frame corresponding to the target image, the video identification, the category of the target image, the serial number of the target image, the frame number of the video frame corresponding to the target image and the offset of the video frame corresponding to the target image. The categories of the target image include a large image and a small image, and the time of capturing the video may be the start time of the video, etc.

It can be appreciated that in practical applications, the absolute position and the relative position may be transmitted simultaneously to perform verification between each other, so as to avoid errors caused by loss of the video frame during transmission.

In a specific embodiment, as shown in FIG. 3B, the location information transmitted between the camera and the storage device is compressed. Specifically, the camera may compress the original position information and transmit the compressed position information to the storage device. The original position information may be part or all of the position information. When the original position information is part of the position information, the original position information includes at least the position information of the target image.

As shown in fig. 4, fig. 4 shows a specific example when the storage device stores the received video and the location information, and after receiving the video and the location information, the storage device stores the video in a storage space corresponding to the video and stores the location information. For convenient storage, the position information can be further processed to obtain an index table of the target image, and the index table is stored. The location information (or index table) may represent the location information of the target picture in the video, where the index table includes a large-scale index table and a small-scale index table, and the large-scale index table and the small-scale index table may be stored separately, that is, all the large-scale index tables are stored in one memory space, and all the small-scale index tables are stored in another memory space.

When the position information is processed, the index table for obtaining the target image may specifically be: and extracting the position information of the target image, generating a template according to a preset index table, and generating an index table of the target image. When the position information of the target image is extracted, the position information may be extracted from a cache or from a memory. The preset index table generation template may be a preset template. Of course, an index table of the target image may be obtained by other methods.

In a specific embodiment, when the index table is generated according to the index table template, fig. 5A shows a schematic diagram of an index table of a large graph, and fig. 5B shows a schematic diagram of an index table of a small graph. As shown in fig. 5A, the index table of the large image includes a camera identifier, a capturing time of a video frame corresponding to the large image, a frame number of the video frame, a picture type, a picture sequence number, a video frame type, a video frame offset, and the like, and the content of the index table can be directly extracted from the received position information. As shown in fig. 5B, the index table of the small image includes a camera identifier, an acquisition time of a video frame where the small image is located, a frame number of the video frame, a picture type, a picture sequence number, a video frame type, a video frame offset, an offset of the small image in the video frame, a size of the small image in the video frame, and the like. The offset of the small image in the video frame is represented by a form of coordinates, the size of the small image in the video frame is represented by a horizontal size and a vertical size, for example, 80X80, the horizontal size is 80, the vertical size is 80, and the content of the index table can be directly extracted from the received position information.

The storage device may not be used to store the target image alone. After receiving a request for reading the target image from the host, the storage device acquires the corresponding target image from the video by using the index table and sends the target image to the host.

After the storage device stores the video, the storage life cycle is set for the video, and after the storage life cycle is finished, the video is deleted, and the storage life cycle can be specifically understood as a fixed duration. In order to avoid that the target image cannot be read out any more after the video is deleted, the storage device may extract the target image from the video at the end of the storage life cycle of the video, store the target image in the corresponding storage space, and update the picture index table (updated index table for describing the storage position of the target image in the storage device). The moment when the storage life cycle ends is a condition that triggers a step of extracting the target image from the video, and after the extraction of the target image is completed, the storage device deletes the video. At a specific point in time, the end of the lifecycle includes: the lifecycle is about to reach the end time point, or a short time after the end time point of the lifecycle. Of course, the storage device may also extract the target image from the video before the storage life cycle of the video is finished, for example: the completion of the extraction operation is performed and stored within 10 minutes before the end of the storage life cycle, and the storage device can delete the video immediately after the end of the storage life cycle.

When extracting a target image from a video according to an index table of the target image, the specific method may be: according to the video identification, at least one video corresponding to the video identification is obtained, then according to the acquisition time of the video, a target video is determined from the at least one video, the target video comprises a target image, the video frame is extracted from the target video according to the time of the video frame in the index table, the type of the video frame, the frame number of the video frame in the video and the offset of the video frame, the target image is obtained from the video frame according to the position information of the target image, if the target image is a large image, the video frame can be determined to be the target image (the large image is taken as a complete video frame for illustration here), and if the target image is a small image, the target image is obtained according to the offset of the picture in the video frame in the index table and the size of the picture in the video frame. In a specific example, as shown in fig. 6A, fig. 6A shows a schematic diagram of extracting a target image. The storage device provides n videos from a storage space corresponding to the video according to the video identification, wherein the n videos comprise video 1, video 2, …, video n-1 and video n, m videos comprise video k, … and video j according to the video identification, a target video comprises the m videos according to the acquisition time of the video, a target image is determined from n video frames of the target video according to other information in an index table, and the other information comprises the time of the video frames, the video frame type, the frame number of the video frames, the video frame offset, the position information (first absolute position and/or first relative position, second absolute position and/or second relative position) of the target image and the like.

When the target image is extracted from the video, firstly judging whether the target image exists in the video, and if the target image exists, extracting the target image from the video according to an index table of the target image, and storing the target image into a picture storage space. Whether the target image exists in the video or not can be judged according to the index table, specifically, whether the video frame exists in the video or not can be judged through the frame number of the video frame in the index table and the acquisition time of the video frame, if the video frame exists in the video, the target image exists in the video is determined, and if the video frame does not exist in the video, the target image does not exist in the video.

When storing the target image, the large image and the small image may be stored separately or may be stored in a multiplexed manner. The multiplexing storage can be understood as storing a large image or video frame where the target image is and storing the position information of the small image in the large image or video frame when the target image is the small image, thereby realizing the effect that the small image and the large image/video frame are both stored. When the target image is stored, different encoding formats can be adopted to encode the target image and then the target image is stored, and the encoding format can be HEIF format.

After deleting the video, since the position of the specific index in the index table of the current target image is the position in the video, the current index table cannot meet the condition of representing the position of the target image, so that the index table needs to be updated, and the updated index table is used for describing the storage position of the target image, the picture type of the target image, the coding format of the target image during storage, and the like.

Different storage modes are adopted, the updated picture index table is also different, the storage position of the target image and the file name of the target image are newly added in the updated picture index table, and the original position information related to the video, such as the type of the video frame, the offset of the video frame and the like, is deleted. As shown in fig. 6B, fig. 6B shows a schematic diagram of an index table of a small picture when the small picture is multiplexed and stored, and at this time, a video frame corresponding to the small picture is multiplexed when the small picture is stored. Fig. 6C shows that, when the small images are stored in a multiplexing manner, a large image corresponding to the small images is multiplexed, if the large image is a complete video frame, the offset of the small image in the large image does not need to be re-acquired, and if the large image is a partial video frame, the offset of the small image in the large image needs to be re-acquired, and the specific acquisition manner refers to the aforementioned acquisition manner of acquiring the position information of the small image in the video frame, which is not described in detail herein. Fig. 6D shows a schematic diagram of the index table of the updated large graph. Wherein the small image storage type comprises 0 and 1,1 represents small image multiplexing storage, and 0 represents small image independent storage.

After the storage device stores the video, if a request for reading the target image is received, the target image may be extracted from the video or from a storage space of the target image, and the target image may be fed back to the requester. The method specifically comprises the following steps: the method comprises the steps that a target image reading request is in a storage life cycle of a video, and the target image is extracted from the video according to an index table; after the storage life cycle of the video is finished, extracting the target image from the storage space of the target image according to the index table, feeding back the target image to the requester after extracting the target image, and converting the picture format of the target image into a picture format corresponding to the requester when feeding back the target image to the requester, for example, converting the format of the target image into a JPEG format when the requester requests the JPEG format. The method for extracting the target image from the video may refer to the image extraction method shown in fig. 6A in the foregoing embodiment, and will not be described herein.

The storage device may also receive the video stream and the position information of the target image sent by the camera, and the storage device obtains the target image from the corresponding video frame in the video stream according to the position information, and specific embodiments of the storage device may refer to embodiments of the storage device receiving the video and the position information of the target image sent by the camera, and obtaining the target image from the video, which are not described herein again.

Fig. 7 is an interaction schematic diagram of a media data transmission method according to an embodiment of the present application, as shown in fig. 7. The data transmission method of the present embodiment includes the steps of:

s101, a camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video.

The target image comprises a large image and/or a small image, wherein the large image can be a complete image of a certain video frame, or an image occupying an area of the certain video frame exceeding a preset threshold value, and the like; the small picture may be a partial region of a certain video frame. In a particular embodiment, the thumbnail may include only a single target subject, or only a partial region of a single target subject.

The position information includes an absolute position, which may be, for example, a frame number, a time stamp, etc. of the video frame, and a relative position, which may be, for example, an offset relative to the particular video frame.

Before performing the acquisition of the position information of the target image in the video, the camera generates a plurality of original video frames, which the camera uses to generate the video.

S102, the video camera sends the video and the position information to the storage device.

When the video camera transmits the video and the position information to the storage device, the video and the position information may be transmitted simultaneously or non-simultaneously.

S103, the storage device receives the video sent by the camera and the position information of the target image, wherein the target image is a video frame in the video or a part of the video frame in the video.

S104, the storage device acquires the target image from the corresponding video frame of the video according to the position information.

When the storage device acquires the target image according to the position information, the storage device may acquire the target image according to an index table carrying the position information, specifically, may acquire the target image from a video frame corresponding to the video according to the index table, or may acquire the target image from a storage space of the target image according to the index table.

In one possible implementation of the present invention,

in the case that the category of the target image is a small image, the position information comprises a second absolute position and/or a second relative position, wherein the second absolute position comprises one or more of a frame number and a time stamp of a video frame corresponding to the target image in the video, the second absolute position further comprises a position of the target image in the corresponding video frame, and the position comprises coordinates and a size; the second relative position includes an offset of the target image relative to the particular video frame, coordinates and a size of the target image in the corresponding video frame.

In one possible embodiment, the camera does not generate the target image and does not send the target image to the storage device.

In one possible embodiment, the method further comprises:

The target frame may be understood as a video frame in which the target image is located in the foregoing embodiment.

In one possible embodiment of the present invention,

The I-frames in a group of pictures GOP of video are key frames.

In one possible embodiment, the camera acquires a target image from a plurality of original video frames, the target image having an image quality that is an image of a target feature included in the plurality of original video frames.

Target features may be understood as specific features, such as behavior between multiple subjects, etc.

In one possible embodiment, the method further comprises: the storage device receives the video sent by the camera and the position information of the target image;

In one possible embodiment, the method further comprises:

the storage device stores the target image;

the storage device deletes the video.

In one possible embodiment, the method further comprises:

For simplicity of presentation, the present embodiment is not described in terms of definition of a large map, a small map, location information, an index table, etc., and specific reference is made to fig. 2, 3A, 3B, 5A, 5B, etc., and definition of a large map, a small map, location information, an index table, a specific video frame, etc. The video capturing by the camera, the video transmission, etc. are not described in this embodiment, and refer to fig. 1, fig. 3A, fig. 3B, and the related descriptions. For additional definitions, see the description of the embodiments above.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a media data transmission device provided in the present application. The media data transmission device 800 of the embodiment of the present application includes:

a first generation unit 810 for generating a plurality of original video frames;

a second generation unit 820 for generating a video using a plurality of original video frames;

an acquiring unit 830, configured to acquire position information of a target image in a video, where the target image is a video frame in the video or a portion of the video frame in the video;

and a transmitting unit 840 for transmitting the video and the position information to the storage device.

In one possible embodiment of the present invention,

in the case that the category of the target image is a large graph, the position information comprises a first absolute position and/or a first relative position, wherein the first absolute position comprises one or more of a frame number and a timestamp of the target image in the video, and the first relative position comprises an offset of the target image relative to a specific video frame;

in one possible embodiment of the present invention,

In one possible embodiment, the media data transmission means does not generate the target image and does not send the target image to the storage device.

In one possible embodiment, it is further used to:

selecting a target frame in which a target image is positioned from the video;

In one possible embodiment of the present invention,

In one possible embodiment, it is further used to: a target image is acquired from a plurality of original video frames, and the image quality of the target image is an image including target features in the plurality of original video frames.

Referring to fig. 9, fig. 9 is a schematic structural view of a video camera provided in the present application. The camera 900 of the present embodiment includes a processor 910 and a transceiver module 920, where,

a processor 910 for generating a plurality of original video frames, generating a video using the plurality of original video frames; acquiring position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

the transceiver module 920 is configured to send the video and the location information to the storage device.

In one possible embodiment of the present invention,

In one possible embodiment, the processor 910 does not generate the target image and the transceiver module 920 does not send the target image to the storage device.

In one possible embodiment, it is further used to:

selecting a target frame in which a target image is positioned from the video;

In one possible embodiment of the present invention,

As shown in fig. 10, the embodiment of the present application further provides a camera 1000, where the camera 1000 includes a processor 1010, a memory 1020 and a transceiver 1030, where the memory 1020 stores instructions or programs, and the processor 1010 is configured to execute the instructions or programs stored in the memory 1020. When executed, the processor 1010 is configured to perform the operations performed by the processor 920 in the above embodiments, and the transceiver 1030 is configured to perform the operations performed by the transceiver module 902 in the above embodiments.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a media data transmission device provided in the present application. The media data transmission device 1100 provided in the embodiment of the present application includes:

a receiving unit 1110, configured to receive a video transmitted by a camera and location information of a target image, where the target image is a video frame in the video or a portion of a video frame in the video;

An acquiring unit 1120, configured to acquire a target image from a corresponding video frame of the video according to the location information.

In one possible embodiment of the present invention,

in one possible embodiment of the present invention,

In one possible embodiment, the target image is acquired from a corresponding video frame in the video according to the location information when a request for reading the target image is received during the storage life cycle of the video.

In one possible embodiment, it is further used to: when the storage life cycle of the video is finished, acquiring a target image from a corresponding video frame of the video according to the position information; save the target image, and delete the video.

In one possible embodiment, it is further used to:

Referring to fig. 12, fig. 12 is a schematic structural view of a memory device provided in the present application. The storage device 1200 provided in the embodiment of the present application includes a transceiver module 1210 and a processor 1220:

a transceiver module 1210, configured to receive a video transmitted by a camera and location information of a target image, where the target image is a video frame in the video or a portion of a video frame in the video;

the processor 1220 is configured to obtain a target image from a corresponding video frame of the video according to the location information.

In one possible embodiment of the present invention,

In one possible embodiment, during a storage life cycle of the video, when the storage device 1200 receives a request to read the target image, the storage device 1200 acquires the target image from a corresponding video frame in the video according to the location information.

In one possible embodiment, it is further used to:

As shown in fig. 13, the embodiment of the present application further provides a server 1300, where the server 1300 includes a processor 1310, a memory 1320 and a transceiver 1330, where the memory 1320 stores instructions or programs, and the processor 1310 is configured to execute the instructions or programs stored in the memory 1320. The processor 1310 is configured to perform the operations performed by the processor 1220 in the above embodiment, and the transceiver 1330 is configured to perform the operations performed by the transceiver module 1210 in the above embodiment when the instructions or programs stored in the memory 1320 are executed.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium can store a program, and the program can include part or all of the steps of any one of the media data transmission methods described in the above method embodiments when executed.

The present application also provides a program product, wherein when the computer program product is read and executed by a computer, part or all of the steps of any one of the media data transmission methods described in the above method embodiments will be executed.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, storage Disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

The foregoing has outlined rather broadly the more detailed description of embodiments of the present application, wherein specific examples are provided herein to illustrate the principles and embodiments of the present application, the above examples being provided solely to assist in the understanding of the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in light of the ideas of the present application, the present disclosure should not be construed as being limited to the above description.

Claims

1. A method of media data transmission, the method comprising:

the camera generates a plurality of original video frames;

the camera generating a video using the plurality of original video frames;

the camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

the video camera sends the video and the position information to a storage device;

in case the target image is an I-frame image, the position information includes an absolute position;

in the case where the target image is a P-frame image, the position information includes a relative position;

In the case where the target image is a video frame in the video, the relative position includes a first relative position including an offset of the target image relative to an I-frame image;

in the case where the target image is part of a video frame in the video, the relative positions include a second relative position; the second relative position includes a relative position of the target image in a corresponding video frame;

the relative position of the target image in the corresponding video frame includes an offset of the position of the target image relative to a particular identifier.

2. The method of claim 1, wherein the camera does not generate the target image and does not send the target image to the storage device.

3. The method according to claim 1 or 2, further comprising:

the camera selects a target frame where the target image is located from the video;

4. The method according to claim 1 or 2, further comprising:

The target image is an image of the plurality of original video frames that includes a target feature.

5. The method according to claim 1 or 2, further comprising: the storage device receives the video sent by the camera and the position information of the target image;

and the storage equipment acquires the target image from the corresponding video frame of the video according to the position information.

6. The method according to claim 1 or 2, further comprising:

the storage device stores the video sent by the camera and the position information of the target image;

when the storage life cycle of the video is finished, the storage device acquires the target image from the corresponding video frame of the video according to the position information;

the storage device stores the target image;

the storage device deletes the video.

7. A method of media data transmission, the method comprising:

the storage device receives video sent by a camera and position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

the storage device acquires the target image from the corresponding video frame of the video according to the position information;

in the case where the class of the target image is part of a video frame in the video, the relative position includes a second relative position including a relative position of the target image in a corresponding video frame; the relative position of the target image in the corresponding video frame includes an offset of the position of the target image relative to a particular identifier.

8. The method of claim 7, wherein the storage device obtains the target image from a corresponding video frame in the video according to the location information when the storage device receives a request to read the target image during a storage life cycle of the video.

9. The method according to claim 7 or 8, further comprising: when the storage life cycle of the video is finished, the storage device acquires the target image from the corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

10. A media data transmission apparatus, the apparatus comprising:

a second generation unit configured to generate a video using the plurality of original video frames;

a transmitting unit for transmitting the video and the position information to a storage device;

in the case where the class of the target image is part of a video frame in the video, the relative positions include a second relative position; the second relative position includes a relative position of the target image in a corresponding video frame; the relative position of the target image in the corresponding video frame includes an offset of the position of the target image relative to a particular identifier.

11. The apparatus of claim 10, wherein the media data transmission means does not generate the target image and does not send the target image to the storage device.

12. A camera, the camera comprising:

a processor for generating a plurality of original video frames, the plurality of original video frames being used to generate a video; acquiring position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

the receiving and transmitting module is used for transmitting the video and the position information to a storage device;

in the case where the target image is part of a video frame in the video, the relative positions include a second relative position; the second relative position includes a relative position of the target image in a corresponding video frame; the relative position of the target image in the corresponding video frame includes an offset of the position of the target image relative to a particular identifier.

13. The camera of claim 12, wherein the camera does not generate the target image and does not send the target image to the storage device.

14. A storage device, the device comprising:

the receiving and transmitting module is used for receiving the video sent by the camera and the position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

the processor is used for acquiring the target image from the corresponding video frame of the video according to the position information;

15. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1-9.