CN114282052A - Video image localization method and system based on frame feature - Google Patents

Abstract

The present invention relates to the technical field of image data processing, in particular to a method and system for locating video images based on frame features. The method includes the following contents: a video analysis step: splitting a video to be matched into video frames, and extracting from the video frames Extracting image features and storing image features; image processing steps: acquiring the target image, segmenting the target image to generate a target subject image, and extracting target features from the target subject image; image matching step: matching the target features with the image features in turn, calculating Similarity; image screening and push steps: according to the similarity, the push image is screened from the video frame, and the image information is obtained and pushed according to the push image. With this solution, the technical problem of low image matching accuracy in the prior art can be solved.

Description

Video image localization method and system based on frame features

technical field

The present invention relates to the technical field of image data processing, in particular to a frame feature-based video image positioning method and system.

Background technique

The rapid rise of multimedia makes all kinds of videos appear in front of people. During the process of watching videos, users will take screenshots to save and share the pictures they like or are interested in. The idea of viewing the full video or wanting to know where the image came from. In the prior art, a common method is to match the screenshot with the video frame in the video, so as to know the source of the screenshot. However, there is considerable noise in the screenshot, which is equivalent to interference information, and these interference information will affect the accuracy of the matching process, resulting in the inability to accurately match the video frame corresponding to the screenshot, and to obtain the video source accurately.

SUMMARY OF THE INVENTION

One of the objectives of the present invention is to provide a video image positioning method based on frame features, so as to solve the technical problem of low image matching accuracy in the prior art.

Basic scheme 1 provided by the present invention: a video image positioning method based on frame features, including the following contents:

Video analysis step: split the video to be matched into video frames, extract image features from the video frames, and store the image features;

Image processing step: acquiring a target image, segmenting the target image to generate a target subject image, and extracting target features from the target subject image;

Image matching step: match the target features with the image features in sequence, and calculate the similarity;

Image screening and push steps: Screen push images from video frames according to similarity, and obtain and push image information according to push images.

Beneficial effects of Basic Program 1:

The target image is the image that the user wants to know the source of. The target image is segmented through the image processing step to obtain the target subject image. The target subject image contains the key information in the target image. By segmenting the target image, the noise in the target image is removed and the matching process is reduced. interference information in . The video analysis step and the image processing step extract the target feature and the image feature respectively, and the target feature and the image feature are matched through the image matching step, and the matching speed is faster by extracting the feature for matching.

Through the steps of image screening and pushing, the video frames are screened according to the similarity, so as to obtain the push image similar to the target image, and then obtain the image information of the push image to push, and know the position of the push image in the video through the image information. Compared with the prior art, by adopting this solution, by dividing the target image, the interference in the matching process is reduced, and the accuracy of image matching is improved, thereby realizing the precise positioning of the target image in the video.

Further, segment the target image to generate the target subject image, which specifically includes the following contents:

Perform contour recognition on the target image; screen out the target contour from the identified contours as the target subject image according to preset screening conditions.

Beneficial effect: In most images, the effective information is concentrated in each figure in the image. When segmenting the target image, each figure in the target image is obtained through contour recognition, and the identified contours are screened by filtering conditions, thereby obtaining the target subject image, that is, the effective information in the image.

Further, image features are extracted from video frames, including the following:

Preprocessing video frames;

Obtain the low-frequency signal of each pixel in the video frame after preprocessing, and assign each pixel in the video frame according to the low-frequency signal;

Generate image features based on the assignment results.

Beneficial effects: when extracting image features, assign values to each pixel point according to the low-frequency signal of the video frame, so as to obtain image features.

Further, according to the target features and the image features in sequence, the similarity is calculated, which specifically includes the following contents:

Calculate the Hamming distance of the target feature and the image feature in turn, and the Hamming distance is the similarity.

Beneficial effects: By calculating the Hamming distance, the distance between the target feature and the image feature is known, so as to obtain the similarity between the target image and the video frame.

Further, the image information includes a time point and a similarity, and the format of the image information is json.

Beneficial effects: The time point is the time point of the video frame in the corresponding video, and the similarity is the similarity between the video frame and the target image. It is convenient for the user to quickly locate the push image in the corresponding video through the image information, so as to view and confirm the push image and target. Whether the images are the same image. The image information is in json format, which is faster to read and write, easy to use, and has wider compatibility.

Another object of the present invention is to provide a video image positioning system based on frame features.

The present invention provides basic solution 2: a video image positioning system based on frame features, including:

The video splitting module is used to split the video to be matched into video frames;

Also includes:

The target segmentation module is used to segment the acquired target image to generate the target subject image;

A feature extraction module for extracting image features from video frames, and also for extracting target features from target subject images;

The similarity calculation module is used to match the image features in sequence according to the target features, and calculate the similarity;

The information screening module is used to filter out the push images from the video frames according to the similarity, and obtain image information according to the push images.

Beneficial effects of basic plan two:

The target image is the image that the user wants to know the source of. The settings of the image segmentation module, segment the target image to obtain the target subject image. The target subject image contains the key information in the target image. By segmenting the target image, the noise in the target image is removed, reducing Interference information in the matching process.

The setting of the feature extraction module extracts target features and image features from the target image and the split video frame respectively. The setting of the similarity calculation module matches the target feature and the image feature, and the matching speed is faster by extracting the feature.

The settings of the information screening module screen the video frames according to the similarity, so as to obtain the push image similar to the target image, and then obtain the image information of the push image to push, and know the position of the push image in the video through the image information. Compared with the prior art, by adopting this solution, by dividing the target image, the interference in the matching process is reduced, and the accuracy of image matching is improved, thereby realizing the precise positioning of the target image in the video.

Further, the target segmentation module is used to perform contour recognition on the acquired target image, and select the target contour from the identified contours as the target subject image according to preset screening conditions.

Beneficial effect: In most images, the effective information is concentrated in each figure in the image. The settings of the image segmentation module, when segmenting the target image, obtain each figure in the target image through contour recognition, and filter the identified contours through the screening conditions, so as to obtain the target subject image, that is, the effective information in the image.

Further, the feature extraction module is used to preprocess the video frame; it is also used to obtain the low frequency signal of each pixel point in the preprocessed video frame, assign value to each pixel point in the video frame according to the low frequency signal, and generate image features according to the assignment result. .

Beneficial effects: when extracting image features, the feature extraction module reassigns each pixel point through the low-frequency signal of each pixel point, so as to obtain the image feature.

Further, the similarity calculation module is used to calculate the Hamming distance of the target feature and the image feature in turn, and the Hamming distance is the similarity.

Beneficial effect: the setting of the similarity calculation module, through calculating the Hamming distance, the distance between the target feature and the image feature is known, so as to know the similarity between the target image and the video frame.

Further, the image information includes a time point and a similarity, and the format of the image information is json.

Beneficial effects: The time point is the time point of the video frame in the corresponding video, and the similarity is the similarity between the video frame and the target image. It is convenient for the user to quickly locate the push image in the corresponding video through the image information, so as to view and confirm the push image and target. Whether the images are the same image. The image information is in json format, which is faster to read and write, easy to use, and has wider compatibility.

Description of drawings

FIG. 1 is a logical block diagram of an embodiment of a video image positioning system based on frame features of the present invention.

Detailed ways

The following is further described in detail by specific embodiments:

Example

A video image localization method based on frame features, including the following:

Video analysis step: split the video to be matched into video frames, extract image features from the video frames, and store the image features. The video to be matched is all the videos stored in the system. In other embodiments, when the image to be located is the image stored by the user's own screenshot, the video to be matched is the video that the user has browsed.

Extracting image features from video frames, including the following: preprocessing the video frame; obtaining the low-frequency signal of each pixel in the preprocessed video frame, and assigning a value to each pixel in the video frame according to the low-frequency signal; according to the assignment result Generate image features. Specifically, an image size is preset, and the video frame is scaled according to the image size to obtain a video frame with the same size as the image; the scaled video frame is converted into a grayscale image; the preset discrete cosine transform is called, and the The image calculation generates a DCT matrix. The DCT matrix includes the DCT value of each pixel point. According to the DCT matrix, the DCT average value of the grayscale image is calculated, and the value is assigned according to the DCT value and the DCT average value of each pixel point. When the pixel point DCT value is greater than or equal to When the DCT average value is used, the hash value of the pixel point is assigned as 1, and when the DCT value of the pixel point is less than the DCT average value, the hash value of the pixel point is assigned as 0; the hash code is generated according to the assignment result of each pixel point. , in this embodiment, the hash value is combined into an integer from left to right and from top to bottom, which is a hash code, and the hash code is stored in an npy file, which is an image feature. npy files refer to numpy-specific binary files, and numpy is a package for data processing in python.

Image processing steps: acquiring a target image, segmenting the target image to generate a target subject image, and extracting target features from the target subject image.

The target image is segmented to generate the target subject image, which specifically includes the following content: a screening condition is preset, and the screening condition is the contour with the largest area, that is, the contour with the most pixels. Perform contour recognition on the target image; select the target contour from the identified contours as the target subject image according to the preset screening conditions, that is, count the pixels of the identified contours, and select the contour with the most pixels as the target contour.

Extracting target features from the target subject image is the same as extracting image features from video frames, and specifically includes the following contents: preprocessing the target subject image; Assign values to each pixel in the target subject image; generate target features according to the assignment results. Specifically, the image size is preset, and the target subject image is scaled according to the image size to obtain a target subject image with the same size as the image; the scaled target subject image is converted into a grayscale image; the preset discrete cosine transform is called, Calculate and generate a DCT matrix according to the grayscale image. The DCT matrix includes the DCT value of each pixel point. Calculate the DCT average value of the grayscale image according to the DCT matrix, and assign values according to the DCT value and the DCT average value of each pixel point. When the pixel point DCT value When greater than or equal to the DCT average value, assign the hash value of the pixel point to 1, when the pixel point DCT value is less than the DCT average value, assign the hash value of the pixel point to 0; according to the assignment result of each pixel point To generate a hash code, in this embodiment, the hash value is combined into an integer from left to right and from top to bottom, which is stored in binary, which is a hash code, and the hash code is stored in an npy file, which is the target feature.

Image matching step: Match with the image features in sequence according to the target features, and calculate the similarity. The image matching step specifically includes the following contents: calculating the Hamming distance of the target feature and the image feature in turn, that is, calculating the Hamming distance between the hash code corresponding to the target feature and the hash code corresponding to the image feature, and characterizing the similarities and differences between the two hash codes by the Hamming distance, that is, Hamming distance is the similarity between target features and image features.

Image screening and push steps: Screen push images from video frames according to similarity, and obtain and push image information according to push images. Image information includes time point and similarity, and the format of image information is json. Specifically, the video frames are sorted from high to low according to the similarity, a corresponding number of video frames are selected as the push image according to the preset push threshold, and the time point in the video of the video frame corresponding to the push image is obtained according to the push image, Generate image information corresponding to the push image according to the time point and similarity, and integrate the push information into json format. In this embodiment, the push threshold is 4, that is, the top 4 video frames in the sorting order are selected as push images.

The frame feature-based video image positioning system, using the above frame feature-based video image positioning method, as shown in Figure 1, includes a video splitting module, a target splitting module, a feature extraction module, a similarity calculation module and an information screening module.

The video splitting module is used to split the video to be matched into video frames, extract image features from the video frames, and store the image features. The video to be matched is all the videos stored in the system. In other embodiments, when the image to be located is the image stored by the user's own screenshot, the video to be matched is the video that the user has browsed. Specifically, the video splitting module is used for preprocessing the split video frames. The preprocessing includes: presetting an image size, scaling the video frame according to the image size to obtain a video frame with the same size as the image; converting the scaled video frame into a grayscale image. The video splitting module is also used to obtain the low-frequency signal of each pixel in the preprocessed video frame, assign values to each pixel in the video frame according to the low-frequency signal, and generate image features according to the assignment result. Call the preset discrete cosine transform, calculate and generate the DCT matrix according to the grayscale image, the DCT matrix includes the DCT value of each pixel, calculate the DCT average value of the grayscale image according to the DCT matrix, and calculate the DCT value and DCT average value of each pixel point according to the DCT value. Carry out assignment, when the DCT value of the pixel point is greater than or equal to the DCT average value, assign the hash value of the pixel point to 1, when the pixel point DCT value is less than the DCT average value, assign the hash value of the pixel point to 0 ; Generate a hash code according to the assignment result of each pixel, in the present embodiment, the hash value is combined into an integer from left to right from top to bottom, which is a hash code, and the hash code is stored by using an npy file, which is an image feature. npy files refer to numpy-specific binary files, and numpy is a package for data processing in python.

The target segmentation module is used for segmenting the acquired target image to generate a target subject image. Specifically, the target segmentation module presets a screening condition, and the screening condition is the contour with the largest area, that is, the contour with the most pixels. The target segmentation module is used to identify the contour of the obtained target image, and select the target contour from the identified contour as the target subject image according to the screening conditions, that is, count the pixels of the identified contour, and select the contour with the most pixels as the target. contour.

The feature extraction module is used to extract target features from the target subject image. Specifically, the feature extraction module is used to preprocess the target subject image, and the preprocessing includes: presetting the image size, scaling the target subject image according to the image size, and obtaining the target subject image with the same size as the image; The target subject image is converted to a grayscale image. The feature extraction module is also used to obtain the low-frequency signal of each pixel in the target subject image after preprocessing, and assign values to each pixel in the target subject image according to the low-frequency signal; and generate target features according to the assignment result. Call the preset discrete cosine transform, calculate and generate the DCT matrix according to the grayscale image, the DCT matrix includes the DCT value of each pixel, calculate the DCT average value of the grayscale image according to the DCT matrix, and calculate the DCT value and DCT average value of each pixel point according to the DCT value. Carry out assignment, when the DCT value of the pixel point is greater than or equal to the DCT average value, assign the hash value of the pixel point to 1, when the pixel point DCT value is less than the DCT average value, assign the hash value of the pixel point to 0 ; Generate hash code according to the assignment result of each pixel, in the present embodiment, from left to right from top to bottom, the hash value is combined into an integer, adopt binary to save, it is hash code, and hash code adopts npy file to store , which is the target feature.

The similarity calculation module is used to sequentially match with the image features according to the target features, and calculate the similarity. Specifically, the similarity calculation module is used to calculate the Hamming distance of the target feature and the image feature in turn, that is, to calculate the Hamming distance of the hash code corresponding to the target feature and the hash code corresponding to the image feature, and to characterize the similarities and differences between the two hash codes by the Hamming distance, that is, Hamming distance is the similarity between target features and image features.

The information screening module is used to filter out the push images from the video frames according to the similarity, and obtain and push the image information according to the push images. Image information includes time point and similarity, and the format of image information is json. Specifically, the information screening module is preset with a push threshold. In this embodiment, the push threshold is 4, that is, the top 4 video frames in the sorting order are selected as the push images. The information screening module is used to sort the video frames from high to low according to the similarity, filter out the corresponding number of video frames as the push image according to the preset push threshold, and obtain the time in the video of the video frame corresponding to the push image according to the push image point, generate image information corresponding to the push image according to the time point and similarity, and integrate the push information into json format.

The above are only the embodiments of the present invention, and the common knowledge such as the well-known specific structures and characteristics in the scheme has not been described too much here. Those of ordinary skill in the art know that the invention belongs to the technical field before the filing date or the priority date. Technical knowledge, can know all the prior art in this field, and have the ability to apply conventional experimental means before the date, those of ordinary skill in the art can improve and implement this scheme in combination with their own ability under the enlightenment given in this application, Some typical well-known structures or well-known methods should not be an obstacle to those skilled in the art from practicing the present application. It should be pointed out that for those skilled in the art, some modifications and improvements can be made without departing from the structure of the present invention. These should also be regarded as the protection scope of the present invention, and these will not affect the implementation of the present invention. Effectiveness and utility of patents. The scope of protection claimed in this application shall be based on the content of the claims, and the descriptions of the specific implementation manners in the description can be used to interpret the content of the claims.

Claims (10)

1. a video image positioning method based on frame feature, is characterized in that, comprises the following content: Video analysis step: split the video to be matched into video frames, extract image features from the video frames, and store the image features; Image processing step: acquiring a target image, segmenting the target image to generate a target subject image, and extracting target features from the target subject image; Image matching step: match the target features with the image features in sequence, and calculate the similarity; Image screening and push steps: Screen push images from video frames according to similarity, and obtain and push image information according to push images. 2. The frame feature-based video image positioning method according to claim 1, wherein the target image is segmented to generate a target subject image, specifically comprising the following content: Perform contour recognition on the target image; screen out the target contour from the identified contours as the target subject image according to preset screening conditions. 3. the video image location method based on frame feature according to claim 1, is characterized in that: extract image feature from video frame, specifically comprise the following content: Preprocessing video frames; Obtain the low-frequency signal of each pixel in the video frame after preprocessing, and assign each pixel in the video frame according to the low-frequency signal; Generate image features based on the assignment results. 4. the video image positioning method based on frame feature according to claim 1, is characterized in that: carry out matching with image feature successively according to target feature, calculate similarity, specifically comprise the following content: Calculate the Hamming distance of the target feature and the image feature in turn, and the Hamming distance is the similarity. 5 . The frame feature-based video image positioning method according to claim 1 , wherein the image information includes a time point and a similarity, and the format of the image information is json. 6 . 6. Video image positioning system based on frame features, including: The video splitting module is used to split the video to be matched into video frames; It is characterized in that it also includes: The target segmentation module is used to segment the acquired target image to generate the target subject image; A feature extraction module for extracting image features from video frames, and also for extracting target features from target subject images; The similarity calculation module is used to match the image features in sequence according to the target features, and calculate the similarity; The information screening module is used to filter out the push images from the video frames according to the similarity, and obtain image information according to the push images. 7. The frame feature-based video image positioning system according to claim 6, wherein the target segmentation module is used to carry out contour identification to the obtained target image, and screen out the identified contour according to a preset screening condition The target contour is taken as the target subject image. 8. The frame feature-based video image positioning system according to claim 6, wherein the feature extraction module is used to preprocess the video frame; it is also used to obtain the low frequency signal of each pixel in the preprocessed video frame , assign values to each pixel in the video frame according to the low-frequency signal, and generate image features according to the assignment results. 9 . The frame feature-based video image positioning system according to claim 6 , wherein the similarity calculation module is used to sequentially calculate the Hamming distance of the target feature and the image feature, and the Hamming distance is the similarity. 10 . 10 . The frame feature-based video image positioning system according to claim 6 , wherein the image information includes a time point and a similarity, and the format of the image information is json. 11 .

Images