Disclosure of Invention
The invention aims to provide a marketing video auditing system based on AI, which aims to solve the problems in the background technology.
In order to solve the technical problems, the invention provides the following technical scheme: a marketing video auditing system based on AI comprises a preprocessing module, an auditing model establishing module and a visual optimization module, wherein the preprocessing module is used for preprocessing a video to be audited, the auditing model establishing module is used for establishing a model based on video auditing, the visual optimization module is used for further optimizing the audited video, the preprocessing module comprises a visual frame extracting module, a feature recognition module, a single-frame data quantity recognition module and a mathematical curve fitting module, the visual frame extracting module extracts the number of video frames which can be continuously recognized by human eyes to serve as a unit of visual frame, the feature recognition module is used for performing feature recognition on the extracted visual frame, the single-frame data quantity recognition module is used for performing data quantity recognition on the visual frame of each unit, and the mathematical curve fitting module is used for fitting the data quantity of the visual frame of each unit to be a curve.
According to the technical scheme, the auditing model establishing module comprises a feature migration module, a convolution layering module, an original domain accumulation module and an identification domain selection module, wherein the feature migration module is used for model migration of features identified in the preprocessing process, the convolution layering module is used for convolution layering according to feature types, the original domain accumulation module is used for providing multiple features of an original data model, and the identification domain selection module is used for selecting an identification domain matched with the original data model according to a preprocessing result.
According to the technical scheme, the visual optimization module comprises a multidirectional filtering module, a single-frame confidence coefficient calculating module and a feedback module, the multidirectional filtering module is used for carrying out multidirectional filtering on the video to be examined, the single-frame confidence coefficient calculating module is used for carrying out confidence coefficient calculation on the split single-frame image, and the feedback module is used for carrying out feedback based on a time point on the video to be examined according to the confidence coefficient.
According to the technical scheme, in the system, the operation method of the preprocessing module comprises the following steps:
determining a visual frame, and extracting a multi-unit visual frame from the video;
performing feature recognition on the extracted multi-unit visual frame, wherein the feature recognition content comprises direct features, semantic features and repeated features;
carrying out data quantity identification statistics on the visual frame of each unit, and fitting a data quantity change curve by taking time as a reference;
and screening out the visual frames needing further examination by combining the identification characteristics and the data volume change curve.
According to the technical scheme, the screening method of the visual frames needing further auditing comprises the following steps:
after the feature recognition is carried out on the visual frame of a unit, a feature matrix of the visual frame of the unit is generated
;
Performing weighted average on elements in the feature matrix to generate a weight value of the unit visual frame
;
Setting an audit weight threshold
When it comes to
Time marks the time point of the unit visual frame and takes the unit visual frame as a candidate auditing visual frame;
amount of data contained in each unit visual frame
Identifying and outputting, and fitting time points
Graph with data amount, wherein
Is a set of data arrays of data amount in bytes, and the change of data amount contained in unit visual frame is obeyed by time point
The curve change of (c);
calculating the change trend of the visual frame data of each adjacent time point, and calculating the data change degree
;
And judging whether the data change is abnormal or not according to the AI, and marking and uploading abnormal points.
According to the technical scheme, the data change degree
The calculation method comprises the following steps:
in the formula, a whole video channel is represented
After the extraction of the identification frames of the units, the average data amount of each unit identification frame,
representing the variance value of all unit identification frames, directly reflecting each segmentA degree of data change for the frame is identified, wherein,
is as follows
The unit identifies the amount of data of the frame, in bytes,
the number of frames is identified for the total decimation,
the larger the data size is, the more obvious the change of the identification frame data of the section is, and the larger the fluctuation is;
and after the identification is finished, judging whether the data change is normal according to the AI, and carrying out time point marking on the abnormal identification frame and uploading.
According to the technical scheme, after the abnormal identification frame is uploaded, the verification model is used for further verification, and the establishment method of the verification model comprises the following steps:
step S1: establishing a characteristic original domain, specifically a characteristic original domain containing all characteristic ranges of the identification frame;
step S2: feature matrix
Carrying out convolution layering with the characteristic original domain, and establishing an auditing model with characteristic layering;
and step S3: after each audit, performing original domain accumulation on the identification of the features in the audit content, and storing the identification in a feature matrix form;
and step S4: after the verification is finished, selecting the identification domains with the corresponding layers for identification operation, and after the identification is passed, canceling the mark of the identification frame content, otherwise, continuously outputting.
According to the above technical solution, the operation method of the vision optimization module further comprises the steps of:
performing multi-directional filtering based on feature layering, wherein the filtering degree is related to the feature layer number of the visual frame;
calculating single frame confidence of unit visual frame according to characteristic parameters
Wherein the characteristic parameters comprise the total characteristic convolution layer number
Number of characteristic layers for filtration
Conversion coefficient of
。
According to the technical scheme, the single frame confidence degree
The calculation method comprises the following steps:
in the formula (I), the compound is shown in the specification,
the feature recognition depth after filtering is represented,
the proportion of the identification depth to the total characteristic convolution layer number is represented, the larger the proportion of the identification depth to the total characteristic convolution layer number is, the less the characteristic redundancy is represented, the higher the safety factor of a single identification frame is, and specifically, the converted coefficient is used
A converted single frame confidence representation;
after single-frame confidence coefficient calculation, marking the visual frames in the auditing model, performing danger feedback on the visual frames with the confidence coefficient lower than the average confidence coefficient, sending the visual frames to a manual auditing end for further auditing, and performing repeated model auditing on the visual frames with the confidence coefficient higher than the average confidence coefficient to realize multiple screening.
Compared with the prior art, the invention has the following beneficial effects: according to the invention, the visual frame extraction module is arranged, so that the video frame number which can be smoothly identified by human eyes is extracted as the visual frame, the AI operation complexity is reduced while the manual review is simulated, and the subsequent curve fitting is facilitated; the data volume in the identification frame of each unit is identified by arranging a single-frame data volume identification module, and a curve change image is fitted by using a time point for auxiliary identification; the feature matrix and the original feature domain are subjected to convolution layering through the convolution layering module, the auditing precision is improved, the original domain is updated, the confidence coefficient of a single identification frame is calculated through the single-frame confidence coefficient calculating module, the content of the identification frame is further screened, the part with abnormal numerical values is submitted to a manual place for further auditing, and the auditing efficiency is improved.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-2, the present invention provides the following technical solutions: a marketing video auditing system based on AI comprises a preprocessing module, an auditing model establishing module and a visual optimization module, wherein the preprocessing module is used for preprocessing a video to be audited, the auditing model establishing module is used for establishing a model based on video auditing, the visual optimization module is used for further optimizing the audited video, the preprocessing module comprises a visual frame extracting module, a feature recognition module, a single-frame data quantity recognition module and a mathematical curve fitting module, the visual frame extracting module extracts the number of video frames which can be continuously recognized by human eyes to serve as a unit of visual frame, the feature recognition module is used for carrying out feature recognition on the extracted visual frame, the single-frame data quantity recognition module is used for carrying out data quantity recognition on the visual frame of each unit, and the mathematical curve fitting module is used for fitting the data quantity of the visual frame of each unit to be a curve.
The auditing model establishing module comprises a feature migration module, a convolution layering module, an original domain accumulation module and an identification domain selection module, wherein the feature migration module is used for performing model migration on the identified features in the preprocessing process, the convolution layering module is used for performing convolution layering according to feature types, the original domain accumulation module is used for providing multiple features of an original data model, and the identification domain selection module is used for selecting an identification domain matched with the original data model according to the preprocessing result.
The visual optimization module comprises a multidirectional filtering module, a single-frame confidence coefficient calculating module and a feedback module, wherein the multidirectional filtering module is used for carrying out multidirectional filtering on the video to be examined, the single-frame confidence coefficient calculating module is used for carrying out confidence coefficient calculation on the split single-frame image, and the feedback module is used for carrying out feedback based on time points on the video to be examined according to the confidence coefficient.
In the system, the operation method of the preprocessing module comprises the following steps:
determining a visual frame, and extracting multi-unit visual frames from the video; the visual frames are extracted by taking the number of video frames which can be smoothly identified by human eyes as a unit, so that the consistency of video audit is ensured;
performing feature recognition on the extracted multi-unit visual frame, wherein the feature recognition content comprises direct features, semantic features and repeated features;
carrying out data quantity identification statistics on the visual frame of each unit, and fitting a data quantity change curve by taking time as a reference;
and screening out the visual frames needing further examination by combining the identification characteristics and the data volume change curve.
The screening method of the visual frames needing further auditing comprises the following steps:
after the feature recognition is carried out on the visual frame of a unit, a feature matrix of the visual frame of the unit is generated
(ii) a After the visual frame is subjected to feature recognition, generating
Line of
A feature matrix of columns, the matrix containing all features of a single visual frame;
performing weighted average on elements in the feature matrix to generate a weight value of the unit visual frame
(ii) a After elements in the feature matrix are weighted and averaged, screening out elements which are least matched and excessively matched with the identification content and the features, generating a weight value of the visual frame, and feeding back an audit weight of the visual frame;
setting an audit weight threshold
When is coming into contact with
Time marking the time point of the unit visual frame, and using the time point as a candidate audit visual frame; auditing weight threshold
The big data is obtained by filtering according to the weight value of the identification content in the historical audit record, and is a dynamic value, the audit content is different every time, and the weight threshold is differentThe value also changes;
amount of data contained in each unit visual frame
Recognizing and outputting, and fitting out time points
Graph with data amount, wherein
Is a set of data arrays of data amount in bytes, and the change of data amount contained in unit visual frame is obeyed by time point
The curve change of (c); counting the data quantity contained in each identification frame, fitting a data change curve with the time point, recording the data flow in a set form, and judging the time point with abnormal data quantity by combining curve data;
calculating the change trend of the visual frame data of each adjacent time point, and calculating the data change degree
;
And judging whether the data change is abnormal or not according to the AI, and marking and uploading abnormal points.
Degree of data change
The calculating method comprises the following steps:
in the formula (I), the compound is shown in the specification,
representing a whole video channel
After the extraction of the identification frames of the units, the average data amount of each unit identification frame,
the variance value of all unit identification frames is expressed, and the data change degree of each identification frame is directly reflected, wherein,
is as follows
The unit identifies the amount of data of the frame, in bytes,
the number of frames is identified for the total decimation,
the larger the data size is, the more obvious the change of the identification frame data of the section is, and the larger the fluctuation is;
and after the identification is finished, judging whether the data change is normal according to the AI, and carrying out time point marking on the abnormal identification frame and uploading.
After the abnormal recognition frame is uploaded, the auditing model is used for further auditing, and the establishing method of the auditing model comprises the following steps:
step S1: establishing a characteristic original domain, specifically a characteristic original domain containing all characteristic ranges of the identification frame; the original features contain larger redundancy, an original feature domain is established after classification is carried out through a classifier, effective features are screened out according to AI and put into the domain, and the complexity of the feature original domain is reduced;
step S2: feature matrix
Carrying out convolution layering on the feature original domain, and establishing an auditing model with feature layering; after convolution of the feature matrix and the elements in the feature original domain, the feature matrix containing a plurality of directions is generatedLayering vectors, accessing the vectors into an audit model, matching most features in a visual frame, uploading the unmatched parts and updating an original domain to complete accumulation of the original domain, wherein the more the original domain is accumulated, the more accurate the feature identification is;
and step S3: after each audit, performing original domain accumulation on the identification of the features in the audit content, and storing the identification in a feature matrix form;
and step S4: and after the auditing is finished, selecting the identification domain with the corresponding layer number for identification operation, and after the identification is passed, identifying the frame content canceling mark, otherwise, continuously outputting.
The method for operating the vision optimization module further comprises the following steps:
performing multi-directional filtering based on feature layering, wherein the filtering degree is related to the feature layer number of the visual frame; the more the number of the characteristic layers is, the richer the video content is represented, the more the hidden characteristics are buried, and the more the common characteristics needing to be filtered are;
calculating single frame confidence of unit visual frame according to characteristic parameters
Wherein the characteristic parameter comprises the total characteristic convolution layer number
Number of characteristic layers for filtration
Conversion coefficient of
。
Single frame confidence
The calculation method comprises the following steps:
in the formula (I), the compound is shown in the specification,
the feature recognition depth after filtering is represented,
the proportion of the identification depth to the total characteristic convolution layer number is represented, the larger the proportion of the identification depth to the total characteristic convolution layer number is, the less the characteristic redundancy is represented, the higher the safety factor of a single identification frame is, and specifically, the converted coefficient is used
A converted single frame confidence representation;
after single-frame confidence coefficient calculation, marking the visual frames in the auditing model, performing danger feedback on the visual frames with the confidence coefficient lower than the average confidence coefficient, sending the visual frames to a manual auditing end for further auditing, and performing repeated model auditing on the visual frames with the confidence coefficient higher than the average confidence coefficient to realize multiple screening.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.