JPH1013832A - Moving picture recognizing method and moving picture recognizing and retrieving method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To directly recognize and retrieve a specific pattern from compressed moving picture data through the use of the parameter of maximum likelihood with respect to a symbol string by extracting a DCT coefficient from image data and transforming its feature vector string to the symbol string. SOLUTION: One frame of image data is divided by MPEG data 1 to obtain the DCT(discrete cosine transformation) coefficient of each unit. Next, a feature extraction part 2 fetches the frame feature vector of a low frequency component. Similarly, the feature vector string of a series of moving picture data is fetched and recorded in a memory for storing feature 3. Next, a quantization part 4 vector-quantizes it and records the symbol string in a symbol storing memory 5. A model parameter estimating part 6 estimates the parameter of such a state transition model as generates this symbol string and records it in a state transmission model storing memory for recognition 7. A likelihood calculating part 8 estimates the parameter of the model of high likelihood for each category of a recognizing object and stores it in a memory for a recognizing result 9. Thereby the specific moving picture pattern is recognized and retrieved from compressed moving picture data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image recognizing method and a moving image recognizing and retrieving method, and more particularly, to recognizing and retrieving a specific moving image pattern from image data of each screen displaying a series of moving images. To a moving image recognition method and a moving image recognition search method.

[0002]

2. Description of the Related Art In recent years, a great deal of research has been conducted on pattern recognition technology for moving images.
The technique described in the following publication (a) is known.

(A) Japanese Patent Application Laid-Open No. 5-46583 The above-mentioned Japanese Patent Application Publication (A) (Japanese Patent Application Laid-Open No. 5-46583) includes:
A mesh feature of a moving object extracted from image data of each screen displaying a moving image is symbolized by vector quantization, a moving image sequence is converted into a symbol sequence, and the symbol sequence is learned and recognized, thereby enabling a human or the like. A method for recognizing each motion of the moving object is described.

[0004] In addition, as a data compression technology for storing or transmitting moving image data, which constitutes a core technology of multimedia, an MPEG (Moving Picture E) is used.
International standard encoding schemes such as xparts Group (international standard for media-integrated moving image compression) and MPEG2 are becoming widespread.

[0005]

As described in the above publication (A) (Japanese Patent Application Laid-Open No. 5-46583), a specific moving image pattern is selected from a conventional series of moving images by using the moving image. When a search is performed using the image pattern itself as a search key, it is necessary to handle a large amount of image data and feature amount data, and there is a problem in that the processing time of data processing increases.

[0006] In addition, standard encoding methods such as MEPG and MEPG2 are becoming widespread, and when a specific moving image pattern is searched from a series of moving images using the moving image pattern itself as a search key, this standard coding method is used. By using moving image data compressed by the encoding method, it is expected that the processing time of data processing will be reduced.

However, there has been no study on an optimal method for searching for a specific moving image pattern from a series of moving images by using moving image data compressed by the standard encoding method.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a method for recognizing a moving image using moving image data compressed by a standard encoding method or the like. An object of the present invention is to provide a technology capable of shortening a processing time.

[0009] Another object of the present invention is to provide a technique capable of shortening the data processing time by using moving picture data compressed by a standard coding method or the like in a moving picture recognition and retrieval method. It is in.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0011]

SUMMARY OF THE INVENTION Among the inventions disclosed in the present application, the outline of a representative one will be briefly described.
It is as follows.

(1) In a moving image recognition method for recognizing a moving image pattern of a series of moving images, image data of each screen for displaying a series of moving images is divided into M × N blocks, and a DCT coefficient of each block is calculated. Extracting, said DC
A step of extracting at least one of the T coefficients as a feature vector of each screen, and a time-series feature vector sequence composed of feature vectors of each screen displaying a specific moving image pattern form a stochastic state transition model. Learning for each of a plurality of specific moving image patterns serving as a recognition key, and a time-series feature vector composed of feature vectors extracted from image data of each screen displaying a series of moving images to be recognized. Outputting, as a recognition result, a moving image pattern of a state transition model in which the likelihood of the sequence with respect to the plurality of state transition models obtained by the learning is maximized.

(2) In the means of (1), the image data of each screen displaying the series of moving images to be recognized is compressed by a standard encoding method, and the feature vector of each screen is It is characterized in that a part of the DCT coefficients included in the image data of each screen compressed by the standard encoding method is used.

(3) In the means of the above (1), a motion vector is used together with a DCT coefficient as a feature vector of each screen.

(4) In the means of (3), the image data of each screen displaying the series of moving images to be recognized is compressed by a standard encoding method, and the feature vector of each screen is It is characterized by using a part of DCT coefficients included in image data of each screen compressed by the standard encoding method and a motion compensation vector.

(5) In a moving image recognition / retrieval method for extracting a time region including a specific moving image pattern from a series of moving images, image data of each screen displaying a series of moving images is M × N. Extracting a DCT coefficient of each block into blocks, extracting at least one of the DCT coefficients as a feature vector of each screen, and a feature of each screen displaying a specific moving image pattern serving as a search key. It consists of a step of learning a stochastic state transition model by a time-series feature vector sequence composed of vectors, and a feature vector extracted from image data of each screen displaying a series of moving images to be searched. A step of outputting, as a search result, a time domain having a high likelihood with respect to the state transition model obtained by the learning in the time-series feature vector sequence. Characterized by including and.

(6) In the means of (5), the image data of each screen displaying the series of moving images to be searched is compressed by a standard encoding method, and the feature vector of each screen is It is characterized in that a part of the DCT coefficients included in the image data of each screen compressed by the standard encoding method is used.

(7) In the means of the above (5), a motion vector is used together with a DCT coefficient as a feature vector of each screen.

(8) In the means of (7), the image data of each screen displaying the series of moving images to be searched is compressed by a standard encoding method, and the feature vector of each screen is It is characterized by using a part of DCT coefficients included in image data of each screen compressed by the standard encoding method and a motion compensation vector.

According to each of the above-mentioned means, DCT is used as the characteristic amount.
A specific moving image pattern is directly recognized and searched from a small amount of moving image data compressed by a standard encoding method such as MEPG or MEPG2 using a coefficient or a DCT coefficient and a motion compensation vector. The processing time of data processing can be reduced.

[0021]

Embodiments of the present invention will be described below in detail with reference to the drawings.

In all the drawings for describing the embodiments of the present invention, components having the same function are denoted by the same reference numerals, and their repeated description will be omitted.

FIG. 1 is a functional block diagram showing a schematic configuration of a moving image recognition and search apparatus to which a moving image recognition method and a moving image recognition and search method according to an embodiment of the present invention are applied.

In FIG. 1, 1 is MEPG data, 2 is a feature extraction unit, 3 is a feature storage memory, 4 is a quantization unit, 5
Is a symbol storage memory, 6 is a model parameter estimator,
7 is a memory for storing a state transition model for recognition, 8 is a likelihood calculator, and 9 is a memory for a recognition result.

Here, for example, an external storage device is used as the recognition state transition model storage memory 7 and the recognition result memory 9, and the MEPG data 1 is
For example, it is stored in an external storage device.

The basic operation of the embodiment of the present invention has three stages of learning and recognition. At the time of learning, parameters of a state transition model for recognition are estimated from learning data, and a recognition category (see FIG. 1). Each of the categories 1 to 6 shown) is stored in the recognition state transition model storage memory 7.

At the time of recognition, the likelihood of the model corresponding to each category stored in the recognition state transition model storage memory 7 by learning is calculated, and the category corresponding to the model having the maximum likelihood is determined. Is performed.

In the moving picture recognition method and the moving picture recognition and retrieval method according to the embodiment of the present invention, the processing up to quantization is the same at the time of learning and recognition.

A moving image recognition method and a moving image recognition search method according to an embodiment of the present invention will be described below with reference to FIG.

First, from the MPEG data 1 to be searched,
The feature extraction unit 2 extracts a DCT coefficient as a feature vector.

Here, the MEPG data 1 will be briefly described.

In the MPEG standardized coding system, DCT (Discrete Cosine Transform: Discrete Cosine Transform) for each block of 8 × 8 pixels in a frame.
rm) Data is compressed using coefficients and quantization, and using motion compensation vector information between frames.

Each frame of normal MPEG data 1 is composed of coded data of any one of three types of I picture, P picture and B picture.

It should be noted that the I picture is intra-frame coded,
A picture means forward inter-frame predictive coding, and a B picture means bidirectional inter-frame predictive coding.

In a normal sequence, one GOP (G
(loop of Picture) starts with an I picture, and arranges a P picture or a B picture at an appropriate interval according to the intensity of motion of an image, required image quality, and the like.

In the embodiment of the present invention, in order to use the DCT coefficients, all the frames are converted into image data which is I pictures and used.

The conversion from the MPEG data 1 composed of I, P, and B pictures to an I picture is performed by directly operating the encoded data as described in, for example, the following document (b). This is possible.

(B) Shin-Fu Chang and David G. Messe
rchmitt: “A New Approach to Decoding and Compositi
ng Motion-Compensated DCT-Based Images ”, Proceedin
gs ofICASSP'93 (1993). FIG. 2 is a diagram showing a schematic configuration of the MEPG data 1 and the DCT coefficient of the MEPG data 1.

As shown in FIG. 2, in the MPEG data 1, one frame of image data is divided into M × N blocks in which one block is composed of 8 × 8 pixels, and a DCT operation is performed for each block. As a result, DCT coefficients indicated by numerals 1 to 64 in the lowermost block in FIG. 2 are obtained.

In the embodiment of the present invention, an appropriate number of DCT coefficients of low frequency components (DCT coefficients in the area E1 shown in FIG. 3) are extracted from the DCT coefficients of the block of 8 × 8 pixels. Is performed on all the blocks, and a numerical sequence in which all the extracted DCT coefficients are arranged is set as a feature vector (f) of the frame.

Assuming that an image of 32 pixels × 32 pixels is used and i DCT coefficients are extracted from each block, since there are 16 blocks in total, the dimension of the feature vector in this case is 16i. .

Since one feature vector (f) is obtained from one frame of image data of the MPEG data 1, a feature vector sequence (F) is obtained from continuous frame (screen) image data for displaying a series of moving images. The feature vector sequence (F) is recorded in the feature storage memory 3.

The DCT coefficients used as the feature vector (f) are the DCT coefficients on the first line in the horizontal direction (in the region E2 shown in FIG. 3), in addition to the appropriate number of low frequency components. DCT coefficient), the DCT coefficient on the first line of the vertical method (the DCT coefficient in the area E3 shown in FIG. 3), or the DCT coefficient on the diagonal line including the DC component (the DCT coefficient in the area E4 shown in FIG. 3) ) May be used.

By using the DCT coefficient on the first line in the horizontal direction (the DCT coefficient in the area E2 shown in FIG. 3) as a feature vector, a specific pattern of a moving image is mainly dominated by horizontal movement. , The less DCT
It is possible to accurately extract a feature of a moving image using a coefficient.

Also, the DC on the first line of the vertical method
By using a T coefficient (a DCT coefficient in an area E3 shown in FIG. 3) as a feature vector, when a specific pattern of a moving image is mainly dominated by vertical motion, a moving image can be accurately formed with a small number of DCT coefficients. It is possible to extract image features.

Further, by using a DCT coefficient on a diagonal line including a DC component (a DCT coefficient in an area E4 shown in FIG. 3) as a feature vector, a specific pattern of a moving image can be used both in a horizontal direction and in a vertical direction. , It is possible to accurately extract the feature of the moving image with a small number of DCT coefficients.

Further, as the feature vector (f), D
It is also possible to use the CT coefficient and the motion compensation vector together, and thereby it is possible to extract the feature of the moving image in more detail.

This feature vector string (F) is
Is converted into a symbol sequence (O) by vector quantization and recorded in the symbol storage memory 5.

That is, each feature vector is converted into a symbol corresponding to the closest representative point vector among the representative points based on a list of representative points prepared for quantization in advance.

This representative point group is called a codebook, and this codebook is created by using the LBG algorithm described in the following document (c) using a part of the feature vectors extracted from each type of motion image. did.

(C) Y. Linde, A. Buzo, RMGray;
Algorithm for Vector Quantizer design ”, IEEE Tra
ns.Commin. vol.COM-28 (1980). This codebook was created using km-m
It may be created by an ean (k-mean) algorithm.

(D) XDHuang, Y. Ariki, MAJack; “Hi
dden Markov Model for Speech Recognition ”, Edinbu
rg Univ. Press (1990). Now, assuming that the codebook is represented by the following equation (1), the feature vector (f) is converted into a symbol (O _t ) shown in the following equation (2).

[0053]

## EQU1 ## C = c ₁ , c ₂ ,. . . . . c _N ... (1)

[0054]

[Number 2] _{O t = v k ····· (2} ) k = argmin j d (f, c j) where, d (x, y) is x, the processing of the distance y so far, the feature vector The sequence (F) is converted into a symbol sequence (O), and the symbol sequence (O) is learned and recognized by a state transition model.

The operation up to this point is the same for both recognition and learning.

As this state transition model, a Hidden Markov (hereinafter, referred to as HMM) model described in the above reference (d) or the following reference (e) is used.

(E) Seiichi Nakagawa; "Speech Recognition by Stochastic Model", IEICE (1990). At the time of learning, the parameters of the HMM model are estimated. At the time of recognition, only the number of categories to be recognized is prepared. H stored in the recognition state transition model storage memory 7
From each of the MM models, the likelihood calculating unit 8 calculates the probability of generating the feature vector sequence (F) to be recognized.

Hereinafter, the HMM model will be briefly described.

The HMM model is a stochastic state transition model, and can be regarded as modeling a source of a time series phenomenon.

FIG. 4 is a conceptual diagram showing the concept of the HMM model.

As shown in FIG. 4, the HMM model has a plurality of states (q _{1 to} q ₅ ), and the probability (a _ij ) of transition from each state (q _{1 to} q ₅ ) to another state. Is given.

State transitions occur stochastically as time advances, and symbols (O _{1 to} O _t ) stochastically change from each state.
Is output.

What can be observed is the output symbol sequence (O =
O ₁ , O ₂ ,. . . , O _t ), and the state cannot be directly observed.

This is the origin of the “hidden” Markov model.

In the application to the motion recognition, each posture during the motion corresponds to a state. Therefore, it is necessary to select an appropriate number of states according to the length and complexity of the motion to be recognized.

In the application to motion recognition, the state transition probability describes the time series pattern of the posture change itself and the change of expansion and contraction, and the symbol output probability describes the fluctuation of each posture and the fluctuation of the observation result of the posture. It can be interpreted that it corresponds to the part that does.

The HMM model is described by the following parameters.

[0068]

S = {s _t }: set of states. s _t is (not observable) t th state _{O = O 1, O 2,} ..., O T; observed symbol sequence (length _{T) A = {a ij |} a ij = Pr (s t + _{1 = j | s t = i} )}: state transition probability a _ij probability transition from state (si) to state (sj) is _{B = {b j (O t} ) | b j (O t) = Pr (O _t | s _t = j)｝: Symbol output probability b _j (k) is the probability of outputting a symbol (υ _k ) in state (sj) π = {π _i | π _i = Pr (s ₁ = i)}: Initial State Probability Next, a procedure for learning and recognizing a time-series pattern (symbol sequence (O)) using the HMM model will be described.

<< Procedure at the time of learning >> The model parameter estimating unit 6 generates a symbol sequence (O) for a symbol sequence (O) obtained from a plurality of learning data provided for each category. The parameters of the appropriate state transition model are estimated and stored in the recognition state transition model storage memory 7.

The recognition system based on the HMM model includes one HMM model for each category.

Assuming that the HMM model for each category to be recognized is λ _i (= {A _i , B _i , π _i }), this λ _i
Is performed using a learning pattern for each category.

Here, learning is nothing less than estimating the parameters of the HMM model that are likely to generate a learning pattern, that is, the state transition probability A _i , the symbol output probability B _i, and the initial state probability π _i .

To estimate the parameters of the HMM model from the learning pattern, the Baun-Welch algorithm described in the above-mentioned reference (d) or (e) is used.

More specifically, a procedure of repeatedly obtaining the parameters of the HMM model with a higher likelihood in order from a certain initial value until it can be considered that the parameters have been sufficiently converged from the likelihood value, change, etc., ie, a certain HMM model Is a procedure for repeatedly obtaining a model parameter having a higher likelihood based on the above parameter.

At each repetition, the convergence can be confirmed by confirming the likelihood value by the forward algorithm described in the above reference (d).

When expressed by a mathematical formula,

[0077]

(Equation 4)

[0078]

(Equation 5)

[0079]

(Equation 6)

[0080]

(Equation 7)

However, here,

[0082]

(Equation 8)

[0083]

(Equation 9)

The meaning of the above expressions is that expression (3) is a re-evaluation of a _ij under the HMM model λ.
Equation (4) is a re-evaluation of b _i (k) under the HMM model λ.

By the above-described procedure, the parameters of the state transition model for recognition corresponding to the learning data can be obtained.

The model for each category obtained in this way is used for recognition.

<< Procedure for Recognition >> The procedure for recognition is as follows.
This is done by calculating the likelihood of the model and selecting the maximum value.

For the pattern to be recognized, λ _i is a symbol sequence (O = O ₁ ,
O ₂ ,. . . . , O _t ), the probability (likelihood) Pr (O
| Λ _i ).

The calculation of the likelihood is performed recursively by the forward algorithm described in the above reference (d).
It can be obtained as follows.

That is, the probability Pr (O│λi) that a certain model λ = {A, B, π} outputs a symbol sequence (O = O ₁ , O ₂ ,..., O _t ) is

[0091]

(Equation 10)

Here, S _F is a set of final states,
α _T (i) is

[0093]

[Equation 11]

The HMM model λ generates a symbol sequence (O = O ₁ , O ₂ ,..., O _t ) and is in a state ( _St = i) at time t. Probability.

This is

[0096]

(Equation 12)

[0097] It is obtained by the recurrence formula.

The model with the maximum likelihood obtained in this way, that is, Pr (O |) obtained from Expressions (1) to (11)
λ _i ), the category (G _k ) for the maximum likelihood λ _i
(K = argmax _i Pr (O | λ _i )) is selected as a recognition result and stored in the recognition result memory 6.

At the time of searching, the MEP to be searched is
A search is performed by scanning the MEPG data 1 to determine which part of the G data 1 has the maximum likelihood for the HMM model corresponding to the search target.

In this case, in order to efficiently obtain the maximum likelihood portion in the MEPG data 1, it is possible to use the HMM spotting algorithm described in the above-mentioned document (e).

As is clear from the above processing flow, H
Recognition by the MM model is performed by maximum likelihood estimation.
The learning is realized in the form of estimating the parameters of the HMM model from the training data.

Since the likelihood calculation is performed from the entire symbol sequence, if a symbol string pattern specific to the category appears, there is an advantage that it is resistant to some movement, expansion and contraction in the time axis direction.

Further, the likelihood up to each time point of the time series pattern of the moving image is obtained, and a threshold processing or the like is performed on the likelihood, whereby a specific time series pattern can be searched.

Next, as an example of an experimental result based on the embodiment of the present invention, a description will be given of an experimental result of confirming two persons' movements on a tennis movement image.

[Experiment 1] In the embodiment of the present invention,
One example of a photograph of the tennis operation image used in Experiment 1 is shown in FIG.

As shown in the lower part of FIG. 5, a person region is extracted from the tennis operation image shown in the upper part of FIG. 5 by background subtraction, and MEPG data created based on an image example in which the person region is extracted is extracted. The recognition performance was evaluated when the DCT count was used as a feature quantity, as a recognition target.

The recognition performance is as follows. The DCT coefficients for each block (8 × 8 pixels) are extracted one by one from the low-order component, that is, 1, 3, 6, 10, 15, 21, and 28 DCT coefficients are extracted.
Each experiment was performed to determine the recognition rate.

The DCT coefficient for each block is 1
In the case of, there is only a DC component.

The image size was of two types, 16 × 16 pixels (1 × 1 block in macroblock units) and 32 × 32 pixels (2 × 2 blocks in macroblock units).

The size of the codebook for quantization is 48 in total for each class size of 8 and 6 classes.
The HMM model is created by the BG algorithm, and has 12 states and 48 symbols.

FIG. 6 is a photograph showing a tennis operation image targeted in Experiment 1 of the embodiment of the present invention.

As shown in FIG. 6, the target tennis operation includes back-hand volley, back-volley,
Back hand stroke (back-stroke),
There are six categories: fore-hand volley, fore-stroke, smash, and service.

For each of the six categories of operations, 1
The motion image data of 0 trials was collected, five trials among them were used as learning data, parameters of the HMM model were estimated, and a recognition experiment was performed using the remaining five trials as test data.

In this case, an experiment was conducted by changing the selection method for selecting 5 trials out of 10 trials into 10 different ones.

Therefore, the recognition rate is 5 × 10 × 6 = 3
It is evaluated based on how many of the 00 recognition experiments were successful.

Tables 1 and 2 show the results of this recognition experiment.

[0117]

[Table 1]

[0118]

[Table 2]

As can be understood from Table 1 and Display 2, the recognition rate is greatly improved by increasing the DCT coefficient used as the feature quantity, and the DCT coefficient of a relatively low frequency component is used for image recognition of human motion. It is found that it is effective as a feature value for

Further, even when the target image is relatively small,
By using DCT coefficients up to high frequency components, 98
%, And it was found that a recognition rate comparable to that of a large image could be realized.

[Experiment 2] In the embodiment of the present invention,
For a series of moving image data including multiple types of operations,
An experiment on application to moving image retrieval was performed.

It was examined whether or not an operation can be searched by selecting an HMM model having the maximum likelihood up to each time point based on the learned HMM models of each operation category.

The screen size used was 32 × 32 pixels, and the DCT coefficient was set to 6 for each block as a feature value.

FIG. 7 is a graph showing the experimental result of Experiment 2 in the embodiment of the present invention.

FIG. 7 is a graph in which the log likelihood of the HMM model of each of the six categories is plotted based on the observations up to each time point.

Therefore, the likelihood is expected to be maximum at the end of the operation.

From the graph shown in FIG. 7, the HM of each target operation is shown.
It can be confirmed that the M models have the maximum likelihood in order,
It can be understood that the operation section can be cut out by the threshold processing.

As a result, it is possible to search for a specific operation pattern in the continuous moving image data.

In the description of the embodiment of the present invention, the MPEG data encoded by the standard encoding method such as MEPG and MEPG2 is used. However, the present invention is not limited to this. -JPE
It goes without saying that data encoded by a standard encoding method such as G can be used.

Although the present invention has been specifically described based on the embodiments of the present invention, the present invention is not limited to the embodiments of the present invention, and various modifications may be made without departing from the gist of the present invention. It goes without saying that it can be done.

[0131]

The effects obtained by the representative inventions among the inventions disclosed in the present application will be briefly described as follows.

(1) According to the present invention, DC is used as the feature value.
Since the T coefficient or the DCT coefficient and the motion compensation vector are used, a specific moving image pattern is directly recognized and searched from a small amount of moving image data compressed by a standard coding method such as MEPG or MEPG2. It is possible to do.

As a result, the processing time of data processing can be reduced.

(2) According to the present invention, the likelihood calculation is performed from the entire feature vector sequence. Therefore, if a feature vector sequence pattern specific to a category appears, a slight movement, expansion and contraction in the time axis direction is performed. Even if there is, it becomes possible to recognize and search a specific moving image pattern with high accuracy.

(3) According to the present invention, by using the DCT coefficient used as the feature value up to the high frequency component,
The recognition rate can be greatly improved, and even when the target image is relatively small, the recognition rate can be improved by using DCT coefficients used as feature amounts up to high-frequency components. is there.

(4) According to the present invention, the present invention can be widely applied to monitoring of suspicious behavior in a bank or a store, clipping of a desired operation portion from a moving image such as sports, and the like.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a schematic configuration of a moving image recognition / search apparatus to which a moving image recognition method and a moving image recognition / search method according to an embodiment of the present invention are applied.

FIG. 2 shows MEPG data 1 and D of MEPG data 1
FIG. 3 is a diagram illustrating a schematic configuration of a CT coefficient.

FIG. 3 is a diagram for explaining a method of extracting DCT coefficients according to the embodiment of the present invention.

FIG. 4 is a conceptual diagram showing the concept of an HMM model (Hidden Markov).

FIG. 5 is a halftone image displayed on a display showing an example of a tennis operation image used in Experiment 1 in the embodiment of the present invention.

FIG. 6 is a halftone image displayed on a display showing a tennis operation image targeted in Experiment 1 of the embodiment of the present invention.

FIG. 7 is a graph showing experimental results of Experiment 2 according to the embodiment of the present invention.

[Explanation of symbols]

2 ... Feature extraction unit, 3 ... Feature storage memory, 4 ... Quantization unit, 5 ... Symbol string storage memory, 6 ... Model parameter estimation unit, 7 ... Recognition state transition model storage memory, 8 ...
Likelihood calculating section 9, memory for recognition result.

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] June 27, 1996

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Claims

[Correction method] Change

[Correction contents]

[Claims]

Claims (16)

[Claims] In a moving image recognition method for recognizing a moving image pattern of a series of moving images, image data of each screen displaying a series of moving images is divided into M × N blocks, and a DCT coefficient of each block is extracted. , Extracting at least one of the DCT coefficients as a feature vector of each screen, and a time-series feature vector sequence composed of feature vectors of each screen displaying a specific moving image pattern. Learning a state transition model for each of a plurality of specific moving image patterns serving as a recognition key, and a feature vector extracted from image data of each screen displaying a series of moving images to be recognized. Recognize a moving image pattern of a state transition model in which the likelihood of a time-series feature vector sequence with respect to a plurality of state transition models obtained by the learning is maximized. Video recognition method characterized by comprising the step of outputting a result. 2. The image data of each screen displaying a series of moving images to be recognized is compressed by a standard encoding method, and each of the image data compressed by the standard encoding method is used as a feature vector of each screen. 2. The method according to claim 1, wherein a part of DCT coefficients included in the image data of the screen is used.
The moving image recognition method described in 1. 3. A feature vector of each screen is DC
The moving image recognition method according to claim 1, wherein a motion vector is used together with the T coefficient. 4. The image data of each screen displaying a series of moving images to be recognized is compressed by a standard encoding method, and each of the image data compressed by the standard encoding method is used as a feature vector of each screen. 4. The moving image recognition method according to claim 3, wherein a part of DCT coefficients included in the image data of the screen and a motion compensation vector are used. 5. The method according to claim 1, wherein 3 to 21 DCT coefficients of low frequency components among DCT coefficients included in image data of each screen compressed by the standard encoding method are used as feature vectors. The image recognition method according to claim 2 or 4, wherein 6. A DCT coefficient on a first horizontal line among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. The image recognition method according to claim 2 or 4, wherein 7. A DCT coefficient on a first line of a vertical method among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. The image recognition method according to claim 2 or 4, wherein 8. A DCT coefficient on a diagonal line including a DC component among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. The image recognition method according to claim 2 or 4. 9. A moving image recognition and retrieval method for extracting a time region including a specific moving image pattern from a series of moving images, wherein image data of each screen displaying a series of moving images is M × N blocks. Extracting a DCT coefficient of each block, extracting at least one of the DCT coefficients as a feature vector of each screen, and a feature vector of each screen displaying a specific moving image pattern serving as a search key. Learning a stochastic state transition model using a time-series feature vector sequence composed of: and a feature vector extracted from image data of each screen displaying a series of moving images to be searched. Outputting, as a search result, a time domain having a high likelihood for the state transition model obtained by the learning in the time-series feature vector sequence. Video recognition retrieval method characterized by Bei. 10. The image data of each screen displaying a series of moving images to be searched is compressed by a standard encoding method, and each image compressed by the standard encoding method is used as a feature vector of each screen. The moving image recognition / retrieval method according to claim 9, wherein a part of DCT coefficients included in the image data of the screen is used. 11. A feature vector of each screen is D
The moving image recognition search method according to claim 9, wherein a motion vector is used together with the CT coefficient. 12. The image data of each screen displaying the series of moving images to be searched is compressed by a standard encoding method, and each image compressed by the standard encoding method as a feature vector of each screen. The moving image recognition search method according to claim 11, wherein a part of DCT coefficients included in the image data of the screen and a motion compensation vector are used. 13. A DCT coefficient of 3 to 21 low frequency components among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. Claim 10 or Claim 1
2. The image recognition search method described in 2. 14. A DCT coefficient on a first horizontal line, among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. Claim 10 or Claim 1
2. The image recognition search method described in 2. 15. A DCT coefficient on a first line of a vertical method, among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. Claim 10 or Claim 1
2. The image recognition search method described in 2. 16. A DCT coefficient on a diagonal line including a DC component among DCT coefficients included in image data of each screen compressed by the standard encoding method, is used as a feature vector. An image recognition and retrieval method according to claim 10 or 12.