JP2005094755A - Method for processing a plurality of videos - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for processing a plurality of videos, which minimizes a combined distortion and satisfies a combined frame rate constraint, for all videos. <P>SOLUTION: Intra-, or inter-compressed frames of each compressed video are acquired at a fixed sampling rate. Joint analysis is applied concurrently and in parallel to the compressed videos to determine a variable and non-uniform temporal sampling rate for each compressed video so that a combined distortion is minimized and a combined frame rate constraint is satisfied. Each compressed video is then sampled at an associated variable and non-uniform temporal sampling rate to produce output compressed videos having variable temporal resolutions. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates generally to compressed video sampling, and more particularly to compressed video sampling subject to resource constraints.

  Multiple video encoding has been considered primarily in two areas: transmission and recording. Multiple video transmissions are primarily applied to broadcast and transmission applications, and multiple video recordings are typically applied to surveillance applications. Although video recording is common in many consumer electronics products, such video recorders typically handle single video encoding rather than multiple simultaneous videos.

  In television broadcast applications, it is common practice to encode multiple videos so that the encoded video bitstream can be broadcast over a single channel with a fixed bandwidth. For example, when N programs and a general 45 Mbps channel bandwidth are generally given to the satellite link, the problem is that the N programs having the maximum quality are encoded into a single channel. To multiplex on top. Since the bandwidth is fixed and the complexity of each program is different, each program is encoded at a variable bit rate (VBR). In this way, a substantially constant distortion can be maintained throughout the entire program. Thus, more complex portions of some videos can be allocated more bits by reducing the number of bits allocated to less complex portions of other videos that are encoded simultaneously.

  The encoding process as described above is called statistical multiplexing. Techniques associated with this process include Haskell's Multiplexing of Variable Rate Encoded Streams (IEEE Transactions on Circuits and Systems for Video Technology, 1994), Wang et al. "Joint Rate Control For Multi-Program Video Coding" (IEEE Transactions on Consumer Electronics, Vol. 42, No. 3, August 1996), US patent issued July 18, 2000 to Yang No. 6,091,455, “Statistical Multiplexer for Recording Video”, US Pat. No. 6,195,388, issued February 27, 2001 to Choi et al. No. "Apparatus and Method for Encoding Multiple Video Programs" and references contained herein. It has been.

  A similar policy describes the encoding of multiple objects in a fixed bandwidth constrained scene in US Pat. No. 5,969,764 issued October 19, 1999 by Sun and Vetro. Yes. This method assigns a bit to each object. US patent application Ser. No. 09 / 579,889, filed May 26, 2000 by Vetro et al., “Method for encoding and transcoding multiple video. objects with variable temporal resolution) describes a method for satisfying the full bandwidth constraint with each object having a different temporal rate in the scene. This method therefore minimizes the component artifacts that occur when encoding multiple objects in a scene at different time rates.

  The prior art method described above encodes multiple videos or objects subject to the full bandwidth constraints of one transmission channel.

  In the prior art, resource constraints other than bandwidth have been considered in processing multiple videos. For example, US Pat. No. 6,052,384 issued to Huang et al. On August 18, 2000, “Using a Receiver Model to Multiplex with Timing Constraints Using Receiver Model”. Refer to Variable Bit-Rate Streams Having Timing Constraints). This patent describes a technique for determining the output bit rate of each stream so that neither the queue of bit streams in the multiplexer nor the buffer of the decoder overflows or underflows. These rates are determined using timing information read from the bitstream and a receiver model that takes into account the operation of the decoder buffer.

  Multiple video transcoding considering both delay and processing timing constraints is described in US Pat. No. 6,275,536, issued August 14, 2001 to Chen et al. “Using multiple programmable processors. Implementation architectures of a multi-channel MPEG video transcoder using multiple programmable processors ”. The input bitstream is first partitioned into processing units. In one architecture, the processing unit is divided into different sub-bitstreams, each with its own queue, and each substream is processed in a corresponding branch. In the second architecture, processing units are assigned to any available processor from a common queue. Independent processing units are processed simultaneously according to the queuing system model, minimizing the average processing time. In contrast to the first architecture, which is parallel processing of multiple branches, the second architecture is a single branch of multiple processing.

  Similar to Chen et al., US Pat. No. 6,008,848, “Video Compression Using Multiple Computing Agents” issued December 28, 2001 to Tiwari et al. A system and method using multiple processors is described. In contrast, Tiwari describes a technique that applies to video encoding and achieves encoding using coarse-grain parallelism performed by multiple processors or compression agents.

  FIG. 1 shows a general system model for encoding multiple videos for surveillance applications. The camera 101 acquires the video 102 for the video recorder 110. Usually, the recorder 110 compresses the video. The compressed video is then stored in memory 120. The video player 130 can then play the stored video.

  FIG. 2 shows details of the recorder 200. The acquired video 102 is sent to the high speed switch 210. This switch samples the analog video signal in time. The samples are supplied to the decoder 220 and the digitized image is encoded by the still image encoder 230 to produce a compressed image. The memory controller 240 writes the compressed image into the allocated space in the memory 120. The stored video can be played back later.

  The main problem with the recorder of FIG. 2 is that still images are encoded. Video temporal redundancy is not used. As a result, the memory 120 must be very large when storing weeks or months of surveillance video. It should be noted that large amounts of surveillance video, especially those taken at night, are of a completely static scene. Significant events are rare.

  FIG. 3 shows a clear solution to the above problem. In this scheme, there is one encoded channel for each video. In the encoded channel, the video is first NTSC decoded 220. For predictive coding such as MPEG used in a video coder, a frame memory 310 is maintained for each video encoder to store a reference picture. Next, input frames from different cameras 101 are encoded separately, and the result is written into the memory 120 using the memory controller 240. The time rate of the input frame can be controlled by sampling uniformly in the fixed period T301. By this sampling, the memory 120 can be used more effectively. The main drawback of this scheme is that video encoder 320 is not fully utilized despite being designed to process full-rate video. Multiple decoders and encoders also increase the cost of the system.

  U.S. Pat. No. 6,314,137 issued to Ono et al. On November 6, 2001 "Video data compression system, video recording / playback system, and video data compression system," "video recording / playback system, and video data compression encoding method" describes a system and method as shown in FIG. 4 that overcomes the above drawbacks. There, a single video encoder 420 is used to encode all videos. Digitized video frames from each camera input 101 are subsampled in time period T and buffered in a respective frame memory 310. In order to achieve predictive coding with a single encoder 420, a series of input video frames corresponding to one camera input is fed to the video encoder, and predictive coding from frames of the same camera input is continuously performed. Like that. An MPEG encoded picture group (GOP) structure allows such independent units to be formed. As such, the memory controller 240 becomes GOP-select and the GOP from each camera input is temporally multiplexed into the encoder according to the controller 410. In that scheme, a single bitstream of all camera inputs is generated. A camera identifier 401 is multiplexed into the encoded bitstream to identify the portion of the video corresponding to a given camera.

  In the above solution, data worth one GOP needs to be stored in each of the frame memories, and the requirements are far greater than the system of FIG. 2 where only one or two reference pictures are required at most. growing. Thus, although the encoding hardware is greatly reduced, the memory requirements are still large and expensive. This drawback cannot be overcome by simply sampling the input video more aggressively. This reduces the temporal resolution of the video, but still needs to buffer the same GOP data. Only shorter GOP periods reduce memory requirements, but this implies more frequent intraframe coding, which means less efficient coding. In the most extreme case, one GOP period degenerates to the still picture coding system shown in FIG.

  High memory requirements are just one drawback of the system of FIG. This problem gets worse in proportion to the scaling of the system for a larger number of videos.

  In large systems, compressed video is often used to reduce bandwidth and storage. Accordingly, it is desirable to provide a system and method for subsampling multiple compressed videos simultaneously.

  One object of the present invention is to provide a low-cost, compression-efficient, scalable, and flexible system for storing resources, especially memory, encoding hardware, and multiple delay-constrained videos. It is.

  It is a further object of the present invention to provide a method for achieving variable and non-uniform temporal resolution between multiple uncorrelated compressed videos.

  The method obtains a compressed input video. Acquire compressed frames of each input video at a fixed sampling rate.

  A composite analysis is applied to the compressed video simultaneously and concurrently to determine a variable and non-uniform temporal sampling rate for each compressed video to minimize the total distortion and satisfy the total frame rate constraint.

  Each compressed video is then sampled at an associated variable non-uniform time sampling rate to produce a compressed output video having a variable temporal resolution.

  FIG. 5 shows individual frames 501 of multiple input videos (videos 1-4) 510 acquired at the same time with a fixed sampling rate by the camera. Over the composite analysis 600 according to the present invention, the frames 502 of the plurality of output videos 520 have a variable and non-uniform temporal sampling rate. The purpose of composite analysis is to minimize the total distortion of all output videos while satisfying the total frame rate constraint.

  Factors that affect the rate and presence of a particular output frame 502 at a given time include compression efficiency, resource constraints, and significant event detection. The compression efficiency may be related to the type of encoder used (eg MPEG-2 or MPEG-4). Resource constraints take into account memory, processing speed, and coding rate. Significant events mean short-term local events that are different from the longer part of the background scene.

  For example, in a surveillance video, a person who enters an entrance scene that is normally stationary and nobody is considered significant. US patent application Ser. No. 10 / 610,467, filed June 30, 2003 by Divakaran et al., “Method for Detecting Short Term Unusual Events in Videos” (book by reference (Incorporated herein). Similarly, in the case of traffic video, a change from smooth motion to no motion indicates an accident.

System Architecture FIG. 6A shows a block diagram of a video encoding system for multiple videos according to the present invention. Video 510 from multiple cameras 601 is first NTSC decoded 602. The digitized video is then input to composite analysis circuit 600 to determine a temporal sampling rate 604 for each video.

  The temporal sampling rate 604 is determined according to a method that takes into account the presence of significant events in the video, compression efficiency and resource constraints. The method and constraints are described in further detail below.

The variable non-uniform time sampling rate of each video is communicated to the controller 610 which triggers the sampling circuit 604. This circuit can non-uniformly sample each video input frame at time t. The sampling times T ₁ (t), T ₂ (t), T ₃ (t), and T ₄ (t) do not necessarily operate at the same rate. These sampling times determine the total frame rate across all videos.

The sampled frame of video 520 is then stored in frame buffer 620. Controller 610 selects the input to be encoded 640 630, and frames are read from the respective frame buffers and communicated to video encoder 640. From the controller, the video encoder can receive the following information 611: the encoding parameters used to encode the video derived by the composite analysis 600, as well as identification information that can be encoded into the bitstream itself. . Identification, ISO / IEC 14496-2 "encoding of audio-visual objects - Part 2: Visual (Coding of audio-visual objects - Part 2: Visual) " MPEG-4 video objects that are described in (2 ^nd Edition, 2001) It can be encoded directly into the bitstream using plain coding syntax.

  The system generates a unique bitstream 650 for each video input, which is then stored in permanent memory 660 for later playback.

  In this system, a minimum number of reference frames, for example one or two frames, is stored for each video. The reason for this low memory requirement is due to the nature of the video encoding process. In contrast to the prior art system for encoding data GOP by Ono et al. Shown in FIG. 4, the system according to the present invention encodes a frame of data. This is made possible by a video encoder 640 that can mark different video inputs and generate separate output bitstreams 650. Controller 610 may select 630 the correct reference frame to use for motion compensation of the current video being encoded. None of these features are part of the system described by Ono et al. These features can provide a scalable system that can encode multiple videos simultaneously.

  The system of FIG. 6B is a slightly different configuration of the system of FIG. 6A. The main difference is a single shared frame buffer 621.

  FIG. 7 illustrates an extension of the core architecture of FIGS. 6A and 6B. The bitstream stored in permanent memory 660 can be transferred to archival memory 703 for long term storage. In this case, the transcoder 701 can be used to further change the bit rate, spatial resolution and / or temporal resolution of the stored bitstream. An analysis circuit 702 is used to configure and set the transcoding parameters. Use any conventional transcoding technique as described by Vetro et al. In "An overview of video transcoding architectures and techniques" (IEEE Signal Processing Magazine, March 2003). Can do.

  Although the above architecture has been shown for four input videos and one video encoder, this design makes the system more capable of input video and video encoders (eg, 16 inputs and 2 video encoders, or 64 inputs). And 4 video encoders) can be easily scaled up.

Composite Analysis Factors that affect the operation of composite analysis 600 include the total distortion caused by skipping frames, the number of encoders incorporated as part of the system, the number of input videos, and the detection of significant events in the scene. And classification, user priority for specific camera inputs, minimum temporal encoding rate for each video, acceptable delay and storage required for analysis.

  The total distortion caused by the skipped frames leads to a complex analysis operation. US patent application Ser. No. 09 / 835,650 filed Apr. 16, 2001 by Vetro et al. “Estimating Total Average Distortion in a Video with Variable Frameskip” , The distortion of the skipped frame is determined from two parts, namely the coding error due to the quantization of the last reference frame and the frame interpolation error due to the change of the video signal between the two instants. The method only considers single video distortion. Here, the total total distortion of all output videos is minimized.

  In the following, distortion caused by coding error is called spatial distortion, and distortion caused by interpolation error is called time distortion. Since video recording systems typically encode all frames with the same quality, the problem of estimating distortion can eliminate the spatial distortion of the frame and focus only on temporal distortion. It is emphasized that this is very different from the formulation that has been studied in the prior art.

  The number of encoders incorporated as part of the system determines the total frame rate, ie the maximum number of frames that can be encoded per unit time. For example, if a single encoder can encode 30 frames per second and four encoders are incorporated as part of the system, the total frame rate is 120 frames / second. For example, the total frame rate can be assigned to the video as follows: If the total total frame rate is 120 fps, each of the 60 surveillance videos of the still scene can be sampled at 1 frame / second, and the two videos with activity can be sampled at 30 frames / second.

In order to formulate the problem based on this constraint, the value N _cap (T) indicates the total number of frames per time unit T. Depending on the number of input videos to the system and the frame rate of those videos, an average temporal sampling rate across all videos can be determined.

The objective function of complex analysis can be described as follows.
Min Σ _i D _i (1) such that N _coded (T) ≦ N _cap (T)
Here, D _i is the time distortion of video i, and N _coded (T) is the total number of frames encoded per unit time T. To maximize the use of coding capability, N _coded (T) = N _cap (T).

The time distortion from (1) in the time interval [t, t + τ] is expressed as follows.
D _i [t, t + τ] = w _i ΣD _{skipped_frames} [t, t + τ] (2)

The above formula sums the skipped frame distortions in a given time interval and considers the multiplicative weighting factor w _i for each video. The factors that affect the purpose of weighting and the weight value will be described later.

  As mentioned earlier, this formulation is not only new in that it only accounts for time distortion (account for), but it also has rate constraints on individual videos that are very typical of prior art formulations. It is also new in that it is not. Instead, there are constraints on the total frame rate of all videos. This is the second novel aspect of this formulation, and to the best of the inventor's knowledge, this function is not present in prior art systems.

  Detection of a significant event in the scene may record a binary action that records whether an event has been detected and stops recording if there is no significant event. This is a very simple operation. However, this is not a very good strategy as it can miss some significant events and record some insignificant events unless the detector is accurate and noise proof.

Instead, it is preferable to include a weighting factor as part of the objective function to be optimized. Assuming that the objective function is based on minimizing the total distortion, use a weight of 1 if there is no significant event, and a larger weight if there is an event (eg, w _i = w _max ). Is used. The value of w _max can be adjusted. A large value encourages composite analysis to encode more frames in which events in the video are detected to minimize total distortion. If no event is detected and w _i = 1, the default mode is to find a frame whose objective function is purely dependent on video distortion. Assuming that the process used to detect events has the ability to distinguish the significance of different events, the weights w _i can take values in the range [1, w _max ].

A further use of the weight w _i in equation (1) is to express the priority of a particular video. For example, if there video is of critical areas (e.g. entry), it is possible to set the W _i greater than 1. This gives a bias to always encode more frames from this input video, as well as event detection. Note that this priority can also act on the weighting that occurs as a result of event detection. In other words, the priority can add a certain value to the weight output from the event detection process.

  Instead of adding priority to a video, weights can also be used to reduce the importance of the relative number of frames that are encoded for a particular video. This can be achieved using a weight less than one. The reason for this is that distortion accumulation caused by skipping frames from the input video is slowed down and fewer frames are encoded.

  However, there may be a minimum time coding rate set by the system. This setting causes a frame of a specific input to be encoded when a predetermined period has elapsed since the last frame was encoded. This minimum time rate is enforced whenever the weight for a particular input is set to zero.

  The last factor that affects the operation of the composite analysis 600 is the acceptable delay (τ). If a larger delay is allowed, the problem is solved taking into account a larger time interval. Although the distortion may be reduced if the time interval is increased, there are two drawbacks to providing this large time interval. The first drawback is the size of the memory. A larger time window implies that more storage capacity is needed to analyze incoming frames. The second drawback is calculation. The larger the time window, the more possible solutions that need to be evaluated. This is a normal technical trade-off that must be made during system design. However, note the flexibility provided by the system described by the present invention.

  Decomposing the problem into a fixed time interval T, where N represents the total number of possible frames encoded in this time interval for all input videos, the optimal solution to the problem is the system But

  Request to evaluate each possible solution.

  Choose the solution with the minimum total distortion across all videos for a given time interval. It should be noted that each possible solution is a combination of distortion calculations between two frames. If M is the total number of input videos and K is the total number of frames in each video period T, then the total number of distinct distortions between frame pairs that must be calculated is

  It is.

  In the embodiment shown in FIG. 8, the parallel input video 801 for the composite analysis 800 is a compressed video of individually compressed frames, ie, intra-frame frames. Examples of videos that are part of the compression category of intra-coded frames include JPEG, JPEG2000, MPEG-1 / 2 I-frame, MPEG-4 I-VOP (video object plane) or any other frame There are videos in which frames are compressed using inner-coded frame compression techniques.

  In networked surveillance systems, the camera input is compressed in this way to reduce bandwidth. At the same time, edit and review operations can still access each frame individually. Using compression of intra-frame encoded frames in this way is quite common in broadcast studios, and video is usually sent internally to different editing stations prior to broadcast, with some modifications. .

  In the present embodiment, the composite analysis 800 still operates according to the objective function and constraints given by equation (1). However, in this case, the time distortion is obtained directly from the compressed area information of the intra-frame encoded frame. This compressed domain information may include discrete cosine transform (DCT) coefficients. Since frame differences and correlations between frames can be achieved very efficiently from the compressed domain, there is no need to perform analysis from frames that have been completely decoded and reconstructed. Instead, analysis 800 is performed on the partially decoded frame.

The output of controller 610 operates in the same manner as above, and sampling circuit 604 is triggered to sample the compressed frame of each video input at time t non-uniformly. As an advantage, the sampling rates T ₁ (t), T ₂ (t), T ₃ (t) and T ₄ (t) may be different. After obtaining the sampling rate, the compressed video of the intra-frame encoded frame is sampled as appropriate, and the sampled frame is recorded directly in the permanent memory 660.

  In the embodiment shown in FIG. 9, the input video 901 for the composite analysis 900 is a compressed video of mutually compressed frames, ie inter-frame frames. Videos that fall into this category include MPEG-1 / 2 P / B frames, MPEG-4 P / B-VOPs, or any other that predicts information in the current frame from another past or future reference frame. There is a video in which the frame is compressed using the above technique.

  In networked camera systems, video is compressed much more than is possible with compression of intra-frame encoded frames. This is especially true for surveillance systems with very high time redundancy in scenes with little motion. One major advantage of this embodiment is that multiple cameras can share the same network and bandwidth. In this case, a part of the frame serves as a reference frame for predicting other frames. Care must be taken to process these frames correctly.

  In this embodiment, the composite analysis 900 estimates temporal distortion from the compressed video. One means of achieving this is US patent application Ser. No. 09 / 835,650, filed Apr. 16, 2001, entitled “Estimating Total Average Distortion” in videos with variable frame skipping. in a Video with Variable Frameskip) ”.

The time distortion E {Δ ² z _{i, k} } between the frame i and the frame k is estimated by the following equation.

Here, (σ ² _xi , σ ² _yi ) represents the variance of the spatial gradient of x and y in frame i, and (σ ² _{Δxi, k} , σ ² _{Δyi, k} ) represents two frames in the x and y directions. Represents the variance of the motion vectors between them.

  Here, Vetro et al. Recognize the practical difficulty in applying the technique to video coding, and the goal is to estimate the distortion of different frame skip factors. The objective is to select a frame skip rate that minimizes distortion. However, since many possible frame motion vectors are not yet available at the time of encoding, motion vector estimation is a difficult calculation.

  However, in the embodiment of FIG. 9, the motion vectors are obtained directly from the compressed video 901. The spatial gradient of the reference frame, such as texture and edge information, can be determined directly from the DCT coefficients. Wang et al. “Survey of compressed-domain features used in audio-visual indexing and analysis” (Journal of Visual Communication and Image Representation, Volume 14, Issue 2, pp. 150-183, June 2003).

  According to the present invention, any other means for measuring the estimated distortion as well as the temporal distortion obtained by equation (3) can be used. The object of the present invention is to minimize the total distortion in all videos and satisfy the total frame rate constraint.

  Based on the result of the composite analysis 900, information about the frames that are missing from the input video is transmitted to the controller 910. The controller signals this information to a time rate transcoder 920, one for each video.

  Note that the complexity of the transcoding operation is significantly lower than the encoding operation. During encoding, a motion vector must be determined. This is not true for transcoding where motion vectors are derived from the input stream. The system can therefore process many videos simultaneously.

  Various transcoding techniques that reduce the time rate are described by Vetro et al., “An Overview of Transcoding Architectures and Techniques” (IEEE Signal Processing Magazine, vol. 20, no. 2, pp. 18 -29, March 2003).

  After reducing the time rate of each video accordingly, the compressed output stream 902 is recorded directly in permanent memory 660.

  It should be understood that the input video may include both compressed and uncompressed video simultaneously.

  Although the invention has been described by way of examples of preferred embodiments, it is understood that various other adaptations and modifications can be made within the spirit and scope of the invention. Accordingly, the purpose of the appended claims is to cover all modifications and variations that fall within the true spirit and scope of the invention.

1 is a block diagram of a conventional video recording and playback system. FIG. 1 is a block diagram of a plurality of video still image encoding systems according to the prior art. FIG. 1 is a block diagram of a video encoding system for multiple videos using multiple encoders according to the prior art. FIG. 1 is a block diagram of a video encoding system for multiple videos using a single encoder according to the prior art. FIG. FIG. 6 is a block diagram of simultaneous full frame rate input video that is analyzed to produce output video with non-uniform temporal resolution according to the present invention. FIG. 2 illustrates a video encoding system for multiple videos using a single encoder and separate memory according to the present invention. FIG. 2 illustrates a video encoding system for multiple videos using a single encoder and shared memory according to the present invention. 1 is a block diagram of a video encoding system for multiple videos using a single encoder and transcoder according to the present invention. FIG. 1 is a block diagram of a video processing system that operates on intra-coded frames of compressed video. FIG. 1 is a block diagram of a video processing system that operates on inter-coded frames of compressed video. FIG.

Claims (19)

Acquiring multiple compressed videos in parallel, acquiring compressed frames for each input video at a fixed sampling rate;
A composite analysis is applied simultaneously and concurrently to the plurality of compressed videos to determine a variable and non-uniform temporal sampling rate for each of the compressed videos, and the sum while minimizing the total distortion for the plurality of compressed videos Satisfy the frame rate constraints of
Processing a plurality of videos including sampling each compressed frame of the compressed video at the associated variable non-uniform time sampling rate to generate a plurality of compressed output videos having a variable temporal resolution. Method. The method of processing a plurality of videos of claim 1, further comprising: storing the plurality of compressed output videos in a permanent memory. The method of processing a plurality of videos according to claim 1, wherein the compressed frame is an intra-frame encoded frame. The method of processing a plurality of videos according to claim 3, wherein the compressed video is a JPEG video. The method of processing a plurality of videos according to claim 3, wherein the compressed video is an MPEG video. The method of processing a plurality of videos of claim 1, further comprising: acquiring the plurality of compressed videos using a plurality of surveillance cameras. The method of processing a plurality of videos of claim 1, further comprising: acquiring the plurality of compressed videos using a plurality of broadcast studio cameras. The method of processing a plurality of videos according to claim 3, wherein the total distortion includes a temporal distortion. The method for processing a plurality of videos according to claim 8, wherein the time distortion is obtained from compressed region information of the intra-frame encoded frame. The method of processing a plurality of videos according to claim 9, wherein the compressed region information includes DCT coefficients. The method of processing a plurality of videos of claim 1, further comprising: partially decoding the plurality of compressed videos before applying the composite analysis. The method of processing a plurality of videos according to claim 1, wherein the compressed frames are inter-frame encoded frames. The method of processing a plurality of videos according to claim 12, wherein the time distortion is directly determined from a motion vector of the inter-frame encoded frame. The method for processing a plurality of videos according to claim 12, wherein the compressed frame is an MPEG-1 / 2 P / B frame. The method of processing a plurality of videos according to claim 12, wherein the compressed frame is an MPEG-4 P / B video object plane. The method of processing a plurality of videos according to claim 12, wherein the total distortion includes a temporal distortion. The time distortion E {Δ ² z _{i, k} } between frame i and frame k is estimated by the following equation:

Here, (σ ² _xi , σ ² _yi ) represents the variance of the spatial gradient of x and y in the frame i, and (σ ² _{Δxi, k} , σ ² _{Δyi, k} ) represents the frame in the x and y directions. The method of processing a plurality of videos according to claim 16, which represents a variance of motion vectors between i and the frame k. The method of processing a plurality of videos according to claim 17, wherein the spatial gradient is directly determined from DCT coefficients of the frame. The method of processing a plurality of videos of claim 12, further comprising transcoding the compressed output video.

Images