WO1991004636A2

WO1991004636A2 - Improvements in or relating to motion adaptive video signal converters

Info

Publication number: WO1991004636A2
Application number: PCT/GB1990/001405
Authority: WO
Inventors: Jonathan George William Pledge; Jeffrey John Gledhill; Peter Anthony Avon
Original assignee: Independent Broadcasting Authority
Priority date: 1989-09-11
Filing date: 1990-09-11
Publication date: 1991-04-04
Also published as: GB8920490D0; WO1991004636A3

Abstract

In a motion-adaptive video signal conversion method and apparatus, a saving in field delays is obtained by using a spatial interpolation when motion is present and, when motion is absent in a particular area of the image, repeating a previous signal portion representing the same spatial location in the image. The evaluation of motion involves averaging the input signal over N x M block of pixels and taking the frame difference of the averaged signal. This frame difference signal may be spread by temporal filtering so as to avoid the moving bar problem. Further savings may be obtained by taking the sum and difference of successive samples of the input signal, subjecting the sum signal to the above motion-adaptive processing whilst the difference signal undergoes a spatial interpolation, and recombining the processed signals.

Description

IMPROVEMENTS IN OR RELATING TO MOTION ADAPTIVE VIDEO SIGNAL CONVERTERS

The present invention relates to methods and apparatus for converting a video signal from one format (i.e. one line structure and field rate) to another format. In particular, the invention concerns motion adaptive video signal converters.

In the field of television signal broadcasting one of the long-standing applications of video signal converters is in the transforming of a television signal in the studio from one "standard" format (e.g. the NTSC format used in the USA) into another "standard" format (e.g. the PAL format used in the UK). This type of "standards conversion" may involve a change both of the number of lines in a field and of the field rate.

A more recent application of video signal converters is in the field of enhanced or high definition television systems (most of which are still at the experimental stage). For example, it is proposed in the article "A new approach to research and development of high-definition television" by Nishizawa and Tanaka (NHK Technical Monograph No. 32, June 1982) to reduce the visibility of the frame rate interline flicker which arises from the interlaced structure of a signal by upconverting a 525 line/frame, 60 fields/second, 2 interlaced fields/frame signal (525/60/2:1) at a domestic receiver to a non-interlaced, i.e. sequential, format (525/60/1 :1 ).

In a standards converter, or an up-converter, the apparatus is required to produce output lines and/or fields which have no direct counterpart in the input video signal. This often involves interpolation between the lines and/or fields of the input video signal. In practice it has been found that when a single fixed interpolation scheme is used to produce all of the required lines of the output signal impairments arise in the image represented by the output signal. This is because for a given spatiotemporal location, A, in a video signal it is not possible to consistently define which neighbouring locations will have a signal value most closely approximating the signal value at A, these vary dependent upon whether or not movement is occurring (and in what direction) in the image represented by the video signal. Accordingly it is preferable to use two (or even more) interpolation schemes and to have some mechanism for selecting the interpolation scheme to be used at a given time dependent upon whether or not movement is present in the video signal portion being processed.

There are two main ways of controlling the selection of interpolation schemes in a motion adaptive converter; either a control signal indicative of the presence, absence, or degree of motion may be included in the input video signal (as in DATV, digitally assisted television), or a motion detector may be included in the motion adaptive converter itself.

Examples of motion adaptive video signal converters incorporating movement detectors may be found in an article "Digital standards converter by adaptive intra-frame line interpolation" by Kinuhata et al (IEEE Trans Communications Vol. COM-26, No. 10 (1978) pp. 1413-1419), in a British patent application published as GB-A-2050109 and in an International patent application published as WO85/04542.

The motion detectors used in the known adaptive video signal converters often assess motion by comparing a video signal X with a temporally adjacent signal Y (i.e. a video signal Y at the same spatial location as X but in the previous frame). Since the video signal is usually in a digital form this comparison is carried out on a pixel by pixel basis. Generally the magnitude of the difference between X and Y (the "frame difference") is used directly as an indication of the degree of motion in the video signal, however in some devices the ratio of the frame difference to the frame sum (X + Y) is used instead.

There are a number of disadvantages associated with using the known motion adaptive video signal converters.

Firstly the known adaptive converters generally incorporate three of more field delays or field memories (at least one of which is used in the chrominance signal path) which increases the cost, size and complexity of the apparatus. Also, each of the field delays requires a bandwidth at least equal to that of the input signal; this is more significant when the input signal has a large bandwidth e.g. when the video signal is a high definition television (HDTV) signal. Secondly problems often arise in the known adaptive converters incorporating motion detectors when the input video signal represents an image including a moving bar. This situation is illustrated by the - A -

diagram shown in Fig. 1. When a point, S, at horizontal location X s in field 0 is on a horizontally moving vertical bar and is to be constructed for the output signal by interpolation, then the motion detector will select an interpolation scheme to be used based on the frame difference at horizontal location X s between the signals for fields -1 and +1. The calculated frame difference will fail to indicate the presence of movement and so the converter erroneously will use an interpolation scheme appropriate to a static area.

In an article "A motion Detector for Television Applications" by B Flannaghan (Conference Proceedings of Digital Processing of Signals in Communications, Loughborough University, April '85, Institute of Electronic and Radio Engineers). A motion detector is described for overcoming the "moving bar" problem mentioned above. However, this upconverter still is subject to the first-mentioned disadvantages of size and complexity and the requirement for field memories of relatively large bandwidth.

The present invention provides motion adaptive video signal conversion apparatus and methods having reduced field delay/memory requirements compared with the previous proposals. Also, techniques to eliminate the moving bar problem may be accommodated in embodiments of the invention.

One embodiment of the present invention provides apparatus and a method for processing a video signal by application of a first interpolation scheme, when a predetermined level of motion is present in the input signal, and outputting the result, and by application of a second interpolation scheme, when a predetermined level of motion is not present in the input signal, and outputting the result, wherein the second interpolation scheme involves repetition of earlier lines of the input signal. Preferably motion is detected in the input signal by obtaining a measure of the signal level for an area of the video image represented by the input signal and comparing this with a corresponding measure of the signal level for a corresponding area of the video image represented by an earlier portion of the input signal.

This embodiment of the invention has the advantage that the video signal conversion may be carried out using a single field delay (or field store) even for a colour video signal.

A further embodiment of the invention provides apparatus and a method for processing a video signal by producing a first signal representing the low frequency component of the input signal, producing a second signal representative of the high frequency component of the input signal, applying a motion adaptive interpolation scheme to the first signal to produce a first interpolated signal, applying a spatial interpolation scheme to the second signal to produce a second interpolated signal, and combining the first and second interpolated signals to produce an output signal.

Preferably the first signal is the sum of successive portions of the input signal and the second signal is the difference of successive portions of the input signal. Also added advantages may be gained if the motion adaptive interpolation scheme used in this further embodiment of the invention is implemented according to the first embodiment described above.

This further embodiment of the invention has the advantage that the video signal conversion may be carried out using one or more field delays having a bandwidth half that of the input signal.

Preferably, when apparatus embodying the invention incorporates a motion detector, measures to avoid the above-described "moving bar" problem will be taken, such as the spatial and temporal spreading of frame differences described in the Flannaghan article.

Further features and advantages of the present invention will become clear from the following description of embodiments thereof, given by way of example, and illustrated by the accompanying drawings, in which:

Fig. 1 illustrates diagrammatically three fields of a video signal including a horizontally-moving vertical bar;

Fig. 2 shows in block diagrammatic form an upconverter for a colour video signal using known techniques;

Fig. 3 shows in block diagrammatic form an upconverter simplified according to a first embodiment of the invention;

Fig. 4 shows in block diagrammatic form the structure of a preferred movement detector for use in the Fig. 3 upconverter;

Fig. 5 shows in block diagrammatic form an upconverter according to a second embodiment of the invention; and

Fig. 6 is a schematic representation of a technique used in the Fig. 5 upconverter illustrating the relationship between the input and output signals.

An upconverter using known techniques for converting an interlaced colour video signal in the form of digital samples into a sequential signal, doubling the number of lines per field, is shown in block diagrammatic form in Fig. 2. The illustrated upconverter has separate processing paths for luminance samples, Y, and chrominance samples CR Cg.

The luminance processing section performs a motion adaptive interpolation on the input luminance signal. Two field delays 1A, 1B are used to provide samples two fields apart (i.e. a frame apart for input interlaced signals) both to a motion detector 10, for assessing motion in the video image represented by the input signal, and to averaging circuitry 2, 3 for producing a temporally interpolated signal for use when the video image is assessed as being static. A spatially interpolated signal for use when the video image is assessed as "moving" is produced from one- field-delayed input samples a line apart using line delay 5 and averaging circuitry 6, 7.

The motion detector 10 assesses motion in the video image by calculating frame differences on a sample by sample basis. Each frame difference may be compared with a threshold and rejected if below threshold, so as to prevent an incorrect choice of an interpolated signal appropriate to a "moving" image due to noise present on the video image. The output of the motion detector 10 is used as a control signal at a mixer 8, which receives as inputs the temporally and spatially interpolated signals, so as to control selection of one or other of the two inputs, or a mixture of the two. At a line compression device 9 the mixer output is combined with delayed input luminance samples to double the number of lines in the signal. The line compression device 9 compresses each line to half of its original length and outputs the resultant signal so that the duration of each frame of the output signal will be the same as that of each input frame. The bandwidth of the output signal is double that of the input signal.

The chrominance processing section of the Fig. 2 upconverter includes only a spatial, and not a temporal, interpolation stage, provided by the vertical interpolation section 15. However a field delay 11 is required so as to ensure correspondence of the output chrominance signals with the output luminance signals. The above described upconverter may be simplified according to a first embodiment of the invention as shown in Fig. 3. Those elements of Fig. 3 which correspond to elements of Fig. 2 have been given the same reference numerals.

Instead of averaging two signals a frame apart to produce a temporally interpolated signal for use when the video image is assessed as being static, the upconverter of Fig. 3 produces and uses a one-field- delayed signal. This one-field-delayed signal is slightly more sensitive to noise than the averaged signal used in the Fig. 2 upconverter. However the simplification of the "static" signal path enables a saving of two field delays to be made, one from the luminance processing section and one from the chrominance processing section (since no compensating delay is now required to ensure the proper timing of output chrominance signals with output luminance signals).

When an upconverter is implemented using a single field delay as shown in Fig. 3 the motion detector cannot as before assess motion by measuring frame differences on a pixel by pixel basis, instead a new type of motion detector 100 is required.

In a preferred version of this embodiment of the invention a motion detector as illustrated in Fig. 4 is used. In this motion detector 100 the input luminance signal is averaged over a block of N x M samples representing an area of the video image, by an averaging circuit 101. The average values are fed to a circuit 102 which calculates frame differences, i.e. it calculates the difference between the average luminance value of a block at a given spatial location and the average luminance value of a correspondingly located block in the preceding frame. The capacity of the store or delay needed for this frame difference calculation is 1 /NxM capacity of a full frame store or delay. For example, a suitable block size would be about 16 pixels x 4 lines in which case the store/delay in the frame difference calculation device 102 would need a capacity of only 1/64 of a frame.

The block averaging performed in the preferred motion detector of Fig. 4 has an effect similar to that of a spatial filter and improves the performance of the upconverter in relation to image areas of slow movement.

In the preferred movement detector the frame difference signals are filtered in a circuit 103 to "spread" the signals spatially and temporally. This reduces the effect of isolated points and avoids the moving bar problem. Details of the techniques needed to implement this "spreading" are given in the Flannaghan article referred to above.

The upconverter of Fig. 3 could also be used on a video signal containing data indicative of the locations in the signal where significant movement occurs. In this case the motion detector 100 need only strip off the data and use it as a direct indication of motion, or absence (or degree) of motion, in the signal.

An upconverter according to a second embodiment of the invention is shown in block diagrammatic form in Fig. 5.

In the Fig. 5 upconverter the input luminance signal of sampling rate f-| , is split into two signals each of sampling rate 1/2 fi . One of the signals is the sum of pairs of input samples, the other is the difference of the same pairs. These two signals can be recombined to form the original f-j signal as shown in Figure 6. It can be considered that the sum signal has been low pass filtered, and the difference signal high pass filtered, where the filters are crude 1 1 and 1 -1 filters.

In the case of the upconverter of Figure 5, upconversion is applied to both 1/2 f-| paths separately. The upconversion in the low frequency path is fully adaptive, switching between temporal and vertical interpolation. The upconversion in the high frequency path is vertical only interpolation. In operation, moving picture areas cause the upconversion in the low frequency path to switch to vertical interpolation, and after combination at the output the result is exactly as if an f-] signal had been converted directly. In stationary areas, temporal interpolation is applied to the low horizontal frequency components, and vertical interpolation is applied to the high frequency components.

This second embodiment enables field delays of halved bandwidth to be used, i.e. stores/buffers having halved capacity and halved reading/writing rate may be used.

If the upconverter of Fig. 5 is implemented using the circuitry of Figs. 3 and 4 to perform the adaptive interpolation then an input video signal can be upconverted using just one field store/delay having a bandwidth half that of the input signal.

The present invention finds application in the upconversion and "standards" conversion of video signals in generally, e.g. whether transmitted television signals, signals recorded on video tape or signals representing image data transmitted by telephone, and whether in composite format, such as NTSC or PAL, or in MAC format.

A particularly advantageous application of the present invention is in the upconversion of extended definition television signals in the MAC type format having an increased bandwidth, e.g. 27 MHz luminance sampling rate rather than the usual 13.5 MHz. A conventional motion adaptive upconverter such as that illustrated in Fig. 2 would require three 27 MHz field delays to double the number of lines in such a signal whereas according to the present invention only a single 13.5 MHz field delay would be required (using a Fig. 5 type upconverter incorporating Fig. 3 type circuitry for the adaptive interpolation).

Furthermore, if the above-described up¬ conversion of an extended definition MAC type signal were to be carried out in a broadcast signal receiver, or the like, then existing MAC decoder chips, such as the Philips/Plessey/Nordic VLSI chip, could still be used as described below.

In general a barrier to the exploitation of the extra analogue bandwidth in such an extended definition MAC type signal is that current MAC semiconductor chip sets cannot decode it, and development of versions to work at twice the usual sampling rate might take a long while. However, this type of extended definition MAC signal provides better performance than ordinary MAC, both because of upconversion, and because of increased bandwidth. If only one of these two features is taken advantage of, the improvement over ordinary MAC may not be considered to be worthwhile.

A solution is to sample the received MAC waveform at 40.5 MHz, and then pass the even samples to one decoder at 20.25 MHz, and the odd samples to a second. The two 13.5 MHz (Y) outputs can be combined to produce the 27 MHz decoded signal required for input to the upconverter. The Philips/Plessey/Nordic VLSI chip set is particularly suited to this approach because the vision processing is in a separate i.e. (MV1710) and only this i.e. needs to be duplicated. Some care may be needed to ensure that the cut point is not visible, but preliminary study suggests that this is not a serious problem.

The specific embodiments described above use motion detectors to produce an indication of the motion present in the input video signal. It is to be understood that in some embodiments of the invention the indication of motion may be derived from data accompanying the input video signal (i.e. the video signal may have associated therewith information indicating whether or not there is motion in the video signal, or indicating the degree of motion present) .

Claims

CLAIMS :

1. A motion-adaptive video signal convertor, comprising: means for providing an indication of the motion in a video signal input to the convertor; means for performing a spatial interpolation on a portion of the input signal; means for delaying the input signal; means responsive to the indication of the motion output by the providing means for selecting between the output of the spatial interpolator and the output of the delaying means and for gajting the selected signal to the output of the convertor, wherein the selection means is adapted to select the output of the spatial interpolator when the output of the providing means indicates the presence of motion in the input signal and to select the output of the delaying means when the output of the providing means indicates the absence of motion in the input signal.

2. A convertor according to claim 1 , wherein the providing means comprises a motion detector, the motion detector comprising: means for processing a portion of the input signal to generate a value representative of an area of the video image, means for delaying said value, and means for comparing said delayed value with a subsequent value generated by the proe- essing means representative of a corresponding area of the video image, whereby to evaluate the motion in the input signal and output a signal indicative of the evaluated motion.

3. A convertor according to claim 1 or 2, wherein the providing means is a motion detector adapted to output a binary signal whose value depends on whether the evaluated motion is greater or less than a threshold value.

4. A convertor according to claim 2 or 3, wherein a temporal filter is arranged to operate on the output of the comparing means.

5. A motion-adaptive video signal convertor, comprising: means for producing a first signal representative of the low frequency component of an input video signal; means for producing a second signal representative of the high frequency component of the input signal; a motion-adaptive interpolator adapted to receive said first signal as an input and to output a first interpolated signal; a spatial interpolator adapted to receive said second signal as an input and to output a second interpolated signal; and means for combining said first and second interpolated signals to produce an output signal.

6. A convertor according to claim 5, wherein the means for producing the first signal is adapted to sum successive portions of the input signal and the means for producing the second signal is adapted to take the difference of successive portions of the input signal.

7. A convertor according to claim 5 or 6, wherein the motion-adaptive interpolator comprises a motion adaptive video signal convertor according to claim 1, 2, 3 or 4.

8. A convertor according to claim 5, 6 or 7, and further comprising: a duplicate arrangement of first signal producing means, second signal producing means, motion adaptive interpolator, spatial interpolator and combining means; means for sampling the input signal; means for feeding half of the input signal samples to the first signal producing means and second signal producing means of the first arrangement; means for feeding the other half of the input signal samples to the first signal producing means and second signal producing "means of the duplicate arrangement; and means for combining the outputs of the first and duplicate arrangements.

9. A convertor according to claim 8, wherein the feeding means are arranged to feed odd input signal samples to the first arrangement and even input signal samples to the duplicate arrangement.

10. A motion-adaptive method of converting a video signal, comprising the steps of: producing a signal indicative of the motion in the input video signal; performing a spatial interpolation on a portion of the input signal; delaying the input signal; selectively gating the spatially interpolated signal to the output of the convertor, when the output signal of the producing step is indicative of the presence of motion in the input signal, and the delayed signal to the output of the convertor, when the output signal of the producing step is indicative of the absence of motion in the input signal.

11. A method according to claim 10, wherein the producing step comprises processing a portion of the input signal to generate a value representative of an area of the video image, delaying said value, and comparing said delayed value with a subsequent value generated by the processing means representative of a corresponding area of the video image, whereby to evaluate the motion in the input signal, and outputting the result of the comparison.

12. A method according to claim 10 or 11, wherein the producing step comprises comparing a signal representative of the degree of motion in the video image with a threshold level and outputting a binary signal whose value depends on the result of the comparison.

13. A method according to claim 11 or 12, wherein the producing step further comprises temporally filtering the output signal indicative of motion.

14. A motion-adaptive method of converting a video signal, comprising the steps of: producing a first signal representative of the low frequency component of the input signal; producing a second signal representative of the high frequency component of the input signal; applying a motion-adaptive interpolation scheme to the first signal to produce a first interpolated signal; applying a spatial interpolation scheme to the second signal to produce a second interpolated signal; and combining the first and second interpolated signals to produce an output signal.

15. A method according to claim 14, wherein the first signal producing step comprises summing successive portions of the input signal, and the second signal producing step comprises taking the difference of successive portions of the input signal.

16. A method according to claim 14 or 15, wherein the step of applying a motion-adaptive interpolation scheme comprises applying a method according to claim 10, 11, 12 or 13.

17. A method according to claim 14, 15 or 16 and further comprising sampling the input signal; dividing the input signal samples into two halves, each half of the input signal samples being separately subjected to the producing steps, motion-adaptive interpolation step, spatial interpolation step and combining step; and combining the signals output from the combining steps in respect of the two halves of the input signal samples.

18. A method according to claim 17, wherein the dividing step is adapted to separate the odd samples of the input signal from the even samples.