FR2651628A1 - Device for driving acoustic channels via the contents of an image, in an audovisual system - Google Patents

Abstract

The objective of the invention is to enhance a given audiovisual system, by making it possible significantly to increase the coupling between the image and the sound, in such a way as to give the viewers sensory perception effects, especially acoustic effects, which are very much greater than the effects of the original audiovisual sequence. These objectives are achieved, according to the invention, by a device for driving acoustic channels, including means (23A, 23B, 23C) for selective distribution among several acoustic channels (28A, 28B) or one or more source audio signals (24), and/or for selective processing of at least one of the said channels, the said means being driven in real time by means (20A, 20B, 20C) for analysis of at least one of the characteristics (13, 15, 26, 27) of a video signal. The invention applies, for example, equally well to domestic television sets as to cinematographic projection, or also to video conferences, etc..

Description

 Device for controlling acoustic channels by the content of an image, in an audiovisual system.

 The field of the invention is that of improving by signal processing the video-acoustic effects restored in audiovisual systems or media of all kinds. By way of nonlimiting example, one can cite just as well domestic televisions as cinematographic projections, or videoconferences, etc ...

 By "acoustic video" effect is meant the fact of obtaining a good coupling between the visual perception and the acoustic perception. During the representation of an audiovisual sequence, direct or programmed, the spectator or the participant in the audiovisual system must thus be able to obtain a pronounced sensory effect, either intentionally artificial, or intended to reconstruct as faithfully as possible during the representation of a scene through the technological media, the effects that would be likely to be produced in reality.

 The invention provides a system for improving audiovisual rendering, applicable to conventional audiovisual signals providing limited sensory effects, in order to satisfy the requirement of media consumers.

 Thus, in the television sector in particular, the evolution of the systems for broadcasting audio / video programs as well, in the television sector, in particular aims, by increasing the quality of the standards and components used, to create the conditions known as "home cinema". The size of the display screens tends to increase, and the definition of the image to improve (HD TV). Likewise, the acoustic quality improves, with digital and stereophonic sound.

 However, these developments are slow, and should see their culmination at the start of the next millennium, with the standards envisaged today.

 In the meantime, manufacturers are trying to make better use of current broadcast standards, with improvements such as doubling the frame frequency, progressive scanning and noise reduction for the image for example, and sound generation pseudo stereophonic and a "surround" atmosphere (environmental effect) for the audio part.

 In known systems, the pseudo-stereophonic generation from a monophonic signal is carried out by a distribution of the sound spectrum between the left channel and the right channel. Usually, the left channel is enriched with high frequencies, and the right channel with low frequencies. This distribution is due to the fact that in an orchestra, the violins are on the left, and the cellos and double basses on the right.

 It is not uncommon for a third channel to be placed in the center, reproducing the entire frequency spectrum, or only the mid frequencies. This channel can also be provided with a delay line, in order to create a depth effect.

 This device is indeed well suited to the retransmission of a concert. However, in other cases, it does not necessarily provide realistic results. In the case of a dialogue between two characters, for example, this distribution does not correspond to the reality of the image. If, more particularly, the dialogue takes place between a woman and a man, the woman appearing on the left on the screen, we obtain a result opposite to reality, the highest voice being heard on the right and the lowest voice to the left.

 On the other hand, processors are known which make it possible to generate 5 to 6 different channels from a stereophonic signal, using filtering and preset delays. It is thus possible to select an atmosphere among several, for example opera, jazz, reverberation ...

 These different means allow only one generation of passive atmosphere. Indeed, the corrections made to the audio signals are fixed. Thus, whatever the type of scene present on the screen, the soundscape remains unchanged.

 There is also a recording process, Dolby MP (registered trademark). This process allows the distribution of sounds over several acoustic sources, depending on what is happening in the associated image. Thus, the sound effect rendered is fairly close to reality. This process is implemented in many cinemas. However, this distribution of sounds, and the associated sound effects, must be recorded during the making of the film. This system is therefore not applicable for improving audiovisual recordings made with poorer means (monophonic sound). It cannot, without modification of the different standards, be used for example in television.

 There are also known decoders called "surround-sound", which use the above method, for example for playing video discs, and which have means for dynamic distribution of sounds. These decoders can supply an additional acoustic source, placed in the center of the stage, and broadcasting the dialogs. They have for this purpose means for locating speech in an acoustic signal. The result is that the dialogue comes mainly from the center speaker, the sound effects and the music from the side speakers. This improvement does not however restore the sound environment corresponding to the image broadcast. If we take the example of the dialogue between two characters, the sound will only come from the central speaker, whoever is speaking.

 The objective of the invention is to overcome these various drawbacks of the technique.

 More specifically, the object of the invention is to improve a given audiovisual system, by making it possible to significantly increase the coupling between image and sound, so as to give viewers very superior effects of sensory perception, notably acoustic the effects of the original audiovisual sequence. Thus, the implementation of the invention allows in particular the creation of a self-acoustic environment adapted to the characteristics of the visual scene, for example by enlarging the acoustic scene, in audiovisual systems originally limited.

 A complementary objective of the invention is to provide a device suitable for any audiovisual system, whether private (televisions), collective (cinema) or professional (videoconference).

 Another objective of the invention is to provide a device allowing the realization of audiovisual animations or shows, by coupling or enslaving sound effects to previously or jointly generated visual sequences.

 These objectives, as well as others which will appear subsequently, are achieved, according to the invention, by a device for controlling acoustic channels, comprising means for selective distribution between several acoustic channels of one or more audio source signals, and / or for selective processing of at least one of said channels, said means being controlled in real time by means of analysis of at least one of the characteristics of a video signal.

 According to a preferred characteristic of the invention, said means for analyzing the characteristics of a video signal consist of means for spatially splitting at least one image corresponding to said video signal into several zones, and means for analyzing characteristics of each of said zones, each of said zones being selectively associated with one or more of said acoustic channels.

 Advantageously, said analysis means comprise means for detecting and / or quantifying movement in at least one image corresponding to said video signal or in at least one of said zones.

 Advantageously, said means for detecting and / or quantifying movement include means for comparing, over several successive frames, at least one of the characteristics of said video signal.

 According to another advantageous characteristic of the invention, said analysis means comprise means for sub-sampling and averaging at least one of the characteristics of said video signal.

Preferably, said analysis means use at least one of the components of the video signal belonging to the group comprising:
- luminance component;
- chrominance component;
- red component;
- green component;
- blue component;
- line synchronization;
- frame synchronization.

Advantageously, said selective processing means comprise at least one of the means belonging to the group comprising:
- amplification;
- amplitude modulation;
- frequency modulation;
- phase modulation;
- delays;
- filtering;
- spectral distribution;
- spectral shift;
- reverberation.

 According to another preferred characteristic of the invention, said means for selective processing of at least one of said channels comprise gain control means with amplitude modulation.

 Advantageously, said gain control means comprise a modulation rate limiter.

 According to another advantageous characteristic, said gain control means comprise a control speed limiter.

 Preferably, at least one of the characteristics of said video signal is used as a means of validation and / or of selection of said selective processing means.

 According to another preferred characteristic of the invention, at least one of said zones is selectively associated with a sound or a range of sounds and / or a sound effect.

 Advantageously, the device controls a musical synthesizer.

Other characteristics and advantages of the invention will appear on reading the following description of three preferred embodiments of the invention, given by way of illustration and limitation, and the appended drawings, in which
- Figure 1 shows the block diagram of a device for generating two
pseudo-stereophonic acoustic channels from an audio signal
monophonic, according to the invention;
- Figure 2 shows the simplified block diagram of a following device
the invention of generating 4 acoustic channels from an audio signal
stereophonic;
- Figure 3 shows the block diagram of an improvement to the devices of the
Figures 1 and 2 to increase the contrast between two channels
acoustic
- Figure 4 is a block diagram of a device according to the invention allowing
control of a musical synthesizer.

 According to the invention, the device allows the improvement or the creation of a sound environment associated and adapted to a video image.

 More specifically, the present invention relates to a device for controlling acoustic channels controlled in real time by the content of at least one image. This device can in particular be applied to the creation of an acoustic environment that is self-adapting to the characteristics of the visual scene, for example by enlarging the acoustic scene, in audiovisual systems originally limited.

 One application of the invention is the generation of a number of acoustic channels greater than the number of source audio signals, by analysis of the movements in the associated image, the device according to the invention increasing the gain of the acoustic channels associated with the zones in which movement has been detected. In this preferred application of the invention, an original principle is thus implemented: the fact that, as a general rule, everything that makes noise moves. We will find an illustration of this principle by considering that, in a scene implementing a dialogue between two characters, the image of the one who speaks is generally more agitated than that which listens, if only by the movement of the lips. Moreover, although it is clear that this analysis is not general, the fact remains that the principle actually provides enriched sound effects in connection with the content of the image.

 The invention can also be used in combination with any sound effect or adaptation of known type, for example modulation, delay, phase shift, etc.

 The invention can also be used to generate sounds, and therefore music, on a musical synthesizer, thus allowing the production of audiovisual shows, the music being linked to the content of the video image, either synthetic image or image. of a person moving for the purpose of generating said music.

 FIG. 1 shows a device for active generation of a pseudo-stereophonic environment from a monophonic audio signal associated with a video signal according to the invention.

This first preferred embodiment of the invention relates in particular to conventional television receivers, such as televisions according to SECAM, PAL or NTSC standards, having two speakers, left 11A and right 11B
The various video signals, chrominance 26, luminance 13, line synchronization 15 and frame synchronization 27 are directed into the usual video processing circuits. On the other hand, they are used, according to the invention, to identify movements in the areas of the image, in order to act on the distribution of sounds between the different speakers. Thus, it is the video signal which permanently and in real time controls the distribution of the acoustic signals (amplitude, frequency, delay) between the left and right speakers.

 For example, the simple case of two fixed characters, one on the left, the other on the right of the screen, interviewed by a third person in the center. The problem is to favor the speaker located on the side of one of the interviewees when he speaks, and to provide the two side speakers with the same signal when the central interviewer is speaking (or to feed a center speaker).

 The solution consists in carrying out an analysis of the movement between successive images in order to detect the movement of the lips of the character having the floor.

 The information concerning the barycenter of the moving area is translated into right-left location information, which itself controls the distribution of the acoustic signals between left and right speakers.

 It is clear from this example that it is not a question of analyzing all the pixels of successive images, but rather the pixels of averaged images, in much fewer numbers. Since in this case it is only a question of obtaining a right-left location, the pixels of the averaged image are vertical stripes, having the height of the image. It seems that 3 to 5 of these vertical pixels, of width 1/3 to 1/5 of the screen width, are sufficient to obtain a precise left / right location. In addition, image analysis can only be done on luminance signals. Finally, in order to avoid artifacts and sudden switching from one speaker to another, we will limit the modulation rate of this gain control system at approximately + 33%, so that at most, the ratio of the intensities provided by the loudspeakers is (1 + 0.33) / (1 - 0.33) = 2. Detection motion requiring the comparison of 2 successive images, i.e. a minimum duration of 2 x 1/25 = 0.8 seconds, the gain control device will also include a control speed limiter, having a time constant of 0.1 to 0.3 seconds.

 In the case of FIG. 1, we have chosen to detect the movements in three areas of the screen of equal widths: left 1OAX center 10B, right 10 ,.

 The objective is to distribute the sound intensity between 2 left speakers 11A and right 11B, located on either side of the screen 12.

 The distribution is made from the luminance signal 13.

 First of all, a generator 14 synchronized at the line frequency by the line synchronization component 15 supplies 16A pulses of duration = 1/3 of the line duration (ie 64/3, US in PAL / SECAM). From these pulses are generated, by delay means 17A and 17BX, two other trains of pulses 16B and 16C, delayed respectively by 1/3 and 2/3 of the line duration, and of the same duration (1 / 3 of the line duration). These three pulse trains 16A, 16B, 16C are sent respectively to three analog multipliers 18A '18B' î8ç, which all three receive, on their second input, the luminance signal 13. It therefore emerges from these 3 multipliers 3 signals 19A'19B '19CX luminance corresponding respectively to the left, center and right parts of the image.

Each of these partial luminance signals 19A, 19BS l9, is summed during the frame period, in one of the memories 20A, 20B, 2OCX controlled by the frame synchronization signal 38. At the end of the frame period (frame 1) the sums obtained , or Ci Zlc, Sîd (analog signals) are memorized. The same operation is repeated during the following frame (frame 2), hence the sums S2g, # 2c, S2d. At the end of this second frame the differences are made:
s left weft = 6 12 g = S1g - S2g
S weft center = 6 12 C = Sîc - # 3c
6 straight weft = S 12 d = # 1d - # 2d
Similarly, the differences # 23g, # 23c, 623d are calculated for frames 2 and 3, and so on.

The signals s consist of a series of pulses of variable amplitudes (positive or negative) at the frame frequency. These pulses are then rectified in dual cycle, and integrated with a time constant of a fraction of a second (0.1 to 0.3 seconds) by the rectifier integrators 21A, 21B, 21C
These three integrated signals 22A '22B' 22C, therefore slowly variable, are used for 'B' C 'LLC) to respectively control the gains of three amplifiers 23A 23B, 23 are subjected to the same monophonic audio signal 24. acts to manufacture two sound channels 28A and 28B, corresponding to the two speakers 11A and 11B, the signals from the amplifiers with left gain control 23A and center 23B are summed, with weighting, in the consumer 25A, to form the left channel 28A . Similarly, the straight channel 28B is obtained by summing in the summator 25B 'the signals from the amplifiers 23B and 23c.

 The details of the circuits of the functional blocks are not described, because those skilled in the art can design these functions in digital or analog technology.

 This example is not limitative. We can first of all conceive of synthesizing more than two sound channels from the same monophonic signal. Similarly, it is possible to envisage, by the same techniques, performing a high-low sound distribution of the image, and not left-right. By combining these two methods, four sound sources for example can be controlled by the four image areas.

 Figure 2 shows the block diagram of an extension of the device of Figure 1 in case the audio signal broadcast with the image is stereophonic.

Four acoustic channels are generated here
left - up 30A and left - down 30B from the left audio channel 31A;
right - top 30C and right - bottom 30D from the right audio channel 31B
The partial video signals 32A, 32B, 32C and 32D, corresponding to the 4 quarters of the screen 33A'33B '33C and 33DX are generated by means 34 for cutting the image, from the video signals 35.

 Each of said partial video signals 32A, 32B, 32c, 32D undergoes in the respective blocks 36A, 36B, 36Ç and 36D an analysis of movement identical to that specified in the case of FIG. 1.

 Blocks 36A and 36B control the gain of amplifiers to which the left source audio channel 31A is subjected. Blocks 36C and 36D act identically for the right source audio channel 31B.

 Four distinct acoustic channels 30A, 30B, 30C and 30D are thus obtained, intended respectively for the four speakers 37AX 37BX 37C and 37D.

 Improvements can be brought to the device by combining the process described and the known art: delays introduced on the rear acoustic channels, simulating the reverberant atmospheres, widening of the stereophonic scene by selective phase shifts, realization of standard atmospheres, such as opera, outdoor, etc ...

 In FIGS. 1 and 2, the gain control amplifiers are such that their gain increases when the control voltage increases. FIG. 3 shows an improvement to the devices of these two figures making it possible to increase the contrast between two channels, for example left channel and right channel, by proceeding as follows: when a movement is detected in the left part of the image, the gain of the left channel is increased and that of the right channel is decreased, and vice versa for a movement to the right.

 This is achieved by doubling the win orders.

 The left gain control signals 41A and right 41B drive the amplifiers 42A and 42D, to which the monophonic signal 43 is subjected. On the other hand, the two gain control signals 41A and 41B are inverted by two inverters 44A and 44B which generate decreasing voltages when their input voltages are increasing. signals 45A and 45B from these two inverters drive two amplifiers 42B and 42c, identical to amplifiers 42A and 42D.

 The signals 46A and 46C from the gain control amplifiers 42A and 42C are then summed by the summator 47A 'to give the signal 48A, intended for the left acoustic channel. Similarly, signals 46B and 46D are summed to provide the signal 48B, intended for the right acoustic channel.

 So far, only luminance signals have been used for motion detection and gain controls. You can choose to perform these same operations from the chrominance signals. It goes without saying that more precise motion detection and gain control can be obtained by simultaneously exploiting the luminance and chrominance signals.

 An extension of this latter combination consists in using the luminance signals for motion detection, and the chrominance signals to arbitrate, according to a given criterion, the gain control of the acoustic channels. This criterion can be a hue: to use the example of characters appearing on the screen, the sound distribution between right and left lane can only be activated if the moving area has the color "flesh". This process prevents "silent" movements from giving rise to an incorrect distribution of the acoustic signals between the sound channels.

 The device according to the invention, illustrated above by examples of control of the level of acoustic sources carried out in real time by the characteristics of the associated image, can be extended to control of any type, such as delays, phase shifts, spectral distributions, etc., depending on the desired effects.

 This device makes it possible to improve listening comfort. Other applications are also possible, such as the generation of sounds, therefore of music.

 It is indeed possible to associate a sound with a movement in the image, the characteristics of this sound being for example a function of the spatial location of the movement in the image.

 Under these conditions, the display screen behaves like a keyboard, each zone of the image with which a sound is associated playing the role of a key.

 Figure 4 presents two possibilities of using this application, respectively by computer generated images, or by analyzing the movements of a character.

 The video screen 50 is divided into N vertical bands 51 and P horizontal bands 52.

The corresponding video signals 53 are cut out according to the N × P image areas 54, and any movements are detected in each of these areas, in the processing block 55, according to the device of the invention developed above. Each of these
N x P image areas 54 corresponds to a control signal 56, which controls a musical synthesizer 57.

 Thus, a video image can drive the generation of sounds. This process is probably of little interest if any images are used, although it can provide surprising results. On the other hand, computer-generated images 58, imagined and calculated with the aim of creating an audiovisual spectacle, take on their full interest.

 I1 is the same for images obtained from a video camera 59 looking at an "instrumentalist" 60. The latter can "play" this new instrument, without contact, only by the movements of his hands, his body , of his head, etc ...

 Many other effects and applications of the invention can be envisaged for this audiovisual spectacle objective. We can cite, by way of example, the association of a particular color with a sound or a sound effect. The creators of the audiovisual field will find other applications to the present invention, for the production of shows, psychedelic animations and other audiovisual shows.

Claims (13)

 1. Device for controlling acoustic channels in an audiovisual system, characterized in that it comprises means for selective distribution (23A, 23B, 23c) between several acoustic channels (28A, 28B; 30A, 30B, 30C, 30D) d '' one or more audio source signals (24; 31A, 31B), and / or selective processing of at least one of said channels, said means being controlled in real time by analysis means (2OA 20B '20C; 36A , 36B, 36C, 36D ,; 55) of at least one of the characteristics (13, 15, 26, 27) of a video signal (35; 53).  2. Device according to claim 1, characterized in that said means for analyzing the characteristics (13, 15, 26, 27) of a video signal consist of spatial cutting means (18A, 18B, 18C; 34) d '' at least one image corresponding to said video signal in several zones, (10A, 1OBR 10; 33A, 33B, 33CR 33D; 54) and of analysis means (20A, 20B, 20C; 36A, 36B, 36C, 36D,; 55) characteristics of each of said zones, each of said zones being selectively associated with one or more of said acoustic channels (28A, 28B ,; 30A, 30B, 30C, 30D) -  3. Device according to claim 1 or 2, characterized in that said analysis means (20A'20B '20C; 36A, 36B, 36C, 36D ,; 55) comprise means for detecting and / or quantifying movement in at least one image corresponding to said video signal (35; 53) or in at least one of said areas (10A, 1OBX 10; 33A, 33B, 33CX 33D; 54).  4. Device according to claim 3, characterized in that said means for detecting and / or quantifying movement comprise comparison means (20A, 20B, 20C) on several successive frames of at least one of the characteristics of said video signal.  5. Device according to any one of claims 1 to 4, characterized in that said analysis means comprise means for subsampling and averaging at least one of the characteristics of said video signal.  6. Device according to any one of claims 1 to 5, characterized in that said analysis means use at least one of the components of the video signal belonging to the group comprising:  - luminance component (13);  - chrominance component (26);  - red component;  - green component;  - blue component;  - line synchronization (15);  - frame synchronization (27);  7. Device according to any one of claims 1 to 6, characterized in that said selective processing means comprise at least one of the means belonging to the group comprising: amplification,  - amplitude modulation;  - frequency modulation;  - phase modulation;  - delays;  - filtering;  - spectral distribution;  - spectral shift;  - reverberation.  8. Device according to any one of claims 1 to 7, characterized in that said means for selective processing of at least one of said channels comprise amplitude modulation gain control means (23A, 23B, 23c).  9. Device according to claim 8, characterized in that said gain control means (23A, 23B, 23c) comprise a modulation rate limiter.  10. Device according to claim 8 or 9, characterized in that said gain control means (23A, 23B, 233 comprise a control speed limiter.  11. Device according to any one of claims 1 to 10, characterized in that at least one of the characteristics of said video signal is used as means of validation and / or selection of said selective processing means.  12. Device according to any one of claims 2 to 11, characterized in that at least one of said zones (54) is selectively associated with a sound or a range of sounds and / or a sound effect.  13. Device according to claim 12, characterized in that it controls a musical synthesizer (57).