WO2004001699A2 - Method for enhancement of listener perception of sound spatialization - Google Patents

Abstract

Presentation of sensory stimuli in a manner that enhances the listener's perception of the three dimensional location of specific sounds so that the listener may recognize, comprehend and react to the sound in a manner that improves entertainment, education, safety, work performance and impact on psychological and physiological states.

Description

INTERNATIONAL APPLICATION FOR PATENT UNDER THE PATENT COOPERATION TREATY

METHOD FOR ENHANCEMENT OF

LISTENER PERCEPTION OF SOUND

SPATIALIZATION

[0001] This document claims priority to and the benefit of the filing date of a provisional application filed in the United States entitled "A Method for Enhancement of Sound Recordings," assigned serial No. 60/390,637, and filed 24 June 2002, which is hereby incorporated by reference.

TECHNICAL FIELD [0002] The present invention relates generally to the field of audio engineering.

Specifically, the present invention comprises a method of enhancing a listener's perception of sounds produced from signals that have been modified by any one of a number of sound directionalization and localization techniques (hereinafter collectively referred to as sound spatialization techniques). By applying the method of the present invention, one is able to create a contextual sensory frame of reference in which the spatialization techniques are more effective, thereby giving the audio engineer the ability to apply the spatialization techniques toward specific objectives.

BACKROUND ART [0003] It is well established that the brain is able to determine the location of a sound source relative to a listener as well as the character of the space from which that sound originated based upon auditory cue information presented to the listener within the sound. We have all experienced this phenomenon whether by, for example, involuntary turning our heads rapidly when we hear a loud sound in our vicinity or by tilting our heads upward to look at an airplane we hear flying overhead. The brain relies on four basic types of auditory cue information to determine the source of an incoming sound.

[0004] The first basic type of auditory cue information is the difference in amplitude of a single sound as it is received by each of a listener's two ears. For example, a sound originating from somewhere on the right side of a listener's head will have a greater amplitude when it reaches the right ear than will the same sound when it reaches the left ear. This particular amplitude difference would cue the listener's brain that the sound emanated from the right side of the listener's head.

[0005] The second basic type of auditory cue information is the time delay between the receipts of a single sound by each of a listener's two ears. Again, for example, a sound originating from somewhere on the right side of a listener will reach the listener's right ear before it reaches the listener's left ear. Such an interaural delay would cue the listener's brain that the sound emanated from the right side of the listener's head.

[0006] The third basic type of auditory cue information is the difference in arrival time between a particular sound and attenuated, delayed reoccurrences of that same sound resulting from the reflection of the original sound off of various objects in the listener's environment, such as walls. The listener's brain subconsciously compares the subtle differences between an original sound and its reflections to determine the size and character of the bounded or unbounded space from which the original sound emanated. Just as an echo (or lack thereof) allows a listener to determine the depth of a well, a slightly delayed reverberant sound (or lack thereof) allows a listener to subconsciously note the position and/or acoustic composition of objects in the listener's environment. The listener's brain associates longer delays between an original sound and its various reverberations with larger spaces or rooms, and shorter such delays with more confined spaces or rooms. The brain may also associate a larger or smaller number of reflections with more or less acoustically cluttered areas, respectively, depending on the difference in delay times between such reflections.

[0007] The fourth basic type of auditory cue information is the frequency response imparted to original or reflected sounds by the listener's body. As sound waves propagate, they are modified depending upon the nature of the media through which they travel. The brain is tuned to the specific modifications to a sound wave generated as a result of the shape and character of the listener's body, head, and ear, and the incident direction of the sound. These modifications represented in the field of binaural signal processing by head-related transfer functions (HRTF's), to denote the primary cause of their variability. It is important to note that any HRTF is really a pair of frequency responses, corresponding to the independent modifications made to sound waves before arriving at the left and right eardrums, respectively.

[0008] The brain is capable of making extremely precise subconscious comparisons with respect to each of these types of auditory cue information, and uses those comparisons to give the listener a fairly precise notion of the direction and location in a three-dimensional space surrounding the listener from which a particular sound is originating.

[0009] However, when recording sounds with the intention that they be reproduced, professional audio engineers generally record such sounds so that the recording will include only as much information as is necessary to capture the bare sound itself. The aim is to prevent the inclusion of sound artifacts in the recording in order to give the engineer as much control as possible during the post-recording mixing process over how a reproduction will sound. Accordingly, most professional recording environments utilize monaural microphones and recording chambers with walls designed to dampen reflections and reverberations with the specific purpose of excluding these kinds of auditory cue information from the sound recording being made. Consequently, sounds reproduced from such recordings contain almost no cue information that would allow a listener's brain to place the sound in a virtual space with respect to the listener's head upon reproduction.

[0010] The advent of stereo sound was the first attempt to include auditory cue information in binaural sound signals. Stereo sound enables an audio engineer to place a sound, in general, in the left, right, or center portions of a sound field by adjusting the amplitude of the signal delivered to each of the two stereo channels (i.e. left and right). While this method is effective in placing a sound image, the spatial extent of such placement is extremely limited; using such a method to place sounds only allows an audio engineer to place a sound or sounds along an imaginary one-dimensional line that can be drawn between the two speakers creating the sounds. In addition, when such sounds are played back on headphones, a listener will perceive them as originating somewhere between their ears. This sensation can be extremely uncomfortable for the listener because it defies the brain's understanding of the possible emanation location of sounds. For example, the brain knows intuitively that a piano cannot fit between the listner's ears, and that a piano could thus not create sound from within the listener's head. However, a recording of a piano processed using conventional stereo techniques will, when reproduced on headphones, nonetheless appear to originate from between the listener's ears.

[0011] Technological advances in the field of binaural signal processing since the advent of stereo sound have made more manageable the reproduction of sounds that contain particular desired auditory cue information. The art in this field describes numerous methods by which sound signals can be imparted with various characteristics related to the four types of auditory cue information described above such that when a listener is presented with sounds generated from such signals, the listener's mind may be tricked into believing that those sounds are emanating from a location other than the sound reproduction device from which the sound is actually being produced.

[0012] The terminologies used to describe these varying methods are as diverse as the methods themselves and include, but are not limited to, concepts such as sound localization, sound spatialization, and sound shuffling. For example, the prior art presents a multitude of methods whereby monaural sound signals are convolved with various amplitude- and frequency-response filters to form a signal that contains auditory cue information that emulates actual cue information as if the sound originated from the selected location. One popular method is to use a digital signal processor to convolve a finite-impulse response filter generated from a head-related transfer function corresponding to the desired directional characteristic of the sound to be generated with a monaural signal representing that sound. The prior art also presents methods of expanding the sound field projected by a stereo pair or pairs of sound reproduction devices, such as by phase-shifting part of all of the various signals, a technique sometimes referred to as sound shuffling.

[0013] Unfortunately, all of the methods of sound spatialization in the prior art suffer from one serious deficiency: the limitation of the human brain to accurately and consistently interpret audio cues. When spatialization techniques are applied to a given sound, the listener may be unable to interpret the audio cues, may misinterpret the audio cues, or may subordinate the audio cues to other sensory stimuli. In such events, the listener may gain little or know information from a given sound or may interpret the sound as originating from a location other than that intended. The result is that the listener may not react to the sound, may react differently than desired or expected, or may become confused from conflicting stimuli. For example, sound directionalization techniques in the prior art poorly differentiate between sounds intended to appear to emanate from in front or behind of a listener's head. That is, sounds intended to appear to emanate from directly in' front of a listener's head are not easily distinguishable from sounds intended to appear to emanate from directly behind a listener's head. This is primarily due to the insensitivity of the human auditory system in distinguishing between such sounds - usually a listener's eyes help to determine whether a sound is emanating from the front or back or the listener's head. [0014] The present invention improves the utility of sound towards a variety of objectives.

DISCLOSURE OF INVENTION [0015] It is an object of the present invention to enhance a listener's perception of sounds generated from signals modified by the various sound spatialization techniques of the prior art. This is accomplished by providing an auditory frame of reference against which a listener's brain can more accurately recognize, comprehend and react to a spatialized sound or sounds being presented.

MODES FOR CARRYING OUT THE INVENTION [0016] In one embodiment of the present invention, an input monaural signal is split into an identical plurality of monaural signals. The amplitude of each monaural signal is independently adjustable. Each monaural sound signal is directed to a device suitable for imparting user-specified time delays and reverberations to sound signals. The output from said device is directed to a digital signal processor (DSP) capable of splitting a monaural sound signal into two identical channels and imparting a user-specified frequency response to each channel. A personal computer provides the DSP with the proper HRTF from which the DSP derives the desired frequency response and imparts said frequency response to each channel of the binaural signal. The left and right channels of each binaural signal are combined and the resulting two-channel sound signal is reproduced via headphones or loudspeakers, or recorded on a medium capable of storing two- channel sound signals.

[0017] When reproduced, a sound or sounds processed using the above technique would then be combined in series and/or in parallel with one or more unprocessed sounds from which the reproduced sound was created. The unprocessed sounds thereby serve as an auditory frame of reference against which a listener's mind can more easily interpret the spatialization auditory cue information presented by the processed sounds.

[0018] Multiple layers of sound can be presented in this manner, some of which have spatialized qualities and some of which do not. The contrast created between the processed sounds that include the auditory cue information and the unprocessed sounds which do not include such information forces a listener's mind to more easily recognize those sounds that have been spatialized using any of the methods within the prior art.

[0019] The contrast results in quicker interpretation, a higher degree of understanding, and a higher retention rate of the information being presented to the listener.

[0020] The techniques of the present invention as described herein would be equally applicable to any kind of sound reproduction that includes any type of auditory cue information intended to spatialize the reproduced sounds in the mind of a listener.

Claims

CLAIMSWhat is claimed:

1. A method for the creation of a three-dimensional audio frame of reference for a listener in a three dimensional location comprising: presenting a spatialized auditory stimulus to said listener, wherein the apparent three dimensional origin of said auditory stimulus is different from said three dimensional location of said listener; presenting a separate sensory stimulus to said listener, wherein said separate sensory stimulus has a discrete apparent three dimensional location as measured on an x/y/z scale, wherein said discrete apparent three dimensional location of said separate sensory stimulus is different from both said three dimensional location of said listener and of said apparent three dimensional location of said auditory stimulus, wherein the discrete apparent three dimensional location of the separate sensory stimulus is obvious to the listener; and wherein said separate sensory stimulus has a discrete apparent three dimensional location that is perceivably different from said spatialized auditory stimulus; wherein said auditory stimulus and said separate sensory stimulus are presented substantially concurrently.

2. The method of claim 1 wherein said separate sensory stimulus is a single auditory stimulus.

3. The method of claim 1 wherein said separate sensory stimulus is a single visual, tactile, olfactory or psychological stimulus.

4. The method of any one of claims 1-3 wherein said separate sensory stimulus additionally comprises a plurality of sensory stimuli.

5. The method of claim 1 or 2 wherein a combination of sound directionalization and a reverse room simulation is applied to a single sound source to improve said listener's perception of the three dimensional location of said auditory stimulus.

6. The method of claim 5 wherein different reverse room simulations are applied to different sound stimuli to improve the listener's perception of the difference between the three dimensional locations of individual sound stimulus.

7. The method of claim 5 wherein said sound stimuli have already been mixed into a stereo sound recording without application of any spatialization techniques and a three dimensional frame of reference is achieved by: spatializing the left stereo signal to simulate a sound source with a point of origin that is to the left of the listener; and spatializing the right stereo signal to simulate a sound source with a point of origin that is to the right of the listener.

8. A method of improving listener recognition of sound stimuli by following the method of any one of claims 1-7.

9. A method of improving listener comprehension of sound stimuli by following the method of any one of claims 1-7

10. A method for improving listener reaction to sound stimuli by following the method of any one of claims 1-7.

11. The method of claim 8, wherein the objective is to entertain the listener.

12 The method of claim 8, wherein the objective is to improve the safety of the listener.

13 The method of claim 8, wherein the objective is to improve the work performance of the listener.

14 The method of claim 8, wherein the objective is to influence or alter the psychological state of the listener.

15 The method of claim 8, wherein the objective is to influence or alter the physiological state of the listener.

16 The method of claim 9, wherein the objective is to entertain the listener.

17 The method of claim 9, wherein the objective is to improve the safety of the listener.

18 The method of claim 9, wherein the objective is to improve the work performance of the listener.

19 The method of claim 9, wherein the objective is to influence or alter the psychological state of the listener.

20 The method of claim 9, wherein the objective is to influence or alter the physiological state of the listener.

21 The method of claim 10, wherein the objective is to entertain the listener.

22 The method of claim 10, wherein the objective is to improve the safety of the listener.

23. The method of claim 10, wherein the objective is to improve the work performance of the listener.

24. The method of claim 10, wherein the objective is to influence or alter the psychological state of the listener.

25. The method of claim 10, wherein the objective is to influence or alter the physiological state of the listener.