MX2007006051A - Methods, apparatus and system for film grain cache splitting for film grain simulation - Google Patents

Abstract

The present invention provides a method, apparatus and system for film grain cache splitting for film grain simulation. In one embodiment of the present invention a method for storing film grain patterns includes storing at least a first portion of film grain patterns in an internal memory and storing at least a secondportion of the film grain patterns in an external memory. That is, in the present invention a method for film grain cache splitting for film grain simulation includes splitting the storage of film grain patterns between an internal cache and an external memory. In one embodiment of the present invention, the internal cache is integrated into an integrated circuit chip of a decoder.

Description

METHODS, APPARATUS AND SYSTEM FOR DIVISION OF COPY OF FILM GRAIN FOR GRAIN SIMULATION MOVIE Cross Reference with Related Requests The application claims the priority of U.S. Provisional Patent Application Serial No. 60 / 630,049 filed on November 22, 2004, which is hereby incorporated by reference in its entirety.

Field of the Invention The present invention, in general, relates to the simulation of film grain and more particularly, to methods and systems for economical, and efficient, film grain simulation implementations.

BACKGROUND OF THE INVENTION The film grain is formed into film images during the development process. The film grain is clearly visible in high definition (HD) images and becomes a distinctive consideration of cinema that is very desirable to preserve throughout the image processing and delivery chain. However, the conservation of the film grain is a challenge for the current encoders since the compression gains related to the temporal prediction can not be exploited. Due to the random nature of the grain, encoding without visual loss can only be achieved at high bit rates. Loss coders tend to suppress film grain when they are filtered at high frequencies typically associated with noise and fine textures. In the newly created H-264 / MPEG-4 AVC video compression standard, and in particular, the I Amendment of fidelity interval extensions (FRExt) (JVT-K051, ITU-T Recommendation H.264 / ISO / IEC 14496-10 International Standard with Amendment 1, Redmond, USA, June 2004), a complementary improvement information (SEI) message has been defined. Such a message describes the characteristics of the film grain with respect to attributes such as size and intensity and allows the video decoder to simulate the view of the film grain on a decoded image. The H.264 / MPEG-4 AVC standard specifies the parameters present in the SEI message of film grain, how to interpret them and the syntax to be used to encode the SEI message in a binary format. However, the standard does not specify the exact procedure to simulate the film grain after receiving the SEI message of film grain. Film grain simulation is a relatively new technology used in post-production to simulate film grain in a computer-generated material, as well as during the restoration of old films. For these applications, there is a software called Cineon® from Eastman Kodak Co, Rochester, NY and Grain Surgery ™ from Visual Infinity. These tools usually operate on the basis of interaction and are complex to implement, which makes them unsuitable for real-time coding applications. In addition, none of these tools has the ability to interpret the SEI message of film grain characteristics as specified by the H.264 / AVC video coding standard.

Brief Description of the Invention The present invention provides a method, apparatus and system for dividing the buffer of the film grain for the simulation of film grain. In one embodiment of the present invention, a method for storing the film grain patterns includes storing at least a first portion of the film grain patterns in an internal memory and storing at least a second portion of the grain patterns. of film in an external memory. In an alternative embodiment of the present invention, an apparatus for the simulation of film grain includes a means for receiving at least one encoded image and complementary information including the characterization information of the film grain for use in the simulation process of film grain, an internal storage means for storing at least a first portion of the film grain patterns and an external storage means for storing at least a second portion of the film grain simulation patterns. In an alternative embodiment of the present invention, a system for simulating the film grain includes a decoder for receiving at least one encoded image and a complementary information message including the characterization information of the film grain to be used in the process of film grain simulation, an internal storage medium for storing at least a first portion of the film grain patterns, an external storage medium for storing at least a second portion of the film grain simulation patterns, where the internal storage medium is located in the decoder.

BRIEF DESCRIPTION OF THE DRAWINGS The teachings of the present invention will be better understood by considering the following detailed description in conjunction with the accompanying drawings, in which: Figure 1 illustrates a high-level block diagram of a video decoder subsystem having capabilities simulation of film grain according to one embodiment of the present invention; and Figure 2 illustrates a high-level block diagram of a typical arrangement of the film grain database of Figure 1. It should be understood that the drawings are intended to illustrate the concepts of the invention and not necessarily they are only possible configuration to illustrate the invention. To facilitate understanding, identical reference numbers have been used, where it is possible to point out the identical elements that are common in the Figures.

Detailed Description of the Invention The present invention advantageously provides methods, apparatus and systems for dividing the buffer of the film grain for the simulation of the film grain. Although the present invention is mainly described within the context of a subsystem of the video decoder for application, for example the IC designs for consumer HD DVD players, the specific embodiments of the present invention will not be treated as limiting the scope of the invention. . Those skilled in the art will be able to appreciate and understand from the teachings of the present invention, that the concepts thereof can be applied, with advantage in any film grain simulation process, for example, in media players / receiver devices, decoders, transcoders, television sets and the like. Figure 1 illustrates a high-level block diagram of a video decoding subsystem having film grain simulation capabilities in accordance with one embodiment of the present invention. The video decoder sub-system 100 of Figure 1 illustratively comprises a video decoder (illustratively an H.264 decoder), a video display and a graphics engine and a film grain simulator 108, a host interface 110 , an interface controller 112 (illustratively a RAM interface controller), and a memory 114 (illustratively an external RAM) implemented as a film grain buffer to store at least a small subset of grain patterns of film from the 104 database of remote film grain. The video display and the graphics engine and the film grain simulator 108 of Figure 1 illustratively also comprise internal storage capabilities illustrated as the internal film grain buffer 109. Although in Figure 1, the internal film grain buffer 109 is illustrated as located in the video display and graphics engine and the film grain simulator 108, in alternative embodiments of the present invention, the internal memory of The film grain of the present invention can be located inside the video decoder 106 and other components of the video decoder sub-system 100 of Figure 1. Figure 1 also illustrates a host CPU 102 and a permanent storage program memory 104 ( illustratively a remote permanent storage memory) comprising a film grain database. Although in the video decoder sub-system 100 of Figure 1, the host CPU 102 and the remote film grain database 104 are illustrated as comprising separate components, in alternative embodiments of the present invention, the database 104 of remote film grain can be located in the permanent memory of the CPU 102. In addition, although in the 100 video decoder sub-system of Figure 1, the video decoder 106, the video display and the graphics engine 108, the host interface 100 and the interface controller 112 are illustrated as comprising separate components, in alternative embodiments of the present invention, the video decoder 106, the video display and graphics engine 108, host interface 100 and video controller 112 may comprise a single component and may be integrated into a single integrated microcircuit (SoC) system. In such an embodiment, the video decoding subsystem 100 of FIG. 1 will comprise an internal buffer 109 of microcircuit film grain and an external film grain buffer 114. Further, although in the video decoder sub-system 100 of Figure 1, the means for storing the film grain patterns are illustrated as the external RAM memory 114 (buffer), the internal buffer memory 109 and a base 104 of remote film grain data, in alternative embodiments of the present invention, essentially any accessible storage medium can be implemented to maintain a sub-group of the film grain patterns and the total number of film grain patterns. Such means may include storage disks, a magnetic storage medium, an optical storage medium and essentially any storage medium. In addition, one or more storage media can be implemented for each of the storage devices. Also, although the film grain database 104 of FIG. 1 is shown as being remote from the external RAM memory 114 and the internal buffer memory 109, in alternative embodiments of the present invention, the storage medium of FIG. Film grain patterns can be located near or at a greater distance from each other. In film grain simulation systems such as the video decoder sub-system 100 of Figure 1, the remote film grain database 104 is typically very large. In one embodiment of the present invention, the H.264 video decoder 106, the video display and the graphics engine 108, the host interface 110, the interface controller 112 and the external RAM memory 114 comprise components of an HD player. DVD. The film grain patterns from the remote film grain database 104 are necessary to access a sample rate for example, from the HD DVD player. Therefore, quick access to the film grain database 104 is necessary. In a video decoder sub-system 100 of Figure 1, in accordance with the present invention, only a small portion of the remote film grain database 104 is used during the film grain periods of the improvement information. complementary (SEI), which are leveled to develop a buffer technique to reduce complexity. More specifically, the film grain simulation process of Figure 1 requires the decoding of the SEI messages of the film grain, transmitted in 10-bit currents, of the International Standard ITU-T Rec. H. 264 IISO / IED 14496, as specified in Amendment 1 (Fidelity Interval Extensions), which are included here as a reference in their totals. In one embodiment of the present invention, the film grain SEI messages are sent with preceding (intra-encoded) images I and only the film grain SEI message precedes a particular image I.

In one embodiment of the present invention and in accordance with the specifications of the standard, the remote film grain database 104 of the film grain patterns is composed of 169 patterns of 4,096 film grain samples, each representing an image of 64x64 film grain. For example, Figure 2 illustrates a high-level block diagram of a typical array of a film grain database of Figure 1. Figure 3 illustrates a film grain pattern of 64x64 samples with an i_displacement in the x axis and j_ displacement on the y axis. Figure 2 also illustrates the 169 film grain patterns of various types. In the film grain 104 database, each film grain pattern is synthesized with the use of a different pair of cutoff frequencies in accordance with a frequency filtration model of the specifications of the standard. The cutoff frequencies transmitted in the SEI message are used to obtain access to the remote film grain database 104 of the film grain patterns during the film grain simulation process. The film grain database 104 is stored in the ROM, in the Flash or in another permanent storage device, such as the film grain database 104 of the video decoder sub-system 100 of Figure 1, and it typically does not change. The 04 grain of film database contains random film grain patterns in a wide variety of film grain shapes and sizes. However, for a specific video content sequence, only a small sub-group of this database is needed to effectively simulate the film grain. The specification limits the number of film grain patterns to a small subgroup for any SEL message period. Therefore, the present invention implements small intermediate film grain stores, such as external RAM memory 114 and internal memory 109 buffer memory, which is updated upon receiving the SEL messages Typically, the remote film grain database 104 is stored in the permanent storage of the host CPU 102 or at the site of the host CPU 102. However, it is the video decoder 106 and the video display and the graphics engine 108 that need quick access to the film grain 104 database. As such, and in accordance with the present invention, the external memory 114 and the internal buffer 109 are provided for quick access to at least one subset of film grain patterns. That is, at least a small sub-group of the necessary or more implemented film grain patterns for the SEI message period are transferred and stored in the external memory 114 and in the internal buffer 109 as described below . More specifically, in accordance with the present invention, a solution that minimizes the overall cost of the design of a film grain simulation system, such as the video decoder sub-system 100 of Figure 1 is to divide the storage of the film grain patterns between the internal buffer of IC 109 of the decoder and the remaining external memory 114. For example, in one implementation, where a total of 10 film grain patterns are to be stored, when the internal buffer 109 stores N film grain patterns, then the external memory 114 stores the 10-N grain patterns of remaining film. By dividing the storage of the film grain patterns between the internal buffer 109 and the external memory 114 in accordance with the present invention, it provides for reduced internal memory size requirements, resulting in a reduced microcircuit area and a width of average and reduced memory band over solutions that have only an external memory to store the film grain patterns. In various embodiments of the present invention, the memory bandwidth (BW) required for the simulation of film grain in accordance with the present invention, can be reduced to zero since not all stored film grain patterns are used for a specific film content. . In embodiments of the present invention, different buffer divisions may be used to store the necessary film grain patterns. That is, in accordance with the present invention, any division is possible. The more patterns of film grain are stored in the internal buffer 109, the lower the probability that BW memory will be needed in the worst case. Furthermore, since not all the film grain buffer is necessary during a given content simulation, in many cases, the BW memory is significantly reduced. For example, in one embodiment of the present invention where ten (10) film grain patterns are to be stored, when half (5) film grain patterns are stored in the internal buffer, such as the buffer 109 of the subsystem 100 of the video decoder of Figure 1, then the size of the internal memory is half the total memory required to store the 10 film grain patterns. In such embodiment of the present invention, the memory bandwidth for most of the content is reduced below 36 Mbytes / sec, and in some cases, it will be much smaller. When in the example described above, only one (1) film grain pattern of ten is to be stored in the internal buffer (for example, N = 1), then only a small amount of the internal buffer is needed in such embodiment of the present invention. Such an embodiment of the present invention only requires a very small additional area to provide the buffer required to store only one film grain pattern. In such an embodiment, the BW memory can be greatly reduced by a significant amount, since the most frequently implemented film grain pattern can be placed in the internal buffer. In an alternative embodiment of the present invention, an internal buffer and an external memory are implemented to store the luma and chroma components separately. That is, the luma can be placed in the internal buffer, while the chroma can be placed in the external memory. In this embodiment of the present invention, it is guaranteed that in the worst case, the BW memory for the simulation of film grain is 36 Mbytes / sec, (only chroma) and the size of the internal buffer only needs to accommodate the portion luma of the buffer. However, such embodiments of the present invention require that the specifications of the film grain simulation include a definition of the division between the luma buffer size and the chroma buffer size to configure the internal buffer and the external memory. In one embodiment of the luma / chroma division of the present invention, where only one chroma component is stored in the external memory, the BW memory is decreased to 10 Mbytes per second. Such a modality requires more internal buffering bolt less than a maximum. In another embodiment of the present invention, the SEI message of the film grain simulation process includes additional information indicating a priority order for the stored film grain patterns. This order of priority is used for example, by the video decoder subsystem 100 of Figure 1, to store the most frequently required film grain patterns in the internal buffer of the IC of the decoder, therefore optimizing the use of the internal buffer and minimizes the external BW memory. For film grain simulation processes, this can be achieved with a new syntax element SEI, characterized by the equation one (1) as follows: Fg_patron_priority_pecifies the pairs (h, v) of the cutoff frequencies in order of priority. (h, v) = (comp_model_value (j) (i) (1), comp_model_value (j) (i) (2)). (1) In another embodiment of the present invention, a priority order of the film grain patterns is derived from a standardized film grain SEI message. That is, since the SEI message contains a list of intensity ranges, each with its own film grain parameters, the intensity ranges can be listed in accordance with its priority (instead of being listed with intensity range limits). in increment). It should be noted that this change is compatible with the H.264 / MPEG AVC standard. Then, for each color component, the first N film grain patterns are stored in the internal buffer, since those first N film grain patterns are the most implemented film grain patterns. In addition, rules can be generated to give priority among color components. For example, until the first N / 2 and patterns of film grain, until the first N / 4 U patterns film grain, and to the first patterns N / 4 V grain patterns of film are placed in the internal buffer , while the remaining film grain patterns are stored in the external memory. Having described various modalities for the methods, apparatuses and systems for the division of film grain buffer for the simulation of film grain (which are intended to be illustrative and not limiting), it should be noted that persons experienced in the can make changes and modifications in light of these teachings. Therefore, it should be understood that the changes can be made in particular embodiments of the disclosed invention, which are within the scope and spirit of the invention, as indicated by the appended claims. While the foregoing is directed to various embodiments of the present invention, other embodiments of the invention may be contemplated without departing from the basic scope thereof. As such, the appropriate scope of the invention is determined in accordance with the claims that follow.

Claims (21)

1. A method for storing film grain patterns, characterized in that it comprises: storing at least a portion of the film grain patterns in an internal memory; and storing at least a second portion of the film grain patterns in an external memory.

The method according to claim 1, characterized in that the internal memory is located in a video decoder.

The method according to claim 1, characterized in that the internal memory is located in a microcircuit integrated in the video decoder.

The method according to claim 3, characterized in that only a small portion of the film grain patterns are stored in the internal memory, such that only a small additional area of microcircuit is required to provide the internal memory.

The method according to claim 1, characterized in that the internal memory stores at least the film grain patterns implemented more in the film grain simulation process.

The method according to claim 1, characterized in that the internal memory and the external memory together store the film grain patterns required in the film grain simulation process.

The method according to claim 1, characterized in that the luma components and the chroma components of the film grain patterns are stored separately in an internal memory and in an external memory.

The method according to claim 7, characterized in that the luma components are stored in the internal memory and the chroma components are stored in the external memory.

The method according to claim 1, characterized in that the order of priority for storing the film grain patterns in the internal memory and the external memory is derived from a complementary information message of standardized film grain.

A method for storing film grain patterns for a film grain simulation process, characterized in that it comprises: dividing the storage of the film grain patterns between an internal buffer and an external memory.

The method according to claim 10, characterized in that the internal buffer is located in a video decoder.

The method according to claim 10, characterized in that the buffer is located in a microcircuit integrated in the video decoder.

13. The method according to claim 12, characterized in that only a small portion of the film grain patterns are stored in an internal buffer, so that only a small additional microcircuit area is required to provide the internal buffer.

The method according to claim 10, characterized in that the internal buffer memory stores at least the film grain patterns most implemented in the film grain simulation process.

The method according to claim 10, characterized in that the order of priority for storing the film grain patterns in the internal memory and in the external memory is derived from a complementary information message of standardized film grain.

16. An apparatus characterized in that it comprises: means for receiving at least one encoded image and complementary information including film grain characterization information for use in the film grain simulation process; an internal storage means for storing at least a first portion of the film grain patterns; and an external storage means for storing at least a second portion of the film grain simulation patterns.

The apparatus according to claim 15, characterized in that the means for receiving comprises a decoder.

18. The apparatus according to claim 16, characterized in that the internal storage means comprises an internal buffer.

The apparatus according to claim 16, characterized in that the internal storage medium comprises at least more film grain patterns implemented in the film grain simulation process.

20. A system for simulating the film grain, characterized in that it comprises: a decoder for receiving at least one encoded image and a complementary information message including the film grain characterization information for use in a grain simulation process of film; an internal storage means for storing at least a first portion of the film grain patterns; and an external storage means for storing at least a second portion of the film grain simulation patterns.

21. The system according to claim 20, characterized in that the internal storage means is located in the decoder.