JP5482394B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus for processing image data used when rendering by applying a texture.

  Conventionally, as this type of image processing method, a three-dimensional model is rendered in real time and displayed on a display (see, for example, Patent Document 1), and a three-dimensional model is rendered in advance to create a bitmap image. It has been proposed to save and display the display by reading a bitmap image.

Japanese Patent Application Laid-Open No. 07-152925

  In the former method, since it is necessary to perform the rendering process at a cycle shorter than the display cycle of the screen, a high calculation capability is required. Therefore, depending on the computer to be used, there is a shortage in computing ability, and high quality rendering such as ray tracing cannot be performed. On the other hand, since the latter method only displays a bitmap image, it is possible to display a high-quality image by creating a bitmap image by performing high-quality rendering in advance. Then you can't replace it with a different texture later. Further, such image data is large in size and is required to be managed efficiently depending on the capacity of the mounted memory.

An image processing method and an image processing apparatus according to the present invention are mainly intended to efficiently manage a rendering image of a three-dimensional model.
The image processing method and the image processing apparatus of the present invention employ the following means in order to achieve the main object described above.

  According to the image processing method of the present invention, the correspondence between the coordinates of the rendered image obtained by pasting the texture on the three-dimensional model and the coordinates of the texture, the color of each pixel of the rendered image, and the pixel of the texture Image drawing information indicating a correspondence relationship between colors is acquired, and the image drawing information indicating a correspondence relationship between the coordinates of the rendered image and the coordinates of the texture is compressed by a linear compression method.

  In the image processing method of the present invention, among the image drawing information indicating the correspondence between the texture and the rendered image, at least the image drawing information indicating the correspondence between the coordinate of the rendered image and the coordinate of the texture is linear. Compress using the compression method. Thereby, the rendered image is managed efficiently and while suppressing the deterioration of the image quality.

  In such an image processing method of the present invention, the image drawing information indicating the correspondence between the color for each pixel of the rendered image and the color for each pixel of the texture may be a JPEG compression method.

  In the image processing method of the present invention, data may be compressed by linearly approximating one of the values of the rendered image coordinates and the texture coordinates by triangulation. In this way, the compression rate can be increased by a relatively simple process while suppressing the deterioration of the image quality.

  In the image processing method of the present invention, a predetermined pattern in which different gradation values are set for each coordinate is pasted and rendered as a texture on a three-dimensional model, and a rendered image obtained as a bitmap image by the rendering is analyzed Thus, the correspondence between the coordinates of the rendered image and the coordinates of the texture is set and saved as image drawing information, and when displaying a desired texture as an image, the rendering is performed based on the saved image drawing information. The desired texture may be arranged and displayed in the image. By doing so, it is possible to display an image obtained by rendering the three-dimensional model by replacing a desired texture, and it is possible to reduce the processing burden as compared with the case where the three-dimensional model is rendered and displayed in real time. Here, the display of the image includes an image that is drawn in units of frames and displayed as a moving image. In this aspect of the image processing method of the present invention, the correspondence relationship may be derived by specifying the corresponding coordinates of the predetermined pattern from the gradation value of each coordinate of the rendered image. In the image processing method of the present invention of these aspects, the predetermined pattern is a pattern having the same number of bits when the texture coordinates are expressed in binary numbers, and each pattern has coordinates expressed in binary numbers. It is also possible to associate each bit of, and set the gradation value of each coordinate of each pattern to a value corresponding to the value of the associated bit. In this way, the correspondence can be set more accurately. In this case, the binary number may be a gray code (alternate binary number). In this way, since only 1-bit change always occurs when moving to an adjacent coordinate, it is possible to suppress erroneous data from being acquired due to an error in the gradation value of the image. In the image processing method of the present invention according to these aspects, in addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, the first solid pattern formed by solid painting with a minimum gradation value is further described. A bias value, which is a gradation value of the first solid pattern in the rendered image, is rendered by pasting on a three-dimensional model, and the correspondence relationship between the color of each pixel of the rendered image and the color of each pixel of the texture It is also possible to store the image drawing information as shown and to convert the gradation value of the desired texture based on the stored bias value into the gradation value of the rendered image for display. . By doing this, it is possible to reflect the effect of rendering the three-dimensional model that does not depend on the original texture. In this case, the bias value may be compressed by a linear compression method. Furthermore, in the image processing method of the present invention of these aspects, in addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, a first solid pattern that is further solid-coated with a minimum gradation value And the second solid pattern formed by applying a solid color with the maximum gradation value are pasted on the three-dimensional model and rendered, respectively, and the gradation value of the second solid pattern and the first solid pattern in the rendered image are rendered. A gain that is a deviation from the gradation value of the paint pattern is calculated and stored as the image drawing information indicating the correspondence between the color of each pixel of the rendered image and the color of each pixel of the texture, and the saved gain is stored in the stored gain. Based on this, the gradation value of the desired texture may be converted into the gradation value of the rendered image and displayed. In this way, it is possible to reflect the effect of the original texture gradation value among the effects of rendering the three-dimensional model. In this case, when n (n is a natural number) textures are arranged and displayed on the rendered image, (n-1) first solid coating patterns and one second solid coating pattern are displayed. And a first set group consisting of n sets of first solid coating patterns, each of which has a different location where the second solid coating pattern is applied to the three-dimensional model. 2 sets are rendered on the 3D model for each set and rendered, and the first set group is rendered for each set. By comparing the gradation value of the rendered image obtained by rendering the set with each set of the first set group, the three-dimensional model Identifying a texture region which is a texture pasted region can also be made to calculate the gain with respect to the specified texture region. In this way, the texture area can be specified more easily. When the texture and the rendered image are composed of a plurality of color components (for example, three colors), the first set group and the second set are pasted and rendered on the three-dimensional model for each color component, and each color component is rendered. What is necessary is just to calculate a gain.

  The image processing apparatus according to the present invention provides a correspondence relationship between coordinates of a rendered image obtained by pasting a texture on a three-dimensional model and the coordinates of the texture, a color for each pixel of the rendered image, and a pixel for each pixel of the texture. Image drawing information acquisition means for acquiring image drawing information indicating a correspondence relationship with a color, and compression means for compressing the image drawing information indicating a correspondence relation between the coordinates of the rendered image and the coordinates of the texture by a linear compression method And a gist of the above.

  In the image processing apparatus of the present invention, among the image drawing information indicating the correspondence between the texture and the rendered image, at least the image drawing information indicating the correspondence between the coordinate of the rendered image and the coordinate of the texture is linear. Compress using the compression method. As a result, the rendered image can be managed efficiently and while suppressing deterioration in image quality.

Explanatory drawing explaining a special texture. Explanatory drawing explaining a special texture. Explanatory drawing of a non-texture area | region and a texture area | region. The block diagram which shows the outline of a structure of the computer 20 used for an image processing method. The flowchart which shows an example of a special texture production | generation process. Explanatory drawing which shows an example of a special texture. Explanatory drawing which shows a mode that a special texture is rendered for every set. 10 is a flowchart illustrating an example of rendered image analysis processing. Explanatory drawing explaining bias Bc, t (x, y) and gain Gc, t (x, y). The flowchart which shows an example of a compression process. The flowchart which shows an example of a linear compression process. Explanatory drawing which shows the relationship between the plane (alpha) and the pixel C. FIG. Explanatory drawing which shows the relationship between plane (beta) and the pixel P. FIG. Explanatory drawing which shows a mode that a point set G = {A, B, C, P} is triangulated Explanatory drawing which shows the mode of linear compression. Explanatory drawing which shows the texture for three replacements. Explanatory drawing which shows an example of the slide show display of the texture for replacement. Explanatory drawing which shows the special texture of a modification. Explanatory drawing which shows a mode that it renders using the special texture of a modification. Explanatory drawing which shows the special texture of a modification.

Next, an outline of an embodiment of the present invention will be described.
One embodiment of the present invention is a two-dimensional image (a two-dimensional image) showing a state in which a two-dimensional texture is pasted on a virtual three-dimensional structure and the pasted three-dimensional structure is viewed from a predetermined direction. Hereinafter, the technique is related to a rendering process for generating a rendered image). That is, an embodiment of the present invention is a mode for providing a compression method for efficiently managing image data handled when the rendering process is performed. In the compression method, a texture is pasted by the rendering process. This is applied when a rendered image in which a texture area coexists with an area where the texture is not pasted is generated.

  In the present embodiment, in order to enable the above-described rendering processing with fewer resources, the rendered image after the texture is pasted on the three-dimensional model is compared, and the rendered image is generated from the texture. The coefficient is defined in advance as image drawing information.

This coefficient is a mathematical expression that defines the correspondence between the pixel position of an arbitrary texture and the pixel position of the rendered image and the color change between the pixels, and is calculated through the following steps.
(A) Identification of texture to be pasted (B) Identification of influence of rendering processing on luminance (C) Identification of correspondence of pixel positions These steps are not for an arbitrary texture but for specifying the above-mentioned coefficients This is carried out using a predetermined pattern texture (hereinafter referred to as a special texture). Note that the size (number of vertical and horizontal pixels) of the special texture is set to the same number as the above-described two-dimensional texture. Hereinafter, the outline of (A) to (C) will be described.

(A) Identification of Texture to be Pasted FIG. 1A is a diagram for explaining an outline of rendering processing. In the rendering processing in the present embodiment, assuming that n (n is a natural number) two-dimensional texture is pasted on each surface of a virtual three-dimensional structure, the texture is pasted. A two-dimensional image obtained by viewing the three-dimensional structure from a predetermined direction is set as a rendered image. Therefore, in the rendered image, a maximum of n textures are visually recognized according to the direction of the three-dimensional structure. In FIG. 1A, a rendered image when a two-dimensional texture T _{1 to} T ₆ is pasted on a virtual rectangular parallelepiped R is shown as I ₀ , and six textures T _{1 to} T ₆ are displayed. Of these, an example is shown in which _three textures T _{1 to} T ₃ are visually recognized. In the example of FIG. 1A, it is assumed that a virtual rectangular parallelepiped R is placed on a table, and a shadow S of the rectangular parallelepiped R is formed on the table.

  Since such a rendering process is realized by pasting a specific texture on a specific surface of a three-dimensional structure, a single texture always corresponds to a certain coordinate of the rendered image, and a plurality of textures There is no correspondence. That is, there is a one-to-one relationship between the coordinates of the rendered image and the coordinates of the texture. Therefore, in the present embodiment, n special textures are generated in which only one specific sheet is white and other than the specific one is black, and further, a special texture in which all n sheets are black is generated. Then, by comparing the rendered images, the position of the rendered image corresponding to the white special texture which is the above-mentioned specific sheet is specified. Here, the white special texture corresponds to the second solid pattern, and the black special texture corresponds to the first solid pattern.

FIGS. 1B to 1D show examples of special textures that are pasted in place of the textures T _{1 to} T ₆ shown in FIG. 1A with six rectangles shown on the right side. In FIG. 1B, the texture T ₁ is shown. only special texture corresponding white, special texture corresponding to the texture T ₂ through T ₆ indicates an example in which black. FIG. 1C shows an example in which the special texture corresponding to the textures T _{1 to} T ₆ is black. The rendered images generated by the rendering process based on the special texture shown in FIGS. 1B and 1C are shown as I ₁ and I ₂ , respectively.

In this embodiment, the position of the rendered image corresponding to the special texture of the second solid pattern is specified by comparing the rendered images I ₁ and I ₂ generated as described above. That is, when the luminances of the rendered images I ₁ and I ₂ are compared for each pixel of the rendered image, the luminance is different in the pixels to which the special texture of the second solid pattern is pasted, but the first solid color There is no difference in the pixels to which the special texture of the pattern is pasted. Therefore, for example, if the process of determining whether the luminance of the rendered image I ₁ is greater than the luminance of the rendered image I ₂ is performed for each pixel, the luminance of the rendered image I ₁ is that of the rendered image I ₂ . For a pixel determined to be larger than the luminance, it can be determined that the special texture of the second solid coating pattern is attached.

That is, as shown in FIG. 1B, the rendered image I ₁ when the texture T ₁ is rendered as a special texture of the second solid pattern, and all the textures as the first solid as shown in FIG. 1C. The position of the rendered image corresponding to the texture T ₁ is specified by comparing the rendered image I ₂ when the rendering process is performed as the special texture of the paint pattern. Similarly, as shown in FIG. 1D, by comparing the rendered image I ₃ and the rendered image I ₂ when the rendering process is performed using the texture T ₂ as the special texture of the second solid pattern, the texture T ₂ is compared. The position of the rendered image corresponding to ₂ is specified. Therefore, if the above processing is performed for each rendered image obtained by rendering a special texture in which any one of n textures is white and the others are black, the position where n textures are pasted in the rendered image Is identified.

(B) Identification of Effect of Rendering Process on Brightness Next, the influence of the rendering process on luminance is specified for each pixel of the rendered image. As shown in FIG. 1A, the rendering process is performed by pasting each texture T _{1 to} T ₆ on a virtual three-dimensional structure, and then the light from the light source set at the virtual position or the three-dimensional structure. The brightness value of the rendered image is specified on the assumption that the image is affected by a predetermined shadow or the like due to the shape of the structure. Therefore, the predetermined influence can be defined for each pixel of the rendered image. Further, in the present embodiment, the luminance value of each pixel of the rendered image, constant component that does not depend on the texture T ₁ through T ₆ (hereinafter, referred to as bias B) to be proportional to the brightness of the texture T ₁ through T ₆ It is considered to be defined by the sum of the proportional component (hereinafter referred to as gain G).

Therefore, if the bias B and the gain G are specified for each pixel of the rendered image, the influence of the rendering process on the luminance can be specified. As shown in FIG. 1C, this effect is caused by rendering the image I _{2 in} which all the n textures are replaced with the special texture of the first solid pattern and the rendering process is performed as shown in FIGS. 1B and 1D. One of the specified textures is specified using the special texture of the second solid pattern, and the other is replaced with the special texture of the first solid pattern, and the rendered images I ₁ and I ₃ are used for specification. be able to.

That is, in the rendered image I ₂ shown in FIG. 1C, all n sheets are special textures of the first solid pattern, and the special texture of the first solid pattern is pasted on a virtual three-dimensional structure. If each pixel of the rendered image generated in this way has a significant luminance value (a luminance value greater than 0), it can be seen that the luminance value is a significant value without being affected by the original texture. . Therefore, the luminance value at each pixel of the rendered image generated by pasting the special texture of the first solid pattern on the virtual three-dimensional structure can be defined as the bias B.

  On the other hand, the gain G can be specified by subtracting the bias B from the luminance value at each pixel of the rendered image corresponding to the portion to which the special texture of the second solid pattern is pasted. Therefore, the gain G of each pixel is specified by subtracting the bias B from each pixel of the rendered image corresponding to the position where the special texture of the second solid pattern is pasted.

(C) Identifying correspondence between pixel positions Next, correspondence between pixel positions before and after rendering is identified. That is, the correspondence relationship between the position of each pixel of the texture and the position of each pixel of the rendered image is specified. The correspondence is specified by associating the coordinates (x, y) of the rendered image with the coordinates (X, Y) of the texture based on the rendered image obtained by rendering the special texture. However, in this embodiment, the correspondence is specified by specifying the correspondence between the x coordinate and the X coordinate and performing the process of specifying the correspondence between the y coordinate and the Y coordinate. A special texture (corresponding relationship setting pattern) for performing each is generated.

2A to 2C are diagrams illustrating examples of special textures generated for specifying the correspondence between the x coordinate and the X coordinate. In these drawings, a coordinate system is adopted in which the position of the pixel in the horizontal direction is defined by the X coordinate and the x coordinate, and the position of the pixel in the vertical direction is defined by the Y coordinate and the y coordinate. Further, here, since a texture having a value range of 1 to 8 is assumed for both the X coordinate and the Y coordinate, the special texture also has a value range of 1 to 8 for both the X coordinate and the Y coordinate. In FIG. 2A, the value of the X coordinate in one of the special textures is indicated by an arrow. The special texture generated for specifying the correspondence between the x coordinate and the X coordinate is the same pattern for all n textures. For example, the example shown in FIG. 2A shows an example of a special texture generated when ₆ textures T _{1 to} T ₆ are pasted on a virtual three-dimensional structure as shown in FIG. 1A. In the example shown in FIG. 2A, the patterns of the special textures that are used instead of the _six textures T _{1 to} T ₆ are all the same.

  However, in the special texture generated to specify the correspondence between the x coordinate and the X coordinate, the same number of patterns as the number of bits of the X coordinate are generated. For example, in the example illustrated in FIGS. 2A to 2C, the X coordinate range is 1 to 8 and 3 bits, so three types of special textures are generated. That is, the special texture patterns are different in each of FIGS. 2A, 2B, and 2C, but when compared with the special textures shown in FIG. 2A, the n special texture patterns are the same.

  Furthermore, in this embodiment, the patterns of the three types of special textures are different so that the luminance permutation obtained by sequentially extracting the luminances of the same X coordinate values from the three types of special textures is different in all the X coordinate values. Is set. For example, the permutation obtained by extracting the luminance of the X coordinate value 1 in the order of FIGS. 2A, 2B, 2C is (black, black, black), and the luminance of the X coordinate value 2 is extracted in the order of FIGS. 2A, 2B, 2C. The permutation obtained is (white, black, black). As described above, when the pattern of the special texture is set so that the permutations of all the X coordinate values are all different, the three types of rendered images at the respective coordinates after the rendering processing of the three types of special textures are performed. Based on the luminance value, the X coordinate value of the original special texture can be specified. For example, if the permutation of the brightness of the original special texture is specified by sequentially referring to the brightness value at each coordinate of three types of rendered images, the X coordinate value of the special texture corresponding to each coordinate of the rendered image is specified. can do.

More specifically, in the example shown in FIGS. 2A to 2C, the permutation of the brightness of the original special texture specified by sequentially extracting the brightness values at the position P ₁ of the rendered images I ₄ , I ₅ , and I _6. Is (black, black, black). This permutation corresponds to the brightness of the X coordinate value 1 of the original special texture, and the brightness of other X coordinate values does not correspond to this permutation. Therefore, it can be specified that the image of the X coordinate value 1 of the special texture is pasted at the position P ₁ of the rendered image from which this permutation has been extracted.

  Therefore, in this embodiment, n special textures of the same number as the number of bits of the X coordinate of the special texture are generated and rendered, and any of the n textures is pasted according to (A) above. The luminance value at each position of the rendered image that is specified to be attached is extracted in order. Then, by specifying the X coordinate value of the original special texture based on the extracted permutation, the correspondence relationship between the x coordinate and the X coordinate is specified.

  Such a correspondence relationship between the x-coordinate and the X-coordinate is preferably specified by, for example, expressing the X-coordinate value by a gray code. The special texture patterns shown in FIGS. 2A to 2C are patterns used when the correspondence between the x coordinate and the X coordinate is specified using the gray code. That is, in these examples, in order to specify the correspondence between the x coordinate and the X coordinate using the gray code, the value of the least significant bit when the X coordinate value is expressed by the gray code is 0. The pattern shown in FIG. 2A is generated by setting the case of black 1 to white. Similarly, the pattern shown in FIG. 2B is assumed to be black when the value of the bit one higher than the least significant bit (referred to as the middle bit) in the case where the X coordinate value is expressed in gray code is 0, and white when it is 1. The pattern shown in FIG. 2C is generated with black when the value of the most significant bit is 0 and white when 1. That is, the special texture pattern shown in FIG. 2A is determined based on the least significant bit value when the X coordinate value is expressed by a Gray code, and the special texture pattern shown in FIGS. It is determined based on the middle bit and the most significant bit value when expressed in code. That is, in the example shown in FIGS. 2A to 2C, the same number of patterns as the number of bits 3 are formed, and the least significant bit, the middle bit, and the most significant bit are associated with FIGS. 2A, 2B, and 2C, respectively. It has been.

  FIG. 2D is a diagram illustrating generation of the pattern. As shown in FIG. 2D, the Gray code indicating the X coordinate value 1 is (000), the least significant bit is 0, the middle bit is 0, and the most significant bit is 0. Therefore, the brightness of the X coordinate value 1 of the special texture shown in FIG. 2A is black, the brightness of the X coordinate value 1 of the special texture shown in FIG. 2B is black, and the brightness of the X coordinate value 1 of the special texture shown in FIG. Become. Similarly, the Gray code indicating the X coordinate value 2 is (001), the least significant bit is 1, the middle bit is 0, and the most significant bit is 0. Therefore, the brightness of the X coordinate value 2 of the special texture shown in FIG. 2A is white, the brightness of the X coordinate value 2 of the special texture shown in FIG. 2B is black, and the brightness of the X coordinate value 2 of the special texture shown in FIG. Become. By continuing the same processing to the X coordinate values 3 to 8, the patterns of FIGS. 2A to 2C are generated.

  In this way, if the patterns of multiple types of special textures are determined by the Gray code, the brightness of the original special texture specified from the brightness value of each coordinate of the rendered image generated from the multiple types of special textures. Thus, it is possible to determine a value expressing the X coordinate value of the original special texture in the gray code. Here, the luminance of the original special texture specified from the luminance value of each coordinate of the rendered image is, for example, a value obtained by subtracting the above-described bias B from the luminance value of the rendered image is larger than ½ of the gain G. It can be specified by determining whether or not. That is, when the original special texture is black, the value obtained by subtracting the above-mentioned bias B from the luminance value of the rendered image is almost 0, and when the original special texture is white, the luminance value of the rendered image is The value obtained by subtracting the bias B is almost the same as the gain G. Therefore, if the value obtained by subtracting the bias B at each position from the luminance value at each position of the rendered image is larger than ½ of the gain G at each position, the original special texture is white, and the luminance value at each position of the rendered image. If the value obtained by subtracting the bias B at each position is equal to or less than ½ of the gain G at each position, the original special texture can be regarded as black.

  Therefore, the rendered image is generated by rendering the special texture generated based on the least significant bit value when the X coordinate value is expressed in gray code, and the original special value is obtained from the luminance value of each coordinate of the rendered image. When it is determined that the texture is white, the least significant bit value of the X coordinate value of the gray code expression at that position is set to 1. When the original special texture is determined to be black from the luminance value of each coordinate of the rendered image, the least significant bit value of the X coordinate value of the Gray code expression at that position is set to 0. By performing the above processing for each of a plurality of types of special textures, it is possible to determine all bits of the X coordinate value of the gray code representation corresponding to each position of the rendered image, and as a result, the x coordinate And the X coordinate can be defined.

For example, in the example shown in FIG. 2A, the value obtained by subtracting the bias B (x, y) from the luminance value at the position P ₁ (x, y) of the rendered image I ₄ is ½ of the gain G (x, y). Since it is the following, it is determined that the original special texture is black. Accordingly, the least significant bit of the X coordinate value of the texture corresponding to the position P ₁ (x, y) is 0. Similarly, in the example shown in FIG. 2B, the value obtained by subtracting the bias B (x, y) from the luminance value at the position P ₁ (x, y) of the rendered image I ₅ is 1 / of the gain G (x, y). Since it is 2 or less, it is determined that the original special texture is black. Therefore, the middle bid of the X coordinate value of the texture corresponding to the position P ₁ (x, y) is zero. In the example shown in FIG. 2C, the value obtained by subtracting the bias B (x, y) from the luminance value at the position P ₁ (x, y) of the rendered image I ₆ is equal to or less than ½ of the gain G (x, y). Therefore, it is determined that the original special texture is black. Therefore, the most significant bit of the X coordinate value of the texture corresponding to the position P ₁ (x, y) is 0. Therefore, the Gray code representation of the X coordinate value corresponding to the position P ₁ (x, y) is (000), and the X coordinate value 1 is specified.

  The processing as described above identifies the permutation of the original special texture luminance based on the luminance value at each coordinate of a plurality of types of rendered images, and identifies the X coordinate value from the permutation, thereby rendering the rendered image. This is a process substantially equivalent to determining the X coordinate value corresponding to each coordinate. It is possible to specify the Y coordinate value corresponding to each coordinate of the rendered image by performing the same process for the Y coordinate value. That is, if a determination similar to the determination in the X coordinate value is performed with a special texture obtained by rotating the patterns shown in FIGS. 2A to 2C by 90 degrees, the correspondence between the Y coordinate value represented by the Gray code and the y coordinate It becomes possible to specify the relationship.

  If the steps (A) to (C) are executed as described above, the texture coordinates (X, Y) corresponding to the arbitrary coordinates (x, y) of the rendered image can be specified. Further, it is possible to specify the influence of the rendering process on the luminance at an arbitrary coordinate (x, y) of the rendered image. Therefore, if information indicating the correspondence between coordinates specified in each step and the effect of rendering processing on luminance is stored, processing for generating a rendered image from an arbitrary texture based on the stored information is performed. It is possible to execute at high speed with very few resources. In addition, even if the virtual three-dimensional structure assumed in the rendering process, the position of illumination, etc. are not clear, that is, only the image after rendering and the texture before processing are clear, the rendering process Can be reproduced.

  In the present embodiment, as described above, the correspondence relationship between the texture before rendering and the rendered image is expressed by a function. That is, the coordinate value (x, y) of the rendered image is used as a variable, and for each coordinate value, I (x, y) indicating the texture to be pasted, bias B (x, y), gain G (x, y) If the texture coordinates X (x, y) and Y (x, y) are defined as image drawing information, a rendered image can be generated from an arbitrary texture.

  Here, the bias B (x, y), the gain G (x, y), the coordinate X (x, y), and the coordinate Y (x, y) are bias B (x, y), This is a function in which the tone values of the gain G (x, y), the coordinate X (x, y), and the coordinate Y (x, y) change. In the present embodiment, rendering is performed by compressing these functions. The configuration is such that images are managed efficiently and while suppressing deterioration in image quality. That is, the bias B (x, y) and the gain G (x, y) are compressed at a high compression rate by the JPEG compression method, and the coordinates X (x, y) and the coordinates Y (x, y) are linear. The configuration is such that deterioration in image quality is suppressed by a compression method.

  Specifically, the coordinate X (x, y) indicates the correspondence between the texture coordinate X and the coordinate (x, y) of the rendered image, and the coordinate Y (x, y) indicates the coordinate Y of the texture. The correspondence relationship with the coordinates (x, y) of the rendered image is shown. Therefore, when an error occurs in the correspondence between coordinates due to the lossy compression method, an error occurs between the original position where the texture should be pasted and the position where the texture is actually pasted in the rendered image. Here, if the error changes abruptly between adjacent pixels of the texture, the quality of the rendered image is significantly degraded. However, if the error changes gradually for each pixel of the texture, the degradation of the quality of the rendered image is suppressed. Is done.

  Therefore, in the present embodiment, the coordinates X (x, y) and the coordinates Y (x, y) are compressed by a linear compression method. That is, for example, in the JPEG compression method, since an image is divided into MCUs (Minimum Coded Units) and subjected to compression processing, block noise in which an error rapidly changes at an MCU boundary may occur. However, in linear compression, compression is performed so that the gradation value changes linearly between adjacent pixels. Therefore, even if an error occurs due to compression, the error is gradually reduced for each texture pixel. Will change. Therefore, by compressing the coordinates X (x, y) and the coordinates Y (x, y) by the linear compression method, it is possible to suppress deterioration in image quality.

  The linear compression can be performed by various methods. For example, when the coordinate X (x, y) is expressed by a three-dimensional graph having the value of the coordinate X on the xy plane. It can be realized by expressing the unevenness in the height direction indicated by the value of the coordinate X on the graph as an approximate unevenness in which the entire surface is constituted by a polygon (for example, a triangle). In addition, when defining approximate unevenness so that all of the surfaces are triangular, the configuration is such that the triangle is determined so that the projection of the triangle onto the xy plane becomes a triangle divided by Delaunay triangulation. It can be adopted. The approximation can be realized by increasing the number of triangles by triangulation until the error between the approximate coordinate X value indicated by the triangle and the true value of the coordinate X is within a predetermined value.

  Next, embodiments of the present invention will be described together with specific examples with reference to the drawings. FIG. 4 is a configuration diagram showing an outline of the configuration of the computer 20 and the viewer 40 used in the image processing method according to the embodiment of the present invention. The computer 20 of this embodiment is a general-purpose computer comprising a central processing unit (CPU), a ROM for storing processing programs, a RAM for temporarily storing data, a graphic processor (GPU), a hard disk (HDD), a display 22 and the like. It is configured as a computer, and its functional blocks include 3D modeling data (hereinafter referred to as 3D model) indicating a virtual 3D structure and texture data (hereinafter referred to as texture) to be attached to this. A storage unit 31 for storing, a special texture generation processing unit 32 for generating a special texture for preprocessing to be pasted on the three-dimensional model, a rendering processing unit 34 for generating a bitmap image by rendering the three-dimensional model, Bits obtained by rendering Includes a rendered image analysis processing section 36 for analyzing the rendered image as a map image, and a compression processing unit 38 compresses the various kinds of data obtained by analyzing the rendered image and the rendered image analysis processing unit 36 which generated.

  The special texture generation processing unit 32 is a processing unit that generates a special texture to be pasted on the three-dimensional model rendered by the rendering processing unit 34. Specifically, the special texture generation processing unit 32 has a gradation value range of 0.0 to 1.0. Among them, a white solid pattern (second solid pattern) having a gradation value of 1.0, a black solid pattern (first solid pattern) having a gradation value of 0.0, and a value of 0.0 A vertical stripe pattern (correspondence setting pattern) in which gradation values of 1.0 appear alternately in the horizontal direction, and a horizontal stripe pattern in which gradation values of 0.0 and 1.0 appear alternately in the vertical direction. A pattern (correspondence setting pattern) is generated. The role of each pattern will be described later.

  The rendering processing unit 34 is a processing unit that functions when software for 3D rendering is installed in the computer 20. The rendering processing unit 34 performs rendering by pasting the texture generated by the special texture generation processing unit 32 on the three-dimensional model. Thus, a bitmap image is reproduced frame by frame at a predetermined frame rate (for example, 30 times or 60 times per second) to display a moving image. In the present embodiment, the rendering process is performed using a ray tracing method that calculates and renders reflection of an object surface, light refraction, and the like while tracing light from a light source.

  The rendered image analysis processing unit 36 analyzes the bitmap image (rendered image) generated by the rendering processing unit 34 to freely replace desired image data such as a photograph in place of the special texture, and has been rendered. Image drawing information for enabling an image to be displayed on the viewer 40 side is generated.

  The compression processing unit 38 is a processing unit for compressing the image drawing information generated along with the analysis by the rendered image analysis processing unit 36. In the present embodiment, the overall compression rate is suppressed while suppressing deterioration in image quality. In order to improve the data, the data is compressed by using different types of compression methods. Details of the compression method will be described later.

  The viewer 40 according to the present embodiment stores a rendered image obtained by the rendering processing unit 34 of the computer 20, image rendering information analyzed by the rendered image analysis processing unit 36 and compressed by the compression processing unit 38. A storage unit 41, an input processing unit 42 that controls input of image data such as photographs stored in the memory card MC, image data input by the input processing unit 42, and rendered images stored in the storage unit 41 , A development processing unit 44 that decodes (decompresses) image drawing information, and a drawing processing unit 46 that combines and renders image data input to the rendered image as a texture. The viewer 40 sequentially reads a plurality of image data stored in the memory card MC according to an instruction from a user, and pastes the read image data on a rendered image of a three-dimensional model using image drawing information, and sequentially reproduces the image data. The slide show display can be performed.

  Next, the operations of the special texture generation processing unit 32, the rendering processing unit 34, the rendered image analysis processing unit 36, and the compression processing unit 38 of the computer 20 of the present embodiment configured as described above, the decompression processing unit 44 of the viewer 40, The operation of the drawing processing unit 46 will be described. First, the processing of the special texture generation processing unit 32 will be described. FIG. 5 is a flowchart illustrating an example of the special texture generation process.

In the special texture generation process, first, the target set number i for identifying which set of the plurality of sets is initialized to a value 1 (step S100), and the RGB color for the target set number i is initialized. N special textures are generated for each component (step S110), the target set number i is incremented by 1 (step S120), the target set number i and the value n are compared (step S130), and the target When the set number i is less than or equal to the value n, the process returns to step S110 to repeat the process of generating n special textures for the next target set number i, and when the target set number i exceeds the value n, the next process is performed. move on. Here, the generation of the special texture with the target set number i from the value 1 to the value n shifts the target texture number j from the first to the nth by the value 1 from the first as shown in the following equation (1). While comparing the target texture number j and the target set number i, the target texture number j that matches the target texture number j is in the gradation value range from the minimum value 0.0 (black) to the maximum value 1.0 (white). A white solid special texture (second solid paint pattern) is generated by setting a gradation value of 1.0 to all coordinates (X, Y). This is done by generating a black solid special texture (first solid coating pattern) by setting a gradation value of 0.0 to the coordinates (X, Y). Here, “c” in the formula (1) indicates a value corresponding to each color of the RGB values of the image data, “n” indicates the number of textures arranged on one screen, and “b” indicates the texture coordinates. Indicates the number of bits when expressed as a binary number, and “Tc, i, j (X, Y)” is the color component c, the target set number i, and the coordinate (X, Y) of the special texture in the target texture number j. Indicates a gradation value (hereinafter the same). Here, for all color components, a pattern in which the maximum gradation value is set for all coordinates is called white solid, and a pattern in which the minimum gradation value is set for all coordinates is called black solid. (Hereinafter the same).

When a special texture having a target set number i having a value 1 to a value n (a first set group consisting of n sets) is generated, then n number of color components having a target set number i having a value (n + 1) are generated. A special texture is generated (step S140) (second set), and the target set number i is incremented by 1 (step S150). Here, the generation of the special texture having the target set number i of the value (n + 1) is performed on all coordinates (X, X, X) for all the target texture numbers j from 1 to n as shown in the following equation (2). This is done by generating a solid black special texture by setting a gradation value of 0.0 to Y).

When the special texture having the target set number i of the value (n + 1) is generated, the [i− (n + 2)] when the texture coordinates are expressed in an alternating binary number (gray code) with respect to the target set number i. N special textures for each color component of the vertical stripe pattern corresponding to the bit are generated by the following equation (3) (step S160), the target set number i is incremented by 1 (step S170), and the target set number i Is compared with the value (n + b + 1) (step S180), and when the target set number i is equal to or smaller than the value (n + b + 1), the process returns to step S160 to generate n special textures for the next target set number i. Repeatedly, when the target set number i exceeds the value (n + b + 1), the process proceeds to the next process. With the above processing, a correspondence setting pattern for setting the correspondence between the x coordinate of the rendered image and the X coordinate of the special texture is generated. Here, “gray (a)” in Equation (3) is a gray code (alternate binary code) representation of the numerical value a, and “and (a, b)” is the logical product of a and b for each bit. Shown (same below). The target set numbers i from (n + 2) to (n + b + 1) are the (b-1) th bit (least significant bit) to 0th bit (most significant bit) when the texture coordinates are expressed in binary numbers, respectively. The gradation value of 1.0 (white) is set when the value of the bit corresponding to the target set number i is 1, and the value of the corresponding bit is 0. A special texture having a vertical stripe pattern is generated by setting a gradation value of 0.0 (black). In this embodiment, the texture coordinates are expressed in alternating binary numbers. For example, if the texture number n is 3 and the coordinates are 3 bits (b = 3), the target set number i is the first number. As a special texture of value 5 indicating 2 bits (least significant bit), a black gradation value is set for the X coordinate value 1 and a white gradation value is set for the X coordinate values 2 and 3, and the X coordinate value 4 , 5, black gradation values are set, X coordinate values 6, 7 are set to white gradation values, and X coordinate value 8 is set to black gradation values. Further, as a special texture having a target set number i with a value 6 indicating the first bit, a black gradation value is set for the X coordinate values 1 and 2, and a white gradation value is set for the X coordinate values 3 to 6. As for the X coordinate values 7 and 8, a black gradation value is set, and as the special texture having the target set number i indicating the 0th bit (most significant bit), the X coordinate values 1 to 4 are black. A gradation value is set, and for the X coordinate values 5 to 8, a white gradation value is set.

When the special texture having the target set number i of the value (n + 2) to the value (n + b + 1) is generated, the [i− (n + b + 2) when the y coordinate of the texture is expressed as an alternating binary number with respect to the target set number i. )] N special textures for each color component of the horizontal stripe pattern corresponding to the bit are generated by the following equation (4) (step S185), the target set number i is incremented by 1 (step S190), and the target set is set. The number i is compared with the value (n + 2b + 1) (step S195). When the target set number i is equal to or smaller than the value (n + 2b + 1), the process returns to step S185 to generate n special textures for the next target set number i. When the process is repeated and the target set number i exceeds the value (n + 2b + 1), the generation of all the special textures is completed. This routine is terminated. With the above processing, a correspondence setting pattern for setting the correspondence between the y coordinate of the rendered image and the Y coordinate of the special texture is generated. The target set numbers i from (n + b + 2) to (n + 2b + 1) are the (b-1) th bit (least significant bit) to 0th bit (most significant bit) when the texture coordinates are expressed in binary numbers, respectively. And when the value of the bit corresponding to the target set number i is 1, the gradation value of 1.0 (white) is set, and the value of the corresponding bit is 0. Sometimes a special texture with a horizontal stripe pattern is generated by setting a gradation value of 0.0 (black). In this embodiment, the texture coordinates are expressed by alternating binary numbers. For example, if the texture number n is 3 and the y coordinate is 3 bits (b = 3), the target set number i is As a special texture having a value 8 indicating the second bit (least significant bit), a black gradation value is set for the Y coordinate value 1 and a white gradation value is set for the Y coordinate values 2 and 3, and the Y coordinate value is set. Black gradation values are set for 4 and 5, white gradation values are set for Y coordinate values 6 and 7, and black gradation values are set for Y coordinate value 8. As the special texture having the target set number i of 9 indicating the first bit, a black gradation value is set for the Y coordinate values 1 and 2, and a white gradation value is set for the Y coordinate values 3 to 6. A black gradation value is set for the Y coordinate values 7 and 8, and a special texture with the target set number i indicating the 0th bit (the most significant bit) is a black value for the Y coordinate values 1 to 4. A gradation value is set, and a white gradation value is set for the Y coordinate values 5 to 8. FIG. 6 shows a list of special textures generated when the number of textures n is 3 and the number of coordinate bits b is 3.

  For each set, the rendering processing unit 34 performs rendering processing by pasting the corresponding n special textures onto the three-dimensional model. FIG. 7 shows the rendering process. In this embodiment, since the three-dimensional model is rendered as a moving image, the number of textures n is 3 and the number of bits b is 3, the rendering process for a total of 10 sets is performed and 10 sets of moving images are generated. Will be. This moving image is composed of bitmap images (rendered images) generated for each frame from frames 1 to T. In FIG. 7, bitmap images having a common frame number are extracted from each of 10 sets of rendered images.

  Next, processing for analyzing a rendered image generated by the rendering processing unit 34 will be described. FIG. 8 is a flowchart illustrating an example of a rendered image analysis process executed by the rendered image analysis processing unit 36.

In the rendered image analysis process, first, as shown in the following equation (5), a variable It () indicating the texture number to be pasted to the coordinates (x, y) of the rendered image at each frame number t (= 1 to T). x, y) is initialized to a value of 0 (step S200), a white solid area (coordinates) in the rendered image of the set numbers 1 to n in the target frame t is specified, and a variable It (x , y) is set (step S210). As shown in the following equation (6), this processing is performed by sequentially shifting the target set number i from No. 1 to n, and the gradation value of the rendered image of the target set number i (the gradation value for each color component). This can be done by comparing the sum of the tone values of the rendered image with the set number (n + 1) (the sum of the tone values for each color component). That is, in the target set number i, the special texture having the texture number equal to the number i is set to white solid, and in the set number (n + 1), the special texture having all texture numbers is set to black solid. Therefore, when the gradation value of the rendered image with the target set number i is larger than the gradation value of the rendered image with the set number (n + 1), the special texture with the texture number i is pasted to the coordinates (x, y). You can see that it was done. Here, “w” in equation (5) indicates the size in the width direction of the rendered image, and “h” indicates the size in the height direction of the rendered image. In addition, “Ac, i, t (x, y)” in the equation (6) is the rank of the coordinate (x, y) of the rendered image at the color component c, the set number i (1 to n), and the frame number t Indicates the key value (hereinafter the same).

Subsequently, the gradation value of the rendered image with the set number (n + 1) is set as the bias Bc, t (x, y) by the following equation (7) (step S220), and the variable It (x, y) is a value. The gain Gc, t (x, y) is calculated by the following equation (8) for the coordinates (x, y) of the rendered image that is not 0, that is, the region where the texture is pasted (step S230). Here, “Ac, It (x, y), t (x, y)” in the equation (8) is the color component c and the set number i and the frame number t stored in the variable It (x, y). Indicates the gradation value of the coordinates (x, y) of the rendered image. FIG. 9 shows the relationship between the bias Bc, t (x, y) and the gain Gc, t (x, y). When rendering with a texture attached to a 3D model, as shown in the figure, the offset that does not depend on the gradation value of the original texture corresponds to the bias Bc, t (x, y), and the gradation of the original texture The gradient of the change in the gradation value of the rendered image with respect to the change in value corresponds to the gain Gc, t (x, y).

Then, the coordinates (X′t (x, y), Y′t (x, y)) of the texture gray code expression are initialized to the value 0 by the following equation (9) (step S240), and the set number (n + 2) Correspondences between the coordinates (x, y) of the rendered image (n + 2b + 1) and the coordinates (X′t (x, y), Y′t (x, y)) of the texture are set (step S250). Here, the correspondence relationship of coordinates is set by the following equation (10). Specifically, the gradation value Ac of the rendered image of the set number (i + n + 1) while sequentially shifting the number i from 1 to b. , i + n + 1, t (x, y) minus the bias Bc, t (x, y) (the sum for each color component) is the gain Gc, t (x, Whether or not y) is greater than the value divided by 2 (sum of each color component), that is, whether or not the coordinates (x, y) are white in the white and black vertical stripe pattern in the set number (i + n + 1) When the value is white, a value 1 is set to the corresponding (i-1) -th bit value of the coordinate X't (x, y) in the alternating binary number expression, and the number i is sequentially shifted from the 1st to the bth. While subtracting the bias Bc, t (x, y) from the gradation value Ac, i + b + n + 1, t (x, y) of the rendered image of the set number (i + b + n + 1) (for each color component Total Is larger than the value obtained by dividing the gain Gc, t (x, y) of the rendered image with the set number i by the value 2 (the sum for each color component), that is, white and black horizontal stripes at the set number (i + b + n + 1) It is determined whether or not the coordinates (x, y) of the pattern are white. If the coordinates are white, the value of the (i−1) -th bit corresponding to the coordinates Y′t (x, y) is set to the value 1. It is done by. Here, “or (a, b)” in the expression (10) indicates a logical sum of bits a and b.

When the correspondence relationship of coordinates is set, the coordinates (X't (x, y), Y't (x, y)) of the texture of the gray code expression are decoded using the following equation (11) to obtain the decoded coordinates (Xt (x, y), Yt (x, y)) is calculated (step S260), and it is determined whether or not the processing has been completed for all the frames having values 1 to T (step S270). When the process is not completed, the next frame is set as the target frame t, and the process returns to step S210 to repeat the process. When the process is completed for all frames, the process is terminated. Here, “gray ⁻¹ (a)” in Expression (11) indicates a value obtained by decoding the Gray code a, and “Xt (x, y)” indicates the coordinates (x, y) represents the x coordinate of the texture, and “Yt (x, y)” represents the y coordinate of the texture corresponding to the coordinate (x, y) of the rendered image of frame number t. In this embodiment, since the origin of the coordinates (X′t (x, y), Y′t (x, y)) is (1,1), the value 1 is set to the value obtained by decoding the Gray code. It is adding. Image rendering information includes variable It (x, y), bias Bc, t (x, y), gain Gc, t (x, y), and coordinates (Xt (x, y), Yt (x, y)) And are included.

  Next, processing for compressing image drawing information obtained by analysis by the rendered image analysis processing unit 36, particularly processing for compressing coordinates (Xt (x, y), Yt (x, y)) will be described. FIG. 10 is a flowchart illustrating an example of the compression processing executed by the compression processing unit 38.

  In the compression process, first, the bias Bc, t (x, y) and the gain Gc, t (x, y) are compressed by the JPEG compression method (step S300). Next, the coordinates (Xt (x, y), Yt (x, y)) are compressed by the linear compression method (step S310), the compressed data is saved (step S320), and this routine is terminated. Note that the above compression processing may be performed by separating the texture area to which the texture is pasted and the non-texture area to which the texture is not pasted, and performing either one or both of the areas. Further, It (x, y) as image drawing information may be compressed.

  FIG. 11 is a flowchart showing an example of the linear compression process executed in step S310 of FIG. The linear compression processing in the present embodiment is a method in which each of the coordinates Xt (x, y) and Yt (x, y) of the rendered image is linearly approximated by triangulation. That is, in the graph in which the coordinate value Xt is the height in the xy plane, the upper space of the xy plane is covered with a plurality of triangles approximated to this height, and the difference between the plurality of triangles and the coordinate value Xt is below a predetermined reference Set to be. Then, the surface of the triangle is regarded as the coordinate value Xt at the coordinates (x, y). Specifically, first, a target texture area is set (step S400). That is, since the coordinate Xt (x, y) is defined only for the coordinate (x, y) where It (x, y) ≠ 0, that is, the coordinate to which the texture is pasted, all coordinates (x , y), one of the regions where It (x, y) ≠ 0 is set as the target texture region. If there is only one region where It (x, y) ≠ 0, that one region is the target texture region, and the region where It (x, y) ≠ 0 is separated in the xy plane. If there are a plurality of target texture areas, each area is set as a target texture area, and the processing from step S410 is performed on each target texture area. In this embodiment, each of the coordinates Xt (x, y) and Yt (x, y) is linearly compressed for the texture area, but arbitrary coordinate values Xt and Yt are set for the non-texture area. The configuration may be such that linear compression is performed collectively for all coordinates (x, y).

  When the target texture area is set, the average gradation value za is calculated by taking the average of the gradation values z (coordinate value Xt or coordinate value Yt) in the target texture area (step S410), and in the target texture area A deviation Δz between the gradation value z and the average gradation value za of each pixel is calculated (step S420), and it is determined whether or not the calculated deviation Δz is larger than a threshold value zref (step S430). When the deviation Δz for any pixel is equal to or smaller than the threshold value zref, it is determined whether or not there is a next texture area (step S560), and when there is a next texture area, the process returns to step S400 to make this texture area the target texture area. When the deviation Δz for any pixel is larger than the threshold value zref, the pixel A (x1, y1, z1) with the maximum gradation value z1 and the pixel B (with the gradation value of the minimum z2) are repeated. x2, y2, z2) are searched (step S440), a straight line orthogonal to the AB line and parallel to the xy plane and a plane α including two points A and B are set (step S450). The pixel C (x3, y3, z3) having the longest distance L1 is searched (step S460). FIG. 12 shows the relationship between the plane α and the pixel C. When the pixel C is searched in this way, it is determined whether or not the distance L1 is greater than the threshold value Lref (step S470). When the distance L1 is less than or equal to the threshold value Lref, the process proceeds to the next process, and when the distance L1 is greater than the threshold value Lref A set G = {A, B, C} is set (step S480), and a plane β including the three points A, B, C is set (step S490). FIG. 13 shows the relationship between the plane β and the pixel P. Next, the pixel P (xp, yp, zp) having the longest distance Lp from the plane is searched (step S500). When S500 is executed for the first time, the plane for which the distance Lp is calculated is the plane β set in step S490, and when S500 is executed for the second time or later, the plane for which the distance Lp is calculated is the step. This is the plane reset in S540.

  Then, it is determined whether or not the distance Lp is larger than the threshold value Lref (step S510). When the distance Lp is equal to or smaller than the threshold value Lref, the process proceeds to the next process. When the distance Lp is larger than the threshold value Lref, a pixel P is newly added. The set point set G is set (step S520), the set point set G is triangulated (step S530), and the plane is reset (step S540). FIG. 14 is an explanatory diagram showing a state in which the point set G = {A, B, C, P} is triangulated. As shown in the figure, a triangular plane including three points A, B, and C and a triangular plane including three points A, B, and P are divided by a straight line AB that is shared by both, and each line is divided by a straight line AB. Reset the plane by extending the triangular plane. When the plane is reset, the process returns to step S500 to search for a new pixel P (xp, yp, zp) having the largest distance Lp with the reset plane (step S510), and until the distance Lp becomes equal to or smaller than the threshold Lref. The processes in steps S500 to S540 are repeated. When the distance Lp is less than or equal to the threshold value Lref in step S510, the set plane is cut out to the shape of the target texture area (step S550), and when there is a next texture area (step S560), the process returns to step S400 to return to this texture area. Set the target texture area and repeat the processing of steps S400 to S560. When there is no next texture area, the planes cut out for each texture area are combined (step S570), and this is x for each column (y coordinate direction). Scanning is sequentially performed in the coordinate direction (step S580), the starting point a, the inclination Δa, and the length l obtained thereby are stored (step S590), and this process is terminated. The stored information indicating the starting point a, the slope Δa, and the length l is information indicating the linearly compressed coordinates Xt (x, y) or coordinates Yt (x, y). FIG. 15 shows the state of linear compression. In the linear compression, as shown in the figure, the starting points a0, a1, a2 and the slopes Δa0, Δa1, Δa2 and the lengths l0, l1, l2 are obtained for the scanning line s1, and are obtained for each scanning line s2, s3,. This is done by repeatedly saving the result.

Image drawing information (bias Bc, (x, y), gain Gc, t (x, y), coordinates Xt (x, y), Yt (x, y))) compressed by the compression processing unit 38 of the computer 20 and The variable It (x, y) is stored in the storage unit 41 of the viewer 40, and the coordinates Xt (x, y) and Yt (x, y) linearly compressed by the expansion processing unit 44 of the viewer 40 are decompressed for linear compression. The bias Bc, t (x, y) and the gain Gc, t (x, y) JPEG compressed by the processing are decompressed by the JPEG decompression processing and used for the rendering processing by the rendering processing unit 46. The drawing processing unit 46 reads a plurality of image data such as photographs stored in the memory card MC as replacement textures and combines them into a rendered image using the following equation (12) and sequentially draws the textures. It is possible to perform a slide show reproduction that displays the rendered image of the three-dimensional model while replacing. Here, “Uc, It (x, y) (Xt (x, y), Yt (x, y))” in Expression (12) is the coordinate (X of the replacement texture in the color component c and texture number i. , Y) indicates a gradation value (0.0 to 1.0), and “Pc, t (x, y)” indicates the coordinates (x, t) of the display image (rendered image) in the color component c and frame number t. y) represents the gradation value (0.0 to 1.0). As shown in the equation (12), the gradation value Pc, t (x, y) of the display image is set with respect to the texture arrangement region where the variable It (x, y) is not 0 (the coordinate of the display image ( Bias Bc, t (x) by multiplying the tone value of the replacement texture coordinate (Xt (x, y), Yt (x, y)) corresponding to x, y) by the gain Gc, t (x, y) , y) is set. FIG. 16 shows three replacement textures having texture numbers 1 to 3, and FIG. 17 shows a state in which the replacement texture of FIG. 16 is arranged and rendered on the rendered image. As described above, since the coordinates Xt (x, y) and Yt (x, y) are compressed by linear compression, when the replacement texture is placed while suppressing the data volume with a relatively high compression ratio The distortion of the image can be suppressed.

  According to the image processing method of the embodiment described above, the coordinates (x, y) as one of the image drawing information obtained by pasting and rendering the special texture on the three-dimensional model and analyzing the rendered image The image rendering information (Xt (x, y), Yt (x, y)) that shows the correspondence between the texture and the texture coordinates (X, Y) is compressed by the linear compression method. However, the data can be compressed with a high compression rate. On the computer 20 side, the vertical stripe pattern for x coordinate and the horizontal stripe pattern for y coordinate corresponding to the value of each bit when coordinates (X, Y) are expressed in binary are used as a special texture. Render by pasting to the dimensional model and analyzing the rendered image obtained as a bitmap image by rendering, and the coordinates of the rendered image (x, y) and texture coordinates (Xt (x, y), Yt ( x, y)) is set and saved as image drawing information, and when the image is displayed using the rendered image on the viewer 40 side, the texture coordinates (Xt (x ), y), Yt (x, y)), based on the gradation value of the display image, it is rendered at the coordinates (x, y) of the display image. In addition, the processing load can be reduced as compared with the case where the three-dimensional model is rendered and displayed in real time. In addition, since the tone value of the texture is converted by using the gain Gc, t (x, y) and the bias Bc, t (x, y) and the tone value of the display image is set, the three-dimensional model is rendered. It is also possible to reflect the effects of refracted light, specular reflection, and shadows. Furthermore, since a vertical striped pattern and a horizontal striped pattern corresponding to alternating binary numbers are formed as a special texture for specifying the correspondence of coordinates, a change of 1 bit always occurs when moving to adjacent coordinates. Therefore, it is possible to suppress erroneous data from being acquired due to an error in the gradation value of the image.

  In this embodiment, the coordinates Xt (x, y) and Yt (x, y) are compressed by the linear compression method, but the present invention is not limited to this, and the bias Bc, t (x, y) and The gain Gc, t (x, y) may be compressed by linear compression. Further, the coordinate system to be linearly compressed may be an xy coordinate system. That is, the coordinate correspondence is defined as xt (X, Y), yt (X, Y), and the height xt, yt, etc. on the XY plane is expressed by linear approximation using triangulation. May be. Further, instead of the above-described JPEG compression method, the compression may be performed by other compression methods such as JPEG2000, GIF, TIFF, deflate compression or the like.

In the present embodiment, a three-dimensional pattern is obtained by using a vertical stripe pattern for x coordinates and a horizontal stripe pattern for y coordinates corresponding to the value of each bit when coordinates (X, Y) are expressed in binary notation as a texture. Image rendering information is generated by pasting and rendering the model and analyzing the rendering result. However, the pattern used is not limited to this, and the gradation (gradation value) gradually increases in the x-coordinate direction (lateral direction). It is also possible to use a pattern that changes to a pattern and a pattern in which shading gradually changes in the y-coordinate direction (vertical direction). In this case, a single pattern of the set number (n + 2) obtained by the following equation (13) is used instead of the vertical stripe pattern of the set numbers (n + 2) to (n + b + 1) obtained by the equation (3) described above and the equation Instead of the horizontal stripe pattern of the set numbers (n + b + 2) to (n + 2b + 1) obtained by (4), one pattern of the set number (n + 3) obtained by the following equation (14) may be used.

When the pattern of Expression (13) and the pattern of Expression (14) are used, the setting of the coordinate correspondence can be obtained by the following Expression (15). FIG. 18 shows an example of the special texture, and FIG. 19 shows a state where the special texture of FIG. 18 is pasted on the three-dimensional model and rendered. Thereby, the number of special textures to be generated can be reduced.

  In the present embodiment, the special texture of the vertical stripe pattern whose target set number i is a value (n + 2) to a value (n + b + 1) corresponds to the value of each bit when the coordinates are expressed in alternating binary numbers, and the target set number The special texture of the horizontal stripe pattern with i being the value (n + b + 2) to the value (n + 2b + 1) is assumed to correspond to the value of each bit when the coordinates are expressed in alternating binary numbers. It may be generated as a value corresponding to the value of each bit when expressed in decimal. An example of the special texture in this case is shown in FIG.

  In the present embodiment, an image is reproduced by the viewer 40, but any device such as a mobile phone with a liquid crystal screen or a printer may be used as long as the device can reproduce an image.

  It should be noted that the present invention is not limited to the above-described embodiment, and it goes without saying that the present invention can be implemented in various modes as long as it belongs to the technical scope of the present invention.

  20 computers, 22 displays, 31 storage units, 32 special texture generation processing units, 34 rendering processing units, 36 rendered image analysis processing units, 38 compression processing units, 40 viewers, 41 storage units, 42 input processing units, 44 decompression processings Part, 46 drawing processing part, MC memory card.

Claims (13)

An image showing the correspondence between the coordinates of a rendered image rendered by pasting a texture on a three-dimensional model and the coordinates of the texture, and the correspondence between the color of each pixel of the rendered image and the color of each pixel of the texture Get drawing information
An image processing method comprising: compressing the image drawing information indicating a correspondence relationship between the coordinates of the rendered image and the coordinates of the texture by a linear compression method. The image processing method according to claim 1, wherein the image drawing information indicating a correspondence relationship between a color for each pixel of the rendered image and a color for each pixel of the texture is compressed by a JPEG compression method. The image processing method according to claim 1, wherein the linear compression method compresses data by linearly approximating one of the values of the rendered image coordinates and the texture coordinates by triangulation. . The image processing method according to any one of claims 1 to 3,
Render by pasting a predetermined pattern with different gradation values for each coordinate as a texture to a 3D model,
By analyzing the rendered image obtained as a bitmap image by the rendering, a correspondence relationship between the coordinates of the rendered image and the coordinates of the texture is set and stored as the image drawing information,
An image processing method, wherein when displaying a desired texture as an image, the desired texture is arranged and displayed in the rendered image based on the stored image drawing information.   The image processing method according to claim 4, wherein the correspondence relationship is derived by specifying the corresponding coordinates of the predetermined pattern from the gradation value of each coordinate of the rendered image.   The predetermined pattern is the same number of bits as the number of bits when the texture coordinates are expressed in binary, and each pattern is associated with each bit when the coordinates are expressed in binary. 6. The image processing method according to claim 4, wherein the gradation value of each coordinate is set to a value corresponding to the value of the associated bit.   The image processing method according to claim 6, wherein the binary number is a gray code (alternate binary number). An image processing method according to any one of claims 4 to 7,
In addition to the correspondence setting pattern for setting the correspondence between the coordinates of the rendered image and the coordinates of the texture as the predetermined pattern, a first solid pattern that is further solid-coated with the minimum gradation value Render by pasting to a 3D model,
A bias value, which is a gradation value of the first solid pattern in the rendered image, is stored as the image drawing information indicating a correspondence relationship between a color for each pixel of the rendered image and a color for each pixel of the texture. ,
An image processing method for converting and displaying the gradation value of the rendered image by offsetting the gradation value of the desired texture based on the stored bias value.   The image processing method according to claim 8, wherein the bias value is compressed by the linear compression method. An image processing method according to any one of claims 4 to 9,
In addition to the correspondence setting pattern for setting the correspondence between the coordinates of the rendered image and the coordinates of the texture as the predetermined pattern, the first solid coating pattern formed by solid painting with the minimum gradation value and the maximum A second solid pattern that is solid with gradation values is pasted on the three-dimensional model and rendered,
By calculating a gain that is a deviation between the gradation value of the second solid pattern and the gradation value of the first solid pattern in the rendered image, the color of each pixel of the rendered image and the pixels of the texture Save as the image drawing information indicating the correspondence with each color,
An image processing method for converting a gradation value of the desired texture into a gradation value of the rendered image based on the stored gain and displaying the converted gradation value. The image processing method according to claim 10, comprising:
In the case where n (n is a natural number) textures are arranged and displayed on the rendered image, (n−1) sheets of the first solid pattern and one second solid pattern are included. A second set comprising n sets, a first set group in which the second solid coating pattern is applied to the three-dimensional model for each set, and a set of n first solid coating patterns. And render each set to the 3D model,
The gradation value of each rendered image obtained by rendering the first set group for each set and the gradation value of the rendered image obtained by rendering the second set are the first values. An image processing method for identifying a texture area, which is an area where a texture is pasted on the three-dimensional model, by comparing each set of the set groups, and calculating the gain for the identified texture area.   The image processing method according to claim 1, wherein the image is drawn in units of frames and displayed as a moving image. An image showing the correspondence between the coordinates of a rendered image rendered by pasting a texture on a three-dimensional model and the coordinates of the texture, and the correspondence between the color of each pixel of the rendered image and the color of each pixel of the texture Image drawing information acquisition means for acquiring drawing information;
Compression means for compressing the image drawing information indicating the correspondence between the coordinates of the rendered image and the coordinates of the texture by a linear compression method;
An image processing apparatus comprising: