JP4349733B2 - Method and apparatus for providing non-realistic cartoon outline in 3D video graphics system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to 3D computer graphics, and more particularly to non-realistic 3D imaging. More particularly, the present invention relates to a method and apparatus for generating and displaying a line on a shadow of a three-dimensional object by a computer and other boundaries found on the edges thereof.
[0002]
[Prior art]
Much of the research on computer graphics has tended to focus on generating realistic images and has been quite successful so far. Nowadays, computer-generated images are becoming very realistic and cannot be distinguished from photographs. For example, we often see dinosaurs, aliens, and so on that appear to be very realistic and real due to special effects created by computers. The computer-based flight simulator trained by new pilots is very real and almost reproduces the actual flight conditions. Even in low-cost home video game systems, an amazing level of realism has been realized, and game players are covered in snow and ice, for example, running on a track in a real race car You can feel the illusion of walking in a medieval castle, like skiing down a slope. In most games, this illusion of realism greatly enhances the gameplay experience.
[0003]
However, in some cases, an unreal image may be preferable. For example, in interactive video and computer games, instead of realistically simulating the real (or fantasy) world, a cartoon world with many unrealistic parody-like characters will be created on purpose. If it is displayed, it can be provided with entertainment value. For example, in such a game, an effort is made to reproduce a hand-drawn comic book with movement, voice, and interactivity.
[0004]
In such a three-dimensional computer system, one of the desirable visual effects is to make sure that the shadow lines of the display object and other boundaries seen at the edges thereof are clear and reliable. Such a border makes certain images clearer. For example, it becomes easier for the user to distinguish different faces such as the outline of a cartoon character, a landscape hill or mountain, and the edge of a wall. Furthermore, such a boundary line tends to create a desirable impression as if it were hand-drawn by a cartoonist.
[0005]
[Problems to be solved by the invention]
Here, a line that distinguishes from another object adjacent to the edge of the polygon that forms a certain object is one of the methods that give an unreal effect such as a boundary line around the edge of the object such as a character. There is a way to show clearly.
[0006]
However, if the border with another object is clearly shown, the image processing becomes extremely complicated. As a result, in systems with limited resources, this method results in delays in image generation and reduced efficiency. Efficiency is extremely important in low-cost 3D graphics systems such as interactive 3D game systems. Using more efficient techniques to generate visual effects reduces the burden on scarce system resources and further improves visual effects overall without significantly affecting speed or other performance be able to.
[0007]
[Means for Solving the Problems and Effects of the Invention]
The present invention solves this problem by providing an efficient technique for displaying shadow lines and borders at their edges in 3D graphics systems, such as home video game consoles.
[0008]
According to the first invention, after the image is drawn in the frame buffer, the boundary line is generated. In the present invention, the values stored in the frame buffer are used to determine which pixels are located on the contour or other edges of the object, and the border color is selectively blended with these pixels and displayed.
[0009]
Further, according to an embodiment, the edge of the contour is arranged by comparing the depth value of a certain pixel with the depth value of the surrounding pixels. For example, the “distance” value is calculated based on the absolute value of the distance between the depth of a certain pixel and the surrounding pixels. Then, based on this distance value, the desired border color is blended with the color value of that pixel. For example, the distance value is used to calculate the alpha value of the pixel, and this alpha value is used to control the amount of border color blended into the pixel color.
[0010]
In another example, depth changes are further used. For example, the alpha value of the pixel is calculated based on the distance value and a further provided value that is a function of the pixel depth, and the alpha value changed according to the depth value of the pixel is obtained. Based on the alpha value changed by this depth, the border color is blended with the pixel color.
[0011]
In a further embodiment, the border is drawn on some internal edge where the cartoonist will draw the border, rather than on the contour of the object. For example, consider a state where a cartoon character is bending an arm in front of the body. In this case, the cartoonist would draw a border around the character's arm outline. Even if this character bends the arm at the front of the torso so that the edge of the contour of the arm is located inside the entire contour. According to a further feature of the present invention, different identification values are assigned to each pixel of the image. This identification value is typically stored in a frame buffer memory area or the like prepared for the alpha value. These identification values are used to identify an edge with a boundary line and an edge without a boundary line.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows an example of a three-dimensional video graphics system 1005 for displaying boundaries according to the present invention.
[0013]
The system 1005 is responsive to interactive real-time input from, for example, the game controller 1007 and / or other manual input devices, and displays on a display device 1009 (eg, a home color television receiver, video monitor, or other display). Display video. The system 1005 operates under the control of a computer program such as a video game program stored on an external storage medium 1011 (eg, a replaceable video game cartridge, CD-ROM, or other optical disc).
[0014]
In this example, system 1005 includes a processor 1010, a three-dimensional graphics processor 1020 and a memory 1030. The processor 1010 provides a three-dimensional graphics command to the graphics score processor 1020. The graphics coprocessor 1020 interactively generates a 2D view in the 3D world according to the 3D graphics command. For example, the graphics coprocessor 1020 includes a hardware three-dimensional graphics pipeline. This pipeline manipulates graphics primitives such as polygons defined in 3D space to generate pixels that represent 2D images in 3D space projected onto a field of view based on an arbitrarily chosen viewpoint. . In this example, the user can use the game controller 1007 to interactively change the viewpoint in real time.
[0015]
The graphics coprocessor 1020 stores the generated display pixel in the frame buffer 1040 in the memory device 1030. The frame buffer 1040 includes a color frame buffer 1042 and a depth (Z) buffer 1044. In this example, the color frame buffer 1042 stores a two-dimensional array of red, green and blue (RGB) color values and corresponding alpha (A) values. There may be a one-to-one relationship between the stored RGB color values and the pixels displayed on the display device 1009. Alternatively, the frame buffer 1042 may store the subsample. The Z buffer 1044 stores a depth value (for example, a distance in the z direction with respect to the viewpoint) for each pixel or sub-pixel stored in the color frame buffer. As is well known, the Z-buffer 1044 is used for a variety of purposes (eg, removal of invisible surfaces) when the graphics pipeline “builds” the image.
[0016]
In the preferred embodiment, the contents of the color frame buffer 1042 and / or the z buffer 1044 selectively blend the border color with the pixel color values stored in the color frame buffer 42 to define object boundaries. Also plays a role. Specifically, the system 1005 may include a pixel filter 50. The pixel filter 50 processes the pixel values of the image rendered by the 3D graphics coprocessor 1020 and selectively applies them on the contours and / or other edges of the object displaying the border color. . The pixel filter 50 in the preferred embodiment operates at the pixel post-processing stage after the image data is rendered in the frame buffer 1040 or other pixel memory. For example, the pixel filter 50 may apply a boundary line or the like to the image data drawn and buffered by the three-dimensional graphics processor 1020 during the process of combining the image data from the graphics processor and the image data into the memory 1030. Non-realistic image elements may be added.
[0017]
FIG. 2 is a more detailed schematic diagram of an entire interactive 3D computer graphics system 1005 in which the present invention may be implemented. System 1005 can be used to play interactive 3D video games with attractive stereo sound. Various games can be played by inserting an appropriate storage medium such as the optical disc 1011 into the optical disc playback apparatus 1134. A game player can interact with the system 1005 in real time by operating an input device such as a manual controller 1007. The controller 1007 includes various operating devices such as a joystick, a button, a switch, a keyboard, and a keypad.
[0018]
In this example, main processor 1010 receives input from manual controller 1007 (and / or other input devices). The main processor 1010 interactively responds to such user input and executes a video game or other graphics program provided from, for example, the external storage device 1011. For example, the main processor 1010 can perform various real-time interactive control functions in addition to collision detection and moving image processing.
[0019]
The main processor 1010 generates a three-dimensional graphics command and a voice command, and transmits them to the graphics and voice coprocessor 1020. The graphics and audio coprocessor 1020 processes these commands to display a compelling image on the display device 1009 and generate stereo audio on the stereo speakers 1137R, 1137L, or an appropriate audio generator.
[0020]
System 1005 includes a TV encoder 1140. The TV encoder receives the image signal from the coprocessor 1020 and converts it into a composite video signal suitable for display on a standard display device 1009 (eg, a computer monitor or a home color television receiver). The system 1005 also includes an audio codec (compressor / decompressor) 1138.
[0021]
The audio codec 1138 compresses / decompresses the digitized audio signal (and may convert between digital analog audio signal formats). The audio codec 1138 receives the audio input via the buffer 1141 and transmits it to the coprocessor 1020 (for processing in the coprocessor 1020, other audio signals generated by the coprocessor and / or streaming of the optical disc device 1134). For example, mixing with other audio signals received via the audio output).
[0022]
The coprocessor 1020 stores the voice related information in the memory 1144 dedicated to the voice task. Coprocessor 1020 provides the processed audio output signal to audio codec 1138 for compression and conversion to an analog signal (eg, via buffer amplifiers 1142L and 1142R) for playback by speakers 1137L and 1137R. To do.
[0023]
Coprocessor 1020 can communicate with various peripheral devices in system 1005. For example, the parallel digital bus 1146 may be used for communication with the optical disc device 1134. The serial peripheral bus 1148 may be used for communication with various peripheral devices. Examples of the peripheral device include a ROM and / or a real-time clock 1150, a modem 1152, and a flash memory 1154. An additional external serial bus 1156 may be used for communication with expansion memory 1158 (eg, a memory card).
[0024]
FIG. 3 is a block diagram of example components within the coprocessor 1020. The coprocessor 1020 is an integrated circuit, and includes a three-dimensional graphics processor 1107, a processor interface 1108, a memory interface 1110, an audio digital signal processor (DSP) 1162, and an audio memory interface (I / F) 1164. An audio interface and mixer 1166, a peripheral controller 1168, and a display controller 1128.
[0025]
The three-dimensional graphics processor 1107 performs a graphics processing task, and the audio digital signal processor 1162 performs an audio processing task. Display controller 1128 accesses and retrieves image information from memory 1030 and provides it to TV encoder 1140 for display on display device 1009. The audio interface and mixer 1166 serves as an interface with the audio codec 1138, and receives audio of different sources (eg, streaming audio input from the disk 1011, output of the audio DSP 1162, external audio input received via the audio codec 1138, etc.). It is also possible to mix. The processor interface 1108 is an interface related to data and control between the main processor 1010 and the coprocessor 1020. The memory interface 1110 is an interface related to data and control between the coprocessor 1020 and the memory 1030. In this example, the main processor 1010 accesses the main memory 1108 via the processor interface 1108 and the memory interface 1110 that is a part of the coprocessor 1020. Peripheral controller 1168 provides data between coprocessor 1020 and the various peripheral devices described above (eg, optical disk device 1134, controller 1007, ROM and / or real time clock 1150, modem 1152, flash memory 1154, and memory card 1158). Interface for control. The audio memory interface 1164 provides an interface with the audio memory 1144.
[0026]
FIG. 4 is a more detailed view of the 3D graphics processor 1107 and its associated components within the coprocessor 1020. The three-dimensional graphics processor 1107 includes a command processor 1114 and a three-dimensional graphics pipeline 1116. The main processor 1010 exchanges a graphics data stream with the command processor 1114. The command processor 1114 receives such a display command, analyzes (acquires additional data required for processing from the memory 1030), analyzes the vertex command stream for graphics processing and drawing for three-dimensional processing and drawing. 1116. The graphics pipeline 1116 generates a three-dimensional image based on these commands. The processed image information may be transferred to the main memory 1030 and made accessible by the display controller 1128. The display controller 1128 displays the frame buffer output of the pipeline 1116 on the display device 1009. The memory request arbitration circuit 130 arbitrates memory access in the graphics pipeline 1116, the command processor 1114, and the display unit 128.
[0027]
As shown in FIG. 4, the graphics pipeline 1116 may include a conversion unit 1118, a setup / rasterizer 1120, a texture unit 1122, a texture environment unit 1124, and a pixel engine 1126. In the graphics pipeline 1116, the conversion unit 1118 may perform various three-dimensional conversion operations, and may further perform illumination effects and texture effects. The processing performed by the conversion unit 1118 includes, for example, conversion from an object space having a shape input for each vertex to screen space, conversion of input texture coordinates, calculation of projected texture coordinates, polygon clipping, and calculation of illumination intensity for each vertex. And bump mapping texture coordinate generation. The setup / rasterizer 1120 includes a setup unit. The setup unit receives the vertex data from the conversion unit 1118 and transmits triangular setup information to a plurality of rasterizers. The rasterizer performs edge rasterization, texture coordinate rasterization, and color rasterization. The texture unit 1122 performs various tasks related to texturing. This task includes multi-texture processing, post-cache texture decompression processing, texture filtering, raised bump mapping, shading using projected textures, and BLIT using alpha transparency and depth. The texture unit 1122 outputs the filtered texture value to the texture environment unit 1124. The texture environment unit 1124 blends the polygon color and the texture color, and performs texture fog and other environment-related functions. The pixel engine 1126 performs z buffering and blending, and stores data in a frame buffer memory on the integrated circuit.
[0028]
As shown in FIG. 5, the graphics pipeline 1116 may include a DRAM memory 1126a that stores frame buffer information locally. The frame buffer 1126a on the integrated circuit is periodically written in the main memory 1030 that is not provided on the integrated circuit so that the display unit 1128 on the integrated circuit can access it. For example, the pixel filter 50 operates to add a contour such as a boundary line to the cartoon during the writing process. The frame buffer output of the graphics pipeline 1116 (which is finally stored in the main memory 1030) is read by the display unit 1128 for each frame. The display unit 1128 provides digital RGB pixel values for display on the display device 1009.
[0029]
Some of the components of the video game system described above can also be implemented as other than the home video game console configuration described above. Generally, home video game software is written to run on a specific home video game system. This video game software may be “ported” (converted) to the different systems, taking into account the hardware and software configuration of the different systems. Such “porting” often requires access to the video game source code. In addition, there is a method for causing the second system to emulate the first system as a method for executing the game software in a system having a different configuration. If the second system can successfully emulate or simulate the hardware and software resources of the first system, the second system can successfully execute the image executable in the video game software binary; There is no need to access the source code of the game software.
[0030]
In home video games, an emulator is a system different from a system for a game program, that is, a system designed to be able to execute a game program. As an example, an emulator system provides a hardware and / or software configuration (platform) that is different from that of a system for game software. The emulator system includes software and / or hardware components that mimic the software and / or hardware components of the system for game software. For example, the emulator system may include a general-purpose digital computer such as a personal computer that executes a software emulator program that simulates the hardware and / or firmware of the game software system.
[0031]
The emulator software may be developed so that a game for a home video game system as shown in FIGS. 1 to 4 using a console can be executed on a personal computer or other general-purpose digital computer. For such general-purpose digital computers (for example, IBM and Macintosh personal computers and compatible computers), a 3D graphics pipeline corresponding to DirectX and other standard 3D graphics command application program interface (API) is provided. Some have a 3D graphics card. Some have a stereo sound card that provides high-quality stereo sound based on a standard set of sound commands. Such multimedia hardware-equipped personal computers that run emulator software may have sufficient performance to approximate the graphics and sound performance in a dedicated home video game console hardware configuration. Emulator software controls the hardware resources on the personal computer platform to do what the home video game console platform does (though game programmers wrote game software for this platform), 3D graphics, Simulate sound, peripherals, and other functions.
[0032]
FIG. 6 shows an example of the entire emulation process. This example of emulation processing uses a host platform 1201, an emulator element 1303, and game software that can execute a binary image on a storage medium such as a ROM or an optical disk 1305 or another storage device. Host 1201 may be a general purpose or dedicated digital computing device such as a personal computer or other type of game console.
[0033]
The emulator 1303 is executed on the host platform 1201 to convert commands, data, and other information from the storage medium 1305 into a format that can be processed by the host 1201 in real time. For example, the emulator 1303 takes out the program instructions for the home video game platform as shown in FIGS. 1 to 4 and converts the program instructions into a format that can be executed or processed by the host 1201. As an example, the game program is written to be executed on a platform using Z-80, MIPS, IBM PowerPC, or other specific processing device, and the host 1201 is different (for example, Intel Corporation). ), The emulator 1303 takes one or a series of program instructions from the storage medium 1305 and converts them into one or more Intel program instructions.
[0034]
Similarly, the emulator 1303 extracts graphics commands and audio commands for processing by the graphics and audio coprocessor shown in FIG. 3, and the graphics hardware and / or software available in the host 1201 and audio processing resources are processed. Convert to a format that you can do. As an example, emulator 1303 converts such commands into commands that can be processed by a particular graphics and / or sound card of host 1201 (eg, using standard DirectX or a sound application program interface (API)). It may be converted.
[0035]
The emulator 1303 used to provide part or all of the functions of the video game system as described above is a graphic that simplifies or automates the selection of various options and screen modes for games executed using the emulator. A user interface (GUI) may be provided. As an example, such an emulator 1303 may further have higher functionality than the host platform originally assumed by the video game software.
[0036]
FIG. 7 shows a personal computer based host 1201 suitable for use with emulator 1303. The personal computer system 1201 includes a processing unit 1203 and a system memory 1205. A system bus 1207 connects various system elements such as the system memory 1205 and the processing unit 1203. The system bus 1207 may be any of several types of bus structures (one of various bus architectures is used for each bus structure) such as a memory bus or memory controller, a peripheral bus, and a local bus. .
[0037]
The system memory 1207 includes a read only memory (ROM) 1252 and a random access memory (RAM) 1254. The ROM 1252 stores a basic input / output system (BIOS) 1256, which includes a basic routine that helps to transfer information between elements in the personal computer system 1201 at the time of start-up.
[0038]
Personal computer system 1201 further includes various drives and associated computer readable media. The hard disk drive 1209 reads from and writes to a (typically fixed) magnetic hard drive 1211. The magnetic disk drive 1213 reads from and writes to a removable “floppy” and other magnetic disks 1215. The optical disc drive 1217 reads from and attaches to a removable optical disc 1219 such as a CD-ROM or other optical medium, and also writes depending on the configuration. The hard disk drive 1209, magnetic disk drive 1213, and optical disk drive 1217 are connected to the system bus 1207 by a hard disk drive interface 1221, a magnetic disk drive interface 1223, and an optical drive interface 1225, respectively. These drives and associated computer readable media store computer readable instructions, data structures, program modules, game programs, and other data for the personal computer system 1201 in a nonvolatile manner.
[0039]
In other configurations, other computer readable media (eg, magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random storage media that can store computer accessible data can be stored. An access memory (RAM), a read-only memory (ROM), or the like may be used.
[0040]
Many program modules including the emulator 1303 may be stored in the hard disk 1211, removable magnetic disk 1215, optical disk 1219 and / or ROM 1252 and / or RAM 1254 in the system memory 1205. Such program modules may include an operating system that provides graphics and sound APIs, one or more application programs, other program modules, program data, and game data. The user inputs commands and information to the personal computer system 1201 using an input device such as a keyboard 1227 and a pointing device 1229. Other input devices include a microphone, joystick, game controller, satellite dish, scanner, and the like. Such an input device is often connected to the processing device 1203 via a serial port interface 1231 connected to the system bus 1207. For example, a parallel port, a game port, a universal serial bus (USB), etc. It may be connected by another interface. A monitor 1233 and other types of display devices are connected to the system bus 1207 via an interface such as a video adapter 1235.
[0041]
The personal computer 1201 may further include means such as a modem 1154 for communicating with a wide area network 1152 such as the Internet. The modem 1154 may be internal or external and is connected to the system bus 1207 via the serial port interface 1231. The personal computer 1201 may further include a network interface 1156 that allows communication with a remote computing device 1150 (eg, another personal computer) via the local area network 1158 (such communication is It may be performed via the wide area network 1152 or other communication paths, such as a dial-up system or other communication means). The personal computer system 1201 typically includes other peripheral output devices such as standard peripherals such as printers.
[0042]
For example, video adapter 1235 may include a 3D graphics pipeline chipset. This chipset performs 3D graphics drawing at high speed in response to 3D graphics commands issued based on a standard 3D graphics application programmer interface such as Microsoft DirectX. In addition, a series of stereo speakers 1237 are connected to the system bus 1207, which provides hardware and built-in software that supports high quality stereo sound generation based on sound commands provided by the bus 1207. This is done through a sound generation interface such as a “sound card”. Such hardware capabilities allow the host 1201 to provide graphics and sound performance that is fast enough to play a video game stored on the storage medium 1305.
[0043]
(Example of non-realistic cartoon-like contour processing method according to the present invention)
Cartoon outline / border processing according to the present invention may be advantageously performed in hardware and / or software. Assuming there is enough time, after the graphics coprocessor 1020 renders an image in the frame buffer and before the image is displayed, the processor 1010 accesses the frame buffer 1040 to perform pixel post-processing. The border color is blended into the color frame buffer 1042. Alternatively, hardware and / or software is provided in the graphics coprocessor 1020 to perform this function. As described above, the coprocessor 1020 utilizes existing hardware support such as alpha blending and z-buffer arithmetic in the graphics pipeline to perform pixel post-processing / filtering and to create a border around the object's edge. May be given. Alternatively, the processor 1010 and / or the display unit 1128 may perform pixel filtering processing based on the contents of the frame buffer 1040.
[0044]
When the coprocessor 1020 completes this pixel post-processing, the changed pixel value is written back to the frame buffer memory 1030. The contents are accessed from the frame buffer memory 1030 by a video generation circuit in the coprocessor, and an image is generated on the display device 1009. Alternatively, the changed value is sent elsewhere (eg, directly to display device 1005). In order to increase the transmission efficiency, the data in the frame buffer 1040 may be double buffered or DMA may be used. If the frame buffer 1040 is read and processed line by line, the line buffer may be used to hold the Z value of the previously located line.
[0045]
FIG. 8 is a flowchart illustrating an example of a pixel filtering routine 100 according to a preferred embodiment, which performs pixel post-processing boundary lines for the pixel filter 50. The pixel filter routine 100 of FIG. 8 is performed for each of the pixels [x] [y] stored in the frame buffer 1040 (block 102). The pixel filter 100 reads pixel color values (pixel R [x] [y], pixel G [x] [y], pixel B [x] [y]), and further reads a pixel depth value (Z [x]). [y]) is read (block 104). In addition, the pixel filter routine 100 also reads the depth values of the pixels around the pixel [x] [y] (also block 104). In this example, the pixel filter routine 100 reads the depth values of two adjacent pixels. That is, the depth value Z [x] [y] in the neighboring pixel that is “left” next to the predetermined pixel in the xy array, and the peripheral pixel that is truly “below” the predetermined pixel in the xy array Is the depth value Z [x] [y-1]. See FIG. 9 for the above. If the frame buffer 1040 is read and processed line by line, the line buffer can be used to hold the Z value of the previous line.
[0046]
The pixel filter routine 100 determines whether the pixel [x] [y] is the edge of the object by, for example, the following Dz operation (see block 106). For example, the pixel depth value (Z [x] [ y]) and the depth value of one or more neighboring (adjacent in this example) pixels.
DzDx = | Z [x] [y] −Z [x−1] [y] |
DzDy = | Z [x] [y] −Z [x] [y−1] |
Dz = max (DzDx, DzDy)
When performing the above calculation, the pixel filter 100 discriminates the difference between the pixel depth and each of two adjacent pixels, that is, takes the absolute value of these differences (whether an object is near or far from another object). The larger of the two obtained absolute values is selected. This result is a distance value Dz, which is a value that measures how far a certain pixel is from a certain peripheral pixel in the z direction. This operation is performed to test whether the pixel is on the edge of the object. For example, if the pixel is on the edge of the object, Dz is generally large (typically because neighboring pixels have different depths), and if the pixel is not on the edge of the object, Dz Is generally smaller (because neighboring pixels are similar in depth).
[0047]
In another example, the distance value Dz can be calculated by taking the sum of the distance values of two neighboring pixels. For example, the following formula is an example.
Dz = DzDx + DzDy
Other methods are also possible (for example, changing the calculation method, changing the surrounding pixels to be used, or changing the number of surrounding pixels).
[0048]
The pixel filter routine 100 in this example changes the color in proportion to the distance value Dz calculated as described above. For example, the pixel filter 100 scales / modifies the coefficient of the obtained distance value Dz, changes the resulting value, and clamps the resulting value to establish a pixel blend factor such as the following (see block 108): ).
Alpha = Clamp ((Scale factor * Dz + Base factor), 0, 1)
Here, when the clamp is clamp (x, 0, 1), the function returns 0 when x is 0 or less, returns 1 when x is 1 or more, and returns x when 0 <x <1. It is.
[0049]
Assuming that the scale factor is a positive value, the resulting alpha value increases with increasing Dz (eg, clamped to approximately 1 or 1), and decreases with decreasing Dz. (E.g., clamped to approximately zero or zero). Any method can be used to establish the pixel blend factor. For example, another method for calculating the pixel blend factor is shown below.
Gray = Clamp ((Dz-Coefficient A) * Coefficient B, 0, 1)
[0050]
The pixel filter 100 in this example uses a pixel blend coefficient obtained by blending processing (for example, alpha blending), and selectively applies a predetermined border color (for example, line R, line G, line B) to a color frame. The pixel color values (pixel R [x] [y], pixel G [x] [y], pixel B [x] [y]) obtained from the buffer 1042 are blended. An example of calculation for performing this blending is as follows.
New pixel. R [x] [y] = old pixel. R [x] [y] * (1-alpha) + line. R * alpha
New pixel. G [x] [y] = old pixel. R [x] [y] * (1-alpha) + line. G * Alpha
New pixel. B [x] [y] = old pixel. R [x] [y] * (1-alpha) + line. B * Alpha
In accordance with these operations, the blended (“new”) pixel color value is almost always an initial value when the distance value Dz is small, and becomes a border color when the distance value Dz is large.
[0051]
The blend operation described above limits the border color to black (eg, line.R = 0, line.G = 0, line.B = 0) and / or the color of the white canvas (line.R = 255, line .G = 255, line .B = 255) can be used for simplification (eg done for further optimization). For further optimization, a format other than RGB color, for example, YUV (in this case, the brightness Y is controlled so as to ensure that the boundary line is clearly visible) is used.
[0052]
After blending, the blended pixels are buffered and displayed on the display device 1009 (block 112). For example, the pixel filter routine 100 is performed on all the pixels in the frame buffer 1040, and the resulting blended frame buffer contents are displayed on the display device 1009.
[0053]
Perspective processing affects the results obtained by the above processing in some cases. Consider the case where there is a pixel PixA in the distance and a pixel PixB in the vicinity as shown in FIG. Due to the conversion, DzDy of the distant pixel PixA becomes larger than DzDy of the distant pixel PixB. As a result, it becomes more difficult to detect the boundary line in the entire image using unprocessed DzDx and DzDy values. In order to overcome this problem, the Z value may be corrected according to the Z range. For the correction, for example, a log2 (n) function is used as follows for range conversion.
DzDx = | log2 (Z [x] [y]) − log2 (Z [x−1] [y]) |
DzDy = | log2 (Z [x] [y]) − log2 (Z [x] [y−1]) |
If the processor 1010 has a floating point calculation function and the Z value is stored in, for example, the following floating point format, the calculation of the log2 (n) function is very convenient.
[3-bit exponent] [11-bit mantissa]
[0054]
FIG. 11 is obtained by changing the flow chart shown in FIG. 8 and adding this log2 (n) range conversion formula.
[0055]
FIG. 12 is another example of a post-processing pixel filter 200 that performs depth modulation of the alpha value used to control blending. Blocks 204, 206, and 208 shown in FIG. 12 are identical to the corresponding blocks 104, 106, and 108 shown in FIG. 8 (FIG. 11 when range correction is applied). Block 210 of FIG. 12 is added to calculate alpha A as a function (eg, product) of the value calculated in block 208 and a further value that is a function of pixel depth z. For example,
Alpha = Alpha Dz * Alpha Z
The resulting alpha value, that is, the alpha value used for blending control (see block 212) described in connection with block 210 of FIG. 8, also depends on the pixel depth.
[0058]
(Further examples of cartoon outline processing)
Depending on the calculation method, the boundary line application algorithm may not provide satisfactory results in the comic outline processing. Figure 13 Such an example is shown in FIG. Figure 13 As described above, the cartoon character 300 is provided with a border line on the shadow edge 302. Figure 13 From this, it can be seen that this cartoon character 300 has a right hand and a wrist, and the arm is bent at the front. In the above approach, depending on the case (depending on how far your character's arm is from the torso), the edges around the right hand, wrist, and arm are determined to be internal rather than shadow edges, No border is given to those points. Figure 13 Now, when the cartoon outline processing is performed only on the shadow edge 302 of the character, the cartoon character 300 partially disappears or becomes slightly unclear. However, the user expects a border where the hands, wrists and arms are clearly shown (from memories such as coloring books, hand-drawn anime comics, and / or comic books).
[0059]
In order to make the cartoon character 300 appear as if it is hand-drawn, it is preferable to add a border line to the inner edge 304 as well. In this example, the inner edge defines the hand, wrist and arm that the character bends at his front. Figure 14 Then, a boundary line is given to the inner edge 304 of the character 300. This inner edge 304 becomes the edge of the contour line when the character 300 extends its arm outward, 14 In the position of the arm shown in FIG.
[0060]
According to another aspect of the invention, 14 At the inner edge as shown in FIG. 2, the cartoon outline processing is automatically performed. That is, different identification values are assigned to the pixels depending on the portion of the object. For example, this identification value is specified by assigning bits in the frame buffer 1040 and / or DRAM 1126a, which is normally used to encode alpha information. This specified identification value may be used to determine whether or not to add a boundary line to the pixel position. For example, an identification value of a certain pixel is compared with an identification value of a pixel in the vicinity (adjacent) of the pixel. When a predetermined relationship between the identification values of these two pixels is recognized, no boundary line is given. For example, when the identification values are the same, since these two pixels are on the same plane, no boundary line is attached. However, when the identification values of these two pixels are in another predetermined relationship, a boundary line is given.
[0061]
Figure 15 An example is shown in (A) to (C). Figure 15 (A) is a perspective view of the object 319 roughly divided into three parts 320, 322 and 324. Figure 15 (B) is a plan view of the same object 319. The target portion 320 is a rectangle, the target portion 322 is a circle, and the target portion 324 is a cone. Here, the graphics artist has a boundary 330 (see FIG. 5) at a portion where the cone 324 visually contacts the quadrangle 320 but does not contact the circle 322, or a portion where the circle contacts the rectangle. 15 (See (A)). In this case, the pixels within the square 320, the circle 322, and the cone 324 are coded with different identification values. For example, pixels within square 320 are coded with identification value “1”, pixels within circle 322 are coded with identification value “2”, and pixels within cone 324 are coded with identification value “3”. Figure 15 (C) shows an example of the alpha part of the frame buffer 1040 and / or 1126a that stores the coding information (the hatched cells are the cells that give the border color).
[0062]
In the pixel post-processing stage, tests are performed on various identification values in the frame buffer. A border line is not given to pixels having the same identification value, such as adjacent pixels (all such pixels exist on the same plane). In addition, when the identification value of a certain pixel is different from that of an adjacent pixel below a certain reference, no boundary line is given (for example, when the identification value of pixel k does not differ by two or more from the identification value of pixel k + 1, No boundary line is given). However, if the identification value of a certain pixel differs from that of an adjacent pixel by more than a certain reference, a boundary line is given (for example, if the identification value of pixel k differs by two or more from the identification value of pixel k + 1, the boundary line Is applied to the pixel k).
[0063]
Figure 16 These are the flowcharts which show an example of the pixel post-processing routine for providing a boundary line in a present Example. Routine 350 includes a loop (blocks 352-362) that is performed for each pixel [i] [j] in the image stored in frame buffer 1040 and / or 1126a. As described above, in the image generation process, when drawing an image in the frame buffer, an identification value is set for each part of the object in the frame buffer to which bits are normally assigned to normally store the alpha value. To do. In routine 350, these alpha values (here ID values) are tested to determine whether to draw a border. In this example, routine 350 determines the alpha (ID) value of pixel [i] [j] and the alpha (e.g., pixel [i-1] [j] and pixel [i] [j-1]) alpha ( The ID) value is retrieved (block 352). Thereafter, in the routine 352, the following calculation is performed (block 354) to determine the difference between the alpha (ID) value of the pixel [i] [j] and the alpha (ID) value of the adjacent pixel.
diffX = | (alpha [i] [j] −alpha [i−1] [j]) |
diffY = | (alpha [i] [j] −alpha [i] [j−1]) |
[0064]
Thereafter, the routine 350 tests the difference values obtained by the calculation, diffX and diffY, to determine whether one exceeds a predetermined difference value (eg, arbitrarily fixed or 1). Programmable threshold) (block 356). If at least one of the difference values exceeds a predetermined difference value, the routine 350 sets the color of pixel [i] [j] to the border color (block 358). Accordingly, in this example, when the alpha slope is −1 to +1, the pixels are assumed to be on the same plane. Steps 352 through 358 are performed sequentially for each pixel in the image (blocks 360, 362).
[0065]
As a variation of routine 350, the object is coded with a special alpha identification value (eg, 0x00) to identify pixels within the object that should be ignored to provide a border (see FIG. 17 reference). This is useful when drawing an object to which a boundary line is not applied as a bitmap (for example, an explosion animation).
[0066]
Figure 18 Using the routine 350 above, 13 And figure 14 It shows how to draw a boundary line efficiently on the object 300 shown in FIG. In this example, the object is coded with a different alpha (ID) value for each part. For example, the object 300 includes two arms 311 a and 311 b and a body 309. Each arm includes a hand 313, a wrist 315, a lower arm 312, an elbow 308, an upper arm 310, and a shoulder 317. These sites are coded with different alpha IDs as follows:
Body Alpha ID
Left hand 313a 1
Left wrist 315a 2
Lower left arm 312a 3
Left elbow 308a 4
Upper left arm 310a 5
Left shoulder 317a 6
Body 309 7
Right shoulder 317b 8
Upper right arm 310b 9
Right elbow 311b 10
Lower right arm 312b 11
Right wrist 315b 12
Right hand 313b 13
According to the above alpha ID coding example, in routine 350, the diagram 18 Draw a border line as shown by the bold line. However, no boundary line is drawn at the part delimiters (indicated by dotted lines).
[0067]
The above coding can also be used to draw a boundary line between the parts of the same object 300. In traditional coloring books and hand-drawn anime comics, cartoon-like contour processing is also applied to these delimiters to further define joints and make muscles imagine. Figure 19 In (A) to (C), a joint 308 (that is, an elbow) connecting the upper arm 310 and the lower arm of the character 300 is particularly shown. Using the above coding, 19 As shown in (B) and (C), when the upper arms 310 and 312 are positioned adjacent to (contact with) each other by bending the joint 308, in the routine 350 (the alpha ID of the lower arm 310 and the upper arm 312) A border segment 316 is provided between the portions 310 and 312 (based on whether the difference from the alpha ID is greater than 1).
[0068]
Thus, the present invention provides an effective technique for imparting a non-realistic effect such as a cartoon outline in three-dimensional computer graphics. These techniques are used at the pixel post-processing stage (ie, after the 3D graphics pipeline is rendered in the frame buffer) and do not require pixel information other than that provided by the 3D graphics pipeline (ie. , Color, depth and alpha pixel information). Thus, the pixel post-processing techniques of the present invention are effective at “downstream” of a frame buffer (eg, aliasing or other post-processing operation from a portion of the buffer on the circuit to a buffer outside the circuit). In addition, it does not substantially complicate the 3D graphics pipeline or other components of the system.
[0069]
Although the present invention has been described in connection with what is presently considered to be the most realistic and preferred embodiments, the present invention is not limited to the disclosed embodiments, and various modifications and equivalents It is interpreted as including the composition.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an example of a three-dimensional video graphics system for embodying the present invention.
FIG. 2 is a more detailed schematic diagram of an entire interactive 3D computer graphics system 1005 in which the present invention may be implemented.
FIG. 3 is a block diagram of example components within the coprocessor 1020;
FIG. 4 is a more detailed view of the three-dimensional graphics processor 1107 and its associated components within the coprocessor 1020.
FIG. 5 is a diagram exemplarily showing a screen effect in the present invention.
FIG. 6 is a diagram showing an example of the entire emulation process.
FIG. 7 shows a personal computer based host suitable for use with emulator 1303;
FIG. 8 is a flowchart illustrating an example of a pixel filtering routine 100 according to a preferred embodiment.
FIG. 9 is a schematic diagram illustrating an example in which the pixel filter routine 100 reads out the depth values of two adjacent pixels.
10 is a schematic diagram showing perspective processing when a pixel PixA is in the distance and a pixel PixB is in the vicinity. FIG.
11 is a flowchart in which a range conversion formula of log2 (n) is added by changing the flowchart shown in FIG.
FIG. 12 is an example of a post-processing pixel filter 200 that performs depth modulation of alpha values used to control blending.
[Figure 13 FIG. 11 is a diagram showing an example of a cartoon character by a cartoon outline process.
[Figure 14 FIG. 10 is a diagram showing another example of a cartoon character by a cartoon outline process.
[Figure 15 FIG. 11 is a diagram showing a further embodiment of the present invention using an object identification value for specifying a boundary line portion of a cartoon outline processing.
[Figure 16 In the present embodiment, this is a flow chart showing an example of a pixel post-processing routine for providing a boundary line.
[Figure 17 FIG. 10 is a schematic diagram showing a modification of routine 350.
[Figure 18 FIG. 10 is a diagram showing an example of object identification value coding performed to obtain a cartoon outline effect.
[Figure 19 FIG. 10 is a diagram showing another example of object identification value coding performed to obtain a cartoon outline effect.

Claims (12)

An interactive 3D image processing apparatus,
At least one manual controller capable of real-time operation input by the user;
A storage medium for storing three-dimensional data for drawing a character composed of at least one part;
A buffer memory for storing image data including color data and depth data for each pixel; a three-dimensional graphics pipeline connected to the buffer memory;
A filter connected to the buffer memory and applying contour processing to the image data to generate a border around the character;
The three-dimensional graphics pipeline draws image data corresponding to the character in the buffer memory based on at least real-time operation input by the user and the three-dimensional data representing the character,
The storage medium stores different identification values for each of the plurality of parts,
The three-dimensional image processing device, wherein the filter selectively changes the color data based on pixel depth data stored in the buffer memory and an identification value of the part to generate a boundary line.   The filter obtains a difference between depth data corresponding to the pixel and depth data corresponding to one pixel around the pixel, and when the difference exceeds a predetermined threshold, the pixel The three-dimensional image processing apparatus according to claim 1, wherein the color data corresponding to is changed. The buffer memory further includes an alpha value for each pixel,
The three-dimensional image processing apparatus according to claim 1, wherein the filter sets an alpha value corresponding to the pixel to a value corresponding to depth data of the pixel.   The filter selectively sets color data stored in the buffer memory by setting an alpha value corresponding to the pixel to a value corresponding to a difference between the depth of the pixel and the depth of surrounding pixels. The three-dimensional image processing apparatus according to claim 3, wherein the three-dimensional image processing apparatus is changed to:   The pixel is designated as P [x, y] in the image, and the filter has a first pixel designated with a depth of the pixel as P [x-1, y], and P [x , Y−1], the three-dimensional image processing apparatus according to claim 1, wherein the three-dimensional image processing apparatus compares the depth with the depth of the second pixel designated as y−1]. A computer graphics system including data for rendering a three-dimensional image composed of a plurality of parts and storing identification values for the plurality of parts, and a pixel memory for storing color data and depth data corresponding to the pixels of the image A three-dimensional image processing method,
(A) obtaining a user's real-time operation input from at least one manual controller;
(B) reading pixel data from the pixel memory, and selectively changing the color data of the pixel based on the depth data corresponding to the pixel and the identification value of the part;
(C) drawing a three-dimensional image based on the data for drawing the three-dimensional image, the changed color data, and the real-time operation input;
A three-dimensional image processing method comprising: The step of rendering the three-dimensional image obtains a difference between depth data corresponding to the pixel and depth data corresponding to at least one adjacent pixel of the pixel, and thereby the difference exceeds a predetermined threshold value 7. The method according to claim 6 , wherein the color data corresponding to the pixel is changed. The image memory further includes an alpha value for each pixel,
The method of claim 6 , wherein the step of rendering the three-dimensional image sets an alpha value corresponding to the pixel to a value corresponding to depth data of the previous pixel. Step of drawing the three-dimensional image, an alpha value corresponding to the pixel, and sets a value corresponding to the difference between the depth of the depth and at least one of the surrounding pixels of the pixel, claim 8 The method described in 1. The pixel is designated as P [x, y] in the image, and the step of drawing the three-dimensional image designates the depth of the pixel as P [x-1, y]. The method according to claim 6 , characterized in that it compares the depth of one pixel and the second pixel specified to be P [x, y-1]. A computer-readable storage medium storing a program for causing a computer to execute a process of drawing a boundary line on an image,
A reading step of reading out a depth value of each pixel of the image from a memory;
A determination step of determining whether each pixel is an edge of the object using the read depth value;
Changing the pixel on the edge to the border color, and causing the computer to execute
In the determining step, the computer calculates a difference between a depth value of a pixel and a depth value of a pixel around the pixel, determines whether the edge of the object based on the calculated difference,
In the changing step, the computer changes the pixel on the edge to the boundary line color by blending the color of the image before the change and the color of the boundary line at a ratio corresponding to the difference. A storage medium that records the program. An image processing apparatus that executes a process of drawing a boundary line on an image,
Reading means for reading out the depth value of each pixel of the image from the memory;
Discrimination means for discriminating whether each pixel is an edge of the object using the read depth value;
Changing means for changing the pixel on the edge to the color of the boundary line,
The determination unit calculates a difference between a depth value of a pixel and a depth value of a pixel around the pixel, determines whether the edge of the object based on the calculated difference,
The image processing apparatus is configured to change the pixel on the edge to the color of the boundary line by blending the color of the image before the change and the color of the boundary line at a ratio corresponding to the difference .

Images