JP7355960B2 - System and method for efficient multi-GPU rendering of geometry by geometry analysis during rendering - Google Patents

Description

The present disclosure relates to graphic processing, and more specifically to multi-GPU cooperation when rendering images for applications.

In recent years, online services that enable online or cloud gaming in a streaming format between a cloud gaming server and a client connected via a network have been continuously promoted. Streaming formats provide the availability of on-demand game titles, the ability to run more complex games, the ability to network between players for multiplayer games, the ability to share assets between players, and the ability to instantly communicate between players and/or spectators. It has become even more popular due to the ability to share experiences, allow friends to watch friend-play video games, and allow friends to participate in their friend's ongoing gameplay.

A cloud gaming server may be configured to provide resources to one or more clients and/or applications. That is, the cloud game server may be configured with resources capable of high throughput. For example, there are limits to the performance that individual graphics processing units (GPUs) can achieve. To render more complex scenes or use more complex algorithms (e.g. materials, lighting) when generating scenes, use multiple GPUs to render a single image. may be desirable. However, even utilization of these graphics processing units is difficult to achieve. Additionally, even if multiple GPUs are present to process images for an application using traditional technology, there is no ability to support a corresponding increase in both the number of screen pixels and the geometry density (e.g., four It is not possible for a GPU to write four times as many pixels and/or process four times as many vertices or primitives to an image).

The embodiments of the present disclosure have been made against this background.

Embodiments of the present disclosure may be performed by performing geometry analysis during rendering to generate information used for dynamic allocation of screen space to the GPU for rendering of image frames, and/or prior to rendering. multi-GPU rendering of geometry for applications, etc. by performing geometry analysis during the rendering phase and/or by performing timing analysis during the rendering phase to redistribute GPU responsiveness allocation during the rendering phase. , relates to using multiple GPUs (graphics processing units) in conjunction to render a single image.

Embodiments of the present disclosure disclose a method for graphics processing. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes using multiple GPUs in conjunction to render an image frame that includes multiple pieces of geometry. The method includes generating information about a plurality of geometry pieces and their relationships to a plurality of screen regions on a GPU during a pre-pass phase of rendering. The method includes allocating screen areas to GPUs based on the information for rendering the pieces of geometry in a subsequent phase of rendering.

In other embodiments of the present disclosure, a computer system is disclosed that includes a processor and a memory coupled to the processor, the memory storing instructions that, when executed by the computer system, perform graphics processing. cause a computer system to execute a method for The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes using multiple GPUs in conjunction to render an image frame that includes multiple pieces of geometry. The method includes generating information about a plurality of geometry pieces and their relationships to a plurality of screen regions on a GPU during a pre-pass phase of rendering. The method includes allocating screen areas to GPUs based on the information for rendering the pieces of geometry in a subsequent phase of rendering.

Still other embodiments of the present disclosure disclose a non-transitory computer readable medium storing a computer program for graphics processing. A computer-readable medium includes program instructions for rendering graphics for an application using multiple graphics processing units (GPUs). The computer-readable medium includes program instructions for cooperatively using multiple GPUs to render an image frame that includes multiple pieces of geometry. The computer-readable medium includes program instructions for generating information about a plurality of geometry pieces and their relationships to a plurality of screen regions on a GPU during a pre-pass phase of rendering. The computer-readable medium includes program instructions for allocating screen areas to GPUs based on the information for rendering the pieces of geometry in a subsequent phase of rendering.

Embodiments of the present disclosure disclose a method for graphics processing. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes partitioning the responsiveness of processing multiple geometry pieces of an image frame during an analysis pre-pass phase of rendering among multiple GPUs, each of the multiple geometry pieces being assigned to a corresponding GPU. It will be done. The method includes determining an overlap of each of the plurality of geometry pieces with each of the plurality of screen regions in an analysis pre-pass phase. The method includes generating information about the plurality of geometry pieces and their relationships to the plurality of screen regions at the plurality of GPUs based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. including. The method includes allocating screen areas to GPUs based on the information for rendering the pieces of geometry during a subsequent phase of rendering.

In other embodiments of the present disclosure, a computer system is disclosed that includes a processor and a memory coupled to the processor, the memory storing instructions that, when executed by the computer system, perform graphics processing. cause a computer system to execute a method for The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes partitioning the responsiveness of processing multiple geometry pieces of an image frame during an analysis pre-pass phase of rendering among multiple GPUs, each of the multiple geometry pieces being assigned to a corresponding GPU. It will be done. The method includes determining an overlap of each of the plurality of geometry pieces with each of the plurality of screen regions in an analysis pre-pass phase. The method includes generating information about the plurality of geometry pieces and their relationships to the plurality of screen regions at the plurality of GPUs based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. including. The method includes allocating screen areas to GPUs based on the information for rendering the pieces of geometry during a subsequent phase of rendering.

Still other embodiments of the present disclosure disclose a non-transitory computer readable medium storing a computer program for graphics processing. A computer-readable medium includes program instructions for rendering graphics for an application using multiple graphics processing units (GPUs). The computer-readable medium includes program instructions for partitioning a responsibility for processing multiple pieces of geometry of an image frame during an analysis pre-pass phase of rendering among multiple GPUs, each of the multiple pieces of geometry comprising: Assigned to the corresponding GPU. The computer readable medium includes program instructions for determining an overlap of each of the plurality of geometry pieces with each of the plurality of screen regions in an analysis pre-pass phase. The computer-readable medium generates information about the plurality of geometry pieces and their relationships to the plurality of screen regions at the plurality of GPUs based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. Contains program instructions for The computer-readable medium includes program instructions for allocating screen areas to GPUs based on the information for rendering the pieces of geometry during a subsequent phase of rendering.

Embodiments of the present disclosure disclose a method for graphics processing. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes using multiple GPUs in conjunction to render an image frame that includes multiple pieces of geometry. The method subdivides one or more of the pieces of geometry into smaller pieces during rendering of an image frame, and divides the responsiveness of rendering smaller parts of these geometries between multiple GPUs. each smaller portion of the geometry is processed by a corresponding GPU. The method includes, for pieces of geometry that are not subdivided, dividing the responsibility of rendering the pieces of geometry among a plurality of GPUs, each of these pieces of geometry being processed by a corresponding GPU.

In other embodiments of the present disclosure, a computer system is disclosed that includes a processor and a memory coupled to the processor, the memory storing instructions that, when executed by the computer system, perform graphics processing. cause a computer system to execute a method for The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes using multiple GPUs in conjunction to render an image frame that includes multiple pieces of geometry. The method subdivides one or more of the pieces of geometry into smaller pieces during rendering of an image frame, and divides the responsiveness of rendering smaller parts of these geometries between multiple GPUs. each smaller portion of the geometry is processed by a corresponding GPU. The method includes, for pieces of geometry that are not subdivided, dividing the responsibility of rendering the pieces of geometry among a plurality of GPUs, each of these pieces of geometry being processed by a corresponding GPU.

Still other embodiments of the present disclosure disclose a non-transitory computer-readable medium storing a computer program for graphics processing. A computer-readable medium includes program instructions for rendering graphics for an application using multiple graphics processing units (GPUs). The computer-readable medium includes program instructions for cooperatively using multiple GPUs to render an image frame that includes multiple pieces of geometry. The computer-readable medium subdivides one or more of the plurality of pieces of geometry into smaller pieces during rendering of an image frame, and provides responsibility for rendering the smaller portions of these geometries between the plurality of GPUs. , and each smaller part of the geometry is processed by a corresponding GPU. The computer-readable medium includes program instructions for partitioning, for pieces of geometry that are not subdivided, the responsibility of rendering the pieces of geometry among multiple GPUs, each of these pieces of geometry Processed by

Other aspects of the disclosure will be apparent from the following detailed description, taken in conjunction with the accompanying drawings, which serve as examples of the principles of the disclosure.

The present disclosure can be best understood by reference to the following detailed description, taken in conjunction with the accompanying drawings.

and/or by performing geometry analysis during rendering to generate information used for dynamic allocation of screen area to the GPU for further rendering passes of the image frame, according to embodiments of the present disclosure. Multi-GPU (graphics processing) of geometry for applications by performing geometry analysis before phasing and/or by subdividing pieces of geometry and assigning smaller parts of the resulting geometry to multiple GPUs. unit) for serving games over a network between one or more cloud game servers configured to implement multiple GPUs working together to render a single image, including rendering FIG. 2 is a diagram of the system. 1 is a diagram of a multi-GPU architecture in which multiple GPUs cooperate to render a single image, according to an embodiment of the present disclosure. FIG. By performing geometry analysis during rendering, and/or by performing geometry analysis before rendering, and/or by subdividing pieces of geometry and resolving the resulting geometry, according to embodiments of the present disclosure. FIG. 2 is an illustration of multiple graphics processing unit resources configured for multi-GPU rendering of geometry for an application by allocating smaller portions to multiple GPUs. 1 is a diagram of a rendering architecture that implements a graphics pipeline configured for multi-GPU processing, such that multiple GPUs work together to render a single image, according to an embodiment of the present disclosure. FIG. FIG. 2 is an illustration of a screen subdivided into quadrants when performing multi-GPU rendering, according to an embodiment of the present disclosure. FIG. 3 is an illustration of a screen that is subdivided into multiple interleaved regions when performing multi-GPU rendering, according to an embodiment of the present disclosure. 3 illustrates object testing for screen regions when multiple GPUs collaborate to render a single image, according to an embodiment of the present disclosure; FIG. 2 illustrates testing a portion of an object against a screen area when multiple GPUs collaborate to render a single image, according to an embodiment of the present disclosure; FIG. 2 is a flowchart illustrating a method of graphics processing including multi-GPU rendering of geometry for an application by performing geometry analysis during rendering, according to an embodiment of the present disclosure. FIG. 3 is a screen diagram illustrating dynamic allocation of screen area to a GPU for geometry rendering based on an analysis of the geometry of a current image frame performed during rendering of the current image frame, according to an embodiment of the present disclosure; . FIG. 6 illustrates rendering of an image frame including four objects including a Z pre-pass phase and a geometry phase for rendering the image frame, where the Z pre-pass phase is used for dynamic allocation of screen area, according to an embodiment of the present disclosure; is executed to generate information to the GPU for geometric rendering of the image frame. FIG. 6 is a diagram illustrating rendering of an image frame including four objects including a Z pre-pass phase and a geometry phase for rendering the image frame, where the Z pre-pass phase is used for dynamic allocation of screen area, according to an embodiment of the present disclosure; is executed to generate information to the GPU for geometric rendering of the image frame. FIG. 6 illustrates rendering of an image frame including four objects including a Z pre-pass phase and a geometry phase for rendering the image frame, where the Z pre-pass phase is used for dynamic allocation of screen area, according to an embodiment of the present disclosure; is executed to generate information to the GPU for geometric rendering of the image frame. While rendering an image frame, an entire object or a portion of an object is used for geometry rendering based on an analysis of the geometry of the current image frame performed during the Z pre-pass phase of rendering, according to an embodiment of the present disclosure. 3 illustrates rendering image frames using dynamic allocation of screen area to GPUs based on screen area. of an image frame to perform a Z pre-pass phase of rendering to generate information used for dynamic allocation of screen space to the GPU for geometry rendering of the image frame according to an embodiment of the present disclosure. FIG. 3 illustrates interleaving GPU assignments to pieces of geometry. 2 is a flowchart illustrating a method of graphics processing including multi-GPU rendering of geometry for an application by performing geometry analysis before rendering, according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an analysis pre-pass performed before the image frame rendering phase, in accordance with an embodiment of the present disclosure, where the analysis pre-pass is used to dynamically allocate screen space to the GPU for geometry rendering of the image frame; Generate information to be used. Pieces of geometry and screen area when performing an analysis pre-pass to generate information used for dynamic allocation of screen area to GPUs for geometry rendering of image frames, according to an embodiment of the present disclosure. FIG. 3 illustrates calculation of exact overlap between. Pieces of geometry and screen space when performing an analysis pre-pass to generate information used for dynamic allocation of screen space to GPUs for geometry rendering of image frames, according to an embodiment of the present disclosure. FIG. 3 is a pair of diagrams illustrating calculation of approximate overlap between. such as when performing a Z pre-pass phase on a piece of geometry to generate information used to dynamically allocate screen space to a GPU for geometry rendering of an image frame, according to an embodiment of the present disclosure; A method for graphics processing, including multi-GPU rendering of geometry for an application, by performing timing analysis during the rendering or analysis phase to redistribute GPU responsiveness allocation during the rendering or analysis phase. FIG. A variety of GPU allocations that perform a Z pre-pass phase of rendering to generate information used for dynamic allocation of screen space to GPUs for geometry rendering of image frames, according to an embodiment of the present disclosure. It is a figure showing dispersion. FIG. 3 is a diagram illustrating the use of multiple GPUs to render pieces of geometry in a screen area, according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating rendering pieces of geometry out of order with their corresponding draw calls, according to an embodiment of the present disclosure. 2 illustrates components of an example device that can be used to implement aspects of various embodiments of this disclosure.

Although the following detailed description includes many specific details for purposes of illustration, those skilled in the art will recognize that many variations and modifications to the following details will fall within the scope of this disclosure. . Accordingly, the aspects of the disclosure described below are presented without loss of generality to or imposition of limitations on the claims that follow this description.

Generally speaking, there are limits to the performance that an individual GPU can achieve, derived from, for example, limits on how large a GPU can be. To render more complex scenes or use more complex algorithms (e.g. materials, lighting, etc.), multiple GPUs can be used in conjunction to generate and/or render a single image frame. is desirable. For example, rendering responsiveness can be determined based on information determined from geometric analysis of objects and/or pieces of geometry (e.g., parts of objects, primitives, polygons, vertices, etc.) within an image frame. Split between multiple GPUs. This information provides the relationship between the potentially interleaved geometry and each screen area. This may allow the GPU to render the geometry more efficiently or avoid rendering it all together. In particular, various embodiments of the present disclosure provide for analysis of the geometry of image frames and dynamically and flexibly allocate responsibilities for rendering image frames among GPUs, with each GPU ultimately specific to that image frame. (i.e., the next image frame may have a different association of GPUs to screen regions). Through geometric analysis and dynamic allocation of rendering responsiveness to the GPU on a per-image frame basis, embodiments of the present disclosure can be used to increase pixel count (i.e., resolution) and complexity, and/or increase geometric complexity. and/or an increase in the amount of processing per vertex and/or primitive. Specifically, various embodiments of the present disclosure provide multi-GPU rendering of geometry for applications by performing geometry analysis during rendering that dynamically allocates screen space to GPUs for geometry rendering of image frames. A method and system configured to perform a geometry analysis is described, wherein the geometry analysis is based on information defining a relationship between geometry rendered for an image frame and a screen area. Geometry analysis information is generated during rendering, eg, during a Z pre-pass before geometry rendering. Specifically, the hardware is configured such that the prepass generates information that is used to assist in intelligently allocating screen space to the GPU when performing geometry rendering during subsequent phases of rendering. be done. Other embodiments of the present disclosure provide a method for multiplying the geometry of an application by performing geometry analysis prior to the rendering phase to dynamically allocate screen space to the GPU for that phase of image frame rendering. A method and system configured to perform GPU rendering is described, where the geometry analysis is based on information that defines a relationship between geometry rendered for an image frame and a screen area. For example, the information is generated in a pre-pass performed prior to rendering, such as using a shader (eg, software). This information is used to intelligently allocate screen space to GPUs when performing geometry rendering. Yet other embodiments of the present disclosure subdivide pieces of geometry, such as those processed or generated by draw calls, into smaller portions of geometry and use multiple GPUs for rendering those smaller portions of geometry. A method and system are described that are configured such that each smaller portion of the geometry is allocated to a GPU. As an advantage, for example, this allows multiple GPUs to render more complex scenes and/or images in the same amount of time.

With the above general understanding of various embodiments, details of example embodiments will now be described with reference to various drawings.

Throughout this specification, references to "application" or "game" or "video game" or "gaming application" are meant to refer to any type of interactive application that is directed through the execution of input commands. For illustrative purposes only, interactive applications include applications for gaming, word processing, video processing, video game processing, and the like. Furthermore, these terms are interchangeable.

Throughout this specification, various embodiments of the present disclosure are described for multi-GPU processing or geometry rendering for applications using an exemplary architecture with four GPUs. However, it is understood that any number of GPUs (eg, two or more GPUs) can cooperate when rendering the geometry of an application.

FIG. 1 is a diagram of a system for performing multi-GPU processing when rendering images (eg, image frames) for an application, according to one embodiment of the present disclosure. The system is configured to provide games over a network between one or more cloud game servers in accordance with embodiments of the present disclosure, and more specifically, coordinates multiple GPUs to provide applications for It is configured to render a single image of an image frame during or before rendering, e.g. to dynamically allocate screen space to the GPU for image frame geometry rendering. When performing geometry analysis for , in which each smaller portion of the geometry is assigned to a GPU. Cloud gaming involves running a video game on a server to generate game-rendered video frames, which are then sent to a client for display. Specifically, system 100 is configured for efficient multi-GPU rendering of application geometry by pre-testing on interleaved screen regions before rendering.

While FIG. 1 illustrates an implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system, other embodiments of the present disclosure provide efficient multi-GPU rendering of geometry of an application. Rendering is provided by performing area tests while rendering within a standalone system, such as a personal computer or game console, including a high-end graphics card with multiple GPUs.

It is also understood that multi-GPU rendering of geometry may be performed in various embodiments (eg, within a cloud gaming environment or standalone system) using physical GPUs, or virtual GPUs, or a combination of both. For example, a virtual machine (e.g., an instance) is a host hardware (e.g., located in a data center) that utilizes one or more components of the hardware layer, such as multiple CPUs, memory modules, GPUs, network interfaces, communication components, etc. can be created using a hypervisor ( These physical resources can be placed in racks, such as racks of CPUs, racks of GPUs, and racks of memory, and are used for assembly and access of components used by the instance (such as when building virtualized components of the instance). The top of the rack switch facilitates access to physical resources within the rack. Typically, a hypervisor can present multiple guest operating systems in multiple instances configured with virtual resources. That is, each of the operating systems may be configured with a corresponding set of virtualized resources supported by one or more hardware resources (eg, located in a corresponding data center). For example, each operating system may be supported with a virtual CPU, multiple virtual GPUs, virtual memory, virtualized communication components, and so on. Additionally, instance configurations can be transferred from one data center to another to reduce latency. GPU utilization defined for a user or game can be used when saving a user's game session. GPU utilization may include any number of configurations described herein to optimize fast rendering of video frames for gaming sessions. In one embodiment, GPU utilization defined for a game or user can be transferred between data centers as a configurable setting. The ability to transfer GPU usage allows for efficient migration of gameplay from data center to data center when users connect to play games from different geographic locations.

The system 100 provides games via a cloud gaming network 190 and, according to one embodiment of the present disclosure, the games are played remotely from a client device 110 (e.g., a thin client) of a corresponding user playing the game. is being executed. System 100 provides control of a game over network 150 to one or more users who play one or more games over cloud gaming network 190 in either single-player or multiplayer mode. Can be done. In some embodiments, cloud gaming network 190 may include multiple virtual machines (VMs) running on a host machine's hypervisor, one or more virtual machines being available to the host's hypervisor. The game processor module is configured to execute a game processor module utilizing available hardware resources. Network 150 may include one or more communication technologies. In some embodiments, network 150 may include fifth generation (5G) network technology with advanced wireless communication systems.

In some embodiments, communication may be facilitated using wireless technology. Such technology may include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. 5G networks are digital cellular networks in which the service area covered by a provider is divided into small geographical areas called cells. Analog signals representing sound and images are digitized by the phone, converted by an analog-to-digital converter, and transmitted as a stream of bits. All 5G wireless devices within a cell connect to local antenna arrays within the cell and low-power automated transceivers (transmitters and receivers) via frequency channels allocated by the transceiver from a pool of frequencies that are reused in other cells. ) to communicate via radio waves. Local antennas are connected to the telephone network and the Internet by high bandwidth optical fiber or wireless backhaul connections. Similar to other cell networks, mobile devices that move from one cell to another are automatically transferred to the new cell. It is understood that a 5G network is just one example type of communication network, and embodiments of the present disclosure may utilize previous generations of wireless or wired communications, as well as later generations of wired or wireless technologies following 5G. I want to be

As shown, cloud gaming network 190 includes a game server 160 that provides access to multiple video games. Game server 160 may be any type of server computing device available in the cloud and may be configured as one or more virtual machines running on one or more hosts. For example, game server 160 may manage a virtual machine that supports a game processor that instantiates instances of a user's game. Thus, multiple game processors of game server 160 associated with multiple virtual machines are configured to execute multiple instances of one or more games associated with multiple users' gameplay. As such, the backend server support provides streaming of gameplay media (eg, video, audio, etc.) for multiple gaming applications to corresponding multiple users. That is, game server 160 is configured to stream data (eg, rendered images and/or frames of corresponding gameplay) back to corresponding client device 110 via network 150 . In that way, computationally complex gaming applications can continue to run on the backend server in response to controller inputs received and forwarded by the client device 110. Each server may render images and/or frames and then encode (eg, compress) and stream them to a corresponding client device for display.

For example, multiple users may access cloud gaming network 190 via communication network 150 using corresponding client devices 110 configured to receive streaming media. In one embodiment, client device 110 is a thin client that provides an interface with a backend server (e.g., cloud gaming network 190) configured to provide computational functionality (e.g., includes game title processing engine 111). It can be configured as In another embodiment, the client device 110 may be configured with a game title processing engine and game logic for local processing of at least some of the video game and streaming content generated by the video game running on a backend server. or for other content provided by backend server support. For local processing, the game title processing engine includes the basic processor-based functionality for running the video game and services related to the video game. In that case, game logic can be stored on local client device 110 and used to run the video game.

Each of the client devices 110 may be requesting access to a different game from the cloud gaming network. For example, cloud gaming network 190 may be running one or more game logic built on game title processing engine 111 to be executed using CPU resources 163 and GPU resources 365 of game server 160. You can. For example, game logic 115a that works with the game title processing engine 111 is executed on the game server 160 of one client, and game logic 115b that works with the game title processing engine 111 is executed on the game server 160 of a second client. , and game logic 115n in conjunction with game title processing engine 111 may be executed on game server 160 of the Nth client.

In particular, a client device 110 of a corresponding user (not shown) may request access to a game via a communication network 150, such as the Internet, and display images generated by a video game executed by a game server 160. (eg, image frames), where the encoded images are delivered to the client device 110 for display in association with a corresponding user. For example, a user may interact through client device 110 with an instance of a video game running on a game processor of game server 160. More specifically, instances of video games are executed by game title processing engine 111. Corresponding game logic (eg, executable code) 115 implementing the video game is stored and accessible via a data store (not shown) and is used to run the video game. Game title processing engine 111 may support multiple video games using multiple game logic (eg, game applications), each of which is selectable by a user.

For example, client device 110 is configured to interact with game title processing engine 111 associated with the corresponding user's gameplay, such as via input commands used to drive the gameplay. In particular, client device 110 may receive input from various types of input devices such as game controllers, tablet computers, keyboards, gestures captured by video cameras, mice, touch pads, and the like. Client device 110 may be any type of computing device having at least memory and a processor module, and may be connected to game server 160 via network 150. Backend game title processing engine 111 is configured to generate rendered images, which are distributed over network 150 for display on a corresponding display associated with client device 110. For example, via a cloud-based service, game rendered images may be delivered by an instance of a corresponding game (eg, game logic) running on game execution engine 111 of game server 160. That is, client device 110 is configured to receive encoded images (e.g., encoded from game rendered images generated through execution of a video game) and display the rendered images on display 11. be done. In one embodiment, display 11 includes an HMD (eg, displays VR content). In some embodiments, rendered images may be streamed wirelessly or wired to a smartphone or tablet directly from a cloud-based service or via a client device 110 (e.g., PlayStation® Remote Play). I can do it.

In one embodiment, game server 160 and/or game title processing engine 111 includes the basic processor-based functionality for running games and services related to game applications. For example, game server 160 may provide processor-based services including 2D or 3D rendering, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, raytracing, shadowing, culling, transformations, artificial intelligence, etc. It includes central processing unit (CPU) resources 163 and graphics processing unit (GPU) resources 365 configured to perform functions. In addition, the CPU and GPU groups support memory management, multithread management, quality of service (QoS), bandwidth testing, social networking, managing social friends, communicating with social networks of friends, communication channels, text messages, instant messaging, Services may be implemented for gaming applications, including in part chat support. In one embodiment, one or more applications share certain GPU resources. In one embodiment, multiple GPU devices may be combined to perform graphics processing for a single application running on a corresponding CPU.

In one embodiment, cloud gaming network 190 is a distributed gaming server system and/or architecture. Specifically, a distributed game engine that executes game logic is configured as a corresponding instance of a corresponding game. Generally, a distributed game engine takes the functions of a game engine and distributes those functions to be performed by multiple processing entities. Individual functionality may be further distributed across one or more processing entities. Processing entities may be configured in various configurations, including physical hardware and/or as virtual components or machines, and/or as virtual containers, where a container is configured on a virtualized operating system. It is different from a virtual machine because it virtualizes an instance of a running game application. The processing entity may utilize and/or rely on servers and their underlying hardware on one or more servers (compute nodes) of cloud gaming network 190, where the servers may can be placed on a rack. Coordination, assignment, and management of the performance of their functions to the various processing entities is done by a distributed synchronization layer. As such, the execution of those functions is controlled by the distributed synchronization layer to enable generation of media (e.g., video frames, audio, etc.) for the gaming application in response to controller input by the player. . A distributed synchronization layer distributes critical game engine components/functions across distributed processing entities (e.g., via load balancing) so that they are distributed and rebuilt for more efficient processing. It is possible to execute it efficiently.

FIG. 2 is a diagram of an example multi-GPU architecture 200 in which multiple GPUs work together to render a single image of a corresponding application, according to one embodiment of the present disclosure. According to various embodiments of the present disclosure, the multi-GPU architecture 200 dynamically allocates screen area to GPUs for geometry rendering of image frames during or before rendering, and/or for example, when drawing of a piece of geometry in an image frame when subdividing a piece of geometry as processed or produced by a call into smaller pieces of geometry and assigning those smaller pieces of geometry to multiple GPUs for rendering. It is configured to perform geometry analysis, and each smaller portion of the geometry is assigned to a GPU. Although not explicitly described or illustrated, it is understood that many architectures are possible in various embodiments of the present disclosure in which multiple GPUs cooperate to render a single image. Multi-GPU rendering of geometry for applications, e.g. by performing area tests during rendering, can also be implemented between one or more cloud gaming servers in a cloud gaming system, or a personal computer or computer with multiple GPUs. It can also be implemented within standalone systems such as game consoles that include high-end graphics cards.

Multi-GPU architecture 200 includes a CPU 163 and multiple GPUs configured for multi-GPU rendering of a single image (also referred to as an "image frame") for an application, and/or each image of a series of images for an application. include. Specifically, as described above, the CPU 163 and GPU resources 365 are used for 2D or 3D rendering, physics simulation, script creation, audio, animation, graphic processing, lighting, shading, rasterization, ray tracing, shadowing, culling, and conversion. , configured to perform processor-based functions, including artificial intelligence, etc.

For example, although four GPUs are shown in GPU resources 365 of multi-GPU architecture 200, any number of GPUs may be utilized when rendering images for an application. Each GPU is connected to a corresponding dedicated memory, such as random access memory (RAM), via a high speed bus 220. Specifically, GPU-A is connected to memory 210A (e.g., RAM) via bus 220, GPU-B is connected to memory 210B (e.g., RAM) via bus 220, and GPU-C is connected to memory 210A (e.g., RAM) via bus 220. GPU-D is connected to memory 210D (eg, RAM) via bus 220.

Additionally, each GPU is connected to each other via a bus 240, which, depending on the architecture, is approximately equal in speed to or similar to bus 220 used for communication between the corresponding GPU and its corresponding memory. It can be slower. For example, GPU-A is connected to each of GPU-B, GPU-C, and GPU-D via bus 240. Further, GPU-B is connected to each of GPU-A, GPU-C, and GPU-D via a bus 240. Additionally, GPU-C is connected to each of GPU-A, GPU-B, and GPU-D via bus 240. Additionally, GPU-D is connected to each of GPU-A, GPU-B, and GPU-C via bus 240.

CPU 163 connects to each GPU via a slow bus 230 (eg, bus 230 is slower than bus 220 used for communication between a corresponding GPU and its corresponding memory). Specifically, the CPU 163 is connected to each of GPU-A, GPU-B, GPU-C, and GPU-D.

In some embodiments, the four GPUs are separate GPUs, each on its own silicon die. In other embodiments, four GPUs may share a die to take advantage of high speed interconnects and other units on the die. Still other embodiments may use it as either a single more powerful GPU or as four less powerful "virtual" GPUs (GPU-A, GPU-B, GPU-C and GPU-D). There is one physical GPU 250 that can be configured as follows. In other words, GPU-A, GPU-B, GPU-C, and GPU-D each have sufficient functionality to operate the graphics pipeline (as shown in Figure 4), and the entire chip operates the graphics pipeline. (as shown in Figure 4), and the configuration can be flexibly switched between the two configurations (eg, between rendering passes).

FIG. 3 illustrates processing or processing during or before rendering and/or, e.g., by draw calls, to dynamically allocate screen space to the GPU for geometry rendering of image frames, according to various embodiments of the present disclosure. Perform geometry analysis on a piece of geometry in an image frame when subdividing such a generated piece of geometry into smaller pieces of geometry and assigning those smaller pieces of geometry to multiple GPUs for rendering 3 is a diagram of a graphics processing unit resource 365 configured for multi-GPU rendering of the geometry of an image frame generated by an application, with each smaller portion of the geometry allocated to a GPU. For example, game server 160 may be configured to include GPU resources 365 in cloud gaming network 190 of FIG. As illustrated, the GPU resource 365 includes a plurality of GPUs such as GPU 365a, GPU 365b, . . . GPU 365n. As mentioned above, various architectures implement multi-GPU rendering of geometry between one or more cloud gaming servers in a cloud gaming system, or in a personal computer or gaming console that includes a high-end graphics card with multiple GPUs. Implementing multi-GPU rendering of geometry within a standalone system, etc., working together to render a single image by performing multi-GPU rendering of the application's geometry through region testing during rendering, etc. It can include multiple GPUs.

Specifically, in one embodiment, the game server 160 allows multiple GPUs to work together to render a single image and/or to render each of one or more images in a series of images during application execution. The application is configured to perform multi-GPU processing when rendering a single image of the application. For example, in one embodiment, game server 160 may include a CPU and GPU group configured to perform multi-GPU rendering of each of one or more images in a series of images of an application; The CPU and GPU groups may implement graphics and/or render pipelines for applications. CPU and GPU groups can be configured as one or more processing devices. As mentioned above, GPUs and GPU groups can include CPUs 163 and GPU resources 365, which can be used for 2D or 3D rendering, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, late Configured to perform processor-based functions including lacing, shadowing, culling, transformation, artificial intelligence, etc.

GPU resources 365 are responsible for rendering objects (e.g., writing the object's pixel colors or normal vector values to a multiple render target - MRT) and executing synchronous computation kernels (e.g., full-screen effects on the resulting MRT). The synchronized computations to be performed and the objects to be rendered are specified by commands contained in a plurality of rendering command buffers 325 that are executed by the GPU. Specifically, GPU resources 365 are configured to perform synchronous computations upon rendering objects and executing commands from rendering command buffer 325 (e.g., during execution of a synchronous computation kernel), where the commands and/or operations may depend on other operations to be performed in sequence.

For example, GPU resources 365 are configured to perform synchronous computations and/or rendering of objects using one or more rendering command buffers 325 (e.g., rendering command buffer 325a, rendering buffer 325b...rendering command buffer 325n). has been done. In one embodiment, each GPU within GPU resource 365 may have its own command buffer. Alternatively, when substantially the same set of objects is being rendered by each GPU (e.g., because the size of the region is small), the GPUs in GPU resource 365 may use the same command buffer or the same set of command buffers. Can be done. Additionally, each GPU in GPU resource 365 may support functionality whose commands are executed by one GPU but not another GPU. For example, a flag in a drawing command or predicate in a rendering command buffer allows a single GPU to execute one or more commands in the corresponding command buffer, while other GPUs ignore the commands. For example, rendering command buffer 325a may support flag 330a, rendering command buffer 325b may support flag 330b, and rendering command buffer 325n may support flag 330n.

The performance of synchronous computations (eg, execution of synchronous computation kernels) and rendering of objects are part of the overall rendering. For example, if a video game is running at 60Hz (e.g., 60 frames/second), all object rendering and synchronized computation kernels for an image frame typically execute within about 16.67ms (e.g., one frame at 60Hz). must be completed. As mentioned above, operations when rendering objects and/or executing synchronous computation kernels are ordered such that operations may depend on other operations (e.g., commands in the rendering command buffer are (It may be necessary to complete execution before other commands in that rendering command buffer are executed.)

Specifically, each of the rendering command buffers 325 contains commands that affect the corresponding GPU configuration (e.g., commands that specify the position and format of a render target), as well as commands that render objects and/or synchronize computation kernels. Contains various types of commands, including commands to execute. To illustrate, the synchronous calculations performed when running a synchronous calculation kernel include the ability to perform full-screen effects when objects are all rendered to their corresponding one or more multiple render targets (MRTs). may be included.

Further, when GPU resource 365 renders an object for an image frame and/or executes a synchronous computation kernel when generating an image frame, GPU resource 365 configures the be done. For example, GPU 365a is configured to perform its rendering or computational kernel execution in a particular manner via its registers 340 (eg, register 340a, register 340b...register 340n). That is, the value stored in register 340 is used by GPU 365a 365 hardware when executing commands in rendering command buffer 325 that are used to render objects and/or execute image frame synchronization calculation kernels. Define the context (eg, GPU configuration or GPU state). Each of the GPUs in GPU resources 365 allows GPU 365b to be configured via its registers 350 (e.g., register 350a, register 350b...register 350n) to perform its rendering in a particular manner or compute kernel execution. may be similarly configured. GPU 365n is then configured via its registers 370 (eg, register 370a, register 370b...register 370n) to perform its rendering or computational kernel execution in a particular manner.

Examples of GPU configurations include the location and format of render targets (such as MRT). Further, other examples of GPU configurations include operating procedures. For example, when rendering an object, the Z value of each pixel of the object can be compared to the Z buffer in various ways. For example, pixels of an object are written only if the object's Z value matches the value in the Z buffer. Alternatively, pixels of an object may be written only if the object's Z value is less than or equal to the value in the Z buffer. The type of test performed is defined within the GPU configuration.

FIG. 4 is a simplified diagram of a rendering architecture implementing a graphics pipeline 400 configured for multi-GPU processing, such that multiple GPUs work together to render a single image, according to one embodiment of the present disclosure. It is. Graphics pipeline 400 is illustrative of a general process for rendering images using a 3D (three-dimensional) polygon rendering process. The graphics pipeline 400 for rendered images outputs corresponding color information for each pixel to the display, where the color information can represent texture and shading (eg, color, shadowing, etc.). Graphics pipeline 400 may be implementable within client device 110, game server 160, game title processing engine 111, and/or GPU resources 365 of FIGS. 1 and 3. In other words, various architectures implement multi-GPU rendering of geometry between one or more cloud game servers in a cloud gaming system, or in a standalone system such as a personal computer or game console containing a high-end graphics card with multiple GPUs. Implementing multi-GPU rendering of geometry within the system, including multiple It can include a GPU.

As shown, the graphics pipeline receives input geometry 405. For example, geometry processing stage 410 receives input geometry 405. For example, input geometry 405 may include vertices within the 3D gaming world and information corresponding to each of the vertices. A given object within the gaming world can be represented using a polygon (e.g., a triangle) defined by vertices, and the corresponding polygon surface is then processed via the graphics pipeline 400. , to achieve the final effect (e.g. color, texture, etc.). Vertex attributes include normal (e.g., which direction is perpendicular to the geometry at that location), color (e.g., RGB - triple red, green, blue, etc.), and texture coordinates/mapping information. It can be done.

The geometry processing stage 410 is responsible for (and can perform) both vertex processing (eg, via a vertex shader) and primitive processing. Specifically, geometry processing stage 410 not only outputs a set of vertices that define a primitive and delivers them to the next stage of graphics pipeline 400, but also determines the positions of those vertices (more precisely, homogeneous coordinates) and various other parameters. The location is placed in a location cache 450 for access by later shader stages. Other parameters are placed in parameter cache 460, also for access by later shader stages.

Various operations may be performed by geometry processing stage 410, such as performing lighting and shadowing calculations for primitives and/or polygons. In one embodiment, the geometry stage can process primitives and thus perform backface culling and/or clipping (e.g., testing against view frustums), thereby freeing up downstream stages (e.g., rasterization stage 420, etc.). Reduce. In another embodiment, the geometry stage may generate primitives (eg, have functionality equivalent to a traditional geometry shader).

The primitives output by the geometry processing stage 410 are provided to a rasterization stage 420 that converts the primitives into raster images made up of pixels. Specifically, rasterization stage 420 projects objects in the scene onto a two-dimensional (2D) image plane defined by a viewpoint within the 3D gaming world (e.g., camera position, user eye position, etc.). It is composed of At a simplified level, rasterization stage 420 examines each primitive and determines which pixels are affected by the corresponding primitive. Specifically, rasterizer 420 divides the primitive into pixel-sized fragments, each fragment corresponding to a pixel in the display. It is important to note that one or more fragments may contribute to the color of the corresponding pixel when displaying the image.

As previously mentioned, additional operations such as clipping (identifying and ignoring fragments that are outside the view frustum) and culling to the viewpoint (ignoring fragments occluded by closer objects) are also performed by the rasterization stage 420. can be executed. With respect to clipping, the geometry processing stage 410 and/or the rasterization stage 420 may be configured to identify and ignore primitives that are outside the view frustum defined by the gaming world perspective.

Pixel processing stage 430 uses the parameters and other data created by the geometry processing stage to generate values, such as the resulting color of the pixel. Specifically, the pixel processing stage 430 at its core performs shading operations on the fragments to determine how the color and brightness of the primitives change with the available lighting. For example, pixel processing stage 430 may determine depth, color, normal, and texture coordinates (e.g., texture details) for each fragment, and may also determine appropriate levels of light, darkness, etc. for the fragment. , and the color may be determined. Specifically, pixel processing stage 430 calculates characteristics of each fragment, including color and other attributes (eg, z-depth for distance from the viewpoint, alpha value for transparency). In addition, pixel processing stage 430 applies lighting effects to the fragments based on the available lighting affecting the corresponding fragments. Additionally, pixel processing stage 430 may apply shadowing effects to each fragment.

The output of pixel processing stage 430 includes processed fragments (eg, texture and shading information) and is sent to the next stage of graphics pipeline 400, output merger stage 440. Output merger stage 440 uses the output of pixel processing stage 430 as well as other data, such as values already in memory, to generate the final color of the pixel. For example, output merger stage 440 may perform optional blending of values between the fragment and/or pixel determined from pixel processing stage 430 and the value already written to the MRT for that pixel. Can be done.

The color value of each pixel in the display may be stored in a frame buffer (not shown). These values are scanned into the corresponding pixels when displaying the corresponding image of the scene. In particular, the display reads color values from the frame buffer and displays the image pixel by pixel, line by line, left to right or right to left, top to bottom or bottom to top, or in any other pattern. illuminate pixels using their pixel values.

Embodiments of the present disclosure use multiple GPUs in conjunction to generate and/or render a single image frame. The difficulty with using multiple GPUs is distributing an equal amount of work to each GPU. Embodiments of the present disclosure can provide each GPU with an equal amount of work (i.e., roughly distributing the work), and through analysis of the spatial distribution of the rendered geometry, the number of pixels (i.e., resolution) and complexity dynamically (i.e., from frame to frame) GPU responsiveness to screen area Adjust civility to optimize both geometry work and pixels. In this way, dynamic distribution of GPU responsivity is performed by screen area, as described further below in connection with FIGS. 5A-5B and 6A-6B.

5A-5B show, purely for illustrative purposes, the rendering of a screen subdivided into regions, each region being assigned to a GPU in a fixed manner. In other words, the area allocation to the GPU does not change from image frame to image frame. In FIG. 5A, the screen is subdivided into four quadrants, each of which is assigned to a different GPU. In FIG. 5B, the screen is subdivided into a larger number of interleaved regions, each of which is assigned a GPU. The discussion of FIGS. 5A-5B below is intended to illustrate the inefficiencies that occur when performing multi-GPU rendering for multiple screen regions to which multiple GPUs are assigned. FIG. 8 illustrates more efficient rendering according to an embodiment of the invention.

Specifically, FIG. 5A is an illustration of a screen 510A that is subdivided into quadrants (eg, four regions) when performing multi-GPU rendering. As shown, screen 510A is subdivided into four quadrants (eg, A, B, C, and D). Each quadrant is assigned to one of four GPUs [GPU-A, GPU-B, GPU-C, and GPU-D] in a one-to-one relationship. That is, GPU responsiveness is distributed by fixed area allocation, with each GPU being fixedly allocated to one or more screen areas. For example, GPU-A is assigned to quadrant A, GPU-B is assigned to quadrant B, GPU-C is assigned to quadrant C, and GPU-D is assigned to quadrant D.

Geometry can be culled. For example, CPU 163 may check a bounding box for each quadrant's frustum and request each GPU to render only objects that overlap the corresponding frustum. As a result, each GPU has the responsibility of rendering only a portion of the geometry. For purposes of illustration, screen 510 shows pieces of geometry, each piece being a corresponding object, and screen 510 shows objects 511-517 (eg, pieces of geometry). It is understood that a piece of geometry may correspond to an entire object or a portion of an object (eg, a primitive, etc.). GPU-A does not render the object since there is no overlapping object in quadrant A. GPU-B renders objects 515 and 516 (because part of object 515 is in quadrant B, the CPU's culling test correctly concludes that GPU-B needs to render it). GPU-C renders objects 511 and 512. GPU-D renders objects 512, 513, 514, 515, and 517.

In FIG. 5A, when the screen 510A is divided into quadrants A through D, the amount of work each GPU must perform is Could be very different. For example, quadrant A has no pieces of geometry, but quadrant D has five pieces of geometry, or at least a portion of at least five pieces of geometry. Therefore, GPU-A assigned to quadrant A will be idle, while GPU-D assigned to quadrant D will be disproportionately busy when rendering objects in the corresponding image.

FIG. 5B illustrates another technique for subdividing a screen into regions such that screen 510B is subdivided into multiple interleaved regions when performing multi-GPU rendering, according to an embodiment of the present disclosure. show. Specifically, the screen 510B is divided into multiple regions rather than subdivided into quadrants when performing multi-GPU rendering of a single image or each of one or more images in a series of images. be redivided. For example, screen 510B may be subdivided into regions corresponding to GPUs. In that case, screen 510B is subdivided into a larger number of regions (eg, more than four quadrants) while using the same amount of GPUs (eg, four) for rendering. The objects (511-517) shown on screen 510A are also shown in the same corresponding positions on screen 510B.

Specifically, four GPUs (eg, GPU-A, GPU-B, GPU-C, and GPU-D) are used to render images for the corresponding application. Each GPU has the responsibility of rendering geometry that overlaps the corresponding region. That is, each GPU is assigned to a corresponding set of regions. For example, GPU-A has a responsibility for each region labeled A in the corresponding set, GPU-B has a responsibility for each region labeled B in the corresponding set, and GPU-B has a responsibility for each region labeled B in the corresponding set. C has a responsibility for each region labeled C in the corresponding set, and GPU-D has a responsibility for each region labeled D in the corresponding set.

Furthermore, the regions are interleaved in a specific pattern. With interleaving (and a larger number) of regions, the amount of work each GPU needs to perform can be much more balanced. For example, the pattern of interleaving of screen 510B includes alternating rows including areas ABAB, etc., and areas CDCD, etc. Other patterns of interleaving regions are also supported by embodiments of the present disclosure. For example, the pattern may include a repeating sequence of regions, an evenly distributed region, an uneven distribution of regions, a repeating row of a sequence of regions, a random sequence of regions, a random row of a sequence of regions, and the like.

The selection of the number of regions is important. For example, if the distribution of regions is too fine (eg, the number of regions is too large to be optimal), each GPU still needs to process most or all of the geometry. For example, it may be difficult for the GPU to check an object's bounding box for all areas for which it has responsiveness. Also, even if the bounding box could be checked in a timely manner, the small region size would result in all objects in the image overlapping by at least one region of each of the GPUs, so each GPU would (e.g., a GPU may have to process an entire object even if only part of the object overlaps at least one region in the set of regions allocated to that GPU. processing).

As a result, the choice of the number of regions is important. Selecting too few or too many regions can lead to inefficiencies when performing GPU processing (e.g., each GPU processes most or all of the geometry), or can lead to imbalances. (e.g., one GPU processes more objects than another GPU). In those cases, even though they have multiple GPUs to render images, due to these inefficiencies they do not have the ability to support a corresponding increase in both the number of pixels and the density of the screen (i.e. Four GPUs cannot write four times as many pixels and process four times as many vertices or primitives). Accordingly, embodiments of the present disclosure may generate information (via "geometric analysis") to indicate which object or objects are present in each of the screen regions. Geometry analysis can be performed during or before rendering, and the resulting information is used to dynamically transfer screen areas to the GPU for further rendering of the corresponding image frame, as described further below. can be assigned to That is, the screen area is not fixed to the corresponding GPU, but can be dynamically allocated to the GPU for rendering the corresponding image frame.

6A-6B illustrate performing geometry analysis to dynamically allocate screen space to a GPU for geometry rendering of an entire object and/or a portion of an object in an image frame in various embodiments of the present disclosure. shows the advantage of dividing objects within an image frame into smaller parts. Specifically, multi-GPU rendering of objects is performed on a single image frame by performing geometric analysis on the objects within the screen. Information is generated for "pieces of geometry," which may be entire objects or portions of objects. For example, the piece of geometry may be object 610 or a portion of object 610. Specifically, a GPU is assigned to a piece of geometry (eg, an entire object and/or a portion of an object) to determine a relationship between the geometry and each of a plurality of screen regions. That is, the cooperating GPUs determine information that provides a relationship between each piece of geometry and each screen area. Analysis is performed on the information and screen area is dynamically allocated to the GPU for subsequent rendering of the corresponding image frame. Geometry analysis and subsequent rendering, e.g. during geometry rendering, if the object is associated with a single GPU for geometry rendering (e.g. dynamically allocating all screen area containing the object to a single GPU) ), other GPUs can skip the entire object when rendering an image frame, according to one embodiment of the present disclosure, which results in efficient processing of the geometry. Moreover, dividing an object into smaller parts may further increase efficiency in performing geometry analysis and/or rendering of geometry in a corresponding image frame.

FIG. 6A illustrates the overall object geometry analysis (i.e., the correspondence the amount of geometry used or generated by the draw call. If the entire object is rendered (that is, the geometry used or generated by the draw call is not split into parts), then each GPU that has the rendering responsiveness of the screen area that overlaps with the object will render the entire object. There is a need. Specifically, during geometry analysis, object 610 may be determined to overlap region 620A, and object 610 may also be determined to overlap region 620B. That is, portion 610A of object 610 overlaps region 620A, and portion 610B of object 610 overlaps region 620B. Subsequently, GPU-A is assigned the responsibility to render objects in screen region 620A, and GPU-B is assigned the responsibility to render objects in screen region 620B. Since the object is rendered as a whole, GPU-A is given the task of completely rendering object 610, ie, processing all primitives within the object, including primitives spanning both regions 620A and 620B. In this particular example, GPU-B is also given the task of rendering the entire object 610. That is, duplication of work by GPU-A and GPU-B may occur when performing rendering of the geometry of objects in corresponding image frames. Furthermore, if the number of objects (that is, draw calls) distributed among GPUs is small, it may be difficult to balance the geometry analysis itself.

FIG. 6B illustrates geometric analysis of a portion of an object to determine the relationship of the portion of the object to a screen area when multiple GPUs cooperate to render a corresponding image frame, according to an embodiment of the present disclosure. shows. As shown, the geometry used or generated by the draw calls is subdivided to create these parts of the object. For example, object 610 may be divided into pieces such that the geometry used or generated by the draw call is subdivided into smaller pieces of geometry. In that case, information is generated about the smaller pieces of geometry during geometry analysis to determine the relationship (eg, overlap) between the smaller pieces of geometry and each screen area. Using this information, geometry analysis is performed to dynamically allocate rendering responsiveness for each screen area between GPUs and render smaller pieces of the geometry of the corresponding image frame. Each GPU renders only a smaller piece of geometry that overlaps the responsive screen area when performing rendering of the corresponding image frame. As such, each GPU is assigned a set of screen areas for rendering the corresponding image frame's piece of geometry. In other words, GPU responsiveness is uniquely assigned to each image frame. In this way, there may be less duplication of work between GPUs when performing geometry analysis and/or rendering of the geometry of objects in the corresponding image frame, thereby increasing efficiency when rendering the corresponding image frame. improves.

In one embodiment, the draw calls in the command buffer remain the same, but during rendering, the GPU breaks the geometry into pieces. The piece of geometry may be approximately the same size as the position cache and/or parameter cache is allocated. Each GPU renders or skips these pieces so that it only renders those pieces that overlap with its assigned screen area.

For example, object 610 is divided into parts such that the pieces of geometry used for region testing correspond to these smaller parts of object 610. As shown, object 610 is divided into pieces of geometry "a", "b", "c", "d", "e", and "f". After geometry analysis, GPU-A uses screen area to render pieces of geometry "a", "b", "c", "d", and "e" when rendering the corresponding image frame. 620A. That is, GPU-A can skip rendering the piece of geometry "f". Also, after geometry analysis, GPU-B may be assigned to screen area 620B to render pieces of geometry "d", "e", and "f" when rendering the corresponding image frame. . That is, GPU-B can skip rendering pieces of geometry "a", "b", and "c". As shown, instead of rendering the object 610 completely, only pieces of geometry "d" and "e" are rendered by GPU-A and GPU-B, respectively, so that GPU-A and GPU-B There is little duplication of work between the two.

Multi-GPU Rendering of Geometry by Performing Geometry Analysis During Rendering Along with the detailed description of cloud gaming network 190 (e.g., within game server 160) and GPU resources 365 of FIGS. 1-3, flowchart 700 of FIG. 3 illustrates a method for graphics processing when implementing multi-GPU rendering of the geometry of image frames generated by an application by performing geometry analysis during rendering, according to an embodiment of the disclosure; Specifically, a large number of GPUs work together to generate image frames. Responsibility for a particular phase of rendering is divided among multiple GPUs based on the screen area of each image frame. During rendering of geometry, the GPU generates information about the geometry and its relationship to the screen area. This information is used to allocate GPUs to screen areas, allowing for more efficient rendering. In this manner, multiple GPU resources are used to efficiently perform rendering of objects in image frames during application execution. As mentioned above, various architectures are available for rendering, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system such as a personal computer or game console that includes a high-end graphics card with multiple GPUs. Performing multi-GPU rendering of geometry for an application through area testing during the process can involve multiple GPUs working together to render a single image.

At 710, the method includes rendering graphics using multiple GPUs, and in certain phases, rendering responsiveness is dynamically divided among the multiple GPUs based on screen area. In particular, multi-GPU processing is performed when rendering a single image frame and/or each of one or more image frames of a series of image frames for real-time applications, where each image frame is composed of multiple pieces of geometry. including. In a particular phase, GPU rendering responsivity is dynamically allocated among multiple screen areas of each image frame such that each GPU renders a piece of geometry in its assigned screen area. That is, each GPU has a corresponding division or division of responsiveness (eg, corresponding screen area).

At 720, the method includes using multiple GPUs in conjunction to render an image frame that includes corresponding multiple pieces of geometry. In one embodiment, during rendering, a pre-pass phase of rendering is performed. In one embodiment, this pre-pass phase of rendering is a Z pre-pass in which multiple pieces of geometry are rendered.

To perform the pre-pass phase of rendering, at 720, the method includes partitioning the responsivity of processing multiple pieces of geometry of an image frame during the Z pre-pass phase of rendering among multiple GPUs. That is, each of the plurality of geometry pieces is assigned a corresponding GPU to perform the Z pre-pass, and/or each GPU is assigned a set of screen areas to which it has responsiveness. Thus, pieces of geometry are rendered in a Z pre-pass phase on multiple GPUs to generate one or more Z buffers. Specifically, each GPU renders a corresponding piece of geometry in a Z pre-pass phase to generate a corresponding Z buffer. For example, for a corresponding piece of geometry, a Z-buffer may include a corresponding z value (eg, a depth value) that measures the distance of the piece of geometry from a pixel on the projection plane. Hidden geometry or objects can be removed from the Z-buffer as is known in the art.

In one embodiment, each GPU may have a dedicated Z-buffer. For example, a first GPU renders a first piece of geometry in a Z pre-pass phase to generate a first Z buffer. The other GPU renders the corresponding piece of geometry in the Z pre-pass phase to generate the corresponding Z buffer. In one embodiment, each GPU sends its data in its corresponding Z-buffer to each of the plurality of GPUs such that the corresponding Z-buffer is updated approximately for use in rendering the geometry of the image frame. Make it similar. That is, each GPU is configured to merge data received from all Z-buffers such that each of the GPU's corresponding Z-buffers is updated as well.

At 730, the method includes generating information regarding the plurality of geometric pieces of the image frame and their relationships to the plurality of screen regions. In one implementation, the information is generated during the pre-pass phase of rendering. For example, information may be generated on the first GPU while rendering a piece of geometry, and the information may indicate which screen area the piece of geometry overlaps. As mentioned above, a piece of geometry can be an entire object (i.e., geometry used or produced by an individual draw call) or a portion of an object (e.g., an individual primitive, a group of primitives, etc.). Additionally, the information may include the presence of the piece of geometry within the corresponding screen area. The information may include a conservative estimate of the presence of a piece of geometry within the corresponding screen area. The information may include the pixel area or approximate pixel area (eg, coverage) that the piece of geometry covers in the screen area. The information may include the number of pixels written to the screen area. The information may include the number of pixels written to the Z buffer per piece of geometry per screen area during the Z pre-pass phase of rendering.

At 740, the method includes using this information in a subsequent allocation of screen area to multiple GPUs. Specifically, each GPU is informedly assigned to a corresponding screen area for rendering an image frame during a subsequent phase of rendering, which may be a geometry pass. In this way, the allocation of screen area to the GPU can change from image frame to image frame, ie, can be dynamic.

FIG. 8 shows a diagram for geometry rendering (i.e., rendering a piece of geometry into an MRT) based on an analysis of the geometry of the current image frame performed during rendering of the current image frame, according to an embodiment of the present disclosure. FIG. 8 is a diagram of a screen 800 illustrating dynamic allocation of screen space to GPUs. As shown, screen 800 can be subdivided into regions, each region being approximately equal in size for purposes of illustration. In other embodiments, each of the regions can be of various sizes and shapes. For example, region 810 represents an equal subdivision of screen 800.

The objects and their positions shown on screen 800 are the same as the objects and their positions shown on screen 510A of FIG. 5A and screen 510B of FIG. 5B. For example, objects 511-517 are shown on screen 800. FIG. 5A shows the division of screen 510A into quadrants that are fixedly assigned to GPUs for geometry rendering. FIG. 5B shows the division of screen 510B into regions that are assigned in a fixed manner to the GPU for geometry rendering. FIG. 8 shows the dynamic allocation of screen space to the GPU for the current image frame containing objects 511-517. Allocation is performed for each image frame. That is, in the next image frame, objects 511-517 may be in different positions, and therefore the screen area allocation of the next image frame may be different than the allocation of the current image frame. For example, GPU-A is assigned to set of screen areas 832 and renders objects 511 and 512. GPU-B is also assigned to a set of screen regions 834 and renders objects 513, 515, and 517. GPU-C is assigned to a set of screen regions 836 and renders objects 512, 513, 514, and 517. GPU-D is then assigned to a set of screen areas 838 to render objects 515 and 516. If the object is further divided into parts, there may be less overlap in rendering because smaller parts have less overlap between GPU regions. That is, the draw call in the corresponding command buffer remains the same, but during rendering the GPU draws the geometry into a piece (e.g., an object ) and render or skip those pieces depending on whether they overlap the screen area allocated to that GPU for geometry rendering.

In one embodiment, the allocation of screen space to GPUs may be handled such that approximately equal amounts of pixel work are performed by each GPU when rendering geometry. Pixel shaders associated with objects may differ in complexity, so the amount of screen area covered by corresponding objects is not necessarily equal. For example, GPU-D has a rendering responsiveness of 4 regions and GPU-A has a rendering responsiveness of 6 regions, but their corresponding pixels and/or rendering efforts are approximately equal. obtain. This means that each object has a different rendering cost, and each object can have a higher or lower cost per pixel, primitive, or vertex. This cost per pixel, primitive, vertex, etc. can be made available to each GPU and used to generate information, or can be included as information. Alternatively, costs can be used when allocating screen space.

In one embodiment, crosshatch region 830 does not include geometry and may be assigned to any one of the GPUs. In another embodiment, crosshatch region 830 is not assigned to any GPU. In either case, no geometry rendering is performed on region 830.

In another embodiment, all regions associated with an object are assigned to a single GPU. In this way, all other GPUs can completely skip the object when performing geometry rendering.

9A-9C are diagrams providing a more detailed explanation of the rendering of an image frame showing four objects, which includes a Z pre-pass phase and a geometry phase of rendering. As mentioned above, the Z pre-pass phase is performed to generate information used to dynamically allocate screen area to the GPU for geometry rendering of image frames in accordance with embodiments of the present disclosure. For purposes of illustration, FIGS. 9A-9C illustrate the use of multiple GPUs to render each of a series of image frames. The selection of four GPUs for the examples shown in FIGS. 9A-9C was made purely to illustrate multi-GPU rendering; in various embodiments, any number of GPUs for multi-GPU rendering may be used. It is understood that you can use

Specifically, FIG. 9A shows a screen 900A showing four objects contained within an image frame. For example, an image frame includes object 0, object 1, object 2, and object 3. As shown, screen 900A is divided into multiple regions. For example, screen 900A may be divided into more than four regions, each of which is assigned a corresponding GPU for rendering the current image frame.

In one embodiment, a single command buffer is used by multiple GPUs to render corresponding image frames. A common rendering command buffer may contain draw calls and state settings for each object to perform the Z prepass phase of rendering. A sink (eg, synchronization) operation can be included in the command buffer so that all GPUs start the geometry pass phase of rendering at the same time. The command buffer may contain draw calls and state sets for each object to perform the geometry pass phase of rendering.

In one embodiment, the common rendering command buffer supports the ability for commands to be executed by one GPU but not another GPU. That is, the common rendering command buffer format allows commands to be executed by one or a subset of multiple GPUs. For example, as mentioned above, a flag in a drawing command or predicate in a rendering command buffer allows a single GPU to execute one or more commands in the corresponding command buffer without interference from other GPUs. Can be executed.

FIG. 9B illustrates one or more Z-buffers and information associated with each of the pieces of geometry of a particular image frame and the screen regions and/or sub-regions of the rendered screen, according to an embodiment of the present disclosure. 3 shows the Z pre-pass phase of rendering performed to generate. In the Z pre-pass phase of rendering in FIG. 9B, one strategy is shown in which multiple GPUs can work together to generate one or more Z buffers for a frame of rendering. Other strategies can be implemented to generate one or more Z-buffers.

As shown, each GPU in a multi-GPU architecture is allocated a portion of the geometry. For purposes of illustration, GPU-A is assigned to object 0, GPU-B is assigned to object 1, GPU-C is assigned to object 2, and GPU-D is assigned to object 3. Each GPU renders the corresponding object in the Z prepass phase and renders the corresponding object into its own copy of the Z buffer. For example, in the Z pre-pass phase, GPU-A renders object 0 into its Z buffer. Screen 921 shows the pixel coverage of object 0 as determined by GPU-A and stored in its corresponding Z-buffer. GPU-B also renders Object 1 into its Z-buffer, as screen 922 shows the pixel coverage of Object 1 as determined by GPU-B and stored in the corresponding Z-buffer. In addition, GPU-C renders Object 2 into its Z-buffer as screen 923 shows the pixel coverage of Object 2 determined by GPU-C and stored in the corresponding Z-buffer. Additionally, GPU-D renders object 3 into its Z-buffer as screen 924 shows the pixel coverage of object 3 determined by GPU-D and stored in the corresponding Z-buffer.

The four Z-buffer copies corresponding to the GPUs are then merged. That is, each GPU has a corresponding copy of the Z-buffer in its own RAM (Random Access Memory). In one embodiment, the strategy for building one or more Z-buffers includes having each GPU send its completed Z-buffer to other GPUs. Thus, each of the Z-buffers must be similar in size and format. Specifically, data for each of the Z-buffers is sent to all GPUs to merge and update each of the Z-buffers, which is illustrated by screen 925 showing the pixel coverage of each of the four objects 1-4. and stored in each of the GPU's updated Z-buffers. The object is blank in FIG. 9B, indicating that only Z has been written and no other values (eg, color) have been calculated for each of the pixels of the screen.

In another embodiment, merge time is reduced. Instead of waiting for each Z-buffer to be fully completed by the corresponding GPU before the data is sent to the other GPU, as each GPU writes the corresponding piece of geometry to its Z-buffer, sends the updated screen area Z-buffer data to other GPUs. That is, when a first GPU renders geometry to a corresponding Z-buffer or other render target, the first GPU renders data from the Z-buffer or other render target data, including updated screen area, to the other GPU. Send. By not waiting for each Z-buffer of the corresponding GPU to be completely written before transmitting, some of the time required to merge Z-buffers is removed, thereby reducing merge time.

In another embodiment, another strategy for building Z-buffers includes sharing a common Z-buffer or a common render target among multiple GPUs. For example, the hardware used to perform Z-buffering may be configured such that there is a common Z-buffer or a common render target that is shared and updated by each GPU. That is, each GPU updates a common Z-buffer while rendering one or more corresponding pieces of geometry during the Z-prepass phase of rendering. In the example four-GPU architecture, a first GPU can each update a common Z-buffer or common render target shared by multiple GPUs, thereby adding geometry to a corresponding Z-buffer or other render target. Render. Using a common Z-buffer or a common render target eliminates the need for a merge step. In one embodiment, screen space is allocated to the GPU, simplifying the need for arbitration when accessing a common Z-buffer.

As mentioned above, information is generated during Z-buffer rendering. In one embodiment, a scan converter running as part of rasterization stage 420 of FIG. 4 generates the information. For example, the scan converter can calculate the area of overlap between each piece of geometry and the screen area. In various embodiments, overlap may be measured in pixels, such as between each primitive of a piece of geometry and each screen area. Additionally, the scan converter can sum the areas of overlap as measured for each region to create a total area of overlap for each piece of geometry (eg, pixel by pixel).

This information can be used to allocate screen space to the GPU before the geometry pass begins. That is, one or more of the multiple GPUs can be assigned to the screen area. In one embodiment, the allocation is made such that each GPU's rendering responsiveness (eg, rendering geometry) is approximately equal. In this way, information generated in one phase of rendering (the Z pre-pass phase) is used in another phase of rendering, such as allocating screen space to the GPU for the geometry pass phase of rendering.

As mentioned above, objects may have different rendering costs than other objects. That is, one object may have a higher or lower cost per pixel, or primitive, or vertex than another object. In some embodiments, the cost per pixel/primitive/vertex is available to the GPU, used to generate the information, and/or included in the information. In another embodiment, the cost per pixel/primitive/vertex is used when allocating screen area to the GPU, so that the information generated is Take into account approximate rendering costs. That is, costs are determined for rendering pieces of geometry of an image frame during the geometry phase of rendering. Cost is considered when allocating screen space to the GPU for geometry rendering. For example, in the subsequent allocation of screen space to multiple GPUs, GPUs can be allocated to screen space in such a way that the cost of rendering is divided (equally or unevenly) between the GPUs as desired. Takes into account the approximate rendering cost of a piece of geometry per pixel, primitive, or vertex.

FIG. 9C illustrates the geometry pass phase of rendering performed to render a piece of geometry for a particular image frame, according to one embodiment of the present disclosure. In the geometry pass phase, each GPU renders objects of a particular image frame to screen areas for which it has responsiveness (e.g., based on the GPU's previous assignment to screen areas). Specifically, each GPU renders all objects, but we know (based on the information) that there is no overlap between these objects and the screen area allocated to the GPU for geometry rendering. Excludes objects that are present. Thus, if a piece of geometry does not overlap the screen area allocated to a particular GPU, that GPU can skip rendering that piece of geometry.

As shown, each GPU in the multi-GPU architecture is assigned or allocated to a portion of the screen. For illustration purposes, GPU-A is assigned to one region labeled 931A and renders object 0 (as introduced in Figure 9A) (where other values such as color data are written). (dimmed to show that Screen 931 shows render target data (eg, pixels) for object 0 after geometry rendering. GPU-B is also assigned to two regions labeled 932A to render portions of object 1 and object 2 (the respective portions of those objects are dimmed). Screen 932 shows render target data (eg, pixels) for respective portions of objects 1 and 2 after geometry rendering. Furthermore, GPU-C is assigned to two regions labeled 933A to render portions of object 2 (respective portions that are dimmed). Screen 933 shows render target data (eg, pixels) for each portion of object 2 after geometry rendering. GPU-D is also assigned to three areas labeled 934A to render object 3 (here dimmed to represent other values being written, such as color data) . Screen 934 shows render target data (eg, pixels) for object 3 after geometry rendering.

After rendering the geometry, it may be necessary to merge the render target data produced by each GPU. For example, merging of the geometry data generated during the geometry pass phase of each GPU's rendering is performed, as indicated by screen 935 containing render target data (eg, pixels) for all four objects 0-3.

In one embodiment, the allocation of screen area to the GPU changes from frame to frame. That is, each GPU may have a different screen area responsiveness when comparing allocations of two consecutive image frames. In another embodiment, the allocation of screen area to the GPU may also vary throughout the various phases used in rendering a single frame. That is, the screen area allocation may change dynamically during a rendering phase, such as a geometry analysis phase (eg, Z pre-pass) or a geometry pass phase.

For example, when a geometry phase assignment is made, this assignment may therefore differ from the existing assignment. In other words, GPU-A may now have responsiveness in a screen area where GPU-B previously had responsiveness. This may require the transfer of Z-buffers or other render target data from GPU-B's memory to GPU A's memory. As an example, the information may include a first object in a command buffer to write to a screen area. This information can be used to schedule DMA (direct memory access) transfers, such as transferring screen area Z-buffer data or other render target data from one GPU to another. Following the example above, data from GPU-B's memory (eg, Z-buffer or render target data) may be transferred to GPU-A's memory. In some cases, the later the first screen use occurs when rendering an image frame, the longer the DMA transfer takes.

In another embodiment, once all updates of the Z-buffer or other render target data between GPUs are complete, the information may include the last object in the command buffer to write to the screen area. That information can be used to schedule DMA transfers from the rendering GPU (performed during the Z pre-pass phase of rendering) to other GPUs. That is, this information is used to schedule the transfer of screen area Z-buffers or other render target data from one GPU to another (eg, a rendering GPU).

In yet another embodiment, once all updates of Z-buffers or other render target data between GPUs are complete, the updated data may be broadcast to the GPUs. In that case, the updated data is available if any of the GPUs need it. In another embodiment, data is sent to a particular GPU, such as in anticipation that the receiving GPU will have screen area responsiveness in subsequent phases of rendering.

FIG. 10 illustrates rendering of an image frame using dynamic allocation of screen space to the GPU based on the entire object or portions of the object for geometry rendering, according to an embodiment of the present disclosure; is based on an analysis of the geometry of the current image frame performed during the Z pre-pass phase of rendering performed while rendering the image frame. Specifically, rendering timing diagram 1000A illustrates the rendering of an image frame based on the entire object (ie, the geometry used or generated by the individual draw calls). In contrast, rendering timing diagram 1000B depicts rendering of an image frame based on portions of an object. Advantages exhibited when rendering image frames based on portions of objects include better balancing of rendering performance between GPUs, thus reducing image frame rendering time.

Specifically, rendering timing diagram 1000A shows the rendering of each of four objects 0-3 by four GPUs (eg, GPU-A, GPU-B, GPU-C, and GPU-D), and shows the rendering timing diagram 1000A. Responsibility is distributed across GPUs at object granularity. Objects 0-3 were previously introduced in FIGS. 9A-9C. Various phases of rendering are shown in relation to timeline 1090. Vertical line 1001A indicates the start of rendering of the Z pre-pass. Rendering timing diagram 1000A includes a Z pre-pass phase 1010A of rendering and also shows a phase 1020A that illustrates merging of Z buffer data between GPUs. GPU idle time is indicated using the hashed out area, and the merge phase 1020A may occur during this idle time. A sync point 1030A is provided such that each GPU starts its respective geometry pass rendering phase simultaneously. The rendering timing diagram 1000A also includes a rendering geometry pass phase 1040A for rendering the geometry of the image frame, as described above. A sync point 1050A is provided such that each GPU simultaneously begins rendering the next image frame. Sync point 1050A may also indicate the end of rendering of the corresponding image frame. The total time for rendering an image frame when rendering the entire object is indicated by period 1070. The processing of information to determine each GPU's screen area responsiveness is not shown in the diagram, but may be assumed to be completed before the beginning of geometry pass 1030A.

As shown, the hashed area of rendering timing diagram 1000A during geometry pass phase 1040A indicates GPU idle time. For example, GPU-A will be idle for approximately the same amount of time that GPU-A spends rendering. On the other hand, GPU-B has almost no time to be in an idle state, and GPU-C has no time to be in an idle state.

In contrast, rendering timing diagram 1000B shows the rendering of each of four objects 0-3 by four GPUs (e.g., GPU-A, GPU-B, GPU-C, and GPU-D) and shows the rendering response. Civility is distributed across GPUs at the granularity of parts of an object, such as the pieces of geometry shown in Figure 6B, rather than the entire object. For example, information (eg, overlap with screen area) is generated for a piece of geometry (eg, a portion of an object) rather than the entire object. In this way, the geometry of the image frame used or produced by the draw call (e.g. the entire object) is subdivided into smaller pieces of geometry, and the information produced is about these pieces of geometry. . In some cases, there are limits to the extent to which pieces of geometry can be subdivided.

Various phases of rendering are shown in relation to timeline 1090. Vertical line 1001B indicates the start of rendering of the Z pre-pass. Rendering timing diagram 1000B includes a Z pre-pass phase of rendering 1010B and also shows a hashed out period 1020B during which merging of Z buffer data between GPUs is performed. GPU idle time 1020B in rendering timing diagram 1000B is shorter than idle time 1020A in rendering timing diagram 1000A. As shown, each GPU spends approximately the same amount of time processing the Z pre-pass phase, with little or no idle time. A sync point 1030B is provided so that each GPU starts its respective geometry pass rendering phase at the same time. Rendering timing diagram 1000B also includes a rendering geometry pass phase 1040B for rendering the geometry of the image frame, as described above. A sync point 1050B is provided such that each GPU simultaneously begins rendering the next image frame. Sync point 1050B may also indicate the end of rendering of the corresponding image frame. As shown, each GPU spends approximately the same amount of time processing the geometry pass phase, with little or no idle time. In other words, Z pre-pass rendering and geometry rendering are almost balanced between GPUs. Also, the total time for rendering an image frame when rendering by portion of the entire object is indicated by period 1075. The processing of information to determine each GPU's screen area responsiveness is not shown in the diagram, but may be assumed to be completed before the beginning of geometry pass 1030B.

As shown, rendering timing diagram 1000B illustrates reduced rendering time when rendering responsiveness is distributed across GPUs at the granularity of portions of objects rather than entire objects. For example, time savings 1077 are shown when rendering an image frame at the granularity of a portion of an object.

In addition, according to one embodiment of the present disclosure, this information allows the requirements and/or dependencies of the rendering phase to be relaxed, thereby allowing the rendering phase to be completed while another GPU is still processing the current phase of rendering. , resulting in the GPU proceeding to subsequent phases of rendering. For example, one requirement that the Z pre-pass phase 1020A or 1020B must be completed for all GPUs before any GPU begins the geometry phase 1040A or 1040B may be relaxed. As shown, the rendering timing diagram 1000A includes all GPU sync points 1020A before starting the geometry phase 1040A. However, this information may indicate that (for example) GPU A can begin rendering its assigned region before other GPUs complete the Z pre-pass phase of the corresponding rendering. This may reduce the overall image frame rendering time.

FIG. 11 illustrates a diagram for performing a Z pre-pass phase of rendering to generate information used for dynamic allocation of screen area to GPUs for geometry rendering of image frames, according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating interleaving GPU assignments to pieces of geometry of an image frame. That is, FIG. 11 shows the distribution of rendering responsiveness among multiple GPUs for Z pre-pass. As mentioned above, each GPU is assigned to a corresponding portion of the image frame's geometry, which portion may be further divided into objects, portions of objects, geometries, pieces of geometry, and so on.

As shown in Figure 11, objects 0, 1, and 2 represent the geometry used or generated by the individual draw calls. In one embodiment, the GPU divides each object into smaller pieces of geometry, such as pieces of approximate size to which position caches and/or parameter caches are allocated, as described above. Purely for illustration purposes, object 0 is divided into pieces "a," "b," "c," "d," "e," and "f," like object 610 in FIG. 6B. Further, object 1 is divided into pieces "g", "h", and "i". Furthermore, object 2 is divided into pieces "j", "k", "l", "m", "n", and "o". Pieces can be ordered (eg, a through o) to distribute the responsivity of performing the Z pre-pass phase of rendering.

Distribution 1110 (eg, lines ABCDABCDABCD...) indicates an even distribution of the responsibilities of performing geometry tests across multiple GPUs. Specifically, let one GPU take the first quarter of the geometry (e.g. in a block, GPU-A takes "a", "b", "c" out of about 16 total pieces) and "d" for testing), have the second GPU take the second quarter, etc., the allocation to the GPUs is interleaved. That is, successive pieces of geometry are assigned to different GPUs to perform the Z pre-pass phase of rendering. For example, piece "a" is assigned to GPU-A, piece "b" is assigned to GPU-B, piece "c" is assigned to GPU-C, piece "d" is assigned to GPU-D, Piece "e" is assigned to GPU-A, piece "f" is assigned to GPU-B, and piece "g" is assigned to GPU-C. As a result, there is no need to know the total number of pieces of geometry to process (such as when GPU-A has acquired the first quarter of the pieces of geometry), and processing of the Z pre-pass phase of rendering is done by the GPU. (eg, GPU-A, GPU-B, GPU-C, and GPU-D).

In other embodiments, information generated during rendering of one frame (e.g., a previous image frame) may be used to allocate GPUs to screen area for subsequent frames (e.g., the current image frame). For example, the hardware may be configured to generate information during a geometry pass phase of rendering a previous image frame, such as GPU usage during a geometry pass phase of rendering a previous image frame. Specifically, this information may include the actual number of pixels shaded per piece of geometry per screen area. This information can be used in subsequent frames (eg, rendering of the current image frame) when allocating GPUs to the screen area of the rendering geometry pass. That is, the allocation of screen space to the GPU to perform the geometry pass phase of rendering the current image frame uses the information generated from the previous image frame and the current image frame (if any) as described above. Both of the information generated in the Z pre-pass phase of is taken into account. Therefore, screen area is allocated to the GPU based on information from previous image frames (e.g. GPU usage) and information generated during the Z pre-pass phase of rendering of the current image frame (if any). Assigned.

This information from the previous frame can either just use the overlap area mentioned above (e.g. to generate information for the current image frame), or it can be stored in the Z buffer for each piece of geometry per screen area during the Z pre-pass. Accuracy can be increased over just using the number of written pixels. For example, the number of pixels written to an object's Z-buffer may not correspond to the number of pixels that need to be shaded in a geometry pass due to occlusion of the object by other objects. Using both information from the previous image frame (e.g. GPU usage) and information generated during the Z pre-pass phase of rendering the current image frame, during the geometry pass phase of rendering the current image frame rendering can be more efficient.

The information may also include a number of vertices for each screen region, giving the number of vertices used by a corresponding portion of geometry (eg, a piece of geometry) that overlaps the corresponding screen region. Therefore, when rendering the corresponding piece of geometry later, the rendering GPU can use the vertex count to allocate space for the position and parameter caches. For example, in one embodiment, vertices that are not needed do not have space allocated to them, which can increase rendering efficiency.

In yet another embodiment, there may be processing overhead (either software or hardware) associated with generating information during the Z pre-pass phase of rendering. In that case, it may be beneficial to skip generating information about particular pieces of geometry. In other words, information may be generated for a specific object and not for other objects. For example, no information may be generated for pieces of geometry (eg, objects or portions of objects) that have large primitives and are likely to overlap many screen areas. Objects with large primitives may be skyboxes or large terrain pieces containing, for example, large triangles. In that case, each GPU used for multi-GPU rendering of an image frame would likely need to render those pieces of geometry, and no information would be needed to indicate that. In this way, information may or may not be generated depending on the properties of the corresponding piece of geometry.

Systems and Methods for Efficient Multi-GPU Rendering of Geometry by Performing Pre-Rendering Geometry Analysis FIG. Flowchart 1200A illustrates a method of graphics processing, including multi-GPU rendering of geometry for an application by performing geometry analysis prior to rendering, according to one embodiment of the present disclosure. That is, instead of generating the information during rendering as described in connection with FIGS. 7, 9, and 10, the information is generated prior to rendering, such as during a pre-pass (i.e., a pass that does not write to the Z-buffer or MRT). generated. One or more of the various features and advantages of the various embodiments described with respect to the generation of information during rendering (e.g., the Z prepass phase of rendering) include the generation of information before rendering (e.g., performing geometry analysis). It is understood that these are equally applicable to PrePass) and may not be repeated here to minimize duplication of explanation. As mentioned above, various architectures are available for rendering, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system such as a personal computer or game console that includes a high-end graphics card with multiple GPUs. Performing multi-GPU rendering of geometry for an application through area testing during the process can involve multiple GPUs working together to render a single image.

Specifically, GPU rendering responsivity is dynamically allocated among multiple screen areas of each image frame such that each GPU renders objects in its assigned screen area. The analysis is performed (e.g., in a primitive or computational shader) before geometry rendering to determine the spatial distribution of the geometry within the image frame and dynamically adjust the GPU's responsiveness to the screen area to Render objects within a frame.

At 1210, the method includes rendering graphics for the application using multiple graphics processing units (GPUs). Specifically, a large number of GPUs work together to generate image frames. In particular, multi-GPU processing is performed when rendering a single image frame and/or each of one or more image frames of a series of image frames for real-time applications. Rendering responsiveness is divided among multiple GPUs based on the screen area of each image frame, as described further below.

At 1220, the method includes partitioning a responsibility for processing multiple geometry pieces of an image frame during an analysis pre-pass between multiple GPUs, each of the multiple geometry pieces being assigned to a corresponding GPU. It will be done. The analysis pre-pass is performed before the image frame rendering phase.

In the analysis pre-pass, objects are distributed among multiple GPUs. For example, in a multi-GPU architecture with four GPUs, each GPU processes approximately one-fourth of the object during the analysis pre-pass. As mentioned above, in one embodiment there may be advantages to subdividing an object into smaller pieces of geometry. Additionally, in other embodiments, objects are dynamically assigned to GPUs on a per image frame basis. Dynamically allocating pieces of geometry to the GPU for analysis pre-pass may improve processing efficiency.

Since the analysis pre-pass is performed before the rendering phase, the processing is typically not performed in hardware. That is, the analysis pre-pass can be performed in software, such as using shaders in various embodiments. For example, a primitive shader may be used during the analysis pre-pass such that there is no corresponding pixel shader. Additionally, Z-buffers and/or other render targets are not written during the analysis pre-pass. In other embodiments, compute shaders are used.

At 1230, the method includes determining in analysis a pre-pass phase overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. As mentioned above, a piece of geometry can be an object or a portion of an object (eg, an individual primitive, a group of primitives, etc.). In one embodiment, the generated information includes an accurate representation of the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. In one embodiment, the information includes an estimate of the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions.

At 1240, the method includes generating information about the plurality of geometry pieces and their relationships to the plurality of screen regions based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions. include. The information may simply be that there is an overlap. The information may include the pixel area or approximate pixel area that the pieces of geometry overlap or cover in the screen area. The information may include the number of pixels written to the screen area. The information may include the number of vertices or primitives that overlap the screen area, or an approximate value thereof.

At 1250, the method includes dynamically allocating screen areas to GPUs based on the information to render the pieces of geometry during a geometry pass phase of rendering. That is, the information can be used for subsequent allocation of screen space to multiple GPUs. For example, each GPU is assigned to a corresponding screen area based on the information. In this way, each GPU has a corresponding division of responsiveness (eg, corresponding screen area) for rendering of image frames. Therefore, the allocation of screen area to the GPU may differ from image frame to image frame.

Further, during the geometry pass phase, the method renders the pieces of geometry on each of the plurality of GPUs based on the GPU-to-screen area allocation determined from allocating the plurality of screen areas to the plurality of GPUs. including doing.

FIG. 12B is a rendering timing diagram 1200B illustrating an analysis pre-pass performed before rendering an image frame (eg, during the geometry pass phase of rendering), according to one embodiment of the present disclosure. The analysis pre-pass is dedicated to the analysis of relationships between pieces of geometry and screen areas. The analysis prepass generates information that is used to dynamically allocate screen space to the GPU for geometric rendering of image frames. Specifically, rendering timing diagram 1200B illustrates using multiple GPUs to coordinately render image frames. Rendering responsiveness is divided among multiple GPUs based on screen area. As mentioned above, before rendering the geometry of an image frame, the GPU generates information about the geometry and its relationship to the screen area. This information is used to allocate GPUs to screen areas, allowing for more efficient rendering. For example, prior to rendering, a first GPU generates information about the relationship between a piece of geometry and its screen area, and this information is passed to one or more "rendering GPUs" that render that piece of geometry. Used when allocating space.

Specifically, rendering timing diagram 1200B illustrates the rendering of one or more objects by four GPUs (e.g., GPU-A, GPU-B, GPU-C, and GPU-D) with reference to timeline 1290. Show rendering. As mentioned above, the use of four GPUs is for illustrative purposes only, as one or more GPUs can be included in a multi-GPU architecture. Vertical line 1201 indicates the beginning of a series of rendering phases for an image frame. Vertical line 1201 also indicates the start of analysis pre-pass 1210. In the analysis pre-pass, objects are distributed among multiple GPUs. With four GPUs, each GPU processes approximately one-fourth of the object. A sync point 1230A is provided such that each GPU starts its respective geometry pass rendering phase 1220 at the same time. That is, in one embodiment, sink operation 1230a ensures simultaneous initiation of geometry passes by all GPUs. In another embodiment, as previously described, the sink operation 1230a is not used and the geometry pass phase of rendering is performed for any GPU that finishes the analysis pre-pass, while all other GPUs complete the corresponding analysis pre-pass. can be started without waiting for it to finish.

Sync point 1230b marks the end of the geometry pass phase of rendering the current image frame and also allows each GPU to simultaneously continue subsequent phases of rendering the current frame or begin rendering the next image frame simultaneously. Provided as possible.

In one embodiment, a single command buffer is used by multiple GPUs to render corresponding image frames. The rendering command buffer may include commands to set state and run primitive or computer shaders to perform an analysis pre-pass. Sink operations can be included in the command buffer to synchronize the initiation of various operations by the GPU. For example, a sink operation can be used to synchronize the start of the geometry pass phase of rendering by the GPU. As such, the command buffer may contain draw calls and state settings for each object to perform the geometry pass phase of rendering.

In one embodiment, generation of information is accelerated by using one or more dedicated instructions. That is, a shader that generates information uses one or more specialized instructions to accelerate the generation of information about the relationship between a piece of geometry and its screen area.

In one embodiment, the instructions may calculate the exact overlap between each of the primitives of the piece of geometry and the screen area. For example, FIG. 13A illustrates the process of performing an analysis pre-pass phase to generate information used for dynamic allocation of screen space to GPUs for geometry rendering of image frames, according to one embodiment of the present disclosure. , a diagram 1310 illustrating calculation of exact overlap between a primitive 1350 and one or more screen regions. For example, primitive 1350 is shown to overlap three different regions, and the overlap of each portion of primitive 1350 is precisely determined for each of the regions.

In other embodiments, to reduce the complexity of the instruction implementation, the instruction may perform an overlap area estimation, and the information includes an estimate of the extent to which the primitive overlaps one or more screen areas. Including area. Specifically, the instructions may calculate an approximate overlap between a primitive of a piece of geometry and one or more screen regions. For example, FIG. 13B illustrates the process of performing an analysis pre-pass phase to generate information used for dynamic allocation of screen space to GPUs for geometry rendering of image frames, according to one embodiment of the present disclosure. , a pair of diagrams illustrating the calculation of approximate overlap between a piece of geometry and multiple screen regions;

As shown in the left diagram of FIG. 13B, instructions can use bounding boxes of primitives. Thus, the overlap between the bounding box of primitive 1350 and one or more screen regions is determined. Boundary 1320A indicates the approximate overlap of pieces of geometry 1350 determined through bounding box analysis.

In the right-hand diagram of Figure 13B, the instructions check the screen area for the primitives, exclude screen areas where no piece of geometry overlaps, and create a bounding box for the portion of the primitive that overlaps each screen area. generated. Boundary 1320B indicates the approximate overlap of primitives 1350 as determined by bounding box analysis and overlap filtering. Note that bounding box 1320B in the right diagram of FIG. 13B is smaller than bounding box 1320A in the left diagram of FIG. 13B.

In yet other embodiments, to further reduce the complexity of the instructions, the instructions may generate presence information, such as whether a piece of geometry is present in the screen area. For example, presence information can indicate whether a primitive of a piece of geometry overlaps a screen area. The information may include the approximate existence of the piece of geometry within the corresponding screen area.

In another embodiment, the shader does not allocate space for position or parameter caches. That is, the shader does not perform position or parameter cache allocation, which allows for a high degree of parallelism when performing the analysis pre-pass. This also leads to a corresponding reduction in the time required for analysis pre-pass.

In another embodiment, a single shader is used to perform either the analysis performed in the analysis pre-pass or the rendering in the geometry pass. For example, a shader that generates information can be configured to output information about a piece of geometry and its relationship to the screen area, or to output vertex position and parameter information for use in a later rendering stage. There may be. This can be accomplished in a variety of ways, such as through external hardware state that the shader can check (eg, setting hardware registers), or through inputs to the shader. As a result, the shader performs two different functions to render the corresponding image frame.

As described above, this information is used to allocate regions to the GPU before starting the geometry pass phase of rendering. Information generated during rendering of previous frames (e.g., the actual number of pixels shaded while rendering a piece of geometry) can also be used to allocate screen area to the GPU. Information from previous frames may include, for example, the actual number of pixels shaded per piece of geometry per screen area. That is, screen area is allocated to GPUs based on information generated from previous image frames (eg, GPU usage) and information generated during the analysis pre-pass.

Systems and Methods for Efficient Multi-GPU Rendering of Geometry by Subdividing Geometry With a detailed description of cloud gaming network 190 (e.g., within game server 160) and GPU resources 365 of FIGS. 1-3, line 1110 of FIG. , presents a method for graphics processing including multi-GPU rendering of applications by subdividing the geometry. Objects 0, 1, and 2 represent the geometry used or generated by the individual draw calls. Rather than distributing the entire object (i.e., draw calls) across GPU-A, GPU-B, GPU-C, and GPU-D, the GPUs instead handle each object with a location and/or parameter cache assigned to it. Split geometry into smaller pieces, such as roughly sized pieces. Purely for illustration purposes, object 0 is divided into pieces "a,""b,""c,""d,""e," and "f," like object 610 in FIG. 6B. Further, object 1 is divided into pieces "g", "h", and "i". Furthermore, object 2 is divided into pieces "j", "k", "l", "m", "n", and "o". Distribution 1110 (eg, line ABCDABCDABCD...) indicates an even distribution of rendering (or phases of rendering) responsiveness among multiple GPUs. Because this distribution is more fine-grained than the entire object (i.e., draw calls), the rendering time imbalance between GPUs is reduced and the total rendering time (or rendering phase time) is reduced. Flow diagram 1400A in FIG. 14A and line 1410 in FIG. A method for graphics processing, including rendering, is presented. One or more of the various features and advantages of the various embodiments described with respect to the rendering of FIGS. 7-13 and the generation of information before and during the geometry pass phase of rendering include subdividing the geometry; It is understood that they are equally applicable for use in performing timing analysis and/or may not be repeated here to minimize duplication of description. As mentioned above, various architectures are available for rendering, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system such as a personal computer or game console that includes a high-end graphics card with multiple GPUs. Performing multi-GPU rendering of an application's geometry through area testing during the process can involve multiple GPUs working together to render a single image.

In some embodiments, the responsiveness of GPU rendering is based on multiple screen regions for each image frame, such that each GPU renders objects in its assigned screen region, as previously described with respect to FIGS. 7-13. fixedly or dynamically allocated between In other embodiments, each GPU renders to its own Z-buffer or other render target. Timing analysis is performed in one or more of the phases of rendering (e.g., geometry prepass analysis, Z prepass, or geometry rendering), the purpose of which is to redistribute GPU responsiveness allocation in these phases. . That is, timing analysis is performed during the rendering phase in order to redistribute the allocation of GPU responsiveness during the rendering phase, such as for geometry pieces for geometry rendering of an image frame, in one implementation. such as when executing a Z pre-pass phase to generate information used for dynamic allocation of screen space to GPUs. For example, screen space initially allocated to one GPU may be reallocated to another GPU during the rendering phase (e.g., one GPU may be lagging behind other GPUs during that phase). ).

At 1410, the method includes rendering graphics for the application using multiple graphics processing units (GPUs). In particular, multi-GPU processing is performed when rendering a single image frame and/or each of one or more image frames of a series of image frames for real-time applications. That is, multiple GPUs work together to render corresponding image frames that include multiple pieces of geometry.

At 1420, the method includes partitioning responsiveness for rendering geometry of the graphic among multiple GPUs based on multiple screen regions. That is, each GPU has a corresponding division of responsiveness (a corresponding set of screen areas).

During geometry rendering or geometry analysis, the rendering or analysis time is used to adjust the responsiveness divisions for the object. In particular, at 1430, the method includes determining that the first GPU lags at least one other GPU, such as a second GPU, during a phase of image frame rendering or analysis. At 1440, the method includes dynamically allocating geometry such that the first GPU is allocated less than the second GPU.

For example, dynamic allocation of geometry can be performed during Z-buffer generation, for illustration purposes. Dynamic assignment of geometry may be performed during the analysis pre-pass and/or the geometry pass phase of rendering. Z-Buffer Generation and Z-Pre-Pass When dynamically allocating geometry during analysis, one or more Z-buffers are generated by multiple GPUs and/or collaborated on an image frame during the Z-prepass phase of rendering. and then merged. Specifically, pieces of geometry are divided among GPUs to process the Z pre-pass phase of rendering, and each of the plurality of pieces of geometry is assigned to a corresponding GPU. Instead of using the hardware during the Z pre-pass phase to generate information used to optimize the rendering of the corresponding image frame, the hardware performs an analysis pre-pass to e.g. Can be configured to generate information used to optimize rendering speed of geometry passes.

Specifically, the object can be subdivided into smaller pieces as previously described in FIG. 6B. The rendering responsibilities of pieces of geometry during the Z pre-pass phase of rendering are distributed among the GPUs in an interleaved manner, as described above with respect to distribution 1110 of FIG. 14B, and FIG. Figure 3 illustrates various distributions of GPU allocation to generate information used to dynamically allocate screen area to GPUs for geometric rendering of image frames. Variance 1110 shows the distribution of rendering responsiveness among multiple GPUs for the Z pre-pass. As mentioned above, each GPU is assigned to a corresponding portion of the image frame's geometry, which may be further divided into pieces of geometry. As shown in the distribution 1110, successive pieces of geometry are assigned to different GPUs, resulting in roughly balanced rendering times during the Z pre-pass.

Further balancing of rendering times between GPUs can be achieved by dynamically adjusting the responsivity of rendering pieces of geometry, as shown in distribution 1410. This is the distribution of pieces of geometry to the GPU when performing the Z pre-pass phase of rendering, and is dynamically adjusted during that phase of rendering. For example, distribution 1410 [Row ABCDABCDBCDBBCD] shows an asymmetric distribution of responsibilities that executes the Z pre-pass phase across multiple GPUs. For example, asymmetric distribution may be advantageous if a particular GPU is delayed in Z prepass relative to other GPUs due to being assigned a larger piece of geometry than those assigned to other GPUs.

As shown in variance 1410, GPU-A is spending more time rendering pieces of geometry during the Z pre-pass phase, so it is skipped when assigning pieces of geometry to GPUs. For example, instead of having GPU-A process a piece of geometry "i" of object 1 during Z pre-pass rendering, GPU-B is assigned to render a piece of geometry during the Z pre-pass phase. Therefore, GPU-B is assigned more pieces of geometry than GPU-A during the Z pre-pass phase of rendering. Specifically, during the Z pre-pass phase of rendering, pieces of geometry are deallocated from a first GPU and assigned to a second GPU. Additionally, GPU-B is more advanced than the other GPUs, so it can process more geometry during the Z pre-pass phase. That is, distribution 1410 indicates the iterative assignment of GPU-B to successive pieces of geometry for Z pre-pass rendering. For example, GPU-B is assigned to process geometry pieces "l" and "m" of object 2 during the Z pre-pass phase.

Although the above is presented in terms of "dynamic allocation" of geometry, it is equally valid to view it in terms of "allocation" and "reallocation". For example, as shown in distribution 1410, GPU-A is reassigned because it is spending more time rendering pieces of geometry during the Z pre-pass phase. For example, instead of having GPU-A process a piece of geometry 'i' of object 1 during Z pre-pass rendering, GPU-B is assigned to render a piece of geometry 'i' during the Z pre-pass phase, and GPU-A , may be initially assigned to render a piece of geometry. Additionally, GPU-B is more advanced than the other GPUs, so it can process more geometry during the Z pre-pass phase. That is, distribution 1410 indicates the repeated assignment or reallocation of GPU-B to successive pieces of geometry for Z pre-pass rendering. For example, GPU-B is assigned to process geometry pieces "l" and "m" of object 2 during the Z pre-pass phase. That is, to render a piece of geometry "l" of object 2, GPU-B is assigned, even though that piece of geometry may have been initially assigned to GPU-A. As such, pieces of geometry initially assigned to a first GPU are reassigned to a second GPU (where rendering may be in progress) during the Z pre-pass phase of rendering.

Although the allocation of geometry pieces during the Z pre-pass phase to the GPU may be unbalanced, the processing performed by the GPU during the Z pre-pass phase may turn out to be approximately balanced. (e.g. each GPU spends approximately the same amount of time performing the Z pre-pass phase of rendering).

In another embodiment, dynamic allocation of geometry may be performed during the geometry pass phase of image frame rendering. For example, screen area is assigned to the GPU during the geometry pass phase of rendering based on information generated during the Z pre-pass or analysis pre-pass. Screen area assigned to one GPU may be reallocated to another GPU during the rendering phase. This may improve efficiency, as GPUs that are more advanced than others may be allocated additional screen space, and GPUs that are more advanced than others may be allocated additional screen space. This is because the area can be avoided from being allocated. In particular, multiple GPUs working together generate a Z-buffer of image frames during the Z-prepass phase of rendering. Information is generated during this Z pre-pass about the geometry pieces of the image frame and their relationship to multiple screen regions. Screen area is informedly allocated to the GPU for rendering image frames during the geometry pass phase of rendering. The GPU renders pieces of geometry during the geometry pass phase of rendering based on GPU-to-screen area assignments. The timing analysis is performed during the geometry pass phase of rendering, such that the first piece of geometry initially assigned to the first GPU is reloaded to the second GPU for rendering during the geometry pass phase. may be assigned. For example, in one embodiment, the first GPU may be behind in processing the geometry pass phase of rendering. In another embodiment, a second GPU may be ahead in processing the geometry pass phase of rendering.

15A-15B illustrate various screen area allocation strategies that can be applied to rendering image frames as previously described with respect to FIGS. 7-14.

Specifically, FIG. 15A illustrates the use of multiple GPUs to render a piece of geometry (e.g., the geometry associated with objects 0-3) in a particular screen region, according to one embodiment of the present disclosure. FIG. That is, screen area 1510 may be assigned to multiple GPUs for rendering. This can improve efficiency, for example when there is very dense geometry later in the rendering phase. Allocating screen area 1510 to multiple GPUs typically requires subdividing the screen area so that each GPU can have the responsiveness of a portion or portions of the screen area.

FIG. 15B is a diagram illustrating rendering pieces of geometry out of order with their corresponding draw calls, according to an embodiment of the present disclosure. In particular, the rendering order of pieces of geometry may not match the order of their corresponding draw calls in their corresponding command buffers. As shown in this example, object 0 has priority over object 1 in the rendering command buffer. However, objects 0 and 1 intersect, such as within screen area C. In that case, region C may need to adhere to a strict order of rendering. That is, object 0 needs to be rendered before object 1 in area C.

On the other hand, since objects in area A and area B do not intersect, they can be rendered in any order. That is, when rendering region A and/or region B, object 1 may precede object 0, and vice versa.

In yet another embodiment, if the rendering command buffer can be traversed multiple times, the first traversal renders a specific screen area (e.g., a high-cost area), and a second or subsequent traversal renders the remaining area (e.g. , low-cost region). The rendering order of the resulting pieces of geometry may not match the order of the corresponding draw calls, such as when a first object is rendered in a second traversal. Since load balancing between GPUs is easier in low-cost regions than in high-cost regions, this strategy increases efficiency in rendering the corresponding image frames.

FIG. 16 illustrates components of an example device 1600 that can be used to perform aspects of various embodiments of this disclosure. For example, FIG. 16 shows that by performing geometry analysis during rendering and dynamically allocating screen space to the GPU for geometry rendering of image frames and/or prior to rendering, according to embodiments of the present disclosure. by dynamically allocating screen space to the GPU for geometry rendering of image frames, and/or by subdividing pieces of geometry to create smaller pieces of the resulting geometry. 1 illustrates an example hardware system suitable for multi-GPU rendering of geometry for applications by assigning it to multiple GPUs. This block diagram can or may incorporate a personal computer, server computer, game console, mobile device, or other digital device, each of which may implement embodiments of the present invention. A suitable device 1600 is shown. Device 1600 includes a central processing unit (CPU) 1602 for running software applications and optionally an operating system. CPU 1602 may be comprised of one or more homogeneous or dissimilar processing cores.

According to various embodiments, CPU 1602 is one or more general purpose microprocessors with one or more processing cores. Further embodiments provide one or more microprocessor architectures specifically adapted for highly parallel and computationally intensive applications, such as media and interactive entertainment applications, applications configured for graphics processing during game execution. It can be implemented using several CPUs.

Memory 1604 stores applications and data used by CPU 1602 and GPU 1616. Storage 1606 provides non-volatile storage and other computer-readable media for applications and data and includes fixed disk drives, removable disk drives, flash memory devices, and CD-ROMs, DVD-ROMs, Blu-rays ( trademark), HD-DVD, or other optical storage devices, as well as signal transmission and storage media. User input devices 1608 communicate user input to device 1600 from one or more users, such as a keyboard, mouse, joystick, touch pad, touch screen, still or video recorder/camera, etc. and/or a microphone. Network interface 1609 enables device 1600 to communicate with other computer systems via electronic communications networks, which may include wired or wireless communications across local area networks and wide area networks, such as the Internet. Audio processor 1612 is adapted to generate analog or digital audio output from instructions and/or data provided by CPU 1602, memory 1604, and/or storage 1606. Components of device 1600, including CPU 1602, graphics subsystem including GPU 1616, memory 1604, data storage 1606, user input device 1608, network interface 1609, and audio processor 1612, are connected via one or more data buses 1622. ing.

Graphics subsystem 1614 is further connected to data bus 1622 and components of device 1600. Graphics subsystem 1614 includes at least one graphics processing unit (GPU) 1616 and graphics memory 1618. Graphics memory 1618 includes display memory (eg, a frame buffer) used to store pixel data for each pixel of the output image. Graphics memory 1618 may be integrated into the same device as GPU 1616, may be connected as a separate device from GPU 1616, and/or may be implemented within memory 1604. Pixel data may be provided directly from CPU 1602 to graphics memory 1618. Alternatively, CPU 1602 provides data and/or instructions defining a desired output image to GPU 1616 from which GPU 1616 generates pixel data for one or more output images. Data and/or instructions defining a desired output image may be stored in memory 1604 and/or graphics memory 1618. In one embodiment, GPU 1616 includes 3D rendering functionality that generates pixel data for the output image from instructions and data that define scene geometry, lighting, shading, texture, motion, and/or camera parameters. GPU 1616 may further include one or more programmable execution units that can execute shader programs.

Graphics subsystem 1614 periodically outputs image pixel data from graphics memory 1618 for display on display device 1610 or for projection by a projection system (not shown). Display device 1610 may be any device capable of displaying visual information in response to signals from device 1600, including CRT, LCD, plasma, and OLED displays. Device 1600 can provide, for example, an analog signal or a digital signal to display device 1610.

Other embodiments for optimizing the graphics subsystem 1614 include multi-GPU rendering of an application's geometry by pre-testing the geometry against interleaved screen regions before rendering objects in image frames. can be included. Graphics subsystem 1614 may be configured as one or more processing devices.

For example, graphics subsystem 1614 may be configured, in one embodiment, to perform multi-GPU rendering of an application's geometry with area testing during rendering, and multiple graphics subsystems may perform graphics rendering for a single game. and/or may implement a rendering pipeline. That is, graphics subsystem 1614 includes multiple GPUs that are used to render each of one or more images of an image, or series of images, when executing an application.

In other embodiments, graphics subsystem 1614 includes multiple GPU devices that combine to perform graphics processing for a single application running on corresponding CPUs. For example, multiple GPUs can perform multi-GPU rendering of an application's geometry with area testing during rendering of objects in an image. In other examples, multiple GPUs may perform alternative forms of frame rendering, where in consecutive frame periods, GPU1 renders a first frame, GPU2 renders a second frame, and so on. , when the last GPU is reached, the first GPU renders the next video frame (e.g., if there are only two GPUs, GPU1 renders the third frame). That is, the GPU cycles when rendering frames. Rendering operations can overlap, in which GPU2 can start rendering a second frame before GPU1 finishes rendering the first frame. In another implementation, multiple GPU devices may be assigned different shader operations in the rendering and/or graphics pipeline. The master GPU is performing the main rendering and compositing. For example, in a group containing three GPUs, master GPU1 can perform main rendering (e.g., a first shader operation) and compositing of outputs from slave GPU2 and slave GPU3, while slave GPU2 can perform main rendering (e.g., a first shader operation), and slave GPU2 can perform main rendering (e.g., a first shader operation) and compositing of outputs from slave GPU2 and slave GPU3, The slave GPU3 can perform a third shader (e.g., particle smoke) operation, and the master GPU1 combines the results from each of GPU1, GPU2, and GPU3. In this way, different GPUs can be assigned to perform different shader operations (flag waving, wind, smoke production, flames, etc.) to render video frames. In yet another embodiment, each of the three GPUs may be assigned to a different object and/or portion of the scene corresponding to a video frame. In the embodiments and implementations described above, these operations can be performed in the same frame period (simultaneously in parallel) or in different frame periods (sequentially in parallel).

Accordingly, the present disclosure provides techniques for performing geometry analysis during rendering and dynamically allocating screen space to the GPU for geometry rendering of image frames, and/or performing geometry analysis prior to rendering. , by dynamically allocating screen space to GPUs for geometry rendering of image frames, and/or by subdividing pieces of geometry and assigning smaller parts of the resulting geometry to multiple GPUs. , describes a method and system configured for multi-GPU rendering of geometry for applications.

It is to be understood that the various embodiments defined herein can be combined or assembled into specific implementations using the various features disclosed herein. Accordingly, the examples provided are only some of the possible examples and are not intended to be limiting to the various embodiments, which can define many more embodiments by combining various elements. In certain examples, certain embodiments may include fewer elements without departing from the spirit of the disclosed or equivalent embodiments.

Embodiments of the present disclosure may be implemented in a variety of computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. Embodiments of the present disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through wire-based or wireless networks.

With the above embodiments in mind, it should be understood that embodiments of the present disclosure may employ various computer-implemented operations involving data stored on computer systems. These operations are those requiring physical manipulations of physical quantities. Any of the operations described herein that form part of an embodiment of the present disclosure are useful mechanical operations. The disclosed embodiments also relate to devices or apparatus for performing these operations. The device can be specially constructed for the required purpose. Alternatively, the device may be a general purpose computer that is selectively activated or configured by a computer program stored on the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized equipment to perform the required operations. In some cases.

The present disclosure may also be embodied as computer readable code on a computer readable medium. A computer-readable medium is any data storage device that can store data that can later be read by a computer system. Examples of computer readable media are hard drives, network attached storage (NAS), read only memory, random access memory, CD-ROM, CD-R, CD-RW, magnetic tape, and other optical and non-optical data storage. Including devices. Computer-readable media can include computer-readable tangible media that is distributed over a network-coupled computer system so that computer-readable code is stored and executed in a distributed manner.

Although the method operations have been described in a particular order, other maintenance operations may be performed between the operations, as long as the processing of the overlay operations is performed in the desired manner, or the operations occur at slightly different times. It should be appreciated that the processing operations may be coordinated to occur or may be distributed within the system to allow processing operations to occur at various processing-related intervals.

Although the foregoing disclosure has been described in some detail for clarity of understanding, it will be obvious that certain changes and modifications may be practiced within the scope of the appended claims. Accordingly, the present embodiments are to be regarded as illustrative rather than limiting, and embodiments of the present disclosure are not limited to the details provided herein, but rather within the scope and equivalents of the appended claims. May be changed within the product.

Claims (26)

A method for graphic processing, the method comprising:
Render graphics for applications using multiple graphics processing units (GPUs),
dividing the responsiveness of processing multiple pieces of geometry of an image frame during an analysis pre-pass phase between said plurality of GPUs, each of said plurality of geometry pieces being responsible for performing said processing during said analysis pre-pass phase; is uniquely assigned to the corresponding GPU for
determining an overlap of each of the plurality of pieces of geometry with each of a plurality of screen regions in the analysis pre-pass phase , the analysis pre-pass phase being performed during a frame period ;
information about the plurality of geometry pieces and their relationships to the plurality of screen regions based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions; and the information is executed during the frame period;
dynamically allocating the plurality of screen areas to the plurality of GPUs based on the information for rendering the plurality of geometry pieces during a subsequent phase of rendering performed during the frame period ; Method. 2. The method of claim 1 , wherein the analysis pre-pass phase is performed using a vertex shader or a compute shader. In determining the overlap,
2. The method of claim 1 , wherein the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions is estimated. In said estimation of said overlap:
4. The method of claim 3 , determining an overlap of one or more bounding boxes of one or more primitives of a piece of geometry with each of the plurality of screen regions. 5. The method of claim 4 , wherein one or more screen areas with no overlap are excluded. Further, during a subsequent phase of the rendering, the plurality of GPUs may be configured to 2. The method of claim 1 , wherein multiple pieces of geometry are rendered. Furthermore, it determines the GPU usage when rendering the previous image frame,
2. The method of claim 1 , allocating the plurality of screen areas to the plurality of GPUs based on the information and usage of the GPU when performing the rendering of the previous image frame. the pieces of geometry correspond to geometry used or generated by a draw call, or
The geometry used or generated by the draw call is subdivided into smaller pieces of geometry corresponding to the plurality of pieces of geometry, such that the information is generated for the smaller pieces of geometry. The method described in Section 1 . 2. The method of claim 1 , wherein the information includes an exact or approximate area that a primitive of a piece of geometry occupies in a corresponding region. The information includes the number of shaded pixels per screen area, or
The method of claim 1 , wherein the information includes a number of vertices per screen area. 2. The method of claim 1 , wherein corresponding information may or may not be generated depending on one or more properties of the corresponding pieces of geometry. further determining a plurality of costs for rendering the plurality of geometry pieces during a subsequent phase of the rendering;
2. The method of claim 1 , wherein the plurality of costs are considered when performing dynamic allocation of the plurality of screen areas to the plurality of GPUs . the information is generated by one or more shaders;
2. The method of claim 1 , wherein the one or more shaders use at least one dedicated instruction to accelerate generation of the information. the information is generated by one or more shaders;
2. The method of claim 1 , wherein the one or more shaders do not perform position or parameter cache allocation. the information is generated by one or more shaders;
The method of claim 1 , wherein the one or more shaders are configurable to output the information or output vertex position and parameter information for use in subsequent phases of the rendering. . 2. The method of claim 1 , wherein at least one of the plurality of GPUs is assigned to a screen area before or during a subsequent phase of rendering. 2. The method of claim 1 , wherein screen area initially allocated to a first GPU is reallocated to a second GPU during a subsequent phase of the rendering. 2. The method of claim 1 , wherein screen area is allocated to two or more of the plurality of GPUs. 2. The method of claim 1 , wherein the rendering order of the plurality of geometry pieces does not match the order of corresponding draw calls in a rendering command buffer. The rendering command buffer is shared among the plurality of GPUs as a common rendering command buffer,
20. The method of claim 19 , wherein the common rendering command buffer format allows commands to be executed by only a subset of the plurality of GPUs. Said information allows for relaxation of rendering phase dependencies such that a first GPU proceeds to a subsequent phase of said rendering while a second GPU is still processing said analysis pre-pass phase. 2. The method of claim 1 , wherein: 2. The method of claim 1 , wherein the information is used to schedule the transfer of screen area Z-buffer or render target data from a second GPU to a first GPU. 2. The method of claim 1 , wherein one or more of the plurality of GPUs are part of a larger GPU configured as a plurality of virtual GPUs. A computer system,
a processor;
a memory coupled to the processor and storing instructions that, when executed by the computer system, cause the computer system to perform a method for graphics processing, the method comprising:
Render graphics for applications using multiple graphics processing units (GPUs),
splitting the responsiveness of processing multiple geometry pieces of an image frame during an analysis pre-pass phase of rendering among the plurality of GPUs, each of the plurality of geometry pieces being processed during the analysis pre-pass phase of rendering; uniquely assigned to the corresponding GPU for execution ,
determining an overlap of each of the plurality of pieces of geometry with each of a plurality of screen regions in the analysis pre-pass phase , the analysis pre-pass phase being performed during a frame period ;
information about the plurality of geometry pieces and their relationships to the plurality of screen regions based on the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions; and the information is executed during the frame period;
dynamically allocating the plurality of screen areas to the plurality of GPUs based on the information for rendering the plurality of geometry pieces during a subsequent phase of rendering performed during the frame period ; computer system. 25. The computer system of claim 24 , in the method, the analysis pre-pass phase is performed using a vertex shader or a compute shader. In the method, determining the overlap includes:
25. The computer system of claim 24 , estimating the overlap of each of the plurality of geometry pieces with each of the plurality of screen regions.