CN115089966A

CN115089966A - Video rendering method and system applied to cloud game and related equipment

Info

Publication number: CN115089966A
Application number: CN202210957309.4A
Authority: CN
Inventors: 黄本良; 张鹤翔; 李鑫; 郭建君
Original assignee: Beijing Weiling Times Technology Co Ltd
Current assignee: Beijing Weiling Times Technology Co Ltd
Priority date: 2022-08-10
Filing date: 2022-08-10
Publication date: 2022-09-23
Anticipated expiration: 2042-08-10
Also published as: CN115089966B

Abstract

The embodiment of the application discloses a video rendering method, a system, a storage medium and a server applied to a cloud game, wherein the method comprises the following steps: receiving a game picture display request sent by a user terminal when a cloud game is operated, and acquiring equipment parameter information of the user terminal, wherein the equipment parameter information comprises a first rendering parameter locally supported by the user terminal; comparing the first rendering parameter with a second rendering parameter supported by the cloud server to determine an actual rendering parameter; rendering the game picture based on the actual rendering parameters to obtain a rendered game picture; and coding the rendered game picture, and sending the game picture to the user terminal so as to enable the user terminal to decode and display the game picture. By adopting the embodiment of the application, the adaptation degree of the video picture rendered by the cloud server to different user terminals can be improved, and further the picture is prevented from stretching and deforming.

Description

Video rendering method and system applied to cloud game and related equipment

Technical Field

The application relates to the technical field of cloud games, in particular to a video rendering method and system applied to a cloud game and related equipment.

Background

With the rapid development of the 5G technology, China becomes one of the fastest and most concerned areas of the global cloud game industry, and becomes the first main and passenger area of the global cloud game industry. With the gradual improvement of the industry chain of the cloud game industry, the cloud game industry will form a similar income mode with the traditional sports in the future, and the whole scale of the cloud game industry is not limited.

Cloud games are game modes based on cloud computing, and the main technologies used by the cloud games include a cloud computing technology for completing game running and picture rendering at a cloud end and a streaming media transmission technology for a user terminal and a cloud server.

However, the video picture rendered by the cloud server is not adapted to different user terminals, so that the picture is stretched and deformed during the running process of the cloud game.

Disclosure of Invention

The embodiment of the application provides a video rendering method, a video rendering system, a storage medium and a server applied to a cloud game, which can improve the adaptation degree of video pictures rendered by the cloud server to different user terminals, and further avoid picture stretching deformation.

The technical scheme is as follows:

in a first aspect of the present application, there is provided a video rendering method applied to a cloud game, applied to a cloud server, the method including:

receiving a game picture display request sent by a user terminal when a cloud game is operated, and acquiring equipment parameter information of the user terminal, wherein the equipment parameter information comprises a first rendering parameter locally supported by the user terminal;

comparing the first rendering parameter with a second rendering parameter maximally supported by the cloud server, and determining an actual rendering parameter;

rendering the game picture based on the actual rendering parameters to obtain a rendered game picture;

and coding the rendered game picture, and sending the game picture to the user terminal so as to enable the user terminal to decode and display the game picture.

By adopting the technical scheme, when the user terminal runs the cloud game, the game picture display request is sent to the cloud server, the cloud server receives the request and simultaneously acquires the equipment parameter information of the user terminal, the actual rendering parameters suitable for the user terminal are determined based on the equipment parameter information of the user terminal so as to render the game picture, and then the rendered game picture is coded and sent to the user terminal. The cloud server sets rendering parameters adaptive to the user terminal according to the equipment parameter information of the user terminal, so that the adaptive capacity of the cloud server to different user terminals is improved, and further, the stretching deformation of the picture is avoided.

Optionally, the determining the actual rendering parameter by comparing the first rendering parameter with the second rendering parameter that is maximally supported by the cloud server includes:

comparing whether the second rendering parameter exceeds the first rendering parameter;

if the second rendering parameter exceeds the first rendering parameter, determining the first rendering parameter as an actual rendering parameter;

and if the second rendering parameter does not exceed the first rendering parameter, performing scaling operation on the first rendering parameter to determine an actual rendering parameter.

By adopting the technical scheme, when the rendering parameter is the resolution, the first rendering parameter supported by the user terminal is compared with the second rendering parameter supported by the server to the maximum extent, and if the second rendering parameter exceeds the first rendering parameter, the first rendering parameter is determined to be the actual rendering parameter; if the second rendering parameter does not exceed the first rendering parameter, scaling the first rendering parameter to determine the actual rendering parameter, exerting the terminal performance and resolution support to the maximum, and rendering according to the proportion to avoid stretching deformation of the picture.

and if the second rendering parameter does not exceed the first rendering parameter, determining the second rendering parameter as an actual rendering parameter.

According to the technical scheme, when the rendering parameter is the refresh rate, a first rendering parameter supported by the user terminal is compared with a second rendering parameter supported by the server to the maximum extent, and if the second rendering parameter exceeds the first rendering parameter, the first rendering parameter is determined to be an actual rendering parameter; and if the second rendering parameter does not exceed the first rendering parameter, determining the second rendering parameter as an actual rendering parameter, exerting the performance of the terminal to the maximum, and rendering according to the proportion to avoid stretching deformation of the picture.

Optionally, the scaling the first rendering parameter to determine an actual rendering parameter includes:

determining a scaling, wherein the first rendering parameter is smaller than or equal to the second rendering parameter after being scaled according to the scaling;

and multiplying the first rendering parameter by the scaling ratio to obtain the actual rendering parameter.

By adopting the technical scheme, the scaling ratio is determined by adjusting the first rendering parameter supported by the client according to the second rendering parameter supported by the cloud server to the maximum, and the actual rendering parameter is obtained by multiplying the first rendering parameter by the scaling ratio, so that the adjusted first rendering parameter supported by the client is more matched with the second rendering parameter supported by the cloud server to the maximum.

Optionally, the cloud server includes a video memory and a memory, the rendered game picture is encoded and sent to the user terminal, so that the user terminal decodes and displays the game picture, including:

coding the rendered game picture in the video memory to obtain a coded picture;

and storing the coded picture into the memory, and sending the coded picture in the memory to the user terminal so as to enable the user terminal to decode and display the coded picture.

By adopting the technical scheme, the game picture is rendered in the video memory of the GPU based on the actual rendering parameters, and the game picture is directly coded through the video memory. The resource consumption and resource occupation of copying the data in the video memory to the memory are reduced, and the calculation power of the server is saved.

Optionally, the receiving a game screen display request sent by a user terminal when the cloud game is running, and acquiring device parameter information of the user terminal includes:

receiving a game picture display request sent by a user terminal when a cloud game is operated;

judging whether the game picture display request is a first request or not;

if the game picture display request is a first request, acquiring equipment parameter information from the user terminal;

and if the game picture display request is not a first request, searching the equipment parameter information of the user terminal in a cache.

By adopting the technical scheme, whether the game picture display request sent by the user terminal is the first request is judged, if yes, the equipment parameter information of the user terminal is obtained, and the equipment parameter information of the user terminal is stored in the cloud server after the video rendering is finished so as to be directly called next time; if not, the equipment parameter information of the user terminal is searched in the cache, the computing power of cloud service is saved, and the running speed of the cloud game is improved.

In a second aspect of the present application, there is provided a video rendering method applied to a cloud game, applied to a user terminal, the method including:

sending a game picture display request to a cloud server when a cloud game is operated so that the cloud server can obtain equipment parameter information of the user terminal, wherein the equipment parameter information comprises a first rendering parameter locally supported by the user terminal, and the cloud server can determine an actual rendering parameter after comparing the first rendering parameter with a second rendering parameter maximally supported by the cloud server, render and encode a game picture based on the actual rendering parameter and then send the game picture to the user terminal;

and receiving the coded game picture sent by the cloud server, and decoding and displaying the game picture.

By adopting the technical scheme, when the user terminal runs the cloud game, the game picture display request is sent to the cloud server, the cloud server receives the request and simultaneously obtains the equipment parameter information of the user terminal, the actual rendering parameters suitable for the user terminal are determined based on the equipment parameter information of the user terminal so as to render the game picture, and then the rendered game picture is coded and sent to the user terminal. The cloud server sets rendering parameters adaptive to the user terminal according to the equipment parameter information of the user terminal, so that the adaptive capacity of the cloud server to different user terminals is improved, and the stretching deformation of the picture is avoided.

In a third aspect of the present application, there is provided a video rendering system applied to a cloud game, the system including:

the cloud game system comprises a device information acquisition module, a processing module and a display module, wherein the device information acquisition module is used for receiving a game picture display request sent by a user terminal when a cloud game is operated, and acquiring device parameters of the user terminal, and the device parameter information comprises a first rendering parameter locally supported by the user terminal;

the rendering parameter confirming module is used for comparing the first rendering parameter with a second rendering parameter which is supported by the cloud server to the maximum extent, and determining an actual rendering parameter;

the game video rendering module is used for rendering the game picture based on the actual rendering parameter information to obtain a rendered game picture;

and the game picture coding module is used for coding the rendered game picture and sending the coded game picture to the user terminal so as to enable the user terminal to decode and display the game picture.

By adopting the technical scheme, when the user terminal runs the cloud game, the game picture display request is sent to the cloud server, the cloud server receives the request and simultaneously obtains the equipment parameter information of the user terminal, the actual rendering parameters suitable for the user terminal are determined based on the equipment parameter information of the user terminal to render the game picture, and then the rendered game picture is coded and sent to the user terminal. The cloud server sets rendering parameters adaptive to the user terminal according to the device parameter information of the user terminal so as to render the game picture, and therefore the user terminal can smoothly display the game picture.

In a fourth aspect of the present application, a computer storage medium is provided, which stores a plurality of instructions adapted to be loaded by a processor and to perform the above-mentioned method steps.

In a fifth aspect of the application a server is provided comprising a processor, a memory and a transceiver, wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

In summary, the present application includes at least one of the following beneficial technical effects:

1. the cloud server sets rendering parameters adapted to the user terminal according to the device parameter information of the user terminal, so that the adaptation capability of the cloud server to different user terminals is improved, and the game experience of a user is improved;

2. the rendered game picture is directly coded in the video memory to obtain a coded picture, and the coded picture is stored in the memory, so that the user terminal decodes and displays the coded picture, the action of screenshot on the window picture of the operating system is reduced, or the picture in the video memory is captured and copied to the memory, then copied to the coder by the memory for coding, and finally copied to the memory and sent to the client through the network layer, thereby saving the calculation power of the server and reducing the delay of the cloud game.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic system architecture diagram of a video rendering method applied to a cloud game according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a video rendering method applied to a cloud game according to an embodiment of the present disclosure;

fig. 3 is a schematic view of a scenario in which a plurality of user terminals simultaneously send game screen display requests according to an embodiment of the present application;

fig. 4 is a scene schematic diagram of a rendering manner selected by a user terminal according to an embodiment of the present application;

FIG. 5 is a schematic view of a UI design setting screen scene applied when a user terminal renders according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another embodiment of a video rendering method applied to a cloud game according to an embodiment of the present disclosure;

fig. 7a is a scene diagram of a user mobile terminal in a vertical screen game state according to an embodiment of the present application;

FIG. 7b is a diagram of a scenario of a user mobile terminal in a landscape game state according to an embodiment of the present application;

FIG. 8a is a scene diagram of a user PC terminal in a full-screen game state according to an embodiment of the present application;

FIG. 8b is a diagram of a scenario in a zoom game state of a PC terminal of a user according to an embodiment of the present application;

FIG. 9 is a comparison chart of logical resolution, rendering resolution, and physical allocation rate of a user terminal screen according to an embodiment of the present disclosure;

FIG. 10a is a diagram illustrating a case where a first rendering parameter is included in a second rendering parameter according to an embodiment of the present application;

FIG. 10b is a schematic diagram of a first rendering parameter intersection and a second rendering parameter provided by an embodiment of the present application;

fig. 11 is a schematic structural diagram of a video rendering system applied to a cloud game according to an exemplary embodiment provided in an embodiment of the present application;

fig. 12 is a schematic structural diagram of a rendering parameter confirmation module according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of a parameter adjustment unit according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a game video rendering module according to an embodiment of the present disclosure;

fig. 15 is a schematic structural diagram of a video rendering system applied to a cloud game according to another exemplary embodiment provided in an embodiment of the present application;

fig. 16 is a schematic flowchart of another video rendering method applied to a cloud game according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of a server according to an embodiment of the present application.

Description of reference numerals: 1. the video rendering system is applied to the cloud game; 11. an equipment information acquisition module; 12. a rendering parameter confirmation module; 13. a game video rendering module; 14. a game picture coding module; 15. a display request judging module; 16. a rendering time judgment module; 121. a parameter comparison unit; 122. a parameter adjustment unit; 123. a parameter confirmation unit; 131. a picture coding unit; 132. a picture storage unit; 133. a picture transmission unit; 1221. a proportion determination unit; 1222. a parameter scaling unit; 1000. a server; 1001. a processor; 1002. a communication bus; 1003. a user interface; 1004. a network interface; 1005. A memory.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

In the description of the embodiments of the present application, the words "exemplary," "for example," or "for instance" are used to indicate instances, or illustrations. Any embodiment or design described herein as "exemplary," "e.g.," or "e.g.," is not to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary," "such as," or "for example" are intended to present relevant concepts in a concrete fashion.

In the description of the embodiments of the present application, the term "and/or" is only one kind of association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, B exists alone, and A and B exist at the same time. In addition, the term "plurality" means two or more unless otherwise specified. For example, the plurality of systems refers to two or more systems, and the plurality of screen terminals refers to two or more screen terminals. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the indicated technical feature. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically stated.

The present application will be described in detail with reference to specific examples.

Referring to fig. 1, a system architecture diagram of a video rendering method applied to a cloud game according to an embodiment of the present invention includes a user terminal and a cloud server, where the user terminal refers to an interactive device between the cloud server and a user, and the user terminal is generally located at a convenient place where the user terminal can work with a remote computer by using a communication facility, and is mainly formed by combining a communication interface control device and a dedicated or selected input/output device. A system in which a plurality of distributed terminals are connected to a computer via a communication facility becomes an online system. While typical cloud gaming user terminals include, but are not limited to: android (Android) system devices, mobile operating system (iOS) devices developed by apple, Personal Computer (PC), World Wide Web (Web) devices, Virtual Reality (VR) devices, Augmented Reality (AR) devices.

The cloud server is a simple and efficient computing service which is safe and reliable and has elastically telescopic processing capacity. When the cloud game is established, the most basic content cannot be separated, namely the cloud server, the required game picture can be rendered, compressed, coded and packaged and transmitted through the server, and then a game user can contact the real picture of the game on a user terminal.

The cloud service is provided with a video memory and a memory, wherein the video memory is also called a frame buffer and is used for storing rendering data processed or to be extracted by a video card chip. Like the memory of a computer, the video memory is a component for storing processed graphic information, a picture seen on a display screen is composed of pixel points, each pixel point controls the brightness and color of the pixel point by 4-32 or even 64-bit data, the data must be stored through the video memory and then distributed by a display chip and a core processor, and finally, an operation result is converted into a graphic and output to the display. Before graphics rendering modeling is started on a display card, required material and texture data are generally transmitted into a display memory for modeling rendering, the data are transmitted through a bus, a display chip extracts data stored in the display memory through the bus, besides the modeling rendering data, a large amount of vertex data and work instruction streams need to be exchanged, the data are converted into analog signals through a digital-to-analog converter and output to a display end, and finally, an image seen by a user is displayed.

The memory is used for storing the coding file of the coded game picture.

The cloud game is an online game technology based on cloud computing, and is based on an operation mode of the cloud game, game operation and a cloud server, and a game picture after rendering is processed by the cloud server through coding and the like and then is transmitted to a user through a network. Based on the mode, the game equipment of the user terminal can only need basic audio and video media decompression capacity without high-end hardware resources such as a high-end processor, a display card and the like. Therefore, the cloud game is free from the dependence of a user terminal on hardware, and compared with the traditional game mode, the cloud game can greatly reduce the equipment cost of a player for playing the game. For the cloud server, only the performance of the cloud server needs to be improved without developing a new host, and for the user player, higher image quality can be obtained without purchasing a high-performance computer.

A video encoder refers to a program or device capable of compressing digital video.

Video decoder refers to a program or device capable of decompressing digital video.

When the cloud game is operated, the user terminal sends a game picture display request to the cloud server, and the cloud server obtains the equipment parameter information of the user terminal after receiving the game picture display request. And the cloud server performs data processing on the acquired equipment parameter information and sends the equipment parameter information to the video memory to render the game picture. And compressing and coding the rendered game picture through an encoder, and then sending the game picture to a memory for storage. And the transmitting terminal of the cloud server reads the game picture coded in the storage, and sends the game picture to a decoder of the user terminal for decoding and displaying.

In one embodiment, as shown in fig. 2, a flow diagram of a video rendering method applied to a cloud game is specifically provided, and the method is mainly applied to a cloud server, can also be implemented by relying on a computer program, can be implemented by relying on a single chip, and can also be run on a video rendering system applied to a cloud game based on a von neumann system. The computer program may be integrated into the application or may run as a separate tool-like application.

Specifically, the video rendering method applied to the cloud game includes:

step 101: receiving a game picture display request sent by a user terminal when a cloud game is operated, and acquiring equipment parameter information of the user terminal, wherein the equipment parameter information comprises a first rendering parameter locally supported by the user terminal.

The request is an instruction and a command for instructing the cloud server to work, and may be understood as making a control code for performing a certain operation or function implementation. The game screen display request in the embodiment of the application may be understood as that, when the cloud game is just started to be run, the user terminal instructs the cloud server to execute a code for acquiring the device parameter information of the user terminal, and the cloud server may acquire the device parameter information of the user terminal by executing the code.

The device parameter information, which mainly includes a device hardware configuration (the first rendering parameter) and/or a device network configuration, may be: the integrated circuit card identification code, namely the serial number of the SIM card, namely the ICCID code (card number of the SIM card) of the SIM card inserted into the mobile phone, is composed of 20 digits and is equivalent to an identity card of the mobile phone number; MAC address, network card hardware address, 48 bits (6 bytes) in length, usually expressed as 12 16-ary numbers; the equipment identification code is a 64-bit random integer randomly generated when the system is started for the first time and is stored in a system setting file in a 16-system character string mode; the hardware serial number, a device hardware identifier, is determined by hardware information, written into system attributes at operating system boot time, and has no fixed format.

The first rendering parameters mainly comprise parameters such as the number of displays to be rendered, the resolution of equipment, the refresh rate range of the equipment, the rendering proportion and the like.

The number of the displays to be rendered refers to that a plurality of user terminals send game screen display requests at the same time, and in addition, one terminal can be provided with a plurality of displays, so that each display needs to be matched with different game screens.

The resolution of the equipment is a hardware parameter, which indicates that a plurality of minimum display units are arranged in a display panel of a display screen, and the physical resolution is fixed and unchangeable and is not adjustable.

The rendering resolution (digital image resolution of a video card or video output device) is understood to be "soft resolution" if the physical resolution of the device is understood to be "hard resolution". Usually, the rendering resolution and the physical resolution of the user terminal are kept consistent, in which case each pixel point of the digital image has a corresponding physical display unit on the display to display, i.e. point-to-point display.

The device refresh rate range, which is a characteristic of computer displays and projection devices, defines the frequency and ability of the device to redraw an entire visible display on the screen every second. The refresh rate, also referred to as the vertical refresh or vertical scan rate, is in hertz and may vary depending on the architecture of the display device. The display refresh rate ranges from device to device, with typical computer displays ranging from 60hz to 100 hz depending on their screen size, while for LCD displays the refresh rate is higher due to their advanced technology. Refresh rates below 70 hz are considered negative rates because they can cause screen flicker.

The rendering ratio generally refers to a screen ratio set by running a cloud game, and the higher the rendering ratio is, the higher the performance requirement on the user terminal is. If the proper screen scale is not selected, a stuck may result.

For example, as shown in fig. 3, fig. 3 is a schematic view of a scenario in which a plurality of user terminals simultaneously send game screen display requests, and after the plurality of user terminals simultaneously send the game screen display requests, the game screen display requests are transmitted to a cloud server corresponding to a game through a resource manager at a network side. And after receiving the plurality of requests, the cloud server respectively acquires the equipment parameters of each terminal and calculates the equipment parameters, and then acquires the calculation results to respectively render the game pictures, so that the corresponding pictures are transmitted to the corresponding terminals.

For example, as shown in fig. 4, fig. 4 is a schematic view of a scenario in which a user terminal selects a rendering mode, when the user terminal sends a game screen display request to a cloud server, the cloud server sends an instruction for acquiring user equipment parameter information, and the user terminal pops up a "rendering mode selection" interface box after receiving the instruction, where the rendering mode selection interface box includes a "system automatic matching" mode and a "manual setting" mode.

For another example, as shown in fig. 5, fig. 5 is a schematic view of a UI design setting screen scene applied during rendering by a user terminal, and when the user terminal selects a "manual setting" mode, rendering parameters such as resolution, refresh rate, and rendering ratio may be set according to a prompt or by customization.

Specifically, when a user starts a game, the user terminal sends a game picture display request to the cloud server, after the game picture display request is received by the cloud server, the cloud server sends a command for acquiring user equipment parameter information and acts on the user terminal, the user terminal pops up a rendering mode selection interface frame after receiving the command, the rendering mode selection interface frame comprises a system automatic matching mode or a manual setting mode for the user to select, if the user selects the system automatic matching mode, the user terminal sends own equipment parameter information to the cloud server, and if the user selects the manual setting mode, the rendering parameter information such as resolution, refresh rate and rendering ratio can be set according to prompt or self-definition and sent to the cloud server.

Step 102: and comparing the first rendering parameter with a second rendering parameter which is supported by the cloud server to the maximum extent, and determining an actual rendering parameter.

The second rendering parameter mainly refers to the range of the rendering parameter supported by the server local end when the user selects the game. The second rendering parameters correspond to the first rendering parameters, and include rendering parameters such as resolution, refresh rate, and rendering ratio.

The second rendering parameter may be a range value, and the second rendering parameter that is maximally supported by the cloud service refers to a maximum value within the range.

The actual rendering parameters refer to rendering parameters most suitable for the user terminal.

Specifically, the cloud server compares a second rendering parameter supported by the cloud server with a first rendering parameter supported by the user terminal, and adjusts the first rendering parameter based on a comparison result to obtain an actual rendering parameter most suitable for the user terminal.

Step 103: rendering the game picture based on the actual rendering parameters to obtain the rendered game picture.

Rendering (Render) is the last step of computer animation (CG), and is also the stage of the 3D scene that the image finally conforms to. Cloud rendering in the cloud game is similar to conventional cloud computing, namely, the 3D program is placed in a remote server for rendering, the user terminal clicks a 'cloud rendering' button through global wide area network software or directly in the local 3D program and accesses access resources through high-speed internet, an instruction is sent from the user terminal, the server executes a corresponding rendering task according to the instruction, and a rendering result picture is transmitted back to the user terminal for displaying.

Specifically, cloud game rendering is a process of visualization and effect presentation, a cloud server renders a game picture in real time based on actual rendering parameters, and needs to complete one frame in about 23 milliseconds, and the real-time rendering refers to drawing three-dimensional data into two-dimensional bitmaps according to a graphics algorithm and displaying the bitmaps in real time. The essence is that the real-time calculation and output of image data require rendering one picture in a short time and displaying it, and simultaneously rendering and displaying the next picture.

Step 104: and coding the rendered game picture, and sending the game picture to the user terminal so as to enable the user terminal to decode and display the game picture.

Encoding refers to a way of converting a file in an original video format into a file in another video format by a compression technique. Because the amount of the data of the directly transmitted original picture is huge, more bandwidth resources are occupied, and in order to save the bandwidth, the encoder compresses the game picture before transmission, so that the data of the picture is reduced and the picture becomes a uniform coding format.

Decoding refers to a process of restoring and decoding an encoded game picture by a specific algorithm, or a process of converting an electric pulse signal into information and data represented by the electric pulse signal. Therefore, a large amount of bandwidth resources are saved, the transmitted data are small, and the influence caused by network jitter is reduced.

Specifically, the video memory in the cloud server encodes the rendered game picture, transmits the encoded file to the user terminal, and the user terminal receives the encoded file, decodes the encoded file, and displays the decoded file on the display screen.

Referring to fig. 6, fig. 6 is a schematic flowchart illustrating a video rendering method applied to a cloud game according to another embodiment of the present disclosure.

Step 201: and receiving a game picture display request sent by the user terminal when the cloud game is operated.

The user terminal may be any one of an Android (Android) system device, a mobile operating system (IOS) device developed by apple inc, a Personal Computer (PC), a World Wide Web (Web) device, a Virtual Reality (VR) device, an Augmented Reality (AR) device, and the like, which can be connected to a display screen and can be connected to a network. For example, the screen state information of the mobile phone includes a landscape state and a portrait state, and the screen state information of the computer includes a full screen state or a zoom state. The user terminal sends the state information of the display screen of the user terminal to the cloud server in addition to the instruction of sending the game picture display request.

For example, as shown in fig. 7a and 7b, fig. 7a is a scene diagram in a vertical screen game state of the user mobile terminal, and fig. 7b is a scene diagram in a horizontal screen game state of the user mobile terminal. When the game mode is converted from the landscape screen state to the portrait screen state, the aspect ratio affects the experience of the player on the cloud game, and it is very important to select the appropriate aspect ratio. The aspect ratio describes the width and height of the screen or image and consists of two numbers divided by colon, the first number representing the width of the image and the second number representing the height of the image, most games are designed with 1920 x 1080 standard design resolution. While this resolution has an aspect ratio of 16:9, other popular aspect ratios are 4:3, 16:10, etc.

For another example, as shown in fig. 8a and 8b, fig. 8a is a scene diagram in a full-screen game state of the user PC terminal, and fig. 8b is a scene diagram in a zoom game state of the user PC terminal.

As discussed above, the rendering resolution and the physical resolution of the user terminal are generally consistent, in which case each pixel of the digital image has a corresponding physical display unit on the display for displaying, i.e. point-to-point displaying. However, when the cloud game is executed on the PC side, the size of the game window can be adjusted, which can be understood as enlarging or reducing the game screen, which is the case when the game rendering resolution and the physical resolution of the display are not consistent. In zoom-in mode, the rendering resolution is less than the physical resolution; in the zoom-out mode, the rendering resolution is greater than the physical resolution. If the user selects to set the resolution manually, and the distribution ratio is not automatically matched and set by using the system, if the set resolution is smaller than the physical distribution ratio of the display, the display picture is stretched, deformed and blurred. Some cloud games can set rendering resolution, even if the rendering resolution of the system is high, the interior of the game can still set low rendering resolution, so that the game can run more smoothly on a user terminal configured with a game bottom, and the influence is that the picture can be blurred and deformed.

In the zoom-in mode, the rendering resolution may be smaller than the screen resolution, for example, as shown in fig. 9, fig. 9 is a proportional comparison graph of the logical resolution, the rendering resolution, and the physical allocation rate of the user terminal screen, and assuming that the logical resolution is 375 × 667, the rendering resolution is 1125 × 2201, and is still larger than the screen resolution 1080 × 1920, the screen is not blurred. The content of the screen display and the content of the reduced window are the same at this time, but the physical size of the element is larger.

Step 202: and judging whether the game picture display request is a first request.

Step 203: and if the game picture display request is a first request, acquiring equipment parameter information from the user terminal.

Step 204: and if the game picture display request is not a first request, searching the equipment parameter information of the user terminal in a cache.

Specifically, the cloud server establishes a module list in the cache, the module list refers to a cache space opening area of the cloud server to store data, and in the process of configuring the actual rendering parameters of the user terminal, the device identification codes and the actual rendering parameters of the user terminal are stored in the module list so as to be convenient to call and read at any time. After a user terminal sends a game picture display request, a cloud server searches an equipment identification code of the user terminal in a module list so as to judge whether the game picture display request is a first request, if so, actual rendering parameters are configured for the user terminal according to the following steps, and if not, the actual rendering parameters in the module list are directly called to perform video rendering on the user terminal.

Step 205: and comparing whether the second rendering parameter exceeds the first rendering parameter or not, wherein the rendering parameter is resolution.

Specifically, the first rendering parameter refers to parameter information supported by the user terminal, the second rendering parameter refers to parameter information supported by the cloud server, and if the first rendering parameter refers to the device resolution, the rendering parameters may be understood as range values. Comparing whether the second rendering parameter exceeds the first rendering parameter may be understood as a comparison of the two sets.

For example, as shown in fig. 10a and 10B, a represents a first rendering parameter, B represents a second rendering parameter, fig. 10a is a schematic diagram of a case where the first rendering parameter is included in the second rendering parameter, and fig. 10B is a schematic diagram of a cross of the first rendering parameter and the second rendering parameter. When both of the above-described cases occur, it is indicated that the second rendering parameter range exceeds the range of the first rendering parameter.

Step 206: and if the second rendering parameter does not exceed the first rendering parameter, performing scaling operation on the first rendering parameter to determine an actual rendering parameter.

Step 207: and determining a scaling ratio, wherein the first rendering parameter is less than or equal to the second rendering parameter after being scaled according to the scaling ratio, and multiplying the first rendering parameter by the scaling ratio to obtain the actual rendering parameter.

Step 208: and if the second rendering parameter exceeds the first rendering parameter, determining the first rendering parameter as an actual rendering parameter.

Specifically, when the rendering parameter is the resolution, whether a second rendering parameter range exceeds a first rendering parameter range is judged, and if the second rendering parameter does not exceed the first rendering parameter, the first rendering parameter is scaled to determine an actual rendering parameter. And determining a scaling ratio, wherein the first rendering parameter is less than or equal to the second rendering parameter after being scaled according to the scaling ratio, and multiplying the first rendering parameter by the scaling ratio to obtain the actual rendering parameter. And if the second rendering parameter exceeds the first rendering parameter, determining the first rendering parameter as an actual rendering parameter.

For example, the client supports 1440 × 2560 (i.e., 1440 vertical pixels and 2560 horizontal pixels) resolution rendered images, the server supports 1920 × 1080 resolution images, and the server compresses the resolution of the client, because the horizontal pixels in the resolution supported by the client are larger than the vertical pixels, the horizontal pixels are compressed to 1080 horizontal pixels supported by the server, and the vertical pixels are compressed in the same proportion as the horizontal pixels, so that the actual resolution of the compressed rendering is 1920 × 1080. If the resolution supported by the client is 3000 × 1440 and the resolution supported by the server is 1920 × 1080, the actual resolution after compression is 1920 × 921.

Step 209: and the rendering parameter is a refresh rate, and whether the second rendering parameter exceeds the first rendering parameter is compared.

Step 210: and if the second rendering parameter exceeds the first rendering parameter, determining the first rendering parameter as an actual rendering parameter.

Step 211: and if the second rendering parameter does not exceed the first rendering parameter, determining the second rendering parameter as an actual rendering parameter.

Specifically, when the rendering parameter is the refresh rate, it is determined whether the second rendering parameter range exceeds the first rendering parameter range, if the second rendering parameter does not exceed the first rendering parameter range, the first rendering parameter is directly determined as the actual rendering parameter, and if the second rendering parameter exceeds the first rendering parameter, the second rendering parameter is directly determined as the actual rendering parameter.

For example, if the compression rate supported by the client is 90HZ, and the highest supported compression rate of the server is 60HZ, the actual compression rate is 60 HZ. If the compression rate supported by the client is 30HZ and the highest supported compression rate of the server is 60HZ, the actual compression rate is 30 HZ.

Step 212: rendering the game picture based on the actual rendering parameters.

Step 213: and coding the rendered game picture in the video memory to obtain a coded picture.

Step 214: and storing the coded picture into the memory, and sending the coded picture in the memory to the user terminal so as to enable the user terminal to decode and display the coded picture.

Specifically, a rendering module and an encoder module are embedded in the display memory, the rendering module in the display memory directly receives the input of the game picture, the rendered game picture is directly transmitted to the encoder module with a compatible format, the encoder module encodes the rendered game picture to obtain a game picture encoded bit stream, and the encoded bit stream is transmitted to the user terminal or stored in a memory of the encoded bit stream, or transmitted to the user terminal through the memory.

The following are embodiments of the system of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the system of the present application, please refer to the embodiments of the application method.

Referring to fig. 11, a schematic structural diagram of a video rendering system applied to a cloud game according to an exemplary embodiment of the present application is shown. The video rendering system applied to the cloud game can be realized into all or part of the system through software, hardware or a combination of the software and the hardware. The video rendering system 1 applied to the cloud game includes an apparatus information obtaining module 11, a rendering parameter confirmation module 12, a game video rendering module 13, and a game screen encoding module 14.

The device information acquiring module 11 is configured to receive a game screen display request sent by a user terminal when a cloud game is executed, and acquire device parameters of the user terminal, where the device parameter information includes a first rendering parameter locally supported by the user terminal.

And the rendering parameter confirming module 12 is configured to compare the first rendering parameter with a second rendering parameter that is maximally supported by the cloud server, and determine an actual rendering parameter.

And the game video rendering module 13 is configured to render the game picture based on the actual rendering parameter to obtain a rendered game picture.

And a game picture coding module 14, configured to code the rendered game picture, and send the game picture to the user terminal, so that the user terminal decodes and displays the game picture.

Optionally, as shown in fig. 12, the rendering parameter confirmation module 12 includes a parameter comparison unit 121, a parameter adjustment unit 122, and a parameter confirmation unit 123.

A parameter comparing unit 121, configured to compare whether the second rendering parameter exceeds the first rendering parameter.

The parameter adjusting unit 122, if the second rendering parameter does not exceed the first rendering parameter, performs a scaling operation on the first rendering parameter to determine an actual rendering parameter.

The parameter determining unit 123 determines that the first rendering parameter is an actual rendering parameter if the second rendering parameter exceeds the first rendering parameter.

Optionally, as shown in fig. 13, the parameter adjusting unit 122 includes a scale determining unit 1221 and a parameter scaling unit 1222.

A scale determining unit 1221, configured to determine a scaling ratio, where the first rendering parameter is smaller than or equal to the second rendering parameter after being scaled according to the scaling ratio.

A parameter scaling unit 1222, configured to multiply the first rendering parameter by the scaling ratio to obtain the actual rendering parameter.

Alternatively, as shown in fig. 14, the game video rendering module 13 includes a picture encoding unit 131, a picture storage unit 132, and a picture transmission unit 133.

And the picture coding unit 131 is configured to code the rendered game picture in the video memory to obtain a coded picture.

A picture storage unit 132 for storing the coded picture in the memory.

A picture sending unit 133, configured to send the coded picture in the memory to the user terminal, so that the user terminal decodes and displays the coded picture.

Optionally, as shown in fig. 15, the video rendering system 1 applied to the cloud game further includes:

a display request judging module 15, configured to receive a game screen display request sent by a user terminal when a cloud game is running; judging whether the game picture display request is a first request or not; if the game picture display request is a first request, acquiring equipment parameter information from the user terminal; and if the game picture display request is not a first request, searching the equipment parameter information of the user terminal in a cache.

It should be noted that, when the video rendering system applied to the cloud game provided by the above embodiment executes the video rendering method applied to the cloud game, the division of the functional modules is merely used as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the video rendering system applied to the cloud game and the video rendering method applied to the cloud game provided by the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which is not described herein again.

In one embodiment, as shown in fig. 16, a video rendering method applied to a cloud game is specifically proposed, and the method is mainly applied to a user terminal, can be implemented by relying on a computer program, can be implemented by relying on a single chip microcomputer, and can also run on a video rendering system applied to the cloud game based on a von neumann system. The computer program may be integrated into the application or may run as a separate tool-like application.

Specifically, the video rendering method applied to the cloud game includes:

step 301: the method comprises the steps of sending a game picture display request to a cloud server when a cloud game is operated so that the cloud server can obtain equipment parameter information of a user terminal, wherein the equipment parameter information comprises a first rendering parameter locally supported by the user terminal, and the cloud server can determine an actual rendering parameter after comparing the first rendering parameter with a second rendering parameter maximally supported by the cloud server, render a game picture based on the actual rendering parameter, encode the game picture and send the game picture to the user terminal.

Step 302: and receiving the coded game picture sent by the cloud server, and decoding and displaying the game picture.

The above steps can refer to steps 201 to 214, which are not described herein again.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

Please refer to fig. 17, which provides a schematic structural diagram of a server according to an embodiment of the present application. As shown in fig. 17, the server 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002.

The communication bus 1002 is used to implement connection communication among these components.

The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Processor 1001 may include one or more processing cores, among other things. The processor 1001, which is connected to various parts throughout the server 1000 using various interfaces and lines, performs various functions of the server 1000 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005 and calling data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1001, but may be implemented by a single chip.

The Memory 1005 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 may be used to store an instruction, a program, code, a set of codes, or a set of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 1005 may alternatively be at least one memory device located remotely from the processor 1001. As shown in fig. 17, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a video rendering method application applied to a cloud game.

It should be noted that: in the device provided in the foregoing embodiment, when the functions of the device are implemented, only the division of each functional module is illustrated, and in practical applications, the functions may be distributed by different functional modules as needed, that is, the internal structure of the device may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.

In the server 1000 shown in fig. 17, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; and the processor 1001 may be configured to invoke an application program stored in the memory 1005 for a video rendering method applied to a cloud game, which when executed by one or more processors causes the server to perform the method as described in one or more of the above embodiments.

A server-readable storage medium having instructions stored thereon. When executed by one or more processors, cause a server to perform a method as described in one or more of the above embodiments.

It is clear to a person skilled in the art that the solution of the present application can be implemented by means of software and/or hardware. The "unit" and "module" in this specification refer to software and/or hardware that can perform a specific function independently or in cooperation with other components, where the hardware may be, for example, a Field-ProgrammaBLE Gate Array (FPGA), an Integrated Circuit (IC), or the like.

It should be noted that for simplicity of description, the above-mentioned embodiments of the method are described as a series of acts, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art will recognize that the embodiments described in this specification are preferred embodiments and that acts or modules referred to are not necessarily required for this application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some service interfaces, devices or units, and may be an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: various media capable of storing program codes, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The above description is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure should not be limited thereby. That is, all equivalent changes and modifications made in accordance with the teachings of the present disclosure are intended to be included within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains.

Claims

1. A video rendering method applied to a cloud game is applied to a cloud server, and is characterized by comprising the following steps:

2. The video rendering method applied to the cloud game of claim 1, wherein the rendering parameter is a resolution, and the comparing the first rendering parameter with a second rendering parameter maximally supported by the cloud server to determine an actual rendering parameter comprises:

3. The video rendering method applied to the cloud game of claim 1, wherein the rendering parameter is a refresh rate, and the comparing the first rendering parameter with a second rendering parameter maximally supported by the cloud server to determine an actual rendering parameter comprises:

4. The video rendering method applied to the cloud game of claim 2, wherein the scaling the first rendering parameter to determine an actual rendering parameter comprises:

determining a scaling ratio, wherein the first rendering parameter is smaller than or equal to the second rendering parameter after being scaled according to the scaling ratio;

5. The video rendering method applied to the cloud game of claim 1, wherein the cloud server includes a video memory and a memory, and the encoding of the rendered game screen and the sending of the game screen to the user terminal so that the user terminal can decode and display the game screen includes:

coding the rendered game picture in the video memory to obtain a coded picture;

6. The video rendering method applied to the cloud game of claim 1, wherein the receiving a game screen display request sent by a user terminal when the cloud game is executed, and acquiring device parameter information of the user terminal comprises:

judging whether the game picture display request is a first request or not;

7. A video rendering method applied to a cloud game is applied to a user terminal, and is characterized by comprising the following steps:

8. A video rendering system applied to a cloud game, comprising:

the cloud game system comprises an equipment information acquisition module, a cloud game processing module and a game processing module, wherein the equipment information acquisition module is used for receiving a game picture display request sent by a user terminal when a cloud game is operated and acquiring equipment parameters of the user terminal, and the equipment parameter information comprises a first rendering parameter locally supported by the user terminal;

and the game picture coding module is used for coding the rendered game picture and sending the game picture to the user terminal so as to enable the user terminal to decode and display the game picture.

9. A computer-readable storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to perform the method according to any of claims 1 to 7.

10. A server comprising a processor, a memory for storing instructions, and a transceiver for communicating with other devices, the processor being configured to execute the instructions stored in the memory to cause the server to perform the method of any one of claims 1 to 7.