CN118840520A - Camera distortion simulation method and device and electronic equipment - Google Patents

Abstract

The present application provides a camera distortion simulation method, device and electronic device. In the present application, by using N virtual perspective cameras in the 3D rendering field to render undistorted images for the same 3D scene at the same time, and based on a distorted image and the field of view of the above N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from the N undistorted images, rather than directly processing the existing undistorted images. Compared with the existing conventional image post-processing methods, this can avoid blurring of the image area with obvious distortion during the camera distortion simulation process.

Description

Camera distortion simulation method, device and electronic equipment

Technical Field

The present application relates to image processing, and in particular to a camera distortion simulation method, device and electronic device.

Background Art

The images captured by real cameras are often distorted due to the inconsistency between the magnification of the central area of the lens and the edge area. This distortion is more obvious in images captured by wide-angle lenses or fisheye lenses.

At present, the common method for simulating real camera distortion is the image post-processing method, specifically: a distortion-free image taken by a normal perspective camera is distorted to obtain a distorted image for simulation. However, when this method simulates images with obvious (severe) distortion, such as those collected by the wide-angle lens and fisheye lens mentioned above, if the distortion-free image is distorted so that the central area of the image has obvious (severe) distortion, it may cause the central area of the entire image to be too blurred, affecting the image distortion simulation effect.

Summary of the invention

The present application provides a camera distortion simulation method, device and electronic device to avoid blurring of image areas with obvious distortion during the camera distortion simulation process.

The present application provides a camera distortion simulation method, the method comprising:

Obtaining N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras at the same time for the same 3D scene; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles;

Obtain a distorted image;

Based on the N undistorted images and the distorted images, a distorted simulated image is generated; each simulated pixel point in the distorted simulated image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

The present application also provides a camera distortion simulation device, which includes:

An obtaining unit is used to obtain N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras for the same 3D scene at the same time; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; and,

Obtain a distorted image;

A distortion simulation unit is used to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulated pixel point in the distortion simulation image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

The embodiment of the present application also provides an electronic device. The electronic device includes: a processor and a machine-readable storage medium;

The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

The processor is used to execute machine executable instructions to implement the steps of the above disclosed method.

It can be seen from the above technical solutions that in the present application, by using N virtual perspective cameras in the 3D rendering field to render undistorted images for the same 3D scene at the same time, and based on a distorted image and the field of view of the N virtual perspective cameras, simulated pixel points in the distorted simulated image are sampled from the N undistorted images, rather than directly processing the existing undistorted images. Compared with the existing conventional image post-processing methods, this can avoid blurring of image areas with obvious distortion during camera distortion simulation;

Furthermore, in this embodiment, based on a distorted image and the field of view of the above-mentioned N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from N undistorted images, which is equivalent to splicing and fusing the pixel points in N undistorted images in the real-time rendering process, which can completely eliminate the blurring of the central area of the distorted simulation image caused by the existing post-processing of a ready-made image.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

FIG1 is a flow chart of a method provided in an embodiment of the present application;

FIG2 is a schematic diagram of a distortion-free image provided by an embodiment of the present application;

FIG3 is a schematic diagram of a distorted image provided by an embodiment of the present application;

FIG4 is a flowchart of generating a distortion simulation image provided by an embodiment of the present application;

FIG5 is a hybrid mask diagram provided by an embodiment of the present application;

FIG6 and FIG7 are simulation effect diagrams provided in the embodiments of the present application;

FIG8 is a structural diagram of a device provided in an embodiment of the present application;

FIG. 9 is a structural diagram of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

See Figure 1, which is a flow chart of a method provided in an embodiment of the present application. The method is applied to electronic devices such as servers running 3D scenes, etc., and is not specifically limited in this embodiment. The process can be applied to camera distortion simulation in the field of real-time rendering.

As an embodiment, this embodiment is applied to a 3D scene. In the 3D scene, this embodiment will create N virtual perspective cameras, where N is greater than 1. For example, N is 2.

In one embodiment, the N virtual perspective cameras are deployed at the same location and the orientations of the lenses of the N virtual perspective cameras meet a set similarity requirement (such as consistent orientation, etc.).

In one embodiment, the N virtual perspective cameras have different field of view angles.

Based on the above description, as shown in FIG1 , the process may include the following steps:

Step 101: Obtain N distortion-free images.

In this embodiment, the N undistorted images include the undistorted images rendered by the N virtual perspective cameras at the same time for the same 3D scene. FIG. 2 shows two undistorted images with N being 2 as an example. As described above, the N virtual perspective cameras have different field of view angles. When N is 2, the field of view angles of the two virtual perspective cameras are also different, for example, one of the field of view angles is 90° and the other is 60°. Since the field of view angles of the N virtual perspective cameras are different, the field of view angles of the N virtual perspective cameras will definitely be different. For ease of description, when N is 2, the undistorted image rendered by one of the two virtual perspective cameras having a larger field of view angle, such as 90°, can be called a wide field of view angle image. Correspondingly, the undistorted image rendered by the other virtual perspective camera can be called a narrow field of view angle image. FIG. 2 shows an example of a wide field of view angle image (referred to as a wide viewing angle image) and a narrow field of view angle image (referred to as a narrow viewing angle image).

It should be noted that, in this embodiment, the image resolution of the undistorted images rendered by the N virtual perspective cameras is the same. Based on the different field of view angles of the above-mentioned N virtual perspective cameras, relatively speaking, the undistorted image rendered by the virtual perspective camera with a larger field of view has a wider scene range, which can include all or part of the area of the undistorted image rendered by the virtual perspective camera with a small field of view. For example, the wide field of view image shown in Figure 2 includes the content of the entire narrow field of view image.

However, at the same image resolution, the undistorted image rendered by a virtual perspective camera with a smaller field of view has richer scene details than that rendered by a virtual perspective camera with a larger field of view.

Based on the above description, when generating the distortion simulation image later, the distortion-free image rendered by the virtual perspective camera at each field of view angle will be sampled adaptively. For details, see step 103, which will not be described here.

Step 102: Obtain a distorted image.

As an embodiment, the distorted image is generated by transforming a normal undistorted image according to a set distortion algorithm such as an OpenCV algorithm. FIG3 shows an example of a distorted image.

Optionally, the image resolution of the distorted image is the same as the image resolution of the N undistorted images.

Alternatively, the image resolution of the distorted image is in a multiple relationship with the image resolution of the N undistorted images (but the aspect ratio of the distorted image here is the same as the aspect ratio of the N undistorted images).

Step 103, based on the N undistorted images and the distorted images, a distorted simulated image is generated, wherein each simulated pixel in the distorted simulated image is based on a mapped pixel in the distorted image that has a mapping relationship with the simulated pixel, and the field of view of the N virtual perspective cameras is sampled from the N undistorted images.

As described above, the characteristics of the undistorted image rendered by the virtual perspective camera with a large field of view and the undistorted image rendered by the virtual perspective camera with a small field of view, when generating the distorted simulation image, this embodiment triggers the generation of the distorted simulation image from an angle with higher pixel accuracy and clearer distortion effect as much as possible. For example, for the overlapping area in N undistorted images, each simulated pixel point in the simulated area having a mapping relationship with the overlapping area in the distorted simulation image is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view, for example, specifically, the pixel information of the mapped pixel point having a mapping relationship with the simulated pixel point is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view as the pixel information of the simulated pixel point.

That is, when there is an overlapping area in N undistorted images, each simulated pixel point in the simulated area in the distorted simulated image that has a mapping relationship with the overlapping area is sampled from the undistorted image rendered by the virtual perspective camera with the smallest field of view. Based on the characteristics of the undistorted image rendered by the virtual perspective camera with a small field of view, such as richer scene details, the accuracy of the simulated pixel points in the simulated area can be made higher, and the distorted effect can be made clearer. The following describes step 103 by way of example, which will not be described here.

At this point, the process shown in Figure 1 is completed.

It can be seen from the process shown in FIG1 that, in this embodiment, by using the non-distorted images rendered by N virtual perspective cameras in the 3D rendering field at the same time for the same 3D scene screen, and based on a distorted image and the field of view of the N virtual perspective cameras, simulated pixel points in the distorted simulated image are sampled from the N non-distorted images, rather than directly processing the existing non-distorted images. Compared with the existing conventional image post-processing method, this can avoid blurring of the image area with obvious distortion during the camera distortion simulation process;

Furthermore, in this embodiment, based on a distorted image and the field of view of the above-mentioned N virtual perspective cameras, simulated pixel points in the distorted simulation image are sampled from N undistorted images, which is equivalent to splicing and fusing the pixel points in N undistorted images in the real-time rendering process, which can completely eliminate the blurring of the central area of the distorted simulation image caused by the existing post-processing of a ready-made image.

The above step 103 is described below:

As an embodiment, in the above step 103, based on the N undistorted images and the distorted images, generating the distorted simulated image includes: performing the following step a for each simulated pixel point in the simulated image template to obtain the distorted simulated image:

Step a: Based on the mapped pixel points in the distorted image that have a mapping relationship with the simulated pixel points and the field of view of N virtual perspective cameras, a sampling image is determined from N undistorted images; based on the pixel parameters of the mapped pixel points in the sampling image that have a mapping relationship with the simulated pixel points, the pixel parameters of the simulated pixel points in the simulated image template are corrected.

As an embodiment, the pixel parameter here is, for example, a pixel value, etc., which is not specifically limited in this embodiment.

To facilitate understanding of step 103, FIG4 illustrates an example flow chart of generating a distortion simulation image provided in an embodiment of the present application.

See Figure 4, which is a flowchart of generating a distortion simulation image provided by an embodiment of the present application. As shown in Figure 4, the process may include the following steps:

Step 401, traverse each untraversed simulated pixel point in the simulated image template in order, and use the traversed simulated pixel point as the current pixel point.

For example, in this embodiment, each untraversed simulated pixel point in the simulated image template is first traversed in order from front to back and from left to right, and the traversed simulated pixel point is used as the current pixel point.

Step 402 : determining a sampled image from the N undistorted images based on the mapped pixel points in the distorted image that have a mapping relationship with the current simulated pixel point and the field of view of the N virtual perspective cameras.

As an embodiment, the pixel information of any pixel in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute. Here, for example, the first specified attribute corresponds to a blue channel, and the second specified attribute corresponds to an alpha channel. Correspondingly, the first attribute value corresponding to the first specified attribute is a blue channel value; the second attribute value corresponding to the second specified attribute is an alpha channel value.

Based on this, this embodiment can determine the sampling image from the above-mentioned N undistorted images with the help of the coordinate information of the mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, the first attribute value corresponding to the first specified attribute and the second attribute value corresponding to the second specified attribute in the pixel information of the mapping pixel point, and the field of view of the N virtual perspective cameras.

For example, based on the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the first attribute value corresponding to the first specified attribute and the second attribute value corresponding to the second specified attribute in the pixel information of the mapped pixel point, a reference decision coefficient is determined. The reference decision coefficient here is used to determine the target sampling decision coefficient for selecting the sampled image, as described below. Then, based on the reference decision coefficient and the field of view of the N virtual perspective cameras, the target sampling decision coefficient is determined; finally, based on the target sampling decision coefficient, the sampled image is determined from the N undistorted images.

As an embodiment, the reference decision coefficient is determined based on the coordinate information of a mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapping pixel point.

For example, the reference decision coefficient is expressed by the following formula: .

in, represents the reference decision coefficient, represents the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A vector representing a first attribute value corresponding to a first specified attribute in the pixel information of the mapped pixel, such as the above-mentioned blue channel value, and a second attribute value corresponding to a second specified attribute, such as the above-mentioned opaque channel value.

As an embodiment, there are many ways to implement the above-mentioned determination of the target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras, such as: determining the trigonometric function value corresponding to the specified trigonometric function of the field of view angle of each virtual perspective camera; determining the target sampling decision coefficient based on the reference decision coefficient and each sampling decision sub-coefficient; any sampling decision sub-coefficient is obtained by performing a specified operation on the trigonometric function values corresponding to the field of view angles of two virtual perspective cameras.

Optionally, the trigonometric function value corresponding to the specified trigonometric function for determining the field of view angle of each virtual perspective camera may be, for example, a tangent value for determining the field of view angle of each virtual perspective camera.

Optionally, the above-mentioned determination of the target sampling decision coefficient based on the reference decision coefficient and each sampling decision sub-coefficient may be, for example, as follows: first input each sampling decision sub-coefficient and the reference decision coefficient into a specified algorithm to determine the target sampling decision coefficient. The specified algorithm may be set according to actual needs, and the present embodiment does not specifically limit it. For example, each sampling decision sub-coefficient is multiplied by its corresponding proportional weight to obtain a first result, the first result obtained is added to the reference decision coefficient to obtain a second result, the second result is divided by the sum of each sampling decision sub-coefficient, and the final result is determined as the target sampling decision coefficient. Here, the proportional weight corresponding to any sampling decision sub-coefficient may be the same, for example, both may be 1/N; or, the proportional weights corresponding to at least two sampling decision sub-coefficients are different, for example, they may be set according to actual needs, and the present embodiment does not specifically limit it.

For ease of understanding, the above-mentioned specified algorithm is illustrated below by taking N as 2 as an example.

For example, optionally, the target sampling decision coefficient may be specifically expressed by the following specified algorithm:

;

in, is the horizontal field of view of the virtual perspective camera with the smallest field of view, It is the horizontal field of view of the virtual perspective camera with the largest field of view. Represents the target sampling decision coefficient. represents the sampling decision sub-coefficient, represents the above reference decision coefficient.

It should be noted that the above description is only an example description using N as 2. The method of specifying the algorithm when N is other values is the same as the description of the specified algorithm above, which will not be repeated here.

As an embodiment, based on the target sampling decision coefficient, a sampling image is determined from N undistorted images, for example: from the decision coefficient range currently corresponding to the field of view angle of each virtual perspective camera that has been obtained, the target decision coefficient range in which the target sampling decision coefficient is located is determined; and the undistorted image rendered by the virtual perspective camera that currently has a field of view angle corresponding to the target decision coefficient range is determined as the sampling image.

For example, taking N as 2, the sampling image can be determined from N undistorted images based on the target sampling decision coefficient: if the target sampling decision coefficient is within the decision coefficient range corresponding to the narrow field of view, such as the range of 0 to 1, then the undistorted image rendered by the virtual perspective camera with the smallest field of view is determined as the above sampling image; otherwise, the undistorted image rendered by the virtual perspective camera with the largest field of view is determined as the above sampling image.

Step 403, based on the pixel parameters of the mapped pixel points in the sampled image that have a mapping relationship with the current simulated pixel point, correct the pixel parameters of the current simulated pixel point in the simulated image template; if the current simulated pixel point is the last simulated pixel point traversed in the simulated image template, the simulated image template at this time is determined as the above-mentioned distorted simulated image, otherwise, return to the step of traversing each untraversed simulated pixel point in the simulated image template in sequence.

As an embodiment, the above-mentioned correction of the pixel parameters of the simulated pixel point in the simulated image template based on the pixel parameters of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point can specifically be: modifying the pixel value of the simulated pixel point in the simulated image template to the pixel value of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point.

At this point, the process shown in FIG. 4 is completed.

The process shown in FIG4 finally realizes the generation of the distorted simulation image based on N undistorted images and distorted images.

In this embodiment, in the distorted simulated image, in order to avoid gaps caused by stitching, for the area to be adjusted starting from the boundary line and along the direction toward the center area of the image, the simulated pixel points in the area to be adjusted can be adjusted based on the undistorted image rendered by each virtual perspective camera to avoid the above-mentioned gap. Here, the boundary line is determined based on adjacent boundary simulated pixel points in the distorted simulated image. The adjacent boundary simulated pixel points are sampled from the undistorted images rendered by two different virtual perspective cameras.

For example, taking N as 2, the boundary line is a boundary line between pixel points in the undistorted image with a wide field of view and pixel points in the undistorted image with a narrow field of view.

As an embodiment, the above-mentioned adjustment of the simulated pixel points in the area to be adjusted may be, for example: for each simulated pixel point in the area to be adjusted, a mapping pixel point mapped by the simulated pixel point is selected from the undistorted image rendered by each virtual perspective camera, the pixel parameters of each mapping pixel point, such as pixel values, are fused, and the fusion result is determined as the pixel parameter, such as pixel value, of the simulated pixel point.

Still taking N as 2 as an example, the above-mentioned area to be adjusted is the area from the boundary line to the center of the image in the distorted simulation image. Of course, if N is greater than 2, the above-mentioned area to be adjusted is the area from the boundary line to another boundary line in the direction toward the center of the image in the distorted simulation image.

As an embodiment, for each simulated pixel point in the above-mentioned area to be adjusted, a mapped pixel point mapped by the simulated pixel point can be selected from the undistorted images rendered by each virtual perspective camera, and the pixel parameters of each mapped pixel point, such as the pixel value, are linearly fused, and the fusion result is determined as the pixel parameter, such as the pixel value, of the simulated pixel point. This step can be achieved by a mixed mask map as shown in FIG5. In the mixed mask map shown in FIG5, pure white indicates that the pixel points in the narrow field of view image are fully used, black indicates that the pixel points in the wide field of view image are fully used, and the gray transition part at the junction is the fusion result after linear fusion of the pixel points at the same position in the narrow field of view image and the wide field of view image.

The method provided by the embodiment of the present application is described above. In order to make the effect of the above method more obvious, Figures 6 and 7 show examples of clear effect pictures after the distortion simulation of a narrow viewing angle image, and clear effect pictures after the distortion simulation of a wide viewing angle image and a narrow viewing angle image. By combining a narrow viewing angle image or a wide viewing angle image and a narrow viewing angle image for distortion simulation, it is possible to avoid blurring of the image area with obvious distortion during the camera distortion simulation process, thereby ensuring that the entire image is clear.

The following is a description of the device provided in the embodiment of the present application:

See Figure 8, which is a structural diagram of a device provided in an embodiment of the present application. As shown in Figure 8, the device may include:

An obtaining unit is used to obtain N distortion-free images; the N distortion-free images include distortion-free images respectively rendered by N virtual perspective cameras for the same 3D scene at the same time; N virtual perspective cameras are deployed in the 3D scene, N is greater than 1, the N virtual perspective cameras are deployed at the same position and the orientations of the lenses of the N virtual perspective cameras meet the set similarity requirements; the N virtual perspective cameras have different field of view angles; and,

Obtain a distorted image;

A distortion simulation unit is used to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulated pixel point in the distortion simulation image is based on a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and the field of view angles of the N virtual perspective cameras are sampled from the N undistorted images.

As an embodiment, the N distortion-free images have the same resolution;

The distorted image is generated according to a set distortion algorithm;

The image resolution of the distorted image is the same as the image resolution of the N undistorted images, or the image resolution of the distorted image is a multiple of the image resolution of the N undistorted images.

As an embodiment, generating a distortion simulation image based on the N distortion-free images and the distortion image includes:

The following steps are performed for each simulated pixel point in the simulated image template to obtain a distorted simulated image: pixel information of any pixel point in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute;

The generating a distortion simulation image based on the N distortion-free images and the distortion image comprises:

For each simulated pixel point in the simulated image template, a reference decision coefficient is determined based on coordinate information of a mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel point; the reference decision coefficient is used to determine a target sampling decision coefficient for selecting a sampling image;

Determine a target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras;

Based on the target sampling decision coefficient, determining a sampling image from the N undistorted images;

The pixel parameters of the simulated pixel point in the simulated image template are corrected based on the pixel parameters of the mapped pixel point in the sampled image that has a mapping relationship with the simulated pixel point.

As an embodiment, the first specified attribute corresponds to a blue channel, and the first attribute value corresponding to the first specified attribute is a blue channel value;

The second specified attribute corresponds to an opaque alpha channel, and the second attribute value corresponding to the second specified attribute is an opaque channel value;

The reference decision coefficient is determined based on the coordinate information of a mapping pixel point in the distorted image that has a mapping relationship with the simulated pixel point, and a vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapping pixel point.

As an embodiment, the reference decision coefficient is expressed by the following formula: ;

in, represents the reference decision coefficient, represents the coordinate information of the mapped pixel point in the distorted image that has a mapping relationship with the simulated pixel point, A vector consisting of a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute in the pixel information of the mapped pixel.

As an embodiment, determining the target sampling decision coefficient based on the reference decision coefficient and the field of view angles of N virtual perspective cameras includes:

Determine trigonometric function values corresponding to the specified trigonometric functions of the field of view angles of each virtual perspective camera;

Based on the reference decision coefficient and each sampling decision sub-coefficient, a target sampling decision coefficient is determined; any sampling decision sub-coefficient is obtained by performing a specified operation on the trigonometric function values corresponding to the field of view angles of any two virtual perspective cameras.

As an embodiment, determining a sampling image from the N distortion-free images based on the target sampling decision coefficient includes:

The target decision coefficient range in which the target sampling decision coefficient is located is determined from the decision coefficient range currently corresponding to the field of view angle of each virtual perspective camera that has been obtained; and the distortion-free image rendered by the virtual perspective camera that currently has a field of view angle corresponding to the target decision coefficient range is determined as the sampling image.

As an embodiment, the distortion simulation unit further determines the area to be adjusted in the distortion simulation image starting from the boundary line and along the direction toward the center area of the image; the boundary line is determined based on adjacent boundary simulation pixel points in the distortion simulation image, and the adjacent boundary simulation pixel points are sampled from the undistorted images rendered by two different virtual perspective cameras respectively;

For each simulated pixel point in the area to be adjusted, a mapping pixel point mapped by the simulated pixel point is selected from the undistorted images rendered by each virtual perspective camera, pixel parameters of each mapping pixel point are fused, and the fusion result is determined as the pixel parameter of the simulated pixel point.

As an embodiment, if N is 2, the area to be adjusted is the area from the boundary line to the center of the image in the distorted simulation image.

At this point, the structural description of the device shown in FIG. 8 is completed.

The embodiment of the present application also provides a hardware structure of the device shown in FIG8. See FIG9, which is a structural diagram of an electronic device provided in the embodiment of the present application. As shown in FIG9, the hardware structure may include: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of the present application.

Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device, which can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A camera distortion simulation method, the method comprising:

obtaining N undistorted images; the N undistorted images comprise undistorted images respectively rendered by N virtual perspective cameras for the same 3D scene picture at the same moment; n virtual perspective cameras are deployed in the 3D scene, N is larger than 1, the N virtual perspective cameras are deployed at the same position, and the directions of the lenses of the N virtual perspective cameras meet the set similarity requirement; the view angles of the N virtual perspective cameras are different;

obtaining a distorted image;

Generating a distortion simulation image based on the N undistorted images and the distorted image; each simulation pixel point in the distorted simulation image is obtained by sampling from the N undistorted images based on a mapping pixel point with a mapping relation with the simulation pixel point in the distorted image and the view angles of the N virtual perspective cameras.

2. The method of claim 1, wherein the resolution of the N undistorted images is the same;

The distorted image is generated according to a set distortion algorithm;

the image resolution of the distorted image is the same as the image resolution of the N undistorted images, or the image resolution of the distorted image has a multiple relationship with the image resolution of the N undistorted images.

3. The method of claim 1, wherein the pixel information of any pixel point in the distorted image includes a first attribute value corresponding to a first specified attribute and a second attribute value corresponding to a second specified attribute;

the generating a distortion simulation image based on the N undistorted images and the distorted image includes:

Determining a reference decision coefficient based on coordinate information of a mapping pixel point with a mapping relation with the simulation pixel point in the distorted image and a first attribute value corresponding to a first designated attribute and a second attribute value corresponding to a second designated attribute in the pixel information of the mapping pixel point for each simulation pixel point in a simulation image template; the reference decision coefficients are used for determining target sampling decision coefficients for selecting a sampling image;

Determining a target sampling decision coefficient based on the reference decision coefficient and the field angles of the N virtual perspective cameras;

determining a sampling image from the N undistorted images based on the target sampling decision coefficient;

and correcting the pixel parameters of the simulation pixel points in the simulation image template based on the pixel parameters of the mapping pixel points with the mapping relation with the simulation pixel points in the sampling image.

4. A method according to claim 3, wherein the first specified attribute corresponds to a Blue channel, and the first attribute value corresponding to the first specified attribute is a Blue channel value;

The second attribute corresponds to an opaque alpha channel, and the second attribute value corresponding to the second attribute is an opaque channel value;

The reference decision coefficient is determined according to the coordinate information of the mapping pixel point with the mapping relation with the simulation pixel point in the distorted image and a vector formed by a first attribute value corresponding to a first designated attribute and a second attribute value corresponding to a second designated attribute in the pixel information of the mapping pixel point.

5. The method according to claim 3 or 4, characterized in that the reference decision coefficient is represented by the following formula:；

wherein, Representing the reference decision coefficients of the set of coefficients,Coordinate information representing a mapped pixel point having a mapping relation with the simulated pixel point in the distorted image,And a vector composed of a first attribute value corresponding to the first designated attribute and a second attribute value corresponding to the second designated attribute in the pixel information representing the mapped pixel point.

6. The method of claim 3, wherein the determining target sampling decision coefficients based on the reference decision coefficients and the field angles of the N virtual perspective cameras comprises:

Determining a trigonometric function value corresponding to a designated trigonometric function of the field angle of each virtual perspective camera;

Determining a target sampling decision coefficient based on the reference decision coefficient and each sampling decision sub-coefficient; any sampling decision sub-coefficient is obtained by carrying out specified operation on trigonometric function values corresponding to the field angles of any two virtual perspective cameras.

7. A method according to claim 3, wherein said determining a sampled image from said N undistorted images based on said target sampling decision coefficient comprises:

Determining a target decision coefficient range in which the target sampling decision coefficient is positioned from the decision coefficient range currently corresponding to the obtained field angle of each virtual perspective camera; and determining an undistorted image rendered by a virtual perspective camera currently having a field angle corresponding to the target decision coefficient range as the sampling image.

8. The method according to claim 1, characterized in that the method further comprises:

determining a region to be adjusted in the distorted simulation image from a boundary line and along a direction towards a central region of the image; the boundary line is determined based on adjacent boundary simulation pixel points in the distortion simulation image, wherein the adjacent boundary simulation pixel points are respectively obtained by sampling undistorted images rendered by two different virtual perspective cameras;

And selecting mapping pixel points mapped by the simulation pixel points from undistorted images rendered by the virtual perspective cameras for each simulation pixel point in the region to be adjusted, fusing pixel parameters of the mapping pixel points, and determining a fusion result as the pixel parameters of the simulation pixel points.

9. A camera distortion simulation apparatus, the apparatus comprising:

An obtaining unit configured to obtain N undistorted images; the N undistorted images comprise undistorted images respectively rendered by N virtual perspective cameras for the same 3D scene picture at the same moment; n virtual perspective cameras are deployed in the 3D scene, N is larger than 1, the N virtual perspective cameras are deployed at the same position, and the directions of the lenses of the N virtual perspective cameras meet the set similarity requirement; the view angles of the N virtual perspective cameras are different; and

Obtaining a distorted image;

a distortion simulation unit configured to generate a distortion simulation image based on the N undistorted images and the distorted image; each simulation pixel point in the distorted simulation image is obtained by sampling from the N undistorted images based on a mapping pixel point with a mapping relation with the simulation pixel point in the distorted image and the view angles of the N virtual perspective cameras.

10. An electronic device, characterized in that, the electronic device includes: a processor and a machine-readable storage medium;

The machine-readable storage medium stores machine-executable instructions executable by the processor;

the processor is configured to execute machine executable instructions to implement the method steps of any one of claims 1-8.