CN115134505B

CN115134505B - Preview picture generation method and device, electronic equipment and storage medium

Info

Publication number: CN115134505B
Application number: CN202110322170.1A
Authority: CN
Inventors: 徐翔; 巩家兴; 白春玉; 郭传鑫
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2021-03-25
Filing date: 2021-03-25
Publication date: 2024-06-21
Anticipated expiration: 2041-03-25
Also published as: CN115134505A

Abstract

The disclosure relates to a preview picture generation method and device, electronic equipment and storage medium. Wherein the method comprises the following steps: under the condition that a first lens is switched to a second lens, determining optical anti-shake offset of a target lens with an optical anti-shake function in the first lens and the second lens in the switching process; performing offset correction on the pre-correction image obtained by shooting the target lens based on the optical anti-shake offset to eliminate the angle of view offset of the pre-correction image due to the optical anti-shake function of the target lens, so as to obtain a corrected image; and generating an excessive frame image in the switching process based on the corrected image, and generating a preview picture corresponding to the switching process based on the excessive frame image and the corrected image.

Description

Preview picture generation method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of image processing, and in particular, to a preview screen generating method and apparatus, an electronic device, and a storage medium.

Background

In the process of image acquisition, if the used lens is switched, the jump of a preview picture is easily caused, and the visual experience of a user is influenced.

In order to solve the above-mentioned problems, the related art proposes a method capable of smoothly transition a preview screen in a shot-switched scene. According to the method, image processing is carried out on the acquired image according to displacement change in the lens switching process so as to obtain an excessive frame image; and the preview picture is excessively smooth in a mode of displaying the excessive frame image in the preview picture corresponding to the shot switching process.

However, most of the lenses in the past have an optical anti-shake function, and the position of the lens can be automatically adjusted to avoid the occurrence of frame shake during shooting without lens switching. When the method is applied to the lens with the optical anti-shake function, the position of the lens is adjusted while the lens is switched, so that deviation exists between the determined lens position change process and the actual change, and further the preview picture cannot be smoothly and excessively caused by the excessive frame image obtained based on the position change process.

Disclosure of Invention

The present disclosure provides a preview display method and apparatus, an electronic device, and a storage medium, which can smoothly transition a preview when a lens having an optical anti-shake function is switched.

According to a first aspect of the present disclosure, there is provided a preview screen generating method, including:

under the condition that a first lens is switched to a second lens, determining optical anti-shake offset of a target lens with an optical anti-shake function in the first lens and the second lens in the switching process;

performing offset correction on the pre-correction image obtained by shooting the target lens based on the optical anti-shake offset to eliminate the angle of view offset of the pre-correction image due to the optical anti-shake function of the target lens, so as to obtain a corrected image;

and generating an excessive frame image in the switching process based on the corrected image, and generating a preview picture corresponding to the switching process based on the excessive frame image and the corrected image.

According to a second aspect of the present disclosure, there is provided a preview screen generating apparatus including:

A determination unit that determines an optical anti-shake offset of a target lens having an optical anti-shake function in a first lens and a second lens in a switching process in a case of switching from the first lens to the second lens;

A correction unit that performs offset correction on a pre-correction image captured by the target lens based on the optical anti-shake offset amount to eliminate a view angle offset of the pre-correction image due to an optical anti-shake function of the target lens, and obtains a corrected image;

And a generation unit that generates an excessive frame image in the switching process based on the corrected image, and generates a preview screen corresponding to the switching process based on the excessive frame image and the corrected image.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor implements the method of the first aspect by executing the executable instructions.

According to a fourth aspect of the present disclosure there is provided a computer readable storage medium having stored thereon computer instructions which when executed by a processor perform the steps of the method according to the first aspect.

In the technical scheme of the disclosure, under the condition that the first lens is switched to the second lens, the optical anti-shake offset of the target lens with the optical anti-shake function is acquired, and the offset correction is performed on the image shot by the target lens based on the acquired optical anti-shake offset, so that the angle of view offset of the target lens due to the optical anti-shake function is eliminated, and the corrected image is further obtained. On this basis, an excessive frame image corresponding to the switching process can be generated based on the corrected image, and a corresponding preview screen can be generated based on the excessive frame image.

It should be appreciated that the optical anti-shake function of the lens can effectively avoid the frame shake caused by the device shake without switching the lens. However, in the process of switching the lenses, the optical anti-shake function may misconsider the lens movement caused by the lens switching as the device shake, so as to adjust the position of the lens. It is easy to understand that if there is no adjustment of optical anti-shake, the positions of the main optical axes of the two lenses are consistent before and after lens switching, and no shift of the angle of view occurs; when the lens has an optical anti-shake function, the optical anti-shake adjusts the positions of the lenses, which inevitably leads to inconsistent positions of the main optical axes of the two lenses, and further leads to the deviation of the angle of view. It can be seen that, in this scenario, if the influence of the optical anti-shake on the captured images is not considered, a certain angular displacement occurs in the images captured by the two lenses respectively compared with each other, and then, the excessive frame image generated based on at least one of the two images also causes the angular displacement, which eventually results in the occurrence of a frame shake in the generated preview image.

In the technical scheme of the disclosure, the optical anti-shake offset of the target lens with the optical anti-shake function is preferentially determined, and offset correction is performed on the photographed image based on the optical anti-shake offset, so as to eliminate the angle of view offset of the photographed image to the target lens due to the optical anti-shake function, and obtain a corrected image. And generating an excessive frame image based on the corrected image. In other words, the technical scheme of the present disclosure can eliminate the influence of optical anti-shake on the image by a post-processing mode, and avoid the problem that the image obtained by two-lens shooting has a shift in angle of view due to adjustment of optical anti-shake. Obviously, because the image shot by the target lens does not have the angle of view deviation, compared with the image shot by the target lens and the image shot by the target lens, the generated excessive frame image does not have the angle of view deviation, so that the preview image generated by the target lens and the excessive frame image does not generate the condition of image shake, and smooth transition in the lens switching process is realized.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1A is a schematic view of a shot cut shown in an exemplary embodiment of the present disclosure;

FIG. 1B is another shot cut schematic diagram illustrating an exemplary embodiment of the present disclosure;

FIG. 2 is a flow chart of a preview screen presentation method according to an exemplary embodiment of the present disclosure;

FIG. 3 is a flowchart of another preview screen presentation method shown in an exemplary embodiment of the present disclosure;

FIG. 4 is a block diagram of a preview screen presentation apparatus according to an exemplary embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" depending on the context.

In order to solve the above-mentioned problems, the related art proposes a method capable of smoothly transition a preview screen in a shot-switched scene. According to the method, image processing is carried out on the acquired image according to displacement change in the lens switching process so as to obtain an excessive frame image; the frame content in the over-frame image is substantially identical to the first image captured through the first lens and the second image captured through the second lens, but the field angle size (or zoom factor) is in between. Therefore, when the preview screen is displayed, the over-frame image may be used as an over-image of the first image and the second image, so that the preview screen is excessively smoothed.

However, most of the lenses in the past have an optical anti-shake function, and the positions of the lenses can be automatically adjusted according to the shake of the device under the condition that the used lenses are kept unchanged, so as to avoid the image shake caused by the shake of the device in the image shooting process.

When the method is applied to a lens with an optical anti-shake function, if the lens is switched, the optical anti-shake function (actually detecting whether the lens is displaced or not, if so, the device is considered to shake, and then the lens is reversely adjusted) can mistakenly consider the switching of the lens as the shake of the device, and then the position of the lens is automatically adjusted. In other words, the lens is also adjusted in position by the optical anti-shake device at the same time as the lens is switched. Specifically, due to the adjustment of the optical anti-shake function to the lens position, the positions of the main optical axes of the two lenses are inconsistent before and after lens switching, so that the angle of view is also deviated between the two captured images. On this basis, the generated excessive frame image also has a field angle offset from the two images, eventually leading to a problem of picture shake in the preview picture generated based on the image obtained by the two-shot photographing and the excessive frame image.

The reason for generating the picture jitter is illustrated:

If both lenses before and after the lens switching do not have the optical anti-shake function, the situation before and after the lens switching is as shown in fig. 1A. In fig. 1A, the area between the solid lines Y and Y 'is a shooting channel of the electronic device, and before the lens is switched, the electronic device collects an image through the combination of the first image sensor 1 and the lens 1', and at this time, the main optical axis of the lens 1 is X; after the lens is switched, the electronic device collects an image through the combination of the second image sensor 2 and the lens 2', and at this time, the main optical axis of the lens 2 is still X. Therefore, when the two lenses have no optical anti-shake function, the main optical axes of the two lenses are consistent when the two lenses shoot images.

If the first lens before the lens is switched has an optical anti-shake function, and the second lens after the lens is switched does not have an optical anti-shake function, the situation before and after the lens is switched is shown in fig. 1B. Similarly to fig. 1A, the region between the solid lines Y and Y 'in fig. 1B is also a photographing channel of the electronic device, and before the lens is switched, the electronic device collects an image through the combination of the first image sensor 1 and the lens 1', and at this time, the optical anti-shake function mistakes the operation of the lens switching as the device shake, so that the lens 1 'is subjected to position adjustment so that its main optical axis is deviated, and the main optical axis is X' when the first image is photographed; after the lens is switched, the electronic device acquires an image through the combination of the second image sensor 2 and the lens 2', and at this time, the main optical axis of the lens 2 is still X. Obviously, when the lens has an optical anti-shake function, the main optical axes of the two lenses are inconsistent when images are shot, which inevitably causes a shift in the angle of view. Accordingly, the over-frame image generated on the basis of the method also has a field angle offset between the two images obtained by shooting, so that the phenomenon of picture jitter occurs in the generated preview picture.

It can be seen that the preview screen generation method in the related art is only suitable for switching between lenses without an optical anti-shake function. If any one of the two lenses before and after switching has an optical anti-shake function, smooth transition of the preview image cannot be achieved.

Therefore, the present disclosure proposes a preview method to avoid the problem that in the related art, the application scenario that the optical anti-shake function is not considered on the lens is not considered, and the generated excessive frame image cannot make the preview image in the lens switching process unable to be smooth and excessive.

Fig. 2 is a preview screen generation method according to an exemplary embodiment of the present disclosure. As shown in fig. 2, the method may include the steps of:

Step 202, determining an optical anti-shake offset of a target lens with an optical anti-shake function in a first lens and a second lens in a switching process when the first lens is switched to the second lens.

From the above analysis, when the lens has an optical anti-shake function, the optical anti-shake function can automatically adjust the position of the lens. In the related art, the influence of the optical anti-shake function on the position adjustment of the lens on the captured image is not considered, so that the generated excessive frame image and the image captured by the two lenses have a field angle offset, and further the problem of picture shake occurs after the excessive frame image is added in the preview picture in the related art.

In view of this, in the present disclosure, when the lenses are switched and one or both of the two lenses before and after the switching have the optical anti-shake function, the offset amount of the optical anti-shake to the lenses is preferentially determined, and the offset correction is performed on the image captured by the lenses based on the offset amount, so as to eliminate the influence of the optical anti-shake function on the captured image. On this basis, an excessive frame image is generated based on the corrected image, and a preview image is further generated from the corrected image and the excessive frame image. It should be understood that the present disclosure eliminates the influence of the optical anti-shake function on the image, so that there is no angular shift between the images obtained through the two lenses, and correspondingly, there is no angular shift between the excessive frame image obtained based on this and the images obtained through the two lenses, so as to avoid the phenomenon that the image shake occurs in the preview image obtained finally due to the fact that the optical anti-shake function is not considered in the related art.

And 204, performing offset correction on the pre-correction image shot by the target lens based on the optical anti-shake offset to eliminate the angle of view offset of the pre-correction image caused by the optical anti-shake function of the target lens, so as to obtain a corrected image.

In the present disclosure, a lens having an optical anti-shake function among two lenses involved in a lens switching process is referred to as a target lens; in the lens switching process, the lens displacement amount generated by adjusting the lens position due to the optical anti-shake function is called optical anti-shake offset. In actual operation, the optical anti-shake offset can be determined by monitoring the adjustment operation of the optical anti-shake device on the lens; the attitude change information and the position change information of the electronic equipment can be monitored through sensors such as a gyroscope and the like, and then the optical anti-shake device is reversely pushed to adjust the lens according to an optical anti-shake mechanism, so that the position offset is generated. Of course, the foregoing examples are merely illustrative, and it should be understood that only the manner in which the foregoing optical anti-shake offset can be detected may be applied to the present disclosure, and in particular, how to determine the optical anti-shake offset may be determined by those skilled in the art according to actual needs, which is not limited in this disclosure.

It should be understood that the phenomenon that the optical anti-shake causes the angle of view of the image captured by the target lens to deviate is essentially caused by the fact that the optical anti-shake is performed after the position of the target lens is adjusted (i.e. the target lens is displaced on the plane of the target lens itself, or the target lens is displaced on the plane perpendicular to the main optical axis), and the specific principle can be referred to as the description of fig. 1A and fig. 1B above.

In the present disclosure, in order to avoid a situation in which an optical anti-shake function causes a field angle shift to occur in an image captured by a target lens, when the lens is switched, an optical anti-shake shift amount that is actually determined may be: the first optical anti-shake offset of the target lens on the plane of the target lens; and performing offset correction on the pre-correction image captured by the target lens based on the first optical anti-shake offset amount. In actual operation, the pixel offset corresponding to the pre-correction image may be determined according to the first optical anti-shake offset, and the pre-correction image may be subjected to pixel offset according to the pixel offset. It should be noted that the first optical anti-shake offset that is actually determined includes an offset direction, and correspondingly, the pixel offset that is determined according to the first optical anti-shake offset includes an offset direction. How to define the direction can be determined by a person skilled in the art according to actual needs, and the disclosure is not limited to this, and only needs to follow the principle of "the determined pixel offset amount, and the corrected image obtained through offset correction can eliminate the angle of view offset of the image before correction due to the optical anti-shake function".

Step 206, generating an excessive frame image in the switching process based on the corrected image, and generating a preview picture corresponding to the switching process based on the excessive frame image and the corrected image.

In the present disclosure, after the corrected image is obtained based on the optical anti-shake offset amount, an excessive frame image may be generated based on the corrected image in various ways.

In an embodiment, the corrected image may be obtained by performing a scaling process on the corrected image after the corrected image is obtained. In actual operation, the target magnification corresponding to the over-frame image is generally related to "the distance offset between the two lenses before and after the lens switching". The distance offset value refers to: a difference in distance between the first lens and the corresponding image sensor and a distance between the second lens and the corresponding image sensor. From the imaging principle, the distance offset is the image distance change value before and after lens switching. The distance offset is the difference between the distances M and N, as described in connection with fig. 1A.

It should be appreciated that, for any lens equipped in an electronic device, the distance between the any lens and the corresponding image sensor is typically determined by a technician setting at the hardware design stage, being one or more (possibly a lens group comprising multiple lenses) fixed values, without regard to optical anti-shake positioning of the lenses. Therefore, the distance offset determined based on the distance between the two lenses and their image sensors is also a fixed value. In practical application, the value can be recorded in the storage space so as to be read and used at any time; this value may also be calculated temporarily based on the distances between the two lenses and the respective sensors (i.e., the distances M and N described above) when needed for use.

In practice, the distance offset is related to the target magnification of the over-frame image, because the distance offset is related to the first magnification of the image captured by the first lens and the second magnification of the image captured by the second lens, and the magnification of the over-frame image is related to the first magnification and the second magnification. For example, if the magnification of the image captured by the first lens is 1 and the magnification of the image captured by the second lens is 2, the preview screen can be maximally excessively smoothed when the magnification of the excessive frame image is 1.5. It should be noted that, although the example is described by taking the case that the preview image is smoothly excessively large when the target magnification of the excessive frame image is the intermediate value between the first magnification and the second magnification, in practical application, the preview image is not necessarily smoothly excessively large when the intermediate value is taken, and particularly, how to take the target magnification of the excessive frame image may be determined by a technician according to an actual test.

In this embodiment, the actual zooming process may be determined according to the actual situation, for example, in the case where there is only one target lens (i.e., only one lens has an optical anti-shake function before and after switching), the corrected image corresponding to the target lens may be zoomed based on the determined target magnification on the premise that the corrected image is obtained, so as to obtain an over-frame image. Of course, since there is no angular shift between the corrected image and the image captured by the other lens, it is also possible to perform scaling processing on the image captured by the other lens, and the magnification adopted for the image is obviously different from the target magnification described above, requiring additional determination. In the case where both lenses are target lenses (i.e., both lenses before and after switching have an optical anti-shake function), one of the two lenses may be selected and scaled when corrected images corresponding to both target lenses are obtained.

In practical applications, the optical anti-shake function not only adjusts the target lens on the plane where the target lens is located, but also adjusts the target lens in the direction of the main optical axis of the target lens. Obviously, the adjustment in this direction causes the above-described change in the distance offset amount, which in turn affects the magnification of the excessive frame image. It can be seen that the distance offset determined when the original hardware design is still adopted necessarily results in inaccurate magnification of the determined excessive frame image.

Therefore, in this embodiment, a second optical anti-shake offset of the target lens in the main optical axis direction of the target lens may also be obtained, and the distance offset may be corrected based on the second optical anti-shake offset, so as to determine the target magnification corresponding to the excessive frame image based on the corrected distance offset. It should be understood that in this case, it is equivalent to acquiring the adjustment data of the optical anti-shake for the target lens in the main optical axis direction, and correcting the above-described distance offset based on the adjustment data, and the problem of inaccurate target magnification determination of the excessive frame image due to the influence of the optical anti-shake on the distance offset being not considered is avoided.

In another embodiment, since the optical anti-shake function is eliminated from shifting the pre-correction image captured by the target lens after the shifting correction, if only one of the two lenses is the target lens, there is no shifting between the post-correction image corresponding to the target lens and the image captured by the other lens, and therefore the post-correction image and the pre-correction image captured by the other lens may be image-synthesized to obtain an excessive frame image; accordingly, if both lenses are target lenses, there is no angular shift between the corrected images corresponding to the two target lenses, so that the corrected images corresponding to the two lenses can be combined to obtain an excessive frame image. Obviously, there is no angular shift between images for synthesizing the over-frame image, and therefore, the synthesized over-frame image does not have any angular shift. In actual operation, any kind of synthesizing algorithm may be used to synthesize the above excessive frame image, which is only required to ensure that the magnification of the excessive frame image is between two images, which is not limited in this disclosure.

In this embodiment, two images for synthesizing the excessive frame image, corresponding to the first lens and the second lens, respectively, may be referred to as a first image to be fused corresponding to the first lens and a second image to be fused corresponding to the second lens, respectively. If the lens has an optical anti-shake function, the image directly captured by the lens is referred to as a pre-correction image, and the image obtained by offset correction is referred to as an over-corrected image, then the first image to be fused is necessarily the pre-correction image or the post-correction image corresponding to the first lens, and the second image to be fused is necessarily the pre-correction image or the post-correction image corresponding to the second lens.

In the present disclosure, when the corrected image is acquired, and the over-frame image is acquired, a preview screen corresponding to the shot switching process can be generated based on the over-frame image and the corrected image.

In an embodiment, the first lens and the second lens have the optical anti-shake function, and then the first corrected image corresponding to the first lens, the over-frame image, and the second corrected image corresponding to the second lens may be displayed in sequence as preview images of the switching process.

In another embodiment, only one of the first lens and the second lens has an optical anti-shake function, and a corrected image corresponding to the one having the optical anti-shake function, an over-frame image, and an image captured by the other having no optical anti-shake function are sequentially displayed in order of lens switching as preview images of the switching process.

Specifically, in the case where only the first lens of the two lenses has the optical anti-shake function, the corrected image, the excessive frame image, and the image captured by the second lens corresponding to the first lens may be displayed in order as a preview image of the lens switching process. In the case that only the second lens has the optical anti-shake function in the two lenses, the image captured by the first lens, the excessive frame image and the corrected image corresponding to the second lens can be displayed in sequence to be used as a preview picture of the lens switching process.

It should be noted that, in the present disclosure, the operation of generating the preview screen may be performed again in the case where the shot switching is completed. In other words, the image obtained by shooting is processed to obtain an excessive frame image after the images obtained by shooting through the two lenses are obtained. The present disclosure may also start performing the operation of generating the preview screen already during the course of the shot switching, for example, in the case where the first shot has an optical anti-shake function, the operation of switching the first shot to the second shot may be performed while offset correction is performed on the pre-correction image captured by the first shot. In other words, the lens switching operation and the image processing operation are performed in parallel. It should be understood that, since the process of generating the preview screen is in the millisecond level, whether or not it is performed in parallel with the shot switching operation, it can be accomplished without feeling the user, and how to perform it can be set by those skilled in the art according to actual needs.

In addition, the technical scheme of the disclosure can be applied to any type of mobile terminal, and only the mobile terminal is provided with at least two lenses and at least one lens has an optical anti-shake function. For example, the mobile terminal may be a smart phone, a tablet computer, or the like. The specific application of the technical solution of the present disclosure to what mobile terminal may be determined by those skilled in the art according to actual needs, which is not limited in this disclosure.

According to the technical scheme, when the first lens is switched to the second lens, the optical anti-shake offset of the target lens with the optical anti-shake function is acquired, and the image shot by the target lens is offset corrected based on the acquired optical anti-shake offset, so that the angle of view offset of the target lens due to the optical anti-shake function is eliminated, and the corrected image is further obtained. On this basis, an excessive frame image corresponding to the switching process can be generated based on the corrected image, and a corresponding preview screen can be generated based on the excessive frame image.

In the technical scheme of the disclosure, an optical anti-shake offset of a target lens with an optical anti-shake function is preferentially determined, offset correction is performed on a shot image based on the optical anti-shake offset, so as to eliminate the angle of view offset of the shot image to the target lens due to the optical anti-shake function, a corrected image is obtained, and an excessive frame image is generated based on the corrected image. In other words, the technical scheme of the present disclosure can eliminate the influence of optical anti-shake on the image by a post-processing mode, and avoid the problem that the image obtained by two-lens shooting has a shift in angle of view due to adjustment of optical anti-shake. Obviously, because the image shot by the target lens does not have the angle of view deviation, compared with the image shot by the target lens and the image shot by the target lens, the generated excessive frame image does not have the angle of view deviation, so that the preview image generated by the target lens and the excessive frame image does not generate the condition of image shake, and smooth transition in the lens switching process is realized.

Further, if the corrected image is scaled, an excessive frame image is obtained. The determined target magnification corresponding to the over-frame image will directly affect the smoothness of the preview screen. For example, when the magnification of the excessive frame image is close to the image captured by the first lens but is far from the image captured by the second lens, the effect of the frame jump occurs inevitably when the excessive frame image jumps to the image corresponding to the second lens. In the process of lens switching, the target magnification is related to the distance offset, and the optical anti-shake function of the lens affects the distance offset. In view of this, the present disclosure further provides a method for determining a second optical anti-shake offset that affects the distance offset, and determining a target magnification for generating an excessive frame image after correcting the distance offset, so as to avoid the problem that the generated excessive frame image cannot smooth and excessively preview images due to inaccurate determination of the target magnification.

Next, taking an example that a main lens of the smart phone before lens switching has an optical anti-shake function, and a wide-angle lens after lens switching does not have an optical anti-shake function, the technical scheme of the disclosure is described.

Fig. 3 is another preview screen generation method shown in an exemplary embodiment of the present disclosure. As shown in fig. 3, the method may include the steps of:

step 301, the camera APP is started.

In this embodiment, the smart phone is usually pre-equipped with a camera APP to meet the shooting requirements of the user. In actual operation, a user can start the camera APP by triggering an APP icon corresponding to the camera APP on the mobile phone desktop.

And 302, displaying an image shot by the main lens as a preview picture.

In this embodiment, after the camera APP is started, an image is collected through the currently used lens and the image sensor corresponding to the currently used lens, and is displayed as a preview image.

In practical applications, a smart phone is generally equipped with a plurality of lenses, wherein the smart phone includes a main lens, and the main lens can meet shooting requirements of users in most scenes. When the camera APP is activated, image acquisition is typically performed by default through the lens. Therefore, after the camera APP is started, image acquisition can be performed through the main lens, and the acquired image is displayed as a preview picture.

Step 303, when detecting the triggering operation of the user on the lens switching control, switching the main lens to the wide-angle lens.

In this embodiment, after the camera APP is started, a corresponding display interface may be displayed. The lens switching control used for switching the lens can be further displayed in the display interface, so that when a user needs to switch the lens to meet different shooting requirements, the lens currently used by the mobile phone is switched by triggering the lens switching control.

In this embodiment, the switching to the wide-angle lens is taken as an example, and thus, the lens switching control may correspond to the switching between the main lens and the wide-angle lens.

Step 304, obtaining an optical anti-shake offset corresponding to the main lens in the process of switching from the main lens to the wide-angle lens.

In this embodiment, only the main lens has an optical anti-shake function. When the mobile phone is produced, a corresponding optical anti-shake device can be configured for the main lens, so that the main lens has an optical anti-shake function.

When the mobile phone switches the lens based on the triggering operation of the user on the lens switching control. Since the main lens has an optical anti-shake function, it is necessary to determine an optical anti-shake shift amount of the main lens due to the optical anti-shake function to perform shift correction on a pre-correction image acquired through the main lens.

In step 305, a pixel offset corresponding to the image captured by the main lens is determined based on the acquired optical anti-shake offset.

In this embodiment, the optical anti-shake offset refers to a displacement amount caused by the optical anti-shake device adjusting the main lens. The displacement amount causing the angle of view offset in the corrected image obtained by shooting is as follows: displacement on the plane of the main lens. In actual operation, in order to cancel out the angular displacement of the field of view due to this displacement amount in the screen, the direction of the pixel displacement amount determined based on the optical anti-shake displacement amount is generally opposite to the direction of the optical anti-shake displacement amount. For example, in the case of an optical anti-shake offset: +1mm, which means that the optical anti-shake device adjusts the main lens such that the main lens is displaced upward by 1mm, the determined pixel offset may be-1 mm, so that the pixels in the image before correction are all offset downward by 1mm. Of course, this example only takes "when the main lens is shifted up by 1mm, the pixels in the image are shifted down by 1mm to exactly offset the angle of view shift caused by the lens shift" as an example, and in practical application, the pixels in the image specifically need to be shifted in which direction to offset the angle of view shift caused by the lens shift, which should be determined by those skilled in the art according to the practical situation, and this embodiment is not limited.

And 306, performing pixel offset on the image shot by the main lens according to the pixel offset to obtain a corrected image.

In the present embodiment, since only the main lens has the optical anti-shake function, it is only necessary to perform offset correction on the pre-correction image obtained by photographing through the main lens. It is to be understood that there is no angular field shift between the corrected image obtained by correction and the image obtained by photographing through the wide-angle lens after switching.

Step 307, obtaining the magnification determined according to the distance offset between the main lens and the wide-angle lens.

In the present embodiment, the excessive frame image is generated in such a manner that the corrected image is scaled, and a specific generation manner has been elucidated in the above embodiment, and a description thereof will not be repeated in the present embodiment.

And step 308, scaling the corrected image according to the magnification to generate an excessive frame image.

Step 309, after the lens switching is completed, the obtained image is captured by the wide-angle lens.

It should be noted that the present embodiment is merely exemplified by "the step of generating the excessive frame image is performed directly after the above-described pre-correction image is obtained", and in actual operation, the present step may be performed in synchronization with a series of steps of generating the excessive frame image or may be performed before generating the excessive frame image, and the present embodiment is not limited thereto.

Step 310, sequentially displaying the corrected image, the excessive frame image and the graph obtained by wide-angle lens shooting, so as to be used as a preview picture corresponding to the lens switching process.

According to the technical scheme, when the main lens with the optical anti-shake function is switched to the wide-angle lens without the optical anti-shake function, the intelligent mobile phone can determine the displacement of the main lens caused by the optical anti-shake function, and carry out pixel offset on the image shot by the main lens based on the displacement so as to offset the view angle offset of the optical anti-shake function on the image in the lens switching process. On the basis, an excessive frame image is generated based on the corrected image obtained through pixel offset, so that the situation of picture jitter in the finally obtained preview picture is avoided.

Fig. 4 is a block diagram of a preview screen generating apparatus according to an exemplary embodiment of the present disclosure. Referring to fig. 4, the apparatus includes a determination unit 401, a correction unit 402, and a generation unit 403.

A determination unit 401 that determines, in the case of switching from a first lens to a second lens, an optical anti-shake offset amount of a target lens having an optical anti-shake function in the first lens and the second lens during switching;

A correction unit 402 that performs offset correction on a pre-correction image captured by the target lens based on the optical anti-shake offset amount to eliminate a viewing angle offset of the pre-correction image due to the optical anti-shake function of the target lens, and obtains a corrected image;

The generating unit 403 generates an excessive frame image in the switching process based on the corrected image, and generates a preview screen corresponding to the switching process based on the excessive frame image and the corrected image.

Alternatively to this, the method may comprise,

The determining unit 401 is further adapted to: determining a first optical anti-shake offset of the target lens on a plane of the target lens;

The correction unit 402 is further used to: and determining a pixel offset corresponding to the pre-correction image based on the first optical anti-shake offset, and performing pixel offset on the pre-correction image according to the pixel offset.

Optionally, the generating unit 403 is further configured to:

Acquiring a distance offset in the switching process; the distance offset is the difference between the distance between the first lens and the corresponding image sensor and the distance between the second lens and the corresponding image sensor;

And determining a target magnification corresponding to the corrected image based on the distance offset, and performing scaling processing on the corrected image according to the target magnification to obtain the excessive frame image.

Alternatively to this, the method may comprise,

The determining unit 401 is further adapted to: determining a second optical anti-shake offset of the target lens in the main optical axis direction of the target lens;

the generating unit 403 is further adapted to: and correcting the distance offset based on the second optical anti-shake offset, and determining a target magnification based on the corrected distance offset.

Optionally, the generating unit 403 is further configured to:

synthesizing a first image to be fused corresponding to the first lens and a second image to be fused corresponding to the second lens into an excessive frame image;

the first image to be fused is an image before correction or an image after correction corresponding to the first lens; the second lens to be fused is an image before correction or an image after correction corresponding to the second lens; in the case that any lens does not have an optical anti-shake function, the pre-correction image corresponding to any lens is an image captured by any lens.

Optionally, the generating unit 403 is further configured to:

Sequentially displaying a first corrected image corresponding to the first lens, the excessive frame image and a second corrected image corresponding to the second lens as preview pictures of the switching process under the condition that the first lens and the second lens have optical anti-shake functions;

When only one of the first lens and the second lens has an optical anti-shake function, the corrected image corresponding to the one having the optical anti-shake function, the excessive frame image, and the image captured by the other having no optical anti-shake function are sequentially displayed in order of lens switching as preview images of the switching process.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

Correspondingly, the disclosure further provides a preview screen generating device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the preview screen generation method according to any of the above embodiments, for example, the method may include: under the condition that a first lens is switched to a second lens, determining optical anti-shake offset of a target lens with an optical anti-shake function in the first lens and the second lens in the switching process; performing offset correction on the pre-correction image obtained by shooting the target lens based on the optical anti-shake offset to eliminate the angle of view offset of the pre-correction image due to the optical anti-shake function of the target lens, so as to obtain a corrected image; and generating an excessive frame image in the switching process based on the corrected image, and generating a preview picture corresponding to the switching process based on the excessive frame image and the corrected image.

Accordingly, the present disclosure also provides an electronic device including a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by the one or more processors, where the one or more programs include instructions for implementing the preview screen generation method according to any of the foregoing embodiments, for example, the method may include: under the condition that a first lens is switched to a second lens, determining optical anti-shake offset of a target lens with an optical anti-shake function in the first lens and the second lens in the switching process; performing offset correction on the pre-correction image obtained by shooting the target lens based on the optical anti-shake offset to eliminate the angle of view offset of the pre-correction image due to the optical anti-shake function of the target lens, so as to obtain a corrected image; and generating an excessive frame image in the switching process based on the corrected image, and generating a preview picture corresponding to the switching process based on the excessive frame image and the corrected image.

Fig. 5 is a block diagram illustrating an apparatus 500 for implementing a preview screen generation method according to an exemplary embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, or the like.

Referring to fig. 5, an apparatus 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input/output (I/O) interface 512, a sensor component 514, and a communication component 516.

The processing component 502 generally controls overall operation of the apparatus 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 502 can include one or more modules that facilitate interactions between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

The memory 504 is configured to store various types of data to support operations at the apparatus 500. Examples of such data include instructions for any application or method operating on the apparatus 500, contact data, phonebook data, messages, pictures, videos, and the like. The memory 504 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 506 provides power to the various components of the device 500. The power components 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 500.

The multimedia component 508 includes a screen between the device 500 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front lens and/or a rear lens. The front lens and/or the rear lens may receive external multimedia data when the apparatus 500 is in an operation mode, such as a photographing mode or a video mode. Each of the front lens and the rear lens may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 510 is configured to output and/or input audio signals. For example, the audio component 510 includes a Microphone (MIC) configured to receive external audio signals when the device 500 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 further comprises a speaker for outputting audio signals.

The I/O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 514 includes one or more sensors for providing status assessment of various aspects of the apparatus 500. For example, the sensor assembly 514 may detect the on/off state of the device 500, the relative positioning of the components, such as the display and keypad of the device 500, the sensor assembly 514 may also detect a change in position of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration/deceleration of the device 500, and a change in temperature of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 516 is configured to facilitate communication between the apparatus 500 and other devices in a wired or wireless manner. The apparatus 500 may access a wireless network based on a communication standard, such as WiFi,2G or 3G,4G LTE, 5G NR (New Radio), or a combination thereof. In one exemplary embodiment, the communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 504, including instructions executable by processor 520 of apparatus 500 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The foregoing description of the preferred embodiments of the present disclosure is not intended to limit the disclosure, but rather to cover all modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present disclosure.

Claims

1. A preview screen generation method, comprising:

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The determining the optical anti-shake offset of the target lens includes: determining a first optical anti-shake offset of the target lens on a plane of the target lens;

The offset correction of the image captured by the target lens based on the optical anti-shake offset includes: and determining a pixel offset corresponding to the pre-correction image based on the first optical anti-shake offset, and performing pixel offset on the pre-correction image according to the pixel offset.

3. The method of claim 1, wherein the generating the over-frame image in the switching process based on the corrected image comprises:

4. The method of claim 3, wherein the step of,

The determining the optical anti-shake offset of the target lens includes: determining a second optical anti-shake offset of the target lens in the main optical axis direction of the target lens;

the determining a target magnification corresponding to the corrected image based on the distance offset includes: and correcting the distance offset based on the second optical anti-shake offset, and determining a target magnification based on the corrected distance offset.

5. The method of claim 1, wherein the generating the over-frame image in the switching process based on the corrected image comprises:

the first image to be fused is a pre-correction image or a post-correction image corresponding to the first lens, the second image to be fused is a pre-correction image or a post-correction image corresponding to the second lens, and at least one of the first image to be fused and the second image to be fused is a post-correction image corresponding to the corresponding lens; in the case that any lens does not have an optical anti-shake function, the pre-correction image corresponding to any lens is an image captured by any lens.

6. The method of claim 1, wherein the generating a preview screen corresponding to the switching process based on the over-frame image and the corrected image comprises:

7. A preview screen generating apparatus, comprising:

8. The apparatus of claim 7, wherein the device comprises a plurality of sensors,

The determination unit is further adapted to: determining a first optical anti-shake offset of the target lens on a plane of the target lens;

The correction unit is further configured to: and determining a pixel offset corresponding to the pre-correction image based on the first optical anti-shake offset, and performing pixel offset on the pre-correction image according to the pixel offset.

9. An electronic device, comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the method of any of claims 1-6 by executing the executable instructions.

10. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method according to any of claims 1-6.