CN115880193A

CN115880193A - Image processing method, image processing device, electronic equipment and storage medium

Info

Publication number: CN115880193A
Application number: CN202111145746.8A
Authority: CN
Inventors: 赵鹏; 高飞; 弓殷强; 陈彦哲; 李屹
Original assignee: Appotronics Corp Ltd
Current assignee: Shenzhen Appotronics Corp Ltd
Priority date: 2021-09-28
Filing date: 2021-09-28
Publication date: 2023-03-31

Abstract

The embodiment of the application provides an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of projection. The method described in this embodiment is applied to a processor of a projection system, the projection system includes at least two projection components, and the method includes: acquiring a target projection image, and if the bit depth of the target projection image is higher than that of each projection component, determining a plurality of projection images to be projected corresponding to at least two projection components based on the target projection image so as to enable the sum of the display brightness of each projection image to be equal to that of the target projection image; and then, controlling each projection component to project each image to be projected corresponding to each projection component, and enabling a plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection component. The image with the bit depth higher than the projection assembly is divided into a plurality of images to be projected, and a plurality of projection assemblies are used for projection, so that the purpose of projecting the high bit depth image by using the projection assembly with lower bit depth can be achieved.

Description

Image processing method, image processing device, electronic equipment and storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.

Background

High bit-depth images typically contain very fine image data that can present a richer and realistic picture to a user. However, the high bit depth image has a high performance requirement for the projection module, and the low bit depth projection module cannot accurately project the high bit depth image.

Disclosure of Invention

The application provides an image processing method, an image processing device, an electronic device and a storage medium, so as to overcome the defects.

In a first aspect, an embodiment of the present application provides an image processing method applied to a processor of a projection system, where the projection system includes at least two projection assemblies, and the method includes: acquiring a target projection image; if the bit depth of the target projection image is higher than that of each projection component, determining a plurality of projection images to be projected corresponding to the at least two projection components based on the target projection image so as to enable the sum of the display brightness of each projection image to be equal to that of the target projection image; and controlling each projection assembly to project each image to be projected corresponding to each projection assembly, and enabling the plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection assembly.

In a second aspect, an embodiment of the present application further provides an image processing apparatus applied to a processor of a projection system, where the projection system includes at least two projection assemblies, and the apparatus includes: the device comprises an acquisition module, a processing module and a control module. The acquisition module is used for acquiring a target projection image; the processing module is used for determining a plurality of images to be projected corresponding to the at least two projection assemblies based on the target projection image if the bit depth of the target projection image is higher than the bit depth of each projection assembly, so that the sum of the display brightness of each image to be projected is equal to the display brightness of the target projection image; and the control module is used for controlling each projection assembly to project each image to be projected corresponding to each projection assembly, so that the plurality of images to be projected are displayed in a superposition manner, wherein each image to be projected corresponds to one projection assembly.

In a third aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, where the memory stores a computer program, and the processor, by calling the computer program, performs the following operations by calling the computer program: acquiring a target projection image; if the bit depth of the target projection image is higher than that of each projection assembly, determining a plurality of projection images to be projected corresponding to the at least two projection assemblies based on the target projection image so that the sum of the display brightness of each projection image to be projected is equal to the display brightness of the target projection image; and controlling each projection assembly to project each image to be projected corresponding to each projection assembly, and enabling the plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection assembly.

In a fourth aspect, embodiments of the present application further provide a projection system, which includes a processor and at least two projection assemblies, where the processor is configured to perform the above method.

In a fifth aspect, embodiments of the present application further provide a computer-readable storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, which is loaded and executed by a processor to implement the above method.

According to the image processing method, the image processing device, the electronic device and the storage medium, firstly, a target projection image is obtained, and if the bit depth of the target projection image is higher than that of each projection assembly, a plurality of images to be projected corresponding to at least two projection assemblies are determined based on the target projection image, so that the sum of the display brightness of each image to be projected is equal to the display brightness of the target projection image. And then controlling each projection assembly to project each image to be projected corresponding to each projection assembly, and enabling the plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection assembly. This application is through dividing into a plurality of projection images of treating with the image that the bit depth is higher than the projection subassembly, uses many projection subassemblies to throw, can reach the purpose that uses the projection subassembly projection high bit depth image of lower bit depth.

Additional features and advantages of embodiments of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of embodiments of the present application. The objectives and other advantages of the embodiments of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of a projection system provided by an embodiment of the present application;

FIG. 2 is a flow chart of a method of image processing according to an embodiment of the present application;

FIG. 3 is a block diagram illustrating a projection process of a projection system according to another embodiment of the present application;

FIG. 4 illustrates a method flow diagram of an image processing method provided by yet another embodiment of the present application;

FIG. 5 is a flow chart of a method of image processing according to another embodiment of the present application;

FIG. 6 is a diagram illustrating a relationship between brightness of an image to be projected according to an embodiment of the present application;

FIG. 7 is a diagram illustrating pixel values of an image to be projected according to another embodiment of the present application;

FIG. 8 is a block diagram of an image processing apparatus provided in an embodiment of the present application;

fig. 9 shows a block diagram of an electronic device provided in an embodiment of the present application;

fig. 10 shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

At present, the most common image bit depth is usually 8 bits, but in a high bit depth image, the bit depth can be as high as 10 bits, 12 bits or 14 bits, and so on, so the high bit depth image can contain very fine image data, the displayed color gradation is richer, richer and real pictures can be displayed for users, and the application of the high bit depth image is more and more extensive. In practical applications, the projection device with low bit depth can only display an image within a positioning depth range due to the limitation of hardware conditions, while the projection device with high bit depth has a higher requirement on the bit depth of the projection device, and the projection device with high bit depth is relatively short, but the projection device with low bit depth cannot accurately project the image with high bit depth.

The inventor finds in research that the bit depth range of a plurality of projection assemblies with lower bit depth can be combined in an image overlapping mode to realize the bit depth display in a higher range, so that the overlapped image seen by human eyes and the original high bit depth image show a consistent display effect, and the purpose of using the projection assemblies with lower bit depth to project the high bit depth image can be achieved.

Therefore, in order to improve the above-mentioned defects, the application embodiments provide an image processing method, an apparatus, an electronic device, and a storage medium, where an execution subject of the method may be the image processing apparatus provided in the embodiments of the application, or an electronic device integrated with the image processing apparatus, a projection device integrated with the image processing apparatus, and the like, and the image processing apparatus may be implemented in hardware or software. The electronic device may be a tablet Computer, a game console, an electronic book reader, a multimedia player, a smart watch, a mobile phone, a PC (Personal Computer), or the like. The electronic device is provided with a plurality of application programs, and a user can use the plurality of application programs to realize different functions and purposes.

Referring to fig. 1, a schematic diagram of a projection system 100 according to an embodiment of the present disclosure is shown, where the projection system 100 includes a processor 110 and at least two projection assemblies 120 according to an embodiment of the present disclosure. In some embodiments, the processor 110 divides the acquired image into a plurality of images to be projected according to the performance of each projection assembly 120, and then sends the images to be projected to each projection assembly 120 and controls each projection assembly 120 to project the images. In order to make the plurality of images to be projected seen by human eyes coincide, the projection area 130 of each projection assembly 120 displaying the images to be projected may also coincide. It is understood that the projection system 100 may further include a projection screen, and each projection assembly 120 may project the image to be projected onto the projection screen, so that the projection area 130 of each projection assembly 120 may coincide on the projection screen.

Referring to fig. 2, a flowchart of an image processing method provided in an embodiment of the present application is shown, where the image processing method can be applied to, for example, the processor 110 of the projection system 100 shown in fig. 1, and the projection system 100 includes at least two projection assemblies 120. The method comprises the following steps: step S210 to step S230.

Step S210: a target projection image is acquired.

In the embodiment of the present application, projection of an image indicated by a projection instruction may be started according to the projection instruction of a user. Wherein, firstly, an original image which is appointed to be projected by a user should be acquired, and the original image can be used as a target projection image to be processed. The target projection image may be a single picture, or may be a video frame in a video stream. If the target projection image is an independent picture, the target projection image can be processed according to the sequence of transmitting the picture to the processor in the embodiment of the application; if the target projection image is a video frame in a segment of video stream, the target projection image can be processed according to the playing sequence of the video frame in the video stream, and optionally, in order to ensure that a user can see continuous and smooth video pictures after projection, the projection component can also project the image to be projected obtained after the video frame is processed according to the playing sequence of the video frame in the video stream.

In some embodiments, the target projection image may originate from an external device or may be stored in memory internal to the projection system. The external device may be an electronic device such as a mobile phone or a Personal Computer (PC), or may be an imaging device such as a camera or a video recorder. The processor may receive the target projection image transmitted by the external device or the memory according to the projection instruction of the user.

Step S220: and if the bit depth of the target projection image is higher than that of each projection assembly, determining a plurality of images to be projected corresponding to the at least two projection assemblies based on the target projection image so as to enable the sum of the display brightness of each image to be projected to be equal to the display brightness of the target projection image.

As one way, if the bit depth of the target projection image is lower than or equal to the bit depth of any projection component in the projection system, the target projection image can be projected directly using the projection component with the bit depth higher than or equal to the target projection image. However, in practical applications, since the bit depth of the high bit depth image is usually as high as 10 bits, 12 bits, 14 bits, etc., and the bit depth of the projection module is usually less than 10 bits, for example, most projection modules have a bit depth of 8 bits, it is difficult to achieve the high requirement of the high bit depth image on the display performance of the projection module. Therefore, in the embodiment of the present application, if the bit depth of the target projection image is higher than the bit depth of each projection assembly in the projection system, the target projection image may be divided into a plurality of images to be projected, where each image to be projected corresponds to one projection assembly. Each image to be projected will illustratively be projected by one projection assembly in the projection system, and thus, the image to be projected may be determined based on the target projection image and the number and performance of all projection assemblies in the projection system.

In some embodiments, the projection assemblies and the images to be projected may be in a one-to-one relationship, for example, one projection assembly projects one image to be projected, so that each image to be projected may be projected by a different projection assembly, and each image to be projected is determined based on the performance of the target projection image and the projection assembly that projected the image to be projected. For example, if the performances other than the bit depth are the same, the projection module may be configured to project the image to be projected, which is closer to the target projection image, as the bit depth of the projection module is higher.

In other embodiments, the projection assembly and the image to be projected may be in a one-to-many relationship. For example, one projection assembly may correspondingly project two images to be projected, wherein the two images to be projected may be used as main and standby images, and the main image may be determined and then the standby image may be determined according to the target projection image and the projection assembly projecting the two images to be projected.

Furthermore, it is understood that, in order to make the plurality of images to be projected by the projection system and the original target projection image present a consistent display effect, the sum of the display brightness of each image to be projected may be equal to the display brightness of the target projection image.

Step S230: and controlling each projection assembly to project each image to be projected corresponding to each projection assembly, and enabling the plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection assembly.

In the embodiment of the application, after a plurality of images to be projected are determined, the processor of the projection system controls each projection assembly to respectively project each image to be projected corresponding to the projection assembly, and the plurality of images to be projected are also displayed in a superposition manner, so that the superposition brightness of the superposition display of the plurality of images to be projected can be equal to the sum of the display brightness of each image to be projected, the superposition brightness is equal to the display brightness of the target projection image, and the display effect is consistent with that of the original target projection image.

In some embodiments, in addition to the requirement that the projection areas of each projection component can be overlapped, each projection component needs to be controlled to synchronously project all the images to be projected, so that the images to be projected can be displayed in the same projection area at the same time, and the effect of overlapping and displaying a plurality of images to be projected is achieved. For example, to implement synchronous projection, after determining a plurality of images to be projected, the processor may simultaneously transmit a synchronization signal and the images to be projected to the corresponding projection components. If only one projection image to be projected is correspondingly projected by one projection assembly, all the projection assemblies can be controlled to project the corresponding projection image to be projected within the same projection time through the synchronous signal; if one projection component correspondingly projects two or more to-be-projected images, when the projection component is controlled, the projection component can be controlled to project the to-be-projected images at a high frequency, and the effect of synchronous projection is realized by utilizing the visual persistence effect of human eyes. For example, when the first image to be projected and the second image to be projected are projected by using the same projection assembly, if the switching frequency of the two images to be projected is faster than the persistence time, when the first image to be projected disappears, the second image to be projected will be displayed according to the switching frequency, and the human eye can keep the picture of the first image to be projected in the persistence time, and because the projection areas of the first image to be projected and the second image to be projected are overlapped, the effect of the human eye is that the two images are completely overlapped.

If the target projection image is a video frame in a segment of video stream, in some exemplary embodiments, the projection component in the projection system may simultaneously project a plurality of to-be-projected images corresponding to the same video frame, so that the to-be-projected images achieve the effect of overlapping display, and when playing the next video frame, also project a plurality of to-be-projected images corresponding to the next frame at the same time.

As shown in fig. 3, in some embodiments, one projection module in the projection system may be used as a main projection, the other projection modules except the main projection may be used as auxiliary projections, different images to be projected may be respectively allocated to the main projection and the auxiliary projections according to the respective performances of the projection modules, and then the main projection and the auxiliary projections may be controlled to respectively project each image to be projected corresponding to the projection module, so as to enable the plurality of images to be projected to be displayed in a superimposed manner. In particular, one projection component with the highest performance may be used as the main projection, for example, the bit depth is the highest or the projection brightness is the highest, and since the performance of the main projection is the highest, the closer the image to be projected that is projected corresponding to the main projection is to the target projection image, the maximum projection brightness may be assigned to the image to be projected that is projected corresponding to the main projection, and the like. It should be understood that there may be one or more main projections in the projection system, for example, multiple identical projection modules with higher performance in the projection system may be used as the main projections; the projection system may not distinguish between the main projection and the auxiliary projection, for example, if the performance of each projection component in the projection system is the same, each projection component may be assigned the same image to be projected.

It is to be understood that the target projection image according to the embodiment of the present application may not only be a high bit depth image, but a plurality of projection assemblies with bit depths lower than that of the target projection image may be used to project the target projection image by using the method according to the embodiment of the present application as long as the bit depth of the target projection image is higher than that of each projection assembly in the projection system. For example, if the bit depth of the target projection image is 8 bits, at least two projection components with a bit depth of 4 bits may be used to project the target projection image.

To sum up, the embodiment of the present application provides a technical solution, first, a target projection image is obtained, and if a bit depth of the target projection image is higher than a bit depth of each projection assembly, a plurality of to-be-projected images corresponding to the at least two projection assemblies are determined based on the target projection image, so that a sum of display brightness of each to-be-projected image is equal to display brightness of the target projection image. And then controlling each projection assembly to project each image to be projected corresponding to each projection assembly, and enabling the plurality of images to be projected to be displayed in a superposition mode, wherein each image to be projected corresponds to one projection assembly. This application is through dividing into a plurality of projection images of treating with the image that the bit depth is higher than the projection subassembly, uses many projection subassemblies to throw, can reach the purpose that uses the projection subassembly projection high bit depth image of lower bit depth.

In some embodiments of the present application, if the bit depth of the target projection image is higher than the bit depth of each projection component in the projection system, the target projection image may be divided into a plurality of images to be projected according to the number and performance of the projection components in the projection system. In some embodiments, the bit depth n of the target projection image is less than the highest bit depth of the image that can be displayed by the projection system shown in the embodiments of the present application.

The highest bit depth of the image that can be displayed by the projection system can be calculated according to the highest projection brightness and the bit depth of each projection component in the projection system. Taking a projection system composed of projection components with the bit depths of k stations of 8 bits as an example, the maximum value and the minimum value of the bit depth of the projection system can be obtained through calculation. The highest projection brightness of the k-th projection assembly may be denoted L1, L2, \8230;, lk, respectively, the highest projection brightness by the projection system may be L = L1+ L2+ \8230; + Lk, wherein if the highest projection brightness of the k-th projection assembly is the largest, the maximum value of the bit depth of the projection system may be Mmax = log2 ((L1 + L2+ 8230; + Lk)/(L1/256)), and the minimum value of the bit depth may be Mnin = log2 ((L1 + L2+ \8230; + Lk)/(Lk/256)). Further, the highest bit depth of the image that can be displayed by the projection system can be set between Mnin and Mmax (which can be Mnin or Mmax). Furthermore, the use of high bit depths for projection in practical applications may accelerate the wear of the projection components, so that the highest bit depth of the image that can be displayed by the projection system may be the minimum of the bit depths Mnin = log2 ((L1 + L2+ \8230; + Lk)/(Lk/256)).

In some embodiments, if the bit depth of the target projection image is higher than the bit depth of each projection component in the projection system and is also smaller than the highest bit depth of the image that can be displayed by the projection system shown in the embodiments of the present application, the target projection image may be divided into a plurality of images to be projected, where each image to be projected corresponds to one projection component.

Each image to be projected will illustratively be projected by one projection assembly in the projection system, and thus, the image to be projected may be determined based on the target projection image and the number and performance of all projection assemblies in the projection system. Specifically, in some embodiments of the present application, referring to fig. 4, step S220 may include: step S410 to step S420.

Step S410: and acquiring the brightness step value of each projection assembly and the brightness step value of the projection system.

In the embodiment of the application, the display brightness of a plurality of images to be projected is determined according to the brightness step value of each projection assembly, and in addition, the display brightness of the target projection image is related to the brightness step value of the projection system. In some embodiments, the brightness step value of each projection assembly may be determined based on the highest projection brightness of the projection assembly and the bit depth of the projection assembly, and the brightness step value of the projection system may be determined based on the highest projection brightness of the projection system and the bit depth of the target projection image. Referring to fig. 5, step S410 may include: step S510 to step S530.

Step S510: and acquiring the highest projection brightness and the bit depth of each projection assembly.

In an embodiment of the present application, the highest projection brightness of each projection component in the projection system may be obtained first.

As one mode, the maximum projection brightness of each projection module is a fixed parameter, and may be based on the maximum projection brightness measured during the manufacturing of the projection module. As another mode, if the maximum projection brightness of the projection module is unknown, each projection module may be controlled to perform projection with the maximum brightness, and the maximum projection brightness of each projection module may be obtained by performing brightness measurement on the projection area of each projection module. Optionally, the brightness measurement may be performed by using a nine-point method, and the specific measurement manner may be: the projection component to be measured is installed at a position 2.4 meters away from the projection curtain, the projection component to be measured is controlled to project with the highest brightness, the illuminance of each point on nine crossing points in the shape of the Chinese character 'tian' of the projection curtain is measured by an illuminometer, the illuminance of each point is multiplied by the area to obtain the brightness of 9 points of a projection picture, and finally the average value of the brightness of the 9 points is taken to obtain the highest projection brightness of the projection component.

In addition, in some embodiments, the bit depth of each projection component is also obtained.

Step S520: and determining the brightness step value of each projection assembly based on the highest projection brightness of each projection assembly and the bit depth of each projection assembly.

In embodiments of the present application, the brightness step value for each projection component may be determined based on the highest projection brightness for each projection component and the bit depth for each projection component.

Taking a certain projection component i in the projection system as an example, the highest projection brightness of the projection component i is recorded as L _i Let the bit depth of the projection component i be n _i Taking the bit depth of the target projection image as 10 bits as an example, then n _i For example, it may be 8-bit, L _i Can be obtained by the method described in the above embodiment. In some embodiments, the bit depth of a projection component may be used to represent the number of brightness steps that the projection component may project an image, and the brightness step value of the projection component represents the highest brightness value that each brightness step of the projection component may display, e.g., the bit depth n of projection component i _i It is understood that the projection module i has

A brightness step, the brightness step value of the projection module i can then be ≥>

Step S530: and determining the brightness step value of the projection system based on the highest projection brightness of the projection system and the bit depth of the target projection image.

In embodiments of the present application, the brightness step value of the projection system may be determined based on the highest projection brightness of the projection system and the bit depth of the target projection image.

As can be seen from the foregoing embodiments, the projection areas of each projection module in the projection system displaying the image to be projected may coincide, and therefore the maximum projection brightness that can be displayed by the projection system can be obtained according to the maximum projection brightness of each projection module. In some embodiments, the sum of the maximum projection brightness of each projection assembly may be used as the maximum projection brightness of the projection system. For example, if there are k projection modules in the projection system, and the maximum projection brightness of each projection module is L1, L2, \8230;, lk, respectively, the maximum projection brightness of the projection system composed of the k projection modules may be L = L1+ L2+ \8230; + Lk.

Similar to the calculation method of the brightness step value of each projection component, if it is determined that the maximum projection brightness of the projection system is L and the bit depth of the target projection image is n, since the bit depth can be used to represent the number of brightness steps of the image, the brightness step value of the projection system can be understood as the maximum brightness value that each brightness step of the entire projection system can display, that is, the brightness step value of the projection system can be obtained by dividing the maximum projection brightness L of the projection system by the number of brightness steps of the target projection image, i.e., the brightness step value of the projection system can be L/2^ n.

Step S420: and determining the display brightness of each image to be projected based on the display brightness of the target projection image, the brightness step value of the projection system and the brightness step value of the projection assembly, wherein the display brightness of the target projection image is equal to the sum of the display brightness of each image to be projected.

In the embodiment of the present application, since the display brightness of the target projection image is equal to the sum of the display brightness of each of the to-be-projected images, and based on the meaning of the brightness step value (for example, the brightness step value of the projection system may be understood as the highest brightness value that can be displayed by each brightness step of the entire projection system), the display brightness of each to-be-projected image may be finally determined and obtained based on the display brightness of the target projection image, the brightness step value of the projection system, and the brightness step value of the projection assembly.

Alternatively, in order to better understand the calculation process of the display luminance of each image to be projected, the concept of pixel values will be introduced for explanation.

In this embodiment of the application, the pixel value may be used to represent a color depth of each pixel in an image, and exemplarily, the pixel value may be a gray-scale value of the pixel, or a gray-scale luminance obtained by performing gamma decoding on the gray-scale value. It is understood that the pixel value of each pixel in an image may determine a specific frame of an image.

In some embodiments, the display brightness of the target projection image is equal to the product of the brightness step value of the projection system and the pixel value of the target projection image, and the display brightness of each image to be projected is equal to the product of the brightness step value of each projection assembly and the pixel value of each projection assembly.

The process of determining the display luminance of the target projection image will be described below, taking the grayscale luminance P of the pixel a on the target projection image as an example. The pixel value of the image may also be understood as the brightness of the color, i.e. the specific step number of the display brightness, and then the pixel value (taking the gray-scale brightness P as an example) is multiplied by the brightness step value of the projection system to obtain the display brightness of the pixel a of the target projection image in the projection system, and the display brightness of the pixel a may be P × L/2, for example ⁿ . Based on the method, the display brightness of all the pixels in the target projection image can be determined, and the display brightness of all the pixels in the target projection image is taken as the display brightness of the target projection image.

In some embodiments, the pixel value of each pixel point in the image to be projected may determine the display brightness of the image to be projected, and further determine the specific picture presented by the image to be projected. Therefore, to obtain the display brightness of the image to be projected by each projection component in the projection system, the pixel value of each pixel point in each image to be projected can be determined first.

In the embodiment of the application, the display brightness of the image to be projected by each projection assembly in the projection system can also be determined according to the pixel value of each pixel point in the image to be projected and the brightness step value of the projection assembly, wherein in order to ensure that the coincident image seen by human eyes and the original target projection image present a consistent display effect, the display brightness of the target projection image should be equal to the sum of the display brightness of each image to be projected. Therefore, in this embodiment, the pixel value of each pixel point in the image to be projected by each projection component may be determined according to the display brightness of the target projection image and the brightness step value of the projection component.

In some embodiments, similar to the calculation of the display brightness of the target projection image, for the pixel a ' of the image j to be projected, where the pixel a ' corresponds to the pixel a of the target projection image, the display brightness of the pixel a ' may be calculated from the grayscale brightness P of the pixel a _j And the luminance step value of the projection unit projecting the image to be projected. Taking k projection assemblies and one projection assembly correspondingly projecting one image to be projected as an example, since the coincidence brightness requires the display brightness of the target projection image to be consistent, as shown in fig. 6, the brightness relationship can be obtained as follows: the display luminance 1 of the image to be projected 1, the display luminance 2 of the image to be projected 2, \8230, and the sum of the display luminances k of the image to be projected being equal to the display luminance of the target projection image, the equations can be listed as follows:

P×L/2 ⁿ ＝P ₁ ×L ₁ /2 ⁿ¹ +P ₂ ×L ₂ /2 ⁿ² +…+P _k ×L _k /2 ^nk

in addition, P is ₁ 、P ₂ 、…、P _k Respectively representing the gray-scale brightness of the pixel A' of each of the k images to be projected corresponding to the pixel A of the target projection image (where the pixel value may represent gamma decoding)The obtained gray scale brightness), n1, n2, \8230, nk respectively represent the bit depth of k projection components, L ₁ 、L ₂ 、…、L _k Respectively, the maximum projection brightness of k projection modules.

In some embodiments, the luminance step value L of the first projection module can be set as a positive integer ₁ /2 ⁿ¹ Display luminance P × L/2 of target projection image as divisor ⁿ Obtaining the gray scale brightness P obtained by performing gamma decoding on the pixel value of the pixel A' of the image to be projected of the projection assembly through modulus taking ₁ And a remainder T1, followed by a luminance step L of the second projection element ₂ /2 ⁿ² Taking the modulus of the remainder T1 as a divisor to obtain the gray scale brightness P obtained by gamma decoding the pixel value of the pixel A' of the image to be projected of the projection assembly ₂ And the remainder T2, and so on until obtaining the brightness step value L of the kth projection component _k /2 ^nk Taking the remainder obtained by the last calculation as a divisor, obtaining the gray scale brightness P obtained by gamma decoding the pixel value of the pixel A' of the image to be projected of the projection assembly _k . Finally, can be respectively P ₁ 、P ₂ 、…、P _k The pixel value of the pixel a' of each image to be projected can be obtained.

Based on the method, the pixel value of each pixel point in each image to be projected can be determined. In particular, each pixel in each image to be projected corresponds to one pixel in the target projection image.

In some embodiments, since the maximum projection brightness and the bit depth of each projection module are different, different calculation orders for each projection module may result in different images to be projected. In some exemplary embodiments, the projection component with the largest brightness step value in the projection system may be used as the main projection, and the pixel value of each pixel point in the image to be projected corresponding to the main projection is first calculated.

In other embodiments, the pixel value of each pixel of the target projection image and the image to be projected may include at least pixel components of three color channels, and for each color channel of each image to be projected, the pixel component of each color channel of each image to be projected may be determined based on the display luminance corresponding to each color channel and the luminance step value of the image to be projected. Taking the RGB color mode as an example, the target projection image and the image to be projected are both divided into a first component of a red channel, a second component of a green channel, and a third component of a blue channel in the RGB color mode.

It will be appreciated that the display brightness of the target projection image may comprise a display brightness corresponding to each of the color channels, and thus, in the target projection image, the display brightness corresponding to each color channel may be equal to the product of the brightness step value of the projection system and the pixel component corresponding to that color channel. Similarly, the display brightness of each image to be projected may also include the display brightness corresponding to each color channel, and therefore, in each image to be projected, the display brightness corresponding to each color channel is equal to the product of the brightness step value of each projection component and the pixel component corresponding to the color channel.

Still taking a certain pixel a of the target projection image as an example, as shown in fig. 7, for three color components of the RGB color mode, the following three luminance relationships can be obtained for the k pixels a' to be projected images to be projected 1, 2, \ 8230for the image to be projected k:

R×L/2 ⁿ ＝R ₁ ×L ₁ /2 ⁿ¹ +R ₂ ×L ₂ /2 ⁿ² +…+R _k ×L _k /2 ^nk

G×L/2 ⁿ ＝G ₁ ×L ₁ /2 ⁿ¹ +G ₂ ×L ₂ /2 ⁿ² +…+G _k ×L _k /2 ^nk

B×L/2 ⁿ ＝B ₁ ×L ₁ /2 ⁿ¹ +B ₂ ×L ₂ /2 ⁿ² +…+B _k ×L _k /2 ^nk

wherein R is multiplied by L/2 ⁿ Display luminance, gxL/2, representing red channel of target projection image pixel A ⁿ Representing the green channel of a pixel A of a target projection imageDisplay luminance, B × L/2 ⁿ Display luminance, R, representing the blue channel of target projection image pixel A ₁ 、R ₂ 、…、R _k Respectively representing the gray-scale brightness, G, corresponding to the first component of the pixel A' in each image to be projected ₁ 、G ₂ 、…、G _k Respectively representing the gray scale brightness, B, corresponding to the second component of the pixel A' in each image to be projected ₁ 、B ₂ 、…、B _k Respectively representing the gray scale brightness corresponding to the third component of the pixel A' in each image to be projected. Optionally, since the pixel value is a positive integer, the first component, the second component, and the third component of the pixel a' in each image to be projected can be obtained by taking the brightness step value of each projection component modulo the display brightness corresponding to the three channels, respectively, in the manner in the above embodiment.

Also, based on the above method, the pixel value may be determined from the pixel component of each pixel in each image to be projected.

It is to be understood that, if the target projection image is different from the color pattern or the included color components of the projection component, for example, the target projection image is in RGB mode, and the projection component is in YUV mode, after the pixel value of the image to be projected in the color pattern or the included color components of the target projection image is calculated, the pixel value of the image to be projected may also be converted into the pixel value of the color pattern or the corresponding color components adopted by the projection component.

According to the method and the device, the pixel value of each pixel point in each image to be projected can be obtained through calculation, and then the pixel value of each pixel point in the image to be projected can be multiplied by the brightness step value to obtain the display brightness of each image to be projected corresponding to each projection component. Illustratively, still taking the gray-scale brightness as the pixel value, after obtaining the gray-scale brightness of the pixel point a in the image 1 to be projected, the display brightness of the pixel point a may be expressed as a product of the gray-scale brightness and the brightness step value of the corresponding projection component, and the display brightness of the pixel point a may be, for example, P ₁ ×L ₁ /2 ⁿ¹ 。

Based on the method, the display brightness of all pixels in the image to be projected can be determined, and the display brightness of all pixels in the image to be projected is taken as the display brightness of the image to be projected.

In some embodiments, the projection component may project the images to be projected at a display brightness of each image to be projected.

In some embodiments, before controlling the projection assemblies to project the images to be projected, overlap correction may be performed on each projection assembly, and after the overlap correction, corresponding pixels on all the images to be projected that correspond to the same pixel on the target projection image fall to the same point on the projection area after being projected by each projection assembly, so that the projection areas of each projection assembly may coincide.

In some approaches, the overlay correction may be performed mechanically. For example, a fixed display area may be determined, for example, the position of the projection screen is used as the display area of each projection component, and then the position of each projection component is moved in sequence, and the projection area of each projection component is aligned with the display area accurately, that is, the projection area of each projection component is ensured to coincide. In other embodiments, overlay correction may be performed digitally. For example, a fixed display area may be determined, the projection area of each projection assembly may be adjusted to completely cover the display area, and then the projection area and the display area may be precisely aligned using four-point correction or eight-point correction. In some exemplary embodiments, both of the above correction methods may be used simultaneously, such as a coarse adjustment mechanically followed by a fine adjustment digitally.

Alternatively, if the lens of the projection assembly has large distortion, the distortion may also be corrected. For example, a checkerboard or other image is projected onto a flat projection screen, a calibrated camera or measurement instrument is used to capture pixel distortion of the checkerboard or other image, and the checkerboard or other image is corrected based on the pixel distortion.

In the embodiment of the application, after the overlay correction, the manner of projecting the image to be projected by the projection assembly may be controlled by referring to the content described in the foregoing embodiment, and the projected multiple images to be projected are displayed in a superimposed manner, and a condition that the superimposed brightness obtained by superimposing the multiple images to be projected is consistent with the display brightness of the target projection image is satisfied, so that a display effect that the superimposed image seen by human eyes and the original target projection image present a consistent display effect may be achieved.

To sum up, the embodiment of the present application provides a technical solution, first obtain a target projection image, then obtain the highest projection brightness and the bit depth of a projection component, and then obtain the highest projection brightness of a projection system based on the highest projection brightness of each projection component. And then determining the display brightness of the target projection image according to the highest projection brightness of the projection system, the bit depth of the target projection image and the pixel value of each pixel point in the target projection image. If the bit depth of the target projection image is higher than the bit depth of each projection assembly, the highest projection brightness of each projection assembly is obtained, then the brightness step value of each projection assembly is determined based on the highest projection brightness of each projection assembly and the bit depth of each projection assembly, and the pixel value of each pixel point in each to-be-projected image corresponding to each projection assembly is determined based on the display brightness of the target projection image and the brightness step value of each projection assembly. And finally, controlling the projection assembly to project each to-be-projected image corresponding to each projection assembly at the display brightness, so that the plurality of to-be-projected images are displayed in a superposition manner, and the sum of the display brightness of each to-be-projected image is equal to the display brightness of the target projection image. This application is through dividing into a plurality of projection images of treating with the image that the bit depth is higher than the projection subassembly, uses many projection subassemblies to throw, can reach the purpose that uses the projection subassembly projection high bit depth image of lower bit depth, and a plurality of projection subassembly coincidence projections have still enlarged whole projection system's projection luminance scope simultaneously.

Referring to fig. 8, a block diagram of an image processing apparatus according to an embodiment of the present application is shown. In particular, the image processing apparatus is applied to a processor of a projection system, the projection system comprises at least two projection components, and the apparatus comprises: an acquisition module 810, a processing module 820, and a control module 830.

An acquisition module 810 for acquiring a target projection image; a processing module 820, configured to determine, based on the target projection image, a plurality of to-be-projected images corresponding to the at least two projection assemblies if the bit depth of the target projection image is higher than the bit depth of each projection assembly, so that a sum of display brightness of each to-be-projected image is equal to display brightness of the target projection image; the control module 830 is configured to control each projection assembly to project each to-be-projected image corresponding to each projection assembly, so that the multiple to-be-projected images are displayed in a superposition manner, where each to-be-projected image corresponds to one projection assembly.

Optionally, the processing module 820 may include: the brightness step acquisition module is used for acquiring the brightness step value of each projection assembly and the brightness step value of the projection system; and the display brightness determining module is used for determining the display brightness of each image to be projected based on the display brightness of the target projection image, the brightness step value of the projection system and the brightness step value of the projection assembly, wherein the display brightness of the target projection image is equal to the sum of the display brightness of each image to be projected.

Further, the display brightness of the target projection image is equal to the product of the brightness step value of the projection system and the pixel value of the target projection image, and the display brightness of each image to be projected is equal to the product of the brightness step value of each projection assembly and the pixel value of each projection assembly.

In some embodiments, the brightness step acquisition module may include: the component acquisition module is used for acquiring the highest projection brightness of each projection component and the bit depth of each projection component; a first determining module, configured to determine a brightness step value of each projection component based on a highest projection brightness of each projection component and a bit depth of each projection component; and the second determination module is used for determining the brightness step value of the projection system based on the highest projection brightness of the projection system and the bit depth of the target projection image.

Optionally, the maximum projection brightness of the projection system is equal to the sum of the maximum projection brightness of each projection component.

In some embodiments, the pixel value of each of the images to be projected includes at least pixel components of three color channels, and the display brightness of each of the images to be projected includes a display brightness corresponding to each of the color channels. Based on this, in each image to be projected, the display brightness corresponding to each color channel is equal to the product of the brightness step value of each projection assembly and the pixel component corresponding to the color channel.

In other embodiments, the pixel values of the target projection image include at least pixel components of three color channels, and the display brightness of the target projection image includes a display brightness corresponding to each of the color channels. In the target projection image, the display brightness corresponding to each color channel is equal to the product of the brightness step value of the projection system and the pixel component corresponding to the color channel.

Optionally, the image processing apparatus may further include: and the correction module is used for performing overlapping correction on each projection assembly to enable the projection area of each projection assembly to be overlapped.

Optionally, the component obtaining module may include: and the measuring module is used for controlling each projection component to project at the highest brightness, and measuring the brightness of the projection area of each projection component to obtain the highest projection brightness of each projection component.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

Referring to fig. 9, a block diagram of an electronic device provided in an embodiment of the present application is shown, specifically, one or more of the following components of the electronic device: a processor 910, a memory 920, and one or more applications. One or more applications may be stored in the memory 920 and configured to be executed by the one or more processors 910, the one or more applications configured to perform the methods as described in the aforementioned method embodiments.

The electronic device may be any of various types of computer system devices that are mobile, portable, and perform wireless communications. Specifically, the electronic device may be a mobile phone or a smart phone (e.g., an iPhone-based phone), a Portable game device (e.g., nintendo DS, playStation Portable, game Advance, iPhone), a laptop computer, a PDA, a Portable internet device, a music player, a data storage device, a projection device that integrates the image processing apparatus described herein, and the like, other handheld devices and devices such as a smart watch, a smart bracelet, an earphone, a pendant, and the like, and other wearable devices (e.g., such as electronic glasses, electronic clothes, an electronic bracelet, an electronic necklace, an electronic tattoo, an electronic device, or a Head Mounted Device (HMD)).

The electronic device may also be any of a number of electronic devices including, but not limited to, cellular phones, smart watches, smart bracelets, other wireless communication devices, personal digital assistants, audio players, other media players, music recorders, video recorders, cameras, other media recorders, radios, medical devices, vehicle transport equipment, calculators, programmable remote controls, pagers, laptop computers, desktop computers, printers, netbook computers, personal Digital Assistants (PDAs), portable Multimedia Players (PMPs), moving picture experts group (MPEG-1 or MPEG-2) audio layer 3 (MP 3) players, portable medical devices, projection devices, and digital cameras, and combinations thereof.

In some cases, the electronic device may perform a variety of functions (e.g., playing music, displaying videos, storing pictures, and receiving and sending telephone calls). The electronic device may be, for example, a cellular telephone, media player, other handheld device, wristwatch device, pendant device, earpiece device, or other compact portable device, if desired.

Processor 910 may include one or more processing cores. The processor 910 interfaces with various components throughout the electronic device using various interfaces and lines to perform various functions of the electronic device and to process data by executing or performing instructions, applications, code sets, or instruction sets stored in the memory 920 and invoking data stored in the memory 920. Alternatively, the processor 910 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 910 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 910, but may be implemented by a communication chip.

The Memory 920 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 920 may be used to store instructions, applications, code sets, or instruction sets. The memory 920 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, and the like), and is configured to obtain a target projection image, determine a plurality of images to be projected corresponding to the at least two projection components based on the target projection image if a bit depth of the target projection image is higher than a bit depth of each projection component, so that a sum of display brightness of each image to be projected is equal to display brightness of the target projection image, and then control each projection component to project each image to be projected corresponding to each projection component, so that the plurality of images to be projected are displayed in a superposition manner, where each image to be projected corresponds to one projection component. The data storage area can also store data (such as a phone book, audio and video data, chatting record data) and the like created by the electronic equipment in use.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the processor 910 and the memory 920 of the electronic device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

It should be noted that, a person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by the relevant hardware instructed by a computer program, and the computer program can be stored in a computer readable storage medium, which can include but is not limited to: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, and the like.

Referring to fig. 10, a block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. The computer-readable storage medium 1000 has stored therein a program code 1010 that can be called by a processor to execute the image processing method described in the above-described method embodiments.

The computer-readable storage medium 1000 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 1000 includes a non-volatile computer-readable storage medium. The computer readable storage medium 1000 has storage space for program code 1010 for performing any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 1010 may be compressed, for example, in a suitable form.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments. In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium.

While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are exemplary and should not be construed as limiting the present application and that changes, modifications, substitutions and alterations in the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

The terms "first" and "second," and the like, in the description and in the claims of embodiments of the present invention are used for distinguishing between different objects and not for describing a particular order of the objects. For example, the first region and the second region, etc. are for distinguishing different regions, and are not for describing a particular order of the regions. In the description of the embodiments of the present invention, the meaning of "a plurality" means two or more unless otherwise specified.

The term "and/or" herein is an association relationship describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The symbol "/" herein denotes a relationship in which the associated object is or, for example, a/B denotes a or B.

In embodiments of the present invention, words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present application, and are not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. An image processing method applied to a processor of a projection system, the projection system comprising at least two projection assemblies, the method comprising:

acquiring a target projection image;

if the bit depth of the target projection image is higher than that of each projection component, determining a plurality of projection images to be projected corresponding to the at least two projection components based on the target projection image so as to enable the sum of the display brightness of each projection image to be equal to that of the target projection image;

and controlling each projection component to project each image to be projected corresponding to each projection component, so that the plurality of images to be projected are displayed in a superposition manner, wherein each image to be projected corresponds to one projection component.

2. The method of claim 1, wherein determining a plurality of images to be projected corresponding to the at least two projection assemblies based on the target projection image comprises:

acquiring a brightness step value of each projection component and a brightness step value of the projection system;

and determining the display brightness of each image to be projected based on the display brightness of the target projection image, the brightness step value of the projection system and the brightness step value of the projection assembly, wherein the display brightness of the target projection image is equal to the sum of the display brightness of each image to be projected.

3. The method of claim 2,

the display brightness of the target projection image is equal to the product of the brightness step value of the projection system and the pixel value of the target projection image;

the display brightness of each image to be projected is equal to the product of the brightness step value of each projection assembly and the pixel value of each projection assembly.

4. The method of claim 2, wherein obtaining the brightness step value for each projection module and the brightness step value for the projection system comprises:

acquiring the highest projection brightness of each projection assembly and the bit depth of each projection assembly;

determining a brightness step value of each projection assembly based on the highest projection brightness of each projection assembly and the bit depth of each projection assembly;

and determining the brightness step value of the projection system based on the highest projection brightness of the projection system and the bit depth of the target projection image.

5. The method of claim 4, wherein the maximum projection brightness of the projection system is equal to the sum of the maximum projection brightness of each projection unit.

6. The method according to claim 3, wherein the pixel value of each of the images to be projected includes at least pixel components of three color channels, the display brightness of each of the images to be projected includes a display brightness corresponding to each of the color channels, and the display brightness of each of the images to be projected is equal to a product of a brightness step value of each of the projection components and the pixel value of each of the projection components, and the method comprises:

in each image to be projected, the display brightness corresponding to each color channel is equal to the product of the brightness step value of each projection assembly and the pixel component corresponding to the color channel.

7. The method of claim 3 wherein the pixel values of the target projection image include at least pixel components of three color channels, the display brightness of the target projection image includes a display brightness corresponding to each of the color channels, the display brightness of the target projection image is equal to the product of the brightness step value of the projection system and the pixel values of the target projection image, comprising:

in the target projection image, the display brightness corresponding to each color channel is equal to the product of the brightness step value of the projection system and the pixel component corresponding to the color channel.

8. The method according to claim 1, wherein before controlling each of the projection assemblies to project each of the images to be projected corresponding to each of the projection assemblies so that the plurality of images to be projected are displayed in superposition, the method further comprises:

and performing overlapping correction on each projection assembly to enable the projection area of each projection assembly to coincide.

9. The method of claim 4, wherein said obtaining a maximum projection brightness for each of said projection assemblies comprises:

and controlling each projection assembly to project at the highest brightness, and measuring the brightness of the projection area of each projection assembly to obtain the highest projection brightness of each projection assembly.

10. An image processing apparatus, for use in a processor of a projection system, the projection system including at least two projection components, the apparatus comprising:

the acquisition module is used for acquiring a target projection image;

the processing module is used for determining a plurality of images to be projected corresponding to the at least two projection assemblies based on the target projection image if the bit depth of the target projection image is higher than the bit depth of each projection assembly, so that the sum of the display brightness of each image to be projected is equal to the display brightness of the target projection image;

and the control module is used for controlling each projection component to project each image to be projected corresponding to each projection component, so that the plurality of images to be projected are displayed in a superposition manner, wherein each image to be projected corresponds to one projection component.

11. An electronic device, comprising a processor, a memory, the memory storing a computer program, the processor being configured to execute the image processing method according to any one of claims 1 to 9 by calling the computer program.

12. A projection system comprising a processor and at least two projection components, the processor being configured to perform the method of any of claims 1 to 9.

13. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the method of any of claims 1 to 9.