JP2021061510A - Projection control device, method, program, and storage medium - Google Patents

Abstract

To provide a projection control device that can detect pixel shifts of a plurality of projectors without using a special test pattern.SOLUTION: A projection control device includes an acquisition unit that acquires a captured image obtained by capturing a region of a projection surface including at least a superimposed region in which a first projection image projected on the projection surface by a first projector and a second projection image projected on the projection surface by a second projector overlap, and a detection unit that analyzes the captured image, and detects the magnitude and the direction of the displacement between the first projection image and the second projection image in the superimposed region.SELECTED DRAWING: Figure 2

Description

The present invention relates to a projection control device that detects a deviation of a plurality of projected images projected by a plurality of projectors.

When an image is projected from a projector onto a screen and displayed, pixels may be projected in an overlapping manner using a plurality of projectors in order to improve the projection brightness or the projection resolution.

In Patent Document 1, in order to analyze the pixel overlap of a projection using a plurality of projectors, a test pattern is used to detect the pixel overlap deviation (hereinafter referred to as pixel deviation) and the plurality of projectors. A technique for aligning the projection position of is disclosed.

Japanese Unexamined Patent Publication No. 2011-182076

However, in the above-mentioned conventional technique, in order to detect the pixel shift of the projection display of a plurality of projectors, a function of displaying a unique test pattern that can identify each projector is required, which causes a problem that the configuration of the device becomes complicated. was there.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a projection control device capable of detecting pixel deviations of a plurality of projectors without using a special test pattern.

The projection control device according to the present invention is a region of the projection surface including at least an overlapping region in which the first projection image projected on the projection surface by the first projector and the second projection image projected on the projection surface by the second projector overlap. The acquisition means for acquiring the captured image obtained by capturing the image and the captured image are analyzed to detect the magnitude of the displacement and the direction of the displacement between the first projected image and the second projected image in the superimposed region. It is characterized in that it is provided with a detection means for

According to the present invention, it is possible to detect pixel shifts of a plurality of projectors without using a special test pattern.

The figure which shows the structure of the projection system which concerns on 1st Embodiment of this invention. A block diagram showing the configuration of a projector. The conceptual diagram for demonstrating the pixel shift characteristic. The conceptual diagram for demonstrating the pixel shift characteristic. A flowchart showing the processing of the projector. The flowchart which shows the pixel shift correction processing in 2nd Embodiment. The flowchart which shows the image division processing in 3rd Embodiment. The conceptual diagram of the divided image in 3rd Embodiment. The figure which shows the structure of the projector in 4th Embodiment. The flowchart which shows the processing of the projector in 4th Embodiment. The flowchart which shows the processing of the projector in 5th Embodiment. Conceptual diagram of multi-projection and blend section. The figure which shows the structure of the image projection system of 6th Embodiment. The conceptual diagram of the user interface in the 7th Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

(First Embodiment)
FIG. 1 is a diagram showing a configuration of a projection system including a plurality of projectors according to the first embodiment of the present invention. The projection system 160 includes a projector 100a, a projector 100b, a screen 150, and a signal source 152.

The projector 100a and the projector 100b are projection devices that project a projected image onto a screen based on image data input from the signal source 152. Details of the functions of the projector 100a and the projector 100b will be described later. When the projector 100a and the projector 100b are described together, it is referred to as the projector 100. When each is shown individually, it is referred to as a projector 100a and a projector 100b. The projection system 160 is a stack projection system in which the projection image A projected by the projector 100a and the projection image B projected by the projector 100b are superimposed and projected on the screen.

The screen 150 is a projection surface on which an image is projected by the projector 100a and the projector 100b. The screen 150 may be a wall surface or may not be a flat surface.

The signal source 152 is an output device that outputs image data to be input to the projector 100a and the projector 100b. The signal source 152 is, for example, a PC or a video distributor.

In this embodiment, two projectors 100a and 100b are shown as a plurality of projectors, but it goes without saying that three or more projectors may be used.

FIG. 2 is a block diagram showing the configuration of the projector 100.

The projector 100 includes an imaging unit 101, a Fourier transform unit 102, a pixel shift analysis unit 103, an output unit 104, a memory 105, an image input unit 110, an image processing unit 111, a light source 112, a panel 113, a projection optical system 114, a CPU 120, and a bus. 121 is provided.

The imaging unit 101 can capture the periphery of the screen 150, and captures an image projected on the screen 150 by a plurality of projectors 100 (projectors 100a and 100b in this embodiment) to generate an captured image. The imaging unit 101 acquires an image captured by capturing an area of the screen 150 including a region (overlapping area) in which a plurality of images projected on the screen 150 by a plurality of projectors 100 overlap each other. In the present embodiment, the entire image projected by the projector 100a and the entire image projected by the projector 100b are projected so as to overlap each other. Therefore, the imaging unit 101 captures the images projected by the projectors 100a and 100b, respectively. The imaging unit 101 outputs the captured image to the Fourier transform unit 102.

The image pickup unit 101 is obtained by a lens that acquires an optical image of a subject, an actuator that drives the lens, a microprocessor that controls the actuator, an image pickup element that converts an optical image acquired via the lens into an image signal, and an image pickup element. It is provided with an AD conversion unit or the like that converts the generated image signal into a digital signal. The image pickup unit 101 may be provided as an external device (imaging device) of the projector 100. In that case, the projector 100 includes a connection interface that connects to the image pickup unit 101, and acquires an captured image from the image pickup unit 101 via the connection interface.

The Fourier transform unit 102 performs a two-dimensional Fourier transform after acquiring an image obtained by capturing a projected image of the screen from the image pickup unit 101, generates a frequency spectrum, and outputs the frequency spectrum to the pixel shift analysis unit 103.

The resolution of the captured image acquired by the imaging unit 101 may be higher than the resolution of the projected image. In such a case, the photographing unit 101 stores the photographed image in which the range including the periphery of the screen is photographed in the memory 105, and then the CPU 120 cuts out the projected image portion from the photographed image and outputs the projected image portion to the Fourier transform unit 102. Then, the Fourier transform unit 102 converts the cut out image into a frequency spectrum and outputs it to the pixel shift analysis unit 103.

The pixel shift analysis unit 103 analyzes the frequency spectrum generated by the Fourier transform unit 102 and detects the pixel shift projected by a plurality of projectors. The pixel shift analysis unit 103 detects the pixel shift by comparing the pixel shift characteristic, which will be described later, with the frequency spectrum generated by the Fourier transform unit 102, for example, by pattern matching.

The output unit 104 notifies the outside of the pixel shift detection result of the pixel shift analysis unit 103. For example, the pixel shift detection result is transmitted to an external server via a network (not shown). The memory 105 temporarily stores a control program and data as a work memory.

The image input unit 110 has a function of receiving a video signal from an external device (signal source 152 in FIG. 1), and is, for example, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal or an SDI (Serial Digital Interface) terminal. , DisplayPort® (DP) terminal and the like. Here, the external device is a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, a game machine, or the like that outputs a video signal. The image input unit 110 transmits the received video signal to the image processing unit 111.

The image processing unit 111 has a function of changing the number of frames, the number of pixels, the image shape, etc. of the video signal received from the video input unit 110 and transmitting it to the panel 113, for example, from a microprocessor for image processing. Become. The image processing unit 111 does not have to be a dedicated microprocessor. For example, the CPU 120 may execute the same processing as the image processing unit 111 by a program stored in the memory 105. The image processing unit 111 can execute functions such as frame thinning processing, frame interpolation processing, resolution conversion processing, OSD superimposition processing such as menus, distortion correction processing (keystone correction processing), and edge blending processing. In addition to the video signal received from the video input unit 110, the image processing unit 111 can also perform the above-mentioned change processing on the image or video reproduced by the CPU 110.

The light source 112 is a light source that outputs (irradiates) light to the panel 113. The light source 112 may be any member as long as it can emit light, and may be, for example, a halogen lamp, a xenon lamp, a high-pressure mercury lamp, an LED (Light Emitting Diode), a laser, or the like.

The panel 113 is a modulation panel that modulates the light emitted from the light source 112 and outputs an image (light representing an image). The panel 113 transmits the light emitted from the light source 112 at a transmittance (transmittance distribution) based on the image data input from the image processing circuit 111. Alternatively, the panel 113 reflects the light emitted from the light source 112 with a reflectance (reflectance distribution) based on the image data input from the image processing circuit 111. In the present embodiment, it is assumed that the panel 113 is a transmissive liquid crystal panel. The panel 113 is not limited to the liquid crystal panel.

The projection optical system 114 is an optical system for projecting the image output from the panel 113 onto the screen 150. The projection optical system 114 is composed of a plurality of lenses, an actuator for driving the lens, and the like. By driving the lens by the actuator, the projection image is enlarged / reduced, the projection position is shifted, and the focal point of the projection optical system 114 is focused. Make adjustments. The projected image is an image displayed on the screen by projecting the image output from the panel 113, and the projection position is a position on the screen 150 on which the image output from the panel 113 is projected.

The CPU 120 is a processor for controlling the operation of the projector 100. The CPU 120 executes a program read from the memory 105 to execute a control (function) described later. The CPU 120 may be composed of a plurality of processors. Further, a part or all of the control executed by the CPU 120 may be executed by an electronic circuit without using a program.

Subsequently, the pixel shift characteristic will be described with reference to FIG. Pixel overlap and pixel misalignment characteristics when pixels are projected on top of each other using two projectors 100a and 100b will be described.

It is assumed that the pixels 200 of the projected image projected by the projector 100a are projected on the screen 150 so as to be superimposed on the corresponding pixels 201 of the projected image projected by the projector 100b.

However, the projection position of the projected image may shift due to changes in the projection position of the projector 100 over time. In this case, the projection positions of the pixel 200 and the pixel 201 do not match.

3A to 3E are schematic views showing the relationship between the projection positions of the pixel 200 and the pixel 201, respectively. Needless to say, pixels other than the pixels 200 and 201 are also projected on the screen 150.

FIG. 3A shows the pixels 200 of the projector 100a (circles filled with white) and the pixels 201 of the projector 100b (diagonal lines) when the two projectors 100a and 100b project the pixels on the screen 150 so as to overlap each other. Indicates a state in which (circles) overlap (a state in which there is no pixel shift).

FIG. 3B shows a state in which the pixels 200 and the pixels 201 are vertically displaced. FIG. 3C shows a state in which the pixels 200 and the pixels 201 are displaced in the horizontal direction. FIG. 3D shows a state in which the pixels 200 and the pixels 201 are displaced in the oblique direction. FIG. 3E shows a state in which the pixels 200 and the pixels 201 are displaced in an oblique direction different from that in FIG. 3D.

The pixel shift characteristic can be determined from an image (frequency spectrum image) obtained by applying a two-dimensional Fourier transform process to the captured image. 3 (f) to 3 (j) show frequencies when the positions of the projected image projected by the projector 100a and the projected image projected by the projector 100b are in the relationship shown in FIGS. 3 (a) to 3 (e), respectively. A spectrum image is shown.

Frequency spectrum Near the center of the image shows the intensity of the high frequency component of the image shift. In addition, the intensity of the low frequency component is shown near the four corners of the frequency spectrum image. In the following, when the frequency spectrum is divided into four, the frequency spectrum in which the first quadrant and the third quadrant and the second quadrant and the fourth quadrant are exchanged will be referred to as a pixel shift characteristic. By this replacement, a low frequency component is contained near the center, and a high frequency component is contained as the distance from the center increases.

FIG. 3 (f) shows a frequency spectrum image in a state where the projected images of the two projectors 100a and 100b are overlapped on the screen 150 (a state without pixel deviation) as shown in FIG. 3 (a).

FIG. 3 (g) shows a frequency spectrum image in a state where the projected images of the two projectors 100a and 100b are vertically displaced on the screen 150 as shown in FIG. 3 (b). At this time, a dark line in the horizontal direction appears in the frequency spectrum image.

FIG. 3H shows a frequency spectrum image in a state where the projected images of the two projectors 100a and 100b are horizontally displaced on the screen 150 as shown in FIG. 3C. At this time, a dark line in the vertical direction appears in the frequency spectrum image.

FIG. 3 (i) shows a frequency spectrum image in a state where the projected images of the two projectors 100a and 100b are shifted in the licking direction on the screen 150 as shown in FIG. 3 (d). At this time, a dark line in the licking direction appears in the frequency spectrum image.

FIG. 3J shows a frequency spectrum image in a state where the projected images of the two projectors 100a and 100b are shifted in the licking direction on the screen 150 as shown in FIG. 3E. At this time, a dark line in the licking direction appears in the frequency spectrum image.

That is, when the projected image is deviated, a dark line in the direction perpendicular to the deviating direction appears in the frequency spectrum image.

Further, there is a correlation not only in the direction in which the pixels are displaced, but also in the amount of pixel-to-pixel deviation and the pixel deviation characteristic. 4 (a) to 4 (e) show frequency spectrum images when the pixels 201 and 202 are displaced in the horizontal direction, respectively. FIG. 4A shows a pixel shift characteristic when the pixel is shifted by one pixel in the horizontal direction. FIG. 4B shows a pixel shift characteristic when the pixel is shifted by two pixels in the horizontal direction. FIG. 4C shows a pixel shift characteristic when the pixel is shifted by 4 pixels in the horizontal direction. FIG. 4D shows a pixel shift characteristic when the pixel is shifted by 8 pixels in the horizontal direction. FIG. 4E shows the pixel shift characteristic when the shift is 10 pixels in the horizontal direction. That is, it can be seen that the larger the deviation amount, the larger the number of dark lines appearing in the frequency spectrum image. Similarly, there is a correlation between the amount of pixel shift and the change in dark lines in the vertical and diagonal directions.

Therefore, the direction of deviation of the projected image on the screen 150 can be determined from the frequency spectrum image based on the captured image obtained from the Fourier transform unit 102. Specifically, a plurality of frequency spectrum images (reference images) associated with various magnitudes and directions of deviations in advance are stored in a storage medium such as a memory. Then, by comparing the frequency spectrum image based on the captured image obtained from the Fourier transform unit 102 with the reference image, the magnitude and deviation of the projected image on the screen 150 can be determined. For comparison between the frequency spectrum image and the reference image, a known comparison method such as pattern matching can be adopted.

The relationship between the characteristics of the frequency spectrum image and the magnitude and direction of the deviation is not limited to the above example. The appearance of features can change by changing the calculation method.

FIG. 5 is a flowchart showing a deviation notification process executed by the projector 100. The deviation notification process is executed in a state where the projector 100 is projecting an image by stack projection. It is assumed that the deviation notification process is repeatedly executed when the user has previously set to enable the deviation notification function. The deviation notification process may be executed by both the projectors 100a and 100b, or may be executed by either one. In the present embodiment, it is assumed that the deviation notification function of the projector 100a is enabled and the deviation notification function of the projector 100b is disabled.

In S1, the imaging unit 101 captures an image projected on the screen to generate a captured image.

In S2, the Fourier transform unit 102 performs a two-dimensional Fourier transform on the captured image generated in S1 to generate a frequency spectrum image.

In S3, the pixel shift analysis unit 103 acquires the pixel shift characteristics (magnitude, direction) of the projected images of the projectors 100a and 100b projected on the screen 150 from the frequency spectrum image generated in S2.

In S4, the CPU 120 determines whether or not the pixel shift has occurred based on the analysis result of the pixel shift analysis unit 103. When the CPU 120 detects the occurrence of pixel shift, the process proceeds to S5. If the CPU 120 does not detect the occurrence of pixel shift, the process proceeds to S6. The determination process of S4 may be a determination process of proceeding to S5 when the magnitude of the shift indicated by the pixel shift characteristic is equal to or greater than the threshold value, and proceeding to S6 when the magnitude of the shift is not equal to or greater than the threshold value.

In S5, the CPU 120 performs a process of notifying the occurrence of pixel shift. The CPU 120 outputs the detection result of the pixel shift analysis unit 103 to the external server via the output unit 104 via the network.

In S6, the CPU 120 determines whether or not the user has instructed the end of the shift notification process. For example, it is assumed that the end instruction is input by the user operating the menu screen or the like to disable (off) the deviation notification function. When the end instruction is input, the process of the flowchart of FIG. 4 is ended, and when the end operation is not performed, the process returns to S1.

As described above, according to the first embodiment, it is possible to detect the pixel shift of the stack projection while the plurality of projectors 100 continue the stack projection. In this method, it is not necessary to use a test pattern for detecting pixel deviation, so that even if pixel deviation is detected during viewing of a moving image, viewing is prevented from being hindered.

In the above embodiment, the configuration for detecting the pixel shift including the imaging unit 101, the Fourier transform unit 102, the pixel shift analysis unit 103, and the output unit 104 has been described as being mounted on the projector 100. However, all or part of the functional parts for detecting these pixel shifts may be mounted on an external control device of the projector 100. As the external control device, a control device created for projection control, a personal computer, or the like can be used.

(Second embodiment)
Hereinafter, a second embodiment of the present invention will be described. In the first embodiment, a method of detecting pixel shift without using a test pattern while continuing stack projection has been described. In the second embodiment, a configuration in which the pixel shift correction process is performed after the pixel shift detection of the first embodiment will be described. Since the configuration of the projector 100 of the second embodiment is the same as the configuration of the first embodiment shown in FIG. 2, the description thereof will be omitted.

The pixel shift correction process of the second embodiment will be described with reference to the flowchart of FIG. The deviation correction process is executed in a state where the projector 100 is projecting an image by stack projection. It is assumed that the deviation correction process is repeatedly executed when the user has previously set to enable the deviation correction function. The deviation correction process may be executed by both the projectors 100a and 100b, or may be executed by either one. In the present embodiment, it is assumed that the deviation correction function of the projector 100a is set to be valid and the deviation correction function of the projector 100b is set to be invalid. That is, the projector 100a detects the pixel shift characteristic and corrects the projection position of the projected image of the projector 100a according to the detected pixel shift characteristic.

In S100, the stack-projected screen is photographed in the same manner as in S1 of the first embodiment. In S101, the Fourier transform process of the captured image is performed in the same manner as in S2 of the first embodiment. In S102, the pixel shift analysis is performed in the same manner as in S3 of the first embodiment.

In S103, the CPU 120 acquires the pixel shift detection result from the pixel shift analysis unit 103, and determines whether or not the pixel shift has occurred. When the CPU 120 detects the occurrence of pixel shift, the process proceeds to S104. If the occurrence of pixel shift is not detected, the process proceeds to S106.

In S104, the CPU 120 acquires the pixel shift detection result from the pixel shift analysis unit 103, and estimates the pixel shift direction and the pixel shift amount.

In S105, pixel shift correction processing is performed. The CPU 120 adjusts the shift amount of the projection position of the projection optical system 114 based on the pixel shift direction and the pixel shift amount obtained in S104. It is also possible to correct the pixel deviation by using the distortion correction processing function of the image processing unit 111.

In S106, the CPU 120 determines whether or not the user has performed an end operation (an operation to end the flowchart of FIG. 6). When the end operation is performed, the flowchart of FIG. 6 ends, and when the end operation is not performed, the process returns to S100.

As described above, according to the second embodiment, it is possible to correct the pixel deviation of the stack projection while the plurality of projectors 100 continue the stack projection. In this method, since it is not necessary to use a test pattern for detecting pixel deviation, even if pixel deviation is detected during video viewing, viewing is prevented from being hindered.

(Third Embodiment)
Hereinafter, a third embodiment of the present invention will be described. As described above, in the projector 100 of the first and second embodiments, it is possible to detect and correct the pixel shift without using the test pattern while continuing the stack projection. However, the correction conditions are not uniform on the screen 150 due to the shape of the screen (distortion, etc.), the angle of the screen 150 with respect to the projector 100, and the like. For example, when the projection direction of the projector 100 is greatly tilted with respect to the screen, that is, when the projector 100 is installed at an angle, it is necessary to perform relatively complicated correction. Therefore, in the third embodiment, it is possible to perform complicated correction by dividing and analyzing the projected contents. Since the configuration of the projector 100 of the third embodiment is the same as the configuration of the first embodiment shown in FIG. 2, the description thereof will be omitted.

FIG. 7 is a flowchart for explaining the processing of the third embodiment. The deviation correction process is executed in a state where the projector 100 is projecting an image by stack projection. It is assumed that the deviation correction process is repeatedly executed when the user has previously set to enable the deviation correction function. The deviation correction process may be executed by both the projectors 100a and 100b, or may be executed by either one. In the present embodiment, it is assumed that the deviation correction function of the projector 100a is set to be valid and the deviation correction function of the projector 100b is set to be invalid. That is, the projector 100a detects the pixel shift characteristic and corrects the projection position of the projected image of the projector 100a according to the detected pixel shift characteristic. In S201, the stack-projected screen is photographed in the same manner as in S1 of the first embodiment. The photographed image 300 generated by the imaging unit 101 is shown in FIG. The CPU 120 stores the captured image 300 in the memory 105. As an example, the resolution of the captured image 300 is 3840 × 2160.

In S202, in order to divide and analyze the captured image 300, the CPU 120 calculates an area for dividing the image. Here, as an example, it is assumed that the captured image 300 is divided into 24 (6 divisions in the horizontal direction and 4 divisions in the vertical direction). The resolution of each of the divided images from the divided image 301 to the divided image 324 of FIG. 8 is 640 × 540. The CPU 120 generates the coordinates of each divided image using the coordinates of the captured image 300 and stores them in the memory 105.

In S203, a process of generating a divided image from the captured image 300 is performed. The CPU 120 reads out the coordinates of each divided image stored in the memory 105, cuts out an image corresponding to the coordinates of the divided image from the captured image 300 also stored in the memory 105, generates a divided image, and then generates a divided image, and then the Fourier transform unit 102. Output to.

In S204, the Fourier transform process of the divided image is performed. The Fourier transform unit 102 performs a two-dimensional Fourier transform process on each of the divided images, generates a frequency spectrum image of the divided images, and then stores the divided images in the memory 105.

In S205, pixel shift analysis is performed using the frequency spectrum image of the divided image. The pixel shift analysis unit 103 analyzes the pixel shift characteristics of the frequency spectrum images of all the divided images (24 divided images) stored in the memory 105, and detects whether or not the pixels of the stack projection are shifted.

In S206, it is determined whether or not the CPU 120 has detected the occurrence of pixel shift in the divided image. When the CPU 120 detects the occurrence of pixel shift, the process proceeds to S207. If the occurrence of pixel shift is not detected, the process proceeds to step S209.

In S207, similarly to S104, the CPU 120 performs a process of estimating the pixel shift direction and the pixel shift amount for each divided image.

In S208, as in S105, pixel shift correction processing is performed for each region corresponding to each divided image. Based on the pixel shift direction and the pixel shift amount estimated in S207, the CPU 120 corrects the pixel shift in the region corresponding to the divided image by using the distortion correction processing function of the image processing unit 111.

In S209, it is determined whether or not the CPU 120 has completed the processing of all the divided images (divided images 301 to 324). If all the divided images have been processed, the process proceeds to S210, and if the processing of all the divided images has not been completed, the process returns to S203 to continue the processing of the divided images.

In S210, the CPU 120 determines whether or not the user has performed an end operation (an operation to end the flowchart of FIG. 7). When the end operation is performed, the flowchart of FIG. 7 ends, and when the end operation is not performed, the process returns to S201.

As described above, according to the third embodiment, in a situation where the screen is distorted or the projector 100 does not face the screen, the projection area is divided and pixel deviation is detected with high accuracy. , Can be corrected.

(Fourth Embodiment)
Hereinafter, a fourth embodiment of the present invention will be described. As described above, in the projector 100 of the first to third embodiments, it is possible to detect and correct the pixel deviation without using the test pattern while continuing the stack projection. However, when the resolution of the projected image is larger than the resolution of the imaging unit (analysis condition), there is a problem that the analysis accuracy of the pixel shift is lowered. Therefore, in the fourth embodiment, a configuration in which the imaging unit has a zoom function will be described. Since the configuration of the projector 700 of the fourth embodiment is partially the same as the configuration of the first embodiment shown in FIG. 2, the description of the overlapping portion will be omitted.

FIG. 9 is a configuration diagram of the projector 700. The difference from the projector 100 of the first embodiment is that the imaging unit 701 is provided with a zoom lens and has a zoom function.

The projector 700 includes an image pickup unit 701, a zoom ratio calculation unit 702, a zoom control unit 703, a Fourier transform unit 102, a pixel shift analysis unit 103, an output unit 104, a memory 105, an image input unit 110, an image processing unit 111, and a light source 112. It includes a panel 113, a projection optical system 114, a CPU 120, and a bus 710.

The image pickup unit 701 is the same as the image pickup unit 101, but is provided with a zoom lens, and a part of the projected image on the screen can be magnified and photographed by using the zoom function. The zoom ratio calculation unit 702 determines the zoom ratio so that the image can be taken under the same conditions as the pixel shift characteristic stored in the memory 105 in advance. The zoom control unit 703 controls the zoom lens provided in the image pickup unit 701 by using the zoom ratio determined by the zoom ratio calculation unit 702.

Subsequently, the processing of the projector 700 will be described with reference to the flowchart of FIG. The deviation notification process is executed in a state where the projector 700 is projecting an image by stack projection. It is assumed that the deviation notification process is repeatedly executed when the user has previously set to enable the deviation notification function. The deviation notification process may be executed by both the projectors 700a and 700b, or may be executed by either one. In the present embodiment, it is assumed that the deviation notification function of the projector 100a is enabled and the deviation notification function of the projector 100b is disabled.

In S300, the zoom ratio calculation unit 702 calculates the zoom ratio of the zoom lens of the image pickup unit 701 so that the image captured by the image pickup unit 701 has the same conditions as the image that generated the pixel shift characteristic.

In S301, the zoom ratio of the zoom lens provided in the imaging unit 701 is set based on the zoom ratio calculated by the CPU 120 in S300.

In S302, the image pickup unit 701 stores the captured image of the stack projection of the screen in the memory 105.

In S303, a Fourier transform process is performed on the captured image to generate a frequency spectrum image in the same manner as in S2 of the first embodiment.

In S304, pixel shift analysis is performed in the same manner as in S3 of the first embodiment.

In S305, it is determined whether or not the CPU 120 has detected the occurrence of pixel shift. If the CPU 120 detects the occurrence of pixel shift, the process proceeds to S306. If the occurrence of pixel shift is not detected, the process proceeds to step S307.

In S306, the process of outputting the pixel shift to the external server is performed as in S5 of the first embodiment.

In S307, the CPU 120 determines whether or not the user has performed an end operation (an operation to end the flowchart of FIG. 10). When the end operation is performed, the flowchart of FIG. 9 ends, and when the end operation is not performed, the process returns to S302.

Although the description is omitted in this embodiment, the pixel shift correction can be performed by estimating the pixel shift direction and the pixel shift amount from the pixel shift detection result as in the second embodiment.

As described above, according to the fourth embodiment, when the resolution of the projected image is larger than the resolution of the imaging unit, the plurality of projectors 700 continue the stack projection without degrading the analysis accuracy of the pixel shift. As it is, the pixel shift can be corrected.

(Fifth Embodiment)
Hereinafter, a fifth embodiment of the present invention will be described. As described above, in the projector 700, it is possible to detect the pixel shift without degrading the analysis accuracy by using the zoom function of the imaging unit. However, even in the case of multi-projection in which the resolution is increased by a plurality of projectors 100, it is necessary to accurately overlap the pixels at the projection boundary of each projector (hereinafter referred to as the blend portion), and the pixel shift occurs in the blend portion. It is necessary to detect and correct.

Therefore, in the fifth embodiment, a configuration in which the analysis location of the projection of the screen is changed between the stack projection and the multi-projection will be described. Since the configuration of the projector 100 of the fifth embodiment is the same as the configuration of the first embodiment shown in FIG. 2, the description thereof will be omitted.

FIG. 11 is a flowchart illustrating the processing of the fifth embodiment. The deviation correction process is executed in a state where the projector 100 is projecting an image by multi-projection or stack projection. It is assumed that the deviation correction process is repeatedly executed when the user has previously set to enable the deviation correction function.

In S400, the screen is photographed by multi-projection or stack projection as in S1 of the first embodiment.

In S401, a process of determining a method of projecting the plurality of projectors 100 onto the screen is performed. The projection method is set in advance by the user operating the operation unit provided on the projector 100. The CPU 120 acquires the set projection method setting, and proceeds to S402 when multi-projection is selected as the projection method. If stack projection is selected as the projection method, the process proceeds to S405.

In S402, an image of the blended portion of the multi-projection is acquired. The CPU 120 acquires the coordinates of the blend portion from the edge blending function unit in the image processing unit 111 and generates an image of the blend unit. Hereinafter, the processing of the blend portion will be described with reference to FIG.

FIG. 12 shows a captured image 800 captured by the imaging unit 101 of the multi-projected image of the screen 150. The captured image 800 is an image captured by the imaging unit 101, which is formed by projecting the images projected by the four projectors 100. The captured image 801 is an image captured of what is projected by the first projector 100. Hereinafter, the captured images 802, 803, and 804 are images captured by the second, third, and fourth projectors 100, respectively.

The blending portion 810 is a portion in which pixels are superimposed and projected in order to make the boundary between the projected image 801 and the projected image 802 inconspicuous. Similarly, the blend portion 811 is formed from an image of a boundary portion between the projected image 803 and the projected image 804. Similarly, the blend portion 812 is formed from an image of a boundary portion between the projected image 801 and the projected image 803. Similarly, the blend portion 813 is formed from an image of a boundary portion between the projected image 802 and the projected image 804. The width of the blend portion can be determined from a preset value. It can also be determined from the amount of edge blending of the projector 100.

Returning to the flowchart of FIG. 11, in S403, a process of performing a two-dimensional Fourier transform on the image of the blended portion is performed. The frequency spectrum image obtained by performing the two-dimensional Fourier transform of the image of the blend unit 810 is referred to as the frequency spectrum image A. Similarly, the frequency spectrum images obtained by performing the two-dimensional Fourier transform of the blend unit 811, the blend unit 812, and the blend unit 813 are referred to as frequency spectrum images B, C, and D, respectively.

In S404, the pixel shift analysis of the blended portion is performed. The pixel shift analysis unit 103 analyzes the pixel shift characteristics of the frequency spectrum images A, B, C, and D, and detects whether or not the pixels of the image in the blend section are shifted. Further, the CPU 120 estimates the pixel shift amount and the pixel shift direction from the analysis result of the pixel shift analysis unit 103, and stores them in the memory 105. Then, the process proceeds to S407.

On the other hand, the process when the stack projection is selected in S401 will be described.

In S405, in order to detect pixel shift during stack projection, the captured image 800 is subjected to a two-dimensional Fourier transform on the captured image 800 and converted into a frequency spectrum image in the same manner as in S2 of the first embodiment. Perform the processing to be performed.

In S406, similarly to S3 of the first embodiment, the pixel shift analysis unit 103 analyzes the pixel shift characteristic of the frequency spectrum image and detects whether or not the pixels of the projected image are shifted. Further, the CPU 120 estimates the pixel shift amount and the pixel shift direction from the analysis result of the pixel shift analysis unit 103 and stores them in the memory 105.

In S407, it is determined whether or not the CPU 120 has detected the occurrence of pixel shift using the result of analysis in S404 or S406. When the CPU 120 detects the occurrence of pixel shift, the process proceeds to S408. If the occurrence of pixel shift is not detected, the process proceeds to step S410.

In S408, it is determined whether or not to perform pixel shift correction. If the amount of pixel misalignment is equal to or greater than a certain value, the process proceeds to S409 to correct the pixel misalignment. If the pixel shift amount is less than a certain value, the process proceeds to S410.

In S409, pixel shift correction processing is performed. The CPU 120 corrects the pixel shift by using the distortion correction processing function of the image processing unit 111 by using the setting of the projection method and the pixel shift amount and the pixel shift direction stored in the memory 105.

In S410, the CPU 120 determines whether or not the user has performed an end operation (an operation to end the flowchart of FIG. 11). When the end operation is performed, the flowchart of FIG. 11 is terminated, and when the end operation is not performed, the process returns to S400.

As described above, according to the fifth embodiment, by detecting the stack projection and the multi-projection and switching the analysis method, the projection is stopped while the stack projection or the multi-projection of the plurality of projectors 100 is continued. Pixel misalignment can be detected and corrected without the need for. Also in this embodiment, since it is not necessary to use a test pattern for detecting pixel deviation, even if pixel deviation is detected and corrected during video viewing, viewing is prevented from being hindered.

(Sixth Embodiment)
In the fifth embodiment described above, the projector 100 can detect and correct the pixel deviation while determining the stack projection and the multi-projection and continuing the projection. However, it is conceivable that the imaging unit 101 is independent of the projector. For example, a Web camera, a digital camera, a surveillance camera, or the like is used. Therefore, in the sixth embodiment, a configuration for performing pixel shift analysis using a Web camera and a personal computer will be described.

FIG. 13 is a diagram showing a configuration of an image projection system according to a sixth embodiment. The projector 950 constituting the image projection system 900 of the sixth embodiment is the image pickup unit 101, the Fourier transform unit 102, the pixel shift analysis unit 103, and the output unit 104 from the projector 100 of the first embodiment shown in FIG. The configuration is such that the functional part for detecting pixel deviation such as is deleted, and the others are common.

The image projection system 900 is configured by connecting a camera 901, a personal computer 910, and a plurality of projectors 950.

The camera 901 captures an area including a screen on which the projector 950 stacks. The image captured by the camera 901 is sent to the personal computer 910 as image data via the network, and the personal computer 910 stores the captured image data in the memory 912.

The personal computer 910 includes an image input I / F 911, a memory 912, a CPU 913, a graphics compositing unit 914, an image output I / F 915, and a bus 916.

The image input I / F 911 is a network interface that acquires image data taken from the camera 901 via a network. For example, the USB (Universal Serial Bus) standard is used for networks.

The memory 912 stores the image data input from the image input I / F 911. It also stores the control program of the CPU 913. Further, it is used as a work memory by the CPU 913.

The CPU 913 processes the image data and performs the pixel shift analysis process performed by the pixel shift analysis unit 103 of the first embodiment. Further, each component of the image input I / F 911, the memory 912, the graphics compositing unit 914, and the image output I / F 915 is controlled via the bus 916.

The graphics synthesizing unit 914 synthesizes graphics with an image and outputs the graphics to the image output I / F 915. Further, the graphics synthesizing unit 914 synthesizes graphics with an image and stores them in the memory 912.

The image output I / F 915 reads out the image stored in the memory 912, converts it into a video signal, and then outputs the image to an external device using a communication cable. For example, HDMI (registered trademark) and DisplayPort (registered trademark) are used as communication standards.

The projector 950 includes an image input I / F 110, an image processing unit 111, a memory 105, a CPU 120, a light source 112, a panel 113, and a projection optical system 114. The projector 950 projects an image output by the personal computer 910.

As described above, according to the sixth embodiment, even in a configuration in which the camera and the pixel shift analysis function are independent of the projector, the pixel shift can be corrected without stopping the projection. Also in this embodiment, since it is not necessary to use a test pattern for detecting pixel deviation, even if pixel deviation is detected and corrected during video viewing, viewing is prevented from being hindered.

(7th Embodiment)
In the first to sixth embodiments described above, the plurality of projectors can correct the pixel shift without stopping the projection by using the test pattern while continuing the stack projection. However, when installing a projector, there are cases where it is desired to superimpose the shift information on the screen and check the pixel shift. Therefore, in the seventh embodiment, the case where the deviation information is superimposed on the projected image will be described. The configuration of the projection system of this embodiment is the same as that of the first embodiment.

FIG. 14A is a conceptual diagram of a user interface (hereinafter referred to as UI) for the user to correct pixel deviation while looking at the frequency spectrum. The projected image 400 is a projected image stacked and projected on the screen by the projector 100.

In the frequency spectrum image 401, the CPU 120 two-dimensionally Fourier transforms the captured image obtained by capturing the projected image 400 by the imaging unit 101, and the resulting frequency spectrum image is superimposed and displayed on the projected image 400. The user adjusts the position of the projector so that the frequency spectrum image has no streaks.

FIG. 14B is a conceptual diagram illustrating the UI 452 when the adjustment is performed while looking at the UI (user interface) indicating the pixel shift direction when the pixel shift at the time of installation of the seventh embodiment is adjusted. ..

The divided area 450 is a divided area for dividing and analyzing the captured image obtained by capturing the projected image 400. Here, as an example, it is assumed that the captured image is divided into 15 (5 in the horizontal direction and 3 in the vertical direction). The CPU 120 generates each divided image using the coordinates of the captured image 400 and stores it in the memory 105.

The UI 452 is an example of displaying the result of performing the two-dimensional Fourier transform process and the pixel shift analysis after the CPU 120 reads the divided image stored in the memory 105. The user can adjust the position of the projector while looking at the arrow of UI452.

As described above, according to the seventh embodiment, the user can adjust the position of the projector 100 while looking at the pixel shift information superimposed on the projected image.

(Other embodiments)
The present invention also supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads the program. It can also be realized by the processing to be executed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

100: Projector, 101: Imaging unit, 102: Fourier transform unit, 103: Pixel shift analysis unit, 104: Output unit, 105: Memory, 110: Image input unit, 111: Image processing unit, 112: Light source, 113: Panel , 114: Projection optics, 120: CPU, 121: Bus, 150: Screen, 152: Signal source

Claims (19)

Acquisition of an image of an image of the projection surface including at least a superposed area in which the first projection image projected on the projection surface by the first projector and the second projection image projected on the projection surface by the second projector overlap. Means and
A detection means that analyzes the captured image to detect the magnitude of the positional deviation between the first projected image and the second projected image in the superimposed region and the direction of the deviation.
A projection control device comprising. The claim is characterized in that the detection means detects the magnitude of the deviation and the direction of the deviation based on a frequency spectrum image obtained by applying a two-dimensional Fourier transform process to the captured image. The projection control device according to 1. Further provided with a storage means for storing a plurality of reference images indicating a plurality of frequency spectrum images associated with the magnitude and direction of the shift in advance.
2. The detection means is characterized in that an image showing the frequency spectrum image is compared with the plurality of reference images stored in the storage means to detect the magnitude and direction of the deviation. The projection control device according to. The projection control device according to any one of claims 1 to 3, further comprising a connecting means for connecting to an imaging means for imaging a region of the projection surface including the superposed region. Any one of claims 1 to 4, further comprising a control means for controlling at least one of the first projector and the second projector so as to correct the deviation based on the detection result of the detection means. The projection control device according to the section. The projection control device according to claim 5, wherein the control means controls the zoom of the projector based on the analysis conditions of the captured image by the detection means. The projection control device according to any one of claims 1 to 6, further comprising an output means for outputting the detection result of the detection means to the outside. 1. The detection means is characterized in that a portion of the captured image analyzed by the detection means is changed according to whether the first projector and the second projector perform stack projection or multi-projection. 7. The projection control device according to any one of 7. The projection control device according to any one of claims 1 to 8.
Projection means for projecting images and
A projection device characterized by comprising. Acquisition of an image of an image of the projection surface including at least a superposed area in which the first projection image projected on the projection surface by the first projector and the second projection image projected on the projection surface by the second projector overlap. Process and
A detection step of analyzing the captured image to detect the magnitude of the positional deviation between the first projected image and the second projected image in the superimposed region and the direction of the deviation.
A projection control method characterized by having. The claim is characterized in that the detection step detects the magnitude of the deviation and the direction of the deviation based on a frequency spectrum image obtained by applying a two-dimensional Fourier transform process to the captured image. 10. The projection control method according to 10. It further has a storage step of storing a plurality of reference images showing a plurality of frequency spectrum images associated with the magnitude and direction of the shift in advance.
11. The detection step is characterized in that an image showing the frequency spectrum image is compared with the plurality of reference images stored in the storage step to detect the magnitude and direction of the deviation. The projection control method described in. The projection control method according to any one of claims 10 to 12, wherein the projection control method includes a connection step for connecting to an imaging means for imaging an area of the projection surface including the superimposed area. Any one of claims 10 to 13, further comprising a control step of controlling at least one of the first projector and the second projector so as to correct the deviation based on the detection result of the detection step. The projection control method described in the section. The projection control method according to claim 14, wherein in the control step, the zoom of the projector is controlled based on the analysis conditions of the captured image in the detection step. The projection control method according to any one of claims 10 to 15, further comprising an output step of outputting the detection result of the detection step to the outside. 10. The detection step is characterized in that a portion of the captured image to be analyzed in the detection step is changed depending on whether the first projector and the second projector perform stack projection or multi-projection. The projection control method according to any one of 16 to 16. A program for causing a computer to function as each means of the projection control device according to any one of claims 1 to 8. A computer-readable storage medium that stores a program for causing the computer to function as each means of the projection control device according to any one of claims 1 to 8.

Images