JP2020048115A - Projection apparatus, projection method and program - Google Patents

Abstract

To make high resolution by pixel shift compatible with visibility improvement of a moving image by insertion of a black image.SOLUTION: A projection apparatus includes a move part for moving the projection position of a projection image between multiple positions in synchronism with multiple subframe periods dividing one frame, a projection part outputting the projection image in synchronism with the multiple subframe periods, and an image processing part processing the projection image for creating the projection image. The image processing part generates multiple subframes from the input image of one frame, outputs subframes corresponding to the projection positions out of the multiple subframes in the subframe period, as a projection image, when the projection positions are controlled to prescribed positions, out of the multiple positions, and outputs a black image in other subframe period, as the projection image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a projection apparatus using a pixel shifting technique and a moving picture visibility improving technique by inserting black.

  2. Description of the Related Art In recent years, a projection system that realizes a high resolution equal to or higher than a panel resolution by a pixel shifting technique has been proposed. The pixel shifting technique is a technique for switching and displaying image light projected without changing the projection position and image light shifted in the projection position with an accuracy of one pixel or less. These image lights on the projection plane are superimposed with the pixels shifted, and the resolution can be improved in a pseudo manner. In Patent Literature 1, one frame is divided into four sub-frames, and each projection position is displayed in a time-division manner by a 1/2 pixel in a horizontal direction, a vertical direction, and a horizontal / vertical direction using a pixel shifting device. As a result, high resolution is realized. Hereinafter, the combination of the pixel shift direction and the amount in Patent Document 1 is referred to as horizontal and vertical four-direction pixel shift in this specification. In the horizontal and vertical four-direction pixel shift, by dividing one frame into four, the amount of information in one frame is four times as large as that in non-pixel shift.

  In addition, by following an object in a moving image with eyes, an afterimage of the object before the current frame is integrated in the brain, and is perceived as tailing, which may reduce the sense of resolution. On the other hand, in Patent Literature 2, the image of one frame is divided into a plurality of sub-frames, and a part of the plurality of sub-frame images is replaced with a black image to reduce residual image information of an object, thereby lowering the resolution of a moving image. Has been realized. Hereinafter, the moving image visibility improvement technology in Patent Document 2 is referred to as black insertion in this specification.

JP 2017-027024 A JP 2011-221500 A

  In order to realize the above-described horizontal and vertical pixel shifting in four directions, it is necessary to display an image four times in one frame period. That is, it is necessary to drive the panel at a quadruple speed as compared with the non-pixel shift operation. Also, in order to improve the visibility of a moving image by inserting black, it is necessary to display an image twice in one frame period. That is, it is necessary to drive the panel at twice the driving speed as compared to when the black image is not inserted. Therefore, in order to achieve both high resolution by shifting the pixels in the horizontal and vertical directions and improvement in the visibility of moving images by inserting black, it is necessary to increase the panel speed by 4 × 2 to 8 ×. However, setting the panel driving speed to 8 times speed imposes a large load on the band, and requires a panel that is expensive. Particularly, in recent years, SHV (Super High Vision), HFR (High Frame Rate), and the like that require high-speed driving have appeared. SHV requires 60 frames / sec progressive scanning at 8K resolution (7680 × 4320), and HFR requires 120 frames / sec progressive scanning. It is difficult to apply the 8 × speed drive as described above to a SHV / HFR compatible projection device that requires a high frame rate.

  SUMMARY OF THE INVENTION It is an object of the present invention to provide a projection apparatus that achieves both high resolution by pixel shift and improvement of moving image visibility by black insertion.

A projection device according to one aspect of the present invention has the following configuration. That is,
Moving means for moving the projection position of the projection image between the plurality of positions in synchronization with the plurality of sub-frame periods obtained by dividing one frame period;
Projection means for outputting the projection image in synchronization with the plurality of sub-frame periods,
Image processing means for processing an input image to generate the projection image,
The image processing means,
Generating a plurality of sub-frames from one frame of the input image;
Outputting a sub-frame corresponding to the projection position among the plurality of sub-frames as the projection image in a sub-frame period in which the projection position is controlled to a predetermined position among the plurality of positions; To output a black image as the projection image.

  According to the present invention, it is possible to provide a projection device that achieves both high resolution by pixel shift and improvement in moving image visibility by black insertion.

FIG. 2 is a block diagram illustrating a configuration example of a projection device according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration example of an image processing unit according to the first embodiment. FIG. 4 is a block diagram illustrating another configuration example of the image processing unit according to the first embodiment. 5 is a flowchart illustrating projection processing according to the first embodiment. 5 is a flowchart illustrating projection processing according to the first embodiment. 9 is a flowchart illustrating another projection process according to the first embodiment. 9 is a flowchart illustrating another projection process according to the first embodiment. FIG. 4 is a diagram illustrating a timing signal according to the first embodiment and output images of respective units of an image processing unit. The figure which showed the projection position of horizontal and vertical 4 direction pixel shift. FIG. 8 is a diagram showing sampling positions of each thinned image in which pixels are shifted in four directions in the horizontal and vertical directions. FIG. 4 is a diagram for explaining a mechanism for increasing resolution in diagonal pixel shift. FIG. 4 is a diagram illustrating a moving image visibility improvement mechanism by black insertion according to the first embodiment. FIG. 9 is a block diagram illustrating a configuration example of an image processing unit according to a second embodiment. 9 is a flowchart illustrating projection processing according to the second embodiment. 9 is a flowchart illustrating projection processing according to the third embodiment. FIG. 13 is a diagram illustrating a timing signal according to a third embodiment and output images of respective units of an image processing unit. The figure explaining the moving image visibility improvement mechanism by black insertion of 3rd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the following embodiments. The following embodiments do not limit the invention according to the claims, and all combinations of the features described in the embodiments are not necessarily essential for solving the invention.

  Note that each functional block described in the present embodiment does not necessarily need to be individual hardware. That is, for example, the functions of some functional blocks may be executed by one piece of hardware. Further, the function of one functional block or the functions of a plurality of functional blocks may be executed by the cooperative operation of several pieces of hardware. Further, each functional block may be executed by a computer program developed on a memory by the CPU.

<First embodiment>
Hereinafter, a method for achieving both high resolution by shifting the pixels in the four directions in the horizontal and vertical directions and improving the visibility of the moving image by inserting black will be described according to the first embodiment of the present invention.

  FIG. 1 is a block diagram illustrating a configuration example of a projection device 200 according to the first embodiment. In the projection device 200, the CPU 210, the RAM 211, the ROM 212, the operation unit 213, the communication unit 214, the image input unit 220, the light source control unit 230, the image processing unit 240, and the light modulation element control unit 250 are connected by a bus 199. Further, a pixel shift device control unit 284, a projection optical system control unit 286, and a timing signal generation unit 287 are also connected to the bus 199.

  The CPU 210 controls each operation block of the projection device 200. The ROM 212 stores a control program describing the processing procedure of the CPU 210. The RAM 211 functions as a work memory of the CPU 210, and temporarily stores control programs and data. Further, the CPU 210 can temporarily store the still image data and the moving image data received from the communication unit 214 in the RAM 211, and can reproduce each image and video using the program stored in the ROM 212.

  The operation unit 213 includes, for example, a switch and a dial, and receives a user's instruction and transmits an instruction signal to the CPU 210. The operation unit 213 may transmit a predetermined instruction signal to the CPU 210 based on a signal received by a signal receiving unit (such as an infrared receiving unit) that receives a signal from a remote controller. The CPU 210 receives the instruction signal received from the operation unit 213 and the control signal input via the communication unit 214, and controls each operation block of the projection device 200.

  The image input unit 220 receives an image transmitted from an external device. Here, the external device may be any device such as a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, and a game machine as long as it can output an image signal. Further, the image input unit 220 can read an image recorded on a medium such as a USB flash memory or an SD card. The image processing unit 240 includes, for example, a microprocessor for image processing, performs predetermined image processing on an image signal received from the image input unit 220, and transmits the image signal to the light modulation element control unit 250. Details of the image processing unit 240 will be described later.

  The light source control unit 230 includes a control microprocessor, and performs on / off control of the light source 260, control of the light amount, and the like. The light source control unit 230 does not need to be a dedicated microprocessor. For example, the CPU 210 may execute the same processing as the light source control unit 230 by executing a program stored in the ROM 212. The light source 260 outputs light for projecting an image on a screen (not shown), and may be, for example, a halogen lamp, a xenon lamp, a high-pressure mercury lamp, or the like. The color separation unit 261 includes, for example, a dichroic mirror and a prism, and separates light output from the light source 260 into red (R), green (G), and blue (B). In the case where an LED or the like corresponding to each color is used as the light source 260, the color separation unit 261 is unnecessary.

  The light modulation element control section 250 controls a voltage applied to each light modulation element (pixel) of the light modulation element groups 270R, 270G, 270B based on the image signal output from the image processing section 240. Accordingly, the light modulation element control unit 250 adjusts the light modulation rate of each light modulation element of the light modulation element groups 270R, 270G, and 270B. The light modulation element group 270R is a set of light modulation elements corresponding to red, and adjusts the light modulation rate of red light among the lights output from the light source 260 and separated by the color separation unit 261. The light modulation element group 270G is a light modulation element corresponding to green, and adjusts the light modulation rate of green light of the light output from the light source 260 and separated by the color separation unit 261. The light modulation element group 270B is a light modulation element corresponding to blue, and adjusts the light modulation rate of blue light among the lights output from the light source 260 and separated by the color separation unit 261.

  The color synthesizing unit 280 includes, for example, a dichroic mirror and a prism, and synthesizes red (R), green (G), and blue (B) light transmitted through the light modulation element groups 270R, 270G, and 270B. The light obtained by the synthesis of the color synthesis unit 280 is sent to the pixel shift device 283. At this time, the light modulation element groups 270R, G, and B are controlled by the light modulation element control section 250 so as to have a light modulation rate corresponding to the image input from the image processing section 240. Therefore, when the projection light is projected onto the screen by the projection optical system 285, an image corresponding to the image input by the image processing unit 240 is displayed on the screen.

  The pixel shifting device 283 is a device that can shift a projection position so that incident image light can be projected to a desired position. For example, the pixel shifting device 283 includes a parallel plate glass and four actuators. The pixel shifting device 283 changes the attitude of the glass by supporting four points on the top, bottom, left and right of the glass with four actuators and controlling the tilt by displacing the support points. The image light transmitted through the color synthesizing section 280 is transmitted through the glass. At this time, the pixel position at the time of projection can be shifted in the horizontal direction and the vertical direction by changing the attitude of the glass with the actuator.

  The pixel shifting device control unit 284 controls the attitude of the pixel shifting device 283. The pixel shift device control unit 284 adjusts the amount of pixel shift by controlling the displacement of the support point of the actuator that supports the above-mentioned parallel plate glass. The pixel shifting device 283 is not limited to this configuration, and may be, for example, a birefringent optical system. By changing the voltage applied to the birefringent optical system, it is possible to change the projection position of the image light transmitted through the birefringent optical system and shift the pixels. In the case of this configuration, the pixel shift device control section 284 controls the voltage applied to the birefringent optical system.

  The projection optical system control unit 286 has a microprocessor for control and controls the projection optical system 285. However, the projection optical system control unit 286 does not need to be a dedicated microprocessor. For example, the same function as the projection optical system control unit 286 is realized by the CPU 210 executing a predetermined program stored in the ROM 212. You may. The projection optical system 285 has a plurality of lenses and a lens driving actuator, and projects the light output from the pixel shift device 283 on a screen. The projection optical system 285 can perform enlargement, reduction, and focus adjustment of a projection image by driving a lens with an actuator.

  The timing signal generation section 287 generates a timing signal SIG to be given to the image processing section 240, the light modulation element control section 250, and the pixel shift device control section 284. The timing signal SIG is a signal indicating which one of the 1st to 4th subframes the subframe generated when the projection apparatus 200 is driven at 4 × speed. Details of the method of generating the timing signal SIG will be described later.

  The communication unit 214 receives a timing signal SIG, still image data, moving image data, and the like from an external device. The communication method of the communication unit 214 is not particularly limited. For example, a communication method using a wireless LAN, a wired LAN, USB, Bluetooth (registered trademark), infrared light, or the like can be used. Further, the communication unit 214 transmits a timing signal SIG, still image data, moving image data, and the like to the external device. When the terminal of the image input unit 220 is, for example, an HDMI (registered trademark) terminal, the communication unit 214 may perform CEC communication via the terminal. Here, the external device may be any device as long as it can communicate with the projection device 200, such as glasses, a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, a game machine, and a remote controller. Is raised.

  Note that the light modulation element control unit 250, the light source control unit 230, the pixel shift device control unit 284, and the projection optical system control unit 286 of the present embodiment are singly or plurally capable of performing the same processing as each of these blocks. There may be a microprocessor. Alternatively, for example, the CPU 210 may execute a predetermined program stored in the ROM 212 to execute the same processing as the above-described blocks.

  Next, an internal configuration of the image processing unit 240 will be described with reference to FIG. 2A. The image processing unit 240 includes a memory control unit 241, an image memory 242, a thinning unit 243, a selection unit 244, and a black insertion unit 245. These units are connected to the CPU 210 and the like via the bus 199.

  The memory control unit 241 writes the input image of each frame into the image memory 242 and reads the input image a plurality of times during one frame period in order to generate a plurality of subframes for one frame of the input image. In the present embodiment, the input image is read four times during one frame period in order to generate four subframes for one frame of the input image. Therefore, the memory control unit 241 controls the input image to be read from the image memory 242 at a frequency four times the writing frequency. Reading of data from the image memory 242 is performed according to the timing signal SIG input from the timing signal generation unit 287. When the frequency of the vertical synchronization signal of the input frame image is 60 Hz, the memory control unit 241 in the present embodiment reads data from the image memory 242 at a cycle corresponding to 240 Hz. In this way, the memory control unit 241 reads the input image IMG from the image memory 242 four times and supplies it to the thinning unit 243 during one frame period of the input image.

  The thinning unit 243 generates thinned images DIV_A to DIV_D by sampling an image at a predetermined sampling position with respect to an output image of the image (input image IMG) input from the memory control unit 241. Details of a method of generating a thinned image will be described later. Thus, four sub-frames (thinned-out images DIV_A to DIV_D) are generated by the memory control unit 241 and the thinning-out unit 243. The selection unit 244 selects one of the thinned images DIV_A to DIV_D generated by the thinning unit 243 based on the timing signal SIG, and outputs it as an image IMG_C. The relationship between the timing signal SIG and the image output by the selection unit 244 will be described later. Note that a broken line indicating that the black insertion signal K_SIG is input to the selection unit 244 is a configuration used in the third embodiment.

  The black insertion unit 245 replaces the image IMG_C (any one of the thinned images DIV_A to D) output from the selection unit 244 with a black image based on the black insertion control signal K_SIG and the timing signal SIG. The black insertion control signal K_SIG is a signal indicating ON / OFF of the black insertion function set by the user via the operation unit 213 or the communication unit 214. For example, when a 1-bit signal is 0, the function is OFF. Time is designed to indicate that the function is ON. When the black insertion control signal K_SIG is ON and the timing signal SIG indicates a predetermined black insertion timing, the black insertion unit 245 replaces the image IMG_C with a black image and outputs the image IMG_O. When the black insertion control signal K_SIG indicates OFF, the black insertion unit 245 outputs the image IMG_C as it is as the image IMG_O. Further, even when the black insertion control signal K_SIG is ON, the black insertion unit 245 outputs the image IMG_C as it is as the image IMG_O when the timing signal SIG indicates the timing at which the black image is not inserted.

  The internal configuration of the projection device 200 and the image processing unit 240 according to the first embodiment has been described above.

  Next, the projection processing according to the first embodiment will be described with reference to the flowcharts in FIGS. 3A and 3B. In this example, the description will be made on the assumption that the vertical and horizontal resolution of the input image is twice the vertical and horizontal display resolution of each projection device. That is, when the resolution of the projection device is FHD (Full High Definition: 1920 × 1080), the resolution of the input image is 4K (3840 × 2160). When an instruction to start projection is transmitted from the user of the projection device 200 to the CPU 210 via the operation unit 213, the units included in the projection device 200 perform operations to generate a projection image. It operates according to the flowcharts of FIGS. 3A and 3B.

  First, in S301 of FIG. 3A, the CPU 210 determines whether or not the pixel shift mode has been set via the operation unit 213. If the pixel shift mode is not set, the CPU 210 executes a normal projection process (projection process without pixel shift) in S328. If the pixel shift mode is set in S301, the process proceeds to S302. In S302, the CPU 210 issues an instruction to the timing signal generator 287 to start generating the timing signal SIG.

  FIG. 4 shows a correspondence diagram between the timing signal SIG and the image generated by each image processing unit in the first embodiment. FIG. 4A shows the timing signal SIG in the first embodiment. As shown in FIG. 4A, the timing signal SIG is a signal having a value every フ レ ー ム frame for one frame of the input image. At this time, a period between 0 and 1/4 frame when one frame is divided into four is called a first subframe. Similarly, a period between 1/4 to 1/2 frame is called a 2nd subframe, and a period between 1/2 to 3/4 frame is called a 3rd subframe, and a period of 3/4 to 1 frame is called a 4th subframe.

  Next, the method of generating the timing signal SIG will be described in detail. First, the timing signal generator 287 acquires the reproduction frequency of the input image. Then, a signal having a value that is four times the acquired reproduction frequency is generated. The generated signal becomes the timing signal SIG. For example, for a 60 Hz input image, a 240 Hz timing signal SIG is generated. The timing signal SIG includes, for example, a timing signal pulse and a 2-bit signal line. The 2-bit signal line is controlled so that the value of 0 to 3 is repeated every time a timing signal is generated (SIG (0) is 0, SIG (1) is 1, and SIG (2) is 2). , SIG (3) has the value 3). For example, when the value of the timing signal SIG is 0, it indicates that it is the timing to output the first frame.

  In S303, the CPU 210 determines whether or not to insert a black image. In the present embodiment, it is determined whether or not the black insertion mode is set by the operation unit 213. If it is determined in step S303 that the black insertion mode is set (to insert a black image), the process proceeds to step S304, in which pixel shift projection involving insertion of a black image is performed. On the other hand, if it is determined in step S303 that the black insertion mode is not set (the black image is not inserted), the process proceeds to step S316, in which the pixel is shifted and projected without inserting the black image.

  The processing after S304 is processing in the black insertion mode. In S304, the CPU 210 sets the black insertion control signal K_SIG to ON, and initializes the variable i to 0. In S305, the memory control unit 241 in the image processing unit 240 waits for reception of the timing signal SIG (i). When receiving the timing signal SIG (i) (YES in S305), the memory control unit 241 reads the input image IMG from the image memory 242 and provides the same to the thinning unit 243 in S306. Note that the memory control unit 241 performs a quadruple speed driving process on the input image IMG. Specifically, the memory control unit 241 first writes the input image IMG into the image memory 242. Next, the memory control unit 241 reads the input image IMG written in the image memory 242 in accordance with the timing signal SIG. Since the timing signal SIG is a signal having a speed four times the reproduction frequency of the input image, the same image is read from the image memory 242 four times in one frame period. FIG. 4B illustrates a state where the memory control unit 241 outputs the input image IMG four times in accordance with the timing signal SIG.

  In S307, the thinning unit 243 generates thinned images DIV_A to DIV_D from the input image IMG supplied from the memory control unit 241. In the present embodiment, the pixel paths of the thinned images DIV_A to DIV_D are projected slightly differently by the pixel shifting device 283 to realize high definition by pixel shifting. FIG. 5 shows the relationship of the projection positions on the projection plane with respect to the thinned images DIV_B to DIV_A. In FIG. 5, a white square indicates a pixel position on the projection plane of the thinned image DIV_A, and this is set as a reference position.

  In the present embodiment, the projection position of the projection image is sequentially moved between four positions shifted by a predetermined distance (for example, 0.5 pixel) in the horizontal direction and / or the vertical direction. FIG. 5A shows the projection position of the thinned image DIV_B with respect to the projection position (reference position) of the thinned image DIV_A. As shown in FIG. 5A, the thinned image DIV_B is projected at a position shifted by 0.5 pixel in the horizontal direction with respect to DIV_A. FIG. 5B shows the projection position of the thinned image DIV_C with respect to the projection position of the thinned image DIV_A. As shown in FIG. 5B, the thinned image DIV_C is projected at a position shifted by 0.5 pixel in the horizontal and vertical directions with respect to DIV_A. FIG. 5C shows the projection position of the thinned image DIV_D with respect to the projection position of the thinned image DIV_A. As shown in FIG. 5C, the thinned image DIV_D is projected at a position shifted by 0.5 pixel in the vertical direction with respect to DIV_A.

  As described above, in the horizontal and vertical four-pixel shifting, each thinned image is slightly shifted on the projection plane and projected. In order to realize high definition by pixel shift, it is necessary to generate each thinned image by changing the thinned position with respect to the input image IMG according to each projection position. Next, a method of generating each of the thinned images DIV_A to DIV_D will be described with reference to FIG. FIGS. 6A to 6D show sampling positions of the thinned images DIV_A to DIV_D with respect to the input image IMG.

  FIG. 6A shows the sampling position of the thinned image DIV_A. As shown in the figure, the thinned image DIV_A is obtained by sampling the even numbered pixels in the horizontal and vertical directions with respect to the input image IMG. FIG. 6B shows a sampling position of the thinned image DIV_B. As shown in the drawing, the thinned image DIV_B is obtained by sampling odd-numbered pixels in the horizontal direction and even-numbered pixels in the vertical direction with respect to the input image IMG. can get. This is because the thinned image DIV_B is projected at a position shifted by 0.5 pixel in the horizontal direction with respect to DIV_A as described in FIG. 5A, and the sampling position is also horizontal with respect to the thinned image DIV_A. It is necessary to sample at a position shifted to

  FIG. 6C shows the sampling position of the thinned image DIV_C. As shown in the figure, the thinned image DIV_C is obtained by sampling the odd numbered pixels in the horizontal and vertical directions with respect to the input image IMG. This is because the thinned image DIV_C is projected at a position shifted by 0.5 pixel in the horizontal and vertical directions with respect to DIV_A as described with reference to FIG. 5B, and the sampling position is also horizontal with respect to the thinned image DIV_A. It is necessary to sample at a position shifted in the vertical direction. FIG. 6D shows a sampling position of the thinned image DIV_D. As shown in FIG. 6, the thinned image DIV_D is obtained by sampling even-numbered pixels in the horizontal direction and odd-numbered pixels in the vertical direction with respect to the input image IMG. can get. This is because the thinned image DIV_D is projected at a position shifted by 0.5 pixel in the vertical direction with respect to DIV_A as described with reference to FIG. 5C, and the sampling position is also perpendicular to the thinned image DIV_A. It is necessary to sample at a position shifted to

  Through the above processing, the thinning unit 243 generates four types of thinned images DIV_A to DIV from the input image IMG as four subframes. The effect of high definition by shifting pixels using these thinned images will be described later.

  Returning to FIG. 3A, in S308, the selection unit 244 selects a thinned image corresponding to the value of the received timing signal SIG from the thinned images DIV_A to DV output by the thinning unit 243, and sets the selected black image as the IMG_C in the subsequent stage. 245. In this example, DIV_A is selected for SIG (0), DIV_B for SIG (1), DIV_C for SIG (2), and DIV_D for SIG (3).

  Hereinafter, an example of a method of confirming in which subframe period the received timing signal SIG is a signal will be described. Hereinafter, the timing signal indicating the first subframe period is set to SIG (0). Similarly, the timing signal in the second sub-frame period is SIG (1), the timing signal in the third sub-frame period is SIG (2), and the timing signal in the fourth sub-frame period is SIG (3). The first timing signal SIG input to the selection unit 244 is the timing signal SIG (0) indicating the first subframe of the first frame of the input image. As described above, SIG (0) to SIG (3) (1st subframe to 4th subframe) can be identified by expressing 0 to 3 with 2 bits using two signal lines of the timing signal SIG. The configuration for identifying SIG (0) to SIG (3) (1st subframe to 4th subframe) is not limited to this. For example, each unit using the timing signal SIG may count the received timing signal, and identify SIG (0) to SIG (3) based on the remainder obtained by dividing the count value by 4. For example, the number of receptions of the timing signal SIG is counted, and if the number obtained by dividing the count by 4 is 0, the first subframe is 1, the 2nd subframe is 1, the 2nd subframe is 2, and the 3rd subframe is 3. If so, it may be determined that the frame is the 4th subframe.

  As described above, it can be determined from the timing signal SIG which subframe the current display timing is. Further, the above-described confirmation method is an example, and the determination may be performed by another method.

  Next, the relationship between each subframe period and the thinned image output by the selection unit 244 in the corresponding period will be described with reference to FIG. As shown in FIG. 4C, the selection unit 244 selects and outputs the thinned image DIV_A in the first subframe and the thinned image DIV_B in the second subframe. Similarly, the selection unit 244 selects and outputs the thinned image DIV_C in the 3rd subframe and the thinned image DIV_D in the 4th subframe. Therefore, the selection unit 244 sequentially outputs the thinned images DIV_A, DIV_B, DIV_C, and DIV_D in one frame. The projection order of each thinned image is not limited to this. If the four types of thinned images are output once each in one frame, and the optical path of the projection light set by the pixel shifting device 283 is switched so that each of the thinned images is projected to the correct position, it is observed that , Any projection order.

  In S309, the black insertion unit 245 determines whether the timing signal SIG is SIG (1) or SIG (3). When it is determined that the timing signal SIG is SIG (1) or SIG (3), in S310, the black insertion unit 245 replaces the image in the subframe period with a black image and outputs it as an image IMG_O to insert black. I do. On the other hand, when the timing signal SIG is SIG (0) or SIG (2) in S309 (NO in S309), in S311, the black insertion unit 245 outputs the image IMG_C output from the selection unit 244 as the image IMG_O, and outputs the black image. No insertion is performed. With the above processing, the black insertion unit 245 outputs the thinned images DIV_A and DIV_C as IMG_O for the timing signals SIG (0) and SIG (2), and outputs the black image for the timing signals SIG (1) and SIG (3). Output as image IMG_O.

  According to the above processing, the projection position of the projection image moves sequentially between the four positions shifted by a predetermined distance in the horizontal direction and / or the vertical direction. Then, one of the plurality of sub-frames is output as a projection image at two projection positions arranged in an oblique direction (diagonal direction) among these four positions. It should be noted that projecting a set of thinned images DIV_A and DIV_C whose projection positions are shifted in the diagonal direction and replacing the other images with black images makes full use of the effect of increasing the resolution by shifting pixels even when black is inserted. That's why. Although DIV_A and DIV_C are used as a set of thinned images whose projection positions are shifted in a diagonal direction, a set of thinned images DIV_B and DIV_D may be projected, and the other images may be replaced with black images. This is because whichever of the sets is selected as the projection image, the thinned image can be projected while being shifted in the diagonal direction. Hereinafter, the pixel shift performed by using the thinned image whose projection position is shifted in the diagonal direction is referred to as diagonal pixel shift. The mechanism for increasing the resolution by shifting pixels in the diagonal direction and the mechanism for improving the visibility of moving images by inserting black in the first embodiment will be described later.

  A black insertion unit 245 in the case where the thinned images DIV_A and DIV_C are selected as display images in the middle of FIG. 4D and black is inserted, and the thinned images DIV_B and DIV_D are selected and displayed in the middle of FIG. 3 shows an output image IMG_O. When a set of the thinned images DIV_B and DIV_D is used as a projection image, the process may branch to S310 in the case of i = 0 or i = 2 in S309.

  In step S312, in response to an instruction from the CPU 210, the pixel shifting device control unit 284 controls the pixel shifting device 283 based on the timing signal SIG to change the optical path of the projection image so that the projection position becomes P (i). Since the method of determining which sub-frame the signal is from based on the timing signal SIG has been described above, it is omitted. Note that the pixel shift device control unit 284 changes the optical path based on the subframe identification information derived from the timing signal SIG.

  Specifically, in the first subframe, the attitude of the pixel shifting device 283 is controlled so that the thinned image DIV_A is projected on the reference position P (0) shown in FIG. 5, and the optical path is changed. In the second sub-frame, the attitude of the pixel shifting device 283 is controlled so that the thinned image DIV_B is projected at a position P (1) shifted by 0.5 pixel in the horizontal direction from the reference position shown in FIG. change. In the 3rd sub-frame, the attitude of the pixel shifting device 283 is controlled such that the thinned image DIV_C is projected at a position P (2) shifted by 0.5 pixel in the horizontal and vertical directions from the reference position shown in FIG. To change. In the 4th sub-frame, the attitude of the pixel shift device 283 is controlled so that the thinned image DIV_C is projected at a position P (3) shifted by 0.5 pixel in the vertical direction from the reference position shown in FIG. change.

  In S313, the CPU 210 instructs each image processing block to project the output image IMG_O from the black insertion unit 245 from the projection optical system 285 as a projection image. Specifically, the light modulation element control unit 250 changes the modulation rates of the light modulation element groups 270R, G, and B based on the image IMG_O to generate projection image light. The above processing of S305 to S313 is repeated for i = 0 to 3 (S314, S315). When it is determined that i = 3 in S314, it is determined that the processing of one frame of the input image has been completed, and the process proceeds to S326. The processing of S301 to S315 is repeated until the end of projection is instructed (S326, S327).

  On the other hand, if it is determined in S303 that the black image is not to be inserted (not in the black insertion mode), the process proceeds to S316 in FIG. 3B. In this case, any of the plurality of subframes is output as a projection image in all of the plurality of subframe periods. In S316, the CPU 210 sets the black insertion control signal K_SIG to OFF, and initializes the variable i to 0. The processing of S317 to S320 is the same as that of S305 to S308. Since black insertion is not performed, regardless of the value of i, in S321, the black insertion unit 245 outputs IMG_C supplied from the selection unit 244 as it is as IMG_O. Steps S322 to S325 are the same as steps S311 to S315. If it is determined that i = 3 in S324, the process proceeds to S326.

  In S326, CPU 210 determines whether or not the end of projection has been instructed by the user via operation unit 213. If the end of projection has not been instructed, the process proceeds to S327, where the CPU 210 shifts the processing target to the next frame, and returns the process to S301. On the other hand, if it is determined in S326 that the instruction to end the projection has been issued, the CPU 210 ends this processing.

  As described above, according to the first embodiment, the projection position is controlled to the first position of the plurality of positions and the second position that is separated from the first position by a predetermined distance in the horizontal and vertical directions. One of the plurality of sub-frames is output as a projection image during the sub-frame period. Then, a black image is output as a projection image during a sub-frame period in which the projection position is controlled to a position other than the first position and the second position among the plurality of positions. This achieves both high definition by pixel shift and improvement of moving image visibility by black insertion.

  Next, a description will be given of a mechanism for improving image quality by shifting pixels in the diagonal direction, which is employed in the present case, using the projection system. FIG. 7 is a schematic diagram of a high-definition mechanism for shifting pixels in the diagonal direction. For the purpose of explanation, as an example, a diagonally shaded original image composed of a rectangle having a width of two pixels from the upper left to the lower right as shown in FIG. 7A is used. Each of the squares shown in FIG. 7A represents one pixel in the original image, a black square represents a pixel of 0 when the gradation value of the image is binary, and a white square represents a pixel. A pixel having a gradation value of 1 is shown.

  FIGS. 7B to 7E show thinned images DIV_A to DIV_D generated from the original image. The sampling position of each thinned image from the original image is as described with reference to FIG. 6, and the results of thinning at the aforementioned positions are shown in FIGS. 7B to 7E. In this example, the thinned images DIV_A to DIV_D are the same image. As shown in FIGS. 7B to 7E, the size of one pixel of the thinned image is four times the size of the original image.

  FIGS. 7F to 7H show the results of horizontal pixel shift, vertical pixel shift, and diagonal pixel shift performed using the above-described thinned image. FIG. 7F shows a result of horizontal pixel shift generated by combining the thinned images DIV_A and DIV_B. As shown in FIG. 7F, in the horizontal pixel shift, the resolution in the horizontal direction is increased, but the resolution in the vertical direction is not improved. FIG. 7G shows a result of vertical pixel shift generated by combining the thinned images DIV_A and DIV_D. As shown in FIG. 7G, in the vertical pixel shift, the resolution in the vertical direction is increased, but the resolution in the horizontal direction is not improved. FIG. 7H shows the result of the diagonal pixel shift adapted to black insertion in the present case, and is generated by combining the thinned images DIV_A and DIV_C. As shown in FIG. 7H, it can be seen that in the diagonal pixel shift, the resolution in the vertical and horizontal directions is improved, and high definition is achieved.

  As described above with reference to FIG. 7, in order to realize high definition by shifting pixels in both the vertical and horizontal directions, an image thinned out at the reference position and a diagonal direction thinned out from the reference position. It is necessary to perform diagonal pixel shift by combining images. The above is the mechanism for achieving high definition by shifting the diagonal pixels.

  Next, a mechanism for improving the visibility of a moving image by inserting black in the first embodiment using the present projection system will be described with reference to FIG. In the following, for ease of explanation, the description is made on the assumption that the projected image is shifted four times with non-pixel shift.

  FIG. 8A shows an image observed by the user when a black image is not inserted. The horizontal direction in the figure represents the pixel horizontal position of the projected image, and the vertical direction represents the time axis. The square in the figure represents one pixel, a bright pixel represents white, and a dark pixel represents black. In this embodiment, four projection images are output during one frame period, so that one frame period is represented by four pixels in the vertical direction. From this, it can be seen in FIG. 8 that a moving image in which a monochrome 1-pixel stripe image moves in the horizontal direction at 4 pixels / frame is projected four times. Also, a plurality of lines drawn diagonally to the lower right represent the viewpoint movement when the user follows the movement of a certain pixel in the moving image with eyes, and the left oblique line indicates viewpoint 1, viewpoint 2, viewpoint 3, and so on. Viewpoint 4, ... The user can visually recognize the result of time integration of the pixel passing through the oblique line of the viewpoint 1.

  As shown in FIG. 8A, when black is not inserted, the pixel passing through the oblique line of the viewpoint 1 is time-integrated to obtain an average of white pixels and black pixels. Is done. Similarly, the gray level is obtained by temporally integrating the pixels passing through the oblique lines of viewpoint 2, viewpoint 3, and viewpoint 4. Therefore, the image is visually recognized as the gray level in the entire range of the viewpoints 1 to 4, so that even if the user views the projected image in a wide field of view, the image is visually recognized as a constant gray level image. As described above, the high-frequency information included in the input image becomes dull, and if this phenomenon is similarly applied to a natural image or the like, it can be seen that the edge looks blurred.

  On the other hand, FIG. 8B shows an image observed by the user when black is inserted by replacing the other sub-frame image with a black image as in the first embodiment. The input image is the same as in FIG. The dotted-line pixels shown in FIG. 8B indicate pixels that have been replaced with black by black insertion, and the second and fourth sub-frames have been replaced with black images. As in the case of the description of FIG. 8A, when the pixels passing through the oblique line of the viewpoint 1 are integrated, the viewpoint 1 becomes the average of the black and white pixels, which is the gray level. On the other hand, at the viewpoint 2, only black pixels are present, and the gradation is black. Similarly, the viewpoint 3 has a gray level and the viewpoint 4 has a black level. The user recognizes the gray level and a black one-pixel stripe. Comparing the image observed by the user with the input image shows that the high-frequency component can be expressed although the contrast is reduced. In summary, high-frequency components are more likely to remain when black is inserted into the remote subframe in the first embodiment of FIG. 8B than when no black image is inserted as shown in FIG. 8A. You can see that there is. As a result, when the black insertion is performed, the dullness of the edge portion in the moving image is reduced, and a sharp appearance can be realized.

  The above is the description of the mechanism for improving the visibility of a moving image by black insertion in the first embodiment.

  In the first embodiment, the moving image visibility is improved by inserting black. However, when a part of the projected image is replaced with a black image, the projected luminance in one frame is reduced by half for the user. In order to suppress such a decrease in luminance, at the time of black insertion, the light source control unit 230 may control the output light amount of the light source 260 to be higher than when no black image is inserted. For example, the light source control unit 230 that controls the brightness of the light source 260 for projecting the projected image may be configured such that the brightness of the light source 260 when the black image is inserted is changed to the brightness of the light source 260 when the black image is not inserted. Control is performed so as to be brighter than the brightness. By setting the light amount of the light source at the time of black insertion higher than that at the time of non-insertion of a black image, it is possible to suppress a sense of incongruity caused by the user perceiving a change in projection plane luminance when the black insertion function is turned ON / OFF. can do.

  In addition, when black insertion is performed, if the input image is bright and the frequency is low, a bright image and a dark image are alternately projected, which may be perceived as flicker by the user. In order to suppress the flicker, the black insertion unit 245 may perform a process of offsetting the black image to increase the luminance with respect to the black image to be replaced by black insertion. By doing so, the luminance difference between the projection plane luminance when the thinned image generated from the input image is projected and the projection plane luminance when the offset-processed black image is projected becomes smaller, Flicker can be suppressed. At this time, the offset value applied to the black image may use a setting value recorded in the ROM 212 in advance, or the offset information is included in the black insertion control signal K_SIG so that the user can adjust the offset value via the operation unit 213. You may do it.

  In the first embodiment, when the thinning unit 243 generates the thinned images DIV_A to DIV from the input image IMG, pixels at predetermined positions in the input image IMG are sampled and generated. As described above, the four thinned images are generated by referring to all the pixels of the input image IMG once, and in the case of horizontal and vertical pixel shift, the observed image after the pixel shift is missing information from the input image IMG. There is no. On the other hand, in the diagonal pixel shift performed at the time of black insertion, two thinned images (eg, DIV_A and C) are used for pixel shift. Therefore, when the thinned image is created by the above-described method, the observation image becomes the input image IMG. On the other hand, the information amount is halved. If the input image information is missing, jaggies and false images tend to occur as observed image degradation as image quality. In order to suppress this, before the thinning unit 243 generates the thinned images DIV_A to DIV_D, a filtering process may be performed to blur the input image. Since the filtering process is performed using the pixel information around the target pixel when performing the blurring by the filtering process, if a thinned-out image is generated from the filtered image as a result, information loss from the input image can be reduced.

  As described above, according to the first embodiment, by replacing a thinned image that is projected in a diagonally shifted manner with a black image, the black insertion can be performed without reducing the effect of high resolution by pixel shifting as much as possible. It is possible to realize improvement of video visibility. Therefore, it is possible to provide a projection device capable of achieving both pixel shift and black insertion while maintaining the image processing unit 240 and the panel driving speed at 4 × speed.

<Modification>
Although the configuration and operation of the image processing unit 240 have been described with reference to FIGS. 2A and 3A to 3B, the configuration and operation of the image processing unit 240 are not limited thereto. For example, it may have a configuration as shown in FIG. 2B and operate as shown in FIGS. 3C to 3D.

  2B, the thinning unit 243 generates a thinned image corresponding to the timing signal SIG (i) from the image data IMG read by the memory control unit 241 and provides the thinned image to the selection unit 244 as IMG_C. The black insertion unit 245 generates a black image and provides it to the selection unit 244. The black insertion unit 245 may provide a black image to the selection unit 244 regardless of the timing signal SIG and the black insertion signal K_SIG. The selector 244 selects one of IMG_C and a black image according to the timing signal SIG (i) and the black insertion control signal K_SIG, and outputs the selected image as IMG_O. Hereinafter, the projection process according to the modified example will be described with reference to FIGS. 3C to 3D, the same processes as those in FIGS. 3A to 3B are denoted by the same step numbers.

  When the pixel shift drive control mode is set and the black insertion mode is set, the processing of S301 to S306 is executed. In S351, the thinning unit 243 generates a thinned image DIV_i according to SIG (i) (SIG (0) DIV_A, SIG (1) DIV_B, SIG (2) DIV_C, and SIG (3). DIV_D is generated) and supplied to the selection unit 244. The black insertion unit 245 supplies the black image to the selection unit 244 at the timing of the timing signals SIG (1) and SIG (3) or constantly. In S352 and S353, when i = 1 or i = 3, the selection unit 244 selects the black image supplied from the black insertion unit 245, and outputs this as IMG_O. In S352 and S354, the selection unit 244 selects DIV_A or DIV_C and outputs it as IMG_O. S312 to S315 are as described above. When the black insertion mode is set, the thinning unit 243 may control so as not to generate DIV_B and DIV_D at the timing of the timing signals SIG (1) and SIG (3).

  If the pixel shift drive control mode is set and the black insertion mode is not set, the process executes S301 to S303 and S316 to S318. In S361, the thinning unit 243 generates a thinned image DIV_i according to SIG (i) and supplies the thinned image DIV_i to the selecting unit 244. When the black insertion mode is not set, the black insertion unit 245 may stop supplying the black image. In S362, the selection unit 244 selects the thinned image DIV_i supplied from the thinning unit 243, and outputs it as IMG_O. S322 to S325 are as described above.

<Second embodiment>
The first embodiment has been described on the premise that the black insertion is performed when the user instructs the black insertion. In the second embodiment, it is determined whether or not the input image is a moving image, and when it is determined that the input image is a moving image, black insertion is performed. By performing the black insertion only when the input image is a moving image in this manner, the user can observe the projected image with the optimum image quality setting without being particularly conscious of the moving image quality. Hereinafter, the projection device 200 according to the second embodiment will be described. The basic configuration of the projection device 200 is the same as that of the first embodiment (FIG. 1).

  FIG. 9 is a block diagram illustrating an example of the internal configuration of the image processing unit 240 according to the second embodiment. In the second embodiment, the image processing unit 240 in the projection device 200 includes an image determining unit 941 and an image memory 942 connected to the image determining unit 941. Further, instead of the black insertion control signal K_SIG, an image determination signal SIG_MOVIE generated by the image determination unit 941 is used. Note that the same image memory 242 and image memory 942 may be used.

  The image determination unit 941 determines whether the input image is a still image or a moving image, and outputs the determination result to the black insertion unit 245 as an image determination signal SIG_MOVIE. An example of an image determining method by the image determining unit 941 will be described. First, the image discriminating unit 941 writes the input image of the current frame into the image memory 942, and reads out the input image of the previous frame from the image memory 942. Then, the image determining unit 941 obtains a difference between the input image of the current frame and the input image of the previous frame, and regards an area where the difference value is larger than a specific value as having movement from the previous frame. Then, for example, the image determining unit 941 determines that the input image is a moving image when the area of the region having a large difference is larger than a certain area, and determines the input image when the area of the region having a large difference is smaller than the certain area. Is determined to be a still image. Note that the image determination method is not limited to this. For example, when the format of the image input by the image input unit 220 is the moving image format, the image is determined to be a moving image, and when the image is the still image format, the image is determined to be a still image, and the determination result is added to the input image IMG. May be referred to. The format of the image can be recognized, for example, by referring to the header of the file.

  Next, projection processing according to the second embodiment will be described with reference to the flowchart in FIG. The flowchart of FIG. 10 shows a process that replaces S301 to S303 of the projection process (FIG. 3A) of the first embodiment. In FIG. 10, the same processes as those in FIGS. 3A and 3B are denoted by the same step numbers.

  In S1001, the image determining unit 941 in the image processing unit 240 determines whether the input image IMG is a still image or a moving image. The image determination unit 941 outputs the image determination result to the black insertion unit 245 as an image determination signal SIG_MOVIE. The image discrimination signal SIG_MOVIE is, for example, a binary signal. A value of 0 indicates that the input image IMG is a still image, and a value of 1 indicates that the image is a moving image. If it is determined in step S1002 that the image determination result indicates a moving image (SIG_MOVIE = 1), the process proceeds to step S304, in which pixel shift projection with black insertion is performed. The pixel shift projection accompanied by black insertion is as shown in S304 to S315 in FIG. 3A. On the other hand, if the image determination result is a still image (SIG_MOVIE = 0), the process proceeds to S316, in which pixel shift projection without black insertion is performed. The pixel shift projection without black insertion is as shown in S316 to S325 in FIG. 3B.

  The processing for achieving both high definition by pixel shift and improvement in moving image visibility by black insertion in the second embodiment has been described above. Note that the configuration (FIG. 2B) and processing (FIGS. 3C and 3D) described as a modification of the first embodiment can also be applied. In this case, the processing shown in FIG. 10 is the same as the processing of steps S301 to S303 in FIG. 3C.

  As described above, according to the second embodiment, it is possible to provide a projection device that can achieve both pixel shifting and black insertion while maintaining the image processing unit 240 and the panel driving speed at 4 × speed. Further, in the second embodiment, by performing image discrimination of an input image and performing black insertion when it is determined that a moving image has been input, the user can view the image with an optimal image quality setting without being particularly conscious of the image quality of the moving image. Will be able to

<Third embodiment>
In the first embodiment and the second embodiment, both the pixel shift and the black insertion are achieved by replacing one of two sets of two thinned images whose projection positions are shifted in the diagonal direction with a black image. The thinned image and the black image are projected alternately. In the third embodiment, the selection unit 244 changes the output order of the thinned images so that two thinned images shifted in the diagonal direction are continuously output. As a result, the time for displaying the black image is twice as long as that in the first embodiment, and the effect of improving the visibility of the moving image can be further enhanced. Hereinafter, the projection device 200 according to the second embodiment will be described. The basic configuration of the projection device 200 is the same as that of the first embodiment (FIG. 1).

  Further, the internal configuration of the image processing unit 240 in the projection device 200 according to the third embodiment is the same as that of the first embodiment (FIG. 2). In the third embodiment, a black insertion control signal K_SIG indicating the ON / OFF state of the black insertion function is input to the selection unit 244. The selection unit 244 determines an output image in each subframe period based on both the timing signal SIG and the black insertion control signal K_SIG. Details will be described later.

  The black insertion control signal K_SIG is also input to the pixel shift device control unit 284 via the bus 199. Similarly to the selection unit 244, the pixel shifting device control unit 284 determines the attitude of the pixel shifting device 283 based on the timing signal SIG and the black insertion control signal K_SIG. Details regarding the attitude and projection position of the pixel shifting device 283 during each sub-frame period will be described later.

  Next, projection processing according to the third embodiment will be described with reference to the flowchart in FIG. The flowchart in FIG. 11 shows a process that replaces the processes shown in S305 to S315 in FIG. 3A.

  When it is determined that black insertion is to be performed (YES in S303), the black insertion control signal K_SIG is turned on and initialized to i = 0 (S304). When receiving the timing signal SIG (i) (YES in S305), the memory control unit 241 reads the input image IMG from the image memory 242 and provides it to the thinning unit 243 (S306). The thinning unit 243 generates thinned images DIV_A to DIV_D from the input image IMG supplied from the memory control unit 241 (S307). In step S1101, the selection unit 244 selects a thinned image based on the timing signal SIG (i) such that one set of thinned images having a positional relationship shifted in a diagonal direction is continuously output. In the present embodiment, the selection unit 244 selects the thinned image DIV_A when i = 0, the thinned image DIV_C when i = 1, the thinned image DIV_B when i = 2, and the thinned image DIV_B when i = 3. In this case, the thinned image DIV_D is selected and output as IMG_C.

  FIG. 12 is a diagram illustrating the correspondence between the timing signal SIG and the image generated by the image processing unit 240 according to the third embodiment. The timing signal SIG in FIG. 12A and the output image of the memory control unit 241 in FIG. 12B are the same as in the first embodiment (FIGS. 4A and 4B). FIG. 12C illustrates a selection order of the thinned image by the selection unit 244 when black is inserted in the lower part of FIG. As shown in FIG. 12C, when black insertion is performed, thinned-out images DIV_A and DIV_C or DIV_B and DIV_D that are positioned diagonally are output continuously.

  Next, in step S1102, the black insertion unit determines whether i is 2 or 3. If i is 2 or 3, the process proceeds to S310, and the black insertion unit 245 outputs the black image as IMG_O. On the other hand, if i is other than 2 or 3 (i is 0 or 1), the process proceeds to S311, and the black insertion unit 245 outputs the thinned image output by the selection unit 244 as IMG_O. In this way, the black insertion unit 245 replaces the image other than the pair of thinned images having a positional relationship shifted in the diagonal direction with the black image, and continuously outputs the thinned image and the black image. In the third embodiment, the black insertion unit 245 outputs the thinned image DIV_A, the thinned image DIV_C, the black image, and the black image in this order as the output image IMG_O.

  FIG. 12D shows the insertion state of the black image when the thinned images DIV_A and DIV_C are projected in the middle stage. The lower part of FIG. 12D shows the insertion state of the black image when the thinned images DIV_B and DIV_D are projected. The projection control in the lower part of FIG. 12D can be realized, for example, by exchanging the YES branch and the NO branch in S1102 of FIG. As shown in FIG. 12D, the thinned image and the black image are projected in a continuous subframe period. In this way, by switching the order of the output images by the selection unit 244, the black image projection period can be set longer than in the first embodiment.

  In step S1103, the CPU 210 issues an instruction to the pixel shifting device control unit 284, and the pixel shifting device control unit 284 controls the pixel shifting device 283 based on the black insertion control signal K_SIG and the timing signal SIG. The black insertion control signal K_SIG is a control signal indicating that the black insertion function is ON. Based on the timing control signal SIG, the pixel shifting device control unit 284 correctly shifts the thinned-out image of each sub-frame period shown in “when black is inserted” in FIG. 12D to the projection position shown in FIG. The attitude of the pixel shift device 283 is controlled so that the image is projected. More specifically, when black is inserted as shown in the middle part of FIG. 12D, since the first sub-frame is the thinned-out image DIV_A, the pixel shift device 283 is controlled so that it is projected to the reference position (FIG. )). In order to project the thinned image DIV_C in the second sub-frame, the attitude of the pixel shift device 283 is controlled so that the thinned image DIV_C is projected at a position shifted by 0.5 pixel in the horizontal and vertical directions from the reference position (FIG. 5B). In the third and fourth sub-frames, since a black image is projected, there is no particular designation for the attitude of the pixel shift device 283. Even when black is inserted as shown in the lower part of FIG. 12D, the projection position in each sub-frame is determined based on the same concept.

  On the other hand, when black insertion is not performed, the processing described with reference to FIG. 3B is executed. That is, the selection unit 244a outputs the images in the order of the thinned images DIV_A, B, C, and D (the order in which the amount of movement of the pixel shift is the smallest) based on the timing signal SIG, and the black insertion unit 245 outputs the images as they are Output as image IMG_O. FIG. 12D illustrates an output image of the selection unit 244a when black is not inserted in the upper part of FIG. The control processing of the projection position P (i) in S322 of FIG. 3B is the same as in the first embodiment. In other words, based on the timing control signal SIG, the pixel shifting device control unit 284 sets the thinned image of each sub-frame period shown in the upper part of FIG. 12D to the projection position shown in FIGS. 5A to 5C. The attitude of the pixel shift device 283 is controlled so that the image is correctly projected on the pixel.

  As described above, in the third embodiment, the order of movement of the projection position between a plurality of positions differs between when the black image is inserted and when it is not. In the above example, when the black image is inserted, the movement of the projection position is controlled so that the projection position where the sub-frame is used as the projection image and the projection position where the black image is used as the projection image are respectively continuous. I have. On the other hand, when the black image is not inserted, the movement of the projection position is controlled so that the movement amount of the projection position is minimized. The above is the description of the sequence according to the third embodiment until both high definition by pixel shift and improvement in moving image visibility by black insertion are achieved.

  Next, a mechanism in which the effect of improving the visibility of the moving image according to the third embodiment is higher than the effect of improving the visibility of the moving image according to the first embodiment will be described with reference to FIG.

  FIG. 13A shows an image observed by a user when a black image is not inserted. The definition of each axis and figure in the figure is the same as in the first embodiment (FIG. 8). The description will be made on the assumption that the bright pixels of the input image are represented in 255 bits in 8-bit expression, the dark pixels are 0 gradations, and the intermediate gradation pixels are 128 gradations. The moving image speed is 4 pixels / frame as in the first embodiment. At this time, when a pixel passing through the oblique line with respect to the oblique line of the viewpoint 1 is integrated with respect to time, 128 gray levels are obtained. Similarly, when pixels passing through the oblique lines of the viewpoint 2, the viewpoint 3, and the viewpoint 4 are integrated over time, 128 gradations are obtained. As a result, it can be seen that the image observed by the user has a constant gray level, and the frequency components included in the input image are lost due to moving image blur.

  Next, with reference to FIG. 13B, an image observed by the user when black is inserted in the interval sub-frame shown in the first embodiment in the input image of FIG. 13A will be described. When black is inserted into the remote subframe as in the first embodiment, an image having a certain gray level is visually recognized by the user based on the same concept as in FIG. As a result, it can be seen that the frequency components included in the input image are lost due to the moving image blur as in the case where the black insertion is performed but the black insertion is not performed.

  The image observed by the user when black is inserted in the latter half sub-frame of the input image described above will be described with reference to FIG. Pixels passing through the oblique line of the viewpoint 1 continue in the order of 128 gradations, 255 gradations, 0 gradations, and 0 gradations, are time-integrated, and visually recognized as approximately 96 gradation values by the user. Pixels passing through the oblique line of the viewpoint 2 are also time-integrated in the same manner, and are visually recognized as approximately 96 gradation values. On the other hand, the pixels passing through the oblique line of the viewpoint 3 continue in the order of 128 gradations, 0 gradations, 0 gradations, 0 gradations, are time-integrated, and visually recognized by the user as about 32 gradation values. Also, the viewpoint 4 is similarly visually recognized as approximately 32 gradation values. As a result, the image observed by the user has low contrast with respect to the input image, but is visually recognized as an image having frequency components left. When this phenomenon is applied to a natural image, it can be understood that sharpness of an edge portion is likely to remain even for a moving image having a lower frequency component, and that blurring of the moving image is less likely to be visually recognized.

  As described above with reference to FIG. 13, by continuing the subframes to be replaced with a black image, the effect of improving the visibility of a moving image by inserting black can be further enhanced.

  In the third embodiment, when black is not inserted, the images are projected in the order of the thinned images DIV_A, B, C, and D. When black is inserted, the output order of the thinned images is changed, and DIV_A, C, B, and D are changed. And Hereinafter, the reason why the order of the output images is changed depending on whether black is inserted or not is described. When black is not inserted, when the pixels are moved like the thinned images DIV_A, B, C, and D, the amount of pixel movement between subframes is always 画素 pixel, and the minimum amount of movement can be maintained. As a result, it is possible to secure as long as possible the time during which the image is projected at the desired projection position during the sub-frame period, and it is possible to improve the image quality when the pixels are shifted. Therefore, when black is not inserted, the transition direction of the projection position between each sub-frame is desirably either the vertical or horizontal direction in order to shorten the transition time for shifting the projection position as much as possible.

  On the other hand, when black is inserted, as described above, it is desirable that the non-black image be an image having a projection positional relationship shifted in the diagonal direction from the viewpoint of higher definition by shifting pixels. In addition, as described above, it is preferable that black image display by black insertion is performed in a continuous subframe period from the viewpoint of improving moving image visibility. As described above, in order to satisfy all the conditions for high image quality when a black image is not inserted and when it is inserted, it is preferable to change the display order of each thinned image when the black image is not inserted and when it is inserted.

  As described above, according to the third embodiment, it is possible to provide a projection device that can achieve both pixel shifting and black insertion while maintaining the image processing unit 240 and the panel driving speed at 4 × speed. Further, in the third embodiment, a thinned image shifted in the diagonal direction at the time of black insertion is continuously output, and the remaining half frame is replaced with a black image, so that the black image display period is made longer than in the first embodiment. The effect of improving the visibility of the moving image by the insertion can be further enhanced.

  The processing for achieving both high definition by pixel shift and improvement in moving image visibility by black insertion in the third embodiment has been described above. The configuration (FIG. 2B) and the processing (FIGS. 3C and 3D) described as a modification of the first embodiment can be applied to the third embodiment. In this case, in S351 of FIG. 3C, the thinning unit 243 generates a thinned image in the order of DIV_A, DIV_C, DIV_B, and DIV_D while i increases from 0 to 3, as in SS1101 of FIG. The generation of DIV_B and DIV_D may be omitted. Also, in S352 of FIG. 3C, as in S1102, the YES branch may be made when i = 2 or 3, and the NO branch may be made in other cases.

<Other embodiments>
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

200: Projection device, 210: CPU, 211: RAM, 212: ROM, 213: Operation unit, 220: Image input unit, 230: Light source control unit, 240: Image processing unit, 260: Light source, 250: Light modulation element control Unit, 270: light modulation element, 283: pixel shifting device, 284: pixel shifting device control unit, 285: projection optical system, 286: projection optical system control unit, 287: timing signal generation unit

Claims (20)

Moving means for moving the projection position of the projection image between the plurality of positions in synchronization with the plurality of sub-frame periods obtained by dividing one frame period;
Projection means for outputting the projection image in synchronization with the plurality of sub-frame periods,
Image processing means for processing an input image to generate the projection image,
The image processing means,
Generating a plurality of sub-frames from one frame of the input image;
Outputting a sub-frame corresponding to the projection position among the plurality of sub-frames as the projection image in a sub-frame period in which the projection position is controlled to a predetermined position among the plurality of positions; A black image is output as the projection image.   The image processing unit may be configured to control the projection position during a sub-frame period in which the projection position is controlled to a first position of the plurality of positions and a second position separated by a predetermined distance in the horizontal and vertical directions with respect to the first position. Any one of the plurality of sub-frames is output as a projection image, and a black image is projected in a sub-frame period in which the projection position is controlled at a position other than the first position and the second position among the plurality of positions. The projection device according to claim 1, wherein the projection device outputs the image. The moving means sequentially moves a projection position of a projection image between four positions shifted by a predetermined distance in a horizontal direction and / or a vertical direction,
The image processing means outputs one of the plurality of sub-frames as a projection image at two of the four positions that are arranged in a diagonal direction, and outputs a black image at the other two projection positions. The projection device according to claim 1, wherein the output is output as   4. The projection apparatus according to claim 1, wherein the image processing unit increases the luminance of the black image by offsetting the black image and uses the black image as the projection image. 5.   The apparatus according to claim 1, wherein the image processing unit generates the plurality of subframes by thinning out different positions from the input image.   The projection apparatus according to claim 5, wherein the image processing unit generates the plurality of subframes after performing a filtering process on the input image. A determination unit configured to determine whether to perform the insertion of the black image,
The image processing unit outputs any one of the plurality of sub-frames as a projection image in all of the plurality of sub-frame periods when it is determined that the insertion of the black image is not performed. Item 7. The projection device according to any one of Items 1 to 6. Further comprising light source control means for controlling the brightness of the light source for projecting the projection image,
The light source control unit controls the light source such that the brightness of the light source when the black image is inserted is brighter than the brightness of the light source when the black image is not inserted. The projection device according to claim 7.   The projection apparatus according to claim 7, wherein the determination unit determines that the black image is to be inserted when the instruction to insert the black image is received from the user. An image determination unit that determines whether the input image is a moving image,
The determining unit determines that the black image is to be inserted when the input image is determined to be a moving image by the image determining unit, and determines that the input image is not a moving image by the image determining unit. 9. The projection apparatus according to claim 7, wherein it is determined that insertion of a black image is not performed.   The projection apparatus according to claim 10, wherein the image determination unit determines whether the input image is a moving image based on a difference between a plurality of consecutive frames of the input image.   12. The projection according to claim 6, wherein the moving unit changes the order of movement between the plurality of positions when black image insertion is performed and when black image insertion is not performed. apparatus.   The moving means, during the execution of the insertion of the black image, between the plurality of positions so that the projection position where the sub-frame is used as the projection image and the projection position where the black image is used as the projection image are respectively continuous 13. The projection device according to claim 12, wherein the projection position is controlled.   14. The method according to claim 12, wherein the moving unit moves the projection position between the plurality of positions so as to minimize the amount of movement of the projection position when black image insertion is not performed. Projection device. A moving step of moving a projection position of a projection image between a plurality of positions in synchronization with a plurality of sub-frame periods obtained by dividing one frame period;
A projection step of outputting the projection image in synchronization with the plurality of sub-frame periods,
An image processing step of processing an input image to generate the projection image,
In the image processing step,
Generating a plurality of sub-frames from one frame of the input image;
Outputting a sub-frame corresponding to the projection position among the plurality of sub-frames as the projection image in a sub-frame period in which the projection position is controlled to a predetermined position among the plurality of positions; And outputting a black image as the projection image to the projection device.   In the image processing step, during a sub-frame period in which the projection position is controlled to a first position of the plurality of positions and a second position horizontally and vertically separated by a predetermined distance from the first position. Any one of the plurality of sub-frames is output as a projection image, and a black image is projected in a sub-frame period in which the projection position is controlled at a position other than the first position and the second position among the plurality of positions. The projection method according to claim 15, wherein the image is output as an image. In the moving step, the projection position of the projection image is sequentially moved between four positions shifted by a predetermined distance in the horizontal direction and / or the vertical direction,
In the image processing step, one of the plurality of sub-frames is output as a projection image at two projection positions arranged in a diagonal direction among the four positions, and a black image is projected at the other two projection positions. The projection method according to claim 15, wherein the output is performed as: An image determining step of determining whether the input image is a moving image,
When the input image is determined to be a moving image by the image determining step, it is determined that a black image is to be inserted. When the input image is determined to be not a moving image by the image determining step, a black image is inserted. 16. The projection method according to claim 15, further comprising: a determination step of determining not to perform.   16. The method according to claim 15, wherein in the moving step, the order of movement between the plurality of positions is different between when the black image is inserted and when it is not inserted.   A program for causing a computer to execute each step of the projection method according to claim 15.

Images