JP4178657B2 - Panorama shooting method and imaging apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a panoramic imaging technique using an imaging apparatus such as a digital camera.
[0002]
[Prior art]
When obtaining an image of a subject with a wide field of view, such as a landscape, with an imaging device, a plurality of images are obtained by dividing and capturing the landscape within a range of angle of view while moving the camera. Some cameras have a panoramic shooting function that recreates a wide field of view by shifting them seamlessly.
When the panorama shooting mode is selected with an image pickup apparatus equipped with such a panorama shooting function, a part of the image shot immediately before is displayed as an alignment guide on the left or right edge of the LCD screen, and the next subject image is displayed. In some cases, several images are captured while repeating the operation of superimposing a part of the image on the alignment guide portion.
[0003]
In this panorama shooting method, for example, when it is intended to obtain a panoramic image by continuously shooting a landscape as shown in FIG. 11A, first, the landscape portion of FIG. 11B is shot (shutter). The image data is recorded, but the right end (portion A) of the recorded image recorded at this time is left as an alignment guide. When the user moves the camera to the right, the alignment guide portion A at the right end of FIG. 11B is displayed on the left side as shown in FIG. 11C, and a landscape image (through image) being captured is displayed. ). Then, when the through image matches well with this alignment guide portion (FIG. 11 (d)), if the shutter is pressed, panoramic image shooting with a smooth joint can be performed, and a series of panoramic shootings with little vertical deviation can be performed. This is possible (FIG. 11 (e)).
[0004]
[Problems to be solved by the invention]
In the above panorama shooting method, a panoramic image can be obtained by displaying the alignment guide on the finder (liquid crystal screen) with a simple operation and with a smooth seam without any vertical displacement. However, as shown in FIG. When the panorama is shot, when the last image n is shot after shooting the images 1, 2,..., N−1, the end of the last image n and the first image 1 can be matched. There was a problem that the joints were not consistent because they were not.
[0005]
The present invention has been made in order to solve the above-described problems, and in the panorama shooting method and the imaging apparatus capable of easily aligning the joint between the last image and the first image when performing 360 ° panorama shooting. For the purpose of provision.
[0006]
[Means for Solving the Problems]
In order to solve the above problems, the invention according to claim 1 divides and captures a subject image with a wide field of view within a range of an angle of view, and joins a plurality of images obtained by the imaging to subject the subject with a wide field of view. A panoramic image capturing method for obtaining a panoramic image of an image, wherein when the last image of the plurality of images captured to obtain the panoramic image is captured, the last image is displayed in a shooting standby state. The end area of the image captured immediately before is displayed superimposed on one end of the subject image displayed on the screen, and the end area of the first captured image is displayed superimposed on the other end of the subject image. The end region of the captured image is superimposed on one end of the subject image, and the end region of the first captured image is superimposed and displayed on the other end of the subject image. End region and the other end A key operating at least one of the width of the superimposed displayed end region On the basis of By adjusting, it is possible to align the last image with the image captured immediately before and to align the last image with the image captured first.
[0007]
Further, the invention according to claim 2 is characterized in that the superimposed display is formed by transparently displaying an end area of an adjacent captured image at an end of the subject image.
[0008]
Furthermore, the invention of claim 3 is provided with an image sensor on which a subject image is formed, and the subject image with a wide field of view is divided within a range of angle of view and captured multiple times to obtain a panoramic image of the subject image with a wide field of view. In the imaging apparatus configured as described above, display means for displaying the subject image formed on the image sensor during standby for image capturing, and the subject image formed on the image sensor on the display means are displayed. First superimposed display means for superimposing and displaying an end region of the image captured immediately before the plurality of times of imaging on one end of the subject image displayed when being displayed, and the displayed Second superimposed display means for superimposing and displaying an end region of an image captured first among the plurality of times of imaging on the other end of the subject image; The edge region of the image captured immediately before by the first superimposed display means is displayed superimposed on one end of the subject image, and the edge region of the image first captured by the second superimposed display means is In a state of being superimposed on the other end of the subject image Operation key means is provided for varying a width of at least one of the end region of the one end superimposed and the end region of the other end superimposed.
[0009]
Furthermore, the invention of claim 4 further includes start instructing means for instructing start of superimposed display by the second superimposed display means, and the first superimposed display means images the one end of the subject immediately before the imaging. When an instruction from the start instructing unit is given while the end region of the displayed image is superimposed and displayed, superimposed display by the second superimposed display unit is performed.
[0010]
Further, the invention of claim 5 is further characterized by further comprising a storage recording means for storing and recording an image superimposed on the display means as one image file.
[0011]
Furthermore, the invention of claim 6 further includes a superimposition area information detection unit that detects a region width superimposed on the display unit to obtain superimposition area information, and a superimposition area detected by the superposition area information detection unit. Superimposition area information storage means for storing information is provided.
[0012]
Claim 7 The present invention further includes an image storage means for storing an image obtained by the plurality of times of imaging. When, A panoramic image synthesizing unit that obtains the panoramic image by joining and synthesizing the images stored by the image storing unit based on the superimposing region information stored in the superimposing region information storing unit; To do.
[0013]
The invention of claim 8 is further characterized by comprising panoramic image storage means for storing a panoramic image obtained by the panoramic image composition means.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
[Circuit configuration example]
FIG. 1 is a block diagram illustrating an example of a circuit configuration of a digital camera as an embodiment of an imaging apparatus to which the panoramic photographing method of the present invention can be applied. The digital camera 100 includes an optical system 11, a signal conversion unit 12, and a signal. A processing unit 13, a DRAM 14, a control unit 20, an operation unit 30, a display unit 40, a flash memory 50, and a power supply 90 are included. Further, an output unit 15 that outputs an image, shooting information, and the like recorded in the flash memory 50 to an external device such as a personal computer may be provided.
[0015]
The optical system 11 includes an imaging lens 111 and an automatic diaphragm mechanism (not shown), and focuses the luminous flux of the subject image collected through the imaging lens 111 on a subsequent imaging device (CCD). The signal conversion unit 12 includes a CCD, a signal conversion circuit, and the like. The signal conversion unit 12 converts an image formed on the CCD through the optical system 11 in the previous stage into an electrical signal, converts the image into digital data, and outputs the digital data at a constant cycle. . The signal processing unit 13 processes the output from the signal conversion unit 12 in the normal imaging mode to obtain signal components such as digital luminance and color difference multiplex signals (Y, Cb, Cr data), and Y, Cb, Cr data (hereinafter referred to as image data) is transferred to the DRAM 14 and temporarily stored. Further, in order to display a through image (moving image), the captured image data is thinned out and sent to the display unit 40. When the shutter operation is performed, the image data at that time is sent to the display unit 40 to display a still image.
[0016]
The signal processing unit 13 also reads out the image data written in the DRAM 14 at the time of image recording and storage, performs JPEG compression processing, and stores the image recorded in the recording memory (flash memory) 50 at the time of reproduction. The image data is reproduced by decompressing the data.
[0017]
The DRAM 14 is used as a working memory, and is used as an image buffer area for temporarily storing captured images, a working area at the time of JPEG compression / decompression, and the like. In the panoramic shooting mode, the area configuration as shown in FIG. 2 can be adopted.
[0018]
The control unit 20 has a microprocessor configuration including a CPU, a RAM, a program storage ROM, a timer, and the like. The CPU is connected to each of the above-described circuits and a power supply switching switch (not shown) via a bus line, The entire digital camera 100 is controlled by a control program stored in the program storage ROM, and a processing program corresponding to each mode stored in the program storage ROM corresponding to the status signal from the operation unit 30 A panorama shooting processing program is taken out, and execution control of each function of the digital camera 100, for example, execution of imaging and playback of recorded images, etc., and execution control of image alignment processing in the panorama shooting mode, etc. I do.
[0019]
The operation unit 30 includes keys and switches such as a processing mode changeover switch, a function selection button, a display position movement key 35, a shutter button 36, and a processing mode selection switch 37, and when these keys or switches are operated. A status signal is sent to the control unit 20.
[0020]
As the display position movement key 35, a plus / minus (+/−) key for instructing movement in the horizontal direction is used in the embodiment, but a crossbar for designating up / down / left / right movement may be used. Moreover, it is not limited to these. Further, the shutter button 36 has a two-stage pressing structure (a structure in which the shutter can be pressed halfway and fully pressed). In the embodiment, when the shutter button 36 is fully pressed in the panoramic shooting mode, an image within the angle of view is displayed. When the image is picked up and pressed halfway, the image of that angle of view (the nth (last) image), the n−1th image, and the first image (first image) are stitched together (see FIG. 6, FIG. 7 and FIG. 8 (see steps S7 to S9)). Further, a function for switching to the alignment process may be assigned to a key other than the shutter button 36 (such as a function selection button).
[0021]
The display unit 40 is composed of a video encoder, a VRAM, a liquid crystal screen (LCD), and the like. By the display control of the control unit 20, a through image at the time of imaging, a reproduced image at the time of reproduction, and a menu for selection at the time of processing selection (or Icon) and messages for operating instructions are displayed.
[0022]
The flash memory 50 stores and records an image file (FIG. 3) in which the captured image (compressed image data) is stored together with image information of the image. Instead of the flash memory 50, a memory card that can be attached to and detached from the main body of the digital camera 100 may be used.
[0023]
[Example of DRAM layout]
FIG. 2 is an explanatory diagram showing an example of the layout of the DRAM 14 during panoramic shooting. FIG. 2A shows an image file stored in the flash memory 50 for each image captured by the digital camera 100 for each angle of view during panoramic shooting. FIG. 8B shows a layout example of the DRAM 14 in the case of the configuration (FIG. 8), and FIG. It is a layout example of DRAM14 in the case of the structure (FIG.9 (b ')) which preserve | saves and records.
[0024]
In (a), reference numeral 141 denotes a through image buffer for storing the captured through image data. Reference numeral 142 denotes an image buffer for storing the immediately previous captured image. Reference numeral 145 indicates a work area (working area), and reference numeral 143 stores the first image reproduced when the last (n-th) image and the first (first) image are connected. This is an image buffer used for this purpose (FIG. 8 (step S7)). In other cases, it is used as a work area. Note that the image buffer 143 and the work area 145 may be secured separately (when the first image is stored in the image buffer 143 at the time of imaging in step S5 in FIG. 8).
[0025]
In FIG. 5B, reference numeral 141 denotes a through image buffer for storing the captured through image data. Reference numeral 142 ′ denotes a captured image storage area for storing each captured image, and a captured image buffer (1) for storing the nth captured image from the first captured image,... And a captured image buffer (N). Reference numeral 144 denotes an overlapping position information storage area for storing overlapping position information (transmission display position information) of each captured image. Each overlapping position information is stored in the position information column (i) (1 ≦ i ≦ N). ). Reference numeral 145 indicates a work area. It should be noted that the reproduced image buffer (1) for storing the captured image storage area 142 ′ from the first reproduced image to the nth reproduced image when the panoramic image is reproduced, the reproduced image buffer (N ) Can also be used as a reproduced image area.
[0026]
[Image file]
FIG. 3 is a diagram showing an embodiment of an image file. An image file 55 is created for each captured image and is stored and recorded in the flash memory 50. The image file 55 is composed of a header area 56 and an image area 57 as shown in FIG. The header area 56 includes an image file identification area 561 for storing an image file name (identifier) as shown in FIG. 5B, and image information such as the type of shooting method and shooting conditions such as normal shooting, close-up shooting, and panoramic shooting. An image information area 562 is stored, and an image area 57 stores a JPEG-compressed image (image data). Further, in the image information area 562, when the image stored in the image area 57 is an image captured in the panorama shooting mode as shown in (b), the image file name of the panorama shot image immediately before being superimposed at the time of reproduction is displayed. A superposition position field 562 for storing superposition position information (horizontal position on the image) indicating the degree of superposition of the superposition image file field 562-j for storing (identifier) and the immediately preceding panoramic image at the time of reproduction. -K is provided. In the case of 360 ° panoramic shooting, the overlay position information 562 k indicating the degree of overlap between the first image and the nth image is stored in the overlay position column 562-k of the first image file. It becomes.
[0027]
That is, in the embodiment, in the case of panoramic shooting, the overlapping position information of the first and second images is stored as (q2) in the image file of the second image, and the second and third images are stored. Is stored in the image file of the third image as (q3),..., And the overlapping position information of the (n−1) th and nth images is the image of the nth image. Store (qn) in the file. Further, in the case of 360 ° panoramic shooting, as shown in FIG. 6, after the n-1 and nth images and the n and 1st images are aligned, n−1 images are obtained. The overlapping position information of the nth image and the nth image is stored as (Qn) in the nth image file, and the overlapping position information of the nth image and the first image is stored in the image file of the first image. (Q1) is stored. In the embodiment, the superimposition position information is such that the transmission display position (overlap width) of the second to (n-1) th sheets is a fixed value (ie q2 = ... = qn-1), and the (n-1) th sheet. Although the nth sheet and the overlapping position of the nth sheet and the first sheet are the same (Qn = Q1), the present invention is not limited to this. Note that n = N.
[0028]
Further, the same configuration may be used when images captured for each angle of view at the time of panorama shooting are combined and saved and recorded in the flash memory 50 as one image file (combined panorama image file). That is, the overlap position column 562-k stores the head and tail overlap position information of the image. It is not stored in the superimposed image file column 562-j.
[0029]
In the image area 57, each captured image stored in the image buffer 142 ′ of the DRAM 14 is based on the superposition position information stored in the position information fields (1) to (N) of the position information storage area 144. After synthesis (bonding synthesis), JPEG compression processing is performed and stored (FIG. 9 (b ′)).
[0030]
<360 ° panorama shooting method>
FIG. 4 is an explanatory diagram showing the relationship between each image during panorama shooting, FIG. 5 is an explanatory diagram of alignment with an image captured immediately before panorama shooting, and FIG. 6 is the nth image at 360 ° panorama shooting. FIG. 6 is an explanatory diagram of joining of the first image and the first image.
[0031]
In the panoramic photographing method according to the present invention, a continuous landscape is photographed by being divided into images having a certain angle of view as shown in FIG. 4 or FIG. Therefore, it is necessary to match the boundaries of the adjacent images at the time of shooting. Therefore, as shown in FIG. 5, the predetermined width q (q = fixed value) of the right region of the image 70-(i−1) captured immediately before is transmitted to the left side portion (hereinafter referred to as a transmissive display portion) 71 of the through image 60. Display (i = 2 to n−1). Next, the user performs image alignment after performing alignment (moving the digital camera 100 so that the transmissive display image and the image of the transmissive display portion 71 of the through image 60 match).
[0032]
In 360-degree panoramic shooting, the through image, which is the nth image candidate, and the first image are aligned. In this case, by the user's key operation (half-pressing the shutter button 36 in the embodiment). As shown in the example of FIG. 6A, the screen display is switched, and the captured images (n−1 right side region and left side of the first image are displayed on both sides of the through image 60 which is the n th image candidate. Area) is displayed in a transparent manner by widths Q and Q (FIG. 6A). Here, the widths (= transparent display position information) Q and Q of the transmissive display portions 71 and 71 ′ can be expanded and contracted (variable) simultaneously in the horizontal direction by a user operation (FIG. 6B).
[0033]
Therefore, the user adjusts the width of the image to be transmissively displayed (= the width Q of the transmissive display portions 71 and 71 ′) simultaneously (by the same width) by one operation of a predetermined key (for example, the display position movement key 35). Thus, the through image 60 which is the nth image candidate, the n−1th image 70- (n−1), and the first image 70-1 can be aligned (FIG. 6 ( c)).
[0034]
Note that the width of the transmissive display portions 71 and 71 ′ is set to Qn ≠ Q1, one of the transmissive display portion 71 and the transmissive display portion 71 ′ is fixed, and only the other transmissive display portion is made variable by key operation. May be. In the embodiment, the right area of the previous image is transparently displayed on the left part of the through image (in the case of 360 ° panoramic shooting, the left area of the first image is further transparently displayed on the right part of the through image). In order to make it easy to understand the coincidence / non-coincidence at the overlapping part at the time of alignment, a through image is obtained by inverting the right side area of the previous image (in addition, the left side area of the first image in the case of 360 ° panoramic shooting) instead of the transparent display 60 left alignment portions 71 (and right alignment portions 71 ′) may be combined and displayed. Further, the left portion (and the right portion) of the through image 60 may be displayed in an inverted manner. In this way, overlapping portions cancel each other out, so when the images match in the alignment, a certain color (for example, gray) is obtained, and it is easy to determine whether the alignment is successful.
[0035]
[Overlapping position information]
FIG. 7 is an explanatory diagram of a transparent display position (overlapping position information) of an image, (a) is an explanatory diagram of alignment of the through image 60 and the previous image (captured image), and (b) and (c). FIG. 4 is an explanatory diagram of alignment of the nth image, the (n−1) th image, and the first image.
[0036]
As a method for expressing the overlay position information, in the embodiment, the lower left X coordinate of the image that is transparently displayed with the upper left (or lower left) of the through image as the origin is expressed. At this time, the possible value q of the X coordinate is 0 ≦ q ≦ 2x−1 where x is the horizontal size of the image (in the example of a 640 × 480 JPEG image, 0 ≦ q ≦ 1279).
[0037]
In addition, when the through image and the first to n−2th captured images are aligned, the width of the transmissive display portion 71 is set to a constant value q, and the second to n−1th image files are superimposed. In the position column 562-k, the value “q” is stored as overlapping position information. That is, in the example of (a), the right 1/5 region of the immediately preceding image 70-3 (third image) is transparently displayed on the transparent display portion 71 of the through image 60 (a 640 × 480 JPEG image). In the example, q = 127).
[0038]
When the n-th image, the (n-1) th image, and the first image are aligned, an area on the right side of the (n-1) th image 70- (n-1) is initially 5/1. The transparent display portion 71 of the through image 60 that is the nth image candidate is transparently displayed, and the left 1/5 region of the first image 70-1 is transparently displayed on the transparent display portion 71 ′. That is, as shown in (b), in the example of a JPEG image of 640 × 480, the coordinates of the leading ends of the transparent display portions 71 and 71 ′ on the through image 60 are “127” and “512”. The coordinates of the left end of the transmissive display portion 71 ′ are “639” − “512” = “127” on the first image 70-1.
[0039]
Further, when the through image 60 and the first image 70-1 are aligned, if both ends of the image coincide with each other at the position of the width “Q, Q” from the left and right of the through image, the tip of the transparent display portion on the through image is displayed. The coordinates are “Q” and “xQ”, but the coordinates of the transmissive display portion 71 ′ are “Q” on the first image 70-1. That is, as shown in (c), in the 640 × 480 JPEG image, the coordinates of the overlay position of the through image 60 are “135” and “504”, but the left end coordinate of the transparent display portion 71 ′ is the first one. On the image 70-1, “639” − “504” = “135”.
[0040]
[Operation example during panorama shooting]
FIG. 8 is a flowchart showing an operation example of the digital camera 100 at the time of panoramic photographing according to the present invention.
[0041]
When the user operates the processing mode selection switch 37 to select the panorama shooting mode, a status signal is sent from the operation unit 30 to the control unit 20, and the control unit 20 switches the processing mode to the panorama shooting mode and the panorama shooting processing program Start up. As a result, panoramic images can be taken.
[0042]
Step S1: (Image capture)
An image for one frame is captured at predetermined time intervals via the optical system 11, the signal conversion unit 12, and the signal processing unit 13 within the range of the angle of view of the digital camera 100, and stored in the through image buffer 141 of the DRAM 14 (upper Written).
[0043]
Step S2: (Composition of a transparent display portion on a through image)
The control unit 20 cuts out a predetermined width q of the right end of the captured image stored in the image buffer 142 and combines (transparently superimposes) it on the left end portion of the image (through image) stored in the through image buffer 141. It should be noted that since no image is stored in the image storage area 142 until the first first image is captured, the composition to the through image buffer 141 is not performed.
[0044]
Step S3: (Through image display after composition)
The control unit 20 sends an image in which the right region of the immediately preceding image is transparently superimposed on the left side in step S2 to display the through image, and displays a message (for example, “Left end and up / down” Please display the operation guidance message (for example, “OK → shutter fully pressed”, “360 ° → shutter half pressed”) (FIG. 5). Until this time, the composition in step S2 is not performed, so only the message is superimposed on the through image.
[0045]
Step S4: (Determination of imaging / display switching)
The user sees the combined through image and presses the shutter button 36 fully to capture the current angle of view, or presses the shutter button 36 halfway to switch the display for 360 ° panoramic shooting, Since any operation of moving the digital camera to perform the alignment is performed, the control unit 20 checks the status signal from the operation unit 30 and when the user fully presses the shutter button 36, the process proceeds to S5. When the shutter button 36 is half-pressed, the process proceeds to S7, and when alignment is performed, the process proceeds to S15.
[0046]
Step S5: (Imaging)
When the shutter button 36 is fully pressed in step S4, an image within the angle of view is captured. When the shutter 36 is depressed, the control unit 20 sends the image (captured image) stored in the through image buffer 141 to the display unit 40 to display a still image, and stores (overwrites) the captured image in the image buffer 142. . Note that the first captured image may also be stored in the image buffer 143 (see S7).
[0047]
Step S6: (Image file storage record (1 to (n-1) th))
The control unit 20 creates and saves and records an image file to save and record the captured image (in this example, the image stored in the image buffer 142). In this flowchart, it is shown that the process proceeds to S1 after the captured image storage / recording process is performed, but in the embodiment, the captured image storage / recording process and the image capturing process for the next angle of view (S1) are performed. ~ Since the through image display after synthesis (S3) is processed in parallel, after the storage and recording process is completed, the process proceeds to the step being executed in S1 to S4 (recording of the next image after storage and recording of the captured image) Processing may be performed, and after completion, the process may transition to S1).
[0048]
In the save recording process, an image file area is secured on the flash memory 50, and the image stored in the image buffer 142 is subjected to JPEG compression processing and saved and recorded in the image area 57 of the image file 55, and the image file in the header area 56 is also recorded. An image file name is stored in the identification area 561, and an imaging method, predetermined imaging conditions, and the like are stored and recorded in the image information area 562 at corresponding positions. That is, in this embodiment, since the shooting method = panoramic shooting, a code meaning panoramic shooting is stored in the shooting method column 562-1, and the superimposed image file column 562-j has a superimposed image (the immediately preceding image). The file name (“pic003.jpg” in the example of FIG. 7) is stored, and the overlapping position column 562-k has a distance from the left edge of the image (the number of pixels in the horizontal direction (the value “in the example of FIG. 5 is“ q ”,“ 127 ”in the example of FIG. Note that no file name is recorded in the superimposed image file column 562-j of the header area 56 of the first (first) image file, and similarly, no overlapping position information is recorded in the overlapping position column 562-k. (It is recorded after (or before) the storage and recording of the last image (see step S12)).
[0049]
Step S7: (Reproduction of the first (first image) image)
When the shutter button 36 is half-pressed in step S4, the control unit 20 retrieves the image file in which the captured image is first stored in the current panorama shooting from the flash memory 50, reproduces it, and stores it in the image buffer 143 (upper (However, if the first image is also stored in the image buffer 143 in step S5, this step is unnecessary).
[0050]
Step S8: (Composition of transparent display portions on both sides of the through image)
The control unit 20 cuts out a shutter by cutting out a predetermined width q of the right region of the captured image (n−1 image) stored in the image buffer 142 and the left region of the image buffer 143 (first image). When the button 36 is half-pressed, the left side portion and the right side portion of the image stored in the through image buffer 141 are combined (transparent superimposed).
[0051]
Step S9: (Display through image after composition)
The control unit 20 sends the contents of the image buffer 141 synthesized in step S8 to the display unit 40 to display a through image. In addition, a message for prompting the user to align the position (for example, “Please adjust the left and right overlap and top and bottom”) and an operation guidance message (for example, “OK → shutter fully pressed”, “left / right move → +/− key”) It is displayed superimposed on the through image (FIG. 6A).
[0052]
Step S10: (Judgment on presence / absence of imaging operation)
The control unit 20 examines the status signal from the operation unit 30 and transitions to S11 if the user fully presses the shutter button 36, and transitions to S13 otherwise.
[0053]
Step S11: (Recording and saving of the last (nth) image file)
The control unit 20 performs a storage recording process for the image (n-th image) stored in the through image buffer 141. In the storage recording process, an image file area is secured on the flash memory 50, and the image stored in the image buffer 141 is JPEG compressed and stored in the image area 57 of the image file 55, and the image file in the header area 56 is stored. An image file name is stored in the identification area 561, and an imaging method, predetermined imaging conditions, and the like are stored and recorded in the image information area 562 at corresponding positions. That is, a code meaning panoramic shooting is stored in the shooting method column 562-1, and the file name of the transparently displayed image (previous image) is stored in the superimposed image file name column 562-j (example in FIG. 7C). "Pic015.jpg") is stored, and the overlap position column 562-k stores the distance from the left edge of the image (the number of pixels in the horizontal direction (value "Q" in the example of FIG. 6C), FIG. In the example of ")" 135 ") is stored))).
[0054]
Step S12: (First (first) image file header area update processing)
When the storage and recording process in step S11 is completed, the control unit 20 performs an update process on the header area 56 of the image file in which the first (first) image of the panorama shooting stored in the flash memory 50 is recorded, and the header When the update process of the area 56 is finished, the 360 ° panorama shooting process is finished.
[0055]
In the updating process of the header area 56, first, the header area data recorded in the header area 56 of the first panoramic image file stored in the flash memory 50 is read, and a predetermined area in the work area 145 of the DRAM 14 is read. Store in the area. Next, the file name of the last (nth) image (“pic016.jpg” in the example of FIG. 7C) is written in the superimposed image file column 562-j, and one sheet is stored in the superimposed position column 562-k. A value corresponding to the distance from the left end of the eye image (the number of pixels in the horizontal direction), that is, the width of the transmissive display portion 71 ′ (the value “Q” in the example of FIG. 6 (c), In the example, “135”)) is written and updated. Next, the updated header area data can be overwritten on the header area 56 of the first image file of the panorama shooting on the flash memory 50.
[0056]
Step S13: (Transmission display position change operation presence / absence determination)
The control unit 20 examines the status signal from the operation unit 30. If the +/- key 35 is operated, the process proceeds to S14, and the +/- key 35 is pressed even if a predetermined time (2 seconds in the embodiment) elapses. If not operated, the process proceeds to S15. It should be noted that the width of the transparent display portions 71 and 71 ′ is simultaneously expanded by a plus (right) push-down operation (hereinafter, plus operation) of the +/− key 35 and is pushed by a minus (left) direction push-down operation (hereinafter, minus operation). The widths of the transmissive display portions 71 and 71 ′ are simultaneously reduced (FIGS. 6B and 6C).
[0057]
Step S14: (Update of transparent display position)
The control unit 20 further checks the status signal from the operation unit 30, counts the operation time of the +/− key 35, and calculates the movement width based on the count value. Next, when the +/− key 35 is a plus operation, the transmission width Q is updated by adding the movement width α to the transparent display position Q (Q = Q + α: FIG. 6B), and the process returns to S8. In the case of a minus operation, the transmission display position Q is updated by subtracting the movement width β from the transmission display position Q (Q = Q−β: FIG. 6C), and the process returns to S8 (S8 to S10, S13). The operation of S14 is repeated every time the user operates the +/− key 35 in step S13).
[0058]
Step S15: (Positioning operation with previous and first images)
In accordance with the message displayed on the screen, the user performs vertical alignment by slightly moving the camera up and down so that the transmissive display image and the transmissive display portions 71 and 71 ′ of the through image overlap. When the alignment is performed by the user, the process proceeds to S8.
[0059]
Step S16: (Positioning operation with previous image)
In accordance with the message displayed on the screen, the user finely moves the digital camera so that the transparent display image and the image of the transparent display portion 71 of the through image overlap. When positioning by the user is performed, the process proceeds to S1.
[0060]
With the above configuration, when matching the last image in 360 ° panorama shooting with the first image, a partial area of the captured image before and after is transparently displayed on both sides of the through image by display switching, and then the transparent display portion is keyed Since it can be expanded and contracted by operation, it is possible to easily match the both side portions of the through image and the partial regions of the captured images before and after.
[0061]
[Modification 1]
In the example of FIG. 8, the widths of the transmissive display portions 71 and 71 ′ are the same (Q, Q). However, the width of the transmissive display portions 71 and 71 ′ is Qn ≠ Q1, and the transmissive display portion 71 or the transmissive display portion 71 ′. One of the above may be fixed, and in steps S13 and S14, only the width of the other transparent display portion may be added or subtracted by a key operation to be expanded or contracted. Further, the widths of the transparent display portions 71 and 71 ′ may be increased or decreased separately by key operation. The order of steps S11 and S12 may be changed.
[0062]
[Modification 2]
In the operation example shown in FIG. 8, an image file is created for each image based on the captured image and stored in the flash memory 50 (S6, S11, S12), but the DRAM 14 is stored in FIG. The images captured so as to be connected within the range of the angle of view are temporarily stored in the captured image buffers (1) to (N), and the first (first image) and the last (nth image) Then, the captured images stored in the captured image buffers (1) to (N) may be sequentially stored and recorded in the flash memory 50 (FIG. 9 (a), (N)). b)).
[0063]
That is, when the user determines that the connection is good in step S3 in FIG. 8 and fully presses the shutter button 36 in step S4, an image within the angle of view is captured. At this time, as shown below in FIG. 9A, the captured images are temporarily stored in the captured image buffers (1) to (N).
[0064]
Step S5 ′: (Temporary storage of captured image)
When the shutter button 36 is pressed, the image stored in the through image buffer 141 is sent to the display unit 40 to display a still image, and the image is stored in the captured image buffer ′ (i) in the captured image storage area 142 ′. . Here, i is an image counter value and takes a value of 1 to n.
[0065]
Step S6 ': (Temporary storage of transparent display position)
When the image counter value i is 2 or more, the control unit 20 stores the overlapping position “q” when the shutter button 36 is fully pressed in the position information field (i) of the overlapping position field 144, and returns to S1. .
[0066]
Next, in step S4 in FIG. 8, the shutter button 36 is half-pressed, and in steps S7 to S9, the first (first sheet) and last (n-th sheet) images are joined, and then in step S10, the shutter is pressed. When the button 36 is fully pressed, an image within the angle of view is captured. At this time, the captured images stored in the image buffers (1) to (N) are sequentially stored and recorded in the flash memory 50 as shown below in FIG.
[0067]
Step S11 ': (Storage of last (nth) image in image buffer))
The control unit 20 primarily stores the image (n-th image) stored in the through image buffer 141 in the captured image buffer (n) of the captured image storage area 142 ′ of the DRAM 14 and also the image in the header area 56. An image file name is stored in the file identification area 562, and a shooting method, predetermined shooting conditions, and the like are stored and recorded in a corresponding position in the image information area 562. That is, a code meaning panoramic shooting is stored in the shooting method column 562-1, and the file name (image name + serial number n (FIG. 7) in the superimposed image file name column 562-j is the last image transparently displayed). In the example of (c), since n = 16, “pic016.jpg”) is stored, and the position information column (n) of the superimposed position information storage area 144 stores the distance from the left end of the image (FIG. 6C ) Is stored as the value “Q” and in the example of FIG. 7C as “135”).
[0068]
Step S12 ′: (Storing position information of the first (first) image)
When the storage and recording process in step S11 is completed, the control unit 20 superimposes the first (first) image of this panorama shooting stored in the captured image buffer (1) of the captured image storage area 142 ′ of the DRAM. The alignment position information is stored in the position information column (1) of the overlapping position information storage area 144. That is, the value corresponding to the distance from the left end of the first image (the number of pixels in the horizontal direction), that is, the width of the transmissive display portion 71 ′ (the value “Q” in the example of FIG. 6C), FIG. In the example of (c), “135”)) is written in the position information column (1). Next, after setting the repetition counter i = 1, the process proceeds to S13 ′.
[0069]
Step S13 ': (Image information storage record)
The control unit 20 stores the image file name in the image file identification area 561 of the header area 56 of the i-th image file area of the flash memory 50, and the image information area 562 corresponds to a shooting method, predetermined shooting conditions, and the like. Save and record in position. That is, a code meaning panoramic shooting is stored in the shooting method column 561, and the file name (image identification name + number (i)) of the transparently displayed image (previous image) is stored in the superimposed image file column 562-j. The contents of the overlapping position information column (i) of the overlapping position information storage area 144 of the DRAM 14 are stored in the overlapping position column 562-k.
[0070]
Step S14 ': (Save and record of image i)
The control unit 20 performs JPEG compression processing on the image stored in the captured image buffer (i) of the captured image storage area 142 ′ of the DRAM 14 and stores it in the image area 57 of the i-th image file area of the flash memory 50. Record.
[0071]
Step S15 ': (Repeated recording)
The controller 20 ends the panoramic shooting after repeating the operations S13 ′ and S14 ′ i times.
[0072]
[Modification 3]
In the same manner as in the second modification, images captured so as to be connected within the range of the angle of view are temporarily stored in the captured image buffers (1) to (N), and the first (first image) and the last (nth image) are stored. ) And the (n−1) -th image are combined, and when the n-th captured image is photographed, the images are combined and obtained to obtain a panoramic composite image, which is stored in the flash memory 50 as one image file. You may make it do (FIG. 9 (a), (b ')).
[0073]
That is, when the user determines that the connection is good in step S3 in FIG. 8 and the shutter button 36 is fully pressed in step S4, an image within the angle of view is captured. At this time, the images captured as described above in steps S5 ′ and S6 ′ of FIG. 9A are temporarily stored in the captured image buffers (1) to (N).
[0074]
Next, after the shutter button 36 is half-pressed in step S4 in FIG. 8, the first (first sheet), the last (nth sheet), and the (n-1) th sheet are joined in steps S7 to S9. After this, when the shutter button 36 is fully pressed in step S10, an image within the angle of view is captured. At this time, a panoramic composite image is obtained as shown below in FIG. 9B ′, and is stored and recorded in the flash memory 50 as one image file.
[0075]
Step S11 ": (Panorama composite image generation)
The control unit 20 displays the captured images stored in the captured image buffers (1) to (N) of the captured image storage area 142 ′ of the DRAM 14 in the position information fields (2) to (N) of the positional information storage area 144. The image data is stored in the captured storage area 142 ′ in a right-aligned manner based on the values of the images, and a composite image (= panoramic composite image) of 1 to n captured images is generated.
[0076]
Step S12 ": (Panorama composite image saving and recording process)
The control unit 20 performs JPEG compression processing on the panorama composite image synthesized in step S11 ″. In addition, the panorama compressed by securing an image file region including an image region 57 having the same size as the size of the compressed image data. The composite image is saved and recorded in the image area 57, the image file name is stored in the image file identification area 561 of the header area 56, and the image information area 562-1 has a position corresponding to a shooting method, predetermined shooting conditions, and the like. That is, a code meaning panoramic shooting is stored in the shooting method column 561, and the left side area of the first captured image transparently displayed on the n-th captured image is stored in the overlapping position column 562-k. It is not recorded in the superimposed image file field 562-j.
[0077]
[Operation example when compositing panoramic image]
FIG. 10 is a flowchart showing an example of the combined display operation of the panorama shot image in the playback mode. This is an example in which the images of a plurality of image files recorded in the flash memory 50 are played back and combined to obtain a panorama combined image. .
[0078]
Step T0: (Initial setting of counter)
The value i of the image counter used when storing the reproduced image in the reproduced image buffer (= captured image buffer) (1) to (N) of the DRAM 14 is set to zero.
[0079]
Step T1: (Search for panoramic images)
The control unit 20 examines the image file stored in the flash memory 50, and searches for a panoramic image. In this case, the shooting method field 562-1 of the header area 56 is checked to determine whether or not the image is a panoramic image. If the image is a panoramic image, the process proceeds to T2.
[0080]
Step T2: (Search key setting, etc.)
The image file searched in step T1 is an image file storing the first image at the time of panorama shooting. In this embodiment, the image file immediately before the superimposed image file field 562-j in the header area 56 of the image file is stored. Since the image file name storing the image is stored, when successive images are searched using the superimposed image name stored in the superimposed image file field 562-j as a key, the logically previous image is sequentially displayed. Search (reverse search). Therefore, the last (n-th) image at the time of panoramic shooting is stored in the reproduced image buffer (N), and sequentially stored in the reproduced image buffers (N-1), (N-2),. As described above, the control unit 20 extracts the last image file name stored in the superimposed image file field 562-j of the image file storing the first image searched in step T1. The first image file name is held (stored) in the RAM of the control unit 20.
[0081]
Step T3: (Image search)
The control unit 20 searches for an image file stored in the flash memory 50 using the extracted image file name as a key, and if a hit is found, the process proceeds to T4.
[0082]
Step T4: (Reproduction of searched image)
The control unit 20 takes out the compressed image stored in the image area 57 of the image file hit in step T3, performs decompression processing, and reproduces it.
[0083]
Step T5: (Storage of reproduced image etc.)
Next, the control unit 20 stores the reproduced image in the reproduced image buffer (N-i), extracts superposition position information from the superposition display position field 562-k of the header area 56 of the image file, and reads the DRAM 14. Is stored in the position information column (N-i) of the position information storage area 144.
[0084]
Step T6: (End determination of panoramic image)
The control unit 20 stores the image file name (file name of the first image file) held in the RAM and the superimposed image file field 562-j in the header area 56 of the image file hit in step T3. The image file names are compared, and if they match, it is determined that all images obtained by 360 ° panorama shooting have been reproduced (i = n), and the process proceeds to T8.
[0085]
Step T7: (Count up etc.)
If the image file names do not match in step T6, 1 is added to the counter value i (i = i + 1) and the image file hit in step T3 is stored in the superimposed image file field 562-j. The file name is extracted and the process proceeds to T3 as a search key.
[0086]
Step T8: (Composition of reproduced images)
The control unit 20 displays the reproduced images stored in the reproduced image image buffers (1) to (2),... (N-1), (N) of the DRAM 14 in the position information column (2),. Based on the values of (N−1) and (N), the images are stored in the reconstructed image storage area 142 ′ in the left-justified manner, and synthesized images of 1 to n reproduced images (= panoramic composition) Image).
[0087]
Step T9: (Holding position information of the head and tail of the image)
The control unit 20 extracts the position where the head and tail of the panorama composite image overlap (= position of the overlapping portion of the first captured image and the nth captured image) from the position information column (1), and Stored in RAM.
[0088]
Step T10: (Display panorama composite image)
Next, the control unit 20 sends a portion of the size of one frame of the composite image (that is, the composite panorama image) stored in the reproduced image storage area 142 ′ of the DRAM 14 to the display unit 40 for display on the screen.
[0089]
Step T11: (Determination of scroll operation result of panorama composite image)
The user can appropriately view the panoramic view by scrolling the panoramic image left and right by operating the +/- key 35 as appropriate. In this case, the control unit 20 checks the status signal from the operation unit 30 to detect the scroll direction and the scroll width, and the display range (the portion having the size of one frame) includes the front end and the tail of the panoramic image. Transitions to T13.
[0090]
Step T12: (Scroll display (1))
If the display range by the scroll operation does not include the head and tail of the panoramic image, the control unit 20 cuts out the image for one frame of that portion, sends it to the display unit 40 for display, and returns to step T11.
[0091]
Step T13:
When the display range by the scroll operation includes the head and tail of the panoramic image, the control unit 20 positions the head and tail overlapped in the RAM (= the first captured image and the nth captured image). Based on the position of the overlapped part), the head and tail of the panoramic image are pasted together, and an image for one frame including the pasted part is cut out and sent to the display unit 40 for display, and the process returns to T11.
In the case of scrolling, auto scroll display, that is, scroll display may be automatically performed when the +/- key 35 is pressed once.
[0092]
According to the above configuration, since the composite image is sent directly from the DRAM 14 to the display unit 40, it does not take a long time for expansion processing as in the case where the image stored and recorded in the flash memory 50 is combined and reproduced. In addition, since the control unit 20 holds the overlapping position information that connects the head and the tail without any deviation, the head and the tail can be easily aligned, and a 360 ° panorama can be viewed without any discomfort even if scrolling is repeated during playback. it can.
[0093]
[Panorama composite image display operation]
When a panorama composite image is reproduced and displayed, when a panorama composite image file is retrieved in the reproduction mode, the control unit 20 takes out the compressed image stored in the image area 57, performs an expansion process, and reproduces it. The data is stored in the reproduced image storage area 142 ′ of the DRAM 14, the superposition position information is extracted from the superposition position column 562-k of the header area 56 of the image file, and held (stored) in the RAM of the control unit 20. What is necessary is just to be the same as the operation | movement of step T8-T11 of FIG.
[0094]
Although several embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and it goes without saying that various modifications can be made.
[0095]
【The invention's effect】
As described above, according to the first and third aspects of the present invention, when a wide-field subject image is divided and imaged within a range of the angle of view to obtain a panoramic image with a wide field of view, one end of the subject image is obtained. Display the image captured immediately before and the first image captured at the other end. In the state Since the width of the superimposed display can be adjusted, it is possible to obtain a panoramic image with no deviation from the first captured image even when the number of divisions is increased.
[0096]
According to the second aspect of the present invention, since the superimposed display is performed by transparently displaying the end region of the adjacent captured image at the end of the subject image, the alignment is very easy. obtain.
[0097]
According to the fourth aspect of the present invention, since the superimposed display with the first captured image is performed when instructed by the start instructing means, it is not necessary to perform the comparison with the first captured image. It is possible to prevent the display from being superimposed and hindering visual recognition.
[0098]
According to the fifth aspect of the present invention, the image superimposed on the display means can be stored as a single image file, and can be easily combined when panoramic images are combined.
[0099]
Further, according to the sixth aspect of the present invention, since the superimposition area information is stored, the superposition area information can be combined very easily when the panoramic image is synthesized.
[0100]
According to the seventh aspect of the present invention, an image obtained by imaging a plurality of times and the superimposed region information are stored, and the panoramic image is obtained by connecting and synthesizing the images based on the superimposed region information. Therefore, an extremely accurate panoramic image can be obtained.
[0101]
Further, according to the invention of claim 8, since the panoramic image is synthesized and stored, the stored panoramic image can be read and displayed immediately when the panoramic image is desired to be viewed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a circuit configuration example of a digital camera as an embodiment of an imaging apparatus to which a panoramic shooting method of the present invention can be applied.
FIG. 2 is an explanatory diagram showing an example of a layout of a working memory (DRAM) during panoramic shooting.
FIG. 3 is a diagram illustrating an example of an image file.
FIG. 4 is an explanatory diagram illustrating a relationship between images during panoramic shooting.
FIG. 5 is an explanatory diagram of alignment with an image immediately before panorama shooting.
FIG. 6 is an explanatory diagram of stitching of an nth image and a first image.
FIG. 7 is an explanatory diagram of a transparent display position of an image.
FIG. 8 is a flowchart illustrating an operation example of the digital camera during panoramic shooting.
FIG. 9 is a flowchart illustrating an operation example of the digital camera during panoramic shooting.
FIG. 10 is a flowchart showing an example of a panorama shot image display operation under a playback mode.
FIG. 11 is an explanatory diagram of panoramic shooting.
[Explanation of symbols]
14 DRAM (memory means)
20 control unit (first and second image synthesizing means, display control means, superimposed area information detecting means, panoramic image synthesizing means)
35 Plus / minus key (second key, overlap width change instruction means)
36 Shutter button (first key, positioning start instruction means))
50 Flash memory (storage and recording means)
60 Through image (subject image)
71, 71 'Transparent display part (predetermined area at the end)
100 Digital camera (imaging device)
q, Q Overlap width (end area width, area width)

Claims (8)

A panoramic imaging method for obtaining a panoramic image of a subject image with a wide field of view by dividing and capturing a subject image with a wide field of view within a range of angle of view and joining a plurality of images obtained by the imaging,
When taking the last image of the plurality of images taken to obtain the panoramic image, immediately before one end of the subject image displayed on the display unit during the shooting standby state of the last image. The edge region of the captured image is superimposed and displayed, and the edge region of the first captured image is superimposed and displayed on the other end of the subject image, and the edge region of the image captured immediately before is one end of the subject image. And the end region of the first captured image is superimposed and displayed on the other end of the subject image, and the end region of the one end superimposed and the other end of the image are superimposed and displayed. Adjusting the width of at least one of the end regions based on a key operation, aligning the last image with the image captured immediately before, and aligning the last image with the image captured first Features that made it possible Panoramic imaging method to.   The panoramic imaging method according to claim 1, wherein the superimposed display is performed by transparently displaying an end area of an adjacent captured image at an end of a subject image. In an imaging apparatus comprising an imaging device on which a subject image is formed and dividing a wide-field subject image within a range of angle of view and capturing a plurality of times to obtain a panoramic image of the wide-field subject image,
Display means for displaying a subject image formed on the image sensor during standby for image capture;
An end region of an image captured immediately before the plurality of times of imaging at one end of the subject image displayed when the subject image formed on the imaging element is displayed on the display means First superimposed display means for superimposing and displaying;
Second superimposed display means for superimposing and displaying an end region of an image first captured among the plurality of times of imaging on the other end of the displayed subject image;
The edge region of the image captured immediately before by the first superimposed display means is displayed superimposed on one end of the subject image, and the edge region of the image first captured by the second superimposed display means is An operation key means for changing a width of at least one of the end region of the one end superimposed and the end region of the other end superimposed on the other end of the subject image;
An imaging apparatus comprising: Furthermore, it comprises start instruction means for instructing start of superimposed display by the second superimposed display means,
When an instruction by the start instructing unit is given while the end region of the image captured immediately before is superimposed on one end of the subject by the first superimposed display unit, the second superimposed display unit The imaging apparatus according to claim 3, wherein superimposed display is performed.   4. The imaging apparatus according to claim 3, further comprising storage recording means for storing and recording an image superimposed on the display means as one image file. Furthermore, a superimposition area information detection unit that detects the area width superimposed on the display unit and sets the superimposition area information;
Superimposition area information storage means for storing the superposition area information detected by the superposition area information detection means;
The imaging apparatus according to claim 3, further comprising: Furthermore, image storage means for storing an image obtained by the plurality of times of imaging ,
Panoramic image combining means for connecting and combining the images stored by the image storage means based on the overlapping area information stored in the overlapping area information storage means to obtain the panoramic image;
The imaging apparatus according to claim 6 , further comprising:   The imaging apparatus according to claim 7, further comprising a panoramic image storage unit that stores the panoramic image obtained by the panoramic image synthesis unit.

Images