JP2005328497A - Image pickup apparatus and image pickup method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate panoramic whole image of high quality at a low cost which captures movement of a moving object with high precision by shortening image pickup interval. <P>SOLUTION: A unit image which constitutes a part of an image pickup range is imaged by sequentially shifting image pickup direction, image regions of a predetermined size which constitute a part of the unit image which was imaged in an image pickup step are cut out so as to produce mutually overlapping regions, the cut-out image regions are sequentially overlapped, and the whole image showing the whole image pickup range is formed. At this time, the image regions are cut out so that at least two image regions overlap each other in average for the position of each image of the whole image to be generated. As a result, noise component of each of the image regions is averaged, and image quality is improved. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus and an imaging method that generate a panoramic overall image in which a plurality of image areas are pasted by sequentially changing imaging directions.

  Conventionally, an electronic still camera that has been widely used converts light that has passed through a lens by imaging a subject into an image signal by a solid-state imaging device such as a CCD, and records this on a recording medium. The signal can be reproduced. Many electronic still cameras include a monitor that can display a captured still image, and a specific one of the still images recorded so far can be selected and displayed. In this electronic still camera, the shooting range in one shooting is a narrow range limited by the angle of view of the lens, and a wide range of shooting beyond the angle of view of the lens cannot be performed.

  For this reason, camera systems that capture panoramic images have been proposed in recent years. This camera system is classified into a multi-lens system that captures a wide field of view at a time with a plurality of lenses, and a monocular system that continuously captures individual unit images while sequentially shifting the capturing direction using a monocular lens.

  The multi-lens camera system has an advantage that a panoramic overall image can be generated by almost the same operation as a normal camera, but has a disadvantage that the entire system becomes expensive.

  On the other hand, a monocular camera system (for example, refer to Patent Document 1) is relatively inexpensive, but generates a whole image by joining individual unit images taken continuously so that the joints are not conspicuous. The image processing technology to do is essential. In other words, since the individual fields of view are different from each other due to the fact that the shooting direction is sequentially shifted, the amount of positional deviation between adjacent unit images is estimated using a technique such as pattern matching. This is because it is necessary to correct. In addition, in order to obtain panoramic whole images from overlapping unit images, a method for selecting an optimal unit image based on the obtained boundary line by using the Voronoi division method related to the distance from the center of each unit image is proposed. Has been. Further, a method of smoothly connecting adjacent unit images by using an average of pixel values in mutually overlapping image regions, or a method of selecting an optimal unit image by using a median filter has been proposed.

  By the way, this monocular camera system has a discontinuous portion such as a shift when a moving subject is projected near the boundary of unit images acquired as a result of continuous shooting, and as a result, the entire generated finally There is also a problem that the image is broken.

  In order to alleviate this problem, it is only necessary to shoot as many pictures as possible in as short a time as possible. In order to capture the entire panoramic imaging target in a short time, it is necessary to increase the horizontal or vertical turning speed of the camera module having a monocular lens and an image sensor. However, there is a problem that blurring occurs during shooting as the turning speed is increased. In order to suppress this blur, it is necessary to synchronize with the shooting operation and to stop the camera module with high accuracy or to execute control using the camera shake correction mechanism. However, the cost of the entire system becomes high. The problem also arises.

  For example, the panoramic electronic still camera proposed in Patent Document 2 includes means for rotating the camera in conjunction with an operation of reading out an image signal from an image sensor, but in order to shorten the shooting interval, Hardware that repeats turning and stopping at high speed and accuracy is required, and the cost of the entire system cannot be held down.

  In addition, in order to capture the entire panoramic shooting target in a short time, it is inevitably necessary to increase the shutter speed. However, in the conventional panoramic electronic still camera, if the shutter speed is increased, the entire obtained image is obtained. Since the image becomes dark and the noise increases, a high-quality image cannot be generated.

  Further, for the purpose of photographing a panoramic image, there has been conventionally proposed a photographing apparatus that divides a wide photographing range into a plurality of unit images using a monocular lens. However, with this conventional photographing apparatus, it has been impossible to realize a high-quality panoramic photographing function at a low cost while maintaining the merchantability of a consumer digital camera.

  For example, FIG. 2 of Patent Document 2 discloses a camera having a function of capturing a panoramic image and a rotation adapter for attaching an imaging unit of the camera. Although this rotation adapter plays a role as a drive source for rotating the imaging unit, there are some problems when these are actually used.

  The first problem is that the mass balance in the conventional camera is not considered much. In the conventional camera, the adapter is provided with a function of rotating the image pickup unit, and the camera body itself is rotated in order to mount the adapter. Ideally, only some parts such as lenses and image sensors need to be rotated. However, with conventional cameras, the camera itself with heavy components such as a battery must be rotated. In addition, the cost of various parts that support the motor and the rotation operation increases. In addition, when shooting with the camera body supported by hand, the gripping portion is light and the rotating portion is heavy, so the mass balance is poor and difficult to hold.

  The second problem is that it is difficult to operate buttons on the conventional camera and to visually recognize the display screen. That is, various buttons including a shutter button and a display unit are provided on the rotating camera body side. If the button operation is performed while the imaging unit is rotating in such a camera, the resulting image may be blurred, and photographing itself may fail. Normally, it is possible to avoid blur using a release, but it is not common to use a release with a consumer digital camera.

  In addition, since the display unit also rotates in accordance with the rotation operation of the imaging unit, the user has to check the situation while constantly following the display unit, which is difficult to handle.

  In addition, it is cumbersome to carry a rotating adapter that is used only during panoramic shooting and is used infrequently, and if the rotating adapter is always attached to the camera body, the entire camera system cannot be made compact. The portability will be poor.

  For this reason, the rotation adapter used at the time of panoramic imaging has hardly been used as a result.

  As another conventional example, for example, as shown in Patent Document 3, a device is disclosed that includes a turntable on a fixed base and performs shooting while rotating 360 ° to create a panoramic image. This device is specialized for panoramic photography. This device uses a photocoupler to connect the camera and the processing unit in the subsequent stage to continuously take a 360 ° field of view, and uses a rotation angle sensor to measure the rotation angle. It is a large-scale configuration.

  However, if such a configuration is provided to add a panorama shooting function to a general consumer camera, the cost increases. In addition, since these conventional apparatuses are installed on a tripod and connected to an external computer for use, the portability generally required for consumer cameras is poor. In particular, it is not practical to have a large-scale configuration like the conventional apparatus for a camera whose main purpose is to take a normal photograph.

  Furthermore, when a panoramic image is obtained by continuously shooting while rotating or moving to obtain a plurality of shot images and connecting the shot images, the resulting panoramic image has the same direction as the rotation or movement of the camera. There was a problem that direction blur occurred.

Japanese Patent Laid-Open No. 11-46317 JP-A-6-225202 Japanese Patent No. 3348285

  Therefore, the present invention has been devised to solve the above-described problems, and the object of the present invention is to capture a moving subject as a unit image by shooting as many images as possible in as short a time as possible. It is an object of the present invention to provide an imaging apparatus and an imaging method that can reduce unnaturalness such as a shift that occurs in the joint portion of the two and generate a high-quality panoramic overall image at low cost.

  The present invention has been devised in order to solve the above-described problems, and an object of the present invention is to reduce the cost of a general consumer digital camera having a high-quality panoramic shooting function. An imaging apparatus and an imaging method that can be realized with the above.

  Other objects of the present invention and specific advantages obtained by the present invention will become more apparent from the description of embodiments described below.

  In order to solve the above-described problem, the present invention captures a unit image that forms a part of a shooting range by sequentially changing the shooting direction, and sets an image area of a predetermined size that forms a part of the captured unit image. The whole image representing the entire photographing range is generated by cutting out such that overlapping areas are generated and sequentially superimposing the cut out image areas.

  That is, an imaging apparatus to which the present invention is applied has an imaging unit that sequentially changes the shooting direction to capture a unit image that forms a part of the shooting range, and a predetermined part that forms a part of the unit image captured by the imaging unit. Image cutting means for cutting image areas of a size so that overlapping areas are generated, and whole image generating means for generating a whole image representing the entire shooting range by sequentially superimposing the image areas cut by the image cutting means With.

  In addition, an imaging apparatus to which the present invention is applied configures an imaging unit that captures a unit image that forms a part of the imaging range by sequentially changing the imaging direction, and a part of the unit image that is captured by the imaging unit. Image cutting means for cutting out image areas of a predetermined size so as to generate overlapping areas, and a first rotation means that can rotate so that the shooting direction of the imaging means changes sequentially with the connected rotation axis as the rotation center. The entire image generated by the entire image generating means is displayed, and the entire image generated by the entire image generating means is displayed. The display means, the drive means for rotating the rotating shaft, the designation means for designating each timing of the imaging start and imaging end in the imaging means, and each component is moved. And a second casing that is connected to the first casing via a rotating shaft and can be gripped with one hand of the user, and the second casing includes an entire image generation unit and a display unit. Any one or more of driving means, specifying means, and a battery are provided.

  In addition, an imaging method to which the present invention is applied includes an imaging step of capturing a unit image that forms a part of the imaging range by sequentially changing the imaging direction, and a predetermined size that configures a part of the unit image captured in the imaging step An image cutout step for cutting out the image areas so that overlapping areas are generated, and an overall image generation step for generating an overall image representing the entire shooting range by sequentially superimposing the image areas cut out in the image cutout step .

  In addition to the first housing that can be rotated so that the shooting direction of the image pickup unit sequentially changes, the present invention provides a whole image generation unit that generates a whole image that represents the entire shooting range by sequentially pasting unit images. Display means for displaying the generated entire image, drive means for rotating the first housing, designation means for designating each timing of imaging start and end of imaging in the imaging means, and battery for operating each component Are comprised by the 2nd housing | casing which can be hold | gripped with the user's one hand in which any one or more is provided.

  That is, an imaging apparatus to which the present invention is applied has at least imaging means for imaging unit images that constitute a part of the imaging range by sequentially changing the imaging direction, and the imaging means is configured with the connected rotation axis as the rotation center. A first case that can be rotated so that the shooting direction changes sequentially, a whole image generation unit that generates a whole image that represents the entire shooting range by sequentially pasting unit images captured by the imaging unit, and a whole image Display means for displaying the entire image generated by the generating means, driving means for rotating the rotation axis, designation means for designating each timing of imaging start and imaging end in the imaging means, and each component A battery for operation, and a second casing that is connected to the first casing via a rotating shaft and can be gripped by one hand of the user. Any one or more are provided among the display means and the driving means and specifying means and the battery and.

  That is, in the present invention, unit images constituting a part of the photographing range are captured by sequentially changing the photographing direction, and image areas of a predetermined size constituting a part of the unit image picked up in the imaging step are overlapped with each other. The entire image representing the entire photographing range is generated by cutting out the image areas so as to occur and sequentially superimposing the cut-out image areas.

  Thereby, in the present invention, the noise components of the respective image regions can be averaged, and the image quality of the panoramic overall image to be generated can be improved. In addition, even when a moving subject is projected near the boundary of unit images acquired as a result of continuous shooting, it is possible to capture a panoramic image in which discontinuous portions such as a shift are not noticeable.

  Further, in the present invention, in addition to the first casing that can be rotated so that the shooting direction of the image pickup unit changes sequentially, the whole image generation unit that generates the whole image representing the entire shooting range by sequentially pasting unit images. Display means for displaying the generated entire image, drive means for rotating the first housing, designation means for designating each timing of imaging start and end of imaging in the imaging means, and battery for operating each component Are comprised by the 2nd housing | casing which can be hold | gripped with one hand of the user in which any one or more is provided.

  This makes it possible to mount the minimum necessary parts for shooting inside the first housing and reduce the mass of the rotating part. Therefore, the cost of the components that support the first housing and the motor that drives the rotation of the first housing is reduced. Can be lowered.

  Furthermore, according to the present invention, blurring can be made inconspicuous by applying anisotropic filtering to the image of the clipped image region or the generated entire image.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  The present invention is applied to, for example, an imaging apparatus 1 as shown in FIG. The imaging apparatus 1 includes an imaging unit 10 that images a subject. The imaging unit 10 shields a lens 10a for forming image light from the subject and image light incident through the lens 10a. A diaphragm drive unit 10b that adjusts the diaphragm amount by a shutter blade (not shown) and the like, and a CMOS (Complementary Metal-Oxide Semiconductor) image sensor 11 that generates an electrical imaging signal C1 based on an input subject image. .

  The imaging apparatus 1 also uses a CDS (Correlated Double Sampling) circuit 12 for compensating for variations in the imaging signal C1 generated by the CMOS image sensor 11, and an imaging signal C2 supplied from the CDS circuit 12 in analog / digital. An A / D conversion unit 13 that performs conversion processing, and a digital signal processor that temporarily stores image data as a digitized imaging signal C2 supplied from the A / D conversion unit 13 and performs predetermined processing on the image data ( DSP 15, a codec processing unit 16 that encodes image data from the connected DSP 15, and a memory 17 that stores image data supplied from the codec processing unit 16.

  The imaging apparatus 1 includes a D / A conversion unit 18 that performs digital / analog conversion processing on image data supplied from the DSP 15, and a video encoder that converts image data from the D / A conversion unit 18 into a video signal. Unit 19 and a video encoder unit 19, and includes a monitor unit 20 for displaying an image based on the video signal to the user. The imaging apparatus 1 is connected via an internal bus 14 connected thereto. A central processing unit (CPU) 21 for controlling the entire imaging apparatus 1, an operation unit 22 connected to the internal bus 14 for a user to perform various operations, and a transmission from the CPU 21 via the internal bus 14. A timing generator 23 for controlling a signal processing system from the CMOS image sensor 11 to the DSP 15 based on a control signal generated; Which are connected respectively to the scan 14 and a motor 24 and light meter 26.

  The imaging unit 10 executes an automatic aperture control operation, an automatic focus control operation, and the like based on the operation signal supplied from the CPU 21. The imaging unit 10 adjusts the shooting direction in the horizontal and vertical directions based on the operation signal, and opens and closes shutter blades (not shown) according to the aperture value input via the operation unit 22. Adjust the iris amount with.

  The CMOS image sensor 11 generates an imaging signal C1 obtained by converting a subject image incident through the lens unit 10a and the aperture driving unit 10b into an electrical signal, and outputs this to the CDS circuit 12. Note that the CMOS image sensor 11 can select a part of a subject image formed on the imaging surface and efficiently read out only the pixel value of the region.

  The CDS circuit 12 removes the noise of the imaging signal C1 supplied from the CMOS image sensor 11 using a correlated double sampling circuit, or performs a process for amplifying the gain, and uses this as an imaging signal C2 for A / D. The data is output to the conversion unit 13. The A / D conversion unit 13 performs an analog / digital conversion process on the imaging signal C2 supplied from the CDS circuit 12, and outputs this to the DSP 15. Incidentally, the respective operation timings in the CDS circuit 12 and the A / D converter 13 are controlled by the timing generator 23 so as to continue image capture at a constant frame rate.

  The DSP 15 is a block having a signal processing processor (not shown) and an image RAM. The image represented by the imaging signal C2 from the A / D conversion unit 13 is supplied as stream data configured at a constant frame rate under the control of the timing generator 23, and is temporarily stored in the image RAM. The A signal processor (not shown) is set to perform pre-programmed image processing on the image stored in the image RAM. The image processed in the image RAM (not shown) is transmitted to either or both of the codec processing unit 16 and the D / A conversion unit 18.

  The codec processing unit 16 compresses the data amount of the image transmitted from the DSP 15 by a predetermined method. Here, compression coding may be performed based on a standard such as JPEG (Joint Photographic Experts Group).

  The memory 17 is composed of, for example, a semiconductor memory, a magnetic recording medium, a magneto-optical recording medium, or the like, and is a medium for recording the image data compressed by the codec processing unit 16 at a predetermined address. By configuring the memory 17 as a detachable recording medium, the captured image can be transferred to another PC or the like and viewed, or various searches and processing can be executed.

  The monitor unit 20 displays an image converted into an analog signal by the D / A conversion unit 18 and further converted into a video signal by the video encoder unit 19. The monitor unit 20 may be configured with a liquid crystal display element or the like provided on the side surface of the casing of the imaging apparatus 1 so that the user can check the content of the shooting in real time while executing the imaging process.

  The CPU 21 is connected to a ROM for storing a control program to be executed, a DRAM as a work area used for data accumulation and development, and the like as a central processing unit for controlling the entire image pickup apparatus 1 via the internal bus 14. Play the role of The CPU 21 generates an operation signal based on the operation signal D <b> 1 supplied from the operation unit 22 and the information related to the brightness transmitted from the exposure meter 26, and transmits the operation signal to the imaging unit 10 via the internal bus 14.

  The operation unit 22 includes a key for the user to freely adjust the shooting angle of view and the shooting direction, or to freely adjust the aperture value and the exposure time in the imaging unit 10. The operation unit 22 generates an operation signal D1 corresponding to information input by the user, and transmits the operation signal D1 to the CPU 21 via the internal bus 14. Further, the operation unit 22 includes a shutter button 221, and generates an operation signal D 1 for starting or stopping imaging based on a pressing input operation of the shutter button 221 by the user. To the CPU 21.

  The motor 24 is configured as a drive source for turning the imaging unit 10 such as a stepping motor. The motor 24 rotates in response to an operation signal from the CPU 21. Thereby, the imaging direction of the imaging unit 10 can be changed to the horizontal direction or the vertical direction.

  The exposure meter 26 is a sensor for identifying the brightness of the object imaged by the imaging unit 10, and transmits information regarding the identified brightness to the CPU 21.

  A subject image picked up by the image pickup apparatus 1 having the above-described configuration is converted into an electric signal by the CMOS image sensor 11 to become an image pickup signal C1, noise is removed by the CDS circuit 12 to become an image pickup signal C2, and further A Analog / digital conversion processing is performed in the / D conversion unit 13. Further, the image indicated by the imaging signal C2 is stored in an image RAM (not shown) in the DSP 15, subjected to predetermined image processing, and then subjected to digital / analog conversion processing by the D / A conversion unit 18 to be monitored 20. Or is encoded by the codec processing unit 16 and recorded in the memory 17.

  Next, a specific photographing method in the imaging apparatus 1 to which the present invention is applied will be described.

  In this imaging apparatus 1, by continuously shifting the shooting direction in the horizontal direction, a large number of slit-like images are shot at short shooting intervals as shown in FIG. Then, the captured slit-shaped images are joined while being superimposed on each other, thereby synthesizing a panoramic overall image representing the entire imaging range as shown in FIG. In other words, by joining narrow slit images, it is possible to reduce the occurrence of image discontinuity caused by parallax, lens distortion, and the like. Further, by taking these narrow slit images in a short time, it is possible to reduce the occurrence of image discontinuity due to a moving subject.

  FIG. 3 shows a method for cutting out the slit-shaped image area. A subject image incident through the lens unit 10 a and the aperture driving unit 10 b is formed on the imaging surface of the CMOS image sensor 11. Incidentally, the imaging apparatus 1 reads out the pixel values of all the areas of the formed subject image as shown in FIG. 3A and transfers them to the DSP 15 as shown in FIG. 3B. A part of the slit-like area is selected, and only the pixel value of the area is efficiently read out and transferred to the DSP 15 to cut out the image area by two methods B. In the method B, the area of the image area to be read is smaller than that in the method A, and the transfer amount itself can be reduced.

  FIG. 4 shows the relationship between the readout time of the pixel value with respect to the imaging time in these methods A and B. That is, it is indicated that the number of times of photographing per unit time is increased because the readout time can be shortened as the transfer amount is small. That is, when suppressing discontinuous portions when shooting a moving subject, this can be realized by adopting the method B with a short shooting interval. By the way, either method A or B is adopted, the shooting range is the same, and the total shooting area is the same. That is, since the image data size is the same, the total transfer amount is the same.

  In particular, in the method B, the transfer amount can be suppressed as compared with the method A. For this reason, when the next unit image is picked up when the image area is cut out, the shooting time interval can be significantly shortened. As a result, it is possible to minimize the displacement of the position due to the difference in the shooting time of the moving subject across the joint portion of the continuous slit. In addition, since the total shooting area is the same in both method A and method B, the image transfer amount itself is equal, and the cost of the entire hardware including the imaging device 1 does not change between the two.

  In the present invention, further, in this method B, as shown in FIG. 5, the slit-shaped image areas are cut out so that overlapping areas are formed. If the slit width of the image area is quadrupled, the number of image areas that overlap in the overlap area is four as shown in FIG. In such a case, the overlap number L is 4. Next, a panoramic overall image is formed by superimposing the image areas where such overlapping areas are generated. That is, by increasing the shutter speed, photographing with low sensitivity is inevitably executed. However, by superimposing these overlapping image areas, the noise components of each image area can be averaged. For example, by superimposing four image areas photographed with a sensitivity of 1/4, an image photographed with a sensitivity equivalent to a single sensitivity can be obtained, and a high-quality whole image with no noticeable noise component can be created.

  As described above, in the imaging apparatus 1 to which the present invention is applied, image regions of a predetermined size are cut out so that overlapping regions are generated, and the cut-out image regions are sequentially overlapped. An overlapping area generated by superimposing the image areas can be used to improve the image quality. That is, by increasing the shutter speed, even if the entire image to be obtained becomes dark and noise also increases, a high-quality image can be generated by creating an overlapping region.

  Incidentally, in the present invention, this overlap number L is a constituent requirement that two or more image areas overlap on average for each image position of the entire image to be generated (preferably the overlap number is two or more). It is good. As a result, at least two or more image areas are overlapped at each image position, and noise is reduced at all image positions for the overall image to be finally generated, and image quality is improved. Will be.

  Next, a procedure for photographing based on the method B in the imaging apparatus 1 to which the present invention is applied will be described.

  In this method B, as shown in FIG. 6, first, in step S1, various shooting parameters are calculated. In this step S1, information relating to the brightness identified by the exposure meter 26 is acquired, and photographing parameters such as an aperture value and a shutter speed are calculated.

  FIG. 7 is a flowchart showing the operation procedure in step S1 in more detail. When the process proceeds to step S1 and the shooting parameters are calculated, first, the brightness I of the object to be shot is measured via the exposure meter 26 in step S11 shown in FIG. Information regarding the brightness I is transmitted to the CPU 21.

  Next, the process proceeds to step S <b> 12, and the CPU 21 receives an operation signal D <b> 1 sent from the operation unit 22. The operation signal D1 includes various parameters such as an aperture value A, an exposure time S, an overlap number L, and a rotational angular velocity M of the motor 24, which are input by the user via the operation unit 22. Incidentally, in this imaging apparatus 1, the user may specify the values of the respective shooting parameters with specific numerical values in accordance with the intention of shooting, and the CPU 21 itself has the values of some shooting parameters as variables. You can also leave the decision.

  Next, the process proceeds to step S13, and if there is a shooting parameter not specifically specified by the user in step S12, this is obtained by calculation. Each shooting parameter can be associated with each other by a mathematical formula, and shooting parameters not specified by the user in step S12 can be calculated.

The relationship between the rotational angular velocity M of the motor and the exposure time S can be expressed as follows.
M∝1 / S (∝ represents proportionality) (1)
Based on this equation (1), the photographing direction can be changed at an angular velocity that monotonously decreases with respect to the exposure time S, and thus at an angular velocity that is inversely proportional to the exposure time S.

Further, the relationship between the exposure time S, the aperture value A, and the overlap number L with respect to the brightness I measured by the exposure meter 26 described above can be expressed as follows.
I∝1 / S ・ 1 / A2 / 1 / L ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ （2）
Further, the relationship between the slit-shaped image region W and the number L of overlapping images can be expressed as follows.
W∝L (3)
Each proportionality constant in these proportional relationships (1), (2), (3) is determined based on the characteristics of the hardware.

  It should be noted that when there are two or more variables in the proportional relationships (1), (2), and (3) when calculating the shooting parameters not specified by the user, this may not be obtained. . In such a case, these variables may be optimized in such a way as to reflect the intention of photographing set in advance by the user.

  An example in which the shooting parameters are obtained by calculation in step S13 will be described. Note that k1, k2, and k3 used in the following equations are proportional constants that match the characteristics of the hardware.

  For example, when the rotation angle speed M and the aperture value A are given by the user, the exposure time S is first obtained by the following equation (4).

S = k1 / M (4)
Next, the overlap number L is obtained based on the following equation (5).

L = k2 / S / A2 / I (5)
Then, the width W of the slit-like image is obtained based on the following equation (6).

W = k3L (6)
Next, the process proceeds to step S <b> 14, and it is confirmed whether or not the obtained imaging parameters satisfy the constraints of the imaging elements and hardware including the entire imaging device 1. As a result, if the obtained shooting parameters satisfy the hardware constraints, the process proceeds to step S15. On the other hand, if the obtained shooting parameters do not satisfy the hardware constraints, the process proceeds to step S16.

  When the process proceeds to step S15, it is notified that the value of the shooting parameter set by the user is good. On the other hand, when the process proceeds to step S16, it is notified that the value of the imaging parameter is inappropriate, and the user is requested to reset the aperture value A and the rotational angular velocity M. When the imaging parameter is set again by the user who has received such a request, an optimal setting that conforms to hardware constraints is made.

  After executing the details of step S1 shown in FIG. 7, the process proceeds to step S2 shown in FIG. 6, and whether or not the operation signal D1 based on the pressing input operation of the shutter button 221 in the operation unit 22 is generated. Identify. As a result, when the generation of the operation signal D1 can be identified, the process proceeds to step S3. On the other hand, when the generation of the operation signal D1 cannot be identified, the process returns to step S1 and the above-described process is repeatedly executed.

  When the process proceeds to step S3, the imaging unit 10 is turned by driving the motor 24. The value determined in step S1 is adopted as the rotational angular velocity M of the motor. In the following procedure, an example in which the shooting direction of the imaging unit 10 is changed to the horizontal direction will be described.

  Next, the process proceeds to step S4, and a panorama shooting process is executed.

  FIG. 8 is a flowchart showing in more detail the panorama shooting processing procedure in step S4. The imaging unit 10 performs imaging of an image in step S21 shown in FIG. In this imaging, the imaging parameters determined in step S1 are used. The imaging parameters may be updated sequentially by executing the same processing as in step S1 each time step S21 is executed. The subject image captured in this step S21 is converted into an electrical signal by the CMOS image sensor 11, and a part of the slit-shaped image region is selected, and the pixel value of the image region is read out to become the imaging signal C1. . The imaging signal C1 is sequentially converted into the imaging signal C2 as described above, and stored in an image RAM (not shown) in the DSP 15, and then the process proceeds to step S22.

  In step S22, matching processing is performed for calculating the relative displacement between the newly acquired image area and the previously captured image area.

  FIG. 9 is a flowchart showing in more detail the matching processing procedure in step S22.

  First, under the control of the CPU 21, the DSP 15 executes preprocessing such as black level compensation processing and gamma correction processing for a newly acquired image area in step S31 shown in FIG.

  In step S32, a low pass filter is applied to the acquired image area and the result is resampled. When the angular velocity of the motor is high, a high-speed shutter corresponding to the motor is required. In this case, since the image is shot with an exposure time shorter than the normal shooting time, the entire obtained image becomes dark and noise increases. For this reason, this noise is reduced before the matching process is actually performed in step S32. In this embodiment, since the accuracy of the displacement in the horizontal direction is important, a filter is applied so as to average the direction orthogonal to the displacement direction. In the present invention, the vertical direction is averaged as a direction orthogonal to the horizontal direction. In this filtering process, as shown in FIG. 10, when an image region as shown in F1 is newly acquired, the image region F1 is first divided into a plurality of rows and averaged for each column. In the example shown in FIG. 10, when five pixels arranged in the row direction are averaged as one group, one-dimensional image sequences Pa1, Pa2, Pa3 as shown in F2 are obtained. This image sequence is used in the subsequent matching processing.

  When the process proceeds to step S33, normalization is performed on an image area obtained by imaging before the image area F1. The captured image area is sequentially stored in a relatively large storage area for storing a large number of pixels. The image area generated by being superimposed on this storage area is hereinafter referred to as a canvas image. Each pixel constituting the canvas image can be defined by four elements including the number of overlapping L in addition to the three primary color components of R, G, and B as in a normal image. As a new image is captured, the number L of overlaps is increased by one. In the normalization process in step S33, (R / L, G / L, B / L) may be calculated as shown in FIG. An image can be obtained. Each pixel of the entire image generated in this way is composed of denominator data and numerator data, and the sum of pixel values constituting the same pixel position in the overlapping image area is used as numerator data, and the image area at the same pixel position By using the number of superimposed images as the denominator data, it is possible to average the pixel values of each image region and create an entire image with less noise.

  Next, the process proceeds to step S34, and the filtering process and the resampling process in step S32 are applied to the image normalized in step S33. FIG. 12 divides the normalized image into groups each including the number of pixels corresponding to the filtering process in step S32. Then, a filtering process is performed to average the vertical direction as the direction orthogonal to the displacement direction. Then, one-dimensional image sequences Pb1, Pb2, and Pb3 corresponding to the canvas image described above are generated. Incidentally, these image sequences Pb1, Pb2, and Pb3 correspond to the image sequences Pa1, Pa2, and Pa3, respectively.

  Next, the process proceeds to step S35, and the image sequences Pa1, Pa2, Pa3 obtained in step S32 and the image sequences Pb1, Pb2, Pb3 obtained in step S34 are pattern-matched with each other. Then, the relative position displacement between the image sequences is obtained. Since the image sequence to be matched has been subjected to filtering processing in steps S32 and S34, high-precision matching without the influence of noise can be realized. In this pattern matching, for example, the correlation value is obtained while shifting the image sequence Pb1 corresponding to the image sequence Pa1, and the position where the value is the maximum is the position where both image sequences are most correlated (hereinafter referred to as the maximum correlation position). As specified. This is executed for all image sequences.

  After completing the matching process in step S35, the overlay process in step S23 in FIG. 9 is executed. In this superimposition process, the correction for aligning the image area F1 obtained by the above-described new imaging with the maximum correlation position obtained in step S22 is executed. Then, the image area F1 executes processing for adding each pixel on the canvas image. Further, 1 is added to the overlap number L for each pixel on the canvas image every time the addition process is executed. As a result, each pixel value constituting the panoramic overall image can be calculated from the sum of the pixel values of a plurality of image areas, so that an overall image with a good S / N ratio with reduced random noise can be obtained. This makes it possible to produce a natural high-quality overall image.

  In step S24, the generated canvas image is displayed on the monitor unit 20 as the entire image. That is, in step S24, the image area necessary for displaying the canvas image is normalized in the same manner as in step S33 and displayed on the monitor unit 20. FIG. 13 shows an example in which the above-described canvas image is displayed on the display screen of the monitor unit 20. At the start of shooting, no image is displayed on the display screen, but after k seconds, the generated canvas image is displayed from the left end of the display screen. Further, after 2 k seconds, a canvas image to which a new image area is added is sequentially joined to the right side, and after 3 k seconds, a canvas image in which further image areas are superimposed is displayed. By the way, in the display process after 3k seconds, the generated entire image as the canvas image exceeds the size of the screen, so that the newly superimposed image area is scrolled in a preferential manner. . Similarly, in the display process for 4 k seconds, the generated whole image exceeds the size of the screen, so that the newly superimposed image area is scrolled in such a manner that it is displayed preferentially.

  By preferentially displaying the image area newly captured and generated in this way, the user can check the image area generated via the monitor unit 20 in real time even during shooting. it can.

  After finishing the display process in step S24, the process proceeds to step S25, and it is identified via the operation signal D1 whether or not the pressing input of the shutter button 221 in the operation unit 22 is continued. As a result, when the pressing input operation is still continued, the process returns to step S21 and the imaging of the subject is repeated. On the other hand, when it is determined that the pressing input of the shutter button 221 has been completed, the process of step S4 shown in FIG. 8 is ended, and the process proceeds to step S5 shown in FIG.

  In step S5, the rotation of the imaging unit 10 is stopped by stopping the driving of the motor 24.

  Further, when the process proceeds to step S6, when all the areas of the subject imaged on the imaging surface in the CMOS image sensor 11 are read based on the method A, the image of the area is generated as described above. Execute processing to join to the canvas image. The provision of this step S6 is not necessarily essential, but is particularly suitable when photographing a subject as shown in FIG. FIG. 14 shows an example of photographing a person who is under a large tree. By applying this embodiment including this step S6, a portion of a leaf swaying in the wind is changed to a method B suitable for photographing a moving subject. Based on the method A as a normal photographing method, photographing is performed on the person portion.

  Finally, the process proceeds to step S7, where the acquired image data is encoded by the codec processing unit 16 and written into the memory 17 to finish the process.

  As described above, in the imaging apparatus 1 to which the present invention is applied, the image quality can be improved by overlapping the image areas cut out as described above, and the camera can be rotated and stopped at high speed and accurately. Repeated hardware is not necessary, and the overall system cost can be reduced.

  In the above-described embodiment, the case where the shooting direction is moved in the horizontal direction has been described as an example. However, the present invention is not limited to this case, and the present embodiment is also applicable when the shooting direction is moved in the vertical direction. The invention can be applied as well.

  Alternatively, the image areas may be overlapped each time the image pickup unit 10 performs image pickup, and the image areas after the overlap may be discarded. As a result, the memory capacity required for processing can be reduced, so that the hardware implementation cost can be reduced, and a system more suitable for real-time processing can be constructed.

  The present invention may be further applied to an imaging device 3 described below. With respect to the same configuration and elements of the imaging apparatus 3 as those of the imaging apparatus 1 described above, the description of FIG.

  For example, as shown in FIG. 15, the imaging device 3 captures an image of a subject via an imaging unit 10 including a lens unit 10a, and can rotate in the direction of arrow A in FIG. A first casing 31 and a second casing 32 that is disposed below the first casing and can be held with one hand of the user are provided. Incidentally, the second casing 32 is provided with a shutter button 221 and a display unit 43 for the user to visually recognize the captured image.

  FIGS. 16A and 16B show the internal configuration of the imaging apparatus 3. The first housing 31 includes a lens unit 10a, a CMOS image sensor 11, and a first electronic circuit 41 having at least a CDS circuit 12 and an A / D conversion unit 13.

  The second casing 32 is connected to the first casing 31 via a rotation shaft 38, and controls the motor 51 for rotating the rotation shaft 38 and the rotation angle of the rotation shaft 38. In the photo interrupter 52 and the shielding plate 53 for carrying out, a stopper 54 for physically limiting the rotational operation of the rotary shaft 38, and a connecting hole provided in the upper surface of the second housing 32 A bearing 55 that is inserted and realizes a smooth rotation operation by supporting the rotating shaft 38, a second electronic circuit 42 for controlling each component of the imaging device 3, and each component And a battery 44 for operating the element.

  As shown in FIG. 16B, the second electronic circuit 42 includes a DSP 15, a memory 17, a CPU 21, an operation unit 22, a shutter button 221, a display unit 43, a motor, a photo interrupter, or An interface 47 for transmitting / receiving image data to / from an external device is provided, and a wiring 56 for transmitting / receiving an electrical signal to / from the first electronic circuit 41 is connected.

  Incidentally, the second casing 32 is provided with buttons for switching between the general shooting mode and the panoramic shooting mode and various buttons of a general digital camera in addition to the above-described components. Yes.

  The motor 51 is configured by a stepping motor or the like that rotates the rotary shaft 38 at an angular velocity corresponding to the supplied drive pulse. The first housing 31 is connected to one end side of the rotating shaft 38 rotated by the motor 51, and a stopper 54 is fixed to the middle. A shielding plate 53 is fixed to the other end of the rotating shaft 38. That is, the first casing 31 rotates in accordance with the rotation operation of the rotating shaft 38 based on the motor 51, and the shielding plate 53 and the stopper 54 rotate in the same manner. Incidentally, the relative displacement in the rotation angle of the rotating shaft 38 by the motor 51 can be detected by the number of pulses applied to the motor 51.

  The display unit 43 includes all configurations similar to the D / A conversion unit 18, the video encoder unit 19, and the monitor unit 20, and displays an image generated via a liquid crystal display element provided on the side surface of the second housing 32. Since the display unit 43 is provided apart from the first casing 31 rotated by the rotation shaft 38, visibility can be improved without being controlled by the rotation of the rotation shaft 38.

  The photo interrupter 52 includes a light emitter 52a and a light receiver 52b which are provided vertically. The light receiving body 52b continues to receive the optical signal from the light emitting body 52a. However, when the shielding plate 53 is rotated to the vicinity of the photo interrupter 52 according to the rotation of the rotating shaft 38, the light is received. The signal is shielded by the shielding plate 53. That is, it is possible to identify the rotation state of the rotary shaft 38 from the shielding state of the optical signal received by the light receiving body 52b.

  In addition, when the rotating shaft 38 rotates beyond the movable range, the shielding plate 53 may shield the optical signal, and the motor 51 may be stopped according to the shielding. FIG. 17 shows the photo interrupter 52 and the shielding plate 53 taken from the direction B in FIG. As shown in FIG. 17, since the shielding plate 53 does not block the optical signal received by the light receiver 52b within the movable range, the rotating shaft 38 is freely rotated without stopping the motor 51. be able to. On the other hand, when the movable range is exceeded, the shielding plate 53 blocks the optical signal received by the light receiving body 52, and the motor 51 stops. If the movable range is exceeded, the rotational operation of the first casing 31 may be physically suppressed via the stopper 54.

  In this way, by introducing a mechanism that limits the rotation range of the first casing 31 relative to the second casing 32, data transmission from the first casing 31 to the second casing 32 is performed. Without using a special joint part using a photocoupler, it can be realized by using a flexible wiring material used in a normal movable part.

  A user who uses the imaging apparatus 3 having the above-described configuration performs shooting while holding the second casing 32. In such a case, by pressing the shutter button 221 provided outside the second housing 32, the above-described shooting start timing and shooting end timing can be designated. By providing the shutter button 221 at a position where the second casing 32 is touched by the fingertip of the user holding the second casing 32 with one hand, it is possible to support the start and end of photographing with a small movement of the fingertip.

  Accordingly, by holding the second casing 32 with one hand and pressing the shutter button 221, the first casing 31 is gradually rotated from the state shown in FIG. As shown in (b), the shooting direction of the imaging unit 10 changes. Further, by further pressing the shutter button 221, as shown in FIG. 18C, the first casing 31 is further rotated, and as a result, the shooting direction of the imaging unit 10 is further changed. That is, the first casing 31 is configured to be rotatable so that the shooting direction of the imaging unit 10 is sequentially changed around the rotation axis 38 connected.

  Incidentally, when the shutter button 221 is continuously pressed in the state shown in FIG. 18 (c), the rotation of the motor 51 is suppressed by the above-described photo interrupter 52, and then the first casing 31 is moved. Further, it may be rotated in the opposite direction.

  Incidentally, the second casing 32 is mounted with components having large masses such as a battery 44 and a motor 51 and a second electronic circuit 42 composed of many circuits, whereas the first casing 32 is mounted on the second casing 32. On the body 31, components having a relatively small mass such as the lens unit 10a necessary for photographing, the CMOS image sensor 11, the first electronic circuit 41, and the like are mounted. That is, the first casing 31 is configured to have a lighter mass than the second casing 32. As a result, it is possible to reduce the mechanical load on the parts such as the rotary shaft 38 and the bearing 55 for rotating the first casing 31, and further on the motor 51 and the like, and to keep the cost of the entire product low. It is also possible to realize a general consumer digital camera having a panoramic shooting function at a low cost.

  FIG. 19 is a flowchart illustrating a shooting process performed by the imaging device 3.

  The imaging apparatus 3 proceeds to step S61 after performing hardware diagnosis and initialization.

  In step S61, as with a normal camera, the user adjusts the aperture amount, exposure, and the like while using assistance such as an automatic exposure function. As a result, various shooting parameters are determined.

  Next, the process proceeds to step S62, and the pressing of the shutter button 221 is checked to identify the photographing start timing. If the shutter button 221 is pressed, the process directly proceeds to step S63, and if not, the process returns to step S61.

  When the process proceeds to step S63, the imaging unit 10 performs imaging. Data acquired by imaging is transferred to the DSP 15 via the CDS circuit 12 and the A / D converter 13.

  In step S64, the DSP 15 performs image processing to synthesize a panoramic overall image. FIG. 20 shows a state in which continuously taken images are connected to compose a single panoramic overall image. Here, it is assumed that the unit images photographed continuously are photographed so that overlapping regions are generated. With respect to this overlapping region, the whole image as shown in FIG. 20 can be synthesized by connecting them based on a conventionally known synthesis method. Incidentally, as this synthesis method, for example, a synthesis technique such as alpha blending may be used.

  Next, the process proceeds to step S65, and the shooting direction by the imaging unit 10 is slightly changed by rotating the first casing 31 with respect to the second casing 32. The amount of rotation of the first casing 31 is adjusted so that an overlapping area is generated between the unit images taken at least continuously. This rotation amount can also be obtained in advance by geometric calculation when the imaging device 3 is designed. In addition, the rotation amount of the first housing 31 may be dynamically finely adjusted by mounting a function for detecting the amount of shaking of the entire imaging device 3 during shooting.

  In step S65, the rotation amount of the motor 51 is set as small as possible, and the continuous shooting is performed by increasing the number of shots per unit time, thereby shooting the distortion due to parallax and moving subjects. The discontinuity of the image can be eliminated, and it becomes possible to synthesize a high-quality whole image. In order to perform such shooting, intermittent and highly accurate control of the motor 51 is required. As described above, the first casing 31 has a mass higher than that of the second casing 32. Therefore, these can be realized at low cost.

  In step S66, the pressing of the shutter button 221 is checked again to identify the timing of the end of shooting. If it is confirmed in step S66 that the shutter button 221 has been pressed, the process returns to step S63 to continue shooting. On the other hand, if it is not confirmed in step S66 that the shutter button 221 has been pressed, the process proceeds to step S67 in order to shift to the photographing end operation.

  In step S67, the combined panoramic overall image is stored in the memory 17 as a process at the end of shooting.

  As described above, by mounting the shutter button 221 on the second casing 32 that is separated from the first casing 31, it is possible to instruct the start and end of photographing only with a small operation of the fingertip. Moreover, the visibility of the obtained image can be improved by providing the display part 43 in the 2nd housing | casing 32. FIG.

  In addition, since at least the parts necessary for photographing such as the lens unit 10a and the CMOS image sensor 11 are mounted in the first casing 31 that is actually rotated, the mass of the rotating part is reduced. The cost of the components that support the rotating portion of the first housing 31 and the motor 51 that rotationally drives the components can be reduced. In addition, by mounting a component serving as a driving source such as the motor 51 in the second casing 32 held by the user, it is possible to suppress the influence of the vibration accompanying the rotation of the motor 51 as much as possible.

  Further, by limiting the rotation range of the first casing 31 using the photo interrupter 52, data transfer between the rotating portion and the holding portion can be realized using a normal and inexpensive flexible wiring material. .

  In addition, in this imaging device 3, you may make it perform all operation | movement of the imaging device 1 mentioned above. Thereby, in addition to the effect of the imaging device 3 itself, the effects of the imaging device 1 such as the improvement of image quality and the reduction of the cost of the entire system can be achieved synergistically by overlapping the cut-out slit-like image areas. It is also possible to construct a simple shooting system.

  The imaging device 3 to which the present invention is applied is not limited to the above-described embodiment, and the DSP 15, the CPU 21, the display unit 43, the motor 51, the operation unit 22 (shutter button 221), and the battery 44 are included. Any one or more may be provided in the second housing 32.

  By the way, in actual photographing, for example, the machine automatically repeats a series of processes from the first step to the third step.

  First step: The motor is stopped and the camera is fixed so that the camera does not move (does not move).

  Second step: Take a picture.

  Third step: The direction of the camera is changed by the motor.

  In this case, strictly speaking, it is difficult to perform these three steps steadily. That is, the camera has a certain mass, and when this camera is rotated by a motor, it is impossible to stop the rotation of the camera immediately due to the law of inertia. That is, the state of the first step cannot be set immediately after the third step. Therefore, the second step is performed in a state where the camera has a slight blur. Therefore, the image obtained in the second step is an image having a minute blur. This blur is derived from the law of inertia, and is a blur in the same direction as the motor is about to move in the third step. The problem that blurring in the same direction as the direction in which the camera moves occurs in the captured image can be solved as follows.

  In other words, the imaging device 3 cuts out in order to make the blurring of an image inconspicuous when combining unit images captured continuously as described above to synthesize one panoramic overall image. An anisotropic filter process is performed on the image of the image area, that is, each unit image or the generated whole image.

  For example, the image capturing apparatus 3 according to this embodiment is used to perform panning in the horizontal direction, and as shown in FIG. 21, images of cut-out image regions, that is, unit images A1, A2,... A15 are obtained. When a panoramic image as shown in FIG. 22 composed of the images a1 to a15 corresponding to the unit images A1 to A15 is generated, minute blur occurs in the horizontal direction. A filter process for emphasizing components is performed.

  A specific example is shown in FIG. FIG. 23 is an example of photographed image data (slit-like image). An arrow A in FIG. 23 indicates the rotation direction of the imaging device 3. Here, in order to explain the blur easily, it is assumed that a square object is photographed. That is, 46 is a projected image of a square object. The projected image 46 is blurred in the direction of arrow A. That is, among the four sides (edges) 461 to 464 of the projected image 46, the upper side 461 and the lower side 463 are shown as sharp sides (edges), but the right side 462 and the left side 464 are blurred sides ( Edge).

  Therefore, a filter process for emphasizing the high frequency in the direction of the arrow A (in this case, the horizontal direction) is performed. As an example of a filter that emphasizes the high frequency band, for example, as shown in FIG. 24, (−1/3) × (pixel value on the left side of the target pixel) + (5/3) × (value of the target pixel) It is only necessary to calculate + (− 1/3) × (pixel value on the right side of the target pixel).

  Thus, for example, the image data shown in FIG. 23 becomes the image data shown in FIG. 25, and the upper side 471 and the lower side 473 corresponding to the upper side 461 and the lower side 463 are clear as in FIG. The right side 472 and the left side 474 respectively corresponding to the 462 and the left side 464 are also clear.

  Based on this, the imaging device 3 in this embodiment generates a panoramic image by performing the processing shown in the flowchart of FIG.

  That is, first, in step S71, the user sets whether the user is in the normal shooting mode or the panoramic shooting mode (that is, mode setting). Then, the process proceeds to step S2.

  In step S72, the process waits until the user presses the release button (shutter button). If the release button is pressed, the process proceeds to step S73.

  In step S73, the mode set in step S71 is determined. That is, when it is determined that the normal shooting mode is selected, the process proceeds to step S74. If it is determined that the panorama shooting mode is selected, the process proceeds to step S75.

  In step S74, the following processing is performed. That is, the CMOS image sensor 11 is driven by a driver (not shown), and an image signal for one frame is output as serial data. The output signal is converted into a digital signal by the A / D converter 13 and then sequentially sent to the DSP 15. In the DSP 15, separation of each color component in the signal, gain control, gamma correction, color balance adjustment, matrix calculation of the signal component, and the like are performed, and the processed video signal is recorded in the memory 17.

  In step S75, the following processing is performed. That is, the CMOS image sensor 11 is driven by a driver, and an image signal for one frame is output as serial data. The output signal is converted into a digital signal by the A / D converter 13 and then sequentially sent to the DSP 15. The DSP 15 extracts only a part of the signal from the A / D converter 13 (slit-shaped rectangular area), and for each part, separates each color component in the signal, gain control, gamma correction, and color balance adjustment. The signal component matrix calculation is performed, and the processed slit-shaped video signal is recorded in the memory 17. At this time, the CPU 21 sends a rotation signal as a control signal to the motor 24 in conjunction with the writing operation. The motor 24 pans the first casing 31 of the imaging device 3 by a certain rate in synchronization with this rotation signal. After step S75, the process proceeds to step S76.

  In step S76, since the slit-shaped image data is stored in the memory 17, the CPU 21 transfers these slit-shaped images stored in the memory 17 to the DSP 15 again. Then, the process proceeds to step S77.

  In step S77, the DSP 15 applies a high frequency enhancement filter in the same direction as the rotation direction to each of these images. After step S7, the process proceeds to step S8.

  In step S78, the DSP 15 joins these images to generate one panoramic image. Then, the CPU 21 writes the generated panoramic image in the memory 17 in a predetermined format. After step S78, this series of processing ends. At the end, panoramic image data is written in the memory 17.

  Instead of performing high-frequency emphasis filter processing on the captured image data (slit-like image), the panoramic image after stitching is read out from the memory 17 and high-frequency emphasis filter processing is performed by the DSP 15. May be.

  In this way, when shooting in the panorama shooting mode, it is possible to create a panorama image without blur and store the data in the memory 17.

  As described above, when shooting a plurality of images while changing the direction (or position), the direction (referred to as a first direction) is defined. That is, the first direction is a rotation direction during continuous shooting. In order to compensate for blurring that occurs with respect to the first direction, a filter that emphasizes the high frequency is applied to the first direction. This makes it possible to restore a blur-free (inconspicuous) image. Further, the direction in which the high frequency is emphasized is only with respect to the first direction, and the high frequency enhancement filter is not applied to the direction orthogonal to the first direction (referred to as the second direction). . The photographed image is an image having no blur in the second direction, and when high-frequency emphasis is performed in the second direction, an unnatural image (for example, an image having an overshoot or undershoot at an edge portion). In the present invention, since the high-frequency emphasis filter is applied only in the first direction, such a problem does not occur.

It is a figure for demonstrating about the structure of the imaging device of the panoramic image to which this invention is applied. It is a figure for demonstrating about the example which synthesize | combines the panoramic whole image by joining the acquired slit-shaped image, mutually superimposing. It is a figure for demonstrating about the method of cutting out a slit-shaped image area. It is a figure which shows the relationship of the readout time of the pixel value with respect to the imaging time of the methods A and B. It is a figure for demonstrating about the case where it cuts out so that a slit-shaped image area may mutually overlap in the system B. FIG. It is a flowchart which shows the procedure which image | photographs based on the system B in the imaging device to which this invention is applied. It is a flowchart which shows the calculation procedure of various imaging parameters. It is a flowchart which shows the panorama imaging | photography process procedure in detail. It is a flowchart which shows a matching process procedure in detail. It is a figure for demonstrating about the method of averaging the direction orthogonal to a displacement direction. It is a figure for demonstrating normalization of a canvas image. It is a figure for demonstrating about the filtering process of the normalized image. It is a figure for demonstrating about the display method in a monitor part. It is a figure for demonstrating about the example which performs imaging by combining the system A and the system B. FIG. It is a figure for demonstrating about the external appearance of the imaging device of the panoramic image which has a 1st housing | casing and a 2nd housing | casing. It is a figure for demonstrating about another structure of the imaging device to which this invention is applied. It is a figure for demonstrating about the method of controlling rotation of the 1st housing | casing by a photo interrupter. It is a figure for demonstrating about the example which rotates a 1st housing | casing gradually by pressing a shutter button. It is a flowchart which shows the imaging | photography process by an imaging device. It is a figure shown about a mode that a continuous panoramic image is connected and one panoramic whole image is synthesize | combined. It is a figure which shows each unit image of the image area image | photographed and cut out, panning in the horizontal direction with an imaging device. It is a figure which shows the panoramic image which connected and united each unit image shown in FIG. It is a figure which shows the image before a filter process. It is a figure which shows the structural example of a filter typically. It is a figure which shows the image after filter processing. It is a flowchart which shows the imaging | photography process by an imaging device.

Explanation of symbols

1, 3 imaging device, 10 imaging unit, 11 CMOS image sensor, 12 CDS circuit, 13 A / D conversion unit, 14 internal bus, 15 DSP, 16 codec processing unit, 17 memory, 18 D / A conversion unit, 19 video Encoder unit, 20 monitor unit, 21 CPU, 22 operation unit, 23 timing generator, 24 motor, 26 exposure meter

Claims (36)

An imaging unit that sequentially changes the shooting direction and takes a unit image that forms part of the shooting range;
Image cutout means for cutting out an image area of a predetermined size constituting a part of a unit image picked up by the image pickup means so that an overlapping area is generated;
An image pickup apparatus comprising: an entire image generation unit configured to generate an entire image representing the entire photographing range by sequentially superimposing the image areas cut out by the image cutting unit. The imaging apparatus according to claim 1, wherein the imaging unit captures a next unit image when the image region is cut out by the image cutting unit. 2. The imaging apparatus according to claim 1, wherein the image cutout means cuts out the image area so that two or more image areas overlap on average for each image position of the entire image to be generated. An identification means for identifying the brightness of the object imaged by the imaging means;
The image cutout means determines the size of the image region in accordance with the aperture value and the exposure time of the imaging in addition to the brightness identified by the identification means. Item 2. The imaging device according to Item 1. The whole image generation means averages the luminance components of the image areas to be superimposed in a direction orthogonal to the shooting direction, and aligns the image positions with each other based on the distribution of the averaged luminance components. The imaging apparatus according to claim 1. The imaging apparatus according to claim 1, wherein the imaging unit changes the imaging direction at an angular velocity that monotonously decreases with respect to the exposure time of the imaging. The imaging apparatus according to claim 1, wherein the imaging unit changes the imaging direction at an angular velocity that is inversely proportional to the exposure time of the imaging. It further comprises detection means for giving an operation signal based on a user's pressing input operation,
2. The imaging device according to claim 1, wherein the imaging unit executes the imaging when an operation signal is detected by the detection unit, and stops the imaging when the operation signal is not detected by the detection unit. Imaging device. The whole image generating means generates the whole image by joining the unit image to the image area when a new unit image is taken after the imaging is stopped by the imaging means. The imaging apparatus according to claim 8, characterized in that: A display unit for displaying the entire image generated by the entire image generation unit on a screen;
The imaging apparatus according to claim 1, wherein the display unit preferentially displays a newly superimposed image region when the generated whole image exceeds the screen. 2. The overall image generation unit performs superimposition of the image areas each time imaging is performed by the imaging unit, and further discards the image area after completion of superimposition. Imaging device. The imaging apparatus according to claim 1, wherein the whole image generation unit includes each pixel of the whole image to be generated by denominator data and numerator data. The whole image generating means uses the sum of the pixel values constituting the same pixel position in the image area overlapping each other as the numerator data, and sets the number of overlapping image areas at the same pixel position as the denominator data. The imaging device according to claim 12. The imaging apparatus according to claim 1, wherein the whole image generation unit performs anisotropic filter processing on the image of the image region cut out by the image cutout unit. The imaging apparatus according to claim 1, wherein the whole image generation unit performs anisotropic filter processing on the generated whole image. 16. The imaging apparatus according to claim 14, wherein the anisotropic filter process is a filter process that emphasizes a high frequency component only in a rotation or movement direction. An image capturing unit that captures a unit image that forms a part of the capturing range by sequentially changing the capturing direction and an image region of a predetermined size that forms a part of the unit image captured by the image capturing unit are overlapped with each other. A first housing that has at least an image cutting unit that cuts out in such a manner, and is rotatable so that a shooting direction of the imaging unit sequentially changes with a connected rotation axis as a rotation center;
An overall image generating means for generating an entire image representing the entire photographing range by sequentially superimposing the image areas cut out by the image cutting means;
Display means for displaying the whole image generated by the whole image generation means;
Driving means for rotating the rotating shaft;
Designating means for designating each timing of imaging start and imaging end in the imaging means;
A battery for operating each of the above components;
A second housing that is connected to the first housing via the rotating shaft and can be gripped with one hand of a user;
The imaging apparatus, wherein the second casing is provided with any one or more of the whole image generation means, the display means, the drive means, the designation means, and the battery. 18. The imaging apparatus according to claim 17, wherein the driving unit is further provided with a control mechanism for limiting a rotation angle of the rotation shaft within a predetermined range. It has at least an imaging unit that captures unit images that form part of the imaging range by sequentially changing the imaging direction, and can be rotated so that the imaging direction of the imaging unit changes sequentially around the connected rotation axis A first housing,
Whole image generating means for generating a whole image representing the whole photographing range by sequentially pasting unit images picked up by the image pickup means;
Display means for displaying the whole image generated by the whole image generation means;
Driving means for rotating the rotating shaft;
Designating means for designating each timing of imaging start and imaging end in the imaging means;
A battery for operating each of the above components;
A second housing that is connected to the first housing via the rotating shaft and can be gripped with one hand of a user;
The second casing is provided with any one or more of the whole image generating means, the display means, the driving means, the specifying means, and the battery. . The imaging apparatus according to claim 19, wherein the driving unit is further provided with a control mechanism for limiting a rotation angle of the rotation shaft within a predetermined range. An imaging step of capturing unit images that form part of the imaging range by sequentially changing the imaging direction;
An image cutout step of cutting out image areas of a predetermined size constituting a part of the unit image picked up in the image pickup step so that an overlapping area occurs;
An imaging method comprising: an overall image generation step of generating an overall image representing the entire imaging range by sequentially superimposing the image areas cut out in the image extraction step. The imaging method according to claim 21, wherein in the imaging step, the next unit image is captured when the image region is extracted by the image cutting step. The imaging method according to claim 21, wherein in the image cutting step, the image area is cut out so that two or more image areas overlap on average for each image position of the entire image to be generated. An identification step for identifying the brightness of the object to be imaged in the imaging step;
The size of the image area is determined in the image cutting step according to the aperture value and the exposure time of the imaging in addition to the brightness identified in the identification step. Imaging method. In the overall image generation step, the luminance components of the image regions to be superimposed are averaged in a direction orthogonal to the shooting direction, and the image positions are aligned based on the distribution of the averaged luminance components. The imaging method according to claim 21. The imaging method according to claim 21, wherein in the imaging step, the imaging direction is changed at an angular velocity that monotonously decreases with respect to the exposure time of the imaging. The imaging method according to claim 21, wherein in the imaging step, the imaging direction is changed at an angular velocity inversely proportional to the exposure time of the imaging. A detection step of generating an operation signal based on a user's pressing input operation;
The imaging according to claim 21, wherein in the imaging step, the imaging is executed when an operation signal is detected in the detection step, and the imaging is stopped when an operation signal is not detected in the detection step. Method. In the entire image generation step, when the unit image is newly captured after the imaging is stopped in the imaging step, the entire image is generated by joining the unit image to the image region. The imaging method according to claim 25. A display step for displaying the entire image generated in the entire image generation step on the screen;
The imaging method according to claim 21, wherein, in the display step, when the generated whole image exceeds the screen, a newly superimposed image region is preferentially displayed. 23. The whole image generation step, wherein the image regions are overlapped each time the image pickup in the image pickup step is executed, and the image region after the overlap is finished is discarded. Imaging method. The imaging method according to claim 21, wherein, in the entire image generation step, each pixel of the generated entire image is configured by denominator data and numerator data. In the entire image generation step, the sum of pixel values constituting the same pixel position in the image area overlapping each other is used as the numerator data, and the number of overlapping image areas in the same pixel position is used as the denominator data. The imaging method according to claim 21. The imaging method according to claim 21, wherein, in the whole image generation step, anisotropic filter processing is performed on the image of the image region cut out in the image cut-out step. The imaging method according to claim 21, wherein in the whole image generation step, anisotropic filter processing is performed on the generated whole image. 36. The anisotropic filter process in the whole image generation step is a filter process for emphasizing a high frequency component only in the rotation or movement direction. Imaging method.

Images