JP2007011762A - Area extraction apparatus and area extraction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately extract a closed area from an image even when the closed area is observed in a state that a part of four sides of the closed area is disconnected. <P>SOLUTION: A plurality of straight lines are extracted from image data obtained by image pickup and a closed area surrounded by the plurality of extracted straight lines is extracted. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a region extraction apparatus and a region extraction method for extracting a closed region from an image.

In the case of extracting a closed region from an image, conventionally, as disclosed in Non-Patent Document 1, for example, the region is recognized by extracting a rectangular shape within the field of view. That is, the region of interest is recognized by binarizing the image with a predetermined threshold and searching for the closed region.
G.Klinker, D.Stricker, D.Reiners, "Augmented Reality: A Balancing Act Between High Quality and Real-Time Constraints", Proceedings of ISMR '99, 1999, pp.325-346

  In the method of recognizing a closed region by performing a rectangular extraction as described above, it is essential that the four sides appear to be distinguishable from the background, and that the four sides are visible without being interrupted.

  However, for example, an image obtained by taking a photographic print does not necessarily have a contrast that is easily distinguishable from the background on the four sides.

  In addition, when a plurality of photographic prints are placed on a desk or the like, it is not easy to extract a desired photographic print. In particular, in the state where they overlap each other, the four sides of the underlying photographic print do not appear to be continuous, and it is difficult to selectively extract the photographic print.

  The present invention has been made in view of the above points, and an area extraction device and an area that can accurately extract a closed area from an image even when a part of four sides of the closed area is cut off and observed. An object is to provide an extraction method.

One aspect of the region extraction device of the present invention is:
An image input means for inputting an image;
Straight line extraction means for extracting a plurality of straight lines from the image input by the image input means;
Area extracting means for extracting a closed area surrounded by a plurality of straight lines extracted by the straight line extracting means;
It is characterized by comprising.

Another aspect of the region extraction apparatus of the present invention is as follows.
In an image obtained by capturing a subject including an image therein, a side extracting unit that extracts a plurality of sides of the image boundary in the subject;
A region extracting means for extracting a closed region surrounded by a plurality of sides of the image or a plurality of straight lines extending the plurality of sides;
It is characterized by comprising.

Also, one aspect of the region extraction method of the present invention is:
An area extraction method for extracting a closed area from an image,
Enter an image,
Extract multiple straight lines from the input image,
Extracting a closed region surrounded by the plurality of extracted straight lines,
It is characterized by that.

Another aspect of the region extraction method of the present invention is as follows.
An area extraction method for extracting an image included in a subject,
In an image obtained by capturing a subject including an image, a plurality of sides of the image boundary in the subject are extracted,
Extracting a closed region surrounded by a plurality of sides of the extracted image or a plurality of straight lines extending the plurality of sides;
It is characterized by that.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is a case where a part of 4 sides of a closed region is cut and observed, the area | region extraction apparatus and the region extraction method which can extract a closed region from an image accurately can be provided.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
First, with reference to FIG. 1, an outline of a search system to which the region extraction apparatus according to the first embodiment of the present invention is applied will be described.

  That is, the search system includes a digital camera 10, a storage 20, and a printer 30. Here, the storage 20 stores a plurality of image data, and the printer 30 prints out the image data stored in the storage 20. The storage 20 is a memory built in or removable from the digital camera 10, and the printer 30 prints out image data in the memory as the storage 20 in accordance with a printout instruction from the digital camera 10. Alternatively, the storage 20 is a device that can be connected to the digital camera 10 via a connection terminal, a cable, or a wireless / wired network, or a memory that is removed from the digital camera 10 and can transfer image data. There may be. In this case, the printer 30 may be connected to or integrated with the storage 20 and execute a printout in accordance with a printout instruction from the digital camera 10.

  Note that the storage 20 also has a database function configured so that image data can be searched based on feature amounts. In other words, the storage 20 constitutes a feature database that stores a group of feature data (templates) created from digital data of the original image.

  In such a configuration, (1) First, the digital camera 10 shoots a subject including an image of the search source printout 1 once printed out by the printer 30, and the search source print is obtained from the obtained captured image data. An area corresponding to the out 1 image is extracted, and a feature amount of the extracted area is extracted. (2) Then, the digital camera 10 executes a template matching process with the template stored in the storage 20 according to the extracted feature amount. (3) As a result, the image data corresponding to the matched template is read out as the original image data of the search source printout 1. (4) Thereby, the read original image data can be printed out by the printer 30 again.

  The search source printout 1 can use a method of using, as a search source printout, an index print in which a plurality of reduced images are output together, in addition to the printout output in units of one sheet. This is because it is more cost-effective and convenient to select the necessary ones from the index print and print them.

  The search source printout 1 may be a printout made by a printer outside the system (not shown) as long as the original image data exists in the feature amount database configured in the storage 20 in the system. I do not care.

Hereinafter, the search system to which the region extraction apparatus according to the first embodiment is applied will be described in detail.
FIG. 2A is a block diagram of such a search system, and FIG. 2B is an operation flowchart thereof. Note that the digital camera 10 has a captured data search mode in addition to the normal shooting mode, and this operation flowchart shows processing when the search mode is set.

  That is, the user sets at least the search source by the imaging unit 11 of the digital camera 10 in a state where the search source printout 1 desired to be printed again after being set to the search mode is pasted on a table or a wall surface. Photographing is performed so that the printout 1 is not missing (step S11).

  Then, in the digital camera 10, the region extraction unit 12 specifies the image of the search source printout 1 from the image data photographed by the imaging unit 11, and the printout for specifying and extracting the portion. Is cut out (step S12). Details of the printout cutout process will be described later.

  Next, the feature amount extraction unit 13 performs a process of extracting the feature amount from the region of interest specified and extracted by the region extraction unit 12 (step S13). Note that the feature amount may use an arrangement relation of feature points in the image data, or a divided area in the image data according to a predetermined rule, that is, a small area allocated by a predetermined grid. May be used, or may be based on a Fourier transform value for each divided area.

  Next, the matching unit 14 compares the feature amount data extracted by the feature amount extraction unit 13 with a feature amount database of photographed image data configured in the storage 20, and sequentially extracts those having high similarity. A matching process is executed (step S14). That is, as shown in FIG. 3A, the degree of similarity of each photographed image data with the feature amount data is calculated (step S141), and sorting is performed based on the degree of similarity (step S142). Then, original image candidates are selected according to the similarity (step S143). In this selection, a threshold value may be set, or a higher order may be designated in descending order of similarity. In any case, there are two methods: a method of selecting one point with the highest degree of similarity and a method of selecting a plurality of points from those with a high degree of similarity.

  Thereafter, the selected original image candidate image data is read from the storage 20 on the display unit 15 and displayed as an image candidate to be extracted (step S15), and the selection by the user is accepted (step S16). FIG. 4A shows a display screen of the display unit 15 when only one image candidate is displayed. In this display screen, a “previous” and “next” icons 152 representing buttons to be operated when instructing display of other image candidates are displayed next to the display of the image candidates 151, and the image candidates 151 are displayed as desired images. An “OK” icon 153 representing a button to be operated when instructing as data is arranged. The “Previous” and “Next” icons 152 represent the left and right keys of the so-called cross key that the digital camera 10 normally has, and the “Determine” icon 153 is an enter key arranged in the middle of the cross button. Represents something.

  Here, when the cross key corresponding to the “previous” and “next” icons 152 is operated (step S17), the process returns to step S15 and the image candidate 151 is updated and displayed. On the other hand, when the enter key corresponding to the “OK” icon 153 is operated (step S17), the matching unit 14 selects the original image data corresponding to the image candidate 151 stored in the storage 20. Then, the data is sent to the connected printer 30 and printed again (step S18). If the printer 30 is not directly connected to the printer 30 by wire / wireless, a process for performing predetermined marking such as adding a flag to the original image data corresponding to the image candidate 151 stored in the storage 20 is performed. By executing, it is possible to print out by the printer 30 that can access the storage 20.

  Note that, in the display of the image candidates in step S15, a plurality of candidates may be displayed simultaneously. In this case, since the display unit 15 usually installed in the digital camera 10 is naturally a small size of several inches, display of about 4 or 9 points is easy to use. FIG. 4B shows an example in which nine image candidates 151 are displayed. In this case, the thick frame 154 indicating the selected image is moved in accordance with the operation of the left and right keys of the cross key corresponding to the “previous” and “next” icons 152. Although not specifically shown, a so-called page is displayed in which the display of the nine image candidates 151 is changed to the display of the previous and the next nine image candidates in accordance with the operation of the up key and the down key of the cross key. A switching display may be performed.

  Next, the printout cutout process executed in step S12 will be described with reference to the flowchart of FIG.

  That is, first, an edge component is extracted from the image data including the image of the search source printout 1 photographed by the digital camera 10 in order to make it easy to detect a line segment (step S121). This is because, for example, edge components from image data using filters disclosed in Kyoritsu Publishing Co., Ltd., Ozaki et al. “Image Processing”, Nikkan Kogyo Shimbun Co., Ltd., Tezuka et al. “Digital Image Processing Engineering”, etc. Is extracted. In this case, a general filter as shown in FIG. 5 is used as the filter according to the direction of the edge to be detected. Here, an example of extracting edge components in the two directions x and y has been described. However, when the direction of the target search source printout 1 is not specified, edge components are extracted in all directions.

  Next, a linear component is extracted (step S122). For example, linear components are extracted by Hough transform.

  Hough transform is a point by coordinate transformation as disclosed in Kyoritsu Publishing Co., Ltd./Ozaki et al. “Image Processing”, Nikkan Kogyo Shimbun / Tezuka et al. “Digital Image Processing Engineering”, etc. This is a technique for determining a straight line expression from a set of the above and is often used in image processing.

  In the Hough transform, as shown in FIG. 6, the coordinates (X, y) on the image are converted into a Hough curve represented by θ and ρ.

ρ = x cos θ + ysin θ Formula (1)
In the image extracted by the edge extraction filter, the coordinates (x, y) of the pixels exceeding the threshold value are converted into a Hough curve, and a curve is drawn on the θ-ρ image with θ and ρ as coordinate axes. Actually, the coordinates (θ, ρ) of the θ-ρ image are counted up.

  Since the intersection of the Hough curves represents the same straight line, the brightness of the θ and ρ coordinates obtained above (overlap at the intersection) represents the number of pixels existing on the same straight line. The highest luminance coordinate (θ, ρ) on the θ-ρ image is substituted into the above equation (1) to obtain a linear equation.

  Next, each straight line obtained by the Hough transform is decomposed into line segments (step S123). That is, a line segment obtained by extending each straight line is obtained. Then, an area is determined from the connection and shape of each line segment (step S124). That is, an area surrounded by four line segments is searched, and if it is a closed area, it is set as an extraction area. If there is only one search source printout 1 (region of interest) in the captured image field of view, an area surrounded by four straight lines may be searched, but there are a plurality of search source printouts 1. In addition, in the case where some of them overlap, the search source printout 1 below is not surrounded by four straight lines. By searching for each straight line with an extended line segment, it is possible to estimate a closed region for the search source printout 1 under such a line.

  Various methods for recognizing the closed region have been proposed and will not be described here. Here, the edge extraction and the straight line extraction using the Hough transform have been described. However, the method is not limited to these as long as the method detects a straight line.

  7A and 7B show examples of straight line extraction by the above processing. A thick line 16 indicates the extracted straight line. FIG. 7A shows a case where there is only one region of interest in the captured image field of view, and FIG. 7B shows a case where there are a plurality of regions of interest.

  When there are a plurality of regions of interest, a plurality of original image data may be searched, but when narrowing down to one, the following selection can be made.

  For example, the closed region located at the center of the subject image is preferentially selected.

  Alternatively, the obtained plurality of closed areas may be displayed on the display unit 15 of the digital camera 10 so that the user can select a desired closed area by operating the cross key or the like. In this case, partial modification of the user's area may be allowed by such a user interface.

  Further, a closed region having a large area may be preferentially selected.

  Or you may make it select the image (red, blue, etc.) containing many predetermined colors in an extraction area | region.

  Furthermore, a frequency distribution close to a predetermined value in the extraction region may be selected.

  It is also possible to preferentially select a closed region that has little overlap with others, that is, an upper region. There are the following methods for recognizing this state with little overlap.

  Method 1: For the points recognized as being on the same line by the Hough transform, each point in the right diagram in FIG. 6 is again placed on the line in the left diagram in FIG. In this way, line segments on the line are determined, and the vertical relationship between the areas can be obtained from the relationship between each line segment and the extraction area in FIG. 7B. The region above is defined as a region with little overlap.

  Method 2: In a search system, when there are a plurality of templates to be matched, if only a certain template has an extremely low degree of similarity, it can be determined that the portion is shielded by another region.

  In addition to using the above method, the above methods 1 and 2 can be combined to determine a state with less accuracy and less overlap.

  As described above, since the present embodiment is a method for obtaining the intersection of line segments based on the extracted straight line, it is sufficient that a part of each side of the search source printout 1 is visible.

  Accordingly, the target search source printout 1 can be easily extracted from the plurality of search source printouts 1. In particular, even when the search source printouts 1 partially overlap each other, extraction is possible.

  It should be noted that the feature amount database of captured image data configured in the storage 20 as a comparison target used in step S14 needs to be created in advance based on the original image data in the storage 20.

Various methods are conceivable for creating the feature amount database.
For example, when the captured image data is stored in the memory area of the digital camera 10 when the original image is captured, the feature amount is calculated and the database is registered. That is, as shown in FIG. 3B, the digital camera 10 performs shooting (step S201), and the captured image data is stored in the memory area of the digital camera 10 (step S202). Then, feature data is calculated from the stored photographed image data to create template data (step S203), and the created template data is stored in association with the photographed image data (step S204). Therefore, if the storage 20 is a memory built in the digital camera 10, a database is constructed. When the storage 20 is separate from the digital camera 10, both the captured image data and template data stored in the memory area of the digital camera 10 are transferred to the storage 20 and a database is constructed.

  In addition, when the original image data stored in the storage 20 is printed out by the printer 30, it is also a process-efficient method to perform feature amount extraction processing simultaneously with the printout instruction and store it in the database. That is, as shown in FIG. 3C, when printing out the original image data stored in the storage 20, the original image data to be printed is usually selected by a user instruction (step S211). Conditions are set (step S212), and printing is executed (step S213). Normally, the printing process is ended here, but in this example, the feature data is calculated from the selected original image data to create template data (step S214). The original image data is stored in association with it (step S215). In addition, when the template data is created, the matching accuracy between the search source printout 1 and the template data can be improved by reflecting the print conditions. According to such a method, since template data is created only for original image data that may be subjected to matching processing, unnecessary template data creation time and storage capacity can be omitted.

  Of course, badge processing may be used. That is, as shown in FIG. 3D, when there is a collective template creation execution instruction from the user (step S221), the template uncreated original image data in the storage 20 is sorted (step S222), and the sorting is performed. A batch template creation process is executed for the template uncreated original image data (step S223). In the batch template creation process, template data is created by extracting feature amounts from individual template-uncreated original image data (step S223A), and the created template data is stored in the storage 20 in association with the corresponding original image data. (Step S223B).

  Furthermore, it may be processed individually by user input. That is, as shown in FIG. 3E, the user selects one of the original image data in the storage 20 (step S231), and issues an instruction to create template data for the selected original image data. (Step S232), feature data is extracted from the selected original image data to create template data (Step S233), and the created template data is stored in the storage 20 in association with the selected original image data. (Step S234).

  As described above, until now, when trying to reprint image data that has been printed out, the user often searches with reference to auxiliary information (file name, shooting date, etc.) of the image data. According to the search system to which the region extracting apparatus according to this embodiment is applied, the digital camera 10 can access the original image file (image data) simply by capturing the image of the desired search source printout 1. Therefore, it is possible to provide an intuitive and user-friendly search method.

  In addition, it is possible to search not only the original image data itself but also image data having a similar image configuration, and can provide a secondary but novel application. That is, street signs, posters, and the like are photographed in this so-called search mode, and the image data and its feature value data existing in the storage 20 such as a memory attached to the digital camera 10 or a database accessible via communication are used. Similar or identical image data can be easily retrieved.

  The processing in steps S12 to S17 has been described as being performed in the digital camera 10. However, when the storage 20 exists separately from the digital camera 10, such processing is stored in the storage 20 as software. It is also possible to actually activate the digital camera 10 and the storage 20 by dividing the digital camera 10 and the storage 20.

  As described above, since the region extraction apparatus according to the present embodiment is a method for obtaining the intersection of line segments based on the extracted straight line, it is only necessary to see a part of each side of the search source printout 1. Accordingly, the target search source printout 1 can be easily extracted from the plurality of search source printouts 1. In particular, even when the search source printouts 1 partially overlap each other, extraction is possible.

[Second Embodiment]
Next, a search system to which the region extraction apparatus according to the second embodiment of the present invention is applied will be described.

  The present embodiment is characterized by the printout cutout process in step S12 described above, and the others are the same as those in the first embodiment, and a description thereof will be omitted. The present embodiment employs a method of dividing the image and limiting the extracted line segments to extract the closed region.

  That is, in this embodiment, when the search source printout 1 is photographed by the digital camera 10 in the search mode, the guide frame 17 is displayed on the display unit 15 as shown in FIG. Then, the user is allowed to take an image so that one of the upper, lower, left, and right sides of the search source printout 1 enters each of the upper side region, the lower side region, the left side region, and the right side region separated by the guide frame 17. . Note that the width of each region can be set to a width that is easy for the user to shoot by operating the cross key or the like in the setting mode of the digital camera 10.

  Thereby, the captured image data can be similarly divided into four areas of the upper side, the right side, the left side, and the lower side. In the straight line extraction as described in the sixth embodiment, only the horizontal line is detected for the upper side and the lower side area, and the vertical line is detected for the right side and the left side area. By limiting in this way, the amount of calculation can be greatly reduced.

  FIG. 9A is an operation flowchart of printout cutout processing according to the present embodiment. That is, in this embodiment, first, a straight line extraction region in image data including the image of the search source printout 1 photographed by the digital camera 10 is determined based on the current setting of the guide frame 17 (step S125). . That is, as shown in FIG. 9B, four straight line extraction areas are determined for the captured image data 18 in the vertical and horizontal directions. Then, a straight line is extracted from the determined image data 18U, 18D, 18L, 18R of each straight line extraction region (step S126).

  That is, in the first embodiment, the Hough transform is performed on all the straight lines in the image data, and the pixels that are points on the straight lines are counted. In this embodiment, as shown in FIG. The existence angle range of each side in the extraction region is determined.

  The search range is determined as follows according to the processing region size.

Diagonal angle θ = ± atan (processing area height / processing area width)
Vertical line search range: 90-θ to 90 + θ
Horizontal line search range: 0 to θ, 180-θ to 180
That is, as shown in the operation flowchart of FIG. 11, the width and height of the processing area are compared (step S31). If the width is larger, 0 to atan (processing area height / processing area width) and π− A straight line in the range of atan (processing area height / processing area width) to π is extracted (step S32). If the height of the processing area is larger than the width of the processing area, a straight line in a range from π / 2−atan (processing area width / processing area height) to π / 2 + atan (processing area width / processing area height) is obtained. Extract (step S33).

  After limiting the search area in this way, the closed area is determined by the procedure described in the sixth embodiment (step S127). The determined closed region is presented as a thick line 16 on the display unit 15 of the digital camera 10.

  In addition, when the extraction area is not a desired area due to the influence of noise or the like in the image data (for example, when the ground or the horizontal line is detected), the correction can be performed by the following method.

  (A) A straight line having the second highest luminance of the θ and ρ coordinates in the Hough transform is used.

  (B) A straight line manually selected and extracted by the user is used.

  Of course, as described in the sixth embodiment, the user may partially correct the extracted closed region.

  As described above, according to the second embodiment, the detection range is limited and the amount of calculation is reduced by dividing the shooting screen and guiding the search source printout 1 to enter each area. In addition, speeding up can be realized.

  The input image is not limited to a still image. When a moving image is input, it is possible to perform processing for each frame and display the extraction region sequentially. In addition, the information of a plurality of frames is evaluated together, for example, a straight line extracted in any frame regardless of the luminance value of the θ · ρ coordinate in the Hough transform, and a straight line having a large number of appearances in a predetermined plurality of frames. Can be a candidate.

  Further, the input image is not limited to the search source printout 1 printed out by the printer. As shown in FIG. 12A, the display device 40 displaying the image data is photographed, and the image data displayed on the display device 40 is extracted as a region of interest. As shown in FIG. 16 can also be displayed. In this case, as shown in FIG. 9B, the image data 18U, 18D, 18L, and 18R of each straight line extraction region may be replaced with black image data, and the thick line 16 may be displayed in a more distinguishable manner. .

[Third Embodiment]
Next, a search system to which the region extraction apparatus according to the third embodiment of the present invention is applied will be described.

  The present embodiment is characterized by the printout cutout process in step S12 described above, and the others are the same as those in the first embodiment, and a description thereof will be omitted. This embodiment employs a method of extracting a closed region by edge extraction.

  FIG. 13A shows an operation flowchart of printout cutout processing in the present embodiment.

  That is, an edge is extracted from the image data including the image of the search source printout 1 photographed by the digital camera 10 using a filter (step S121). Next, the detected edges are searched (step S128), and a region is determined from the edge shape (step S129).

  A closed region can also be extracted by such a method.

  Also, the printout cutout process may be performed by a method as shown in FIG.

  That is, first, a line segment in the captured image data is detected (step S12A), and a straight line is detected from the detected line segment (step S12B). Then, a frame composed of the four detected straight lines is estimated (step S12C). That is, the region of interest surrounded by the four sides is searched from the captured image data. If there are a plurality of regions surrounded by such four sides, the region with the largest area may be extracted as the region of interest, or the region of interest may be specified from the rectangular aspect ratio. In rare cases, the search source printout 1 itself is warped in the captured image data, and as a result, the region surrounded by the four sides may not be specified. In this case, it is also effective to perform a process that allows even if some of the four sides are formed of loose arcs. Also included here is a process of extracting an area regarded as the search source printout 1 and then normalizing the image data area by affine transformation or the like.

[Fourth Embodiment]
Next, with reference to FIG. 1, the outline of the search system to which the area extracting apparatus according to the fourth embodiment of the present invention is applied will be described.

  That is, the search system includes a digital camera 10, a storage 20, a printer 30, and a personal computer (PC) 50. Here, the storage 20 is a storage device built in the PC 50 or accessible by communication. Further, the PC 50 is configured to be able to read out image data stored in the memory of the digital camera 10 by being connected to the digital camera 10 by wireless / wired connection or by attaching a memory removed from the digital camera 10. .

  In such a configuration, (1) First, the digital camera 10 captures a subject including an image of the search source printout 1 once printed out by the printer 30. (2) The PC 50 extracts a region corresponding to the image of the search source printout 1 from the obtained photographed image data, and extracts a feature amount of the extracted region. (3) Then, template matching processing is executed with the template stored in the storage 20 according to the extracted feature amount. (4) As a result, the image data corresponding to the matched template is read out as the original image data of the search source printout 1. (5) Thereby, the read original image data can be printed out by the printer 30 again.

Hereinafter, a search system to which the region extraction device according to the fourth embodiment is applied will be described in detail.
FIG. 14A is a block diagram of such a search system, and FIG. 15 is an operation flowchart thereof. That is, in this embodiment, image data captured by the digital camera 10 is stored in the storage 20 that is built in or connected to the PC 50 specified by the user, and the processing shown as the PC side in FIG. It is a case to work. The application software is launched in a state where the PC 50 and the digital camera 10 are connected by wire or wirelessly and the communication state is established. This may be a state where the function is activated by an operation of turning on a switch such as “search mode” set in the digital camera 10 or the like.

  When the application software operates as described above, the printout imaging process is executed on the digital camera 10 side (step S11). As shown in FIG. 14B, this is because the image capturing unit 11 of the digital camera 10 at least places the search source printout 1 that the user wants to print out again on a table or a wall surface. By photographing so that the search source printout 1 is not missing, the obtained photographed image data is stored in the storage unit 19 which is a memory in the digital camera 10 (step S111). The stored captured image data is transferred to the connected PC 50 by wire or wireless (step S112).

  Then, in the PC 50, the region extraction unit 51 realized by application software specifies the image of the search source printout 1 from the transmitted captured image data, and specifies and extracts the portion. The printout cutout process is executed (step S12). This printout cut-out process is as described in the first to third embodiments.

  Next, the feature amount extraction unit 52 realized by application software performs a process of extracting feature amounts from the identified and extracted region of interest (step S13). The clipping process and the feature amount extraction process may be performed on the digital camera 10 side. By doing so, the amount of communication from the digital camera 10 to the PC 50 can be reduced.

  Thereafter, the matching unit 53 realized by application software compares the extracted feature amount data with the feature amount database of the photographed image data configured in the storage 20, and sequentially extracts those having high similarity. The matching process is executed (step S14). That is, based on the calculated feature value data, the matching unit 53 on the PC 50 side compares it with the feature value data group included in each image data of the storage 20 (or comprehensively databased). Pick out the closest one. Selecting a plurality of closest feature quantity candidates by setting is also effective for usability. This feature amount data includes designation information of the original image data for which the feature amount has been calculated, and the candidate image is called up accordingly.

  Thereafter, the image data of the selected original image candidate (or candidate image group) is read from the storage 20 and displayed as an image candidate to be extracted on the display unit 54 which is the display of the PC 50 (step S15). Accept selection. At this time, the image data of the selected original image candidate (or candidate image group) is transferred to the digital camera 10 from the PC 50 as it is or appropriately compressed, and displayed on the display unit 15 of the digital camera 10. It does not matter (step S31).

  Then, according to the selection by the operation of the mouse or the like, the original image data corresponding to the image candidate stored in the storage 20 is sent to the connected printer 30 and printed out again (step S18). That is, the displayed original image candidate is determined by the user's judgment and passed to the printing process, so that the user can easily reprint the printed out image, which is the original purpose. At this time, it is not only simply printed, but depending on the user judgment among a plurality of candidate images selected, it means that “similar to the target original image, but similar images are collected”, and similar image data This also leads to the realization of a function for batch search.

  In the present embodiment, the feature amount database may be created when the captured image data is transferred from the digital camera 10 to the storage 20 via the PC 50. That is, as shown in FIG. 14C, transfer of the captured image data from the digital camera 10 to the PC 50 is started (step S241), and the transferred captured image data is stored in the storage 20 by the PC 50. (Step S242), template data is created from the photographed image data (Step S243). Then, the created template data is stored in the storage 20 in association with the captured image data (step S244).

  As described above, also in the fourth embodiment, as in the first to third embodiments, an image of an original image (image) can be obtained by simply capturing an image of a desired search source printout 1 with the digital camera 10. Data) can be accessed, and an intuitive and user-friendly search method can be provided.

  In addition, it is possible to search not only the original image data itself but also image data having a similar image configuration, and can provide a secondary but novel application. That is, street signs, posters, and the like are photographed in this so-called search mode, and the image data and its feature value data existing in the storage 20 such as a memory attached to the digital camera 10 or a database accessible via communication are used. Similar or identical image data can be easily retrieved. Furthermore, an Internet site associated with the data can be displayed on a display such as a PC 50 or a digital camera, or a specific application (such as voice or moving image) can be activated.

  In the above description, the digital camera 10 is used. However, the present embodiment is not limited to this and may be a scanner.

  In addition, the search source printout 1 actually printed out is photographed by the digital camera 10. For example, even if the display device 40 displaying the image obtained by photographing the search source printout 1 is photographed by the digital camera 10. It can be implemented similarly.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

  For example, the digital camera is not limited to a digital still camera that captures a still image, and includes a digital movie camera that captures a moving image. In addition, a camera-equipped mobile phone / PHS, which is a digital camera having a communication function, and a communication device with an imaging function such as a fixed videophone are also included.

  The scope of application of the present invention is not limited to camera-equipped mobile phones and digital cameras, but is expected to be widely used in systems that generally shoot and store digital images with cameras, such as security systems that authenticate with images. it can.

FIG. 1 is a diagram showing a schematic configuration of a search system to which an area extracting apparatus according to a first embodiment of the present invention is applied. 2A is a block diagram of the search system according to the first embodiment, FIG. 2B is a diagram showing an operation flowchart thereof, and FIG. 2C is a diagram showing a detailed flowchart of printout cutout processing. It is. FIG. 3A is a diagram showing a detailed flowchart of the matching process with the DB, and FIGS. 3B to 3E are diagrams showing flowcharts for explaining a method of creating a feature amount database. FIG. 4A is a diagram illustrating a display screen of the display unit of the digital camera when only one image candidate is displayed, and FIG. 4B is a display screen when nine image candidates are displayed. FIG. FIG. 5 is a diagram for explaining the edge extraction filter. FIG. 6 is a diagram for explaining the Hough transform. FIG. 7A is a diagram illustrating an example of line extraction when there is only one region of interest in the captured image field of view, and FIG. 7B is an example of line extraction when there are a plurality of regions of interest. FIG. FIG. 8 is a view for explaining the guide frame and the upper side region, the lower side region, the left side region, and the right side region partitioned by the guide frame in the second embodiment of the present invention. FIG. 9A is a detailed flowchart of the printout cutout process according to the second embodiment, and FIG. 9B is a diagram for explaining the relationship between the captured image data and the image data of each straight line extraction region. FIG. FIG. 10 is a diagram for explaining the existence angle range of each side in each straight line extraction region. FIG. 11 is a diagram showing a detailed flowchart of processing for extracting a straight line from each straight line extraction region. FIG. 12A is a diagram illustrating a display example of the digital camera at the time of photographing with the display device displaying the image data, and FIG. 12B is a diagram illustrating a display example at the end of the region of interest extraction process. FIG. 13A is a diagram illustrating a detailed flowchart of the printout cutout process according to the third embodiment of the present invention, and FIG. 13B is a detailed flowchart illustrating another example of the printout cutout process. It is. FIG. 14A is a block diagram of the search system to which the region extraction apparatus according to the fourth embodiment of the present invention is applied, and FIG. 14B is a detailed flowchart of the printout imaging process. FIG. 14C is a flowchart illustrating a method for creating a feature database. FIG. 15 is a diagram showing an operation flowchart of the search system of the fourth embodiment.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1 ... Search origin printout, 10 ... Digital camera, 11 ... Imaging part, 12, 51 ... Area extraction part, 13, 52 ... Feature-value extraction part, 14, 53 ... Matching part, 15, 54 ... Display part, 16 ... Thick line, 17 ... guide frame, 18 ... photographed image data, 18U, 18D, 18L, 18R ... image data of straight line extraction region, 19 ... storage unit, 20 ... storage, 21 ... matching server of primary feature and primary information DB, 22 ... secondary feature value matching server and secondary information DB, 2n ... n-order feature value matching server and n-order information DB, 30 ... printer, 50 ... personal computer (PC), 40 ... display device, 151 ... Image candidates, 152 ... "Previous" and "Next" icons, 153 ... "Determination" icons, 154 ... Bold frame.

Claims (15)

An image input means for inputting an image;
Straight line extraction means for extracting a plurality of straight lines from the image input by the image input means;
Area extracting means for extracting a closed area surrounded by a plurality of straight lines extracted by the straight line extracting means;
An area extracting apparatus comprising: The straight line extraction means is:
Edge component extraction means for extracting edge components from the image input by the image input means;
Hough transforming means for extracting a linear component by Hough transform from the edge component extracted by the edge component extracting means;
Having
The area extracting apparatus according to claim 1, wherein: In an image obtained by capturing a subject including an image therein, a side extracting unit that extracts a plurality of sides of the image boundary in the subject;
A region extracting means for extracting a closed region surrounded by a plurality of sides of the image or a plurality of straight lines extending the plurality of sides;
An area extracting apparatus comprising:   4. The area extracting apparatus according to claim 3, wherein the image in the subject is a printed out image.   The area extracting apparatus according to claim 3, wherein the image in the subject is an image displayed on an image display apparatus.   4. The area according to claim 3, further comprising selection means for preferentially selecting a closed area within a predetermined frame provided in the subject image among the closed areas extracted by the area extracting means. Extraction device.   4. The area extracting apparatus according to claim 3, further comprising selection means for preferentially selecting a closed area with little overlap among the closed areas extracted by the area extracting means.   4. The area extracting apparatus according to claim 3, further comprising selection means for preferentially selecting a closed area located in the center of the subject image from the closed areas extracted by the area extracting means.   4. The region extraction apparatus according to claim 3, further comprising a selection unit that preferentially selects a closed region having a large area among the closed regions extracted by the region extraction unit.   4. The region extracting apparatus according to claim 3, further comprising a selection unit that allows an operator to select a desired closed region among the closed regions extracted by the region extracting unit.   4. The method according to claim 3, wherein when the closed area is extracted by the area extracting unit, a side within a predetermined frame provided in the subject image is detected, and a closed area constituted by the side is extracted. Region extraction device.   4. The area extracting apparatus according to claim 3, further comprising area correcting means for allowing the operator to partially correct the area after extracting the closed area by the area extracting means.   4. The area extracting apparatus according to claim 3, wherein in the area extracting means, the area is extracted every time an image is input. An area extraction method for extracting a closed area from an image,
Enter an image,
Extract multiple straight lines from the input image,
Extracting a closed region surrounded by the plurality of extracted straight lines,
A region extraction method characterized by that. An area extraction method for extracting an image included in a subject,
In an image obtained by capturing a subject including an image, a plurality of sides of the image boundary in the subject are extracted,
Extracting a closed region surrounded by a plurality of sides of the extracted image or a plurality of straight lines extending the plurality of sides;
A region extraction method characterized by that.

Images