JP5259315B2 - Image search device, digital camera, image search method, and image search program - Google Patents

Description

  The present invention relates to an image search device, a digital camera, an image search method, and an image search program.

  Recently, the capacity of the image memory has been increased, and it has become available at a very low cost. Even in a memory used for an ordinary digital camera, it is possible to store 1000 sheets or more of the number of sheets that can be recorded, which was conventionally about 100 sheets. The increase in the number of recordable images is a great merit in that it can provide an environment in which shooting can be performed without worrying about memory capacity, but on the contrary, a large number of shot images must be handled during playback. In order to search for a desired image when enjoying playback with a digital camera, conventionally, a method of switching and displaying an index screen made up of, for example, 4 × 4 thumbnail images on the display unit of the digital camera is used. However, as described above, when the number of stored photographed images is from several hundred to 1000 or more, it is too inefficient because it takes too much time to search while looking at such an index screen one by one. Is.

  As one of the solutions, there is a method in which a keyword is assigned when an image is recorded, and a desired image is searched based on the keyword or a photographed image (for example, see Patent Document 1).

  On the other hand, recently, an image technique for analyzing a photographed image itself, detecting that the image is a person or a building, and specifying the person or the building has been remarkably advanced. A so-called similar image search technique for searching for an image having a subject similar to the subject using such image analysis has been developed (for example, see Patent Document 2).

JP 2005-354134 A JP 2001-117828 A

  As described above, recent digital cameras and the like are small in size, but can easily record several hundred or more shot images, so it is desirable to have a function for searching for similar images as described above. However, since a digital camera does not include a large display unit or a convenient operation unit such as a mouse unlike a PC device (Pesal Computer), there are significant restrictions on the display area and operation functions. In addition, the digital camera is used by users of a wide range of age groups, and must be targeted for users who are not familiar with PC devices. The situation is the same as for digital cameras in the case of small image viewers and mobile phones with cameras.

  In other words, in a portable image device such as a digital camera, a simple and easy-to-understand operation suitable for a small device can be realized in order to be able to effectively use a similar image search function. It is necessary to develop an image search function that can realize screen display and its operation procedure.

  The present invention has been made in view of the above, and is an image search apparatus, a digital camera, and an image search that can provide an excellent user interface for similar image search that can present a comfortable search environment to the user when searching for similar images. It is an object to provide a method and an image search program.

  In order to solve the above-described problems and achieve the object, an image search apparatus according to the present invention includes a similar image search unit that searches for an image similar to a reference image specified by a selected search condition, and the similar image An image search device comprising a display unit for displaying a search screen for search by a search unit, an operation detection unit for detecting a user operation on the search screen, and a display control unit for controlling display of the search screen A search control unit that controls the similar image search unit so as to search for a corresponding similar image based on a reference image designated based on a user operation on the search screen and a selected search condition, The display control unit displays an image stored in the image storage unit as a reference image region for designating the reference image in a part of the search screen, and further displays the reference image region. A condition icon that is iconified as a search condition area for selecting the search condition, and an area adjacent to the search condition area as a similar image area. The similar image retrieved by the similar image retrieval unit under the retrieval condition is displayed.

  In the image search device according to the present invention, in the above invention, the display control unit specifies the reference image by a user operation on the reference image area, and selects the search condition by a user operation on the search condition area. In this case, the designated reference image is displayed so as to be superimposed on the selected condition icon.

  Further, in the image search device according to the present invention, in the above invention, the display control unit further sets the previous search condition when a search condition is further added by a user operation after the displayed similar image is displayed. A menu screen for selecting a relationship with the added search condition is displayed on the search screen.

  In the image search device according to the present invention, in the above invention, the display control unit may further display a new reference image under the same search condition by a user operation after displaying the searched similar image. In addition to the previous reference image, a new reference image is also displayed over the condition icon.

  The image search device according to the present invention further includes a touch panel disposed on the display unit that displays the search screen in the above invention, and the operation detection unit detects a user operation on the touch panel. Then, the display control unit switches between the image of the reference image area and the condition icon of the search condition area, which are simultaneously displayed on the search screen, by independent movement according to a user operation on the touch panel. Features.

  In the image search device according to the present invention as set forth in the invention described above, the display control unit simultaneously selects at least two adjacent areas of the reference image area, the search condition area, and the similar image area from the search screen. It is characterized by being displayed.

  In the image search device according to the present invention, in the above invention, the display control unit is selected for searching for the similar image to be displayed in the similar image region and the similar image in the search condition region. A division display showing the relation with the condition icon is performed.

  The image search device according to the present invention is characterized in that, in the above invention, the display control unit displays a menu screen for selecting a use of a search result by the similar image search unit on the search screen. To do.

  In the image search device according to the present invention as set forth in the invention described above, the search control unit prepares a plurality of search conditions displayed by the display control unit, and the search conditions include color, person's face, and shooting time. And at least one of a shooting scene, a shooting composition, and a shooting location.

  In addition, a digital camera according to the present invention includes any one of the image search devices described above.

  The image search method according to the present invention displays a similar image search unit that searches for an image similar to the reference image specified by the selected search condition, and a search screen for search by the similar image search unit. A display unit, an operation detection unit that detects a user operation on the search screen, a display control unit that controls display of the search screen, and a reference image specified based on a user operation on the search screen are selected. And a search control unit that controls the similar image search unit so as to search for a corresponding similar image under a search condition. A part of the search screen at the time of image search The image stored in the image storage unit is displayed as a reference image region for designating the reference image, and the region adjacent to the reference image region is further detected. A condition icon that is iconified as a search condition area for selecting a condition is displayed in a freely selectable manner, and the similar image search is performed using the reference image and the search condition as a similar image area as an area adjacent to the search condition area. A similar image retrieved by the section is displayed.

  The image search program according to the present invention searches an image similar to a reference image specified by a selected search condition in a computer of an image search apparatus having a display unit that displays a search screen for search. A similar image search function, an operation detection function for detecting a user operation on the search screen, a display control function for controlling display of the search screen, and a reference image designated based on a user operation on the search screen; A search control function for controlling the similar image search function so as to search for a corresponding similar image under the selected search condition, and the display control function sets a partial area of the search screen as the reference An image stored in the image storage unit as a reference image area for designating an image is displayed in a freely selectable manner, and an area adjacent to the reference image area is displayed as the search condition. A condition icon iconified as a search condition area for selection is displayed in a freely selectable manner, and an area adjacent to the search condition area is selected as a similar image area by the similar image search function using the reference image and the search condition. The retrieved similar image is displayed.

  According to the present invention, there are provided an image search device, a digital camera, an image search method, and an image search program that can provide an excellent user interface for similar image search that can present a comfortable search environment to the user when searching for similar images. be able to.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a rear view illustrating a schematic configuration example of a digital camera including an image search device according to the present invention. The digital camera 1 according to the first embodiment includes a release operation unit 13a that is arranged on the upper surface of the camera body to instruct photographing timing, a zoom operation unit 13b that is arranged on the rear surface of the camera body, a photographing mode, Various operation buttons such as a mode setting operation unit 13c for setting modes such as a reproduction mode and a search mode during the reproduction mode, and a direction operation unit 13d for instructing the up, down, left, and right directions are provided. The release operation unit 13a, zoom operation unit 13b, mode setting operation unit 13c, direction operation unit 13d, and the like constitute the operation unit 13 (see FIG. 2). The digital camera 1 also includes a display unit 23 that occupies most of the area on the back of the camera body. The display unit 23 is for displaying various setting information including a search screen in addition to the captured image and the live view image, and is realized by a thin display device such as an LCD or an ELD. Although not particularly shown, the digital camera 1 includes an imaging optical system such as a photographing lens on the front side of the camera body, and is configured to be able to form a subject image on an imaging element built in the camera body.

  FIG. 2 is a block diagram illustrating a system configuration example of the digital camera 1 including the image search device. The digital camera 1 according to the first embodiment includes a control unit 11 including a CPU that performs instructions and data transfer to each unit constituting the digital camera 1 and controls the operation of the digital camera 1 in an integrated manner. The digital camera 1 also includes an internal data storage unit 12, a camera unit 15, an imaging processing unit 16, a RAM 17, an image processing unit 18, a compression / decompression unit 19, a recording / playback unit 20, a display drive unit 22, and a similar image search unit 25. Each part of the external IF unit 26 is provided. These units are connected to the control unit 11 by a bus line 30.

  The internal data storage unit 12 is an electrically rewritable nonvolatile memory such as a flash memory. The internal data storage unit 12 operates the digital camera 1 and uses a program for the CPU (for example, an image search program) for realizing various functions of the digital camera 1 or during execution of the program. For example, data such as condition icons to be displayed on a search screen, which will be described later, and other display icons and characters are stored in advance.

  The camera unit 15 is for the digital camera 1 to capture a subject image, and includes a photographing lens, an image sensor, an aperture, a shutter, an AF mechanism, an AF drive circuit, a zoom mechanism, a zoom drive mechanism, and the like. Here, the image pickup device is formed of a two-dimensional solid-state image pickup device such as a CCD or a CMOS sensor, and photoelectrically converts a subject image incident through a photographing optical system including a photographing lens and outputs it as an analog electric signal. In addition to including an image sensor driving circuit, the image processing unit 16 performs analog signal processing such as AGC and CDRS on an analog electric signal output from the image sensor, and then converts the analog electric signal into a digital electric signal using an A / D converter. For output. The imaging unit 27 includes the camera unit 15 and the imaging processing unit 16.

  The RAM 17 is made of, for example, SDRAM, and is a memory used as a temporary storage memory or a working area. The RAM 17 temporarily stores a digital electric signal output from the imaging processing unit 16, image data being processed by the image processing unit 18, and the like. Remember.

  The image processing unit 18 performs various types of image processing such as pixel interpolation processing, color correction processing, and gamma processing on the digital electrical signal output from the imaging processing unit 16, and is suitable for recording and display. Process to convert to data. That is, at the time of recording (shooting), the image processing unit 18 performs image processing for recording on the recorded image, and performs simple image processing for display on the live view image for display. . The image processing unit 18 also performs electronic zoom processing when electronic zoom is instructed.

  The compression / decompression unit 19 records image data based on a well-known JPEG method or the like when recording image data of a captured image in the image storage unit 21 or displaying image data recorded in the image storage unit 21. Perform compression processing and decompression processing.

  The recording / reproducing unit 20 is for writing image data to the image storage unit 21 and reading image data from the image storage unit 21, and a unit corresponding to the type of the image storage unit 21 is used. Here, the image storage unit 21 is, for example, a recording medium that can be attached to and detached from the camera body such as an xD-Picture Card (registered trademark) or a Compact Flash (registered trademark) card, or an HDD. One to 1000 sheets or more are used.

  The display drive unit 22 is a driver that includes an encoder, a D / A converter, and the like, and displays various information on the display unit 23. The similar image search unit 25 is a search engine for searching for an image similar to the reference image specified by the user under the search condition selected by the user. Further, the external IF unit 26 is an interface for connecting the digital camera 1 to an external device such as a personal computer, a printer, or a mobile phone according to a standard such as USB, and includes a USB terminal.

  Here, the control unit 11 of the present embodiment includes functional units such as a search control unit 11a, a display control unit 11b, and an operation detection unit 11c as functions executed by the image search program. The operation detection unit 11 c is a functional unit for detecting a user operation on the search screen displayed on the display unit 23 during image search and a user operation on the operation unit 13. Here, in the present embodiment, the touch panel 14 is provided so as to overlap the display unit 23 that displays the search screen, and the operation detection unit 11c detects the user's operation on the touch panel 14, A user operation detection operation for the search screen is executed. The display control unit 11b is a functional unit for controlling the display of the search screen on the display unit 23 and executing various control processes as described later relating to the display of the search screen. Further, the search control unit 11a functions to control the similar image search unit 25 to search for a corresponding similar image using a reference image specified based on a user operation on the search screen and a selected search condition. Part.

  Next, an outline of display control of the search screen 23a by the display control unit 11b when the search mode is set by the operation of the mode setting operation unit 13c will be described. FIG. 3 is an explanatory diagram schematically showing the relationship between display areas of various information related to the search screen 23a. When the search mode is set to the search mode, the display control unit 11b displays the display content that can be displayed on the search screen 23a as an area divided into three vertical columns of the reference image area Ra, the search condition area Rb, and the search image area Rc. Prepare. Then, the display control unit 11b performs control so that at least two regions are simultaneously displayed on the search screen 23a because of the relationship between the size of the search screen 23a in the digital camera 1 and the appropriate display size of each region Ra, Rb, Rc. .

  First, the uppermost reference image area Ra is an area for displaying a plurality of images a (for example, about 8) stored in the image storage unit 21 in a sequential and freely specified manner. This is to allow the user to specify the reference image Pa used for the search processing by the similar image search unit 25. In the example shown in FIG. 3, the captured image a is displayed in a two-column thumbnail format. "1, 2, 3,. The displayed image a can be moved in the left-right direction within the search screen 23a by a scroll operation. Here, basically, the reference image Pa used for the search is specified when the user touches the desired image in the image a displayed in the search screen 23a. For this reason, each image a displayed on the search screen 23a is set to a size that allows a certain number of images to be displayed at the same time, allows the contents of the image to be read, and does not hinder the touch operation with the user's finger or the like. .

  The search condition area Rb located in the middle stage adjacent to the reference image area Ra is an area for displaying the search conditions used for the search process by the similar image search unit 25. This is to allow the user to select a search condition. In the present embodiment, settings are made so that a plurality of search conditions are displayed as condition icons b each iconized. In the example shown in FIG. 3, for example, a plurality of condition icons b are displayed in one row, and the displayed condition icons b are movable in the left-right direction within the search screen 23a by a scroll operation. Here, the condition icon Pb that defines the search condition used for the search is basically selectable by the user touching a desired icon in the condition icon b displayed in the search screen 23a. Yes. Therefore, each condition icon b displayed on the search screen 23a is set to a size that allows a plurality of icons to be displayed at the same time, allows the contents of the icons to be read, and does not hinder the touch operation with the user's finger or the like. Yes. In the present embodiment, the condition icon b is set slightly larger than the image a.

  Further, the similar image region Rc positioned at the lowest position adjacent to the search condition region Rb is a region for displaying the similar image Pc searched by the similar image search unit 25 as a search result. Similar images searched under the search condition corresponding to the designated reference image Pa and the selected condition icon Pb are displayed. In the example shown in FIG. 3, the searched similar images Pc are displayed in a single-line thumbnail format, and the displayed similar images Pc can be moved in the horizontal direction in the search screen 23a by a scroll operation. Here, the similar image Pc is basically set to be displayed in the same size as the image a and the reference image Pa.

  Here, for example, at the stage of specifying the reference image Pa and selecting the search condition in the image search, the display control unit 11b simultaneously displays the two areas of the reference image area Ra and the search condition area Rb on the search screen 23a. Control to display. Then, after the image search is started, as schematically shown in FIG. 3, if the size of the display unit permits, a part of the reference image area Ra, the search condition area Rb, the similar image area Rc, and the like are displayed on the search screen 23a. It is desirable to be controlled so as to be displayed simultaneously.

  Under such display control of the search screen 23a, a display example of the search screen 23a in the search mode of the present embodiment will be described. FIG. 4 is an explanatory diagram showing a basic display example of the search screen 23a in the search mode. First, FIG. 4A shows a display example of the search screen 23a in a state in which the search mode is entered. The display control unit 11b displays a plurality of images a read from the image storage unit 21 in the reference image area Ra in a two-column thumbnail format, and displays a plurality of condition icons b in the search condition area Rb. . Note that the image number of each image a is omitted. Each condition icon b is also displayed with a condition title t in character notation indicating the meaning of the condition icon b.

  Further, the display control unit 11b displays left and right scroll icons Ia and Ib on both the left and right sides (or one side) of the reference image area Ra and the search condition area Rb in the search screen 23a. When it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Ia, the display control unit 11b scrolls the image a displayed in the reference image area Ra leftward or rightward. Similarly, when it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Ib, the display control unit 11b moves the condition icon b displayed in the search condition region Rb leftward or rightward. Scroll. In addition, a position gauge Ga is displayed below the reference image area Ra. The position gauge Ga indicates the display position of the current selected image with respect to all selected images to be searched.

  Here, a search condition for searching for a similar image will be described. In the present embodiment, the search control unit 11a prepares in advance six types of conditions such as shooting time, color, person's face, shooting scene, shooting location, and shooting composition as a plurality of search conditions. The plurality of search conditions are displayed by the display control unit 11b. The “shooting time” is based on the shooting date and time of the reference image, and is used for searching for the same date or near date and time. An icon that looks like a clock is displayed as the condition icon b. “That time” is displayed as the title t. “Color” is a condition that uses a characteristic or dominant color of the reference image as a condition. An icon that represents a checkered pattern of multiple colors is displayed as the condition icon b, and “that color” is displayed as the condition title t. Is displayed.

  The “person's face” is based on the person in the reference image as a condition. An icon representing a smile is displayed as the condition icon b, and “that person” is displayed as the condition title t. “Shooting scene” is a condition that the reference image is a night view, a sunset image, or the like. The condition icon b displays an icon that represents a mountain, a star, or the moon, and the condition title t Displays “that scene”. “Capturing location” is based on the position where the reference image is captured. The condition icon b displays an icon that looks like a map of Japan, and the condition title t displays “that location”. The “photographing composition” is based on the composition of the reference image, the condition icon b displays an icon that looks like two people lined up, and the condition title t displays “that composition”. .

  Among these search conditions, a condition such as a color or a person's face corresponds to one of the feature amounts of each image stored in the image storage unit 21. The conditions such as the shooting time, shooting scene, shooting location, and shooting composition are included in a plurality of pieces of information given at the time of shooting each image.

  First, the display of the search screen 23a as shown in FIG. 4A allows the user to specify a reference image for search and select search conditions. Here, when the desired image a to be designated as the reference image is not displayed on the search screen 23a, the display image a in the reference image region Ra is moved independently by touching the scroll icon Ia. It is switched by. Similarly, when the desired condition icon b to be selected as the search condition is not displayed on the search screen 23a, the condition icon b in the search condition area Rb becomes independent by touching the scroll icon Ib portion. It is switched by movement. Then, the reference image Pa is designated by touching the desired image a portion in the reference image area Ra with a finger or the like. The search condition Pb is selected by touching a desired condition icon b in the search condition region Rb with a finger or the like.

  Here, in the present embodiment, when the desired image a is designated as the reference image Pa, as shown in FIG. 4B, the condition icon b portion of the search condition Pb in which the designated reference image Pa is selected as it is. Up to this point, the reference image Pa and the search condition Pb are selected by performing the drag and drop operation. When the reference image Pa is specified and the search condition Pb is selected in this way, the search control unit 11a uses the specified reference image Pa and the selected search condition Pb to search for similar images by the similar image search unit 25. Start the search process.

  At this time, when the operation detection unit 11c detects that the drag and drop operation has been performed through the touch panel 14, the display control unit 11b is designated by the user's operation as shown in FIG. 4C. The reference image Pa is displayed over the condition icon b of the selected search condition Pb. In other words, the display of the condition icon b is replaced with the display of the reference image Pa by pasting the designated reference image Pa on the selected condition icon b in an overwritten state. Thereby, the search condition and the reference image currently used for the search are specifically shown to the user.

  Then, when the similar image search process by the similar image search unit 25 proceeds, the display control unit 11b sets the similar image Pc obtained as a result of the search to the similar image region Rc by 1 as shown in FIG. Display in column thumbnail format. Here, the search screen 23a is displayed so as to be shifted downward so that the similar image region Rc appears. Also, left and right scroll icons Ic are displayed on both the left and right sides (or one side) in the search screen 23a for the similar image region Rc. When it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Ic, the display control unit 11b scrolls the similar image Pc displayed in the similar image region Rc leftward or rightward. .

  Further, the display control unit 11b displays the position gauge Gc below the similar image region Rc. The position gauge Gc indicates the relative position of the currently displayed similar image with respect to all similar images as search results. Further, the display control unit 11b performs segmented display by the dividing line L. The dividing line L has a predetermined pattern shape so that the similar image region Rc and a part of the search condition region Rb are integrated. The display of the dividing line L clearly shows the relationship between the searched similar image Pc and the condition icon b (replaced reference image Pa) which is the search condition.

  In the example shown in FIG. 4C, the retrieved similar images Pc are displayed in chronological order according to, for example, shooting date / time, but the present invention is not limited to this. For example, the search results may be displayed in order of similarity with the similarity to the reference image Pa, and the display between time series order and similarity order may be switched.

  Subsequently, an example of operation and a display example after FIG. First, FIG. 5 is an explanatory diagram showing a display example when a new reference image is specified by the user under the same search condition after the similar image Pc searched as shown in FIG. 4C is displayed. . First, FIG. 5A shows the same search as the previous time in the search condition area Rb when the user designates the second reference image Pa2 in the display state of the search screen 23a as shown in FIG. 4C. A case where a drag and drop operation is performed on the condition part is shown. When such an operation is performed, the search control unit 11a starts the similar image search process by the similar image search unit 25 according to the current reference image Pa2 and the selected search condition.

  In parallel with such search processing, the display control unit 11b searches the menu screen M1 for selecting the relationship between the previous reference image Pa1 and the current reference image Pa2, as shown in FIG. 5B. It is displayed in the screen 23a. This menu screen M1 performs “new search” for performing a search unrelated to the previous reference image Pa1, “OR search” for ORing both reference images Pa1 and Pa2, and AND of both reference images Pa1 and Pa2. “AND search” to be taken is displayed freely. FIG. 5B shows a state where “OR search” is selected, for example.

  When the operation detection unit 11c detects that “OR search” is selected by the user's operation on the menu screen M1, the display control unit 11b displays the previous time as shown in FIG. In addition to the reference image Pa1, the new reference image Pa2 of this time is displayed over the selected condition icon b, and a character (here, OR) indicating the relationship between the two is displayed.

  Then, when the similar image search process by the similar image search unit 25 proceeds, the display control unit 11b performs both reference images Pa1 as shown in FIG. 5C in the same manner as in FIG. , Pa2 and the similar image Pc obtained as a search result by OR is displayed in the similar image region Rc in the form of a single thumbnail.

  Here, as shown in FIG. 5C, when the operation detecting unit 11c detects that the user has single-touched a desired similar image Pc portion of the similar image region Rc through the touch panel 14, the display control unit 11b. As shown in FIG. 5D, the touch-designated similar image Pc is enlarged and displayed over the entire search screen 23a. At this time, the display control unit 11b also displays a scroll icon Id on both the left and right sides of the search screen 23a. When it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Id, the display control unit 11b enlarges and displays the similar images Pc adjacent to the left and right in the similar image region Rc one by one. . On the other hand, when a portion other than the scroll icon Id is touched, the display control unit 11b returns to the original thumbnail display shown in FIG.

  FIG. 6 is an explanatory diagram showing a display example when a search condition is further added by the user after the similar image Pc searched as shown in FIG. 4C is displayed. First, in FIG. 6A, in the display of the search screen 23a as shown in FIG. 4C, the scroll icon Ib of the search condition area Rb is operated by the user, and the condition icon b is scrolled and changed. It shows how it is.

  FIG. 6B shows a state in which the scroll icons Ia and Ib in the reference image area Ra and the search condition area Rb are respectively operated by the user, and the image a and the condition icon b are changed by scroll movement.

  FIG. 6C shows a state where a new reference image Pa2 is designated from the reference image area Ra and a new search condition (condition icon) Pb2 is selected from the search condition area Rb. Here, in the present embodiment, as in the case described above, when the desired image a is designated as the reference image Pa2, the designated reference image Pa2 is dragged to the condition icon portion of the selected search condition Pb2. By performing the drop operation, the reference image Pa2 is designated and the search condition Pb2 is selected. When the reference image Pa2 is specified and the search condition Pb2 is selected in this way, the search control unit 11a uses the specified reference image Pa2 and the selected search condition Pb2 to search for similar images by the similar image search unit 25. Start the search process.

  In parallel with such search processing, the display control unit 11b searches the menu screen M1 for selecting the relationship between the previous search condition Pb1 and the current search condition Pb2, as shown in FIG. It is displayed in the screen 23a. This menu screen M1 displays “new search” for performing a search unrelated to the previous search condition Pb1, “OR search” for ORing the previous and current search conditions, and AND of the previous and current search conditions. “AND search” to be taken is displayed freely. FIG. 6D shows a state where “AND search” is selected, for example.

  When the operation detection unit 11c detects that “AND search” is selected by the user's operation on the menu screen M1, the search control unit 11a performs a search process that performs an AND operation on the previous and current search conditions. Is executed. Then, when the AND search process of similar images by the similar image search unit 25 proceeds, the display control unit 11b, as shown in FIG. 6E, displays the similar image Pc obtained as a search result by AND of both search conditions. Are displayed in the similar image region Rc in a single-line thumbnail format. Further, the display control unit 11b performs segmented display by the dividing line L. By displaying the dividing line L, the similar image Pc to be displayed in the similar image region Rc and the search conditions Pb1 and Pb2 selected for searching for the similar image Pc in the search condition region Rb (replaced reference images Pa1, The relationship of Pa2) is clearly shown.

  Further, FIG. 7 is an explanatory diagram showing a display example when a new search is further selected by the user after the similar image Pc searched as shown in FIG. 4C is displayed. First, FIG. 7A shows a state before specifying a new reference image and selecting a search condition in the display of the search screen 23a as shown in FIG. 4C.

  FIG. 7B shows a state where the scroll icons Ia and Ib in the reference image area Ra and the search condition area Rb are respectively operated by the user, and the image a and the condition icon b are changed by scroll movement. Furthermore, a new reference image Pa2 is designated from the reference image area Ra, and a new search condition (condition icon) Pb2 is selected from the search condition area Rb. Here, in the present embodiment, as in the case described above, when the desired image a is designated as the reference image Pa2, the designated reference image Pa2 is dragged to the condition icon portion of the selected search condition Pb as it is. By performing the drop operation, the reference image Pa2 is designated and the search condition Pb2 is selected. When the reference image Pa2 is specified and the search condition Pb2 is selected in this way, the search control unit 11a uses the specified reference image Pa2 and the selected search condition Pb2 to search for similar images by the similar image search unit 25. Start the search process.

  In parallel with such search processing, the display control unit 11b searches the menu screen M1 for selecting the relationship between the previous search condition Pb1 and the current search condition Pb2, as shown in FIG. 7C. It is displayed in the screen 23a. This menu screen M1 includes a “new search” for performing a search unrelated to the previous search condition Pb1, an “OR search” for ORing both search conditions, and an “AND search” for ANDing both search conditions. Is displayed freely. FIG. 7C shows a state where, for example, “new search” is selected.

  When the operation detection unit 11c detects that “new search” has been selected by the user's operation on the menu screen M1, the search processing based on the current search condition Pb2 is performed. Then, when the similar image search process by the similar image search unit 25 proceeds, the display control unit 11b displays the similar image Pc2 obtained as the search result based on the search condition Pb2 as shown in FIG. 7D. The area Rc is displayed in a single-line thumbnail format. Further, the display control unit 11b performs segmented display by the dividing line L. The relationship between the similar image Pc2 displayed in the similar image region Rc by the display of the dividing line L and the search condition Pb2 (replaced reference image Pa2) selected for searching for the similar image Pc in the search condition region Rb Is clearly shown.

  FIG. 8 is an explanatory diagram showing a display example of a search result use menu. When the operation detection unit 11c detects that the user has operated the direction operation unit 13d in a predetermined direction on the search screen 23a on which the search result as shown in FIG. 4C is displayed, the display control unit 11b As shown in FIG. 8, a menu screen M2 for selecting the usage of the search result is displayed on the search screen 23a. As options on the menu screen M2, five types of “Save”, “Print”, “Delete”, “Transfer”, and “Reset” are prepared. When “Save” is selected by touch, the search result is stored in the image storage unit 21, for example. When “print” is selected by touch, the search result is printed by, for example, a printer connected to the external IF unit 26. When “Delete” is touch-selected, the search result is deleted. When “transfer” is selected by touching, it is transferred to, for example, a PC or a mobile phone connected to the external IF unit 26. When “reset” is touch-selected, the displayed search result is reset. Here, the search result data includes the search condition and the similar image searched or the date and time of search.

  In the above description, two regions are simultaneously displayed on the search screen 23a because of the relationship between the size of the search screen 23a of the digital camera 1 which is a small portable device and the appropriate display size of each region Ra, Rb, Rc. If there is no particular problem with the size of the search screen 23a and the appropriate display size of each region Ra, Rb, Rc, as shown in FIG. 9, three regions Ra, Rb, Rc are displayed on the search screen 23a. Of course, they may be displayed simultaneously.

  Next, an example of operation control by the control unit 11 for realizing the display control as described above will be described. FIG. 10 is a main flowchart illustrating an example of operation control of search processing executed by the control unit 11. The similar image search process targeted in this embodiment is executed when the search mode in the playback mode is designated by the mode setting operation unit 13b. First, in the playback mode, it is determined whether or not the search mode is designated (step S1). If the search mode is not designated (step S1: No), normal reproduction processing is executed (step S2). On the other hand, if the search mode is designated (step S1: Yes), search mode processing is executed (step S3). In any process, the process is repeated until a predetermined OFF instruction is given (step S4: Yes).

  FIG. 11 is a flowchart showing an example of operation control in the search mode of step S3 executed by the control unit 11. First, when the search mode is designated, the search screen 23a is displayed on the display unit 23 by the display control unit 11b (step S11). The search screen 23a in this case is, for example, as shown in FIG.

  Next, it is determined from the detection result of the operation detection unit 11c whether or not the scroll instruction Ia, Ib in the reference image area Ra or the search condition area Rb has been instructed by the user's touch operation (step S12). When there is a scroll instruction (step S12: Yes), the display control unit 11b executes a scroll process on the search screen 23a (step S13). That is, in the case of a scroll instruction by the scroll icon Ia, the image a is scrolled independently, and in the case of the scroll instruction by the scroll icon Ib, the condition icon b is scrolled independently.

  And it is determined by the detection process of the operation detection part 11c whether the reference | standard image Pa was designated and the condition icon Pb was selected (step S14). Here, for example, as shown in FIG. 4B and the like, the designated reference image Pa is dragged and dropped with respect to the selected condition icon Pb, whereby the reference image Pa and the search condition (condition icon) are displayed. ) It is determined whether or not Pb is confirmed. If the reference image Pa is not specified and the condition icon Pb is not selected (step S14: No), it is determined whether or not there is an instruction to stop (step S15). (S15: No), the process returns to step S12 and waits for a scroll instruction, a reference image designation, or the like.

  When there is a drag and drop operation that means the designation of the reference image Pa and the selection of the condition icon (search condition) Pb (step S14: Yes), the display control unit 11b executes the display process of the search screen 23a. (Step S16). This display process is, for example, a process of changing the display by overwriting and pasting the reference image Pa that has been dragged and dropped with respect to the condition icon Pb as shown in FIG. Then, the search control unit 11a causes the similar image search unit 25 to execute a search process (step S17).

  FIG. 12 is a flowchart illustrating an example of a search execution process executed by the similar image search unit 25. This process may be performed in accordance with, for example, the process disclosed in Japanese Patent Application Laid-Open No. 2001-117928, and an outline is shown here. First, a search condition is set by the selected condition icon b (step S41). Next, the designated reference image Pa is read from the image storage unit 21 (step S42). Then, the feature amount of the reference image Pa is calculated according to the set search condition Pb, and a feature vector is created (step S43).

  On the other hand, a large number of images a stored in the image storage unit 21 are sequentially read out one by one as comparison images (step S44). Then, the feature amount of the comparison image is calculated according to the search condition, and a feature vector is created (step S45). Next, the feature vectors of the reference image Pa and the comparison image a are compared (step S46), and it is determined whether or not the similarity is a certain level or more as a result of comparison (step S47). If the degree of similarity is equal to or greater than a certain value (step S47: Yes), a similar setting is made to indicate that the comparison image a is a similar image (step S48). If the degree of similarity is not equal to or greater than a certain value (step S47: No), the comparison is performed. The dissimilarity setting that the image a is a dissimilar image is performed (step S49). If there is an instruction to cancel (step S50: Yes), the search execution process is canceled halfway and returns, but if there is no instruction to cancel (step S50: No), the last stored in the image storage unit 21. Until the image a becomes a comparative image (step S51: Yes), the processing of steps S44 to S49 is repeatedly executed.

  Returning to FIG. 11, when the search execution process is completed, it is determined whether or not this process is the first time (step S18). If it is the first time (step S18: Yes), the display control unit 11b displays the similar image Pc obtained as a search result in the similar image region Rc of the search screen 23a (step S19). Here, the displayed similar image Pc is a comparative image that is set to be similar in the process of step S48, and is displayed as shown in FIG. 4C.

  On the other hand, when the search execution process is not the first time but the second time or later (step S18: No), the display control unit 11b determines whether the search control unit 11b is shown in FIG. 5B, FIG. 6D, or FIG. ), The menu screen M1 is displayed on the search screen 23a (step S20). Then, it is determined from the detection result of the operation detection unit 11c which process is touched by the user with respect to the display of the menu screen M1 (step S21). Here, when an OR search instruction is given as shown in FIG. 5B, an OR search process is executed as shown in FIG. 5C (step S22). If an AND search instruction is issued as shown in FIG. 6D, an AND search process is executed as shown in FIG. 6E (step S23). Further, when there is a new search instruction as shown in FIG. 7C, a process for resetting the previous result is executed as shown in FIG. 7D (step S24). In any case, the display control unit 11b displays the search result in the similar image region Rc of the search screen 23a (step S19).

  Thereafter, it is determined whether or not there is an instruction to display a search result use menu accompanying the operation of the direction operation unit 13d by the user (step S25). When there is an instruction to display the usage menu (step S25: Yes), the display control unit 11b displays the menu screen M2 on the search screen 23a as shown in FIG. 8 (step S26). If the use item is touched on the menu screen M2 (step S27: Yes), a process corresponding to the specified use item is executed (step S28). Details of the processing of each usage item are omitted.

  Further, when there is no instruction to display such usage menu (step S25: No), or when there is no usage item designation (step S27: No), or when the processing of each usage item is completed, FIG. As schematically shown in (c), it is determined based on the detection result of the operation detection unit 11c whether or not the user has given a single touch instruction to the similar image region Rc (step S29). In particular, when there is no single touch instruction (step S29: No), the process returns to step S12.

  On the other hand, if there is a single touch instruction (step S29: Yes), it is determined whether or not the instructed similar image Pc is currently being enlarged and displayed (step S30). When the enlarged display is not currently being performed (step S30: No), the display control unit 11b enlarges and displays the similar image Pc for which the single touch instruction has been given, on the entire search screen 23a (step S31). That is, the display shown in FIG. 5C is changed to the display shown in FIG. If the enlarged display is currently being performed (step S30: Yes), the display control unit 11b executes a process for returning to the display of the normal search screen 23a (step S32). That is, the display as shown in FIG. 5D is returned to the display as shown in FIG.

  As described above, according to the first embodiment, the search screen 23a can be selected from the reference image area Ra for designating the reference image from the images stored in the image storage unit 21 and the plurality of search conditions. The search condition region Rb for selecting the search condition and the similar image region Rc for displaying the similar image as the search result are displayed adjacent to each other, so that a search environment comfortable to the user when searching for the similar image is provided. An excellent user interface for similar image search that can be presented can be provided. In particular, since the search condition is displayed as an icon as the condition icon b, the contents of the search condition can be easily understood even in a smaller display area than when only the characters are used. A display form suitable for the display unit 23 is obtained. Furthermore, by pasting the designated reference image Pa on the selected condition icon Pb, the display of the condition icon Pb is replaced with the display of the reference image Pa. Therefore, the search condition is determined by the condition icon b. It is not a general thing, but can be specifically shown to the user by an image using the reference image Pa.

  Further, according to the first embodiment, when a search condition is further added by a user operation after the displayed similar image is displayed, the relationship between the previous search condition and the added search condition is selected. The menu screen M1 is displayed on the search screen 23a, so that a plurality of searches with different search conditions can be easily performed. This is convenient when the number of similar images cannot be narrowed down by a single search. It becomes a high quality thing.

  Further, according to the first embodiment, if a new reference image is specified under the same search condition by a user operation after the displayed similar image is displayed, a new reference image is added in addition to the previous reference image. In addition, when the search is performed a plurality of times, two reference images are displayed, and the user can easily understand the relationship between the previous and subsequent searches.

  Furthermore, according to the first embodiment, the operation detection unit 11c detects a user operation on the touch panel 14 disposed on the display unit 23 that displays the search screen 23a, and the user operation on the touch panel 14 is detected. Accordingly, since the image a of the reference image area Ra and the condition icon b of the search condition area Rb displayed simultaneously on the search screen 23a are switched by independent movements, an operation for searching by an intuitive operation is performed. Similar searches can be easily performed even by an unskilled beginner.

  Further, according to the first embodiment, at least two adjacent areas of the reference image area Ra, the search condition area Rb, and the similar image area Rc are displayed on the search screen 23a at the same time. It becomes easy. Furthermore, according to the first embodiment, the relationship between the similar image Pc displayed in the similar image region Rc and the condition icon Pb selected for searching for the similar image Pc in the search condition region Rb is shown. Since the segmented display is performed, the relationship between the selected search condition and the similar image as the search result becomes clear.

  Furthermore, according to the first embodiment, the menu screen M2 for selecting the usage of the search result by the similar image search unit 25 is displayed on the search screen 23a, so that only a similar image is found. In addition, it is possible to easily use the found similar images afterwards.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. The same parts as those shown in Embodiment Mode 1 are denoted by the same reference numerals, and description thereof is also omitted. The second embodiment also shows an application example to the digital camera 1 configured as shown in FIGS. The second embodiment is characterized in that a reference image for search is designated by a live view image. That is, based on the imaging function of the digital camera 1, the subject image formed on the imaging element in the camera unit 15 is displayed on the display unit 23 in real time as a live view image. This live view function is used for searching.

  That is, the imaging unit 27 in the second embodiment outputs a live view image to be displayed on the display unit 23 even in the search mode in addition to outputting the live view image to the display unit 23 in the shooting mode. It is controlled as follows. In addition, the display control unit 11b according to the second embodiment schematically designates a reference image for search from a live view image and selects at least one search condition from a plurality of search conditions prepared in advance. A search screen for displaying on the display unit 23. In addition, the operation detection unit 11 c in the second embodiment has a function of detecting a user operation on the search screen (touch panel) or the operation unit 13 of the digital camera 1. Furthermore, the search control unit 11a according to the second embodiment corresponds to the search condition selected based on the user's operation on the search screen and the reference image specified based on the user's operation during the live view image display. The similar image search unit 25 is controlled to search for similar images.

  Here, an outline of a display control example of the search screen 23b by the display control unit 11b of the second embodiment when the search mode is set by the mode setting operation unit 13c will be described. FIG. 13 is an explanatory diagram schematically illustrating a display control example of the search screen 23b. FIG. 13 is a diagram of a specific subject in front and the digital camera 1 displaying the subject in the search mode as viewed from the back. When the search mode, which is one of the playback modes, is set, in the second embodiment, the imaging unit 27 is activated, and a live view image obtained by imaging the subject 31 is output from the imaging unit 27. Therefore, the display control unit 11b according to the second embodiment includes the live view image Pr output from the imaging unit 27 as display contents to be displayed on the search screen 23b when the search mode is set. This live view image Pr occupies most of the central portion of the search screen 23b according to the shooting range frame 32 and is displayed so that it can be easily seen.

  In addition, a condition icon Pb showing the search condition as an icon is displayed at one corner of the shooting range frame 32, for example, at the upper left, and a condition title t written in text indicating the meaning of the condition icon Pb is also displayed on the right side. Display. Here, also in the second embodiment, as in the case of the first embodiment, for example, six types of shooting time, color, human face, shooting scene, shooting location, and shooting composition are prepared in advance as search conditions. Condition icons are provided. In correspondence with the number of such search conditions, the condition icons are virtualized in one column in the left-right direction, and the condition icon Pb is moved leftward or left by touching the scroll icons Ib on the left and right sides of the search screen 23b. The search condition can be switched by scrolling to the right.

  For example, in the example shown in FIG. 13, the shooting range frame 32 of the condition icon Pb11 “this color” is displayed in the center, and a part of the shooting range frame 32 of the condition icon Pb12 “this person” is displayed on the right side. It shows how it is. Therefore, for example, when the left scroll icon Ib is touched, the condition icon Pb12 “this person” is animated and displayed in the center, and the search condition is switched to the condition icon Pb12 “this person”. At this time, the display of the live view image Pr itself does not change.

  In the case of the second embodiment, when searching for similar images, a subject 31 that can be a reference image having information on a desired search condition is searched for, and the subject 31 is captured and displayed on the search screen 23b as a live view image, and the search-related conditions are set. It will be set. Here, in the second embodiment, the live view image is designated as the reference image based on the release operation of the release operation unit 13a by the user, and the similar image search process is executed as in the case of the original photographing. Control. The release operation unit 13a includes a first stage release operation (1st release) and a second stage release operation (2nd release) as release operations. Therefore, the search control unit 11a starts the similar image search process using the live view image as a temporary reference image in the first release operation, and fixes the live view image as the reference image by performing the second release operation. Control to execute the search process.

  Under such an outline, a display control example of the second embodiment accompanied by a release operation will be described with reference to FIG. In FIG. 14A, the target portion of the subject image is too small in the display state of the live view image Pr as shown in FIG. 13, and therefore the target subject image in the live view image Pr is displayed by the operation of the zoom operation unit 13b. A state is shown in which the release operation unit 13a is operated for the first release while zooming up.

  When such a 1st release operation is detected by the operation detection unit 11c, the search control unit 11a designates the live view image Pr displayed on the search screen 23b as the temporary reference image Pa at the timing of the 1st release operation. In addition, the similar image search unit 25 starts a similar image search operation under the search condition of the selected condition icon Pb11. At this time, the display control unit 11b superimposes and displays the search portion specifying mark Pm indicating the portion used for the selected search condition (condition icon Pb11) in the live view image Pr on the live view image Pr. Let Here, the search portion is automatically specified in the subject according to the selected search condition, and the search portion specifying mark Pm is displayed at the specified position. For example, if “this color” is selected as the search condition, the main color in the screen is selected, and the search portion specifying mark Pm is displayed in the area of the main color. If “this person” is selected as the search condition, the face of the person in the screen is selected, and the search portion identification mark Pm is displayed on the face portion.

  The example shown in FIG. 14A is an image zoomed up from the state of FIG. This is because the apple region at the center of the live view image Pr is enlarged to designate the apple color as the main color. When the color of the apple is selected, a circular search portion identification mark Pm is displayed at the center, and it can be seen that “this color” in the search condition Pb11 indicates the red color of the apple. As a search portion, a portion close to the center of the screen or a portion having a large relative size in the screen is designated preferentially.

  In the search screen 23b, a similar image region Rc for displaying a similar image Pc obtained as a search result is prepared below the display portion of the live view image Pr. Therefore, the display control unit 11b displays the similar image Pc obtained together with the start of the search operation in the similar image region Rc together with the live view image Pr on the search screen 23b. The similar images Pc are displayed, for example, in the searched order (in chronological order). FIG. 14A shows a state in which the display of the similar image Pc in the similar image region Rc is in the middle. This is because if there are many images to be searched, it takes time to complete the search for all the images, and similar images cannot be displayed instantaneously. Further, left and right scroll icons Ic are displayed on both the left and right sides (or one side) in the search screen 23b for the similar image region Rc. When it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Ic, the display control unit 11b scrolls the similar image Pc displayed in the similar image region Rc leftward or rightward. .

  Incidentally, while the operation detecting unit 11c detects that the first release operation as shown in FIG. 14A is continued, the reference image is only temporarily specified, and the image of the digital camera 1 is displayed. The live view image Pr changes as the corner changes. When the live view image Pr changes in this way, the search control unit 11a changes (updates) the temporary reference image Pa according to the change, and similarly searches for a new similar image by the similar image search unit 25. Let it be done.

  Next, FIG. 14B further zooms up the target subject image in the live view image Pr from the state shown in FIG. 14A by the zoom operation of the zoom operation unit 13b, and the release operation unit 13a. Indicates a state in which the 2nd release operation is performed. In the example illustrated in FIG. 14B, the apple of the subject 31 is zoomed up as the life view image Pr, and the entire shooting range frame 32 is a red image. When the operation detection unit 11c detects that the 2nd release operation has been performed, the search control unit 11a fixes the live view image Pr displayed at the timing of the 2nd release operation as the reference image Pa, and searches for similar images. The part 25 searches for the final similar image. In addition, the display control unit 11b displays the similar image Pc obtained as a search result in a single row in the similar image region Rc on the search screen 23b. The similar images Pc are displayed, for example, in the searched order (in chronological order).

  FIG. 14C illustrates a case where the display order of the similar images Pc in the similar image region Rc is switched in the order of similarity by operating the direction operation unit 13d in a predetermined direction in the display state illustrated in FIG. Is shown. When the operation detection unit 11c detects that the direction operation unit 13d in a predetermined direction is operated after the reference image Pc is fixed and the detection result of the similar image Pc is obtained, the display control unit 11b detects the similar image. The display order of the similar images Pc in the region Rc is switched from the time series order to the similarity order. When the direction operation unit 13d in the predetermined direction is operated again, the display order of the similar images Pc in the similar image region Rc is switched again from the similarity order to the time series.

  Further, after the reference image Pa is designated (fixed) from the live view image Pr by the 2nd release operation, the display control unit 11b stores the reference image Pa in the RAM 17, and FIG. 14 (b) and FIG. As shown in (c), the reference image Pa stored in the RAM 17 is fixedly displayed instead of the live view image.

  FIG. 14D is an explanatory diagram showing a display example of a search result use menu. When the operation detection unit 11c detects that the user has operated the direction operation unit 13d in a predetermined direction in the state of the search screen 23b on which the search result as illustrated in FIG. 14C is displayed, the display control unit 11b displays a menu screen M2 for selecting the usage of the search result on the search screen 23b as shown in FIG. 14 (d). This is the same as the case shown in FIG.

  In FIG. 14B, the reference image Pa fixed by the 2nd release operation is displayed in the size of the live view image Pr even after the similar image search is completed. However, the reference image does not change. After the completion, as shown in FIG. 15A, the size of the reference image Pa is automatically reduced and the display area of the similar image area Rc is automatically increased by the control by the display control unit 11b. You may make it display the similar image Pc simultaneously.

  Further, as shown in FIG. 15A, when the operation detecting unit 11c detects that the user has single-touched the desired similar image Pc portion of the similar image region Rc through the touch panel 14, the display control unit 11b As shown in FIG. 15B, the touch-designated similar image Pc is enlarged and displayed over the entire search screen 23b. When any part of the enlarged similar image Pc is touched as shown in FIG. 15B, the display control unit 11b returns to the original thumbnail display shown in FIG. Such an enlarged display is the same in the case of the display shown in FIG. 14B or 14C.

  Next, an example of display control when changing to another search condition will be described with reference to FIG. FIG. 16A shows a display example when the subject 31 that can be the reference image is changed and the search condition is changed from the condition icon Pb11 to the condition icon Pb12 by operating the scroll icon Ib, compared to the case of FIG. It is explanatory drawing shown. At this time, the display control unit 11b superimposes and displays the search portion specifying mark Pm indicating the portion used for the selected search condition (condition icon Pb12) in the live view image Pr on the live view image Pr. Let

  In FIG. 16B, the target portion of the subject image is too small in the display state of the live view image Pr as shown in FIG. 16A, and therefore the purpose in the live view image Pr is determined by the operation of the zoom operation unit 13b. This shows a state in which the release operation unit 13a is being operated for the first release while zooming up the target portion of the subject image.

  When such a 1st release operation is detected by the operation detection unit 11c, the search control unit 11a designates the live view image Pr displayed on the search screen 23b as the temporary reference image Pa at the timing of the 1st release operation. In addition, the similar image search unit 25 performs a similar image search operation under the search condition of the selected condition icon Pb12. At this time, the display control unit 11b also zooms up the search portion specifying mark Pm in accordance with the subject image.

  Further, the display control unit 11b displays the similar image Pc obtained together with the start of the search operation in the similar image region Rc together with the live view image Pr on the search screen 23b. The similar images Pc are displayed, for example, in the searched order (in chronological order). Further, left and right scroll icons Ic are displayed on both the left and right sides (or one side) in the search screen 23b for the similar image region Rc. When it is detected through the touch panel 14 and the operation detection unit 11c that the user touches the scroll icon Ic, the display control unit 11b scrolls the similar image Pc displayed in the similar image region Rc leftward or rightward. .

  Subsequently, in FIG. 16C, the target portion of the subject image in the live view image Pr is further zoomed up by the zoom operation of the zoom operation unit 13b from the state shown in FIG. 16B, and the release operation unit 13a is 2nd. The release operation state is shown. When the operation detection unit 11c detects such a 2nd release operation, the search control unit 11a fixes the live view image Pr displayed at the timing of the 2nd release operation as the reference image Pa, and displays a similar image. The search unit 25 searches for a final similar image. In addition, the display control unit 11b displays the similar image Pc obtained as a search result in a single row in the similar image region Rc on the search screen 23b. The similar images Pc are displayed, for example, in the searched order (in chronological order). In this case, as shown in FIG. 15A, the size of the reference image Pa may be reduced and the similar image region Rc may be increased.

  Next, an example of operation control by the control unit 11 of the second embodiment for realizing the display control as described above will be described. FIG. 17 is a main flowchart illustrating an example of operation control of search processing executed by the control unit 11. The image search process targeted in this embodiment is executed by specifying a search mode in the playback mode by the mode setting operation unit 13b. First, in the playback mode, the search mode is specified. It is determined whether or not there is (step S101). If the search mode is not designated (step S101: No), normal reproduction processing is executed (step S102). On the other hand, if the search mode is designated (step S101: Yes), search mode processing is executed (step S103). In any process, the process is repeated until a predetermined OFF instruction is given (step S104: Yes).

  18 and 19 are flowcharts illustrating an example of operation control in the search mode in step S103 executed by the control unit 11. First, when the search mode is designated, the imaging unit 27 that is not originally used in the reproduction mode is activated (step S111). This is for displaying a live view image for designating a reference image. Then, the search screen 23b is displayed on the display unit 23 by the display control unit 11b (step S112). The search screen 23b in this case is as shown in FIG. 13, for example, and is in a live view image display state.

  Next, it is determined from the detection result of the operation detection unit 11c whether or not there is an instruction to scroll the condition icon Pb by the user's touch operation on the scroll icon Ib (step S113). When there is a scroll instruction (step S113: Yes), the display control unit 11b executes scroll processing on the search screen 23b (step S114). That is, the condition icon Pb is scrolled to the left or right. As a result, the search condition is changed in accordance with the condition icon Pb positioned in the shooting range frame 32 where the live view image Pr is displayed (step S115).

  Whether or not the zoom operation unit 13b has been pressed in such a live view image Pr display state is determined based on the detection result of the operation detection unit 11c (step S116). When the zoom operation unit 13b is pressed (step S116: Yes), zoom switching is performed so that the size of the live view image Pr changes according to the operation amount (step S117). When the zoom operation unit 13b is not pressed (step S116: No), the process of step S117 is jumped.

  Further, in the live view image Pr display state, it is determined from the detection result of the operation detection unit 11c whether or not the release operation unit 13a has been operated for the first release (step S118). If the first release operation has not been performed (step S118: No), it is determined whether or not there is an instruction to stop (step S115). If there is no instruction to stop (step S115: No), the process proceeds to step S113. Returning, the display state of the live view image Pr is continued.

  On the other hand, when the first release operation is performed (step S118: Yes), the search control unit 11a uses the live view image Pr displayed at the timing of the first release operation as the temporary reference image Pa, and the similar image search unit 25. To execute a similar image search process (step S119). The search execution process in this case is the same as in FIG. When this search process is started, display of the search result is also started as shown in FIG.

  Further, it is determined from the detection result of the operation detection unit 11c whether or not the release operation unit 13a has been operated for 2nd release (step S120). If the 2nd release operation has not been performed (step S120: No), it is determined from the detection result of the operation detection unit 11c whether or not the first release operation of the release operation unit 13a has been released (step S121).

  When the 1st release operation is not released (step S121: No), the 1st release operation is in a state of being continued, and the search control unit 11a returns to step S119 to perform the search execution process. In other words, while the operation detection unit 11c detects that the first release operation is continued, the temporary reference image Pa is changed according to the change of the live view image Pr, and a similar image is searched. .

  On the other hand, when the 2nd release operation is performed (step S120: Yes), the search control unit 11a designates the live view image Pr displayed at the timing of the 2nd release operation as the reference image Pa, and the display control unit. 11b stops the live view display of the live view image Pr and fixedly displays the reference image Pa (step S130). Further, as shown in FIG. 15A, the display control unit 11b enlarges and displays the similar image region Rc for displaying the similar image Pc obtained as a similar image search result (step S131). Note that instead of the enlarged display in step S131, the search screen 23b as shown in FIG. 14B or FIG. 14C may be displayed.

  Thereafter, whether or not there has been any instruction operation by the user is determined based on the detection result of the operation detection unit 11c (step S132). For example, when there is a similar image display order change instruction by the operation of the direction operation unit 13d in a predetermined direction (step S133: Yes), the display control unit 11b changes the display order of the similar images Pc in the similar image region Rc. Is displayed (step S134). That is, when the display is in the time series order, the display is changed to the display in the order of similarity, and when the display is in the order of the similarity degree, the display is changed to the display in the time series order.

  Thereafter, it is determined whether or not there is an instruction to display a use menu of a search result accompanying a predetermined operation of the direction operation unit 13d by the user (step S135). When there is an instruction to display the usage menu (step S135: Yes), the display control unit 11b displays the menu screen M2 on the search screen 23b as shown in FIG. 14D (step S136). If the use item is touched on the menu screen M2 (step S137: Yes), a process according to the touch-specified use item is executed (step S138). Details of the processing of each usage item are omitted.

  Further, when there is no instruction to display such usage menu (step S135: No), or when there is no usage item touch designation (step S137: No), or when the processing of each usage item is completed, FIG. As schematically shown in FIG. 15A, it is determined from the detection result of the operation detection unit 11c whether or not the user has given a single touch instruction to the similar image region Rc (step S140). In particular, when there is no single touch instruction (step S140: No), the process proceeds to step S144.

  On the other hand, if there is a single touch instruction (step S140: Yes), it is determined whether or not the instructed similar image Pc is currently being enlarged (step S141). When the enlarged display is not currently being performed (step S141: No), the display control unit 11b enlarges and displays the similar image Pc that has been instructed to be single-touched over the entire search screen 23b (step S142). That is, the display shown in FIG. 15A is changed to the display shown in FIG. If the enlarged display is currently being performed (step S141: Yes), the display control unit 11b executes a process for returning to the normal display of the search screen 23b (step S143). That is, the display as shown in FIG. 15B is returned to the display as shown in FIG.

  In step S144, it is determined again based on the detection result of the operation detection unit 11c whether or not the release operation unit 13a has been operated for the first release. If the first release operation has not been performed (step S144: No), the process returns to step S132 and waits for some user operation. When the first release operation is performed again (step S144: Yes), the search result is reset (step S145), the process returns to step S113, and the live view image display state is set.

  As described above, according to the second embodiment, the reference image Pa to be searched for can be specified in the live view image Pr. Therefore, an appropriate reference image can be selected from among the images stored in the image storage unit 21. The search reference image can be easily obtained and set when searching for similar images. In addition, since each search condition is displayed on the search screen 23b with an iconized condition icon, the contents of the search condition can be easily understood in a smaller display area than in the case of using only characters. The display form is suitable for the small display unit 23 for 1.

  Further, according to the second embodiment, the search portion specifying mark Pm indicating the portion used for the search condition selected in the live view image Pr is superimposed on the life view image Pr and displayed. Therefore, depending on the selected condition, the contents of the search condition can be understood only by viewing the live view image, and the possibility of erroneous selection of the search condition can be reduced.

  Furthermore, according to the second embodiment, the similar image search unit is caused to search for a similar image by using the live view image displayed at the timing of the detected release operation as a designated reference image. Since the operation for designating the reference image, which is an operation similar to shooting, can be performed in the same manner as the shooting operation, it is easy to use. In addition, the designation as the provisional reference image of the life view image and the designation as the final reference image can be performed by distinguishing between the first release operation and the second release operation of the release operation unit 13a. Therefore, the operation is the same as the first stage release operation and the second stage release operation during the photographing operation, and can be operated without confusion.

  Further, according to the second embodiment, after the reference image is designated from the live view images by the release operation in the second stage, the designated reference image is continuously displayed instead of the live view image. As a result, the display is fixed with the finally designated reference image, and the content of the instruction can be confirmed with certainty.

  Further, according to the second embodiment, since the similar image searched by the similar image search unit 25 is displayed together with the live view image, it is easy to compare both images, and the searched similar image is the target image. Even if the user feels that they do not match, it can be used to specify the next reference image.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention, and they are not excluded from the scope of the present invention.

  For example, with respect to each processing function as shown in FIGS. 10 to 12 and FIGS. 17 to 19 in the first and second embodiments, in the block diagram shown in FIG. The display control and the like have been described as being processed by software by the control unit 11, but a specific example of the configuration is not limited to this. It is a design matter to perform these processes in any one of hardware processing and software processing or in an appropriate combination.

  When all or part of the embodiment is executed by software processing, the control unit 11 reads and executes the search program stored in the internal data storage unit 12 such as a flash memory. Since such software processing is realized, a search program related to such software processing is also included in the present invention. A recording medium on which such a search program is recorded is also the present invention. The recording medium for storing the program is not limited to the flash memory, but an optical recording medium such as a CD-ROM or DVD-ROM, a magnetic recording medium such as an MD, a tape medium, or a semiconductor memory such as an IC card. It may be. Further, as a search program, a program obtained from an external recording medium via a network, for example, a program downloaded from a homepage is naturally included. In such a case, a WWW server, an ftp server, and the like that are downloaded by the user are also included in the scope of the present invention.

  In the first and second embodiments, the example in which the image search apparatus is mounted on a digital camera has been described. However, the present invention is not limited to this. The image search device of the first embodiment may be applied to a mobile phone or an image display device (image viewer). The camera of the second embodiment may be applied to a camera unit of a personal computer, a camera unit of a mobile phone, and the like.

It is a rear view which shows the example of schematic structure of a digital camera provided with the image search device of Embodiment 1, 2 of this invention. It is a block diagram which shows the system structural example of a digital camera provided with the image search device of Embodiment 1, 2 of this invention. It is explanatory drawing which shows typically the relationship of the display area of the various information regarding the search screen of Embodiment 1 of this invention. It is explanatory drawing which shows the example of a basic display of the search screen in search mode. It is explanatory drawing which shows the example of a display when a new reference | standard image is further designated with the same search conditions after the display of the similar image searched. It is explanatory drawing which shows the example of a display when a search condition is further added by the user after the display of the similar image searched. It is explanatory drawing which shows the example of a display when a new search is further selected by the user after the display of the searched similar image. It is explanatory drawing which shows the example of a display of the utilization menu of a search result. It is explanatory drawing which shows the modification of the example of a display of a search screen. It is a main flowchart which shows the example of operation control of the search process performed by the control part. It is a flowchart which shows the example of operation control of the search mode performed by the control part. It is a flowchart which shows the example of a search execution process performed by the similar image search part. It is explanatory drawing which shows typically the display control example of the search screen of Embodiment 2 of this invention. It is explanatory drawing which shows the example of a display control accompanying release operation. It is explanatory drawing which shows the modification of display control. It is explanatory drawing which shows the example of a display control when a search condition is changed. It is a main flowchart which shows the example of operation control of the search process performed by the control part. It is a flowchart which shows the first half part of the example of operation control of the search mode performed by the control part. It is a flowchart which shows the latter half part of the example of operation control of the search mode performed by the control part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 11a Search control part 11b Display control part 11c Operation detection part 14 Touch panel 21 Image memory | storage part 23 Display part 23a Search screen 25 Similar image search part Ra Reference | standard image area Rb Search condition area Rc Similar image area a Image Pa Specified Reference image b Condition icon Pb Selected condition icon (search condition)
Pc Similar images found M1 menu screen M2 menu screen

Claims (11)

Based on the selected search criteria, there in the image retrieval apparatus comprising: a similar image retrieval unit for retrieving a similar image similar to the reference image, and a display unit for displaying a search screen for retrieval by the similar image search unit And
An imaging unit that outputs a digital electrical signal by receiving a subject image incident via a photographing optical system and performing photoelectric conversion;
An operation detection unit for detecting a user operation on the search screen;
A display control unit that controls display of the search screen, and displays a live view image corresponding to the digital electric signal output by the imaging unit on the display unit in a moving image in real time ;
Based on the search condition the selected live view image as the designated the reference image by operation of a user on the search screen, the control of the similar image retrieval unit to retrieve similar image retrieval control unit When,
With
The display controller, a part of the area of the search screen, as a reference image area for designating the reference image, to display the live view image to the reference image region, further, adjacent to the reference image area the region, the iconized condition icons freely selected to be displayed as a search condition region for selecting the search condition, further, the said reference image region adjacent to the search area as a similar image area search An image search apparatus that displays the similar image searched by the similar image search unit under conditions. Wherein the display control unit, in the case where the live view image as the reference image is specified by a user operation with respect to the reference image area, when the search condition is selected by a user operation to the search condition region, is the designated The image search apparatus according to claim 1 , wherein the selected condition icon is displayed over the live view image . The display controller, when the search conditions are added by further user operation after displaying the search of similarity image, a menu screen for selecting a relationship between the retrieval conditions that are added to the previous search The image search apparatus according to claim 1, wherein the image search apparatus is displayed on the search screen. A touch panel disposed on the display unit for displaying the search screen;
The operation detection unit detects a user operation on the touch panel,
The display control unit switches between an image of the reference image area and a condition icon of the search condition area, which are simultaneously displayed on the search screen, by independent movement according to a user operation on the touch panel. The image search device according to claim 1.   2. The image according to claim 1, wherein the display control unit simultaneously displays at least two adjacent areas among the reference image area, the search condition area, and the similar image area on the search screen. Search device.   The display control unit performs a segmented display indicating a relationship between a similar image to be displayed in the similar image area and the condition icon selected for searching for the similar image in the search condition area. The image search apparatus according to claim 1, wherein:   The image search apparatus according to claim 1, wherein the display control unit displays on the search screen a menu screen for selecting a use purpose of a search result obtained by the similar image search unit.   The search control unit prepares a plurality of search conditions displayed by the display control unit, and the search conditions include at least one of a color, a human face, a shooting time, a shooting scene, a shooting composition, or a shooting location. The image search apparatus according to claim 1. Digital camera characterized by being equipped with the image search apparatus according to any one of claims 1 to 8. Based on the selected search criteria, the similar image retrieval unit for retrieving a similar image similar to the reference image, the image retrieval apparatus executes and a display unit for displaying a search screen for retrieval by the similar image search unit An image search method for
An imaging step of receiving a subject image incident via a photographing optical system and outputting a digital electric signal by performing photoelectric conversion;
An operation detecting step for detecting a user operation on the search screen;
A display control step of controlling the display of the search screen and displaying a live view image corresponding to the digital electric signal output in the imaging step on the display unit in a moving image in real time ;
A search control step for controlling the similar image search unit to search for the similar image based on a live view image as the reference image specified by a user operation on the search screen and a selected search condition; ,
Including
Wherein the display control step, a part of the area of the search screen, as a reference image area for designating the reference image, to display the live view image to the reference image region, further, adjacent to the reference image area the region, the iconized condition icons freely selected to be displayed as a search condition region for selecting the search condition, further, the said reference image region adjacent to the search area as a similar image area search An image search method comprising: displaying the similar image searched by the similar image search unit under a condition. Computer image retrieval apparatus comprising: a similar image retrieval unit for retrieving a similar image similar to the specified reference image in the selected search condition, and a display unit for displaying a search screen for retrieval by the similar image search unit In addition,
An imaging step of receiving a subject image incident via a photographing optical system and outputting a digital electric signal by performing photoelectric conversion;
An operation detecting step for detecting a user operation on the search screen;
A display control step of controlling the display of the search screen and displaying a live view image corresponding to the digital electric signal output in the imaging step on the display unit in a moving image in real time;
A search control step for controlling the similar image search unit to search for the similar image based on a live view image as the reference image specified by a user operation on the search screen and a selected search condition; ,
And execute
Wherein the display control step, a part of the area of the search screen, as a reference image area for designating the reference image, to display the live view image to the reference image region, further, adjacent to the reference image area the region, the iconized condition icons freely selected to be displayed as a search condition region for selecting the search condition, further, the said reference image region adjacent to the search area as a similar image area search An image search program for displaying the similar image searched by the similar image search unit on condition.

Images