JP5754218B2 - SEARCH DEVICE, SEARCH METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a search device, a search method using the search device, and a program.

  Conventionally, attribute information related to the shooting date and time, the location of the shooting location, etc. is recorded in association with the image information of the shot image, and the image was shot at the date and location specified by the user using the attribute information. An information management technique that makes it possible to search for an image is known (for example, see Patent Document 1). Also, a technique for grouping and classifying a predetermined number of images based on the shooting date and the shooting location in order to facilitate the search for a desired image is also known.

JP 2006-127525 A

  However, in the technique disclosed in Patent Document 1 or the like, when the search target is an image taken in the past, there is a possibility that it is impossible to accurately remember when and where the image was taken. As described above, there is a problem that the search target image cannot be properly searched only by the shooting date and time of the image and the position of the shooting location.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a search device, a search method, and a program that can appropriately search for desired image information.

In order to solve the above problems, the search device of the present invention provides:
First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous A second recording means for recording at least one event information related to an event other than the image capturing that occurs at the timing in association with the event related information related to the occurrence of the event; and a search for searching for a search target When searching for the image information as a search target and an input means for inputting a condition, if other information other than the imaging related information is input as the search condition by the input means, the information is related to the other information. Event information is retrieved from the event information recorded in the second recording means, and is associated with the retrieved event information. Identification means for specifying imaging-related information related to event-related information, and imaging-related information input by the input means or specified by the specification means among the image information recorded in the first recording means The image processing apparatus includes a search unit that searches for image information associated with imaging related information, and an output unit that outputs a search result of the search target by the search unit.

In addition, the search method of the present invention includes:
First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous Search using a search device comprising: second recording means for recording at least one event information related to an event other than imaging of the image that occurs at the timing in association with event related information related to the occurrence of the event A method of inputting a search condition for searching for a search target, and when searching for the image information as a search target, when information other than the imaging related information is input as the search condition Event information related to the other information is searched from the event information recorded in the second recording means, and the searched event information Among the image information recorded in the first recording means, the imaging related information input by the input means or the specified information is specified. And a process of searching for image information associated with the imaging-related information and a process of outputting a search result of the search target.

The program of the present invention is
First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous A second recording unit that records at least one event information related to an event other than imaging of the image that occurs at a timing in association with event-related information related to the occurrence of the event; Input means for inputting a search condition for searching for a search target, and when searching for the image information as a search target, when other information other than the imaging related information is input as the search condition by the input means, Event information related to the other information is searched from the event information recorded in the second recording means, Identification means for specifying imaging-related information related to event-related information associated with event information, imaging-related information input by the input means among image information recorded in the first recording means, or It is characterized by functioning as search means for searching image information associated with the imaging related information specified by the specifying means, and output means for outputting the search result of the search object by the search means.

  According to the present invention, it is possible to appropriately search for desired image information even when it is impossible to accurately remember when and where the image to be searched was taken.

It is a block diagram which shows schematic structure of the portable terminal of Embodiment 1 to which this invention is applied. It is a figure which shows an example of the event recorded as event information in the portable terminal of FIG. It is a figure which shows typically an example of the event recording table recorded on the portable terminal of FIG. 3 is a flowchart illustrating an example of an operation related to an imaging process by the mobile terminal in FIG. 1. It is a flowchart which shows an example of the operation | movement which concerns on the event recording process by the portable terminal of FIG. 3 is a flowchart illustrating an example of an operation related to an image search process by the mobile terminal of FIG. 1. It is a flowchart which shows an example of the operation | movement which concerns on the event search process by the portable terminal of FIG. It is a figure which shows schematic structure of the search system of Embodiment 2 to which this invention is applied. It is a block diagram which shows schematic structure of the imaging device which comprises the search system of FIG. It is a block diagram which shows schematic structure of the mobile telephone which comprises the search system of FIG. It is a block diagram which shows schematic structure of the search device which comprises the search system of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
FIG. 1 is a block diagram showing a schematic configuration of a mobile terminal 100 according to the first embodiment to which the present invention is applied.
As shown in FIG. 1, the mobile terminal 100 includes a central control unit 1, a memory 2, an imaging unit 3, an imaging control unit 4, an image processing unit 5, an operation input unit 6, a transmission / reception unit 7, a communication control unit 8, a position An acquisition unit 9, a date acquisition unit 10, a detection sensor unit 11, an event acquisition control unit 12, a recording unit 13, a search control unit 14, a display unit 15 and the like are provided.
The mobile terminal 100 includes, for example, an imaging device having a communication function, a mobile station used in a mobile communication network such as a mobile phone or a PHS (Personal Handy-phone System), a PDA (Personal Data Assistants), and the like. Has been.

  The central control unit 1 controls each unit of the mobile terminal 100. Specifically, the central control unit 1 includes a CPU (not shown) that controls each part of the mobile terminal 100, and performs various control operations according to various processing programs (not shown).

  The memory 2 includes, for example, a buffer memory that temporarily records image information and the like, a working memory such as the central control unit 1, a program memory that stores various programs and data related to the functions of the mobile terminal 100, etc. (Not shown).

  The imaging unit 3 constitutes an imaging unit. Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and a unit circuit unit 3c.

  For example, although not shown, the lens unit 3a includes a zoom lens, a focus lens, a diaphragm, and the like, and forms an optical image of a subject that has passed through these lenses.

  The electronic imaging unit 3b is disposed on the optical axis of the lens unit 3a. Further, the electronic imaging unit 3b is composed of, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and an optical image that has passed through various lenses of the lens unit 3a is displayed at predetermined intervals. It is converted into a two-dimensional image signal (RGB image data) and output to the unit circuit unit 3c.

  For example, the unit circuit unit 3c is not shown in the figure, but a CDS (Correlated Double Sampling), AGC (Auto Gain Control), ADC (Analog to Digital Converter) is an analog / digital converter. ) Etc. The unit circuit unit 3c holds an analog image signal corresponding to the optical image of the subject output and input from the electronic imaging unit 3b by the CDS at a predetermined period corresponding to the imaging frame rate, and stores the image signal. After amplification by AGC, the amplified image signal is converted into a digital image signal by ADC. Then, the unit circuit unit 3 c transmits a digital image signal to the image processing unit 5.

The imaging control unit 4 controls operations of the lens unit 3a, the electronic imaging unit 3b, the unit circuit unit 3c, and the like constituting the imaging unit 3 when imaging a subject based on a command from the central control unit 1. In other words, the imaging control unit 4 is configured so that the lens unit 3a and the electronic imaging so as to image the subject under predetermined imaging conditions (for example, “aperture value”, “exposure time”, “ISO sensitivity”, “photometry method”, etc.) The operations of the unit 3b and the unit circuit unit 3c are controlled.
Specifically, the imaging control unit 4 includes, for example, a drive source such as a motor and a driver (not shown) that drives the drive source, and the zoom lens and the focus lens of the lens unit 3a are arranged in the optical axis direction. To drive. In addition, the imaging control unit 4 includes, for example, a diaphragm driving unit (not shown) that increases or decreases the diameter of the aperture of the lens unit 3a, and increases or decreases the diameter of the aperture according to the aperture value. The imaging control unit 4 includes, for example, a TG (Timing Generator), a driver for driving the electronic imaging unit 3b, and the like (both not shown), and performs electronic imaging via the TG and driver according to the drive frame rate, exposure time, and the like. The operation timing of the unit 3b is controlled. The imaging control unit 4 controls the operation timing of the unit circuit unit 3c according to the imaging frame rate.

  The imaging control unit 4 may be configured to cause the imaging unit 3 to image the subject at a timing when imaging is instructed by the central control unit 1 based on a predetermined operation of the shutter button of the operation input unit 6 by the user. However, the image capturing unit 3 may automatically capture images at a predetermined timing (for example, a predetermined time interval) without synchronizing with the timing of the imaging instruction via the operation input unit 6 by the user. . That is, in this case, there is a possibility that the composition is not adjusted by the user and a predetermined subject is not captured within the angle of view.

  The image processing unit 5 performs RGB interpolation processing for generating R, G, and B color component data (RGB data) for each pixel with respect to the image signal after A / D conversion sent from the unit circuit unit 3c. YUV conversion processing for generating YUV data consisting of luminance signals (Y) and color difference signals (U, V) from RGB data for each pixel, and digital signal processing for improving image quality such as edge enhancement are performed. Then, the image processing unit 5 sequentially outputs YUV data of each image frame after conversion to the memory 2 and stores it in the memory 2.

The operation input unit 6 is for inputting various instructions to the mobile terminal 100 main body.
Specifically, the operation input unit 6 executes a shutter button related to a subject shooting instruction, up / down / left / right cursor buttons and a determination button related to a selection instruction of a mode, a function, etc., making / receiving a call, sending / receiving an e-mail, etc. Various buttons (not shown) such as communication related buttons related to instructions and numeric buttons and symbol buttons related to text (document) input instructions are provided.
When various buttons are operated by the user, the operation input unit 6 outputs an operation instruction corresponding to the operated button to the central control unit 1. The central control unit 1 causes each unit to execute a predetermined operation (for example, imaging of a subject, incoming / outgoing calls, transmission / reception of an e-mail, etc.) according to an operation instruction output from the operation input unit 6 and input.

Further, the operation input unit 6 inputs a search condition for searching for a search target such as image information or event information as input means.
Here, for example, when image information is a search target, as the search condition, imaging position information (for example, place name and coordinate information) regarding the imaging position of the image and imaging date / time information (for example, month, Imaging related information such as day, day of the week, year, and combinations thereof), keywords related to incidental information (described later) attached to the event that occurred (for example, name, e-mail address, physical condition, weather, etc.) It is done. Further, for example, when event information is a search target, event position information (for example, place name and coordinate information) related to an event occurrence position and event date / time information (for example, month, day) related to the event occurrence date / time are used as search conditions. Event related information, image information itself, keywords related to the subject in the image (eg, human, pet, animal, name, e-mail address, physical condition, weather, etc.) Can be mentioned.

  The operation input unit 6 may include a touch panel provided integrally with the display unit 15, and based on a predetermined operation of the touch panel by the user, an operation instruction corresponding to the predetermined operation is given to the central control unit. 1 may be output.

The transmitter / receiver unit 7 performs a call with an external user of an external device connected via the communication network N.
Specifically, the transmission / reception unit 7 includes a microphone 7a, a speaker 7b, a data conversion unit 7c, and the like. The transmission / reception unit 7 performs A / D conversion processing on the user's transmission voice input from the microphone 7a by the data conversion unit 7c and outputs the transmission voice data to the central control unit 1. Under the control, voice data such as received voice data outputted and inputted from the communication control unit 8 is D / A converted by the data conversion unit 7c and outputted from the speaker 7b.

  The microphone 7a automatically collects sound at a predetermined timing (for example, a predetermined time interval) without being synchronized with the timing of an imaging instruction by the user via the operation input unit 6 as sound collection means. It may be a configuration. That is, in this case, there is a possibility that conversations and external sounds (for example, voices of friends and third parties near the user, sounds of the external environment, etc.) that are not conscious of the user are collected.

The communication control unit 8 transmits and receives data via the communication network N and the communication antenna 8a.
That is, the communication antenna 8a is connected to a predetermined communication method (for example, W-CDMA (Wideband Code Division Multiple Access) method, CDMA2000 method, GSM) adopted by the mobile terminal 100 for communication with a radio base station (not shown). (Global System for Mobile Communications), etc.). The communication control unit 8 transmits / receives data to / from the radio base station via the communication antenna 8a using a communication channel set in the communication method according to a communication protocol corresponding to a predetermined communication method.
That is, the communication control unit 8 transmits / receives voice during a call with an external user of the external device to the external device of the communication partner based on the instruction signal output from the central control unit 1 and input, Send and receive e-mail data.
Note that the configuration of the communication control unit 8 is an example and is not limited thereto, and can be arbitrarily changed as appropriate. For example, although not illustrated, a wireless LAN module is mounted and an access point (Access Point) is provided. The communication network N may be configured to be accessible via

  The communication network N is a communication network that connects the mobile terminal 100 via a wireless base station, a gateway server (not shown), or the like. The communication network N is a communication network constructed by using a dedicated line or an existing general public line, and applies various line forms such as a LAN (Local Area Network) and a WAN (Wide Area Network). Is possible. The communication network N includes, for example, various communication network networks such as a telephone line network, ISDN line network, dedicated line, mobile communication network, communication satellite line, CATV line network, IP network, VoIP (Voice over Internet Protocol). ) Gateways, Internet service providers, etc. are included.

The position acquisition unit 9 acquires position information related to the current position of the mobile terminal 100 main body based on a GPS signal received by a GPS (Global Positioning System) antenna 9a.
That is, the GPS antenna 9a is a GPS signal (for example, almanac (rough orbit information) or ephemeris) transmitted from a plurality of GPS satellites launched in a low earth orbit (only one GPS satellite S is shown in FIG. 1). (Detailed trajectory information) is received at a predetermined timing, and the GPS signal is output to the position acquisition unit 9. The position acquisition unit 9 measures the absolute two-dimensional current position (latitude, longitude) of the terminal body based on the GPS signal output from the GPS antenna 9a, and position information (for example, latitude) , Longitude coordinate information).
Further, the position acquisition unit 9 may acquire the position information related to the current position of the mobile terminal 100 main body at a timing instructed based on a predetermined operation of the operation input unit 6 by the user, or at every predetermined time interval. You may get it.

  The position acquisition unit 9 outputs position information generated at the time of image capturing to the recording unit 13, and is associated with image information as image capturing position information (image capturing related information) regarding the image capturing position of the image. It is recorded in an image recording unit 13a (described later). Similarly, the position acquisition unit 9 outputs position information generated at the time of occurrence of an event to the recording unit 13, and is associated with event information as event position information (event related information) regarding the occurrence position of the event. Is recorded in the event recording unit 13b (described later).

  The position acquisition unit 9 may be configured to acquire, for example, information related to the current position that is input based on a predetermined operation of the operation input unit 6 by the user as position information. That is, based on the place name or address of the current position input based on a predetermined operation of the operation input unit 6 by the user, the position acquisition unit 9 refers to a predetermined database (not shown) and the mobile terminal 100 You may make it acquire the positional information which concerns on the present position (latitude, longitude).

The date acquisition unit 10 acquires the current date and time.
That is, the date and time acquisition unit 10 includes, for example, a timer unit configured with a timer, a timer circuit, and the like, and clocks the current time with this timer unit. Further, the date acquisition unit 10 identifies today's date, day of the week, etc. from the calendar information based on the current time measured by the time measuring unit. Then, the date acquisition unit 10 generates and acquires date information related to the current date based on the current time and today's date.

  Further, the date acquisition unit 10 outputs date information generated at the time of image capturing to the recording unit 13 and is associated with image information as imaging date information (imaging related information) regarding the image capturing date of the image. It is recorded in an image recording unit 13a (described later). Similarly, the date and time acquisition unit 10 outputs the date and time information generated at the time of occurrence of an event to the recording unit 13, and is associated with event information as event date and time information (event related information) regarding the date and time of occurrence of the event. Is recorded in the event recording unit 13b (described later).

The detection sensor unit 11 includes a motion detection sensor 11a, a body state detection sensor 11b, and an environment detection sensor 11c.
Specifically, the motion detection sensor (motion detection means) 11a includes, for example, an acceleration detection sensor, an angular velocity detection sensor, a vibration sensor, a tilt sensor, and the like, and detects the movement and posture of the terminal body. The body state detection sensor 11b is composed of, for example, a body temperature sensor, a heart rate sensor, a myoelectric potential sensor, and the like, and detects the state of the user's body. Moreover, the environment detection sensor 11c is comprised from a brightness sensor, a temperature sensor, a humidity sensor, an atmospheric | air pressure sensor etc., for example, and detects the external environment of a terminal main body.
And each of the motion detection sensor 11a, the body state detection sensor 11b, and the environment detection sensor 11c outputs the detection signal according to the detection result by each sensor. That is, each of the motion detection sensor 11a, the body state detection sensor 11b, and the environment detection sensor 11c may operate at a timing instructed based on a predetermined operation of the operation input unit 6 by the user and output a detection signal. The detection signal may be output by operating at a predetermined timing (for example, a predetermined time interval) without synchronizing with the timing of the imaging instruction by the user via the operation input unit 6.

  Note that each of the motion detection sensor 11a, the body state detection sensor 11b, and the environment detection sensor 11c may be mounted in the apparatus body, for example, or connected externally by a predetermined communication cable. Also good.

The event acquisition control unit 12 controls acquisition of event information related to the event that has occurred. Specifically, the event acquisition control unit 12 includes a first acquisition unit 12a, a determination unit 12b, and a second acquisition unit 12c.
Here, the event is an event other than the image capturing that occurs at a timing asynchronous with the image capturing instruction timing by the user. In addition, the event relates to an operation (for example, incoming / outgoing calls, transmission / reception of e-mail, input of schedules and various documents, etc.) executed based on a predetermined execution instruction operation of the operation input unit 6 by the user. Operation that is performed without requiring a predetermined execution instruction operation of the operation input unit 6 by the user (for example, detection of the movement and posture of the terminal body by the detection sensor unit 11 and the state of the user's body) And the operation-unnecessary event related to the detection of the external environment (meteorology) of the terminal body (see FIG. 2).
FIG. 2 shows an example of an event in association with an event type, an event occurrence timing, and incidental information. However, the present invention is not limited to these, and the event type, event occurrence timing, incidental information, etc. Can be arbitrarily changed as appropriate.

The first acquisition unit 12a acquires event-related information (accompanying information) related to the operation event recorded in the event recording table T (see FIG. 3; described later) of the event recording unit 13b.
That is, as an operation event, for example, when an execution operation such as an outgoing call operation or an incoming call operation by the transmission / reception unit 7 is performed, the first acquisition unit 12 a , Keywords related to the other party related to the call operation or the incoming call operation, for example, name, address, telephone number, etc. are obtained by referring to an address book (not shown), and these are related to the event related to the event. Obtain as information.
Further, as an operation event, for example, when an execution operation such as an email transmission operation or a reception confirmation operation is performed by the communication control unit 8, the first acquisition unit 12a performs an operation such as a transmission operation or a reception confirmation operation. The type and keywords related to the communication partner related to the transmission operation or the reception confirmation operation, for example, name, e-mail address, telephone number, etc. are obtained by referring to an address book (not shown), and these are Acquired as event-related information related to the event.
In addition, when an operation operation is performed by the user via the operation input unit 6 such as an input operation such as creation, editing, or deletion of a schedule or a document, or a browsing operation, the first acquisition unit 12 a And document type, operation type such as input operation and browsing operation, and the contents of the schedule and keywords related to the document, for example, title, etc., and these are acquired as event related information related to the event To do.

The determination unit 12b determines, as a determination unit, whether or not an operation unnecessary event that does not require a predetermined execution instruction operation of the operation input unit 6 by the user for execution of the operation has occurred.
In other words, the determination unit 12b determines from the image based on the image information of the image automatically captured by the imaging unit 3 at a predetermined timing without requiring a predetermined imaging instruction operation of the operation input unit 6 by the user. Subject detection processing for detecting a predetermined subject (for example, “human” or “animal”) is performed, and it is determined whether or not an operation unnecessary event has occurred depending on whether or not a predetermined type of subject has been detected. To do.
Further, the determination unit 12b is identified from the sound based on the sound data automatically collected by the microphone 7a at a predetermined timing without requiring a predetermined sound collection instruction operation of the operation input unit 6 by the user. An operation-free event occurs depending on whether or not a specific word is detected by performing a word detection process that detects words (for example, “words” such as words and phrases, “names” of people and corporations, etc.) Determine whether or not.
Further, the determination unit 12b does not require a predetermined positioning instruction operation of the operation input unit 6 by the user, and is based on the current position of the apparatus main body that is automatically positioned by the position acquisition unit 9 at a predetermined timing. Whether a specific place name (prefecture, city, ward, town, etc. to which the current position belongs) or a specific location of a specific building (for example, Tokyo Tower) is detected by referring to a map database (not shown) Whether or not an operation-unnecessary event has occurred is determined according to whether or not the event has occurred.
Further, the determination unit 12b does not require a predetermined motion detection instruction operation of the operation input unit 6 by the user, and is based on a detection signal that is automatically detected and output by the motion detection sensor 11a at a predetermined timing. Whether a predetermined type of movement (for example, sleeping, studying, exercising, eating, driving a car, taking a walk, jogging, etc.) was detected from the change of the detection signal and the fluctuation pattern per unit time. Whether or not an operation-unnecessary event has occurred is determined according to whether or not the event has occurred. The determination unit 12b determines the type of movement of the terminal main body based on the change in the current position of the apparatus main body measured by the position acquisition unit 9 (for example, while walking, jogging, etc.) It may be determined whether or not a train is moving).
Further, the determination unit 12b is automatically detected and output by the body state detection sensor 11b at a predetermined timing or a predetermined time interval without requiring a predetermined detection instruction operation of the operation input unit 6 by the user. Based on the detected signal, it is determined whether or not an operation-unnecessary event has occurred depending on whether or not physical information (for example, body temperature, blood pressure, pulse, etc.) of the user of the terminal main body has been detected.
Further, the determination unit 12b is based on a detection signal that is automatically detected and output by the environment detection sensor 11c at a predetermined timing without requiring a predetermined detection instruction operation of the operation input unit 6 by the user. It is determined whether or not an operation unnecessary event has occurred depending on whether or not the external environment (for example, weather, temperature, atmospheric pressure, humidity, etc.) of the terminal body has changed.

As subject detection processing, for example, image regions (candidate regions) that are candidates for a predetermined subject are extracted by various image processing such as face detection processing, edge detection processing, and feature extraction processing, and the extracted candidate regions are extracted. Among them, there is a method of detecting a subject that satisfies a predetermined identification condition as a predetermined subject. However, this is an example, and the present invention is not limited to this.
As the word detection process, for example, a speech analysis process is performed on the sound data using a spectrum analyzer or the like, and words such as words or phrases registered in advance in predetermined registration means (for example, the memory 2). However, this is an example and is not limited to this, and can be arbitrarily changed as appropriate.

The second acquisition unit 12c acquires event-related information (accompanying information) related to an operation unnecessary event recorded in the event recording table T of the event recording unit 13b.
That is, when the determination unit 12b determines that an operation unnecessary event has occurred, the second acquisition unit 12c acquires event related information related to the operation unnecessary event (for example, a motion detection event).
Specifically, as an operation unnecessary event, for example, when a predetermined type of subject is detected from image information of an automatically captured image and the operation unnecessary event occurs, the second acquisition unit 12c The type of subject, for example, information related to “human” or “animal” is acquired as event-related information related to the event.
In addition, as an operation unnecessary event, for example, when a specific word is detected from automatically collected sound data and the operation unnecessary event occurs, the second acquisition unit 12c performs the specific word, for example, Information related to “words” such as words and phrases, “names” of people and corporations, and the like is acquired as event-related information related to the event.
Further, as an operation unnecessary event, for example, when the position acquisition unit 9 automatically measures the current position of the apparatus main body and detects a specific place, the second acquisition unit 12c generates the operation unnecessary event. Information relating to the specific place, for example, “Hawaii”, “Tokyo Tower”, etc. is acquired as event-related information related to the event.
Further, as an operation unnecessary event, for example, when a predetermined type of motion is detected and the operation unnecessary event occurs based on a detection signal automatically detected and output by the motion detection sensor 11a, the second operation The acquisition unit 12c obtains information related to the predetermined type of movement, for example, “sleeping”, “study”, “exercising”, and information on the time and distance of the movement, Obtain as information.
Further, as the operation unnecessary event, for example, when the physical information of the user of the terminal body is automatically detected by the body state detection sensor 11b and the operation unnecessary event occurs, the second acquisition unit 12c The body information, for example, “body temperature”, “blood pressure”, “pulse”, etc., and information related to the detection of the body information are acquired as event related information related to the event.
Further, as an operation unnecessary event, for example, based on a detection signal automatically detected and output by the environment detection sensor 11c, a change in the external environment (weather) of the terminal body is detected and the operation unnecessary event occurs. In such a case, the second acquisition unit 12c displays the external environment, for example, “weather”, “temperature”, “barometric pressure”, “humidity”, etc. Etc. is acquired as event related information related to the event.
The second acquisition unit 12c is a weather server (not shown) having a function of opening a Web page (for example, a weather information providing page) on the Internet as a Web (World Wide Web) server instead of the environment detection sensor 11c. Abbreviations) for location information (for example, latitude and longitude coordinate information) and date and time information (for example, 20 △△ year; ○ month ○ day; □ day of the week, ** time, etc.) Corresponding weather (weather) may be acquired via the communication control unit 8.

  The recording unit 13 includes an image recording unit 13a and an event recording unit 13b. The recording unit 13 reads data from the image recording unit 13a and the event recording unit 13b and writes data to the image recording unit 13a and the event recording unit 13b. I do.

The image recording unit 13a records at least one image information for recording encoded in a predetermined compression format (for example, JPEG format) by an encoding unit (not shown) of the image processing unit 5. Specifically, the image recording unit 13a records image information as an Exif format image file, and captures imaging-related information (such as shooting position information and shooting date / time information) related to the image shooting as Exif information. Record in association. That is, the image recording unit 13a captures position information related to the current position of the mobile terminal 100 acquired by the position acquisition unit 9 when the image capturing unit 3 captures an image with respect to image capturing position information (for example, (Name of place, coordinate information, etc.) are recorded in association with the image information of the image. Similarly, when the image recording unit 13 captures an image, the image recording unit 13a obtains date / time information related to the current date / time acquired by the date / time acquisition unit 10 as imaging date / time information (for example, 20Δ) regarding the imaging date / time of the image. △ year; ○ month ○ day; □ day of week, etc.) and recorded in association with the image information of the image.
In addition, although the imaging position information regarding the imaging position of the image and the imaging date / time information regarding the imaging date / time of the image are illustrated as the imaging related information, it is an example and is not limited thereto, and is information related to the imaging of the image. Any change can be made as appropriate. That is, the imaging related information is less related to the event related information than the imaging position information or the imaging date and time information, and the imaging specific related information specific to the imaging of the image, for example, various keywords related to the subject (human , Animals, landscapes, etc.).
As described above, the image recording unit 13a constitutes a first recording unit that records at least one piece of image information of an image captured by the imaging unit 3 in association with imaging related information related to imaging of the image. Yes.

The event recording unit 13b stores the event information related to the generated event in the event recording table T (see FIG. 3) in association with event related information (event position information, event date information, etc.) related to the occurrence of the event. And record at least one. That is, when a predetermined event occurs at a timing asynchronous with the image capturing instruction timing, the image recording unit 13a displays the position information regarding the current position of the mobile terminal 100 acquired by the position acquisition unit 9 of the event. As event position information (for example, place name and coordinate information) related to the occurrence position, it is recorded in association with the incidental information of the event. Similarly, when a predetermined event occurs at a timing asynchronous with the image capturing instruction timing, the event recording unit 13b displays the date and time information related to the current date and time acquired by the date and time acquisition unit 10 Event date and time information (for example, 20 ΔΔ year; ○ month ○ day; □ day of week, etc.) is recorded in association with incidental information of the event.
Here, the incidental information may be any information as long as it is information related to the event, but information specific to each event (event-specific related information) is preferable. In other words, event-specific related information is information related to the occurrence of an event, but is less relevant to imaging-related information than event position information or event date information, and is information specific to an event other than imaging. For example, various keywords or information peculiar to the event may be included.

In addition, the event recording unit 13b acquires event-related information (accompanying information) related to the operation event acquired by the first acquisition unit 12a based on the occurrence of the operation event, and associates with the operation event in the event recording table. Record in T.
Specifically, the event recording unit 13b is an event acquired by the first acquisition unit 12a as an operation event, for example, when an execution operation such as an outgoing call operation or an incoming call operation by the transmission / reception unit 7 is performed. Related information, that is, the type of operation such as a call operation or an incoming call operation, and a keyword related to a call partner related to the call operation or the incoming call operation are acquired. Then, the event recording unit 13b records the acquired keyword in the event recording table T as incidental information of the operation event.
In addition, the event recording unit 13b, for example, the event related acquired by the first acquisition unit 12a when an execution operation such as an email transmission operation or a reception confirmation operation by the communication control unit 8 is performed as an operation event. Information, that is, the type of operation such as a transmission operation or a reception confirmation operation, and a keyword related to a communication partner related to the transmission operation or the reception confirmation operation are acquired. Then, the event recording unit 13b records the acquired keyword in the event recording table T as incidental information of the operation event.
Further, the event recording unit 13b, for example, when an operation operation is performed by the user via the operation input unit 6, such as an input operation such as creation, editing, or deletion of a schedule or a document, or a browsing operation, etc. The event-related information acquired by the first acquisition unit 12a, that is, the schedule and the document type, the operation type such as the input operation and the browsing operation, and the keywords related to the schedule content and the document are acquired. Then, the event recording unit 13b records the acquired keyword in the event recording table T as incidental information of the operation event.

Further, the event recording unit 13b acquires event-related information (accompanying information) related to the operation unnecessary event acquired by the second acquisition unit 12c based on the occurrence of the operation unnecessary event, and associates it with the operation unnecessary event. Record in event record table T.
Specifically, for example, when the determination unit 12b determines that an operation unnecessary event has occurred by detecting a predetermined type of subject from the image information of the automatically captured image, the event recording unit 13b. Acquires the event related information acquired by the second acquisition unit 12c, that is, information related to the type of the subject, and records the acquired information in the event recording table T as incidental information of the operation unnecessary event.
In addition, for example, when the determination unit 12b determines that an operation-unnecessary event has occurred by detecting a specific word from the sound data collected automatically, the event recording unit 13b includes the second acquisition unit. The event related information acquired by 12c, that is, information related to the specific word is acquired, and the acquired information is recorded in the event recording table T as incidental information of the operation unnecessary event.
In addition, for example, when the position acquisition unit 9 automatically measures the current position of the apparatus main body and detects a specific location, the determination unit 12b determines that an operation unnecessary event has occurred. The unit 13b acquires the event related information acquired by the second acquisition unit 12c, that is, information related to the specific location, and records the acquired information in the event recording table T as incidental information of the operation unnecessary event.
Also, for example, when a predetermined type of motion is detected based on a detection signal automatically detected and output by the motion detection sensor 11a, and the determination unit 12b determines that an operation unnecessary event has occurred. The event recording unit 13b acquires the event-related information acquired by the second acquisition unit 12c, that is, the information related to the predetermined type of movement and the time and distance of the movement, and the acquired information does not require the operation. Recorded in the event recording table T as incidental information of the event.
In addition, for example, when the determination unit 12b determines that an operation-unnecessary event has occurred due to the body state detection sensor 11b automatically detecting the physical information of the user of the terminal body, the event recording unit 13b Event-related information acquired by the second acquisition unit 12c, that is, information related to the user's physical information and the time when the physical information is detected, is acquired, and the acquired information is used as incidental information for the operation-free event. Record in T.
Further, for example, an operation unnecessary event occurs by the determination unit 12b when a change in the external environment (weather) of the terminal body is detected based on a detection signal automatically detected and output by the environment detection sensor 11c. When it is determined that the event recording unit 13b has determined, the event recording unit 13b relates to the event related information acquired by the second acquisition unit 12c, that is, the external environment, the time when the change of the external environment is detected, the area where the terminal body exists, Information is acquired, and the acquired information is recorded in the event recording table T as incidental information of the operation unnecessary event.

Specifically, as shown in FIG. 3, the event record table T includes “August 3, 11:12” as the date and time of occurrence (event date information), and “ “Home”, “Communication” as an event type, and “E-mail, reception, friend A,...” As incidental information are associated with each other. ”,“ Home ”as the location of occurrence,“ Physical information acquisition ”as the type of event, and“ 36.3 ° C., blood pressure 123/82, pulse 87,... Also, “August 5, 16:20” as the date of occurrence, “Company” as the location of occurrence, “Document input” as the type of event, and “Plan meeting B minutes, creation as incidental information” , ... "and the date and time of occurrence “August 7, 15: 5”, “Home area” as the location of occurrence, “Motion detection” as the type of event, and “Jogging, 30 minutes, 4 km,... , “September 1st 14:08” as the date of occurrence, “Saitama Prefecture” as the location of occurrence, “weather change” as the type of event, and “snow as ancillary information” , −3 ° C.,... ”.
The event record table T is an example and is not limited to this. The contents of the “occurrence date and time”, “occurrence location”, “event type”, and “accompanying information” to be specified are arbitrarily selected. Can be changed.

  As described above, the event recording unit 13b uses the event related information related to the occurrence of the event as the event information related to the event other than the imaging of the image that occurs at a timing asynchronous with the imaging instruction timing of the image by the imaging unit 3. The second recording means is configured to record at least one in association with each other.

  Moreover, the recording unit 13 may be configured to include a recording medium such as a nonvolatile memory (flash memory), and in this case, may include a mechanism for attaching and detaching the recording medium.

  The search control unit 14 controls the search for image information and events recorded in the recording unit 13. Specifically, the search control unit 14 includes a specifying unit 14a and a search unit 14b.

When searching for image information as a search target, the specifying unit (specifying unit) 14a uses information other than imaging related information (such as shooting position information and imaging date / time information) as a search condition based on a predetermined operation of the operation input unit 6 by the user. When other information is input, event information related to the other information is searched from the event information recorded in the event recording table T of the event recording unit 13b, and is associated with the searched event information. The imaging related information related to the event related information being identified is specified.
Here, as other information, for example, various keywords related to events that are not related to image capturing may be mentioned, but although the relevance to imaging related information is low compared to event position information or event date information, Various keywords or the like (event-specific related information) unique to an event other than imaging may be used.
In addition, the specifying unit 14a determines the relevance between the event-related information and the imaging-related information by using the closeness related to the distance between the imaging position indicated by the imaging position information and the event occurrence position indicated by the event position information, or imaging It is specified based on the closeness related to the time between the imaging date / time indicated by the date / time information and the event occurrence date / time indicated by the event date / time information.

That is, when various keywords (for example, name, e-mail address, physical condition, weather, etc.) are input as search conditions, the specifying unit 14a refers to the event recording table T of the event recording unit 13b and Search for event information with related incidental information. For example, when the name of the keyword “friend A” is input as a search condition by the operation input unit 6, the event “communication” in which the “friend A” is recorded as incidental information is referred to the event recording table T. Search and identify.
Subsequently, the specifying unit 14a specifies imaging related information (shooting position information, shooting date information, etc.) related to event related information (event position information, event date information, etc.) associated with the searched event information. To do. That is, the specifying unit 14a, based on at least one of event position information and event date information associated with the searched event information, captures an image position and event occurrence date related to the event occurrence position. At least one of the imaging dates and times related to is specified.
Specifically, the specifying unit 14a, based on the event position information associated with the event information searched from the event record table T, the imaging position related to the event occurrence position, that is, the occurrence of the event An imaging position within a predetermined distance range (for example, the vicinity) from the position is specified. For example, when the event “communication” accompanied by “friend A” as the incidental information is identified, the identification unit 14a is based on the event position information “home” associated with the event “communication”. Then, “home surroundings” is specified as an imaging position in the vicinity of the event occurrence position.
Further, the specifying unit 14a is predetermined based on the shooting date / time related to the event occurrence date / time, that is, the event occurrence date / time, based on the event date / time information associated with the event information searched from the event record table T. The imaging date and time within the time range is specified. For example, when the event “communication” accompanied by “friend A” as the incidental information is identified, the identification unit 14a identifies the event date / time information “August 3 11” associated with the event “communication”. Based on “time 12 minutes”, “August 3” is specified as an imaging date and time that is close to the event occurrence date and time.

Further, when searching for event information as a search target, the specifying unit 14a receives other information other than event-related information as a search condition based on a predetermined operation of the operation input unit 6 by the user. Image information related to the information is searched from the image information recorded in the image recording unit 13a, and event related information related to the imaging related information associated with the searched image information is specified.
Here, as other information, for example, various keywords related to image capturing that are not related to the occurrence of an event can be cited, but the relevance to the event related information is lower than the image capturing position information or the image capturing date / time information. However, it may be various keywords or the like (imaging-specific related information) specific to image capturing.
That is, when various keywords (for example, humans, animals, etc.) are input as search conditions, the specifying unit 14a performs predetermined subject detection processing (for example, face detection) on the image information recorded in the image recording unit 13a. The image information including the subject related to the keyword is searched. Here, the subject detection process is substantially the same as the subject detection process performed by the determination unit 12b of the event acquisition control unit 12, and a detailed description thereof will be omitted.
Subsequently, the specifying unit 14a specifies event-related information (event position information, event date / time information, etc.) related to the image-related information (imaging position information, image date / time information, etc.) associated with the searched image information. To do. That is, the specifying unit 14a, based on at least one of the imaging position information and the imaging date / time information associated with the searched image information, the occurrence position of the event related to the imaging position of the image and the image At least one of the occurrence date and time of the event related to the imaging date and time is specified.
Specifically, the specifying unit 14a, based on the imaging position information attached as Exif information to the image information retrieved from the image recording unit 13a, the occurrence position of an event related to the imaging position of the image, That is, an event occurrence position within a predetermined distance range (for example, in the vicinity) from the image capturing position is specified. For example, when the image information with “home” attached thereto is input as the imaging position information, the specifying unit 14a generates an event occurrence position in the vicinity of the imaging position of the image based on the imaging position information “home”. As “Near home”.
Further, for example, the specifying unit 14a, based on the imaging date / time information attached as Exif information to the image information retrieved from the image recording unit 13a, the occurrence date / time of an event related to the imaging date / time of the image, The occurrence date and time of the event within a predetermined time range from the image capturing date and time is specified. For example, when the image information attached with “August 3 10:05” or the like is input as the imaging date / time information, the specifying unit 14a captures the imaging date / time information “August 3 10:05”. Based on the above, “August 3” is specified as the event occurrence date and time within a predetermined time range from the image capturing date and time.

In addition, when searching for event information as a search target, the specifying unit 14a specifies designation information (for example, a file name of the image information itself) as a search condition based on a predetermined operation of the operation input unit 6 by the user. Or various keywords related to the image information), event related information related to imaging related information (imaging position information, imaging date information, etc.) associated with the image information specified by the specified information Specify information (event location information, event date information, etc.).
Specifically, the specifying unit 14a searches the image information specified by the specified information from the image information recorded in the image recording unit 13a. At this time, when various keywords (for example, human and animal images) are input as search conditions, a predetermined subject detection process (for example, face detection process) is performed on the image information recorded in the image recording unit 13a. Etc.) to identify image information including a subject related to the keyword.
Thereafter, the specifying unit 14a relates to the imaging position of the image based on at least one of the imaging position information and the imaging date / time information associated with the image information searched from the image recording unit 13a. At least one of an event occurrence position and an event occurrence date and time related to an image capturing date and time is specified. The method for specifying the event occurrence position and the date and time of occurrence is the same as when other information other than event related information is input as a search condition based on a predetermined operation of the operation input unit 6 by the user. The detailed explanation is omitted.

In addition, when searching for event information as a search target, the specifying unit 14a is other than event-related information, designation information specifying image information, and imaging-related information as a search condition based on a predetermined operation of the operation input unit 6 by the user. Information (for example, various keywords and information associated with an event to be searched) is input, image information including a subject related to the information is recorded in the image recording unit 13a. Search from information. Specifically, when various keywords and information are input as search conditions, the specifying unit 14a performs predetermined subject detection processing (for example, face detection processing, etc.) on the image information recorded in the image recording unit 13a. ) To identify image information including a subject related to the keyword or information.
The specifying unit 14a relates to the imaging position of the image based on at least one of the imaging position information and the imaging date / time information associated with the image information retrieved from the image recording unit 13a. At least one of an event occurrence position and an event occurrence date and time related to an image capturing date and time is specified. The method for specifying the event occurrence position and the date and time of occurrence is the same as when other information other than event related information is input as a search condition based on a predetermined operation of the operation input unit 6 by the user. The detailed explanation is omitted.

When searching for image information as a search target, the search unit (search means) 14b uses a search condition based on a predetermined operation of the operation input unit 6 by the user in the image information recorded in the image recording unit 13a. Image information associated with the input imaging-related information (imaging position information, imaging date / time information, etc.) or the imaging-related information specified by the specifying unit 14a is searched.
That is, the search unit 14b is recorded in the image recording unit 13a when imaging related information is input as a search condition for searching image information based on a predetermined operation of the operation input unit 6 by the user. While searching for image information associated with the input imaging-related information in the image information, information other than the imaging-related information (for example, various keywords specific to the event) is used as a search condition. When the image is input, the image information associated with the imaging related information specified by the specifying unit 14a is searched from the image information recorded in the image recording unit 13a. Specifically, the search unit 14b, based on the imaging related information (imaging position information, imaging date information, etc.) in the image information recorded in the image recording unit 13a, At least one of the image information associated with at least one of the image information, that is, the image information associated with the imaging position according to the imaging position information and the image information associated with the imaging date / time according to the imaging date / time information. Search for image information.
For example, the search unit 14b searches the image recording unit 13a for image information associated with the imaging position information related to the imaging position as Exif information based on the imaging position information. In addition, the search unit 14b searches the image recording unit 13a for image information associated with the imaging date / time information related to the imaging date / time as Exif information based on the imaging date / time information.

Further, when searching for event information as a search target, the search unit 14b is based on a predetermined operation of the operation input unit 6 by the user among the event information recorded in the event information table T of the event recording unit 13b. Event related information (event location information, event date information, etc.) input as a search condition or event information associated with event related information specified by the specifying unit 14a is searched.
That is, when the event related information is input as a search condition for searching for event information based on a predetermined operation of the operation input unit 6 by the user, the search unit 14b stores the event information table T of the event recording unit 13b. While searching for event information associated with the input event-related information in the event information recorded in, other information other than event-related information (related to the subject in the image) as a search condition In the event information table T of the event recording unit 13b, the event associated with the event related information specified by the specifying unit 14a is input. Search for information. In addition, the search unit 14b receives the designation information (for example, the file name of the image information itself, various keywords related to the image information, etc.) for specifying the image information as a search condition, event related information, image information Similarly, when information other than designation information for designating and imaging related information (for example, various keywords reminiscent of an event to be searched) is input, the event information table T of the event recording unit 13b is also input. The event information associated with the event related information specified by the specifying unit 14a is searched from among the event information recorded in the event information.
Specifically, the search unit 14b, based on event related information (event location information, event date information, etc.) among event information recorded in the event recording unit 13b, At least one of image information associated with at least one, that is, event information associated with an occurrence position related to event position information and event information associated with an occurrence date related to event date information Search event information.
For example, the search unit 14b, based on the event position information, associates event location information (occurrence location) related to the occurrence location (eg, “home neighborhood”) with an event (eg, “communication”, “ “Physical information acquisition” or the like) is searched in the event record table T. In addition, the search unit 14b, based on the event date / time information, associates event date / time information (occurrence date / time) related to the occurrence date / time (for example, “August 3”, etc.) (for example, “communication”). Etc.) in the event record table T.

The display unit 15 includes a display panel 15a and a display control unit 15b.
That is, the display control unit 15b performs control to read display image information temporarily stored in the buffer memory and display it on the display screen of the display panel 15a. Specifically, the display control unit 15b includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. Then, the digital video encoder reads the luminance signal Y and the color difference signals Cb and Cr of each frame image read from the buffer memory and stored in the VRAM (not shown) from the VRAM through the VRAM controller to a predetermined reproduction frame. The data is read at a rate (for example, 30 fps), and a video signal is generated based on these data and output to the display panel 15a.

  The display panel 15a is, for example, a liquid crystal display panel, and displays an image captured by the electronic imaging unit 3b on the display screen based on a video signal from the display control unit 15b. That is, the display panel 15a displays a live view image while sequentially updating a plurality of frame images generated by imaging a subject at a predetermined frame rate. In addition, the display panel 15a displays a plurality of frame images being recorded as moving images, or displays a rec view image of an image recorded as a still image.

Further, the display unit 15 outputs a search target of the search unit 14b of the search control unit 14, that is, a search result of image information or event information, as an output unit. That is, the display control unit 15b generates image information on a search result screen (not shown) on which candidate images and candidate events are represented according to the search result of the search target by the search unit 14b, and outputs the image information to the display panel 15a. The display panel 15a displays a search result screen corresponding to the image information output and input from the display control unit 15b.
Here, the display order of candidate images and candidate events on the search result screen is, for example, a predetermined operation of the operation input unit 6 by the user such as an order in which priority is given to the distance from the current position, an order in which time from the current position is to be given priority, or the like. Switching display is possible based on.

Next, imaging processing by the mobile terminal 100 will be described with reference to FIG.
FIG. 4 is a flowchart illustrating an example of an operation related to the imaging process.

As shown in FIG. 4, first, the CPU of the central control unit 1 selects an imaging mode based on a predetermined operation of the operation input unit 6 by a user from a plurality of operation modes, and operates according to the imaging mode. After setting each part in the state (step S1), it is determined whether an imaging instruction is input based on a predetermined operation of the shutter button of the operation input unit 6 by the user (step S2).
The determination in step S2 is repeatedly executed until it is determined that an imaging instruction has been input (step S2; YES).

When it is determined in step S2 that an imaging instruction has been input (step S2; YES), the imaging control unit 4 changes the imaging condition of the imaging unit 3 to a predetermined imaging condition (for example, “aperture value”, “ (Exposure time ”,“ ISO sensitivity ”,“ photometry method ”, etc.) (step S3), and controls the operations of the lens unit 3a, electronic imaging unit 3b, and unit circuit unit 3c according to the imaging conditions to image the subject. The image processing unit 5 generates image information of a captured image of the subject (step S4).
Subsequently, the position acquisition unit 9 measures the absolute two-dimensional current position (latitude, longitude) of the apparatus main body based on the GPS signal output from the GPS antenna 9a, and acquires position information ( For example, latitude and longitude coordinate information) is acquired (step S5), and the date acquisition unit 10 specifies today's date, day of the week, etc. from the calendar information based on the current time measured by the time measurement unit, Based on the current time, today's date, and the like, date information relating to the current date is generated and acquired (step S6).
Note that the order of the current position acquisition process in step S5 and the current date acquisition process in step S6 may be reversed.

Thereafter, the image recording unit 13a acquires the image information generated by the image processing unit 5, the imaging conditions set by the imaging control unit 4, the current position (imaging position) acquired by the position acquisition unit 9, and the date and time acquisition. The current date and time (imaging date and time) acquired by the unit 10 is associated and recorded as an Exif format image file (step S7).
That is, the image recording unit 13a sets the position information acquired by the position acquisition unit 9 when the image is captured by the imaging unit 3 as imaging position information (for example, place name and coordinate information) regarding the imaging position of the image. The date and time information acquired by the date and time acquisition unit 10 is recorded in association with the image information of the image as imaging date and time information (for example, 20ΔΔ year; To do.
Thereby, the imaging process is terminated.

Next, event recording processing by the mobile terminal 100 will be described with reference to FIG.
FIG. 5 is a flowchart showing an example of an operation related to the event recording process.

As shown in FIG. 5, first, the CPU of the central control unit 1 determines whether or not the event recording function for recording an event that occurs at a timing asynchronous with the image capturing instruction timing by the user is effective ( Step S11). That is, the CPU of the central control unit 1 determines, for example, whether the “event recording mode” is set and the event recording function is executed based on a predetermined operation of the operation input unit 6 by the user.
The determination in step S11 is repeatedly executed until it is determined that the event recording function is valid (step S11; YES).

If it is determined in step S11 that the event recording function is valid (step S11; YES), the event acquisition control unit 12 determines whether any one of various events has occurred. (Step S12).
Specifically, the event acquisition control unit 12 performs, for example, incoming / outgoing calls, transmission / reception of e-mails, input of schedules and various documents, based on a predetermined execution instruction operation of the operation input unit 6 by the user. An operation event related to the operation, an operation executed without requiring a predetermined execution instruction operation of the operation input unit 6 by the user, for example, detection of the movement and posture of the terminal body by the detection sensor unit 11, It is determined whether or not an operation-unnecessary event related to an operation such as detection of a physical state or detection of an external environment (weather) of the terminal body has occurred.

If it is determined in step S12 that any one event has not occurred (step S12; NO), the CPU of the central control unit 1 returns the process to step S11 and controls the execution of the subsequent processes. To do.
When it is determined in step S12 that any one event has occurred (step S12; YES), the event acquisition control unit 12 outputs an output signal from the operation input unit 6 or a unit circuit of the imaging unit 3. Based on the output signal from the unit 3c, the output signal from the data conversion unit 7c of the transmission / reception unit 7, and the output signals from the motion detection sensor 11a, the body condition detection sensor 11b, and the environment detection sensor 11c of the detection sensor unit 11. The type of event is specified (step S13).

Subsequently, the first acquisition unit 12a and the second acquisition unit 12c of the event acquisition control unit 12 acquire event-related information related to the identified event as incidental information (step S14).
Specifically, the first acquisition unit 12a acquires event related information (accompanying information) related to the operation event. For example, when an operation operation such as an outgoing call operation or an incoming call operation by the transmission / reception unit 7 is performed as an operation event, the first acquisition unit 12a displays the type of operation such as an outgoing call operation or an incoming call operation, A keyword (for example, a name, an address, a telephone number, etc.) related to a call partner related to a call operation or an incoming call operation is acquired by referring to an address book (not shown).
The second acquisition unit 12c acquires event-related information (accompanying information) related to the operation unnecessary event. That is, when the determination unit 12b determines that an operation unnecessary event has occurred, the second acquisition unit 12c uses event-related information related to the operation unnecessary event (for example, a motion detection event) as incidental information of the event. get. For example, a predetermined type of motion is detected by the determination unit 12b based on a detection signal that is automatically detected and output at a predetermined timing by the motion detection sensor 11a of the detection sensor unit 11 as an operation unnecessary event. When it is determined that the operation-unnecessary event has occurred, the second acquisition unit 12c determines the predetermined type of movement (eg, “sleeping”, “study”, “exercising”, etc.) Information related to the time and distance of movement is acquired.
The movement time includes, for example, a start time, an end time, an elapsed time, etc. measured by a timer. Moreover, as the distance of movement, for example, the distance converted at a predetermined conversion rate according to the elapsed time, or the distance calculated from the current position at the start and end of the movement acquired by the position acquisition unit 9 Etc.

Next, the position acquisition unit 9 measures the absolute two-dimensional current position (latitude, longitude) of the apparatus body based on the GPS signal output from the GPS antenna 9a, and position information ( For example, the coordinate information of latitude and longitude) is acquired (step S15), and the date and time acquisition unit 10 specifies today's date and day of the week from the calendar information based on the current time measured by the clock unit, Based on the current time and today's date, date information related to the current date is generated and acquired (step S16).
Note that the order of the current position acquisition process in step S15 and the current date acquisition process in step S16 may be reversed.

Thereafter, the event recording unit 13b acquires the event type, the incidental information acquired by the first acquisition unit 12a or the second acquisition unit 12c, the current position (occurrence position) acquired by the position acquisition unit 9, and the date and time acquisition. The current date and time (occurrence date and time) acquired by the unit 10 is associated and recorded in the event recording table T (step S17).
That is, the event recording unit 13b uses the position information acquired by the position acquisition unit 9 when a predetermined event (such as an operation event or an operation unnecessary event) occurs, as event position information (for example, a place name or The date / time information acquired by the date / time acquisition unit 10 as event date / time information (for example, 20ΔΔ year; It records in association with such incidental information.
As a result, the event recording process ends.

Next, image search processing by the portable terminal 100 will be described with reference to FIG.
FIG. 6 is a flowchart illustrating an example of an operation related to the image search process.
The image search process described below is a process executed when the image search mode is set based on a predetermined operation of the operation input unit 6 by the user.

As shown in FIG. 6, first, the CPU of the central control unit 1 receives a search condition for searching for a search object based on a predetermined operation of the operation input unit 6 by the user (step S21). The control unit 14 determines whether or not at least one of the imaging date and time, the imaging position, and the imaging condition is specified as the search condition (step S22).
Here, if it is determined that at least one of the imaging date and time, the imaging position, and the imaging condition is designated (step S22; YES), the search unit 14b searches for the image information recorded in the image recording unit 13a. In particular, based on any one of the imaging date / time, the imaging position, and the imaging condition specified as the search condition, image information that matches the search condition is searched and specified as a candidate image (step S23).

Next, the search control unit 14 determines whether keywords other than the imaging date, imaging position, and imaging conditions are specified as the search conditions (step S24). Even when it is determined in step S22 that it is not determined that at least one of the imaging date and time, the imaging position, and the imaging condition is designated (step S22; NO), the search control unit 14 The process proceeds to step S24 to determine whether or not a keyword is specified (step S24).
If it is determined that a keyword is specified (step S24; YES), the specifying unit 14a specifies an event associated with the specified keyword and its accompanying information. Specifically, when various keywords (for example, name, e-mail address, physical condition, weather, etc.) are input as search conditions, the specifying unit 14a refers to the event recording table T of the event recording unit 13b, An event (for example, “communication” or the like) related to the keyword (for example, “friend A”) and its accompanying information are specified (step S25).
Subsequently, the specifying unit 14a relates to the specified event (for example, “communication” or the like) and the occurrence position (for example, “home” or the like) related to the event position information associated with the incidental information or the event date / time information. The date and time of occurrence (for example, “August 3, 11:12” or the like) is specified (step S26). Furthermore, the specifying unit 14a captures an imaging position within a predetermined distance range from the event occurrence position (eg, “around home”) and an imaging date / time within a predetermined time range from the event occurrence date (eg, “August 3”). At least one of “day” or the like) (step S27).

  After that, the search unit 14b identifies the image information attached to the imaging position information related to the imaging position specified by the specifying unit 14a as Exif information among the image information recorded in the image recording unit 13a. Image information associated with the imaging date and time information related to the imaging date and time as Exif information is searched and specified as a candidate image (step S28).

Next, the display control unit 15b generates image information of a search result screen on which candidate images are represented according to the search result of the search target by the search unit 14b, outputs the image information to the display panel 15a, and displays the search result screen. (Step S29). Even when it is determined in step S24 that no keyword is specified (step S24; NO), the CPU of the central control unit 1 moves the process to step S29, and the display control unit 15b A search result screen is displayed on the display panel 15a (step S29).
That is, when it is determined in step S24 that a keyword is designated (step S24; YES), the display unit 15 displays the image information searched based on the keyword by the processing in steps S25 to S28. On the other hand, when a search result screen as a candidate image is displayed and it is determined in step S24 that no keyword is specified (step S24; NO), the display unit 15 performs steps S25 to S28. Without passing through the processing, a search result screen is displayed in which the image information searched based on any one of the imaging date / time, the imaging position and the imaging condition specified as the search condition is used as a candidate image (step S29). .

Subsequently, the CPU of the central control unit 1 determines whether an instruction to end the image search process is input based on a predetermined operation of the operation input unit 6 by the user (step S30).
Here, if it is determined that the image search processing end instruction has not been input (step S30; NO), the CPU of the central control unit 1 returns the processing to step S21. Control execution of processing.
On the other hand, if it is determined that an instruction to end the image search process has been input (step S30; YES), the CPU of the central control unit 1 ends the image search process.

Next, event search processing by the mobile terminal 100 will be described with reference to FIG.
FIG. 7 is a flowchart illustrating an example of an operation related to event search processing.
The event search process described below is a process executed when an event search mode is set based on a predetermined operation of the operation input unit 6 by the user.

As shown in FIG. 7, first, the CPU of the central control unit 1 receives a search condition for searching for a search object based on a predetermined operation of the operation input unit 6 by the user (step S31). The control unit 14 determines whether or not at least one of the event type, the incidental information, the occurrence date and time, and the occurrence position is designated as the search condition (step S32).
Here, if it is determined that at least one of the event type, the incidental information, the occurrence date and time, and the occurrence position is designated (step S32; YES), the search unit 14b searches the event recording table T of the event recording unit 13b. , Based on any one of the event type, incidental information, occurrence date / time, and occurrence position specified as the search condition, the event that matches the search condition is searched and identified as a candidate event (step S33).

Next, the search control unit 14 determines whether an image is designated based on designation information such as image information and keywords as a search condition (step S34). In addition, when it is determined in step S32 that it is not determined that at least one of the event type, the incidental information, the occurrence date and time, and the occurrence position is specified (step S32; NO), the search control unit 14 shifts the processing to step S34, and determines whether or not an image is designated (step S34).
If it is determined that an image is designated (step S34; YES), the specifying unit 14a adds Exif to the image information of the designated image among the image information recorded in the image recording unit 13a. The imaging position (for example, “home”, etc.) related to the imaging position information attached as information and the imaging date / time (for example, “August 3, 10: 5”, etc.) related to the imaging date / time information are specified (step S35). ). Furthermore, the specifying unit 14a generates an event occurrence position (for example, “around the home”) within a predetermined distance range from the image capturing position and an event occurrence date / time (for example, a predetermined time range from the image capturing date / time). At least one of “August 3” or the like) is identified (step S36).

  Thereafter, the search unit 14b refers to the event recording table T of the event recording unit 13b, relates to the event associated with the event position information related to the occurrence position identified by the identification unit 14a, and the identified occurrence date and time. The event associated with the event date information is searched and specified as a candidate event (step S37).

Next, the display control unit 15b generates image information of a search result screen in which candidate events are represented according to the search result of the search target by the search unit 14b, outputs the image information to the display panel 15a, and displays the search result screen. (Step S38). In addition, when it is determined in step S34 that an image is not designated (step S34; NO), the CPU of the central control unit 1 moves the process to step S38, and the display control unit 15b A search result screen is displayed on the display panel 15a (step S38).
That is, if it is determined in step S34 that an image has been designated (step S34; YES), the display unit 15 selects an event searched based on the image through the processing in steps S35 to S37. On the other hand, when the search result screen as an event is displayed and it is determined in step S34 that no keyword is specified (step S34; NO), the display unit 15 performs the processing of steps S35 to S37. A search result screen is displayed in which the event searched based on any one of the event type, incidental information, occurrence date and time and occurrence position specified as the search condition without passing through is set as a candidate event (step S38). ).

Subsequently, the CPU of the central control unit 1 determines whether or not an instruction to end the event search process has been input based on a predetermined operation of the operation input unit 6 by the user (step S39).
Here, if it is determined that the end instruction for the event search process has not been input (step S39; NO), the CPU of the central control unit 1 returns the process to step S31, and after step S31, as described above. Control execution of processing.
On the other hand, if it is determined that an instruction to end the event search process has been input (step S39; YES), the CPU of the central control unit 1 ends the event search process.

As described above, according to the mobile terminal 100 of the first embodiment, when searching for image information as a search target, other information other than imaging related information as a search condition based on a predetermined operation of the operation input unit 6 by the user. Is input, the identifying unit 14a searches for event information related to the other information from the event information recorded in the event recording unit 13b, and is associated with the retrieved event information. A search condition that specifies imaging-related information related to event-related information, and is input by the search unit 14b based on a predetermined operation of the operation input unit 6 by the user in the image information recorded in the image recording unit 13a. Image information associated with the imaging related information specified by the specifying unit 14a or the imaging related information specified by the specifying unit 14a, and the search result is output from the display unit 15. Therefore, when imaging related information is input as a search condition, image information associated with the input imaging related information can be searched from the image information recorded in the image recording unit 13a. On the other hand, even when information other than the imaging related information is input as the search condition, the imaging related information specified by the specifying unit 14a in the image information recorded in the image recording unit 13a. It is possible to search for image information associated with.
In other words, when remembering when and where the search target image was taken, not only can the image information be searched using the imaging related information such as the imaging position and the imaging date and time as a search condition, but also when the search target image is Even when it is impossible to accurately remember where the image was taken, a search condition (for example, various keywords) that is not related to the image information is input to search for event information related to the search condition. Event-related information that is associated with the retrieved event information by using the imaging-related information recorded in the image recording unit 13a and the event-related information recorded in the event recording unit 13b. The imaging related information related to the information can be specified. Thereby, the search of the image information matched with the specified imaging related information can be performed appropriately.
In this way, even if you cannot remember the title name, imaging date and time, location, etc. of the image information, if you can remember any part of the related information, you can use it as a keyword to associate the desired image information The search can be performed by various methods, and the operability in the search process of the image information can be improved to speed up the search process. In particular, even a device that records an enormous amount of images and media information for life recorder use, such as recording almost all of life's images and moving images, is recorded. A user-desired image can be searched relatively easily even from among a large amount of past image information.
Therefore, it is possible to appropriately search for desired image information by using a search condition that is not related to the imaging related information as well as the imaging related information.

Further, the relevance between the event related information and the imaging related information is the approximation of the distance between the imaging position indicated by the imaging position information and the event occurrence position indicated by the event position information, and the imaging date and time indicated by the imaging date information. And the occurrence date and time of the event indicated by the event date and time information are specified based on the closeness related to the time, so that the imaging related information related to the event related information is specified, that is, within a predetermined distance range from the event occurrence position. It is possible to appropriately specify the imaging position and the imaging date and time within a predetermined time range from the event occurrence date and time.
Further, based on at least one of the event position information and event date / time information associated with the event information retrieved from the event recording unit 13b, the imaging position and the event At least one of the imaging date and time related to the occurrence date and time is specified, and the image information recorded in the image recording unit 13a is associated with at least one of the specified imaging position and imaging date and time. Since the image position information and image date / time information recorded in the image recording unit 13a and the event position information and event date / time information recorded in the event recording unit 13b are mutually used, the image information is searched. Information can be searched efficiently.

  Further, when searching for image information as a search target, if event-specific related information (for example, various keywords specific to an event) is input as a search condition, event information related to the event-specific related information is Since the event information recorded in the recording unit 13b is searched and the imaging related information related to the event related information associated with the searched event information is specified, the event position information or the event date / time information Even if event-specific related information unique to an event other than imaging is input as a search condition, the event information related to the event-specific related information can be searched even if the relationship with the imaging-related information is low. As a result, event-related information associated with the retrieved event information by mutually using the imaging-related information recorded in the image recording unit 13a and the event-related information recorded in the event recording unit 13b. Imaging-related information related to the image information can be specified, and image information associated with the specified imaging-related information can be appropriately searched.

Further, when searching for event information as a search target, if information other than event-related information is input as a search condition, the specifying unit 14a displays image information related to the other information in the image recording unit 13a. The event related information related to the imaging related information associated with the searched image information is identified from the inside, and the event recorded in the event information table T of the event recording unit 13b by the search unit 14b In the information, search for event related information as a search condition input based on a predetermined operation of the operation input unit 6 by the user or event information associated with the event related information specified by the specifying unit 14a. Since the search result is output from the display unit 15, when event-related information is input as a search condition, the event recording unit While it is possible to search the event information associated with the input event related information among the event information recorded in 3b, other information other than the event related information is input as a search condition. Even in this case, the event information associated with the event-related information specified by the specifying unit 14a can be searched from the event information recorded in the event recording unit 13b.
In other words, if you remember when and where the search target event occurred, you can search for event information using event-related information such as the location and date of the search as search criteria, and when the search target event Even if you cannot remember exactly where it occurred, you can search for image information related to the search condition by entering search conditions that are not related to the event information (for example, various keywords). The imaging-related information that is associated with the searched image information by using the imaging-related information recorded in the image recording unit 13a and the event-related information recorded in the event recording unit 13b. Event related information related to the information can be specified. Thereby, it is possible to appropriately search for event information associated with the specified event-related information.
Therefore, it is possible to appropriately search for desired event information by using not only the event related information but also search conditions that are not related to the event related information.

  In addition, based on at least one of the imaging position information and the imaging date / time information associated with the image information retrieved from the image recording unit 13a, the occurrence position of the event related to the imaging position of the image and At least one of the event occurrence dates and times related to the image capturing date and time is specified, and at least one of the specified occurrence position and occurrence date and time in the event information recorded in the event recording unit 13b Since the associated event information is searched, the imaging position information and imaging date / time information recorded in the image recording unit 13a and the event position information and event date / time information recorded in the event recording unit 13b are mutually used. Thus, the image information can be searched efficiently.

In addition, when searching for event information as a search target, if specification information specifying image information is input as a search condition, the specified specification information entered in the event information recorded in the event recording unit 13b is used. Event-related information related to the imaging-related information associated with the specified image information is identified, so even if you cannot remember exactly when and where the search target event occurred, event information The image-related information recorded in the image recording unit 13a and the event-related information recorded in the event recording unit 13b are input by specifying designation information for designating image information that is not related to the image as a search condition. The event-related information related to the imaging-related information associated with the image information specified by the specified information is used mutually. Can.
Specifically, the occurrence of an event related to the imaging position of the image based on at least one of imaging position information and imaging date / time information associated with the image information specified by the input specification information At least one of the event occurrence date and time related to the position and the image capturing date and time is specified, and among the event occurrence date and occurrence date and time specified in the event information recorded in the event recording unit 13b Event information associated with at least one of the image capturing position information and image capturing date / time information recorded in the image recording unit 13a, and event position information and event date / time information recorded in the event recording unit 13b. It is possible to efficiently search for event information by mutually using.

  In addition, when searching for event information as a search target, information other than event-related information, designation information specifying image information, and imaging-related information as search conditions (for example, various keywords associated with an event to be searched) Or the like) is input, image information including a subject related to the information is searched from the image recording unit 13a, and among the imaging position information and imaging date / time information associated with the specified image information. Based on at least one of the information, at least one of the event occurrence position related to the image capture position and the event occurrence date related to the image capture date is specified and recorded in the event recording unit 13b. Event information associated with at least one of the occurrence position and the occurrence date and time of the identified event Therefore, even if information unrelated to the event information is input as a search condition, the image information recorded in the image recording unit 13a is specified after specifying the image information including the subject related to the information. The event information can be efficiently searched by mutually using the position information and imaging date / time information and the event position information and event date / time information recorded in the event recording unit 13b.

  Furthermore, when searching for event information as a search target, if imaging-specific related information (for example, various keywords specific to image capturing) is input as a search condition, image information related to the imaging-specific related information Is retrieved from the image information recorded in the image recording unit 13a, and event-related information related to imaging-related information associated with the searched image information is specified. Image information related to the imaging-related related information can be searched even if imaging-related related information specific to imaging of an image is input as a search condition. . As a result, the imaging related information associated with the searched image information by mutually using the imaging related information recorded in the image recording unit 13a and the event related information recorded in the event recording unit 13b. Event-related information related to the event information can be specified, and event information associated with the specified event-related information can be appropriately searched.

In addition, since event related information related to an operation event related to an operation executed based on a predetermined execution instruction operation of the operation input unit 6 by the user is acquired and recorded in the event recording unit 13b, the event related information The image information can be properly searched by using.
Specifically, when image information is a search target, a call execution operation as an operation event and a keyword related to the call partner, a communication execution operation as an operation event, a keyword and an operation related to the communication partner Of the input operation of the schedule as an event and the keywords included in the contents of the schedule, the input operation of the document as an operation event and the keywords related to the document, at least one keyword can be used as the event related information, Searching for desired image information and event information can be performed appropriately based on various keywords related to operation events not related to image information.

Further, when an operation unnecessary event related to an operation that is executed without requiring a predetermined execution instruction operation of the operation input unit 6 by the user occurs, event related information related to the operation unnecessary event is acquired. Since it is recorded in the event recording unit 13b, it is possible to appropriately search for image information using the event-related information.
Specifically, when a predetermined type of subject is detected from image information of an image captured at a predetermined timing, information related to the type of the subject may be acquired as event related information related to the event. it can.
In addition, when a specific word is detected from sound information collected at a predetermined timing, information related to the specific word can be acquired as event-related information related to the event.
In addition, when a specific location is detected based on the current position of the terminal body measured at a predetermined timing, information related to the name related to the specific location is used as event related information related to the event. Can be acquired.
In addition, when a predetermined type of motion is detected based on the automatically detected motion information, information related to the predetermined type of motion can be acquired as event related information related to the event.
Therefore, it is possible to appropriately search for desired image information and event information using the above-described various information acquired as event-related information.

[Embodiment 2]
FIG. 8 is a diagram showing a schematic configuration of a search system 200 according to the second embodiment to which the present invention is applied.
As illustrated in FIG. 8, in the search system 200, an imaging device 300, a mobile phone 400, a search device 500, a map server 600, and a weather server 700 are connected via a predetermined communication network N. Then, the search device 500 records the image information and event information acquired by the imaging device 300 and the mobile phone 400, and searches the search information desired by the user in the image information and event information.

First, the imaging apparatus 300 will be described with reference to FIG.
FIG. 9 is a block diagram illustrating a schematic configuration of the imaging apparatus 300.
As illustrated in FIG. 9, the imaging apparatus 300 includes a central control unit 301, a memory 302, an imaging unit 303, an imaging control unit 304, an image processing unit 305, an operation input unit 306, a display unit 307, a communication control unit 308, and position acquisition. A unit 309, a date acquisition unit 310, a recording unit 311, and the like.
The central control unit 301, the memory 302, the imaging unit 303, the imaging control unit 304, the image processing unit 305, the operation input unit 306, the display unit 307, the communication control unit 308, the position acquisition unit 309, the date acquisition unit 310, and the recording unit 311 has substantially the same configuration as each corresponding part of the mobile terminal 100 of the first embodiment, and only different parts from the first embodiment will be described in detail.

  The central control unit 301 controls each unit of the imaging device 300. Specifically, the central control unit 301 includes a CPU (not shown) that controls each part of the imaging apparatus 300, and performs various control operations according to various processing programs (not shown).

  The memory 302 includes, for example, a buffer memory that temporarily records image information and the like, a working memory such as the central control unit 301, a program memory that stores various programs and data related to the functions of the imaging device 300, and the like (all of which (Not shown).

  The imaging unit 303, the imaging control unit 304, and the image processing unit 305 have substantially the same configuration as the imaging unit 3, the imaging control unit 4, and the image processing unit 5 of the mobile terminal 100 of the first embodiment, and detailed description thereof is omitted. To do.

The operation input unit 306 is for inputting various instructions to the imaging apparatus 300 main body.
Specifically, the operation input unit 306 includes a shutter button related to an instruction to shoot a subject, various buttons (not shown) such as up / down / left / right cursor buttons and a determination button related to a selection instruction of a mode, a function, and the like. Yes.
When various buttons are operated by the user, the operation input unit 306 outputs an operation instruction corresponding to the operated button to the central control unit 301. The central control unit 301 causes each unit to execute a predetermined operation (for example, imaging of a subject) in accordance with an operation instruction output from the operation input unit 306 and input.

  The display unit 307 includes a display panel 15a and a display control unit 15b, similar to the display unit 15 of the mobile terminal 100 of the first embodiment.

The communication control unit 308 transmits and receives data via the communication network N and the communication antenna 308a.
That is, the communication control unit 308 includes, for example, a wireless LAN module, and transmits and receives data to and from an access point (not shown) connected to the communication network N via the communication antenna 308a. That is, the communication control unit 308 sends an Exif format image file recorded in the image recording unit 311 a to the communication partner search device 500 based on the instruction signal output from the central control unit 301 and input. Send.
Note that the communication control unit 308 may transmit and receive data according to a predetermined communication method, similarly to the communication control unit 8 of the mobile terminal 100 of the first embodiment.

Similar to the position acquisition unit 9 of the mobile terminal 100 of the first embodiment, the position acquisition unit 309 acquires position information related to the current position of the imaging apparatus 300 main body based on the GPS signal received by the GPS antenna 9a. To do.
Further, the position acquisition unit 309 outputs position information generated at the time of image capturing to the recording unit 311, and is associated with image information as imaging position information (imaging related information) regarding the image capturing position of the image. It is recorded in the image recording unit 311a.

The date acquisition unit 310 generates and acquires date information related to the current date, similar to the date acquisition unit 10 of the mobile terminal 100 of the first embodiment.
Further, the date acquisition unit 310 outputs date information generated at the time of image capturing to the recording unit 311, and is associated with image information as image capturing date information (imaging related information) regarding the image capturing date of the image. It is recorded in the image recording unit 311a.

The recording unit 311 includes an image recording unit 311a, and reads data from the image recording unit 311a and writes data to the image recording unit 311a.
Similar to the portable terminal 100 of the first embodiment, the image recording unit 311a records at least one piece of image information as an Exif format image file, and captures image-related information (capturing position information and imaging date / time information related to image capturing). Etc.) is recorded in correspondence with the image information as Exif information.

The imaging apparatus 300 configured as described above performs an imaging process (see FIG. 4).
The imaging process is substantially the same as the contents of the imaging process performed by the mobile terminal 100 of the first embodiment, and a detailed description thereof is omitted.

Next, the mobile phone 400 will be described with reference to FIG.
FIG. 10 is a block diagram showing a schematic configuration of the mobile phone 400.
As shown in FIG. 10, a mobile phone 400 includes a central control unit 401, a memory 402, a transmission / reception unit 403, a communication control unit 404, an operation input unit 405, a display unit 406, a detection sensor unit 407, an event acquisition control unit 408, A position acquisition unit 409, a date acquisition unit 410, a recording unit 411, and the like are provided.
The central control unit 401, the memory 402, the transmission / reception unit 403, the communication control unit 404, the operation input unit 405, the display unit 406, the detection sensor unit 407, the event acquisition control unit 408, the position acquisition unit 409, the date acquisition unit 410, The recording unit 411 has substantially the same configuration as the corresponding units of the mobile terminal 100 of the first embodiment, and only the parts different from the first embodiment will be described in detail.

  The central control unit 401 controls each unit of the mobile phone 400. Specifically, the central control unit 401 includes a CPU (not shown) that controls each part of the mobile phone 400, and performs various control operations according to various processing programs (not shown).

  The memory 402 is, for example, a buffer memory that temporarily records image information, a working memory such as the central control unit 401, a program memory that stores various programs and data related to the functions of the mobile phone 400, etc. (Not shown).

  The transmitter / receiver unit 403 has substantially the same configuration as the transmitter / receiver unit 7 of the portable terminal 100 of the first embodiment, and a detailed description thereof is omitted.

Similar to the communication control unit 8 of the mobile terminal 100 of the first embodiment, the communication control unit 404 transmits and receives data through the communication network N and the communication antenna 8a.
That is, the communication control unit 404 wirelessly uses a communication channel set in this communication method according to a communication protocol corresponding to a predetermined communication method adopted by the mobile phone 400 for communication with a wireless base station (not shown). Data is transmitted to and received from the base station via the communication antenna 8a. That is, the communication control unit 404, based on the instruction signal output from the central control unit 401 and input, incidental information of various events recorded in the event record table T for the communication partner search device 500, Send event related information.
Note that the communication control unit 404 may be configured by, for example, a wireless LAN module, and may exchange data with an access point (not shown) connected to the communication network N via the communication antenna 8a.

The operation input unit 405 is used to input various instructions to the mobile phone 400 main body.
Specifically, the operation input unit 405 includes up / down / left / right cursor buttons and a determination button related to a mode / function selection instruction, a communication-related button related to an execution instruction such as incoming / outgoing calls, transmission / reception of an e-mail, text ( Various buttons (none of which are shown) such as numeric buttons and symbol buttons related to the input instruction of (document).
When various buttons are operated by the user, the operation input unit 405 outputs an operation instruction corresponding to the operated button to the central control unit 401. The central control unit 401 causes each unit to execute a predetermined operation (for example, incoming / outgoing calls, transmission / reception of e-mails, etc.) in accordance with an operation instruction output from the operation input unit 405 and input.

  The display unit 406 includes a display panel 15a and a display control unit 15b, like the display unit 15 of the mobile terminal 100 of the first embodiment.

  The detection sensor unit 407 includes a motion detection sensor 11a, a body state detection sensor 11b, and an environment detection sensor 11c, similarly to the detection sensor unit 11 of the mobile terminal 100 of the first embodiment.

The event acquisition control unit 408 includes a first acquisition unit 12a, a determination unit 12b, and a second acquisition unit 12c, similar to the event acquisition control unit 12 of the mobile terminal 100 of the first embodiment.
Note that since the mobile phone 400 according to the second embodiment is configured not to include an imaging unit, for example, it is automatically performed at a predetermined timing without requiring a predetermined imaging instruction operation of the operation input unit 405 by the user. An operation unnecessary event for capturing an image is not performed, and processing related to the operation unnecessary event is not performed by the determination unit 12b and the second acquisition unit 12c.

Similar to the position acquisition unit 9 of the mobile terminal 100 of the first embodiment, the position acquisition unit 409 acquires position information related to the current position of the mobile phone 400 based on the GPS signal received by the GPS antenna 9a. To do.
Further, the position acquisition unit 409 outputs the position information generated at the time of occurrence of the event to the recording unit 411, and is associated with the event information as event position information (event related information) regarding the occurrence position of the event. It is recorded in the event recording unit 411b.

The date acquisition unit 410 generates and acquires date information related to the current date, similar to the date acquisition unit 10 of the mobile terminal 100 of the first embodiment.
Further, the date acquisition unit 410 outputs the date information generated when the event occurs to the recording unit 411, and is associated with the event information as event date information (event related information) related to the event occurrence date. It is recorded in the event recording unit 411b.

The recording unit 411 includes an event recording unit 411b, and reads data from the event recording unit 411b and writes data to the event recording unit 411b.
Similar to the portable terminal 100 of the first embodiment, the event recording unit 411b associates event information with event-related information (event position information, event date / time information, etc.) related to the occurrence of the event in the event recording table T. Record at least one.

The mobile phone 400 configured as described above performs an event recording process (see FIG. 5).
The event recording process is substantially the same as the content of the event recording process performed by the mobile terminal 100 of the first embodiment, and a detailed description thereof is omitted.

Next, the search device 500 will be described with reference to FIG.
FIG. 11 is a block diagram illustrating a schematic configuration of the search device 500.
The search device 500 is composed of, for example, a personal computer, and specifically, as shown in FIG. 11, a central control unit 501, a memory 502, a communication control unit 503, a recording unit 504, an operation input unit 505, search control. A unit 506, a display unit 507, and the like.
The central control unit 501, the memory 502, the communication control unit 503, the recording unit 504, the operation input unit 505, the search control unit 506, and the display unit 507 are substantially the same as the corresponding units of the mobile terminal 100 of the first embodiment. Only the parts different from those of the first embodiment will be described in detail.

  The central control unit 501 controls each unit of the search device 500. Specifically, the central control unit 501 includes a CPU (not shown) that controls each part of the search device 500, and performs various control operations according to various processing programs (not shown).

  The memory 502 includes, for example, a buffer memory that temporarily records image information and the like, a working memory such as the central control unit 501, a program memory that stores various programs and data related to the functions of the search device 500, etc. (Not shown).

The communication control unit 503 transmits and receives data via the communication network N and the communication antenna 503a.
That is, the communication control unit 503 includes, for example, a wireless LAN module, and transmits and receives data to and from an access point (not shown) connected to the communication network N via the communication antenna 503a. That is, the communication control unit 503 receives an Exif format image file transmitted from the imaging device 300 of the communication partner.
Further, the communication control unit 503 wirelessly uses a communication channel set by this communication method according to a communication protocol corresponding to a predetermined communication method adopted by the search device 500 for communication with a wireless base station (not shown). Data is transmitted to and received from the base station via the communication antenna 503a. That is, the communication control unit 503 receives incidental information and event related information of various events recorded in the event recording table T transmitted from the mobile phone 400 of the communication partner.

Similar to the recording unit 13 of the portable terminal 100 of the first embodiment, the recording unit 504 includes an image recording unit 13a and an event recording unit 13b, and data from these image recording unit 13a and event recording unit 13b is recorded. Reading and writing of data to the image recording unit 13a and the event recording unit 13b are performed.
In other words, the recording unit 504 causes the image recording unit 13a to record the image file received by the communication control unit 503, and also adds incidental information and event related information of various events received by the communication control unit 503 to the event recording unit. It is recorded in the event record table T of 13b.

  The search control unit 506 and the display unit 507 have substantially the same configuration as the search control unit 14 and the display unit 15 of the mobile terminal 100 of the first embodiment, and detailed description thereof is omitted.

The search device 500 configured as described above performs image search processing (see FIG. 6) and event search processing (see FIG. 7).
The image search process and the event search process are substantially the same as the contents of the image search process and the event search process performed by the mobile terminal 100 according to the first embodiment, and a detailed description thereof is omitted.

Note that the map server 600 has a specific place name corresponding to the position information of the image information and the event information (currently) when the imaging device 300, the mobile phone 400, and the search device 500 are not equipped with a map database (not shown) The search apparatus 500 is provided with information such as the prefecture, city, ward, town, etc. to which the position belongs and a specific building (for example, Tokyo Tower).
That is, the map server 600 has a function of opening a Web page (for example, a place name information providing page) on the Internet as a Web server, and the search device 500 receives the Web from the search device 500 in response to access. Send page information for the page. Specifically, the map server 600 stores the location name in response to the location name transmission request corresponding to the location information (for example, latitude and longitude coordinate information) of the image information and event information from the search device 500. The place name information is transmitted to the search device 500.

  The communication control unit 601 of the map server 600 includes, for example, a modem (MODEM: Modulator / DEModulater), a terminal adapter (Terminal Adapter), and the like, and communicates information with an external device such as the search device 500 via the communication network N. It is for performing communication control. Specifically, for example, the communication control unit 601 receives a request for transmitting a place name corresponding to position information through access from the search device 500. Further, the communication control unit 601 transmits the place name information related to the transmission request acquired from the storage unit 602 to the search device 500 via the communication network N, the wireless base station, or the like.

  The storage unit 602 is configured by, for example, a semiconductor nonvolatile memory, HDD (Hard Disc Drive), or the like, and stores place name information, Web page address information, and the like transmitted to the search device 500. Specifically, the storage unit associates a specific place name (prefecture, city, ward, town, etc. to which the current position belongs) or a specific building with a plurality of position information (for example, latitude and longitude coordinate information). Remember.

In addition, when the weather cannot be identified by the mobile phone 400 or the search device 500, the weather server 700 has location information (for example, latitude and longitude coordinate information) and date / time information (for example, 20ΔΔ year) when the event occurs. ; Month, day, □ day of week, ** hour, etc.) are provided to the search device 500.
In other words, the weather server 700 has a function of opening a Web page (for example, a weather information providing page) on the Internet as a Web server. Send page information for the page. Specifically, the weather server 700 includes the position information (for example, latitude and longitude coordinate information) and date / time information (for example, 20ΔΔ year; ○ month ○ day; □ day of the week, * In response to a weather (weather) transmission request corresponding to * time, etc., the weather at the position and date / time is searched and specified in the storage unit 702, and the weather information is transmitted to the search device 500.

  The communication control unit 701 of the weather server 700 includes, for example, a modem, a terminal adapter, and the like, and is for performing communication control of information with an external device such as the search device 500 via the communication network N. Specifically, for example, the communication control unit 701 receives a meteorological transmission request corresponding to position information and date / time information through access from the search device 500. Further, the communication control unit 701 transmits the weather information related to the transmission request acquired from the storage unit 702 to the search device 500 via the communication network N, the wireless base station, or the like.

  The storage unit 702 includes, for example, a semiconductor non-volatile memory, an HDD, and the like, and stores weather information transmitted to the search device 500, Web page address information, and the like. Specifically, the storage unit stores specific weather (for example, “sunny”, “rain”, etc.), position information (for example, latitude and longitude coordinate information), and date and time information (for example, 20ΔΔ year; ○ Day; □ day of week, ** hour, etc.) are stored in association with each other.

As described above, according to the search system 200 of the second embodiment, as with the mobile terminal 100 of the first embodiment, when remembering when and where the search target image was taken, the imaging position and the imaging date and time It is possible not only to search for image information using image-related information as a search condition, but also when it is impossible to remember exactly when and where the image to be searched was taken, it does not relate to image information. By inputting a search condition (for example, various keywords) to the search device 500, event information related to the search condition can be searched, and imaging-related information and an event recording unit recorded in the image recording unit 13a. The imaging related information related to the event related information associated with the searched event information by mutually using the event related information recorded in 13b It is possible to identify the broadcast. Thereby, the search of the image information matched with the specified imaging related information can be performed appropriately.
Therefore, it is possible to appropriately search for desired image information by using a search condition that is not related to the imaging related information as well as the imaging related information.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, what is illustrated as an operation event or an operation unnecessary event is an example and is not limited thereto, and can be arbitrarily changed as appropriate. Further, the incidental information related to the operation event and the operation unnecessary event is an example, and is not limited to these, and can be arbitrarily changed as appropriate.

  Moreover, in the said embodiment, although the position acquisition part 9 (309) illustrated the structure which acquires the positional information which concerns on the present position (image pick-up position, event occurrence position) using GPS information, The method of acquiring the position is an example, and is not limited to this, and can be arbitrarily changed as appropriate. For example, an RF-ID wireless information tag or a visible light communication information tag attached to a predetermined place (for example, home, office, road sign, car, etc.) or belongings (for example, bag or accessory), or Bluetooth The current position may be specified by receiving and identifying identification information such as a place, belongings, person, and device from a short-range wireless device equipped with (Registered Trademark), TransferJet (Registered Trademark), or the like.

  In addition, the display unit 15 (507) is exemplified as the output unit, but the display unit 15 (507) is not limited thereto. For example, the output unit is connected to a display device (not shown) that displays the output image so as to be able to communicate with the information. The search result data can be output to a display device and the search result can be displayed, or the search result data can be projected on a screen or a wall surface (not shown) to display the search result Any configuration may be used.

  Further, the configurations of the mobile terminal 100 and the search system 200 are merely examples illustrated in the first and second embodiments, and are not limited thereto. At least the first recording unit, the second recording unit, the input unit, Any configuration that includes an identification unit, a search unit, and an output unit can be arbitrarily changed.

In addition, in the first and second embodiments, the functions as the input unit, the identification unit, the search unit, and the output unit are controlled by the central control unit 1 of the mobile terminal 100 or the central control unit 501 of the search device 500. The operation input unit 6 (505), the specifying unit 14a, the search unit 14b, and the display unit 15 (507) are driven. However, the configuration is not limited to this, and the central control unit 1 is not limited thereto. (501) It is good also as a structure implement | achieved when a predetermined | prescribed program etc. are performed by CPU.
That is, a program including an input processing routine, a specific processing routine, a search processing routine, and an output processing routine is stored in a program memory that stores the program. Then, the CPU of the central control unit 1 (501) may function as an input means for inputting a search condition for searching for a search target by an input processing routine. Further, when the CPU of the central control unit 1 (501) searches for image information as a search target by the specific processing routine, if other information other than the imaging related information is input as a search condition by the input means, the other The event information related to the event information is searched from the event information recorded in the second recording means, and the imaging related information related to the event related information associated with the searched event information is specified. You may make it function as a means. Further, the CPU of the central control unit 1 (501) by the search processing routine causes the imaging related information input by the input means or the imaging related specified by the specifying means in the image information recorded in the first recording means. You may make it function as a search means to search the image information matched with information. Further, the CPU of the central control unit 1 (301) may function as an output unit that outputs a search result of a search target by the search unit by an output processing routine.

  Similarly, the determination unit may be realized by a predetermined program or the like being executed by the CPU of the central control unit 1 (501).

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

[Appendix]
Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
<Claim 1>
First recording means for recording image information of an image captured by the imaging means in association with imaging-related information related to imaging of the image;
Recording at least one event information related to an event other than the image capturing of the image that occurs at a timing asynchronous with the imaging instruction timing of the image by the imaging unit in association with the event related information related to the occurrence of the event; Two recording means;
An input means for inputting a search condition for searching the search target;
When searching for the image information as a search target, if information other than the imaging related information is input as the search condition by the input unit, event information related to the other information is stored in the second recording unit. A means for searching for event-related information associated with the event-related information associated with the searched event information;
Search means for searching for image information associated with the imaging-related information input by the input means or the imaging-related information specified by the specifying means among the image information recorded in the first recording means When,
Output means for outputting a search result of the search object by the search means;
A search device comprising:
<Claim 2>
The search means further includes:
When the imaging related information is input by the input unit as a search condition for the image information as a search target, the input unit inputs the image information recorded in the first recording unit. When the image information associated with the imaging related information is searched and information other than the imaging related information is input by the input means, the image recorded in the first recording means 2. The search apparatus according to claim 1, wherein image information associated with the imaging related information specified by the specifying means is searched for in the information.
<Claim 3>
The imaging related information includes at least one information of imaging position information related to the imaging position of the image and imaging date information related to the imaging date and time of the image,
The event related information includes information on at least one of event position information on an event occurrence position and event date information on an occurrence date of the event,
The specifying means further includes:
The relationship between the event-related information and the imaging-related information is the approximation of the distance between the imaging position indicated by the imaging position information and the event occurrence position indicated by the event position information, or the imaging date / time information. The search device according to claim 2, wherein the image is identified based on a closeness related to a time between an imaging date and time indicated by the event and an event occurrence date and time indicated by the event date and time information.
<Claim 4>
The specifying means further includes:
When other information other than the imaging-related information is input as the search condition by the input unit, the information is associated with event information searched from event information recorded in the second recording unit. Based on at least one of the event position information and the event date / time information, specify at least one of an imaging position related to the event occurrence position and an imaging date / time related to the event occurrence date / time,
The search means further includes:
The search apparatus according to claim 3, wherein image information associated with at least one of an imaging position and an imaging date / time specified by the specifying unit is searched.
<Claim 5>
The event related information is low in relevance with the imaging related information compared to the event position information or the event date and time information, and includes event specific related information specific to an event other than imaging,
The specifying means further includes:
When the event-specific related information is input by the input unit as the search condition for the image information as a search target, event information related to the event-specific related information is recorded in the second recording unit The search device according to claim 3 or 4, wherein information is searched from information and imaging related information related to event related information associated with the searched event information is specified.
<Claim 6>
The specifying means is:
When searching for the event information as a search target, if information other than event related information is input as the search condition by the input means, image information related to the other information is stored in the first recording means. Search from the recorded image information, identify event related information related to imaging related information associated with the searched image information,
The search means further includes:
Searching the event information recorded in the second recording means for event related information input by the input means or event information associated with the event related information specified by the specifying means. The search device according to claim 3, wherein
<Claim 7>
The search means further includes:
When the event related information is input by the input unit as a search condition for the event information as a search target, the event information recorded in the second recording unit is input by the input unit. When the event information associated with the event related information is searched and information other than the event related information is input by the input means, the event recorded in the second recording means The search device according to claim 6, wherein event information associated with the event related information specified by the specifying means is searched for in the information.
<Claim 8>
The specifying means further includes:
When information other than event related information is input as the search condition by the input means, the image information associated with the image information searched from the image information recorded in the first recording means Based on at least one of the imaging position information and the imaging date and time information, at least one of the event occurrence position related to the image imaging position and the event occurrence date related to the image imaging date is specified. ,
The search means further includes:
The search device according to claim 6 or 7, wherein event information associated with at least one of an event occurrence position and an event occurrence date and time specified by the specifying means is searched.
<Claim 9>
The specifying means further includes:
When searching for the event information as a search target, if designation information for specifying image information is input as the search condition by the input means, an imaging-related item associated with the image information specified by the specification information The search device according to any one of claims 6 to 8, wherein event-related information related to information is specified.
<Claim 10>
The specifying means further includes:
Information of at least one of the imaging position information and the imaging date / time information associated with the image information designated by the designation information retrieved from the image information recorded in the first recording means And at least one of the event occurrence position related to the image capture position and the event occurrence date related to the image capture date and time,
The search means further includes:
The search apparatus according to claim 9, wherein event information associated with at least one of an event occurrence position and an occurrence date and time specified by the specifying unit is searched.
<Claim 11>
The specifying means further includes:
When searching for the event information as a search target, if information other than the event related information, designation information specifying image information and the imaging related information is input as the search condition by the input means, The image information including the subject to be searched is searched from the image information recorded in the first recording means, and the image capturing position information and the image capturing date / time information associated with the searched image information are searched for. Based on at least one of the information, specify at least one of an event occurrence position related to the image capturing position and an event occurrence date related to the image capturing date and time,
The search means further includes:
The search device according to any one of claims 6 to 8, wherein event information associated with at least one of an event occurrence position and an occurrence date and time specified by the specifying means is searched. .
<Claim 12>
The imaging-related information is less related to event-related information than the imaging position information or the imaging date / time information, and includes imaging-specific related information specific to imaging of an image,
The specifying means further includes:
As the search condition with the event information as a search target, when the imaging-specific related information is input by the input unit, the imaging-specific related information in the image information recorded in the first recording unit 12. The related image information is searched, and event related information related to imaging related information associated with the searched image information is specified. Search device.
<Claim 13>
The event includes an operation event related to an operation executed based on a predetermined execution instruction operation of the operation input unit by the user,
The second recording means further includes
The search device according to any one of claims 1 to 12, wherein event-related information related to the operation event is acquired and recorded based on the occurrence of the operation event.
<Claim 14>
The event related information includes a call execution operation as the operation event and a keyword related to the communication partner, a communication execution operation as the operation event and a keyword related to the communication partner, and a schedule input as the operation event. The search device according to claim 13, comprising at least one keyword among a keyword included in the contents of the operation and the schedule, a document input operation as the operation event, and a keyword related to the document.
<Claim 15>
The event includes an operation unnecessary event related to an operation executed without requiring a predetermined execution instruction operation of the operation input unit by the user,
A determination means for determining whether or not the operation unnecessary event has occurred;
The second recording means further includes
The event-related information related to the operation unnecessary event is acquired and recorded when the determination unit determines that the operation unnecessary event has occurred. Search device.
<Claim 16>
The determination means further includes:
It is determined whether or not the operation unnecessary event has occurred according to whether or not a predetermined type of subject is detected from image information of an image captured by the imaging unit at a predetermined timing,
The second recording means further includes
16. The search device according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the type of the detected subject is acquired and recorded as the event related information.
<Claim 17>
The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a specific word is detected from the sound information collected by the sound collecting means at a predetermined timing,
The second recording means further includes
16. The search device according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the detected specific word is acquired and recorded as the event related information.
<Claim 18>
The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a specific location is detected based on the current position of the device main body measured by the positioning means at a predetermined timing,
The second recording means further includes
The search according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, a name related to the detected specific place is acquired and recorded as the event related information. apparatus.
<Claim 19>
The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a predetermined type of motion is detected based on motion information automatically detected by the motion detection means,
The second recording means further includes
16. The search apparatus according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the type of the detected predetermined movement is acquired and recorded as the event related information.
<Claim 20>
First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous Search using a search device comprising: second recording means for recording at least one event information related to an event other than imaging of the image that occurs at the timing in association with event related information related to the occurrence of the event A method,
A process of entering search conditions for searching the search target;
When searching for the image information as a search target, if other information other than the imaging related information is input as the search condition, event information related to the other information is recorded in the second recording means. A process of searching for event-related information associated with the event-related information associated with the searched event information,
A process of searching for image information associated with the imaging-related information input by the input unit or the specified imaging-related information among the image information recorded in the first recording unit;
A process of outputting the search result of the search target;
The search method characterized by including.
<Claim 21>
First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous A second recording unit that records at least one event information related to an event other than imaging of the image that occurs at a timing in association with event-related information related to the occurrence of the event;
An input means for inputting a search condition for searching a search target;
When searching for the image information as a search target, if information other than the imaging related information is input as the search condition by the input unit, event information related to the other information is stored in the second recording unit. A specifying means for searching from event information recorded in the event information and specifying imaging related information related to event related information associated with the searched event information;
Search means for searching for image information associated with the imaging-related information input by the input means or the imaging-related information specified by the specifying means among the image information recorded in the first recording means ,
Output means for outputting the search result of the search object by the search means;
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Mobile terminal 1 Central control part 3 Imaging part 6 Operation input part 7 Transmission / reception part 7a Microphone 9 Position acquisition part 10 Date acquisition part 11 Detection sensor part 11a Motion detection sensor 12 Event acquisition control part 12b Determination part 13 Recording part 13a Image recording Unit 13b event recording unit 14 search control unit 14a specifying unit 14b search unit 15 display unit 200 search system 300 imaging device 400 mobile phone 500 search device

Claims (21)

First recording means for recording image information of an image captured by the imaging means in association with imaging-related information related to imaging of the image;
Recording at least one event information related to an event other than the image capturing of the image that occurs at a timing asynchronous with the imaging instruction timing of the image by the imaging unit in association with the event related information related to the occurrence of the event; Two recording means;
An input means for inputting a search condition for searching the search target;
When searching for the image information as a search target, if information other than the imaging related information is input as the search condition by the input unit, event information related to the other information is stored in the second recording unit. A means for searching for event-related information associated with the event-related information associated with the searched event information;
Search means for searching for image information associated with the imaging-related information input by the input means or the imaging-related information specified by the specifying means among the image information recorded in the first recording means When,
Output means for outputting a search result of the search object by the search means;
A search device comprising: The search means further includes:
When the imaging related information is input by the input unit as a search condition for the image information as a search target, the input unit inputs the image information recorded in the first recording unit. When the image information associated with the imaging related information is searched and information other than the imaging related information is input by the input means, the image recorded in the first recording means 2. The search apparatus according to claim 1, wherein image information associated with the imaging related information specified by the specifying means is searched for in the information. The imaging related information includes at least imaging position information related to the imaging position of the image , and the event related information includes at least event position information related to the event occurrence position , or the imaging related information relates to at least the imaging date and time of the image. Including imaging date and time information, and the event related information includes at least event date and time information related to an event occurrence date and time,
The specifying means further includes:
The relationship between the event-related information and the imaging-related information is the approximation of the distance between the imaging position indicated by the imaging position information and the event occurrence position indicated by the event position information, or the imaging date / time information. The search device according to claim 2, wherein the image is identified based on a closeness related to a time between an imaging date and time indicated by the event and an event occurrence date and time indicated by the event date and time information. The specifying means further includes:
When other information other than the imaging-related information is input as the search condition by the input unit, the information is associated with event information searched from event information recorded in the second recording unit. Based on at least one of the event position information and the event date / time information, specify at least one of an imaging position related to the event occurrence position and an imaging date / time related to the event occurrence date / time,
The search means further includes:
The search apparatus according to claim 3, wherein image information associated with at least one of an imaging position and an imaging date / time specified by the specifying unit is searched. The event related information is low in relevance with the imaging related information compared to the event position information or the event date and time information, and includes event specific related information specific to an event other than imaging,
The specifying means further includes:
When the event-specific related information is input by the input unit as the search condition for the image information as a search target, event information related to the event-specific related information is recorded in the second recording unit The search device according to claim 3 or 4, wherein information is searched from information and imaging related information related to event related information associated with the searched event information is specified. The specifying means is:
When searching for the event information as a search target, if information other than event related information is input as the search condition by the input means, image information related to the other information is stored in the first recording means. Search from the recorded image information, identify event related information related to imaging related information associated with the searched image information,
The search means further includes:
Searching the event information recorded in the second recording means for event related information input by the input means or event information associated with the event related information specified by the specifying means. The search device according to claim 3, wherein The search means further includes:
When the event related information is input by the input unit as a search condition for the event information as a search target, the event information recorded in the second recording unit is input by the input unit. When the event information associated with the event related information is searched and information other than the event related information is input by the input means, the event recorded in the second recording means The search device according to claim 6, wherein event information associated with the event related information specified by the specifying means is searched for in the information. The specifying means further includes:
When information other than event related information is input as the search condition by the input means, the image information associated with the image information searched from the image information recorded in the first recording means Based on at least one of the imaging position information and the imaging date and time information, at least one of the event occurrence position related to the image imaging position and the event occurrence date related to the image imaging date is specified. ,
The search means further includes:
The search device according to claim 6 or 7, wherein event information associated with at least one of an event occurrence position and an event occurrence date and time specified by the specifying means is searched. The specifying means further includes:
When searching for the event information as a search target, if designation information for specifying image information is input as the search condition by the input means, an imaging-related item associated with the image information specified by the specification information The search device according to any one of claims 6 to 8, wherein event-related information related to information is specified. The specifying means further includes:
Information of at least one of the imaging position information and the imaging date / time information associated with the image information designated by the designation information retrieved from the image information recorded in the first recording means And at least one of the event occurrence position related to the image capture position and the event occurrence date related to the image capture date and time,
The search means further includes:
The search apparatus according to claim 9, wherein event information associated with at least one of an event occurrence position and an occurrence date and time specified by the specifying unit is searched. The specifying means further includes:
When searching for the event information as a search target, if information other than the event related information, designation information specifying image information and the imaging related information is input as the search condition by the input means, The image information including the subject to be searched is searched from the image information recorded in the first recording means, and the image capturing position information and the image capturing date / time information associated with the searched image information are searched for. Based on at least one of the information, specify at least one of an event occurrence position related to the image capturing position and an event occurrence date related to the image capturing date and time,
The search means further includes:
The search device according to any one of claims 6 to 8, wherein event information associated with at least one of an event occurrence position and an occurrence date and time specified by the specifying means is searched. . The imaging-related information is less related to event-related information than the imaging position information or the imaging date / time information, and includes imaging-specific related information specific to imaging of an image,
The specifying means further includes:
As the search condition with the event information as a search target, when the imaging-specific related information is input by the input unit, the imaging-specific related information in the image information recorded in the first recording unit 12. The related image information is searched, and event related information related to imaging related information associated with the searched image information is specified. Search device. The event includes an operation event related to an operation executed based on a predetermined execution instruction operation of the operation input unit by the user,
The second recording means further includes
The search device according to any one of claims 1 to 12, wherein event-related information related to the operation event is acquired and recorded based on the occurrence of the operation event.   The event related information includes a call execution operation as the operation event and a keyword related to the communication partner, a communication execution operation as the operation event and a keyword related to the communication partner, and a schedule input as the operation event. The search device according to claim 13, comprising at least one keyword among a keyword included in the contents of the operation and the schedule, a document input operation as the operation event, and a keyword related to the document. The event includes an operation unnecessary event related to an operation executed without requiring a predetermined execution instruction operation of the operation input unit by the user,
A determination means for determining whether or not the operation unnecessary event has occurred;
The second recording means further includes
The event-related information related to the operation unnecessary event is acquired and recorded when the determination unit determines that the operation unnecessary event has occurred. Search device. The determination means further includes:
It is determined whether or not the operation unnecessary event has occurred according to whether or not a predetermined type of subject is detected from image information of an image captured by the imaging unit at a predetermined timing,
The second recording means further includes
16. The search device according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the type of the detected subject is acquired and recorded as the event related information. The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a specific word is detected from the sound information collected by the sound collecting means at a predetermined timing,
The second recording means further includes
16. The search device according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the detected specific word is acquired and recorded as the event related information. The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a specific location is detected based on the current position of the device main body measured by the positioning means at a predetermined timing,
The second recording means further includes
The search according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, a name related to the detected specific place is acquired and recorded as the event related information. apparatus. The determination means further includes:
Whether or not the operation unnecessary event has occurred is determined according to whether or not a predetermined type of motion is detected based on motion information automatically detected by the motion detection means,
The second recording means further includes
16. The search apparatus according to claim 15, wherein when the determination unit determines that the operation unnecessary event has occurred, the type of the detected predetermined movement is acquired and recorded as the event related information. First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous Search using a search device comprising: second recording means for recording at least one event information related to an event other than imaging of the image that occurs at the timing in association with event related information related to the occurrence of the event A method,
A process of entering search conditions for searching the search target;
When searching for the image information as a search target, if other information other than the imaging related information is input as the search condition, event information related to the other information is recorded in the second recording means. A process of searching for event-related information associated with the event-related information associated with the searched event information,
A process of searching for image information associated with the imaging-related information input by the input unit or the specified imaging-related information among the image information recorded in the first recording unit;
A process of outputting the search result of the search target;
The search method characterized by including. First recording means for recording image information of an image taken by the imaging means in association with imaging-related information related to imaging of the image, and imaging instruction timing of the image by the imaging means are asynchronous A second recording unit that records at least one event information related to an event other than imaging of the image that occurs at a timing in association with event-related information related to the occurrence of the event;
An input means for inputting a search condition for searching a search target;
When searching for the image information as a search target, if information other than the imaging related information is input as the search condition by the input unit, event information related to the other information is stored in the second recording unit. A specifying means for searching from event information recorded in the event information and specifying imaging related information related to event related information associated with the searched event information;
Search means for searching for image information associated with the imaging-related information input by the input means or the imaging-related information specified by the specifying means among the image information recorded in the first recording means ,
Output means for outputting the search result of the search object by the search means;
A program characterized by functioning as

Images