JP7570882B2 - Electronic device, electronic device control method, program, and storage medium - Google Patents

Description

The present invention relates to an electronic device, a control method for an electronic device, a program, and a storage medium.

Electronic devices are known that allow users to operate them by looking at their eyes (gaze input). Gaze input is particularly effective when users want to instantly give operating instructions to electronic devices, such as digital cameras and game consoles.

In general, the gaze is subject to large shaking, so when a pointer (gaze pointer; GUI) indicating the detected position of the gaze is displayed on the screen, if a small pointer (frame) is displayed, the pointer will shake significantly, which may cause discomfort to the user. When a pointer (GUI) indicating the detected position of the gaze is displayed on the screen, if a large pointer (frame) is displayed, it becomes easier to fit the targeted subject within the range of the frame, thereby reducing the discomfort described above. However, as the range is specified roughly, it is difficult to instruct automatic focus adjustment (AF) or subject tracking for small objects (subjects) within the frame.

As a technique for changing the size of a pointer, Patent Literature 1 discloses a technique for changing the size of a touch pointer when the pointer indicating the user's touch position (touch pointer) is placed near a detected subject.

JP 2018-037893 A

However, the technology disclosed in Patent Document 1 changes the size of the pointer depending on the subject, so there is a possibility that the size of the pointer may change contrary to the user's intention.

The present invention aims to provide a technology that allows a user to easily specify an intended position when specifying a position using eye gaze input.

One aspect of the present invention is
A gaze detection means for detecting a gaze position of a user looking at the display means;
A control means for controlling to designate a selection position on the display means in response to a line of sight position detected by the line of sight detection means or a first operation on the operation means,
displaying a first item indicating a first size range at the selection position based on the gaze position detected by the gaze detection means in a first case in which the selection position is designated based on the gaze position detected by the gaze detection means and the first operation is not performed on the operation means;
displaying a second item indicating a range of a second size smaller than the first size at the selection position in a second case in which the selection position is designated based on the gaze position detected by the gaze detection means and the first operation is performed on the operation means;
In the first case, in response to a second operation different from the first operation being performed, selecting an object at a position of the first item based on a predetermined condition;
In the second case, in response to the second operation being performed, an object is selected at the position of the second item without being based on the predetermined condition.
A control means for controlling the
The electronic device is characterized by having the following features.

According to the present invention, when a position is specified using eye gaze input by the user, the user can easily specify the intended position.

FIG. 1 is an external view of a digital camera 100. FIG. 1 is a block diagram of a digital camera 100. 11A and 11B are diagrams illustrating setting of an AF frame by a line of sight in one-point AF. 11A and 11B are diagrams illustrating setting of an AF frame based on the line of sight in face+tracking priority AF. 13 is a flowchart of a shooting mode process. 13 is a flowchart of a shooting mode process. 13 is a flowchart of a camera setting process. 13 is a flowchart of a touch operation response process. 13 is a flowchart of a relative position designation process when the line of sight is enabled. 13 is a flowchart of a relative position designation process when the line of sight is invalid. 13 is a flowchart of an absolute position designation process. 13 is a display example of a setting menu screen. 13 is a display example regarding selection of a character insertion position using line of sight. FIG. 13 is an external view of a headset 1300 and operation members according to Modification 2. 13A to 13C are diagrams illustrating a position designation operation according to Modification 2.

[Embodiment]
<External view of digital camera 100>
A preferred embodiment of the present invention will now be described with reference to the drawings. Figures 1A and 1B show external views of a digital camera 100 as an example of an apparatus to which the present invention can be applied. Figure 1A is a front perspective view of the digital camera 100, and Figure 1B is a rear perspective view of the digital camera 100.

The display unit 28 is a display unit provided on the back surface of the digital camera 100, and displays images and various information. The touch panel 70a can detect touch operations on the display surface (touch operation surface; touch operation member) of the display unit 28. The outside-finder display unit 43 is a display unit provided on the top surface of the digital camera 100, and displays various settings of the digital camera 100, including the shutter speed and aperture. The shutter button 61 is an operating member for issuing shooting instructions (image capture instructions). The mode change switch 60 is an operating member for switching between various modes. The terminal cover 40 is a cover that protects a connector (not shown) that connects the digital camera 100 to an external device.

The main electronic dial 71 is a rotary operation member, and by turning the main electronic dial 71, settings such as the shutter speed and aperture can be changed. The power switch 72 is an operation member that switches the power of the digital camera 100 ON and OFF. The sub electronic dial 73 is a rotary operation member, and by turning the sub electronic dial 73, the selection frame (cursor) can be moved and images can be forwarded. The four-way key 74 is configured so that the up, down, left, and right parts can each be pressed, and processing can be performed according to which part of the four-way key 74 is pressed. The SET button 75 is a push button, and is mainly used to confirm selection items. The multi-controller (hereinafter, MC) 65 can accept directional instructions in eight directions and a press operation on the center part.

The video button 76 is used to start or stop video shooting (recording). The AE lock button 77 is a push button, and pressing the AE lock button 77 in the shooting standby state fixes the exposure state. The enlargement button 78 is an operation button for switching the enlargement mode ON and OFF in the live view display (LV display) of the shooting mode. By turning the enlargement mode ON and then operating the main electronic dial 71, the live view image (LV image) can be enlarged or reduced. In the playback mode, the enlargement button 78 functions as an operation button for enlarging the playback image or increasing its magnification ratio. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 during the shooting mode, the mode transitions to the playback mode, and the latest image among the images recorded on the recording medium 200 (described later) can be displayed on the display unit 28. The menu button 81 is a push button used to perform an instruction operation to display a menu screen, and when the menu button 81 is pressed, a menu screen on which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four-way key 74, the SET button 75, or the MC 65. The line of sight confirmation button 82 is an operating member included in the operation unit 70, and is a push button that instructs the user to select or cancel a subject based on the position of the line of sight pointer, which will be described later. The line of sight confirmation button is located in a position that is easy for the user to operate even when looking through the viewfinder (with the eye placed on the eyepiece unit 16), and is located in a position that can be operated with the thumb of the right hand holding the grip unit 90.

The communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the lens unit 150 (described later; detachable). The eyepiece 16 is an eyepiece of an eyepiece finder (a peer-in type finder), and the user can view the image displayed on the internal EVF 29 (described later) through the eyepiece 16. The eyepiece detection unit 57 is an eyepiece detection sensor that detects whether the user (photographer) has placed his/her eye on the eyepiece 16. The cover 202 is a cover for a slot that stores a recording medium 200 (described later). The grip unit 90 is a holding unit that is shaped so that the user can easily hold it with the right hand when holding the digital camera 100. The shutter button 61 and the main electronic dial 71 are located in a position that can be operated with the index finger of the right hand when the digital camera 100 is held by holding the grip unit 90 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 73 and the line of sight confirmation button 82 are located in a position that can be operated with the thumb of the right hand.

<Configuration block diagram of digital camera 100>
FIG. 2 is a block diagram showing an example of the configuration of the digital camera 100. The lens unit 150 is a lens unit equipped with an interchangeable photographing lens. The lens 103 is usually composed of a plurality of lenses, but FIG. 2 shows only one lens for simplicity. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the digital camera 100, and the communication terminal 10 is a communication terminal through which the digital camera 100 communicates with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via these communication terminals 6 and 10. The lens unit 150 controls the aperture 1 via the aperture drive circuit 2 by the internal lens system control circuit 4. The lens unit 150 adjusts the focus by displacing the lens 103 via the AF drive circuit 3 by the lens system control circuit 4.

The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50.

The imaging unit 22 is an imaging element that converts an optical image into an electrical signal and is configured with a CCD, a CMOS element, etc. The imaging unit 22 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50.

The image processing unit 24 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on the data from the A/D converter 23 or the data from the memory control unit 15. The image processing unit 24 also performs predetermined calculation processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 24. This allows TTL (through-the-lens) type AF (autofocus) processing, AE (automatic exposure) processing, EF (flash pre-flash) processing, etc. to be performed. The image processing unit 24 also performs predetermined calculation processing using the captured image data, and performs TTL type AWB (auto white balance) processing based on the obtained calculation results.

The memory control unit 15 controls the transmission and reception of data between the A/D converter 23, the image processing unit 24, and the memory 32. The output data from the A/D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, the output data from the A/D converter 23 is written to the memory 32 via the memory control unit 15 without going through the image processing unit 24. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A/D converter 23, and image data to be displayed on the display unit 28 and EVF 29. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

The memory 32 also serves as a memory (video memory) for displaying images. The image data for display written to the memory 32 is displayed by the display unit 28 or EVF 29 via the memory control unit 15. The display unit 28 and EVF 29 each perform display according to a signal from the memory control unit 15 on a display device such as an LCD or an organic EL. A live view display (LV) can be performed by sequentially transferring and displaying data that has been A/D converted by the A/D converter 23 and stored in the memory 32 to the display unit 28 or EVF 29. Hereinafter, the image displayed in the live view display is referred to as a live view image (LV image).

The gaze detection unit 160 (reception unit) detects the gaze of the user's eye, which is placed in the eyepiece unit 16, as he or she looks at the EVF 29. The gaze detection unit 160 is composed of a dichroic mirror 162, an imaging lens 163, a gaze detection sensor 164, a gaze detection circuit 165, and an infrared light emitting diode 166.

The infrared light emitting diode 166 is a light emitting element for detecting the user's gaze position within the viewfinder screen, and irradiates infrared light onto the user's eyeball (eye) 161. The infrared light emitted from the infrared light emitting diode 166 is reflected by the eyeball (eye) 161, and the reflected infrared light reaches the dichroic mirror 162. The dichroic mirror 162 reflects only the infrared light and transmits visible light. The reflected infrared light, whose optical path has been changed, forms an image on the imaging surface of the gaze detection sensor 164 via the imaging lens 163. The imaging lens 163 is an optical component that constitutes the gaze detection optical system. The gaze detection sensor 164 is composed of an imaging device such as a CCD image sensor.

The gaze detection sensor 164 photoelectrically converts the incident infrared reflected light into an electrical signal and outputs it to the gaze detection circuit 165. The gaze detection circuit 165 detects the user's gaze position from the movement of the user's eyeball (eye) 161 based on the output signal of the gaze detection sensor 164, and outputs the detection information to the system control unit 50 and the gaze determination unit 170.

The gaze determination unit 170 determines that the user is gazing at a certain area when the period during which the user's gaze is fixed on that area exceeds a predetermined threshold based on the detection information received from the gaze detection circuit 165. Therefore, the area can be said to be a gaze position (gaze area) where the gaze is performed. Note that "the gaze is fixed on a certain area" means, for example, that the average position of the gaze movement is within the area until a predetermined period has elapsed, and the variation (dispersion) is less than a predetermined value. Note that the predetermined threshold can be arbitrarily changed by the system control unit 50. In addition, the gaze determination unit 170 may not be provided as an independent block, and the system control unit 50 may execute the same function as the gaze determination unit 170 based on the detection information received from the gaze detection circuit 165.

In this embodiment, the gaze detection unit 160 detects the gaze using a method called the corneal reflex method. The corneal reflex method is a method of detecting the direction and position of the gaze from the positional relationship between the light reflected by the eyeball (eye) 161 (particularly the cornea) of the infrared light emitted from the infrared light emitting diode 166 and the pupil of the eyeball (eye) 161. Note that the method of detecting the gaze (direction and position of the gaze) is not particularly limited, and methods other than those described above may be used. For example, a method called the scleral reflex method may be used, which utilizes the difference in light reflectance between the iris and white of the eye.

Various camera settings, including shutter speed and aperture, are displayed on the outside viewfinder display 43 via the outside viewfinder display drive circuit 44.

The non-volatile memory 56 is an electrically erasable and recordable memory, such as a Flash-ROM. Constants and programs for the operation of the system control unit 50 are recorded in the non-volatile memory 56. The programs referred to here are programs for executing the various flowcharts described later in this embodiment.

The system control unit 50 is a control unit consisting of at least one processor or circuit, and controls the entire digital camera 100. The system control unit 50 executes the programs recorded in the non-volatile memory 56 described above, thereby realizing each process of this embodiment described below. The system memory 52 is, for example, a RAM, and the system control unit 50 loads constants and variables for the operation of the system control unit 50, programs read from the non-volatile memory 56, etc. into the system memory 52. The system control unit 50 also performs display control by controlling the memory 32, the display unit 28, etc.

The system timer 53 is a timing unit that measures the time used for various controls and the time of the built-in clock.

The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit that switches between blocks to which electricity is applied, and other components, and detects whether a battery is installed, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage for the necessary period to each component, including the recording medium 200. The power supply unit 30 is composed of primary batteries such as alkaline batteries or lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, or Li batteries, an AC adapter, etc.

The recording medium I/F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, etc.

The communication unit 54 transmits and receives video signals and audio signals to and from an external device connected wirelessly or via a wired cable. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can also communicate with an external device via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can receive image data and various other information from an external device.

The attitude detection unit 55 detects the attitude of the digital camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. An acceleration sensor, a gyro sensor, or the like can be used as the attitude detection unit 55. It is also possible to detect the movement of the digital camera 100 (panning, tilting, lifting, whether it is stationary, etc.) using the acceleration sensor or gyro sensor that is the attitude detection unit 55.

The eyepiece detection unit 57 is an eyepiece detection sensor that detects the approach (eyepiece) and departure (eye separation) of the eye (object) 161 to the eyepiece unit 16 of the eyepiece finder (hereinafter simply referred to as the "finder") (approach detection). The system control unit 50 switches the display unit 28 and the EVF 29 between display (display state)/non-display (non-display state) depending on the state detected by the eyepiece detection unit 57. More specifically, at least in the standby state for shooting and when the display destination switching is automatic switching, when the eye is not in contact with the camera, the display destination is set to the display unit 28 and the display is turned on, and the EVF 29 is turned off. Also, when the eye is in contact with the camera, the display destination is set to the EVF 29 and the display is turned on, and the display unit 28 is turned off. For example, an infrared proximity sensor can be used as the eyepiece detection unit 57, and it can detect the approach of some object to the eyepiece unit 16 of the finder that incorporates the EVF 29. When an object approaches, infrared light projected from a light projecting section (not shown) of the eyepiece detection section 57 is reflected by the object and received by a light receiving section (not shown) of the infrared proximity sensor. The amount of infrared light received can also determine how close the object is to the eyepiece section 16 (eyepiece distance). In this way, the eyepiece detection section 57 performs eyepiece detection to detect the proximity of the object to the eyepiece section 16. When an object approaching within a predetermined distance from the non-eyepiece state (non-approaching state) is detected, it is detected that the object has been placed in eye contact. When an object whose approach was detected is moved away from the eyepiece state (approaching state) by a predetermined distance or more, it is detected that the object has been removed. The threshold for detecting eye contact and the threshold for detecting removal of the eye may be different, for example, by providing a hysteresis. In addition, after detecting the eye contact, the eyepiece is in the eye contact state until removal of the eye is detected. After detecting the removal of the eye, the eyepiece is in the non-eye contact state until removal of the eye is detected. Note that the infrared proximity sensor is just one example, and other sensors may be used for the eye proximity detection unit 57 as long as they can detect the approach of an eye or object that can be considered as an eye.

The system control unit 50 controls the line-of-sight detection unit 160 to detect the following states of the line of sight toward the EVF 29.
The line of sight that was not directed toward the EVF 29 is now directed toward the EVF 29. In other words, the start of line of sight input.
- Eye gaze input is being performed on the EVF 29.
- Be sure to focus your gaze on the position of the EVF 29.
The gaze direction is changed to the EVF 29. In other words, the gaze input is terminated.
- A state in which no gaze input is being made to the EVF 29 (a state in which the user is not looking at the EVF 29).

These operations and states, as well as the position (direction) in which the gaze is directed toward the EVF 29, are notified to the system control unit 50 via the internal bus, and the system control unit 50 determines what type of gaze input is being performed based on the notified information.

The operation unit 70 is an input unit that accepts operations from a user (user operations) and is used to input various operation instructions to the system control unit 50. As shown in FIG.
The operation unit 70 includes a mode changeover switch 60, a shutter button 61, a power switch 72, a touch panel 70a, etc. The operation unit 70 also includes, as other operation members 70b, a main electronic dial 71, a sub electronic dial 73, a four-way key 74, a SET button 75, a video button 76, an AE lock button 77, a magnification button 78, a playback button 79, a menu button 81, an MC 65, etc.

The mode changeover switch 60 changes the operation mode of the system control unit 50 to one of still image shooting mode, video shooting mode, playback mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The mode changeover switch 60 allows the user to directly switch to one of these modes. Alternatively, after switching to a list screen of shooting modes with the mode changeover switch 60, the user may selectively switch to one of the multiple displayed modes using other operating members. Similarly, the video shooting mode may also include multiple modes.

The shutter button 61 has a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on when the shutter button 61 is pressed halfway (instruction to prepare for shooting) and generates a first shutter switch signal SW1. The system control unit 50 starts preparation operations for shooting such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing based on the first shutter switch signal SW1. The second shutter switch 64 is turned on when the shutter button 61 is pressed fully (instruction to shoot) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processing operations based on the second shutter switch signal SW2, from reading out the signal from the imaging unit 22 to writing the captured image to the recording medium 200 as an image file.

The touch panel 70a and the display unit 28 can be configured as one unit. For example, the touch panel 70a is configured so that the light transmittance does not interfere with the display of the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Then, input coordinates on the touch panel 70a are associated with display coordinates on the display surface of the display unit 28. This makes it possible to provide a GUI (graphical user interface) that allows the user to directly operate the screen displayed on the display unit 28. The system control unit 50 can detect the following operations or states on the touch panel 70a.

A finger or pen that has not been touching the touch panel 70a touches the touch panel 70a again, that is, the start of touching (hereinafter referred to as touch-down).
A state in which the touch panel 70a is touched with a finger or a pen (hereinafter, referred to as Touch-On).
A finger or a pen is moved while touching the touch panel 70a (hereinafter, referred to as Touch-Move).
The finger or pen that has been touching the touch panel 70a is removed (released), that is, the touch ends (hereinafter, referred to as "touch-up").
A state in which nothing is touching the touch panel 70a (hereinafter, referred to as Touch-Off).

When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. If touch-move is detected, touch-on will also be detected at the same time. Even if touch-on is detected, touch-move will not be detected if the touch position does not move. After it is detected that all fingers or pens that were touching have touched up, touch-off will occur.

These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 via the internal bus. The system control unit 50 then determines what kind of operation (touch operation) has been performed on the touch panel 70a based on the notified information. For touch moves, the direction of movement of the finger or pen moving on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on the change in position coordinates. If a touch move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 70a, quickly moved a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced on the touch panel 70a as if flicking it. If a touch move of a predetermined distance or more at a predetermined speed or more is detected and a touch up is detected, it can be determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation). Furthermore, a touch operation in which multiple points (e.g., two points) are touched together (multi-touch) and the touch positions are brought closer together is called a pinch in, and a touch operation in which the touch positions are moved away from each other is called a pinch out. Pinch out and pinch in are collectively called a pinch operation (or simply a pinch). The touch panel 70a may be any of a variety of touch panel types, including a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or a pen to the touch panel, and either type may be used.

The digital camera 100 may also be provided with an audio input unit (not shown) that transmits audio obtained from a built-in microphone or an audio input device connected via an audio input terminal to the system control unit 50. In this case, the system control unit 50 selects the input audio signal as necessary, performs analog-to-digital conversion, and performs level optimization processing, specific frequency reduction processing, etc. to generate an audio signal.

In this embodiment, the user can set the method of specifying the position of the position indicator (for example, the AF frame) when a touch-move operation is performed in the eye-closed state to either an absolute position specification method or a relative position specification method. The absolute position specification method is a method in which the input coordinates on the touch panel 70a correspond to the display coordinates on the display surface of the EVF 29. In the case of the absolute position specification method, when there is a touch-down on the touch panel 70a, the AF frame is set at a position corresponding to the touched position (position where the coordinates are input) even if there is no touch-move (moves from the position before the touch-down). The position set by the absolute position specification method is a position based on the touch-down position, regardless of the position set before the touch-down. In addition, when there is a touch-move after the touch-down, the position of the AF frame also moves based on the touch position after the touch-move. The relative position specification method is a method in which the input coordinates on the touch panel 70a do not correspond to the display coordinates on the display surface of the EVF 29. In the case of the relative position designation method, if there is only a touchdown on the touch panel 70a and no touch-move, the position of the AF frame does not move from the position before the touchdown. If there is a touch-move after that, regardless of the touchdown position, the position of the AF frame moves in the direction of the touch-move movement from the currently set position of the AF frame (the position that was set before the touchdown) by a distance according to the amount of movement of the touch-move.

As the AF method (AF frame setting method), any one of a plurality of AF methods including "single point AF" and "face + tracking priority AF" can be set. "Single point AF" is a method in which the user specifies one location by using a single point AF frame as the position for performing AF. "Face + tracking priority AF" is a method in which the AF position is automatically set based on the automatic selection conditions when the user does not specify the tracking target. In the automatic AF position setting, if a person's face is detected from the LV image, the face is selected as the AF target subject with priority. If multiple people's faces are detected, one face is selected and set as the AF target subject according to the priority such as the face size is large, the face position is close to the digital camera 100 (close side), the face position is close to the center in the image, the face is a pre-registered individual's face, etc. If a person's face is not detected, a subject other than the face is selected and set as the AF target subject according to the priority such as the face is close to the digital camera 100 (close side), the contrast is high, the subject is a high priority subject such as an animal or a vehicle, or it is a moving object. When a subject to be tracked is designated by a user, the subject to be tracked is set as an AF target subject. That is, the automatic selection condition is a condition in which weighting is performed using at least one element condition such as the example shown below, and the obtained score is equal to or greater than a predetermined threshold value or the obtained score is the highest score.
- It is the face of a detected person.
-The face is large.
The face is positioned close to the digital camera 100 (on the close side).
- The face is positioned close to the center of the image.
- The face of a pre-registered individual.
- Close to the digital camera 100 (close side).
-High contrast.
- High-priority subjects such as animals or vehicles.
・It is a moving object.

<Setting the AF frame by line of sight in single-point AF>
3(a) to 3(f) will be used to explain AF frame movement control using eye gaze input when the AF frame selection method (AF method) is set to "single-point AF" in the digital camera 100. Each of Figs. 3(a) to 3(f) is a display example displayed on the EVF 29 when the AF frame selection method (AF method) is set to "single-point AF."

FIG. 3A shows an example of a display in which the gaze function (a function for specifying an AF position using gaze input) is enabled and the gaze detection unit 160 detects the gaze of the user. A live view (hereinafter, LV) 301 is an LV image captured by the imaging unit 22. A one-point AF frame 302 is an AF frame (an indicator showing a position to be subjected to AF, i.e., a focus adjustment position) when the AF frame selection method (AF method) is set to "one-point AF". The one-point AF frame is set to the center of the screen in the initial state. A gaze pointer 310 is a pointer (indicator, display item, first item) that indicates the position of the gaze input detected by the gaze detection unit 160. The gaze detection unit 160 can acquire the coordinates of a certain point as the position where the gaze input is made, but the gaze pointer 310 is displayed as indicating a range of a certain large predetermined size centered on the position where the gaze input is made. In this way, even if the position detected by the gaze input does not exactly match the position of the target subject that the user wants to select, it is possible to place the target subject within the range indicated by the gaze pointer. That is, it is possible to roughly specify a position by the gaze input. In addition, the gaze pointer 310 is displayed with the gaze position detected by the gaze detection unit 160 averaged over a predetermined period (for example, a period of 30 milliseconds) as the center. In this way, it is possible to prevent excessive movement of the gaze pointer due to the variation in the position of the user's gaze input within a small time period, and it is possible to improve the visibility of the gaze pointer. The gaze of a human being has a characteristic called fixation eye movement, in which the eyeball makes a fine movement even when gazing at a certain point. Therefore, even if an attempt is made to specify a precise position using only the gaze input, it is difficult to specify the position as the user intends, and the operation feels unpleasant to the user. In contrast, by displaying the gaze pointer 310 based on the position averaged over a predetermined period at a first size that is somewhat large, such discomfort can be reduced.

Figure 3(b) is an example of the display on the EVF 29 when the user moves their gaze from the state shown in Figure 3(a) to change the location of the EVF 29 they are looking at. The gaze pointer 310 was at the top right of the screen in Figure 3(a) but has moved to the bottom left of the screen in Figure 3(b). In this way, the position of the gaze pointer 310 also moves in conjunction with the movement of the user's gaze. Note that the single-point AF 302 does not move just by moving the gaze pointer 310. In other words, the position of the single-point AF frame 302 is the same in Figures 3(a) and 3(b).

3(c) is a display example in the EVF 29 when the gaze confirmation button 82 is pressed in the state of FIG. 3(b). When the gaze confirmation button 82 is pressed with the gaze pointer 310 displayed, the one-point AF frame 302 is set (moved) to the gaze input position at that time (the position obtained by averaging the positions detected by the gaze detection unit 160 for a predetermined period). When the gaze pointer 310 is not in contact with the edge of the screen, the gaze pointer 310 is displayed in a range centered on the gaze input position, so the one-point AF frame is displayed at the center of the position where the gaze pointer 310 was. In addition, the position designation by the gaze is confirmed, and the gaze pointer 310 is not displayed. In this way, the position where AF is performed can be moved based on the position of the gaze input. The icon 303 indicates that the confirmation state of the position designation by the gaze must be released when the AF frame is to be moved based on the gaze input again, and indicates the operation method for releasing the confirmation state. The character string "Eye" indicates the gaze confirmation button 82, and indicates that the confirmation state can be released by pressing the gaze confirmation button 82. When the gaze confirmation button 82 is pressed from the state shown in FIG. 3(c), the confirmation state is released and the display returns to the state shown in FIG. 3(a) or FIG. 3(b).

3(d) is a display example when touchdown on the touch panel 70a is detected from the state of FIG. 3(b). When touchdown occurs with the gaze pointer 310 displayed, the one-point AF frame 302 is set (moves) to the gaze input position at that time (the position detected by the gaze detection unit 160 is averaged for a predetermined period). Then, the state becomes one in which the AF frame can be moved by the touch & drag AF function (the state in which the AF frame is in the middle of moving by the touch & drag AF function). The touch & drag AF function is a function that moves the AF frame displayed on the EVF 29 by touching and moving the touch panel 70a, which is located in a different position from the EVF 29. Since the user can accurately specify the desired position by touch operation, if the pointer indicating the touch position (the one-point AF frame 302 itself in this example) is large, it is cumbersome to be unable to specify a fine position. Therefore, a large pointer such as the gaze pointer 310 is hidden, and the state is set to one in which the position is specified by the one-point AF frame 302, which is smaller than the gaze pointer 310.

Figure 3(e) is a display example in which a touch-move to the lower left on the touch panel 70a is detected from the state shown in Figure 3(d), and the single-point AF frame 302 is moved to the lower left in response to the detected touch-move by specifying a relative position.

Figure 3(f) is an example of the display when there is a touch-up from the state of Figure 3(e). The movement of the single-point AF frame 302 by touch and drag has ended, and an icon 303 is displayed. In this way, it is possible to specify the position of the AF frame (selected position) by combining gaze input and touch operation. Note that if the touch panel 70a is further touched from the state of Figure 3(f) or Figure 3(c) to perform a touch move, the single-point AF frame 302 can be moved further in response to the touch move.

In the above example, the position on which the user wants to focus is the license plate of the car (subject) included in the LV image 301. In this case, the one-point AF frame 302 can be set to the position of the license plate as follows. First, as shown in FIG. 3B, the front part of the car in the LV image 301 is looked at, and the front part of the car is quickly and roughly specified with the gaze pointer 310. Then, the position of the one-point AF frame 302 set based on the gaze pointer 310 (the position of the one-point AF frame 302 in FIG. 3D) can be moved and fine-tuned by a touch operation to accurately match the position of the license plate. The amount of movement of the touch move at this time is small because the one-point AF frame 302 has already been set near the license plate based on the gaze input, and the amount of movement is from here. In this way, according to this embodiment, it is possible to quickly and accurately specify the position desired by the user.

<Setting the AF frame by gaze in Face + Tracking Priority AF>
4(a) to 4(g) will be used to explain AF frame movement control using eye gaze input when the AF frame selection method (AF method) is set to "face + tracking priority" in the digital camera 100. Each of Figs. 4(a) to 4(g) is a display example displayed on the EVF 29 when the AF frame selection method (AF method) is set to "face + tracking priority."

Figure 4(a) is an example of a display when the gaze function is enabled and the gaze detection unit 160 is detecting the gaze of the user. The same elements as those explained in Figure 3(a) are given the same reference numerals and explanations are omitted. In face + tracking priority, as in the case of single-point AF, the gaze pointer 310 is displayed at a relatively large first size based on a position averaged over a predetermined period of time. The face frames 401 to 405 are indicators that show the position of the face of a person detected from the LV image. In the state of Figure 4(a), no face is selected.

Figure 4(b) is an example of the display on the EVF 29 when the user moves their gaze from the state shown in Figure 4(a) and changes the location of the EVF 29 they are looking at. The gaze pointer 310 was on the left side of the screen in Figure 4(a), but has moved to the top right of the screen in Figure 4(b).

Figure 4(c) is a display example in the EVF 29 when the gaze confirmation button 82 is pressed in the state of Figure 4(b). When the gaze confirmation button 82 is pressed in the state where the gaze pointer 310 is displayed, a subject to be tracked (also an AF subject) is automatically selected within the range indicated by the gaze pointer 310 at that time according to the above-mentioned automatic selection conditions. In the example of Figure 4(c), among the faces (face frames 402 and 403) whose entirety is included within the gaze pointer 310 shown in Figure 4(b), the face indicated by the face frame 402, which is the face closest to the face, is selected and set as the tracking subject. The tracking frame 412 is displayed on the subject to be tracked, and the face frame is hidden. Then, tracking is started. During tracking, even if the subject to be tracked moves, the tracking frame moves to track the tracking target. Because the gaze pointer 310 narrows down the zone in which the subject is selected, subjects outside the gaze pointer 310 are not selected, and faces and cars indicated by face frames 401 and 405 are not selected. In other words, the tracking target is set from within a range specified by the user quickly and roughly by the gaze, making it possible to select a subject that matches the user's intention better than automatic selection without using the gaze. Also, in FIG. 4(c), the position specified by the gaze is confirmed, and the gaze pointer 310 is hidden. When the gaze confirmation button 82 is pressed from the state of FIG. 4(c), the confirmation state is released, and the display returns to the state of FIG. 4(a) or FIG. 4(b).

FIG. 4D is a display example when touchdown on the touch panel 70a is detected from the state of FIG. 4B. When touchdown occurs with the gaze pointer 310 displayed, the touch pointer 406 is displayed at the gaze input position at that time (the position obtained by averaging the positions detected by the gaze detection unit 160 for a predetermined period). Then, the state becomes one in which the position can be specified by the touch & drag AF function (the state in which the touch pointer 406 is in the middle of moving by the touch & drag AF function). Since the user can accurately specify the desired position by the touch operation, if the pointer indicating the touch position (the touch pointer 406 in this example) is large, it is cumbersome to be unable to specify a fine position. Therefore, a large pointer such as the gaze pointer 310 is not displayed, and the state is one in which the position is specified by the touch pointer 406, which is smaller than the gaze pointer 310. This makes it easier for the user to select the desired face even when the faces are crowded together as shown in the figure.

Fig. 4(e) is a display example in which, from the state of Fig. 4(d), a touch move to the upper right on the touch panel 70a is detected, and the touch pointer 406 is moved to the upper right according to the detected touch move by specifying a relative position. The touch pointer 406 is in a position that almost coincides with the position of the face frame 403 (more precisely, a position where the center of the touch pointer 406 is included within the range of the face frame 403). When a touch-up is performed in this state, the face frame 403 is specified as the tracking target based on the position of the touch pointer 406. Note that instead of the display as shown in Fig. 4(e), when the positional relationship is reached where the face frame can be specified by the touch pointer 406 while the touch pointer 406 is moving before the touch-up, a display (adhesive display) showing the face that would be specified if a touch-up occurs at that time may be performed.

Figure 4(f) shows an example of the suction display. The touch pointer 406 is moved in response to a touch move, and when the face frame 403 is in a position where it can be specified, the touch pointer 406 is made invisible and the face frame 403 is displayed in a different display form from the other face frames. In this way, the user can recognize that if they touch up at this point, the face frame 403 will be specified, making it easier to determine whether or not they have performed a touch move to the desired position.

Figure 4(g) is an example of the display when there is a touch-up from the state of Figure 4(e) or Figure 4(f). Based on the position of the touch pointer 406 immediately before the touch-up, the face frame 403 is set as the tracking target, the tracking frame 413 is displayed, and tracking begins. The movement of the touch pointer 406 by touch and drag is completed, and the icon 303 is displayed. Note that if the touch panel 70a is further touched from the state of Figure 4(g) or Figure 4(c) to perform a touch move, tracking is released and the touch pointer 406 is displayed at the position where the tracking target was located, and the touch pointer 406 can be moved in accordance with the touch move.

In the above example, the position on which the user wants to focus is the face indicated by the face frame 403 included in the LV image 301. In this case, the tracking target (AF position) can be set to the position of the face frame 403 as follows. First, as shown in FIG. 4B, the area is specified roughly and quickly by looking at the part of the LV image 301 near the face frame 403. Then, the touch pointer 406 is moved by a touch operation from the position of the face frame 402, which is the tracking target set based on the gaze pointer 310, to fine-tune it, so that it can be accurately aligned with the face frame 403. The amount of movement of the touch move at this time is small because the face frame 402 near the face frame 403 has already been set based on gaze input, and the amount of movement is from here. In this way, according to this embodiment, it is possible to quickly and accurately specify the position (subject) desired by the user.

Next, we will explain the process that realizes the operations and screen transitions described above.

<Shooting mode processing>
5A and 5B are flowcharts of the shooting mode process in the digital camera 100 of this embodiment. The processes in Fig. 5A and Fig. 5B are processes when the display destination is the EVF 29. The processes in the flowcharts of Fig. 5A to Fig. 10, including Fig. 5A and Fig. 5B, are realized by the system control unit 50 expanding a program stored in the non-volatile memory 56 into the system memory 52 and executing it.
When the CPU 10 is started in the shooting mode, flags, control variables, etc. are initialized and the processing of Figs. 5A and 5B is started.

In S500, the system control unit 50 starts capturing a live view image (LV image) in the imaging unit 22 and displays the captured LV image on the EVF 29.

In S501, the system control unit 50 performs a camera setting process for making various settings related to image capture in response to a user operation. The camera setting process will be described later with reference to FIG.
The details of the determination process will be described later with reference to FIG.

In S502, the system control unit 50 determines whether the line-of-sight function is enabled (whether the line-of-sight AF setting described below is enabled). If it is enabled, the process proceeds to S503, and if it is not enabled (disabled), the process proceeds to S516.

In S503, the system control unit 50 determines whether or not the gaze is detected by the gaze detection unit 160. If it is detected, the process proceeds to S504, and if not (the gaze function is enabled but the gaze is not detected), the process proceeds to S516.

In S504, the system control unit 50 determines whether the gaze pointer display is enabled. If it is enabled, the process proceeds to S505, and if it is not enabled (disabled), the process proceeds to S507.

In S505, the system control unit 50 determines whether the gaze confirmation flag stored in the system memory is 0. The initial value is 0. A gaze confirmation flag of 0 indicates that the gaze confirmation state described above has been released, and the gaze pointer can be moved by the gaze. This is also a "coarse adjustment mode" in which a rough position can be specified by the gaze. On the other hand, a gaze confirmation flag of 1 indicates the gaze confirmation state described above, and is a state in which a rough position has been specified by the gaze once, and it is no longer possible to specify a position by the gaze. This is also a "fine adjustment mode" in which a position can be specified in detail by touch-move. If the gaze flag is 0, proceed to S506, and if not (if the gaze flag is 1), proceed to S507.

In S506, the system control unit 50 displays the gaze pointer 310 on the EVF 29 based on the gaze input position detected by the gaze detection unit 160. As described above, the gaze pointer 310 is displayed in a relatively large first size based on the average position of the gaze input position for a predetermined period. If the averaged gaze detection position is not near the edge of the EVF 29, the gaze pointer 310 is displayed in a range of the first size centered on the gaze input position. If the averaged gaze detection position is near the edge of the EVF 29, the gaze pointer 310 is displayed in a range of the first size adjacent to the edge of the screen close to the gaze input position. If the AF method is set to single-point AF by the processing of S506, the display will be as shown in Figures 3(a) and 3(b) described above, and if it is set to face + tracking priority, the display will be as shown in Figures 4(a) and 4(b) described above.

In S507, the system control unit 50 determines whether the gaze confirmation button 82 has been pressed (i.e., whether an operation has been performed to instruct the execution of position specification by gaze). If the gaze confirmation button 82 has been pressed, the process proceeds to S508, and if not, the process proceeds to S516.

In S508, the system control unit 50 determines whether the gaze confirmation flag stored in the system memory 52 is 0. If the gaze confirmation flag = 0, the process proceeds to S512, and if not (if the gaze confirmation flag = 1), the process proceeds to S509.

In S509, the system control unit 50 sets the gaze confirmation flag to 0. In addition, the icon 303 that was displayed is hidden, and the display returns to the state in which the gaze confirmation has been released.

In S510, the system control unit 50 determines whether the currently set AF method is face + tracking priority AF. If it is face + tracking priority AF, the process proceeds to S511, where tracking is released, and the process proceeds to S504. As a result, for example, when the display of FIG. 4(c) or FIG. 4(g) is performed, the display state of FIG. 4(a) or FIG. 4(b) is transitioned to in response to the line of sight confirmation button 82 being pressed. In S510, if it is determined that the currently set AF method is not face + tracking priority AF (i.e., it is determined that it is single-point AF), the process proceeds to S504. As a result, for example, when the display of FIG. 3(c) or FIG. 3(f) is performed, the display state of FIG. 3(a) or FIG. 3(b) is transitioned to in response to the line of sight confirmation button 82 being pressed.

In S512, the system control unit 50 sets the gaze confirmation flag to 1. In addition, the system control unit 50 displays the icon 303 on the EVF 29 to indicate the gaze confirmation state.

In S513, the system control unit 50 determines whether the currently set AF method is face + tracking priority AF. If it is face + tracking priority AF, the process proceeds to S514, and if it is not (i.e., if it is single-point AF), the process proceeds to S515.

In S514, within a range of a first size indicated by the gaze pointer 310 (within the same range even when the gaze pointer 310 is not displayed), a subject to be tracked is selected based on the automatic selection conditions described above. Then, a tracking frame is displayed around the selected subject (tracking subject), and tracking begins. This causes the display to transition, for example, from that shown in FIG. 4(b) to that shown in FIG. 4(c).

In S515, the system control unit 50 sets the single-point AF frame 302 at the gaze input position (the position obtained by averaging the positions detected by the gaze detection unit 160 for a predetermined period) at the time when the gaze confirmation button 82 is pressed. This causes the display to transition from FIG. 3(b) to FIG. 3(c), for example. In this embodiment, an example has been described in which the single-point AF frame is set at the gaze input position detected by the gaze detection unit 160 in the case of single-point AF. However, this is not limited to this, and even in the case of single-point AF, automatic selection based on the automatic selection conditions may be performed within the range of the gaze pointer 310, as in the case of face + tracking priority AF, and the single-point AF frame 302 may be set at the position of the automatically selected subject.

In S516, the system control unit 50 determines whether or not there has been a touch-down on the touch panel 70a. If there has been a touch-down, the process proceeds to S517; if not, the process proceeds to S518.

In S517, the system control unit 50 performs touch operation response processing in response to a touch operation on the touch panel 70a. The touch operation response processing will be described later with reference to FIG. 7.

In S518, the system control unit 50 determines whether or not any other operation has been performed on the operation unit 70. If any other operation has been performed, the process proceeds to S519; if not, the process proceeds to S520.

In S519, the system control unit 50 performs processing in response to other operations, such as changing various shooting parameters such as the shutter speed, aperture value, and exposure compensation value, and setting the recording image quality and the self-timer.

In S520, the system control unit 50 determines whether the first shutter switch signal SW1 is turned on, i.e., whether the shutter button 61 has been half-pressed and a shooting preparation command has been issued.

In S521, the system control unit 50 determines whether the gaze pointer 310 is being displayed, that is, whether the gaze function is enabled, gaze detection is being performed, the gaze pointer display is enabled, and the gaze confirmation flag = 0. If the gaze pointer 310 is being displayed, the process proceeds to S522, and if not, the process proceeds to S523. Note that instead of determining whether the gaze pointer 310 is being displayed, it may be determined whether the gaze function is enabled, gaze detection is being performed, and the gaze confirmation flag = 0. In this case, if the gaze function is enabled, gaze detection is being performed, and the gaze confirmation flag = 0, the process proceeds to S522 even if the gaze pointer display is disabled (the gaze pointer 310 is not displayed).

In S522, the system control unit 50 selects a subject as the AF target within a range of a first size indicated by the gaze pointer 310 (within the same range even when the gaze pointer 310 is not displayed) based on the automatic selection conditions described above. This is the same process as the selection of the tracking target in S514. Then, AF is performed based on the selected subject (AF target, focus adjustment target). Similarly, AF, AWB, and other processes may be performed based on the selected subject. Note that when the AF method is single-point AF, the AF target may not be selected based on the automatic selection conditions, but may instead be selected within the range of a single-point AF frame centered on the gaze input position at that time.

In S523, the system control unit 50 determines whether the currently set AF method is face + tracking priority AF. If it is face + tracking priority AF, the process proceeds to S524, and if it is not (single-point AF), the process proceeds to S527.

In S524, the system control unit 50 determines whether or not the subject is being tracked. If the subject is being tracked, the process proceeds to S526; if not, the process proceeds to S525.

In S525, the system control unit 50 selects a subject to be the AF target based on the above-mentioned automatic selection conditions for the entire range of the captured LV image. Then, AF is performed based on the selected subject (AF target, focus adjustment target). In the same manner, AF, AWB, and other processing may be performed based on the selected subject. Note that the target is not limited to the entire range of the LV image, and may be within a range of a second size larger than the first size, which is the size of the gaze pointer 310, of the LV image. For example, a range of 80% from the center of the LV image (>first size) may be set as the target range for automatic selection of subjects based on the automatic selection conditions in S525. In this case, it is assumed that there is a low possibility that a main subject to be adjusted for AF exists in the other edge areas, so they are not subject to automatic selection of subjects based on the automatic selection conditions in S525.

In S526, the system control unit 50 performs AF in the tracking frame being tracked (i.e., for the tracking target). Similarly, AF, AWB, and other processes may be performed based on the tracking target.

In S527, the system control unit 50 performs AF in the one-point AF frame that has been set. Similarly, AF, AWB, and other processes may be performed based on the one-point AF frame.

In S528, the system control unit 50 determines whether the second shutter switch signal SW2 is on, that is, whether or not a shooting instruction has been issued by fully pressing the shutter button 61. If SW2 is on, the process proceeds to S530, and if not, the process proceeds to S529.

In S529, the system control unit 50 determines whether SW1 is held on. If it is held on, the process returns to S528, and if not (SW1 is turned off), the process proceeds to S531.

In S530, the system control unit 50 performs a series of image capture processes (the aforementioned image capture process) from exposure in the image capture unit 22 to recording the captured image as an image file on the recording medium 200.

In S531, it is determined whether an end event for the shooting mode has occurred (such as a power-off operation or an instruction to transition to another operating mode such as playback mode). If no end event has occurred, the process returns to S500 and is repeated, and if an end event has occurred, the shooting mode process ends.

<Camera setting process>
The camera setting process in S501 of Fig. 5A described above will be described below. The camera setting process is a process for setting each setting item on the setting menu screen related to photography that is displayed when the menu button 81 is pressed.

FIGS. 11(a) and 11(b) show examples of a setting menu screen related to shooting that is displayed on the EVF 29 or the display unit 28. Setting item 1101 included in the menu screen in FIG. 11(a) is an item for setting the AF method. Setting item 1102 is an item for setting touch-and-drag AF. Setting item 1103 is an item for setting related to the line-of-sight function. Setting item 1104 is an item for setting the operation when the center part of the MC65 is pressed.

Figure 11 (b) is an example of the display of a gaze AF detailed settings menu screen for configuring settings related to the gaze function. This screen is displayed when setting item 1103 in Figure 11 (a) is selected. Setting items 1105 to 1107 are displayed on the gaze AF detailed settings menu screen in Figure 11 (b). Setting item 1105 is an item for setting the gaze function to enabled or disabled. Setting item 1106 is an item for setting the display of the gaze pointer to enabled (displayed) or disabled (not displayed). Setting item 1107 is an item for setting the function of jumping the AF frame to the gaze detection position when SW1 is turned on to enabled or disabled.

Figure 6 is a flowchart showing the details of the camera setting process of S501 in Figure 5A described above.

In S601, the system control unit 50 determines whether an operation to switch the gaze function (gaze AF) ON/OFF (enable/disable) has been performed on the operation unit 70. In this embodiment, the operation to switch the gaze function ON/OFF (enable/disable) is an operation to open a menu screen, select a corresponding setting item (setting item 1105), and switch the setting. In this embodiment, turning gaze AF ON enables the function to input the user's gaze, and turning gaze AF OFF disables the function. If an operation to switch the gaze function ON/OFF has been performed, the process proceeds to S602, and if not, the process proceeds to S603.

At S602, the system control unit 50 switches the gaze function ON/OFF and records the changed settings in the non-volatile memory 56.

In S603, the system control unit 50 determines whether an operation to switch the gaze pointer display ON/OFF (enabled/disabled) has been performed on the operation unit 70. In this embodiment, the operation to switch the gaze determination function ON/OFF (enabled/disabled) is an operation to open a menu screen, select a corresponding setting item (setting item 1106), and switch the setting. In this embodiment, when the gaze pointer display is turned ON, the gaze pointer 310 is displayed as a GUI in response to the user's gaze input, and when the gaze pointer display is turned OFF, the gaze pointer is hidden. If an operation to switch the gaze pointer display ON/OFF has been performed, the process proceeds to S604, and if not, the process proceeds to S605.

In S604, the system control unit 50 switches the gaze pointer display ON/OFF (enabled/disabled) and records the changed settings in the non-volatile memory 56.

In S605, the system control unit 50 determines whether an operation to switch the settings of the touch-and-drag AF function has been performed on the operation unit 70. In this embodiment, the operation to switch the settings of the touch-and-drag AF function is an operation to open a menu screen, select a corresponding setting item (setting item 1102), and switch the setting. In this embodiment, the touch-and-drag AF setting can be selected from either "absolute (the above-mentioned absolute position specification method)" or "relative (the above-mentioned relative position specification method)." If an operation to switch the touch-and-drag AF function has been performed, the process proceeds to S606, and if not, the process proceeds to S607.

In S606, the system control unit 50 switches the settings of the touch-and-drag AF function and records the changed settings in the non-volatile memory 56.

In S607, the system control unit 50 determines whether an operation to switch the AF method has been performed on the operation unit 70. In this embodiment, the operation to switch the AF method is an operation to open a menu screen, select a corresponding setting item (setting item 1101), and switch the setting. If an operation to switch the AF method has been performed, the process proceeds to S608, and if not, the camera setting process ends. Note that, although an example has been described in which either face + tracking priority AF or single point AF can be selected as the AF method in this embodiment, other AF methods (such as zone AF and multi-point AF) may be set.

In S608, the system control unit 50 switches the AF method and records the changed settings in the non-volatile memory 56.

<Touch operation response processing>
FIG. 7 shows a detailed flowchart of the touch operation response process in S517 of FIG. 5A described above.

In S701, the system control unit 50 determines whether the line of sight function is enabled, as in S502. If it is enabled, the process proceeds to S702, and if it is not enabled (disabled), the process proceeds to S708.

In S702, the system control unit 50 determines whether or not a line of sight has been detected, similar to S503. If a line of sight has been detected, the process proceeds to S703; if not, the process proceeds to S708.

At S703, the system control unit 50 sets the gaze confirmation flag to 1.

At S704, the system control unit 50 hides the gaze pointer.

In S705, the system control unit 50 determines whether or not the setting of the touch-and-drag AF in the setting item 1102 in Fig. 11A described above is the relative position designation method. If it is the relative position designation method, the process proceeds to S706, and if not (if it is the absolute position designation method), the process proceeds to S707.

In S706, the system control unit 50 performs a relative position designation process when the line of sight is active. The details of this process will be described later with reference to FIG. 8.

In S707, the system control unit 50 performs absolute value specification processing. Details of this processing will be described later with reference to FIG. 10.

In S708, the system control unit 50 determines whether the touch-and-drag AF setting is the relative position designation method, as in S705. If it is the relative position designation method, the process proceeds to S709, and if not (if it is the absolute position designation method), the process proceeds to S710.

In S709, the system control unit 50 performs a process for specifying a relative position when the line of sight is invalid. The details of this process will be described later with reference to FIG. 9.

In S710, the system control unit 50 performs absolute position specification processing. This processing is similar to the processing in S709, and will be described later with reference to FIG. 10.

In the above example, when the gaze function is enabled (S701-Yes) and the gaze is detected (S702-Yes), it is determined whether the touch and drag AF setting is the relative position value designation method in S705, and when the absolute position designation method is used, the absolute position designation process is performed in S707. However, this is not limited to the above. When the gaze function is enabled (S701-Yes), or when the gaze function is enabled and the gaze is detected (Yes in both 701 and S702), the gaze-enabled relative position designation process may be performed regardless of the touch and drag AF setting. That is, if S701 is Yes, the process of S703 and S704 may be performed without determining S702, and then the process of S706 may be performed without determining S705. Alternatively, if S701 is Yes and S702 is also Yes, the process of S703 and S704 may be performed, and then the process of S706 may be performed without determining S705. By doing this, when the gaze function is available, you can specify a rough position by gaze, and then make further fine adjustments from the position specified by gaze by specifying a relative position using touch operations.

<Relative position designation process when line of sight is enabled>
FIG. 8 shows a detailed flowchart of the line-of-sight-valid relative position designation process in S706 of FIG.

In S801, the system control unit 50 determines whether the AF method is "face + tracking priority AF." If it is "face + tracking priority AF," the process proceeds to S805, and if it is not (in this embodiment, it is "single point AF"), the process proceeds to S802.

In S802, the system control unit 50 displays a single-point AF frame at the gaze input position (the position obtained by averaging the positions detected by the gaze detection unit 160 over a predetermined period) at the time of touch-down. This causes a transition from the display in FIG. 3(b) to the display in FIG. 3(d).

In S803, the system control unit 50 moves the single-point AF frame in response to a touch-move on the touch panel 70a. This movement is performed using a relative position designation method.

In S804, the system control unit 50 judges whether or not there has been a touch-up from the touch panel 70a. If there has been a touch-up, the system control unit 50 displays the icon 303 (which results in the display of FIG. 3(f) described above), and ends the processing of FIG. 8. If there has been no touch-up, the system control unit 50 returns to S803.

In S805, the system control unit 50 determines whether or not the subject is being tracked. If the subject is being tracked, the process proceeds to S810; if not, the process proceeds to S806.

In S806, the system control unit 50 displays a touch pointer 406 (a second item indicating a second size range; a small pointer) indicating the touch position (selected position) at the gaze input position at the time of touch-down. This causes a transition from the display in FIG. 4(b) to that in FIG. 4(d), for example. Here, the gaze input position is, for example, a position obtained by averaging the positions detected by the gaze detection unit 160 over a predetermined period of time.

In S807, the system control unit 50 moves the touch pointer 406 indicating the touch position in response to a touch move (movement instruction operation) on the touch panel 70a. This movement is a movement by the relative position designation method. This causes the display to transition, for example, from FIG. 4(d) to FIG. 4(e). It can also be considered that the system control unit 50 moves the display position of the touch pointer 406 in response to the touch move, not based on the gaze input after the touch pointer 406 is displayed.

In S808, the system control unit 50 determines whether or not there has been a touch-up from the touch panel 70a. If there has been a touch-up, the process proceeds to S809; if not, the process returns to S807.

In S809, the system control unit 50 selects the subject at the position of the touch pointer and starts tracking the subject. The selection at this time is not based on the automatic selection conditions described above. In addition, the icon 303 is displayed. This causes the display to transition, for example, from FIG. 4(e) to FIG. 4(g).

At S810, the system control unit 50 displays a touch pointer 406 (a second item indicating a second size range; a small pointer) indicating the touch position (selected position) at the position of the tracking target at the time of touchdown.

The processes in steps S811 to S813 are similar to the processes in steps S806 to S809 described above, and therefore will not be described here.

In this embodiment, an example has been described in which if a touchdown occurs while the gaze is valid, and tracking is in progress, the touch pointer 406 is displayed at the tracking position, not at the gaze-probable position. However, this is not limited to this, and if the gaze is valid, regardless of whether tracking is in progress or not, if a touchdown occurs, the touch pointer 406 may be displayed at the gaze input position (a position obtained by averaging the positions detected by the gaze detection unit 160 for a predetermined period of time). In this case, if S801 is Yes, the process proceeds to S806 without making the determination in S805.

In addition, when subject tracking is performed by touching up using the relative position designation method (S809, S813, S908 and S912 described below), subject tracking may be started a certain time after touching up. This makes it easier to move the touch pointer when moving the touch pointer by repeatedly performing a series of operations of touching down, touching move, and touching up using the relative position designation method, since subject tracking processing is not triggered every time.

<Relative position specification process when line of sight is disabled>
FIG. 9 is a detailed flowchart of the line-of-sight invalid relative position designation process in S709 of FIG.

In S901, the system control unit 50 determines whether the AF method is "face + tracking priority AF." If it is "face + tracking priority AF," the process proceeds to S902, and if it is not (in this embodiment, it is "single point AF"), the process proceeds to S904.

In S902, the system control unit 50 moves the single-point AF frame in response to a touch move on the touch panel 70a. Note that because gaze input has not been detected and the relative position designation method is used, the position of the single-point AF frame does not move in response to a touch down, but moves by the relative position designation method by a distance corresponding to the touch move from the position of the single-point AF frame that was set before the touch down.

In S903, the system control unit 50 determines whether or not there has been a touch-up from the touch panel 70a. If there has been a touch-up, the process of FIG. 9 ends, and if there has not been a touch-up, the process returns to S902.

In S904, the system control unit 50 determines whether or not the subject is being tracked. If the subject is being tracked, the process proceeds to S909; if not, the process proceeds to S905.

At S905, the system control unit 50 displays a touch pointer 406 (a second item indicating a second size range; a small pointer) indicating the touch position (selection position) in the center of the EVF 29.

The processes in steps S906 to S912 are similar to the processes in steps S807 to S813 in FIG. 7, respectively, and therefore will not be described here.

<Absolute position specification process>
FIG. 10 is a detailed flowchart of the absolute position designation process in steps S707 and S710 in FIG.

In S1001, the system control unit 50 determines whether the AF method is "face + tracking priority AF." If it is "face + tracking priority AF," the process proceeds to S1005, and if it is not (in this embodiment, it is "single point AF"), the process proceeds to S1002.

In S1002, the system control unit 50 displays a single-point AF frame at a position on the EVF 29 that corresponds to the touch position (touch-down position) when touched down.

In S1003, the system control unit 50, in response to the touch-move, moves the single-point AF frame to a position on the EVF 29 that corresponds to the touch position after the touch-move. This movement is performed using an absolute position specification method, and the position of the single-point AF frame after the movement is not based on the distance or direction of the touch-move, but is a position that corresponds to the most recent touch position.

In S1004, the system control unit 50 determines whether or not there has been a touch-up from the touch panel 70a, and if there has been a touch-up, ends the processing of FIG. 10, and if not, returns to S1003.

In S1005, the system control unit 50 displays the touch pointer 406 at a position on the EVF 29 that corresponds to the touch position (touch-down position) at the time of touch-down.

In S1006, the system control unit 50 moves the touch pointer 1006 in response to the touch move to a position on the EVF 29 that corresponds to the touch position after the touch move. This movement is performed using an absolute position specification method, and the position of the touch pointer 1006 after the movement is not based on the distance or direction of the touch move, but is a position that corresponds to the most recent touch position.

In S1007, the system control unit 50 determines whether or not there has been a touch-up from the touch panel 70a. If there has been a touch-up, the process proceeds to S1008; if not, the process returns to S1006.

In S1008, the system control unit 50 selects the subject at the position of the touch pointer and starts tracking the subject. It also displays the icon 303 and ends the processing in FIG. 10.

According to the present embodiment described above, when the user performs gaze input, a large pointer is displayed to roughly select the subject, and when the user performs touch operation, a small pointer is displayed to specify a fine position. This makes it possible to easily specify the intended position when specifying a position using gaze input and touch operation by the user.

In the above embodiment, an example was described in which a small pointer (touch pointer) was displayed in response to a touch operation, but this is not limited to this. It is also possible to specify a position for fine adjustment in response to operation of an operating member such as the MC 65 or the four-way key 74. Therefore, when a position has been roughly specified using the gaze pointer, in response to operation of the MC 65 or the four-way key 74, a fine adjustment position specification pointer (pointer of a second size) may be displayed instead of the gaze pointer. Then, by moving this fine adjustment position specification pointer in response to operation of the MC 65 or the four-way key 74, fine adjustment of the position specification may be performed.

The conditions for canceling the gaze confirmation flag are not limited to the above example. After performing the shooting process once in S530, the gaze confirmation flag may be automatically reset to 0. Furthermore, the gaze confirmation flag may be set to 1 and the gaze function (gaze AF setting) may be "disabled" at the same time, and the gaze flag may be returned to 0 at a timing of the user's choice (the timing when the gaze function was enabled in S602). Furthermore, the gaze flag may be automatically returned to 0 after a certain time has elapsed after the first shutter button 62 is turned off (after the half-press of the shutter button 61 is released).

In addition, although an example of making the pointer smaller after a touch operation has been described, the extent to which the pointer is made smaller may be changed according to the camera settings and use case. For example, in the case of servo AF mode (second AF mode), the size of the gaze pointer or touch pointer may be made larger than that in one-shot AF mode (first AF mode). Here, the servo AF mode is a mode that performs continuous AF, which continues to perform AF at the position of the AF frame. It is suitable to use when photographing a moving subject. The one-shot AF mode is an AF processing mode that stops AF processing (locks the focus; fixes the focus position) once AF is performed unless there is an AF instruction again. Specifically, in the servo AF mode, when a touch operation is not performed during gaze input, a pointer larger than the above-mentioned gaze pointer 310 (a third item indicating a range of a third size larger than the first size) is displayed at a position based on the gaze input. In this way, even if the subject to which AF is to be adjusted is a moving object and does not remain in a narrow range, it is only necessary to fit it into a larger gaze pointer, making it easier to specify the desired subject by gaze. In addition, when AF processing is performed while the gaze pointer is displayed, AF processing may be performed in servo AF mode, and when AF processing is performed while the touch pointer is displayed, AF processing may be performed in one-shot AF mode.

In addition, when the AF method is set to zone AF instead of face + tracking priority AF, the gaze pointer may be enlarged to a size equivalent to the size of the zone in zone AF processing. In other words, the size of the gaze pointer displayed in zone AF may be set to a fourth size larger than the first size displayed in face + tracking priority AF. Here, zone AF is an AF method suitable for photographing moving objects, in which a larger zone is set at a position set by the user, and subjects within that zone that meet the automatic selection conditions are targeted for AF.

In the above embodiment, an example was described in which the range in which AF processing is performed is changed (reduced) in response to a touch operation, but the size of the area in which the subject is detected or the type of subject to be detected may also be changed in response to a touch operation. For example, when AF processing is performed with the gaze pointer displayed, AF processing may be performed on the detected face, and when AF processing is performed with the touch pointer displayed, AF processing may be performed on the detected eyes.

Note that when the display is switched from the gaze pointer to the touch pointer, the position at which the touch pointer is displayed is not limited to the above. For example, the touch pointer may be displayed at the position of the subject detected within the gaze pointer.

<Modification 1>
In the above embodiment, an example has been described in which, when selecting a subject with the digital camera 100, if no touch operation is performed during gaze input, a gaze pointer (large pointer) is displayed, and if a touch operation is performed, a touch pointer (small pointer) is displayed. However, the present application is not limited to subject selection, and can also be applied to position designation for other purposes. That is, even in a position designation operation other than subject selection, the following position selection is possible. When no operation is performed on an operating member (e.g., a touch panel) during gaze input, a large pointer (indicator) is displayed, and the large pointer is moved according to the gaze input to roughly designate the position. Then, within the range roughly designated by the gaze input, a fine position is designated based on the automatic selection conditions. Then, when an operation is performed on an operating member (e.g., a touch panel), the display is switched from the large pointer to a small pointer, and the small pointer (indicator) is moved according to the operation on the operating member to finely designate the position.

FIGS. 12(a) to 12(c) show examples of display when the present application is applied to the operation of selecting the position at which to insert characters in a string of characters, as an example (variant example) of a position designation operation that is different from subject selection.

FIG. 12A shows a display example in which a position for inserting characters is selected by using gaze input from within an input character string of "This is the piel.". A gaze pointer 1200 (following the gaze input position) is displayed in a predetermined range based on the gaze input position. In this state, when a gaze confirmation operation (an operation for instructing execution of position specification by gaze) is performed, a character insertion position is determined based on an automatic selection condition within the range indicated by the gaze pointer 1200. In this modified example, the automatic selection condition is a condition that "among phrases between words, the position closest to the gaze input position." In the character string of "This is the piel.", phrases 1201 to 1203 are detected as shown in the figure. When a gaze confirmation operation is performed with the gaze pointer 1200 at the position of FIG. 12A, a phrase 1203 included in the gaze pointer 1200 is selected, and a character can be inserted at this position by a subsequent character input operation. When the gaze pointer 1200 is present and a position is specified according to a gaze confirmation operation, a position within a word cannot be selected. For example, in the state of FIG. 12(a), a position between "i" and "e" within the word "piel" cannot be selected. This is because, as described above, it is difficult to specify a precise position using a gaze input position due to human characteristics, and by limiting the insertion position to a phrase where character insertion is likely to be performed, it is possible to reduce the selection of an unintended position. In this way, a character insertion position can be quickly and easily selected by simply looking at the position where a character is to be inserted. Also, when a position specification operation is performed using an operating member such as an operation on the MC 65 or the four-way key 74 in the state of FIG. 12(a), the gaze pointer 1200 is hidden and the insertion position cursor 1210 is displayed at the gaze input position. Then, the position of the insertion position cursor 1210 can be moved (fine-adjusted) from the position where the insertion position cursor 1210 is displayed according to an operation using the operating member.

FIG. 12B shows an example of a display when an operation is performed on the MC 65 or the four-way key 74 in the state shown in FIG. 12A. The gaze pointer 1200 is hidden, and an insertion position cursor 1210 is displayed between "i" and "e" inside the word "piel". The insertion position cursor 1210 is a pointer (indicator) of a second size indicating the insertion position, and is smaller than the gaze pointer 1200 of the first size. As shown in the figure, the insertion position cursor 1210 can specify a position between each character inside a word, and can be moved arbitrarily to positions such as positions 1211 to 1215 in response to an operation on the MC 65 or the four-way key 74. For example, when the character "x" is inserted with the insertion position cursor 1210 positioned at the position shown in FIG. 12B, "This is the
The string can be modified to something like "pixel."

<Modification 2>
The present invention can be applied not only to the examples described in the above-mentioned embodiments, but also to situations in which it is required to quickly and accurately select a subject or a specified target. For example, the present invention can be applied when a user wearing a headset including a head mounted display (HMD) selects a target such as a virtual character displayed in a VR (Virtual Reality) image. When no operation is performed on an operating member (for example, a touch panel or an optical tracking pointer that can be touched and pressed) during gaze input, a large pointer (indicator) is displayed and the large pointer is moved according to the gaze input to roughly specify a position. Then, within the range where the position was roughly specified by the gaze input, a fine position is specified based on the automatic selection conditions. Then, when an operation is performed on an operating member, the display is switched from the large pointer to a small pointer (indicator), and the small pointer (indicator) is moved according to the operation on the operating member to finely specify a position.

Figures 13(a) to 13(c) show external views of the headset 1300 and the game controller 1320, which is an operating member.

FIGS. 13(a) and 13(b) show external views of a headset 1300. The headset 1300 mainly comprises an HMD 1310, and in this second modification, the HMD 1310 is a VR device (standalone VR-HMD) that can be used on its own without being connected to a personal computer or the like. The HMD 1310 contains components necessary for applying this embodiment, such as the system control unit 50 and gaze detection unit 160 shown in FIG. 2.

FIG. 13C shows the game controller 1320. The game controller 1320 accepts instructions for displaying a game screen 1400 (described later) displayed on the HMD 1310. The game controller 1320 includes a grip section 1321, a multi-button 1322 that is a depressable operation member having a touchable portion, and a SET button 75. The grip section 1321 is made of a structure and material that allows the user to easily hold the game controller 1320, similar to the grip section 90 of the digital camera 100 described above. The multi-button 1322 can be assigned various functions, such as a function to instruct the movement of a pointer displayed on the game screen 1400. In this second modified example, the multi-button 1322 is an operation member that can accept both a touch operation and a pressing operation. The touch operation member mounted on the multi-button 1322 may be a touch detection mechanism as described above, or an infrared sensor. An infrared sensor is mounted inside the multi-button 1322, and infrared rays are irradiated onto the upper part of the multi-button 1322 (the part touched by the user's finger). When the user touches the upper part of the multi-button 1322, the infrared rays are reflected by the user's finger, and the movement of the user's finger can be detected by detecting the reflected light. Such an optical operating member is called an optical tracking pointer (OTP). Like the touch panel 70a, the OTP can detect a movement operation, which is the movement of the user's finger (operating body) relative to the OTP. Note that the multi-button 1322 is not limited to an OTP, and may be a direction indicating member such as the MC65, the four-way key 74, or a joystick. Note that the HMD 1310 and the game controller 1320 may be connected by wire or wireless communication such as Bluetooth.

Figures 14(a) to 14(f) show examples of game screens displayed on the HMD 1310 in an example (variant) of a position designation operation for designating an attack target in an action game.

Figure 14(a) is an example of a game screen display showing a state in which multiple attack targets 1401-1404 are virtually approaching a user wearing a headset 1300 and playing an action game. At this time, a gaze pointer 1410 is displayed at a position corresponding to the user's gaze detected by the gaze detection unit 160. Then, as the user's gaze moves, the gaze pointer 1410 moves from the position shown in Figure 14(a) to the position shown in Figure 14(b). After that, when a gaze confirmation operation is performed using an operating member (specifically, pressing the multi-button 1322 or pressing the SET button 75), a character to be attacked is determined within the range indicated by the gaze pointer 1410 based on the automatic selection condition. In this modification example 2, the automatic selection condition is set to "the target closest to the user within the range of the gaze pointer."

As shown in FIG. 14(b), when a gaze confirmation operation is performed while targets 1401-1404 to be attacked are present on the game screen, a target frame 1411 is displayed. Specifically, according to the automatic selection conditions, of targets 1401-1403 present within the range of gaze pointer 1410, target 1401 that is virtually closest to the user is selected, and target frame 1411 is displayed (FIG. 14(c)). This allows the user to move their gaze to select target 1401 with target frame 1411 displayed to perform an attack. However, the character that the user actually wants to attack may not be the automatically selected target 1401, but another character (e.g., target 1402) that is within the range of gaze pointer 1410.

Figure 14 (d) shows an example of the display when, in the state of Figure 14 (b), there is a touch operation on the MC 65, the four-way key 74, or the OTP of the multi-button 1322. The gaze pointer 1410 is hidden, and a touch pointer 1412 is displayed. The touch pointer 1412 is the same as the touch pointer 406 in Figure 4. The touch pointer 1412 is smaller than the gaze pointer 1410. By displaying in this way, the user can specify a rough target (position) with the gaze pointer 1410, and then specify a finer target (fine-tune).

Figure 14(e) shows an example of a display when the user moves the touch pointer 1412 from the state of Figure 14(d) to the vicinity of the target 1402 by operating the multi-button 1322 or MC65. Because the user specifies a rough position by line of sight, the distance from the position shown in Figure 14(d) to the position shown in Figure 14(e) is short, and it does not require much operation or time to move the touch pointer 1412.

Figure 14 (f) shows an example of the display when the multi-button 1322 is pressed or touched up in the state of Figure 14 (e). The target frame 1413 is displayed and set based on the display position of the touch pointer 1412 immediately before the multi-button 1322 is pressed, and it can be seen that the target 1402 has been selected as the target for attack.

For example, it is rational to give higher priority to targets closest to the user and select them as targets for attack, since this increases the chances of a successful attack. However, for example, at the time of display in FIG. 14(a), the target 1402 that is furthest away is less likely to be attacked successfully than the target 1401 that is closer, but may receive higher points if the attack is successful. In such a situation, the character that the user wishes to select as a target for attack varies depending on the user's strategy. Therefore, after roughly specifying the range according to the user's line of sight, the user can accurately select the target that the user desires by making fine adjustments through movement/touch operations. Because the range can be roughly specified according to the user's line of sight, the amount of movement/touch operation required for fine adjustments is small, making it possible to select a target quickly. In this way, the user can quickly and accurately specify the position (target) that he or she desires.

In this second modification, it is assumed that one target is designated as the target of attack, but multiple targets may be designated at once. For example, when an operation with a confirmation function is performed at the time when the gaze pointer 1410 in FIG. 14(b) is displayed, all targets within the range of the gaze pointer 1410 may be selected, rather than selecting targets individually from within the range of the gaze pointer 1410. In other words, multiple targets 1401 to 1403 may be selected, rather than just target 1401. Here, the operation with the confirmation function is not the gaze confirmation operation, but another operation such as a decision button. Note that the gaze pointer 1410 can also be considered as an item that indicates a range that can include multiple targets among the targets displayed on the display unit.

The display form of the target frame 1411, the touch pointer 1412, and the target frame 1413 is not limited to the display shown in FIG. 14(a) to FIG. 14(f). Any display may be used as long as it allows visual confirmation that the target 1401 or the target 1402 is selected and that the pointer can be currently operated by touch operation. For example, the silhouette of the target 1401 or the target 1402 may be highlighted, the frame display may blink, or the display color of the frame display may be different from that of the gaze pointer 1410. The touch pointer 1412 is not limited to a circular frame display, and may be displayed with the inside of the circle filled in, or may be displayed as a square frame like the gaze pointer 1410 and the target frame 1411 instead of a circle, and may be displayed in a different display color from the gaze pointer 1410 and the target frame 1411. The touch pointer 1412 can also be considered as an item indicating a range that can include one of the targets displayed on the display unit.

In the above-mentioned embodiment, the present invention is applied to a digital camera, but the present invention is not limited to this example and can be applied to any electronic device that can specify a position by eye-gaze input and operation of an operating member. That is, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, head-mounted displays, and the like. The present invention can also be applied to digital photo frames, music players, game consoles, e-book readers, tablet terminals, smartphones, projection devices, home appliances and in-vehicle devices equipped with displays, and the like. For example, in a soft keyboard on a PC, characters are input one by one by key operation, but when inputting with eye gaze, a desired word can be selected from multiple candidate words.

The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage medium on which the program is stored constitute the present invention.

The various controls described above as being performed by the CPU may be performed by a single piece of hardware, or multiple pieces of hardware may share the processing to control the entire device.

Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

Other Embodiments
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

100: Digital camera 50: System control unit 160: Gaze detection unit

Claims (18)

A gaze detection means for detecting a gaze position of a user looking at the display means;
A control means for controlling to designate a selection position on the display means in response to a line of sight position detected by the line of sight detection means or a first operation on the operation means,
displaying a first item indicating a first size range at the selection position based on the gaze position detected by the gaze detection means in a first case in which the selection position is designated based on the gaze position detected by the gaze detection means and the first operation is not performed on the operation means;
displaying a second item indicating a range of a second size smaller than the first size at the selection position in a second case in which the selection position is designated based on the gaze position detected by the gaze detection means and the first operation is performed on the operation means;
In the first case, in response to a second operation different from the first operation being performed, selecting an object at a position of the first item based on a predetermined condition;
In the second case, in response to the second operation being performed, an object is selected at the position of the second item without being based on the predetermined condition.
A control means for controlling the
1. An electronic device comprising: The control means further executes an AF process on the selected object.
2. The electronic device according to claim 1 . the control means is capable of switching between a first AF mode in which a focus position is fixed after the AF process is performed and a second AF mode in which AF is continued to be performed,
The electronic device according to claim 2, characterized in that, in the second AF mode and in the first case, the electronic device is controlled so as to display a third item indicating a range of a third size larger than the first size at the selection position based on the gaze position detected by the gaze detection means. The control means, in the second case, hides the first item.
4. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. the second operation is an operation on an operating means other than the shutter button, the operating means being arranged at a position where a shutter button can be operated with fingers of a hand holding a grip portion of the electronic device and the operating means is operable with the fingers of a hand holding the grip portion;
5. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. the second operation is an operation for issuing an image capture instruction;
5. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. A gaze detection means for detecting a gaze position of a user looking at the display means;
A control means for controlling to designate a selection position on the display means in response to a line of sight position detected by the line of sight detection means or a first operation on the operation means,
In a first case in which the selection position is designated based on a gaze position detected by the gaze detection means and the first operation is not performed on the operation means,
displaying a first item indicating a first size range at the selection position based on the gaze position detected by the gaze detection means ;
In a second case in which the selection position is designated based on a gaze position detected by the gaze detection means and the first operation is performed on the operation means,
displaying a second item at the selection location indicating a range of a second size smaller than the first size;
hiding the first item;
A control means for controlling the
1. An electronic device comprising: the control means, in response to a movement instruction operation being performed on the operation means while the second item is being displayed, controls to move a display position of the second item in response to the movement instruction operation, not based on a gaze position after the second item is displayed;
8. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. the operation means is a touch operation member, and the first operation is a touch operation on the touch operation member;
9. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. the control means, in the second case, controls to display the second item at the selection position based on the touch operation.
10. The electronic device according to claim 9. the control means, in the second case, controls to display the second item at the selection position based on a gaze position detected by the gaze detection means at a time when the first operation is performed.
10. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. The operation means is a detection means for detecting a movement operation involving a movement of an operation object, and the first operation is the movement operation on the detection means.
12. The electronic device according to claim 1 . the first item is an item indicating a range that can include a plurality of objects among the objects displayed on the display means;
13. The electronic device according to claim 1 . the second item is an item indicating a range that can include one of the objects displayed on the display means;
14. The electronic device according to claim 1, wherein the first and second electrodes are arranged in a first direction. a gaze detection step of detecting a gaze position of a user looking at the display means;
a control step of controlling to designate a selection position on the display means in response to a gaze position detected in the gaze detection step or a first operation on the operation means,
displaying a first item indicating a first size range at the selection position based on the gaze position detected in the gaze detection step , in a first case in which the selection position is designated based on the gaze position detected in the gaze detection step and the first operation on the operation means is not performed;
displaying a second item indicating a range of a second size smaller than the first size at the selection position in a second case in which the selection position is designated based on the gaze position detected in the gaze detection step and the first operation is performed on the operation means;
In the first case, in response to a second operation different from the first operation being performed, selecting an object at a position of the first item based on a predetermined condition;
In the second case, in response to the second operation being performed, an object is selected at the position of the second item without being based on the predetermined condition.
A control step of controlling so that
13. A method for controlling an electronic device comprising: a gaze detection step of detecting a gaze position of a user looking at the display means;
a control step of controlling to designate a selection position on the display means in response to a gaze position detected in the gaze detection step or a first operation on the operation means,
In a first case in which the selection position is designated based on the gaze position detected in the gaze detection step and the first operation on the operation means is not performed,
displaying a first item indicating a first size range at the selection position based on the gaze position detected in the gaze detection step ;
In a second case in which the selection position is designated based on the gaze position detected in the gaze detection step and the first operation is performed on the operation means,
displaying a second item indicating a range of a second size smaller than the first size at the selection position;
hiding the first item;
A control step of controlling so that
13. A method for controlling an electronic device comprising: A program for causing a computer to function as each of the means of the electronic device described in any one of claims 1 to 11. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 11.

Images