KR102339898B1 - Method for unlocking mobile device using authentication based on ear recognition and mobile device performing the same - Google Patents

Abstract

Embodiments may include: acquiring an image of a subject in which at least a part of the subject's body appears in a locked state; if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and confirming whether the set of ear features of the extracted subject satisfies a preset condition. It relates to a method of unlocking a mobile device using ear recognition-based authentication and a mobile device for performing the same.

Description

METHOD FOR UNLOCKING MOBILE DEVICE USING AUTHENTICATION BASED ON EAR RECOGNITION AND MOBILE DEVICE PERFORMING THE SAME

The present invention relates to a mobile unlocking technology, and more particularly, a method of unlocking a mobile device by recognizing a target's ear to confirm that the target is a registered user of the mobile device, and changing the locked state to the unlocked state according to the result of the check and to a mobile device that performs the same.

Since the advent of Apple's iPhone, the demand for mobile devices (including, for example, smart phones, etc.) that has exploded continues to continue at home and abroad.

These mobile devices use touch as a main input method. A touch input not intended by the user causes unnecessary power loss. To lower power dissipation, the mobile device remains locked, configured not to respond to most touch input until the user begins use. In the locked state, the mobile device responds only to a limited input allowed in advance, and the lock is released when a predetermined unlocking condition is satisfied.

Recently, the quantity and importance of the owner's personal information held in the mobile device are also gradually increasing, so the unlocking and security authentication of the mobile device are being combined. Mobile devices are closely located in the lives of modern people, and the time spent interacting with one's mobile devices occupies a significant portion of the total daily activity time, and the frequency of daily use is also not small. Therefore, the input required for authentication for security of the mobile device should not impair user convenience.

In the case of a method of unlocking a mobile device, in the early stage of introduction of a smart phone, most of the unlocking was performed through a user's pattern input or password input (Patent Document 1). However, this direct input method has problems such as exposure to other people. To solve this problem, there is a way to set a more complicated pattern or password, but there is a risk that the user forgets the preset pattern or password, and the user's input process for unlocking the lock is long, which impairs user convenience. It has limitations.

In order to solve these security issues and inconveniences, biometric authentication-based unlocking methods using user's biometric information were used. Most of the unlocking is performed through identity authentication using fingerprint, face, and iris information, and this biometric information is currently being used as a non-face-to-face authentication method for financial business.

Due to the recent trend of minimization of buttons and enlargement of screens, face recognition or iris recognition is preferred over fingerprint recognition. Face recognition or iris recognition has the advantage that authentication is attempted through the built-in front camera, so it does not require a separate sensor and can be used immediately without the need for direct user contact (Patent Documents 2 and 3). However, there is a limitation in that the success rate of authentication is low when wearing accessories such as sunglasses or a mask.

Registered Patent Publication No. 10-1581618 US Patent Publication No. US 20170076077 US Patent Publication No. US 20150269419

Embodiments of the present invention provide a method for unlocking a mobile device for recognizing a target's ear, confirming that the target is a registered user of the mobile device, and changing the locked state to the unlocked state according to the result of the check, and a mobile device for performing the same. can

A mobile device unlocking method using ear recognition-based authentication according to an aspect of the present invention is performed by a mobile device including a recording unit, a display unit, and a processor. The unlocking method includes: detecting an unlocking attempt in a locked state; acquiring an image of a subject in which at least a part of the subject's body appears in a locked state; if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and confirming whether the set of the extracted ear features of the target satisfies a preset condition.

In one embodiment, extracting the set of ear features of the subject comprises: applying the detected ear region to a global feature analysis model configured to extract a first set of features for global features of the ear; and applying the detected ear region to a regional characteristic analysis model configured to extract a second set of features for the regional characteristic of the ear.

In an embodiment, the global feature analysis model or the regional feature analysis model is configured to extract a first set of features or a set of second features by using multiple images obtained from the same training object as a training image, Multiple images may share at least a portion of the training subject's ear.

In an embodiment, the global characteristic of the ear may include one or more of an ear contour and an ear configuration, and the regional characteristic of the ear may include a local pattern present in the ear.

In an embodiment, the step of confirming whether the set of ear features of the extracted target satisfies a preset condition includes pre-stored a set of first features and a set of second features of a first feature of the user of the mobile device calculating a similarity of ear features between the subject and the user by comparing them with the set of and the second set of features; and changing the first interface state to the second interface state when the matching degree based on the similarity is higher than a predetermined threshold.

In one embodiment, calculating the degree of similarity of ear features between the subject and the user comprises: comparing the set of first features of the subject with a pre-stored set of first features of the user of the mobile device to determine the degree of similarity between the subject and the user. calculating a first degree of similarity indicating a degree of similarity of global characteristics of the ear; calculating a second degree of similarity indicating a degree of similarity between the subject's second feature set and the pre-stored set of second features of the user of the mobile device; and calculating a similarity of an ear feature between the target and the user by combining the first degree of similarity and the second degree of similarity.

In one embodiment, the method comprises: before applying to the global characteristic analysis model or the regional characteristic analysis model, when a change in motion of the mobile device is detected, based on the change in motion of the mobile device, the target image or the ear The method may further include correcting a photographing angle of the image of the region.

In one embodiment, the lock state includes a default lock state. In addition, the method of unlocking the mobile device may further include: if it is checked whether the extracted set of ear features of the target satisfies a preset condition, displaying a screen on which one or more application interface objects are displayed.

In one embodiment, the locked state further includes an unlocked state. The method of unlocking the mobile device may further include: before displaying a screen on which one or more application interface objects are displayed, changing the basic lock state to an unlock state and notifying the state change.

In an embodiment, the lock state includes a basic lock state and an application lock state, and the method for unlocking the mobile device includes: when an event occurs in the basic lock state, displaying an interface screen corresponding to the occurrence of the event; and after it is checked whether the extracted set of ear features of the target satisfies a preset condition, executing an operation of an application corresponding to the event.

In an embodiment, the event includes an incoming call, and after it is confirmed whether the set of the extracted subject's ear features satisfies a preset condition, the connection of the incoming call may be performed.

In one embodiment, the lock state includes a basic lock state, an unlock state, and an application lock state, and the method for unlocking the mobile device includes: a screen displayed by one or more application interface objects by receiving an unlock input different from ear recognition; displaying a screen); displaying an interface screen corresponding to the occurrence of an event when an event occurs in the release state; and after it is checked whether the extracted set of ear features of the target satisfies a preset condition, the method may further include executing an operation of an application corresponding to the event. Here, the image of the target is acquired after the event occurs.

In an embodiment, the different unlocking input may include one or more of a touch, a non-touch gesture, a typing, and a voice, and a photographing of a part different from the ear.

In an embodiment, the event includes an incoming call, and after it is confirmed that the set of ear features of the extracted subject satisfies a preset condition, acceptance of the incoming call may be executed.

In an embodiment, the method further comprises: prior to detecting the ear region, when the illuminance level of the mobile device is less than or equal to a threshold, providing the subject with a guide to adjust the distance between the mobile device and the subject can

In one embodiment, the method further comprises: prior to extracting the set of ear features, verifying that the detected ear region includes at least a portion of the subject's ear; when the detected ear area includes at least a part of the subject's ear, checking whether the size of the detected ear area falls within a predetermined size range; and if the size of the detected ear region does not fall within a predetermined size range, providing a guide for adjusting the distance between the mobile device and the object to the subject in order to obtain an ear region belonging to the predetermined size range. may include

In an embodiment, the detecting of an attempt to unlock the lock in the locked state includes detecting a change in the rotation angle of the mobile device within a predetermined range for a specified unit time, and illuminance within a predetermined range for a specified unit time. determining to detect an unlocking attempt upon detecting one or more of the instances of detecting being held; and activating ear recognition for unlocking.

A computer-readable recording medium according to another aspect of the present invention may be readable by a computing device including a processor and store program instructions operable by the computing device. Here, when executed by the processor of the computer, the processor may perform the method of unlocking the mobile device according to any one of claims 1 to 17.

A mobile device according to another aspect of the present invention includes: a photographing device for acquiring an image of a target; display; processor; and a memory storing a program configured to be executed by the processor. Here, the program includes: detecting an attempt to unlock the lock in the locked state; acquiring an image of a subject in which at least a part of the subject's body appears in a locked state; if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and confirming whether the extracted set of ear features of the target satisfies a preset condition.

In one embodiment, the lock status includes a default lock status, and the program: when it is confirmed that the set of ear features of the extracted subject satisfies a preset condition, a screen indicated by one or more application interface objects It may further include instructions for performing; displaying.

In one embodiment, the lock state includes a basic lock state and an application lock state, and the program includes: when an event occurs in the basic lock state, displaying an interface screen corresponding to the occurrence of the event; and after it is confirmed whether the extracted set of ear features of the target satisfies a preset condition, executing the operation of the application corresponding to the event may further include instructions.

In one embodiment, further comprising one or more of a touch unit acquiring a touch input, another input unit acquiring an input different from a touch, and a microphone, wherein the lock state includes a default lock state, an unlock state and an application lock state wherein the program comprises: receiving an unlock input different from ear recognition and displaying a screen represented by one or more application interface objects; displaying an interface screen corresponding to the occurrence of an event when an event occurs in the release state; and after it is confirmed whether the extracted set of ear features of the target satisfies a preset condition, executing the operation of the application corresponding to the event may further include instructions.

The mobile device according to an aspect of the present invention can unlock the lock by recognizing the ear, so that the unlock operation can be performed even when the object that attempts to unlock the device is equipped with an article that covers the face, such as sunglasses or a mask. . In this process, even when wearing accessories such as sunglasses and a mask, it is possible to unlock by performing identity authentication through a mobile device without an issue such as a decrease in recognition performance.

In addition, when an event (eg, an incoming call) that brings the mobile device and an ear close to it occurs, the mobile device may unlock and execute an application corresponding to the event, thereby further enhancing security of the mobile device.

Furthermore, the ear recognition-based unlock technology can be used for non-face-to-face authentication for financial-related tasks.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

In order to more clearly explain the technical solutions of the embodiments of the present invention or the prior art, drawings necessary for the description of the embodiments are briefly introduced below. It should be understood that the following drawings are for the purpose of explaining the embodiments of the present specification and not for the purpose of limitation. In addition, some elements to which various modifications such as exaggeration and omission have been applied may be shown in the drawings below for clarity of description.
1 is a schematic block diagram of a mobile device, according to an embodiment of the present invention;
2 is a diagram for explaining an operation of extracting an ear feature according to an embodiment of the present invention.
3 is a diagram illustrating various feature vectors extractable from an ear characteristic analysis model as images, according to an embodiment of the present invention.
4A and 4B are flowcharts of a process of releasing a basic lock state according to various embodiments of the present disclosure;
5 is a diagram illustrating a target image according to an embodiment of the present invention.
6 is a diagram illustrating a result of releasing a basic lock state according to an embodiment of the present invention.
7 is a diagram illustrating a result of direction correction based on motion information according to an embodiment of the present invention.
8 is a flowchart of a process of unlocking a phone application lock state according to an embodiment of the present invention.
9 is a flowchart of a process of unlocking a phone application lock state according to another embodiment of the present invention.

The terminology used herein is for the purpose of referring to specific embodiments only, and is not intended to limit the present invention. As used herein, the singular forms also include the plural forms unless the phrases clearly indicate the opposite. The meaning of "comprising," as used herein, specifies a particular characteristic, region, integer, step, operation, element and/or component, and includes the presence or absence of another characteristic, region, integer, step, operation, element and/or component. It does not exclude additions.

Although not defined otherwise, all terms including technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Commonly used terms defined in the dictionary are additionally interpreted as having a meaning consistent with the related technical literature and the currently disclosed (disclosed) content, and unless defined, they are not interpreted in an ideal or very formal meaning.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a schematic block diagram of a mobile device, according to an embodiment of the present invention;

In embodiments, the mobile device 100 is configured to obtain an ear image of the subject to recognize the subject's ears and to perform an operation of changing one or more locked states to an unlocked state. The mobile device 100 may be implemented as, for example, a smart phone, a cellular phone, smart glasses, a smart watch, a wearable device, a digital camera, a tablet, a notebook computer, or a laptop computer, but is not limited thereto.

Referring to FIG. 1 , a mobile device 100 includes a memory 110 , a processor 130 ; interface 140; communication module 150; It includes one or more interface units for inputting/outputting information. The interface unit includes at least a recording unit 171, a display unit 174 among a recording unit 171, a touch unit 172, a guitar input unit 173, a display unit 174, a speaker 175, and a microphone 176. ) is included. In some embodiments, the mobile device 100 may further include a gyro sensor 177 and/or an illumination sensor 178 . Also, the mobile device 100 may further include an acceleration sensor (not shown).

Mobile device 100 according to embodiments may be entirely hardware, entirely software, or may have aspects that are partly hardware and partly software. For example, the mobile device 100 may collectively refer to hardware equipped with data processing capability and operating software for driving the same. As used herein, terms such as “unit”, “module,” “device,” or “system” are intended to refer to a combination of hardware and software run by the hardware. For example, the hardware may be a data processing capable computing device including a central processing unit (CPU), a graphic processing unit (GPU), or another processor. In addition, software may refer to a running process, an object, an executable file, a thread of execution, a program, and the like.

The memory 110 may include a nonvolatile memory. The nonvolatile memory may include, for example, SDD, HDD, flash memory, etc., but is not limited thereto and may include other nonvolatile semiconductor memories. In some embodiments, the memory 110 may further include a storage device remotely located outside the mobile device 100 , for example, a distributed storage accessed through a wired/wireless communication network.

The processor 130 is configured to perform data processing and/or image processing. The processor 130 may include a CPU, a GPU, or the like. In FIG. 1 , the processor 130 is illustrated as a single component, but is not limited thereto. For example, it may be implemented as a single chip or multiple chips depending on the design.

The operation of the processor 130 will be described in more detail with reference to FIGS. 2 to 9 below.

The interface 140 connects devices related to input/output of the mobile device 100 with the memory 110 and/or the processor 130 . The interface 140 may be implemented as a combination of software and/or hardware according to connected input/output-related devices. For example, the interface 140 includes an interface for communication, an interface for touch and other input units, an interface for audio, and the like, wherein the audio interface includes an audio circuit (eg, converts data into an electrical signal and transmits the electrical signal) headset jack, etc.), and the interface for touch and other input units may include one or more input controllers that transmit electrical signals corresponding to user input obtained through one or more input units to the processor 130 . have.

The communication module 150 is configured to transmit/receive electromagnetic waves. The communication module 150 converts an electrical signal into an electromagnetic wave or converts an electromagnetic wave into an electrical signal. The mobile device 100 allows a user to make a call or use an application that provides a service on the Internet through the communication module 150 . The communication module 150 may communicate with other devices through various communication methods, including wired communication, wireless communication, 3G, 4G, wired Internet, or wireless Internet, in which an object and an object can network. For example, the communication module 150 is configured to communicate via the Internet, such as the World Wide Web (WWW), a network such as an intranet and/or a cellular telephone network, a wireless network, and wireless communication. The wireless communication is, for example, GSM (Global System for Mobile Network, CDMA (ode division multiple access, W-CDMA) (Wideband Code Division Multiple Access), TDMA (Time Division Multiple Access), Bluetooth, Wi-Fi, Wi wireless communication standards using communication protocols for - Old Interoperability for Microwave Access (MAX), and/or e-mail, instant messaging, short text service (SMS), and the like.

The photographing unit 171 is a component that obtains image data of the object by photographing the object, and may be, for example, various photographing devices including an image sensor, a camera, a video camera, and the like.

The touch unit 172 and other input units 173 are components configured to receive a command related to a user's input. A part of the user's body or another object is used as a pointing object and is input to the touch unit 172 . The touch unit 172 may include, but is not limited to, a pressure sensitive sensor or an electrostatic sensor. The guitar input unit 173 includes, for example, a button, a dial, a switch, a stick, and the like.

The display unit 174 is a component that displays information stored and/or processed in the mobile device 100 , and may include, for example, an LCD, an OLED, a flexible screen, and the like, but is not limited thereto.

Although the touch unit 172 and the display unit 174 are separated in FIG. 1 , in many embodiments, the touch unit 172 and the display unit 174 are implemented as one component to perform input reception and information output. can be For example, the touch unit 172 and the display unit 174 may be implemented as a touch screen in which the touch panel forms a layer structure with the screen. The touch input is input by a pointing object (eg, including a user's body, a tool, etc.).

The speaker 175 converts an electrical signal into a sound wave having a frequency in the audible region. Data processed by the processor 130 or data stored in advance in the memory 110 are converted into electrical signals through the interface 140 and input to the speaker 175, and finally sound waves are output. The microphone 176 converts sound waves into electrical signals. A speaker 175 and a microphone 176 may be used for a phone call when the mobile device 100 is configured to perform a phone application.

The gyro sensor 177 is a sensor that detects the rotation of an object, and converts a physical change (eg, Coriolis force) that occurs when the mobile device 100 has a rotation motion into an electrical signal. is composed The electrical signal is transmitted to the processor 130 .

The illuminance sensor 178 is a sensor for detecting ambient illuminance, and is configured to convert the intensity of illuminance around the mobile device 100 into an electrical signal.

Hereinafter, for clarity of explanation, the present invention will be described in more detail based on the mobile device 100 implemented as a smart phone including a touch screen. However, it will be apparent to those skilled in the art that the above embodiments are merely exemplary, and the present invention is not limited thereto.

Identification based on ear recognition

The mobile device 100 is configured to perform an ear recognition operation that identifies whose ear the ear is in the image.

2 is a diagram for explaining an operation of extracting an ear feature according to an embodiment of the present invention.

The mobile device 100 includes information about one or more candidates. In one embodiment, the candidate may be a person registered as a user of the mobile device 100 .

The information about the candidate includes information to be used for an authentication operation to confirm the identity of the subject. The information on the candidate includes information on characteristics related to the candidate's ears (hereinafter, “ear characteristics”). The ear characteristic information includes a set of ear characteristics comprising ear characteristics expressing individual and unique characteristics of the ear possessed by the candidate. The ear characteristic may be classified as the first characteristic or the second characteristic by the ear characteristic descriptor. The set of ear features is described in more detail below.

In addition, the information on the candidate may further include, for example, candidate identification (ID) information, candidate image information, and the like, but is not limited thereto.

The mobile device 100 acquires an image of the object (eg, by the photographing unit 171 ). The image of the subject is an image in which at least a part of the subject's body appears.

In some embodiments, the image of the subject may be an image of an ear, including at least a portion of the subject's ear. The ear image is acquired by photographing the front part of the face, or by photographing the side part of the face as the center. That is, the image of the target is not limited to the image obtained by photographing the side of the face shown in FIG. 2 .

The mobile device 100 detects an ear region in the image of the object. The mobile device 100 may detect the ear region from the image of the target through the region detection algorithm. The region detection algorithm is, for example, Haar, Convolution Neural Network (CNN), Scale Invariant Feature Transform (SIFT), Histogram of Gradients (HOG), Neural Network (NN), Support Vector Machine (SVM), Gabor method, etc. Including, but not limited to, the algorithm to perform.

Also, the region detection algorithm may be an algorithm configured to suitably detect an ear region by applying an ear characteristic (eg, a morphological characteristic) based on an ear image.

As shown in FIG. 2 , when the target image including the side face of the target is acquired, the mobile device 100 may detect the ear region as shown in FIG. 2 through the above-described region detection algorithm.

Also, the mobile device 100 may generate an image of the ear region. That is, the image of the ear region is extracted as a sub-image from the image of the target.

Hereinafter, it is assumed that the mobile device 100 has detected the ear region, and the feature extraction process is continuously described.

The mobile device 100 extracts a feature representing a characteristic of the ear part included in the ear region. The ear is configured to have a different shape, skin texture, and skin surface composition (eg, dots, etc.) for each person. That is, the intrinsic characteristics of the ear correspond to the point of view of analyzing the ear. Accordingly, when the two ears are compared and analyzed, it may be determined whether the two ears are the same or different based on the unique characteristics of the ears.

Ear characteristics include, for example, the full or partial contour of the ear (eg, ear lobe size, ear canal ratio, etc.), skin texture, skeletal structure (eg, shape of the ear canal), dots, and the like, characteristics indicative of the ear characteristics. The vector includes numerical value(s) of each ear characteristic expressed through the image.

The mobile device 100 extracts features related to one or more ear characteristics from the image of the ear region. To this end, the mobile device 100 includes a pre-modeled ear characteristic analysis model. There may be one or more features extracted by the ear characteristic analysis model, and may constitute a set. A set of features may be referred to as a feature vector.

In an embodiment, the mobile device 100 may extract first features related to the global characteristic of the ear and/or second characteristics related to the regional characteristic of the ear from the image of the ear region. To this end, the mobile device 100 may include a global characteristic analysis model that outputs a first feature vector representing the global characteristic of the ear and/or a regional characteristic analysis model that outputs a second characteristic vector that indicates the regional characteristic of the ear. have.

Global properties relate to, for example, the overall outline or composition within an image. The global characterization model uses the entire ear region image to extract ear characteristics. The global characteristic analysis model uses global characteristic descriptor methods including, for example, Principal Components Analysis (PCA), Local Discriminant Analysis (LDA), Independent Components Analysis (ICA), and the like, and global characteristics expressed in the ear region image. It is possible to extract a feature vector representing , but is not limited thereto.

Regional characteristics are characteristics expressed in a local pattern, and are related to, for example, ear migration, local branch shape such as diurnal notch, points present in the ear, skin texture, wrinkles, and the like. Regional feature analysis models include, for example, Convolution Neural Network (CNN), Local Binary Pattern (LBP), Scale Invariant Feature Transform (SIFT), Learning-based Encoding (LE), Histogram of Oriented Gradient (HOG), etc. A feature vector representing a regional feature expressed in the ear region image may be extracted using the feature descriptor method, but is not limited thereto.

In some embodiments, the global feature analysis model or the regional feature analysis model is configured to extract a first set of features or a second set of features by using multiple images obtained from the same training object as a training image. The multiple images share at least a portion of the training subject's ear.

In certain embodiments, the training image may include, for example, an image centered on the anterior portion of the face and/or an image centered on the side of the face where the ear characteristics analysis model and threshold are centered.

3 is a diagram illustrating various feature vectors extractable from an ear characteristic analysis model as images, according to an embodiment of the present invention.

As described above, the unique characteristic of the ear expressing the object can be classified as a regional or global characteristic, and furthermore, even for the same characteristic, the characteristic descriptors used (for example, to extract the regional characteristic) vary. Therefore, from one ear region image, as shown in FIG. 3 , various feature vectors expressed in various images are extracted. That is, the identification apparatus 100 may analyze the ear characteristics in various aspects by setting the ear characteristics analysis model.

The mobile device 100 checks whether the set of the extracted ear features of the target satisfies a preset condition. In some embodiments, when a preset condition is satisfied, the mobile device 100 may perform one or more operations based on the interface state. Also, verify the identity of the subject.

The preset condition includes whether the ear similarity is higher than a threshold (Tscore).

The mobile device 100 calculates an ear similarity between the subject and the candidate by comparing the previously stored ear features of the candidate with the extracted ear features of the subject.

The feature vector of the target and the feature vector of the candidate are compared for each feature. For example, as shown in FIG. 2 , the first feature vector representing the global characteristic of the target is compared with the first feature vector representing the global characteristic of the candidate, and the second feature vector representing the local characteristic of the target is It is compared with a second feature vector representing the regional characteristics of the candidate.

Various similarity comparison algorithms may be used to calculate a similarity between the ear characteristic of the target and the ear characteristic of the candidate by comparing the feature vector of the target and the feature vector of the candidate stored in advance for each characteristic. The similarity comparison algorithm may include, for example, Euclidean Distance, Cosine Distance, Mahalanobis Distance, and the like, but is not limited thereto.

Also, the mobile device 100 may calculate the final similarity between the ear characteristic of the target and the candidate by concatenating the calculated similarities for each characteristic.

The mobile device 100 determines, for example, a plurality of characteristics-specific similarities (eg, regional characteristics-related similarities and global characteristics-related similarities) through a combination method including SUM, MIN, MAX, PRODUCT, Weighted SUM, SVM, and the like. may be combined, but is not limited thereto.

When the similarity higher than the threshold (Tscore) is calculated and the condition is satisfied, it may be used to determine whether a candidate matching the target exists or to determine whether a previously stored candidate and the target match.

When the condition is satisfied, the mobile device 100 determines a candidate matching the identity of the target. When the candidate is a registered user of the mobile device 100 , a similarity higher than the threshold Tscore is calculated when the target is an actual registered user. Then, authentication is completed and the target is confirmed as a registered user. When the target is different from the registered user, a similarity equal to or lower than the threshold Tscore is calculated, so that authentication fails and it is confirmed that the target is not the registered user.

The threshold value Tscore may be a value determined depending on the extracted ear characteristic, a combination method of similarity, and/or an image characteristic used for extracting the ear characteristic.

For example, when only a regional characteristic is extracted, when only a global characteristic is extracted, or when both a global characteristic and a regional characteristic are extracted, the threshold values Tscore may be different from each other.

The image characteristics indicate characteristics of an image (eg, an ear region image) input to the model. The image characteristics include, but are not limited to, for example, the proportion of the ear in the image, the portion of the ear shown in the image, the angle of the ear, and the like.

However, the threshold Tscore is not limited to the above-described embodiment. For example, the threshold Tscore may be a specific value set by a user input.

In some embodiments, prior to extracting features indicative of an ear characteristic, the mobile device 100 is configured to pre-process the ear image. This will be described in more detail with reference to FIG. 4B below.

User Interface Behavior - Basic Lock and Unlock

The mobile device 100 may have a plurality of interface states. The interface state is an interface mode in which an operation to respond to a user input received by the mobile device 100 is determined in advance according to a corresponding state. The mobile device 100 may display a screen related to an interface state. A change in the interface state may be implemented as a change in a displayed screen or a part of the screen.

In one embodiment, the interface state includes a basic lock state and an unlock state.

Also, the plurality of interface states may include states of a plurality of applications installed in the mobile device 100 .

In the basic locked state, the mobile device 100 is powered on and operable, but ignores most user input. That is, in the basic locked state, the mobile device 100 is prohibited from performing a predetermined set of actions of the mobile device 100 such that the mobile device 100 takes no action in response to the user input and/or operates in response to the user input. The predetermined set of actions includes navigation between the user interface and activation or deactivation of the predetermined set of functions. It also does not respond to user input corresponding to attempts to navigate between user interfaces. The locked state prevents unintentional or unauthorized use of the mobile device 100 or activation or deactivation of a function on the mobile device 100 .

In the default locked state, the mobile device 100 responds to a limited set of user inputs including an input corresponding to an attempt to unlock or an input corresponding to a power off.

In the case of the default unlocking state (hereinafter, “unlocked state”), the device 100 is in a normal operating state and the mobile device 100 detects and responds to a user input corresponding to an interaction with a user interface (UI). In the unlocked state, the mobile device 100 detects and responds to user input for navigation between user interfaces (UIs), input of data, and activation or deactivation of functions.

4A and 4B are flowcharts of a process of releasing a basic lock state according to various embodiments of the present disclosure;

Referring to FIG. 4A , the mobile device 100 may detect an unlock attempt to change the basic lock state to the unlock state from the basic lock state.

The mobile device 100 determines that the release attempt has been made when a specific physical change according to the interaction between the user and the mobile device is detected. The mobile device 100 activates a lock device unlocking operation when an unlocking attempt is detected. That is, the ear recognition operation is activated depending on whether the release attempt is made.

In one embodiment, the unlock attempt comprises any user input. For example, it may be any user input received by touching a certain point in the touch input area within the mobile device 100 or pressing a button.

In some embodiments, the unlock attempt may be detected by sensors included in the mobile device 100 .

For example, when detecting a change in rotation and/or angle within a preset range for a unit time designated by the gyro sensor 177 built-in in the mobile device 100, a preset designated by the acceleration sensor When detecting a change in speed within a predetermined range for a unit time, the mobile device 100 may determine that there is an attempt to release.

Alternatively, it may be determined whether a release attempt is made based on the detection results of the plurality of sensors. For example, when position information is respectively acquired by the gyro sensor 177 and the acceleration sensor, and a change in the terminal position within a predetermined range is detected for a specified unit time, and/or a unit time specified by the illuminance sensor 178 If the illuminance within the preset range is maintained during the period, the mobile device 100 may determine that there is a release attempt.

That is, when one or more of the above-described four cases is detected, for example, the mobile device 100 may determine that there is an unlock attempt, and use it as an operation attempt condition for unlocking the mobile device.

When an unlock attempt is detected, the mobile device 100 may perform an operation of unlocking the mobile device based on ear recognition in the basic lock state.

The mobile device 100 acquires a target image (S410). In some embodiments, the mobile device 100 may induce the subject to photograph a specific body part by relying on an ear feature extraction process. This is because the ear characterization model and threshold may have been modeled based on images of specific body parts.

For example, if the ear characteristic analysis model and the threshold are modeled based on the ear region detected in the image centered on the front part of the face and/or the image centered on the side of the face, as shown in FIG. 5 , the mobile device 100 The specific gravity of the front of the face is higher than the specific gravity of the side of the face, leading to shooting the central part of the front of the face, or, as shown in FIG. 3, the specific gravity of the side of the face is higher than the specific gravity of the front of the face. You may be tempted to take a picture.

Next, the mobile device 100 detects an ear region in the target image of step S410 ( S430 ) and extracts a set of ear features (eg, an ear feature vector) ( S450 ).

In some embodiments, in step S450 , the mobile device 100 provides a subset of features associated with a global feature (eg, a first feature vector) and/or a subset of features associated with a regional feature (eg, a second feature vector). 2 feature vectors) can be extracted.

The mobile device 100 calculates an ear similarity between the subject and the candidate based on the set of ear features of the subject and the set of pre-stored ear features of the candidate ( S470 ).

In some of the above embodiments, the set of the candidate's ear features also includes a subset of features related to global features and/or a subset of features related to regional features. The characteristics of the subject and the candidate depend on the associated ear characteristics to calculate the similarity for each characteristic. The first feature vector of the object and the first feature vector of the candidate are applied to the similarity comparison algorithm to calculate the first similarity between the object and the candidate, and/or the second feature vector of the object and the second feature vector of the candidate are compared with the similarity comparison algorithm is applied to calculate the second degree of similarity between the target and the candidate. When the first and second similarities are calculated, the ear similarity between the target and the candidate is calculated by concatenating the calculated first and second similarities ( S470 ).

The mobile device 100 determines whether the ear similarity in step S470 is higher than a preset threshold Tscore (S480). When it has a high value, the basic lock state is changed to an unlock state (S490).

In some embodiments, when the similarity between the target and the candidate is less than the threshold Tscore, at least a portion of the previous steps S410 to S470 may be repeated within a preset number of times.

In some embodiments having a plurality of candidates, when the calculated similarity between the specific candidate and the target is less than the threshold Tscore and the feature extraction operation and the identification operation are repeated more than a preset number of times, the mobile device 100 is stored in the database. Search for other candidates matching the target in .

In other alternative embodiments, the mobile device 100 may be further configured to perform more user-friendly and precise ear recognition. Since these other alternative embodiments are similar to those of FIG. 4A , differences will be mainly described.

Referring to FIG. 4B , after acquiring the target image ( S410 ), the mobile device 100 checks the ambient illuminance of the mobile device 100 (eg, by the illuminance sensor 178 ) ( S423 ). When the ambient illuminance of the mobile device 100 is less than or equal to the threshold Ti, the mobile device 100 provides a guide for adjusting the distance between the mobile device and the target. The guidance may be, for example, voice guidance such as “Your ear is too close”, but is not limited thereto, and may be performed by non-voice such as vibration or a combination thereof.

Additionally, when the ambient illuminance of the mobile device 100 is higher than the threshold Ti, the mobile device 100 checks whether a change in motion of the mobile device 100 has occurred ( S425 ). When a change in motion of the mobile device 100 is detected (eg, by the gyro sensor 178 ), a shooting angle of the image of the target is corrected based on the change in motion of the mobile device.

As shown in FIG. 7 , when a motion occurs in the mobile device 100 according to the movement of the user's hand, the image obtained in step S410 includes the image before correction. When a change in motion is detected, the mobile device 100 calculates a correction value based on the change in motion, and applies the correction value to correct the image of the object into an image having no change in motion. In some embodiments, the compensation value may be a value having the same magnitude and the same rotation angle included in the motion change, but having the opposite direction.

In step S430 , the mobile device 100 extracts a sub-region estimated to be an ear region from the target image. The mobile device 100 may be further configured to check whether the extracted sub-region includes the actual ear part of the target ( S431 ). If it is determined that the image of the detection area does not actually include the ear part, a guide to induce re-shooting, such as “Please move the equipment,” may be provided.

When it is determined that the region extracted in step S430 is an ear region including the real ear of the target (S431, it is determined whether the detected ear region belongs to a predetermined size range (S433 and S435). The predetermined size range is It consists of a maximum value (Ts_max) and a minimum value (Ts_min).

In some embodiments, when the detected ear region has a size greater than or equal to the maximum value (Ts_max), or has a size less than or equal to the minimum value (Ts_min), the mobile device 100 detects a new ear region belonging to a predetermined size range. To provide a guide for inducing re-photography (S434 and S436). For example, it is possible to provide a guide to the subject to adjust the distance between the mobile device and the subject, such as “Your ears are too far” or “Your ears are too close”.

In some other alternative embodiments, instead of providing a guide for inducing re-imaging to detect a new ear region belonging to a predetermined size range, normalizing the acquired ear region size to fall within a predetermined size range is performed. may be

The mobile device 100 performs a zero-padding method in which a blank space is filled with black pixels in order to satisfy a required image size, or expands the size of the ear region image and then the mobile device in the expanded image Normalization can be performed using a cropping method that cuts out and utilizes some sections corresponding to the image size required by (100), but is not limited thereto, and various size normalization methods that maintain the ratio of the ears can be used. configured to perform a normalization operation.

Through this size control, it is possible to prevent an error in ear recognition by preventing morphological deformation and distortion of the ear region.

Through these steps ( S410 to S490 ), the basic lock state is changed to the unlock state, and the lock of the mobile device 100 is released.

In some embodiments, when the unlock state is changed (ie, when the basic lock is released), as shown in FIG. 6 , a result of the state change may be displayed. When the mobile device 100 changes from the basic lock state to the unlock state, it displays a screen indicated by one or more application interface objects (S490).

Also, the mobile device 100 determines that the ear similarity threshold Tscore and the candidate match the target (S490). If the candidate is a registered user, the identity of the subject is verified as the registered user of the mobile device 100 .

User Interface Behavior - Lock and Allow Applications

The interface state described above may further include an application lock state and an allow state. The application associated with the application lock state may include one or more active applications executable in the default lock state and/or unlock state. The one or more applications respond to an event. Events can occur in the default lock state or unlock state. In some embodiments, the mobile device 100 may display a screen associated with an active application corresponding to the occurrence of an event.

In some embodiments, events include incoming calls, incoming messages, voicemail notifications, and the like. This event relates to providing a voice to the user of the mobile device 100 . In some of the above embodiments, the incoming message is provided to the user through text-to-speech conversion.

In the application lock state, user input to the application is at least partially ignored. That is, in the application lock state, the mobile device 100 is at least partially prohibited from performing an operation set of an application determined in advance as operating in response to a user input for the corresponding application. For example, when an event occurs in the application lock state, the mobile device 100 responds to a user input related to initiating execution of an operation of the application corresponding to the event, but the user input related to the operation after execution It is designed not to react. The one or more active applications may respond to a predetermined user input in an unlocked state or a default locked state.

In the case of the interface permission state (hereinafter, the “allowed state”), the device 100 is a state in which the execution of the application is approved, and the mobile device 100 is a user corresponding to the interaction with the application interface (AU) for the application. Detect and respond to input. That is, operations after the execution of the application are permitted. In the permitted state, the mobile device 100 detects and responds to a user input for performing a function included in the corresponding application. In the application lock state and the application associated with the permission state, an operation that is possible in the permission state may not be performed in the application lock state.

Hereinafter, for clarity of explanation, the present invention will be exemplarily described in more detail as an event as an incoming phone call and an active application corresponding to the event as a phone application.

8 is a flowchart of a process of unlocking a phone application lock state according to an embodiment of the present invention.

Since the embodiment of FIG. 8 is somewhat similar to the embodiment of FIG. 4A , differences will be mainly described.

Referring to FIG. 8 , the mobile device 100 detects a release attempt. The unlock attempt is an attempt to unlock the application, which is detected when an incoming call occurs in the basic lock state.

In some embodiments, the determination condition of the release attempt of FIG. 8 may be set to the same case as that of the release attempt of FIG. 4A .

In some other embodiments, the determination condition of the unlock attempt of FIG. 8 may be different from the unlock attempt of FIG. 4A . Since the determination condition of the release attempt of FIG. 8 is related to the occurrence of an incoming call, it may be set based on a physical change according to the interaction between the mobile device 100 and the user for receiving the incoming call. For example, the determination condition of the release attempt of FIG. 8A may be set when a larger physical change (eg, greater rotational change, acceleration change, etc.) occurs compared to the determination condition of the release attempt of FIG. 4A .

In the application lock state, the mobile device 100 responds only to predetermined actions for the phone application, including accepting or rejecting an incoming call, adjusting the sound, and the like.

In some embodiments, when an incoming call occurs, the mobile device 100 may display an interface screen indicating a notification of the incoming call.

In some embodiments, the unlock attempt of FIG. 8 may include the same user input as the unlock attempt of FIG. 4A . In some other embodiments, the unlock attempt of FIG. 8 may include whether a user input starts an operation of a phone application corresponding to an event. For example, an application unlock attempt corresponding to an incoming call may include user input (eg, moving a phone icon, etc.) that causes the phone application to initiate an action for connecting the incoming call, such as accepting an incoming call. can do.

When the mobile device 100 detects a release attempt, it acquires a target image (S810), detects an ear region (S830), extracts a set of ear features of the target (S850), and calculates the ear similarity between the target and the candidate Calculate (S880). When the ear similarity is higher than the threshold Tscore, the operation of the application corresponding to the event is executed.

In some embodiments, the operation of the application corresponding to the event performed after it is determined that the ear similarity is higher than the threshold Tscore may be an incoming call connection operation. In this case, the ear recognition operation including calculating the ear similarity corresponds to the operation of verifying the input for accepting the incoming call. Even when the mobile device 100 receives a separate user input for accepting the incoming call, if it is determined that the ear similarity is equal to or less than the threshold Tscore, the call is not connected.

In some other embodiments, the operation of the application corresponding to the event performed after it is determined that the ear similarity is higher than the threshold Tscore may be an operation of approving reception of an incoming call and an operation of connecting a call. If it is determined that the ear similarity is less than or equal to the threshold Tscore, an operation of rejecting the incoming call is performed and the call is not connected. In certain embodiments, the reception of the call may be rejected after the ear recognition operation is repeatedly performed within a range of a predetermined number of times.

That is, in this case, the user may perform a user input of acceptance and connection by placing the mobile device 100 around the ear and taking a gesture for receiving a call. No separate user input is required to accept incoming calls.

In some other embodiments, the target image is performed in response to the user's receiving gesture being detected (S805). In certain embodiments, the target image may be taken in the middle of the receive gesture. Here, the middle is any one of the paths between the beginning and the end of the gesture.

For example, for the illumination of the mobile device 100 , before ear recognition is completed, a gesture having a relatively wide interval between the ear and the mobile device 100 is taken, and after connection of a call, the mobile device 100 listens to the ear. In the case of having a close gesture, the target image is obtained in a situation where the gesture is made with a relatively wide gap between the ear and the mobile device 100 .

In addition, the mobile device 100 confirms that the user (ie, the ear recognition target) of the mobile device 100 receiving the incoming call is a registered user, and changes the application lock state to the allow state (S890).

In step S890, both the basic lock and the application lock are released. That is, the application lock state is changed to the allow state, and the default lock state is also changed to the released state. Accordingly, after the execution of the phone application corresponding to the incoming call is terminated, the mobile device 100 may display a screen indicated by one or more application interface objects.

9 is a flowchart of a process of unlocking a phone application lock state according to another embodiment of the present invention.

Since the embodiment of FIG. 9 is similar to the embodiment of FIG. 8 , differences will be mainly described.

Referring to FIG. 9 , the mobile device 100 may release the basic lock based on a user input other than based on ear recognition ( S901 ). The user input is an unlock input different from ear recognition, and may be an input for which image analysis is not required or an input for image analysis different from ear recognition. Inputs for which image analysis is not required include, but are not limited to, one or more of, for example, touch, non-touch gestures, typing, and voice. Here, the touch includes a pattern input. An input for image analysis different from ear recognition may be, for example, photographing an iris for iris recognition or photographing a fingerprint for fingerprint recognition, but is not limited thereto. The input for the different image analysis may be obtained using a signal having a wavelength different from that of the visible ray region.

Even when the basic lock is released by a path different from the ear recognition, the mobile device 100 may display a screen indicated by one or more application interface objects (S903).

Events can occur in the unlocked state. For example, an incoming call may occur as described in FIG. 8 . In step S903, the mobile device 100 is under the application lock state. The mobile device 100 performs an operation of the application corresponding to the event based on the ear recognition operations S910 to S980 .

When the event is an incoming phone call, if the ear similarity is higher than a threshold Tscore, a phone connection operation is performed. In some embodiments, if the ear similarity is higher than the threshold Tscore, the reception of the call and the connection operation may be performed.

Additionally, the event further includes a financial-related event. The application associated with the application lock state may further include an authentication application executable in the default lock state and/or unlock state. The authentication application includes a sub-application that assists the operation of the other application by interworking with the other application. For example, the mobile device 100 may include a non-face-to-face financial application and an authentication application that assists the non-face-to-face financial application.

The financial-related event may be, for example, various non-face-to-face financial operations such as account transfer, online transaction, and the like. Applications corresponding to such events include financial applications and/or authentication applications that assist non-face-to-face finance.

In some embodiments, if the ear similarity is higher than the threshold Tscore, the identification target may be identified as a registered user, or a non-face-to-face financial operation may be performed.

It will be apparent to those skilled in the art that the event-related operation described above with reference to FIGS. 8 and 9 may be modified based on FIG. 4B .

In this way, since the operation of the application corresponding to the event is performed based on whether or not the ear is recognized, the user input for starting the operation of the application is omitted to increase user convenience, or additional identification is performed in the unlocked state to improve security has the advantage of being strengthened.

The method for unlocking the mobile device according to the embodiments described above and the operation performed by the mobile device may be at least partially implemented as a computer program and recorded in a computer-readable recording medium. For example, embodied with a program product consisting of a computer-readable medium containing program code, which may be executed by a processor for performing any or all steps, operations, or processes described.

The computer may be any device that may be incorporated into or may be a computing device such as a desktop computer, laptop computer, notebook, smart phone, or the like. A computer is a device having one or more alternative and special purpose processors, memory, storage, and networking components (either wireless or wired). The computer may run, for example, an operating system compatible with Microsoft's Windows, an operating system such as Apple OS X or iOS, a Linux distribution, or Google's Android OS.

The computer-readable recording medium includes all types of recording identification devices in which computer-readable data is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage identification device, and the like. In addition, the computer-readable recording medium may be distributed in a network-connected computer system, and the computer-readable code may be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present embodiment may be easily understood by those skilled in the art to which the present embodiment belongs.

Although the present invention as described above has been described with reference to the embodiments shown in the drawings, it will be understood that these are merely exemplary, and that various modifications and variations of the embodiments are possible therefrom by those of ordinary skill in the art. However, such modifications should be considered to be within the technical protection scope of the present invention. Accordingly, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (22)

In a mobile device unlocking method using ear recognition-based authentication performed by a mobile device including a recording unit, a display unit, and a processor,
detecting an unlock attempt in a lock state, wherein the lock state includes a default lock state and an unlock state;
receiving an unlock input different from ear recognition to change the default lock state to the unlock state and display a screen indicated by one or more application interface objects;
When an event occurs in the changed release state, displaying an interface screen corresponding to the event occurrence;
acquiring an image of the object in which at least a part of the object's body appears after the event has occurred;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject;
checking whether the set of the extracted ear features of the target satisfies a preset condition; and
and executing an operation of an application corresponding to the event when it is confirmed that the extracted set of ear features of the target satisfies a preset condition.
The method of claim 1 , wherein extracting the set of ear features of the subject comprises:
applying the detected region of the ear to a global characterization model configured to extract a first set of features for global characteristics of the ear; and
and applying the detected ear region to a regional characterization model configured to extract a second set of features for regional characteristics of the ear.
According to claim 2, wherein the global characteristic analysis model,
It is learned to extract a set of first features by using multiple images obtained from the same training object as a training image,
The regional characteristic analysis model is,
It is learned to extract a set of second features by using multiple images obtained from the same training object as a training image,
and the multiple images share at least a portion of the training subject's ear.
3. The method of claim 2,
wherein the global characteristic of the ear comprises one or more of the contour of the ear and the composition of the ear;
wherein the regional characteristic of the ear comprises a local pattern present in the ear.
The method of claim 2, wherein the step of confirming whether the set of ear features of the extracted target satisfies a preset condition comprises:
comparing the first set of features and the set of second features with a pre-stored set of first and second features of a user of the mobile device to calculate a degree of similarity of ear features between the subject and the user; and
and changing the first interface state to the second interface state when the matching degree based on the similarity is higher than a predetermined threshold.
The method of claim 5, wherein calculating the degree of similarity between the subject and the user of the ear feature comprises:
comparing a set of first characteristics of the subject with a pre-stored set of first characteristics of a user of the mobile device to calculate a first degree of similarity representing a degree of similarity between the global characteristics of the subject and the user's ears;
comparing the set of second characteristics of the subject with a pre-stored set of second characteristics of the user of the mobile device to calculate a second degree of similarity indicating a degree of similarity between the global characteristics of the subject and the user's ears; and
and calculating a similarity of an ear feature between the target and the user by combining the first degree of similarity and the second degree of similarity.
3. The method of claim 2,
Before applying to the global characteristic analysis model or regional characteristic analysis model, if a change in motion of the mobile device is detected, correcting the shooting angle of the target image or the image of the ear region based on the change in motion of the mobile device A method of unlocking a mobile device further comprising steps.
According to claim 1,
How to unlock the mobile device:
prior to receiving an unlocking input different from ear recognition, detecting an unlocking attempt in a locked state;
obtaining an image of the subject in which at least a part of the subject's body appears in a locked state before receiving an unlocking input different from ear recognition;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and
and displaying a screen on which one or more application interface objects are displayed when it is confirmed that the extracted set of ear features of the target satisfies a preset condition.
The method of claim 8, wherein the method of unlocking the mobile device comprises:
Before displaying a screen on which one or more application interface objects are displayed, the method further comprising changing the basic lock state to an unlock state and notifying the state change.
According to claim 1,
The lock state further includes an application lock state, wherein the method for unlocking the mobile device includes:
Before receiving an unlock input different from ear recognition, if an event occurs in the basic lock state, displaying an interface screen corresponding to the occurrence of the event - In the basic lock state, the application corresponding to the event enters the application lock state set;
acquiring an image of a target in which at least a part of the target's body is displayed while an interface screen corresponding to the occurrence of the event is displayed;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and
and executing an operation of an application corresponding to the event after it is checked whether the extracted set of ear features of the target satisfies a preset condition.
11. The method of claim 10,
The event includes an incoming call,
and after it is confirmed whether the set of the extracted subject's ear features satisfies a preset condition, the connection of the incoming call is executed.
delete According to claim 1,
wherein the different unlocking input includes one or more of a touch, a non-touch gesture, a typing, and a voice, and a photographing of a portion different from the ear.
According to claim 1,
The event includes an incoming call,
and after it is confirmed whether the set of the extracted subject's ear features satisfies a preset condition, acceptance of incoming calls is executed.
According to claim 1,
Prior to detecting the ear region, if the illuminance level of the mobile device is less than or equal to a threshold, providing a guide for adjusting a distance between the mobile device and the object to the target.
According to claim 1,
prior to extracting the set of ear features, verifying that the detected ear region includes at least a portion of the subject's ear;
when the detected ear area includes at least a part of the subject's ear, checking whether the size of the detected ear area falls within a predetermined size range; and
When the size of the detected ear region does not fall within a predetermined size range, the method further comprising the step of providing a guide to the subject to adjust the distance between the mobile device and the object to obtain an ear region belonging to the predetermined size range How to unlock your mobile device.
The method of claim 1, wherein detecting an attempt to unlock the lock in the locked state comprises:
Detect an unlock attempt when detecting one or more of: detecting a change in the rotation angle of the mobile device within a predetermined range for a specified unit time, and detecting that the illuminance is maintained within a predetermined range for a specified unit time deciding to do; and
A method of unlocking a mobile device comprising the step of activating ear recognition for unlocking.
A computer readable recording medium readable by a computing device including a processor and storing program instructions operable by the computing device, wherein the processor is configured to cause the program instructions to be executed by the processor of the computer. A computer-readable recording medium for performing the method of unlocking a mobile device according to any one of claims 11, 13 to 17.
a photographing device for acquiring an image of an object;
display;
at least one unit of a touch unit for acquiring a touch input, another input unit for acquiring an input different from the touch, and a microphone;
processor; and
a memory storing a program configured to be executed by the processor, the program comprising:
detecting an unlock attempt in a lock state, wherein the lock state includes a default lock state and an unlock state;
receiving an unlock input different from ear recognition to change the default lock state to the unlock state and display a screen indicated by one or more application interface objects;
displaying an interface screen corresponding to the occurrence of an event when an event occurs in the already changed release state;
acquiring an image of the object in which at least a part of the object's body appears after the event has occurred;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject;
checking whether the set of the extracted ear features of the target satisfies a preset condition; and
and executing an operation of an application corresponding to the event when it is confirmed whether the extracted set of ear features of the target satisfies a preset condition.
20. The method of claim 19,
The program is:
prior to receiving an unlocking input different from ear recognition, detecting an unlocking attempt in a locked state;
obtaining an image of the subject in which at least a part of the subject's body appears in a locked state before receiving an unlocking input different from ear recognition;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and
When it is confirmed that the extracted set of ear features of the target satisfies a preset condition, displaying a screen indicated by one or more application interface objects; .
20. The method of claim 19,
The lock state further includes an application lock state, wherein the program includes:
Before receiving an unlock input different from ear recognition, if an event occurs in the basic lock state, displaying an interface screen corresponding to the occurrence of the event - In the basic lock state, the application corresponding to the event enters the application lock state set;
acquiring an image of a target in which at least a part of the target's body is displayed while an interface screen corresponding to the occurrence of the event is displayed;
if the image of the subject includes at least a portion of an ear, extracting a set of ear features of the subject from the image of the subject; and
The mobile device of claim 1, further comprising instructions for performing the step of executing an operation of an application corresponding to the event after it is confirmed whether the extracted set of ear features of the target satisfies a preset condition.
delete

Images