KR20120020970A - Mobile terminal and operation method thereof - Google Patents

Abstract

PURPOSE: A mobile terminal and an operating method thereof are provided to expand a user interface. CONSTITUTION: If an application mode is set according to a command of a user, a control unit displays an indicator matched with a reference brain wave on a display unit(S810,S820). The control unit analyzes a detected brain wave(S830). The control unit calculates a similarity level of the existing brain wave and the analyzed brain wave(S840). If the similarity level is higher than a fixed level, the control unit performs an operation by a command corresponding to the indicator(S850,S860).

Description

Mobile terminal and operation method thereof

The present invention relates to a mobile terminal and a method of operating the same, and more particularly, to a mobile terminal and a method of operating the same in connection with a brain wave of a user to provide a service.

A portable terminal is a portable device that is portable and has one or more functions of making voice and video calls, inputting and outputting information, and storing data. As the functions of the portable terminals are diversified, the portable terminal can provide complex functions such as photographing a picture or a video, playing a music file or a video file, receiving a game, broadcasting, or wireless internet.

On the other hand, human brain activity represents a variety of movements by the interaction or activity of a plurality of neurons (Neuron), the externally recorded scalp brain waves (EEG: Electroencephalogram) is a signal of the fine brain surface, when measuring It changes spatially and temporally according to its state and function.

Here, the recognition method using the brain waves can recognize the neural pattern of the brain, the brain computer interface (BCI :) field to build a direct interface between human and machine using brain waves only to expand the scope of application of brain waves have.

In addition, by applying a brain computer interface (BCI) to a mobile terminal, various service provision methods using the brain waves have been devised.

Accordingly, an object of the present invention is to provide a mobile terminal and a method of operating the mobile terminal for controlling the operation of the mobile terminal in a specific EEG pattern.

Another object of the present invention is to provide a mobile terminal and a method of operating the same, which provide a method for registering and verifying an EEG pattern for controlling the operation of the mobile terminal.

According to an aspect of the present invention, there is provided a method of operating a mobile terminal, the method including: displaying an indicator matched with a reference EEG in an application mode; Calculating the similarity between the brain waves of the user and the reference brain waves; And performing an operation according to a user command corresponding to the indicator when the similarity is greater than or equal to a predetermined value.

In addition, the portable terminal according to the present invention for achieving the above object, a display unit; A storage unit matching and storing the reference EEG and an indicator usable as a user command; And a controller configured to display the indicator on the display unit and to operate the portable terminal according to a command corresponding to the indicator when the user's brain wave has a similarity degree to the reference brain wave.

According to the present invention, since the portable terminal can be operated by the brainwave of the user, the user interface can be extended. In addition, since the portable terminal can be operated by the user's brain waves, the user's movement for operating the portable terminal can be minimized.

1 is a block diagram of a portable terminal according to an embodiment of the present invention;
2 is a perspective view of a portable terminal according to an embodiment of the present invention;
3 is a rear perspective view of the mobile terminal shown in FIG. 2;
4 is a view showing an example of an audio output device connectable to a portable terminal according to the present invention;
5 is a view showing a mobile terminal having a retractable brain wave sensor according to an embodiment of the present invention;
6 is a flowchart illustrating a process of registering a reference EEG according to an embodiment of the present invention;
7 is a flowchart illustrating a method of verifying a reference EEG according to an embodiment of the present invention;
8 is a flowchart illustrating a method of operating a mobile terminal with an EEG according to an embodiment of the present invention;
9 is a reference view illustrating a method of registering a reference EEG according to an embodiment of the present invention;
10 is a reference view illustrating a method of verifying a reference EEG according to an embodiment of the present invention;
11 is a reference diagram illustrating a method of using a reference EEG as a user command in a menu mode of an application mode according to an embodiment of the present invention;
12 is a reference diagram illustrating a method of operating a mobile terminal according to an EEG of a user in a mode of generating a communication event according to an embodiment of the present invention;
FIG. 13 is a reference view for explaining a method of operating a mobile terminal with an EEG of a user in a busy mode according to an embodiment of the present invention; and
14 is a reference diagram for explaining a method of operating a mobile terminal using an EEG of a user in a playback mode of a video file according to an embodiment of the present invention.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The portable terminal described in the present specification includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like. In addition, suffixes "module" and " part "for the components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Therefore, the "module" and "unit" may be used interchangeably.

FIG. 1 is a block diagram of a portable terminal according to an exemplary embodiment of the present invention. Referring to FIG. Referring to FIG. 1, a mobile terminal according to an embodiment of the present invention will be described in terms of components according to functions.

1, the portable terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A memory 160, an interface unit 170, a control unit 180, and a power supply unit 190. Such components may be configured by combining two or more components into one component, or by dividing one or more components into two or more components as necessary when implemented in an actual application.

The wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 113, a wireless Internet module 115, a short distance communication module 117, and a GPS module 119.

The broadcast receiving module 111 receives at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like. The broadcast management server may mean a server for generating and transmitting at least one of a broadcast signal and broadcast related information, or a server receiving at least one of pre-generated broadcast signals and broadcast related information and transmitting the same to a terminal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal. The broadcast related information may also be provided through a mobile communication network, and in this case, may be received by the mobile communication module 113. Broadcast related information may exist in various forms.

The broadcast receiving module 111 may receive a broadcast signal using various broadcast systems, and may receive a digital broadcast signal using a digital broadcast system. In addition, the broadcast receiving module 111 may be configured to be suitable for all broadcasting systems that provide broadcasting signals, as well as the digital broadcasting system. Broadcast signals or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 113 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to voice call signal, video call signal, or text / multimedia message transmission and reception.

The wireless Internet module 115 refers to a module for wireless Internet access, and the wireless Internet module 115 can be embedded in the mobile terminal 100 or externally. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short-range communication module 117 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

A GPS (Global Position System) module 119 receives position information from a plurality of GPS satellites.

The A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 123. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 123 receives an external sound signal by a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 113 and output when the voice data is in the call mode. The microphone 123 may be a variety of noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 130 generates key input data input by the user for controlling the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (constant voltage / capacitance), etc. that may receive a command or information by a user's push or touch operation. In addition, the user input unit 130 may be configured as a jog wheel or a jog method or a joystick that rotates a key, a finger mouse, or the like. Particularly, when the touch pad has a mutual layer structure with the display unit 151 described later, it can be called a touch screen.

The sensing unit 140 senses the current state of the portable terminal 100 such as the open / close state of the portable terminal 100, the position of the portable terminal 100, Thereby generating a sensing signal. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like.

The sensing unit 140 may include a proximity sensor 141, a pressure sensor 143, a motion sensor 145, an EEG sensor 147, and the like. The proximity sensor 141 may detect the presence or absence of an object approaching the portable terminal 100 or an object present in the vicinity of the portable terminal 100 without mechanical contact. The proximity sensor 141 may detect a proximity object by using a change in an alternating magnetic field or a change in a static magnetic field, or by using a change rate of capacitance. Two or more proximity sensors 141 may be provided according to the configuration aspect.

The pressure sensor 143 may detect whether pressure is applied to the portable terminal 100, the magnitude of the pressure, and the like. The pressure sensor 143 may be installed at a portion where the pressure of the portable terminal 100 is required depending on the use environment. If the pressure sensor 143 is installed in the display unit 151, a touch input through the display unit 151 and a greater pressure than the touch input are applied according to a signal output from the pressure sensor 143. The pressure touch input can be identified. Also, the magnitude of the pressure applied to the display unit 151 at the time of the pressure touch input can be determined according to the signal output from the pressure sensor 143. [

The motion sensor 145 detects the position and movement of the portable terminal 100 using an acceleration sensor, a gyro sensor, or the like. An acceleration sensor that can be used for the motion sensor 145 is a device that converts an acceleration change in one direction into an electric signal and is widely used along with the development of MEMS (micro-electromechanical systems) technology. The acceleration sensor measures the acceleration of a small value built in the airbag system of an automobile and recognizes the minute motion of the human hand and measures the acceleration of a large value used as an input means such as a game There are various types. Acceleration sensors are usually constructed by mounting two or three axes in one package. Depending on the usage environment, only one axis of Z axis is required. Therefore, when the acceleration sensor in the X-axis direction or the Y-axis direction is used instead of the Z-axis direction for some reason, the acceleration sensor may be mounted on the main substrate by using a separate piece substrate.

The gyro sensor is a sensor for measuring the angular velocity, and it can sense the direction of rotation with respect to the reference direction.

The EEG sensor 147 is attached to the user's head to detect the EEG generated from the user's brain (EEG). The EEG sensor 147 may be installed at a portion where the EEG is easily detected in the portable terminal 100 or may be installed in an external device (eg, a headset) that can communicate with the portable terminal.

The output unit 150 is for outputting audio data, image data, and haptic data. The output unit 150 may include a display unit 151, an audio output module 153, a haptic module 155, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the portable terminal 100 is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the portable terminal 100 is in the video communication mode or the photographing mode, the photographed or received images can be displayed individually or simultaneously, and the UI and the GUI are displayed.

Meanwhile, as described above, when the display unit 151 and the touch pad form a mutual layer structure and constitute a touch screen, the display unit 151 is an input device capable of inputting information by a user's touch in addition to the output device. May also be used.

If the display unit 151 is configured as a touch screen, it may include a touch screen panel, a touch screen panel controller, and the like. In this case, the touch screen panel is a transparent panel attached to the outside and may be connected to the internal bus of the portable terminal 100. The touch screen panel keeps a watch on the contact result, and if there is a touch input, sends the corresponding signals to the touch screen panel controller. The touch screen panel controller processes the signals, and then transmits corresponding data to the controller 180 so that the controller 180 can determine whether the touch input has been made and which area of the touch screen has been touched.

The display unit 151 may be formed of an e-paper. Electronic paper (e-Paper) is a kind of reflective display, and has excellent visual characteristics such as high resolution, wide viewing angle and bright white background as conventional paper and ink. The electronic paper (e-paper) can be implemented on any substrate such as plastic, metal, paper, and the image is retained even after the power is shut off, and the battery life of the portable terminal 100 is long Can be maintained. As the electronic paper, a hemispherical twist ball filled with a telephone can be used, or an electrophoresis method and a microcapsule can be used.

In addition, the display unit 151 may include a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display. It may include at least one of (3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, the external display unit (not shown) and the internal display unit (not shown) may be simultaneously provided in the portable terminal 100.

The audio output module 153 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 153 outputs sound signals related to functions performed in the portable terminal 100, for example, a call signal reception tone, a message reception tone, and the like. The sound output module 153 may include a speaker, a buzzer, and the like.

The haptic module 155 generates various tactile effects that a user can feel. A representative example of the haptic effect generated by the haptic module 155 is a vibration effect. When the haptic module 155 generates vibration by the tactile effect, the intensity and pattern of the vibration generated by the haptic module 155 may be converted, and may be output by combining different vibrations or sequentially.

In addition to vibration, the haptic module 155 may be provided with an effect of stimulation by an arrangement of pins vertically moving with respect to the contact skin surface, an effect of stimulation through the injection or suction force of air through the injection or inlet, and a stimulus that strikes the skin surface. Various tactile effects can be generated, such as effects of stimulation through contact of electrodes, effects of stimulation using electrostatic force, and effects of reproducing a sense of cold and warm using elements capable of endothermic or exothermic heat. The haptic module 155 may not only deliver the haptic effect through direct contact, but may also implement the haptic effect through the muscles of the user's finger or arm. Two or more haptic modules 155 may be provided according to a configuration aspect of the mobile terminal 100.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input or output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM , And a ROM. ≪ / RTI > In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the portable terminal 100. Examples of the external device connected to the portable terminal 100 include a wired / wireless headset, an external charger, a wired / wireless data port, a memory card, a SIM (Subscriber Identification Module) card, a UIM An audio input / output (I / O) terminal, a video I / O (input / output) terminal, and an earphone. The interface unit 170 receives data from the external device or receives power from the external device and transmits the data to each component in the portable terminal 100 so that data in the portable terminal 100 can be transmitted to the external device .

The interface unit 170 is a path through which power from the cradle connected when the portable terminal 100 is connected to the external cradle is supplied to the portable terminal 100 or various command signals inputted from the cradle by the user are carried And may be a passage to be transmitted to the terminal 100.

The controller 180 typically controls the operation of the respective units to control the overall operation of the mobile terminal 100. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia playback module 181 for multimedia playback. The multimedia playback module 181 may be configured in hardware in the controller 180 or may be configured in software separately from the controller 180.

In particular, the controller 180 analyzes the EEG detected by the EEG sensor 147 and determines whether a signal of a specific frequency band among the analyzed EEG is included in the reference range. In addition, the memory 160 stores a reference range that is brain wave information indicating a specific mental state (eg, excited state, concentrated state, stable state, etc.) of the user.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The mobile terminal 100 having such a configuration can be configured to be operable in a communication system capable of transmitting data through a frame or a packet, including a wired / wireless communication system and a satellite-based communication system. have.

The portable terminal related to the present invention has been described above in terms of components according to functions. Hereinafter, referring to FIGS. 2 and 3, the portable terminal related to the present invention will be further described in terms of components according to appearance. Hereinafter, for convenience of description, a bar type portable terminal having a front touch screen among various types of portable terminals such as a folder type, a bar type, a swing type, a slider type, and the like will be described as an example. However, the present invention is not limited to a bar-type portable terminal, but can be applied to all types of portable terminals including the above-mentioned type.

2 is a perspective view of a portable terminal according to an embodiment of the present invention viewed from the front. Referring to FIG. 2, the case forming the exterior of the portable terminal 100 is formed by the front case 100A-1 and the rear case 100A-2. Various electronic components are embedded in the space formed by the front case 100A-1 and the rear case 100A-2. At least one intermediate case may be further disposed between the front case 100A-1 and the rear case 100A-2. Such cases may be formed by injecting synthetic resin, or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti).

In the main body, specifically, the first front case 100A-1, the display unit 151, the first sound output module 153a, the first camera 121a, the first user input unit 130a, and the old sensor 147 are provided. Can be deployed. In addition, a second user input unit 130b, a third user input unit 130, and a microphone 123 may be disposed on the side surface of the rear case 100A-2.

The display unit 151 includes a liquid crystal display (LCD), organic light emitting diodes (OLED), and the like, which visually express information. Since the touch pad is superimposed on the display unit 151 in a layer structure, the display unit 151 may operate as a touch screen, and may be configured to input information by a user's touch.

The first acoustic output module 153a may be implemented in the form of a receiver or a speaker. The first camera 121a may be implemented to be suitable for capturing an image or a video of a user or the like. The microphone 123 may be implemented in a form suitable for receiving a user's voice, other sounds, and the like.

The first to third user input units 130a, 130b, and 130c may be collectively referred to as the user input unit 130, and may be employed in any manner as long as the user is operating in a tactile manner.

For example, the user input unit 130 may be implemented as a dome switch or a touch pad that may receive a command or information by a user's push or touch operation, or may be operated as a wheel or jog or a joystick that rotates a key. Or the like. In the functional aspect, the first user input unit 130a is for inputting a command such as a start, end, call, etc., and the second user input unit 130b is for inputting a selection of an operation mode. In addition, the third user input unit 130c may operate as a hot-key for activating a special function in the portable terminal 100.

When the user's finger approaches the display unit 151, the proximity sensor 141 installed on the first body detects this and outputs a proximity signal. At this time, it may be configured to output different proximity signals according to the distance the user's finger is located. In general, the distance between the detection object approaching the proximity sensor and outputting the proximity signal is called the detection distance. When the proximity signal output from each proximity sensor is compared using a plurality of proximity sensors having different detection distances, the detection object is detected. You can see how close is.

In addition, a plurality of proximity sensors having different detection areas are arranged to determine which proximity sensor is output from among the proximity sensors to determine which area of the display unit 151 the proximity object approaches, and the detection object. May be determined to move closer to the display unit 151. Accordingly, the controller 180 may control to output a corresponding vibration signal by selecting a touch key corresponding to a close position of a user's finger or the like.

When the user tilts or shakes the mobile terminal, the motion sensor 145 detects the movement of the mobile terminal. The motion sensor 145 generates a signal corresponding to the movement of the mobile terminal and outputs the signal to the controller 180. The controller 180 detects information related to the movement, such as the direction, angle, speed, intensity, and current position of the portable terminal, from the signal generated by the motion sensor 145.

The controller 180 tracks the movement of the mobile terminal based on the information detected from the signal generated by the motion sensor 145. Depending on the components constituting the motion sensor 145, information that can be detected from the signal generated by the gyro sensor may be different. The mobile terminal is provided with a motion sensor 145 according to the motion information of the mobile terminal to be detected. At least one motion sensor 145 may be provided in the mobile terminal. The controller 180 may control the motion sensor 145 to operate only when executing a specific application in response to the information to be detected.

3 is a rear perspective view of the portable terminal shown in FIG. 2. Referring to FIG. 3, a fourth user input unit 130d, a fifth user input unit 130d, and an interface unit 170 may be disposed on the side of the rear case 100A-2, and the rear case 100A-2 may be disposed. The second camera 121b may be additionally mounted on the rear side of the).

The second camera 121b may have a photographing direction substantially opposite to that of the first camera 121a, and may have different pixels from the first camera 121a. For example, in the case of a video call or the like, the first camera 121a has a low pixel so that the user's face is photographed and transmitted to the other party, and the second camera 121b photographs a general subject and does not transmit it immediately. In many cases, it is desirable to have a high pixel.

The mirror 125 and the flash 126 may be further disposed adjacent to the second camera 121b. The mirror 125 allows the user to see his / her own face or the like when the user wants to photograph (self-photograph) the second camera 121b. The flash 126 illuminates the subject when the subject is photographed by the second camera 121b.

A second sound output module (not shown) may be further disposed on the rear case 100A-2. The second sound output module may implement the stereo function together with the first sound output module 153a, and may be used for talking in the speakerphone mode.

In addition, an antenna (not shown) for receiving a broadcast signal may be disposed on one side of the second rear case 100A-2 in addition to an antenna for a call. The antenna may be installed to be pulled out of the rear case 100A-2.

The interface unit 170 serves as a path for allowing the portable terminal 100 to exchange data with an external device. For example, the interface unit 170 may be at least one of a wired or wireless connection terminal for connecting to the earphone, a port for short-range communication, or a power supply terminal for supplying power to the mobile terminal 100. have. The interface unit 170 may be a card socket that accommodates an external card such as a subscriber identification module (SIM) or a user identity module (UIM) and a memory card for storing information.

The rear case 100A-2 is provided with a power supply unit 190 for supplying power to the portable terminal. The power supply unit 190 is, for example, a rechargeable battery, and may be detachably coupled to the rear case 100A-2 for charging.

In the above description, the second camera 121b and the like are disposed on the rear case 100A-2, but the present invention is not limited thereto. In addition, even if the second camera 12b is not separately provided, the first camera 121a may be rotatably formed to be able to shoot up to the photographing direction of the second camera 121b.

4 is a view showing an example of an audio output device connectable to a portable terminal according to the present invention. Referring to FIG. 4, in the audio output apparatus 200, the first sound output unit 201 and the second sound output unit 203 provided on the left and right sides are connected by a frame 205 to be mounted on a user's head. It is configured to be.

The audio output device 200 having the above configuration is connected to the mobile terminal 100 through short-range communication such as wired or Bluetooth, and outputs an audio signal corresponding to the AV data received from the mobile terminal 100. The sound can be transmitted directly to the user's ear.

One or more brain wave sensors 207a, 207b, 207c, and 207d may be disposed inside the frame 205 of the audio output device 200 to measure brain waves from a user's scalp. The brain wave measured by the audio output device 200 may be transmitted to the mobile terminal 100 after a signal processing process.

As such, the brain wave of the user may be disposed in an audio output device such as a headset or earphone connected to the portable terminal 100, or may be measured through the brain wave sensor 147 provided in the portable terminal 100 itself. In addition, by attaching a wireless brain wave device to a hair band, hair pin, glasses, etc. worn by the user, the mobile terminal 100 receives a multi-channel EEG signal generated by processing an EEG signal detected from the user's scalp through another external device. It may be configured to transmit to or directly to the mobile terminal 100.

FIG. 5 is a diagram illustrating a portable terminal having a retractable brain wave sensor according to an exemplary embodiment of the present invention.

As shown in FIGS. 5A and 5B, a through hole 146 is formed in the main body of the portable terminal. The EEG sensor 147 may be connected to the connection rod 148 that transmits the detected EEG signal to the portable terminal and may be provided to be withdrawn from the portable terminal. The connection rod 148 is preferably implemented with a bendable material. Thus, as shown in FIG. 5C, the user can pull out the connecting rod 148 and attach it to the user's head.

As described above, the EEG sensor unit may be provided in the portable terminal or may be provided in the external device. For convenience of explanation, only the case where the EEG sensor is provided in the portable terminal will be described. However, the EEG sensor and the module for analyzing the EEG may be provided in the external device, in this case, the controller receives the EEG analyzed from the external device, it is preferable to use the EEG without a separate analysis process.

6 is a flowchart illustrating a process of registering a reference EEG according to an embodiment of the present invention.

Referring to FIG. 6, the controller sets a registration mode for registering a reference EEG according to a user's command.

In connection with the present invention, a mode of a mobile terminal is divided into a normal mode and an EEG mode. The general mode refers to a mode that can operate according to a user command input through the user input unit, and the brain wave mode refers to a mode that can operate by recognizing a user's brain wave as a specific user command. In addition, the EEG mode is a registration mode for registering a user's brain waves as a specific user command, a verification mode for verifying whether the user's brain waves can be recognized as a registered specific user command, and the application of operating the mobile terminal by applying the user's brain waves as a specific user command. It is divided into modes.

If the registration mode setting is set (S610), the display unit 151 displays candidate indicators that can be registered by matching with the reference EEG (S620). The candidate indicator is an image such as an arrow, an icon, or the like, and is preferably a sensory image in which a user is easily associated with and a unique brain wave pattern can be formed due to the association.

When the user selects one of the candidate indicators as an indicator, the controller 180 determines that an indicator to be matched with the reference EEG is determined (S630).

The controller 180 displays the determined indicator for a predetermined time (S640), and receives the EEG signal detected by the EEG sensor 147 for the predetermined time (S650). Here, the types of brain waves generated in the human brain are divided into alpha waves, beta waves, theta waves, and delta waves according to frequency bands. Alpha waves are brain waves that occur when a human closes his eyes and relaxes his body and has a frequency between 8 Hz and 13 Hz. Beta waves are most of the brain waves that occur when consciousness is awake, with frequencies above 14 Hz to 100 Hz. Theta waves occur in shallow sleep and have frequencies lower than alpha waves (4 Hz to 8 Hz) and are generated at the boundary between perception and dreams. Delta waves are the most frequently measured brain waves in sleep or unconscious states, with frequencies below 4 Hz lower than theta waves. The brain waves utilized in the present invention are preferably alpha waves and beta waves.

In addition, the controller 180 analyzes the brain wave by signal-processing the brain wave received from the brain wave sensor 147 (S660). Specifically, the brain wave input from the brain wave sensor 147 is amplified, the unwanted wave component is removed from the amplified signal, and the signal from which the unwanted wave is removed is converted into a digital signal. The EEG is then analyzed by Fourier transforming the amplified EEG to calculate an output value for each EEG frequency.

In addition, the controller 180 generates a reference EEG based on the EEG received for a predetermined time (S670). The reference EEG is preferably an average value for each frequency of EEG received for a predetermined time.

The controller 180 matches the reference EEG with the determined indicator and stores the same in the command database (S680). The command database stores reference brain waves, an indicator, and a user command matched with each other. Therefore, if the degree of similarity between the EEG of the user and the reference EEG is greater than or equal to a predetermined value, the controller 180 determines that a user command matched with the reference EEG is input and operates the portable terminal according to the command. The indicator preferably includes information about the instruction. However, when the indicator does not include the information on the command, the controller 180 may display the information on the command together with the indicator. In addition, although the type of the indicator and the type of the command are preferably matched one-to-one, a plurality of commands may be matched to one indicator.

As described above, since the user can freely match and store the reference EEG and the indicator, it is possible to implement a user friendly user interface.

7 is a flowchart illustrating a method of verifying a reference EEG according to an embodiment of the present invention.

Referring to FIG. 7, the controller 180 sets a verification mode for verifying a reference EEG according to a user's command (S710). Then, the controller 180 displays information on the candidate indicator to be verified on the display unit 151 (S720). The candidate indicator to be verified is an indicator previously stored in accordance with the reference EEG.

When the user selects one of the candidate indicators, the controller 180 determines an indicator to be verified (S730) and displays the indicator for a predetermined time (S740).

In addition, the controller 180 receives and analyzes the EEG detected by the EEG sensor 147 for a predetermined time (S750). Since the method of analyzing the EEG has been described above, a detailed description thereof will be omitted.

The controller 180 calculates a similarity degree between the analyzed brain wave and the reference brain wave (S760). In calculating the degree of similarity, the controller 180 may determine the degree of similarity by comparing the average value for each frequency of the brain waves received for a predetermined time with the output value for each frequency of the reference EEG in a critical range. The more frequency bands in the critical range, the higher the similarity; the less frequency bands in the same range, the lower the similarity.

In addition, the controller 180 may determine the degree of similarity by converting an output value for each frequency of the user EEG into a relative ratio with respect to the entire frequency band and comparing the relative ratio with the reference EEG. That is, the more frequency bands having a ratio of the critical range, the higher the similarity, and the lower the frequency bands in the same range, the lower the similarity. The threshold range is a range that is regarded as the same range while including the error range, and may be adjusted according to the error range. In addition, the similarity between the user's EEG and the reference EEG may be calculated using a maximum likelihood method or a cross-correlation method.

In the present exemplary embodiment, the output value for each frequency of the EEG received for a predetermined time is compared with the output value for each frequency of the reference EEG, but is not limited thereto. Similarity may also be calculated by comparing the EEG of the user received in real time with the reference EEG.

If it is determined that the similarity is greater than or equal to a predetermined value (S770), the controller 180 generates a notification indicating that the reference EEG is suitable for use as a user command and displays the notification on the display unit 151 (S780). However, if the similarity is determined to be less than the predetermined value (S770-N), the controller 180 generates a notification indicating that the reference EEG is not suitable for use as a user command and displays it on the display unit 151 (S790). .

As described above, it may be verified whether the registered reference EEG is suitable to be utilized as a user command.

8 is a flowchart illustrating a method of operating a mobile terminal with an EEG according to an embodiment of the present invention.

Referring to FIG. 8, when an application mode for applying a reference EEG as a command is set according to a user's command (S810), the display unit 151 displays an indicator matching the reference EEG ( S820). Preferably, the indicator includes information about an instruction. However, when the above information does not include the information on the command, it is preferable to include the information on the command together with the indicator.

 On the other hand, the controller 180 receives and analyzes the brain waves detected by the brain wave sensor 147 (S830). Since the method of analyzing the EEG has been described above, a detailed description thereof will be omitted.

The controller 180 calculates a similarity degree between the analyzed brain wave and the reference brain wave (S840). That is, the similarity is calculated by comparing the frequency-specific output value of the user EEG and the frequency-specific output value of the reference EEG.

If it is determined that the similarity is greater than or equal to a predetermined value (S850-Y), the controller 180 performs an operation according to a command corresponding to the indicator (S860). The command corresponding to the indicator may vary depending on the type of the application mode, which will be described later.

9 is a reference view for explaining a method of registering a reference EEG according to an embodiment of the present invention.

When the user selects the EEG mode in the setting mode of the menu screen, as shown in FIG. 9A, the sub-menu screen 910 of the EEG mode is displayed. By selecting the registration mode item 912 on the sub-menu screen 910, the user may register the reference EEG. Thus, when the registration mode item 912 is selected, at least one candidate indicator screen 920 that can be registered by matching with the reference EEG is displayed, as shown in FIG. 9B. The user selects any one indicator 922 on the candidate indicator screen 920 and selects a confirmation key 924. Then, as shown in FIG. 9C, the enlarged indicator 930 is displayed by enlarging the selected indicator 924. The user selects start key 940. On the other hand, the user preferably thinks about the indicator displayed on the display unit 151. After the start key 940 is selected, the controller 180 receives an EEG from the EEG sensor 147 and analyzes the EEG to calculate an output value for each frequency. The reception and analysis of the brain waves is performed until the end key 950 is selected by the user. When the user selects the end key 950, the controller 180 registers the average value of the output values for each frequency of the brain waves as the reference brain waves. In registering the reference EEG, it is preferable to register by matching the selected indicator 930. Then, it is preferable to display a notification indicating that the registration is completed.

In FIG. 9, the direction indicator is indicated as the candidate indicator, but the type of the candidate indicator is not limited. Any kind of indicator may be used as long as the user can generate a specific pattern of brain waves that can be easily identified by the indicator. In addition, the indicator need not be image data, and may be audio data, haptic data, or a combination of these data.

10 is a reference diagram illustrating a method of verifying a reference EEG according to an embodiment of the present invention.

In the submenu screen of the EEG mode shown in FIG. 9A, a verification mode item is selected. Then, as shown in (a) of FIG. 10, the candidate indicator screen 1010 matching and registered with the reference EEG is displayed. The user selects an indicator 1012 to be verified among the candidate indicators described above. Then, as shown in FIG. 10B, the selected indicator is enlarged to display the enlarged indicator 1020, and the user selects the start key 1030. The user generates an EEG of a specific pattern while looking at the indicator 1020 displayed on the display unit 151. After the start key 1030 is selected, the controller 180 receives an EEG from the EEG sensor 147 and analyzes the EEG to calculate an output value for each frequency. The controller 180 receives and analyzes an EEG until an end key 1040 is selected by a user. When the user selects the end key 1040, the controller 180 calculates a similarity degree between the average value of the EEG frequency-specific output value and the reference EEG. If the similarity is greater than or equal to a predetermined value, as shown in FIG. 10C, a first notification 1050 is generated and displayed on the display unit 151 indicating that the reference EEG is suitable for use as a user command. do. However, if the similarity is less than a predetermined value, as shown in FIG. 10 (d), the display brain 151 generates a second notification 1060 indicating that the reference EEG is not suitable for use as a user command. Mark on.

The user may view the verification result and maintain or cancel the registration of the reference EEG. In this way, it is possible to verify the adequacy of the user command using the EEG.

FIG. 11 is a reference diagram illustrating a method of using a reference EEG as a user command in a menu mode of an application mode according to an embodiment of the present invention.

As illustrated in (a) of FIG. 11, items indicating various services provided by the mobile terminal are displayed on the screen 1110 of the menu mode in the application mode. Any one of the items 1120 is highlighted 1130. In addition, the indicators 1140 and 1150 matched with the reference EEG are displayed on the menu screen. When the user wants to move the highlight 1130, he or she is associated with an indicator associated with the move. While the user is associated with the indicator associated with the movement, the controller 180 receives and analyzes the EEG detected from the EEG sensor. If the analyzed EEG has a similarity or more with a reference EEG associated with the movement by a predetermined value or more, the controller 180 obtains an instruction matched with the reference EEG associated with the movement from the instruction database, and moves the highlight according to the acquired instruction. For example, if a user generates an EEG while viewing the indicator 1142 associated with the movement in the left direction, and the EEG of the user is similar to the reference EEG associated with the movement in the left direction by a predetermined value or more, the controller 180 highlights ( 1130 is moved to the left direction. Thus, as shown in FIG. 11B, the highlight 1130 moves left to overlap the item 1160 in the left direction. In addition, the user may input a command for selecting the highlighted item in association with the indicator 1150 related to the selection.

If the indicator does not include information about the instruction, it is preferable to display the information about the instruction together with the indicator. This is because the portable terminal has various commands depending on the type of application mode, but the type of indicator may be limited.

Hereinafter, when the information on the command is not included in the indicator, a method of displaying information on the indicator and the command and a method of operating the user's brain wave of the mobile terminal will be described.

12 is a reference diagram for explaining a method of operating a mobile terminal according to an EEG of a user in a communication event generation mode according to an embodiment of the present invention.

When a communication event, for example, a call reception call occurs on the idle screen, as shown in FIG. 12, the controller 180 displays a screen 1210 corresponding to the call reception call on the display unit 151. In addition, information 1220 about a command that may be utilized according to generation of a call reception call is displayed. In addition, the indicator 1230 matched with the above command is also displayed. If the user wants to permit the call connection, the user associates an indicator related to the permission of the call connection. On the other hand, if the user's brain wave received from the brain wave sensor 147 has a similarity to the reference brain wave related to the permission of the call connection or more than a predetermined value, the controller 180 displays a screen during a call allowing the call connection. do.

FIG. 13 is a reference diagram illustrating a method of operating a mobile terminal using an EEG of a user in a busy mode according to an embodiment of the present invention.

As illustrated in FIG. 13, the screen 1310 corresponding to the busy mode displays information 1320 about a command that may be utilized in the busy mode and an indicator 1330 matching the command. When the user wants to end the call, the user associates an indicator related to the end of the call, and the EEG sensor 147 receives the EEG of the user. The controller 180 determines that a call termination command has been received when the brainwave of the user has a similarity or more with a reference brainwave associated with the termination of the call, and ends the call, and displays the call termination screen.

14 is a reference diagram for explaining a method of operating a mobile terminal using an EEG of a user in a playback mode of a video file according to an embodiment of the present invention.

As illustrated in FIG. 14, the playback screen 1410 of the video file displays information 1420 about a command that may be utilized in the playback mode and an indicator 1430 matched with the command. When the user wants to end the reproduction, the user associates an indicator related to the termination of the reproduction, and the EEG sensor 147 receives the EEG of the user. If the brain wave of the user has a similarity with a reference brain wave related to the end of reproduction more than a predetermined value, the controller 180 determines that a reproduction end command has been received and ends the reproduction.

In the present invention, for convenience of description, the present invention has been described only when the indicator corresponding to the reference EEG is image data. However, the present invention is not limited thereto, and the indicator may be audio data and haptic data, or may be a combination of the above data.

Meanwhile, the present invention can be embodied as processor readable codes on a processor readable recording medium provided in a mobile terminal such as a mobile station modem (MSM). The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may be implemented in the form of a carrier wave such as transmission through the Internet. The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

110: wireless communication unit 120: A / V input unit
130: user input unit 140: sensing unit
150: output unit 151: display unit
160: memory 170: interface unit
180:

Claims (15)

In an application mode, displaying an indicator matching the reference EEG;
Calculating the similarity between the brain waves of the user and the reference brain waves; And
And performing an operation according to a user command corresponding to the indicator when the similarity is greater than or equal to a predetermined value. The method of claim 1,
If the information about the user command is not the same as the indicator,
And displaying the information on the user command together with the indicator. The method of claim 1,
In a setting mode, displaying the indicator;
Generating the reference EEG based on the received EEG of the user while the indicator is displayed; And
And matching and storing the reference EEG and the indicator. The method of claim 3,
The reference brain wave,
And an average value for each frequency of the user's brain waves received while the indicator is displayed. The method of claim 1,
In the verify mode, displaying the indicator; And
Calculating the similarity between the received brain wave of the user and the reference brain wave while the indicator is displayed;
And displaying the adequacy information for the reference brain wave based on the similarity calculated in the verification mode. 6. The method of claim 5,
The adequacy information
When the similarity calculated in the verification mode is equal to or greater than the predetermined value, it is a notification indicating that the reference EEG is appropriate.
And if the similarity calculated in the verification mode is less than the predetermined value, a notification indicating that the reference EEG is inappropriate. The method of claim 1,
The application mode is any one of a web browsing execution mode, a communication event generation mode, a call mode, a file playback mode, and a menu mode. The method of claim 1,
The user command may vary according to the type of the application mode. The method of claim 1,
If the application mode is the playback mode of the file,
And the user command is any one of forward play, reverse play, pause play, and end play of the file. The method of claim 1,
If the application mode is a menu mode,
And wherein the user command is any one of moving to a position of a cursor or highlight displayed in the menu mode and selecting an item displayed at a position of the cursor or highlight. A display unit;
A storage unit matching and storing the reference EEG and an indicator usable as a user command; And
And a controller configured to display the indicator on the display unit and to operate the portable terminal according to a command corresponding to the indicator when a user's brain wave has a similarity degree to the reference brain wave. Mobile terminal. 12. The method of claim 11,
The control unit,
And the information on the user command is displayed together with the indicator when the information on the user command is not the same as the indicator. 12. The method of claim 11,
The control unit,
In the setting mode, the indicator is displayed, the mobile terminal characterized in that for generating the reference brainwaves based on the brainwaves of the user received while the indicator is displayed. 12. The method of claim 11,
The control unit,
In the verification mode, the indicator is displayed, and the similarity between the brainwave of the user and the reference brainwave received while the indicator is displayed, and the suitability information for the reference brainwave is displayed based on the calculated similarity. Portable terminal. The method of claim 14,
The adequacy information
When the similarity calculated in the verification mode is equal to or greater than the predetermined value, it is a notification indicating that the reference EEG is appropriate.
And the similarity calculated in the verification mode is less than the predetermined value, wherein the reference EEG is a notification indicating that the reference EEG is inappropriate.

Images