WO2021201429A1

WO2021201429A1 - Electronic device and method for controlling audio output of same

Info

Publication number: WO2021201429A1
Application number: PCT/KR2021/002038
Authority: WO
Inventors: 김용호; 김현진; 이다영; 이재영; 이정근
Original assignee: 삼성전자 주식회사
Priority date: 2020-04-01
Filing date: 2021-02-17
Publication date: 2021-10-07
Also published as: KR102693431B1; KR20210122568A

Abstract

Various embodiments of the present disclosure relate to an electronic device and a method for controlling an audio output of same. The electronic device may comprise: a first speaker for outputting a sound signal; a first microphone; a memory for storing at least one wakeup word; and a processor operatively connected to the first speaker, the first microphone, and the memory, wherein the processor is configured to: acquire an external sound through the first microphone while an audio signal is output through the first speaker; identify whether the at least one wakeup word is detected from the acquired external sound; in response to the detection of the at least one wakeup word, perform recognition of a speaking person of the external sound acquired through the first microphone; identify whether the recognized speaking person corresponds to the detected wakeup word; and when the detected wakeup word corresponds to the recognized speaking person, control an output of the audio signal so that a user of the electronic device can recognize an external sound. Various other embodiments are possible.

Description

How to control an electronic device and its audio output

Various embodiments of the present disclosure relate to an electronic device and a method for controlling an audio output thereof.

Portable electronic devices (eg, headsets, earphones, smart glasses, head mounted devices: HMDs) that are worn on the user's body (eg, ears) and can output audio signals is being disseminated. Along with the popularization of portable electronic devices, users who use various services (eg, music listening, video viewing, and voice calls) using portable electronic devices are increasing. For example, a user may connect a portable electronic device to an external electronic device (eg, a smart phone, tablet PC, notebook, desktop, or wearable terminal) by wire or wirelessly to provide various services at any time in daily life. (eg, listening to music, watching videos, making voice calls) can be used conveniently.

When a user wears a portable electronic device, the output of the electronic device (eg, a speaker) is located inside or around the user's ear (eg, in-ear type), or covers the ear (eg, in-ear type). over-ear or closed type). Accordingly, when listening to music using the portable electronic device, the user may not be able to hear an external sound.

Also, in recent years, portable electronic devices having a sound insulation (eg, noise canceling) and/or shielding function for blocking external sounds by hardware or software have been provided. Accordingly, a user wearing the portable electronic device may hear little or no external sound.

In addition, as the portable electronic device is miniaturized, the portable electronic device worn by the user may not be exposed to others. Accordingly, the user may be misunderstood as ignoring his/her call by another person even though he/she does not hear the call of another person due to wearing the portable electronic device.

According to various embodiments of the present disclosure, an external sound is acquired through an input device (eg, a microphone) while using an electronic device (eg, listening to music, watching a video), and when a designated call word is detected from the external sound, the user can use the external sound Provided are an electronic device capable of controlling (or limiting) an audio output so that users can hear a sound, and a method of controlling the audio output thereof.

In addition, various embodiments of the present disclosure control an electronic device capable of reducing unnecessary audio output control in response to an external sound calling a user (other) other than the user by setting a call word for each speaker and controlling the audio output thereof can provide a way to

In addition, various embodiments of the present disclosure provide an electronic device capable of providing information about a direction in which an external sound is generated to a user when a designated call word is detected from an external sound while using the electronic device, and a method of controlling an audio output thereof. can

In addition, various embodiments of the present disclosure provide feedback (notification) to the user that a designated call word has been detected from an external sound and/or feedback (notifying) to the caller (speaker) who called the user that the user can hear the external sound. It is possible to provide an electronic device capable of controlling an audio output thereof and a method for controlling an audio output thereof.

In addition, various embodiments of the present disclosure provide an electronic device capable of controlling an audio output according to whether a specified call word is detected from an external sound when the volume is above a specified level and/or when wearing of the electronic device is detected, and an audio output thereof can provide a way to control

In addition, various embodiments of the present disclosure provide an electronic device capable of adaptively changing a designated call word based on context information (eg, location, time, movement status) of the electronic device, and a method of controlling an audio output thereof. can

An electronic device according to various embodiments of the present disclosure may include, for example, a first speaker outputting a sound signal; a first microphone; a memory for storing at least one call word; and a processor operatively connected to the first speaker, the first microphone, and the memory, wherein the processor acquires an external sound through the first microphone while outputting an audio signal through the first speaker and check whether the at least one call word is detected from the acquired external sound, and perform speaker recognition on the external sound acquired through the first microphone in response to the detection of the at least one call word, and Check whether the recognized speaker and the detected call word match, and when the detected call word and the recognized speaker match, control the output of the audio signal so that the user of the electronic device can recognize an external sound can be

A method of controlling an audio output of an electronic device according to various embodiments of the present disclosure may include, for example, outputting an audio signal through a plurality of speakers; acquiring an external sound through at least one microphone while outputting the audio signal; checking whether at least one designated call word is detected from the acquired external sound; performing speaker recognition on an external sound corresponding to the at least one call word in response to the detection of the at least one call word; checking whether the recognized speaker and the detected call word match; and controlling the output of the audio signal so that the user of the electronic device can recognize an external sound when the detected call word and the recognized speaker match.

An electronic device according to various embodiments of the present disclosure may include, for example, a housing; a first speaker disposed in a first area of the housing and configured to output a left sound signal; a second speaker disposed in the second area of the housing and configured to output a right sound signal; at least one microphone included in the housing; a memory for storing at least one call word set for each speaker; and a processor operatively connected to the first speaker, the second speaker, the at least one microphone, and the memory, wherein the processor outputs an audio signal through the first speaker and the second speaker; , acquiring an external sound through the at least one microphone while outputting the audio signal, confirming whether the at least one call word is detected from the acquired external sound, and corresponding to the detection of the at least one call word, The user of the electronic device performs speaker recognition on an external sound corresponding to the call word, checks whether the recognized speaker and the detected call word match, and when the detected call word and the recognized speaker match may be set to control the output of the audio signal to recognize an external sound.

According to various embodiments of the present disclosure, while wearing an electronic device capable of outputting an audio signal (eg, a headset, earphone, smart glasses, or head-wearable device), a user can easily recognize a call from another person, , the electronic device can automatically control the audio output so that external sounds can be heard, so that the caller can can have a conversation with

In addition, various embodiments of the present disclosure may improve the inconvenience of unnecessarily controlling (limiting) audio output by misrecognizing another's call as a user's call by setting a different call word for each speaker.

In addition, various embodiments of the present disclosure provide direction information in which an external sound calling a user is generated so that the user can easily recognize the location (direction) of the caller.

In addition, various embodiments of the present disclosure may feed back to the user that a call word calling the user is sensed in various ways (visual, auditory, tactile) to easily recognize that the user has been called by another person.

In addition, various embodiments of the present disclosure provide feedback to the caller that the user can hear an external sound, thereby resolving the inconvenience of the caller unnecessarily repeatedly calling the user.

In addition, various embodiments of the present disclosure may reduce current consumption of the electronic device by activating a function of acquiring an external sound when the volume is above a specified level and/or when the electronic device is worn, and detecting a call word.

In addition, various embodiments of the present disclosure may improve the efficiency (eg, speed and/or accuracy) of recognizing a call word by changing the call word for each situation and detecting or preferentially detecting the changed call word.

1 is a diagram illustrating an example in which an electronic device is connected to an external electronic device by wireless communication according to an embodiment of the present disclosure.

2A and 2B are diagrams illustrating an example of controlling an audio output of an electronic device according to an embodiment of the present disclosure;

3 is a flowchart illustrating a method of controlling an audio output between an electronic device and an external electronic device according to an embodiment of the present disclosure.

4A is a flowchart illustrating a method of controlling an audio output of an electronic device according to an embodiment of the present disclosure.

4B is a flowchart illustrating a method of controlling an audio output of an electronic device according to another embodiment of the present disclosure.

5 is a flowchart illustrating in detail an operation of controlling an audio output of an electronic device according to an embodiment of the present disclosure.

6A is a diagram illustrating an example in which an electronic device pauses audio output according to an embodiment of the present disclosure;

6B is a diagram illustrating an example in which an electronic device controls a volume of an audio signal and a volume of an external sound according to an embodiment of the present disclosure;

6C is a diagram illustrating an example in which an electronic device controls a volume of an audio signal according to an embodiment of the present disclosure;

7A is a flowchart illustrating a method in which an electronic device feeds back call origination and speaker direction information to a user according to an embodiment of the present disclosure;

7B is a diagram illustrating an example in which an electronic device feeds back call generation and speaker direction information to a user according to an embodiment of the present disclosure;

8A is a flowchart illustrating a method of feeding back, by an electronic device, that a conversation with a user is possible to a speaker, according to an embodiment of the present disclosure;

8B is a diagram illustrating an example in which the electronic device feeds back to the speaker that a conversation with the user is possible, according to an embodiment of the present disclosure.

9A is a flowchart illustrating a method of controlling audio output by changing a call word based on context information of an electronic device according to an embodiment of the present disclosure.

9B is a diagram illustrating an example of changing a call word based on context information of an electronic device according to an embodiment of the present disclosure.

10 is a flowchart illustrating a method of controlling an audio output of an electronic device according to an embodiment of the present disclosure.

11 is a block diagram of an electronic device and an external electronic device according to an embodiment of the present disclosure.

12 is a block diagram of an electronic device according to another embodiment of the present disclosure.

13A is an exemplary diagram illustrating a method of registering a call word for each speaker according to an embodiment of the present disclosure.

13B is an exemplary diagram illustrating a method of registering a call word for each speaker according to an embodiment of the present disclosure.

13C is an exemplary diagram illustrating a method of registering a call word for each speaker according to an embodiment of the present disclosure.

13D is a diagram illustrating an example in which a call word is set for each speaker according to an embodiment of the present disclosure.

14 is a block diagram of an electronic device in a network environment, according to an embodiment of the present disclosure.

Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. In the present disclosure, specific embodiments are illustrated in the drawings and the related detailed description is described, but this is not intended to limit the various embodiments of the present disclosure to a specific form. For example, it is apparent to those of ordinary skill in the art to which the present disclosure pertains that the embodiments of the present disclosure may be variously changed.

1 is a diagram illustrating an example in which an electronic device is connected to an external electronic device by wireless communication according to an embodiment of the present disclosure, and FIG. 2A is an example of controlling an audio output of the electronic device according to an embodiment of the present disclosure. FIG. 2B is a diagram illustrating an example of controlling an audio output of an electronic device according to an embodiment of the present disclosure.

1 to 2B , an electronic device 100 according to an embodiment of the present disclosure may be an audio output device capable of outputting an audio signal. For example, the electronic device 100 may be a hearable device that may be worn on a user's body and may output an audio signal. The electronic device 100 may include a first type of wireless earphone 101 , a second type of wireless earphone 102 , a headset 103 , a head mounted device (HMD) 104 , or a smart device. It may include smart glasses 105 . The first type of wireless earphone 101 may be connected to a pair of sub-devices (eg, output devices) wirelessly (eg, bluetooth low energy: BLE). The second type of wireless earphone 102 may be connected to a pair of output devices by wire. This is only an example and does not limit the electronic device 100 according to an embodiment of the present disclosure.

According to various embodiments, the electronic device 100 may establish a short-range wireless communication (eg, Bluetooth) channel with the external electronic device 200 . The electronic device 100 may receive and output audio data (eg, music data, audio data included in a video, and call reception data) from the external electronic device 200 . According to some embodiments, the electronic device 100 may output audio data stored in an internal memory.

According to various embodiments, the electronic device 100 acquires an external sound by activating at least one input device (eg, a microphone) while outputting an audio signal corresponding to the received audio data, and obtains an external sound from at least the acquired external sound. You can check if a single specified call word is detected. When at least one designated call word is detected from the external sound, the electronic device 100 may control (or limit) the audio output. For example, as shown in FIG. 2A , when the user 210 calls the user 210 while the user 210 is listening to music while wearing the electronic device 100 on his/her ear, the user 210 is called. Assuming a situation, in response to the output of the audio signal, the electronic device 100 receives

external sounds

230a, 230b, 230c, and 230d, 240), and whether the acquired

external sounds

230a, 230b, 230c, 230d, 240 contain a call word 240 (eg, "Yonghoya") referring to the user 210 can be checked When it is confirmed that the call word 240 is included (or when the call word 240 is detected), the electronic device 100 pauses ( 202), and the caller's voice 250 obtained through an input device (eg, a microphone) may be output through an output device (eg, a speaker). Referring to FIG. 2B , when the call word 240 is detected ( 211 ), the electronic device 100 may stop ( 212 ) playing music and open ( 213 ) the microphone. The opening 213 of the microphone may mean outputting an external sound collected through the microphone through a speaker.

According to an embodiment, the electronic device 100 performs at least one of operations (functions) of pausing playback of an audio signal, muting or decreasing a volume, or turning off a noise canceling function when a call word is detected. You can control the audio output by doing this. A detailed description thereof will be described later with reference to FIGS. 5 to 6C .

According to various embodiments of the present disclosure, when detecting a call word, the electronic device 100 performs speaker recognition based on an external sound including the call word (or corresponding to the call word), and the detected call word and the recognized call word. You can check if the speaker matches. To this end, the electronic device 100 may include (store) a speaker recognition database (DB) for speaker recognition and a call word DB in which the speaker and the calling word are matched. According to some embodiments, speaker recognition may be performed by the external electronic device 200 . For example, the electronic device 100 may transmit at least a part of the acquired external sound to the external electronic device 200 and receive a speaker recognition result. As another example, the electronic device 100 may transmit at least a part of the acquired external sound and the call word to the external electronic device 200 , and receive a result of whether the call word and the speaker match. When the detected call word and the recognized speaker match, the electronic device 100 may control (or limit) an audio output.

According to various embodiments, the electronic device 100 (eg, the HMD 104 or the smart glasses 105) including the display may display information (eg, name) about the recognized speaker on the display.

According to various embodiments of the present disclosure, upon detecting the call word, the electronic device 100 may recognize direction information in which an external sound including the call word is generated and provide the recognized direction information. For example, the electronic device 100 may use various known techniques (eg, an inter-aural time difference (ITD) technique using a time difference between sounds heard in two ears or an inter-aural level (ILD) technique using a difference in sound levels heard in both ears. difference) technology) to output an external sound through an output device to provide a sense of space corresponding to the recognized direction. A detailed description thereof will be described later with reference to FIGS. 7A and 7B .

According to various embodiments, the electronic device 100 may provide feedback (or notification) to the user and/or the speaker who called the user upon detecting the call word. For example, the electronic device 100 may provide feedback to the user that a designated call word is detected from an external sound in various ways. For example, the electronic device 100 outputs a visual (eg, displaying a specified message and/or icon on the display of the smart glasses, providing a specified effect (eg, blinking)), and auditory (eg, a specified sound effect through a speaker). ), and/or tactilely (eg, generating vibrations in a specified pattern). Also, when a designated call word is sensed from an external sound, the electronic device 100 may give feedback (notify) to the speaker who called the user that the user can hear the external sound in various ways. For example, the electronic device 100 may be visually (eg, light an LED included in the electronic device 100 , display an icon on the display of the electronic device 100 ) or auditory (eg, a speaker of the electronic device 100 ). It is possible to provide feedback by outputting the specified sound effect to the outside rather than the inside of the user's ear). A detailed description thereof will be described later with reference to FIGS. 8A and 8B .

According to various embodiments of the present disclosure, in order to reduce battery consumption, the electronic device 100 does not continuously activate at least one input device (eg, a microphone) while outputting an audio signal, and when the volume is greater than or equal to a specified value An external sound may be acquired by activating at least one input device (eg, a microphone). The specified value may be set by a manufacturer or a user as a value capable of recognizing an external sound while the user wears the electronic device 100 . According to an embodiment, in order to reduce battery consumption, the electronic device 100 detects wearing on the user's body while outputting the audio signal, or requests output of the audio signal while being worn on the user's body. In this case, the external sound may be acquired by activating at least one input device (eg, a microphone). The electronic device 100 may include at least one sensor (eg, a temperature sensor for detecting body temperature, a contact sensor) for detecting whether the device is worn or not.

According to various embodiments, the electronic device 100 may adaptively change the call word based on context information. For example, the electronic device 100 may adaptively change the call word based on information (eg, location information, time and/or motion information) acquired through at least one sensor. According to an embodiment, the context information is obtained by the external electronic device 200 , and the electronic device 100 may receive the context information from the external electronic device 200 . A detailed description thereof will be described later with reference to FIGS. 9A and 9B .

According to various embodiments of the present disclosure, the electronic device 100 may determine whether a keyword for executing a specific function other than a call word is detected from the acquired external sound. For example, the electronic device 100 may check whether a keyword for executing a voice recognition function such as ^{“Bixby TM” is detected while an audio signal is being output.} A detailed description thereof will be described later with reference to FIG. 10 .

According to various embodiments, the external electronic device 200 may establish a short-range wireless communication (eg, Bluetooth) channel with the electronic device 100 . For example, the external electronic device 200 may include a smart phone, a tablet PC, a notebook computer, a desktop computer, and a wearable electronic device (eg, a smart watch).

According to various embodiments, the external electronic device 200 may reproduce multimedia data (eg, an audio (eg, music) file or a moving picture file) stored in a memory (not shown) or received from an external server. When connected to the electronic device 100 , the external electronic device 200 may transmit audio data included in multimedia data being played back to the electronic device 100 . Alternatively, when the external electronic device 200 performs a call with another external device while wireless communication is connected to the electronic device 100 , the external electronic device 200 may transmit audio data corresponding to a handset sound to the electronic device 100 .

According to various embodiments, the external electronic device 200 acquires an external sound through at least one input device (eg, a microphone) while transmitting audio data to the electronic device 100 , and analyzes the acquired external sound. It can be checked whether the specified call word is included in the acquired external sound. When the call word is included, the external electronic device 200 may transmit the call word to the electronic device 100 . According to an embodiment, the external electronic device 200 may control (eg, pause) an audio output upon detecting the call word and notify the electronic device 100 of the call word detection.

According to an embodiment, the external electronic device 200 recognizes an external situation using at least one sensor (eg, an illuminance sensor, an image sensor, a microphone), and when it is determined that the external sound can be recognized sound can be obtained. For example, when it is determined that the external electronic device 200 is located in an enclosed space such as a bag or pocket through at least one sensor, at least one input device (eg, a microphone) for acquiring an external sound is deactivated. can do. According to an embodiment, the external electronic device 200 may periodically activate a microphone to determine whether an external sound is detected, and determine whether the external sound is sensed (state).

According to various embodiments, the external electronic device 200 may provide a user interface (UI) for registering a call word. A detailed description thereof will be described later with reference to FIGS. 13A to 13D .

Meanwhile, in FIGS. 1 and 2B , the electronic device 100 receives audio data from the external electronic device 200 through the wireless communication module and acquires an external sound while playing it and confirms whether a designated call word is detected. , the electronic device 100 may obtain an external sound while playing audio data (eg, a music file) stored in an internal memory (not shown) and check whether a designated call word is detected.

According to an embodiment, the electronic device 100 may control the output of the audio signal when at least one specified call word is detected a specified number of times (eg, three times) or more.

Referring to FIG. 3 , an electronic device (eg, the electronic device 100 ) and an external electronic device (eg, the external electronic device 200 ) according to an embodiment of the present disclosure perform short-range wireless communication (eg, in operation 301 ). : Bluetooth communication) channel can be connected.

According to various embodiments of the present disclosure, in operation 303 , the external electronic device may play a multimedia file. The multimedia file may include an audio file (eg, a music file) including audio data and a video file. According to some embodiments, the external electronic device may make a call with another external device.

According to various embodiments of the present disclosure, in operation 305 , the external electronic device may transmit audio data included in the multimedia file to the electronic device. According to an embodiment, the external electronic device may transmit audio data corresponding to a sign language received from the other external device to the electronic device.

According to various embodiments of the present disclosure, in operation 307 , the electronic device may output an audio signal corresponding to audio data. For example, the electronic device may output an audio signal through an output module (eg, a plurality of speakers). The output module may include a first speaker outputting a first acoustic signal (eg, a left acoustic signal) and a second speaker outputting a second acoustic signal (eg, a right acoustic signal).

According to various embodiments of the present disclosure, in operation 309 , the electronic device may acquire an external sound through at least one input device (eg, a microphone). For example, the electronic device may activate at least one microphone and acquire an external sound through the at least one microphone. According to an embodiment, the electronic device may acquire an external sound through at least one input device (eg, a microphone) when a function for limiting audio output through call word detection (hereinafter, call word detection function) is activated. have. According to an embodiment, when the electronic device separately includes a microphone for detecting an external sound (hereinafter, referred to as an external microphone) and a microphone for making a call (hereinafter referred to as a call microphone), the electronic device activates the external microphone and uses the external microphone. External sound can be obtained through

According to various embodiments of the present disclosure, in operation 311 , the electronic device may detect at least one designated call word from an external sound. For example, the electronic device may analyze the external sound to determine whether a designated call word is included in the external sound.

According to various embodiments of the present disclosure, in operation 313 , the electronic device may control the output of the audio signal. For example, the electronic device pauses playback of an audio signal, mutes or decreases the volume, turns off a noise canceling function, or outputs a caller's voice obtained through a microphone to an output device (eg, a speaker) At least one of the (functions) may be performed.

According to various embodiments of the present disclosure, in operations 315 and 317 , similarly to

operations

309 and 311 described above, the external electronic device acquires an external sound through an input device (microphone) included in the external electronic device, and receives the external sound from the external sound. At least one designated call word may be detected. In operation 319 , the external electronic device may notify the electronic device of detection of a call word. Operations 315 to 319 may be omitted. According to an embodiment, the external electronic device may recognize an external situation through at least one sensor, and perform operations 315 to 319 when it is determined that an external sound can be detected through a microphone.

Referring to FIG. 4A , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may output an audio signal in operation 401 . For example, the processor may output an audio signal corresponding to audio data received from an external electronic device (eg, the external electronic device 200) connected through the wireless communication module through a plurality of output devices (eg, a stereo speaker). can According to an embodiment, the processor may output an audio signal corresponding to audio data included in a multimedia file (eg, an audio file (eg, a music file) or a video file) stored in the internal memory through a plurality of output devices. . According to an embodiment, the processor may automatically or according to a user's request turn on (activate) a function (eg, a noise canceling function) capable of reducing or blocking an external sound when an audio signal is output.

The processor according to various embodiments may acquire an external sound in operation 403 . For example, the processor may activate at least one input device (eg, a microphone) in response to the output of the audio signal, and acquire an external sound through the activated at least one input device. According to an embodiment, the electronic device may determine whether the call word detection function is activated, and when the call word detection function is activated, the electronic device may acquire an external sound through at least one input device. According to an embodiment, when the electronic device separately includes a microphone (call microphone) for voice call and a microphone (external microphone) for acquiring an external sound, the processor may activate the external microphone. According to an embodiment, the processor activates an external microphone to acquire an external sound when the volume is greater than or equal to a specified value and/or when it is confirmed that the electronic device is worn on the user's body (eg, ear) through at least one sensor can do.

In operation 405 , the processor according to various embodiments may determine whether a designated call word is detected from an external sound. For example, the processor may analyze an external sound acquired through the input device to determine whether the external sound includes a designated call word.

As a result of checking operation 405, when a designated call word is not detected from an external sound, the processor may proceed to operation 411, which will be described later. On the other hand, when a designated call word is detected from an external sound as a result of checking in operation 405 , in operation 407 , the processor may control the output of the audio signal. For example, the processor may request the external electronic device to pause output of the audio signal, mute or decrease the volume of the audio signal, turn off the noise canceling function, or input An external sound input through the device may be output through an output device. A detailed description of various examples of controlling the output of an audio signal will be described later with reference to FIGS. 5 to 6C .

In operation 409 , the processor according to various embodiments may determine whether resume output of the audio signal is requested. For example, the processor may check whether the output of the audio signal is resumed by the user. According to an embodiment, the processor may detect whether the speaker's utterance corresponding to the call word has ended, and, if the utterance is terminated, may recognize that resumption of the output of the audio signal is requested. The electronic device may include a device (or software module) (eg, end point detection: EPD) for determining whether the utterance has ended.

As a result of checking the operation 409, when the resumption of the output of the audio signal is not requested, the processor may maintain operation 409 . On the other hand, when the resumption of the output is requested as a result of checking in operation 409 , in operation 411 , the processor may determine whether the output of the audio signal is terminated. For example, the processor may determine whether audio data is not received from the external electronic device for a specified period of time (eg, 0.5 seconds) or longer. Alternatively, the processor may check whether multimedia playback is paused, stopped, or ended.

As a result of checking operation 411, if output of the audio signal is not terminated, the processor may return to operation 403 . On the other hand, when the output of the audio signal is terminated as a result of checking in operation 411, the processor may deactivate the activated microphone in operation 403 and end the audio signal output control according to an embodiment of the present disclosure.

Referring to FIG. 4B , an electronic device (eg, the electronic device 100 ) according to another embodiment of the present disclosure may support call word recognition for each speaker.

Operations

431 , 433 , and 435 of FIG. 4B are similar to

operations

401 , 403 , and 405 of FIG. 4A , respectively, and thus a detailed description thereof will be omitted.

In operation 437 , the processor according to various embodiments of the present disclosure may perform speaker recognition for an external sound (hereinafter, referred to as an utterance) corresponding to the calling word. The electronic device may include a speaker recognition DB. According to an embodiment, the electronic device may transmit at least a portion of the utterance to the external electronic device and receive speaker information from the external electronic device.

In operation 439 , the processor according to various embodiments may determine whether the detected call word and the recognized speaker match. For example, the processor may refer to the call word DB and check whether the recognized speaker is "wife" when the call word "honey" is detected. A detailed description of the call word DB will be described later with reference to FIG. 11 .

As a result of checking operation 439, if the detected call word and the recognized speaker do not match, the processor may proceed to operation 445 to be described later. On the other hand, if the detected call word and the recognized speaker match as a result of checking the operation 439, the processor may perform operation 441 .

Operations

441, 443, and 445 of FIG. 4B are similar to

operations

407, 409, and 411 of FIG. 4A, respectively, and thus a detailed description thereof will be omitted.

According to another embodiment of the present disclosure described above, it is possible to improve the efficiency of audio output control by setting a call word for each speaker. For example, another embodiment of the present disclosure may reduce the inconvenience of unnecessarily controlling audio output to an external sound calling a user other than the user.

5 is a flowchart illustrating in detail an operation of controlling an audio output of an electronic device according to an embodiment of the present disclosure, and FIG. 6A illustrates an example in which the electronic device pauses an audio output according to an embodiment of the present disclosure 6B is a diagram illustrating an example in which an electronic device controls a volume of an audio signal and a volume of an external sound according to an embodiment of the present disclosure, and FIG. 6C is an electronic device according to an embodiment of the present disclosure is a diagram illustrating an example of controlling the volume of an audio signal.

5 to 6C , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may determine whether music is being played in operation 501 . As a result of checking in operation 501, if music is being played, in operation 503, the processor may pause music playback or control the volume. For example, when music stored in the electronic device is being played, the processor may instruct the music playback application to pause or control the volume (eg, mute or decrease). As another example, when receiving music from the external electronic device and playing the music, the processor transmits a message requesting to pause the music when a call word 601 is detected to the external electronic device as shown in FIG. 6A ( 602 ) can do. Upon receiving the message, the external electronic device may pause (603) playing the music.

According to some embodiments, the processor may simultaneously output an audio signal and an external signal. For example, as shown in the section of the identification code 607 of FIG. 6B , the processor increases the volume (eg, music volume) of the audio signal when the call word 601 is detected to a specified first value (V ₁ ). It may gradually decrease and gradually increase the volume of the external signal (eg, the utterance 605 ) up to the specified second value V ₂ . As such, as the volume of the external signal is output relatively high, the user On the other hand, when the end of the utterance 605 corresponding to the call word 601 is detected, as shown in section 609 of the identification code of Fig. 6B, the processor adjusts the volume of the audio signal. It is possible to gradually increase to the original state, and to gradually cut off the output of external sound.

According to some embodiments, the processor may control audio output of some of the plurality of output devices. For example, as shown in FIG. 6C , when the electronic device includes a first sub device (eg, a left acoustic device) 611 and a second sub device (eg, a right acoustic device) 612 , the

processor External signals

611a and 612a are respectively obtained through the input device (microphone) included in the first sub device 611 and the input device (microphone) included in the second sub device 612 , and It is possible to compare the intensity and control (eg, mute or decrease the volume) the audio output to the sub-device (eg, the first sub-device 611 in FIG. 6B ) from which the external signal 611a having a large strength is obtained. have. This is because the external sound acquired by the input device (microphone) in the direction in which the speaker is located has a relatively large intensity. According to an embodiment, the processor may control only the audio output to the sub-device in the direction in which the speaker is located so that the user may recognize an external sound.

As a result of checking in operation 501, if music is not being played, in operation 505, the processor may determine whether a current media volume (volume) is equal to or greater than a specified threshold. As a result of checking in operation 505, if the media volume is equal to or greater than the threshold, in step 507, the processor may decrease or mute the media volume to a specified volume.

As a result of checking in operation 505, if the media volume is not equal to or greater than the threshold, in operation 509, the processor may determine whether the noise canceling function is activated. As a result of checking the operation 509 , when the noise canceling function is activated, the processor may deactivate the noise canceling function in operation 511 .

As a result of checking the operation 511 , when the noise canceling function is deactivated, the processor may determine whether a call is in progress in operation 513 . As a result of checking in operation 513 , if a call is in progress, the processor may provide feedback in operation 515 . For example, the processor may feedback to the user that an external call has been detected in various ways. For example, the processor may output a specified sound effect through a speaker. As another example, when the electronic device is a smart glasses or an HMD device, the processor may display a phrase and/or an icon informing the detection of a call word on the display.

As a result of checking operation 513 , if the processor is not in a call, the processor may return to operation 411 (or operation 445 ).

5 is only an example, and an embodiment of the present disclosure is not limited to controlling the audio output in the order of operation 501, operation 505, operation 509, and operation 513 as shown in FIG. 5, and the operation order may be changed have. According to an embodiment, the electronic device may provide a combination of at least one of stopping or controlling the volume of an audio signal, reducing or muting the volume of a media signal, turning off a noise canceling function, or providing feedback. E.g,

7A is a flowchart illustrating a method in which an electronic device feeds back call origination and speaker direction information to a user according to an embodiment of the present disclosure, and FIG. 7B is a flowchart illustrating an electronic device to the user according to an embodiment of the present disclosure. It is a diagram illustrating an example of feeding back information about the origin of a call and the direction of the speaker.

7A and 7B , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may provide direction information in which the speaker is located.

Operations

701, 703, 705, 707, and 709 of FIG. 7A are similar to

operations

431, 433, 435, 437, and 439 of FIG. 4B, respectively, and detailed descriptions thereof will be omitted.

The processor according to various embodiments may recognize a direction in which the speaker is located in operation 711 . The processor may recognize the direction in which the speaker is located by analyzing external sounds acquired through a plurality of input devices (eg, using a magnitude comparison or a reception time difference). For example, when an external sound is detected through the input module located on the right side of the user and little external sound is detected through the input module located on the left side of the user, the processor may determine that the speaker is located on the right side . A technique for recognizing a speaker's position (direction) by analyzing an external sound acquired through the plurality of microphones is a well-known technique and is obvious to those skilled in the art, and thus a detailed description thereof will be omitted.

In operation 713 , the processor according to various embodiments may feed back a call generation and/or a call direction to the user. For example, the processor may output a designated sound effect through the output module. As another example, the processor may display a phrase 722 notifying the detection of the call word 721 on the display of the smart glasses 700 as shown in the diagram of the identification code 720 of FIG. 7B . As another example, in the processor, as shown in the diagram of the identification code 730 of FIG. 7B , the speaker 732 utters in the diagonal direction (about 45 degree direction) 733 at the upper right side with respect to the user 731 and A sense of space may be provided so that the user can recognize that there is. The processor may provide a sense of space corresponding to the recognized direction through various known techniques (eg, an ITD technique or an ILD technique).

Operations

715, 717, and 719 of FIG. 7A are similar to

operations

441, 443, and 435 of FIG. 4B, respectively, and thus a detailed description thereof will be omitted.

According to an embodiment of the present disclosure described above, the user can easily recognize the location of the speaker by detecting the call word and/or providing direction information where the speaker is located.

8A is a flowchart illustrating a method for an electronic device to provide feedback to a speaker that a conversation with a user is possible according to an embodiment of the present disclosure, and FIG. 8B is a flowchart illustrating an electronic device capable of communicating with a user It is a diagram illustrating an example of feeding back to the speaker.

8A and 8B , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may feed back to the caller that the user is in a conversationable state.

Operations

801, 803, 805, 807, 809, and 811 of FIG. 8A are similar to

operations

431, 433, 435, 437, 439 and 441 of FIG. 4B, respectively. to be omitted.

In operation 813 , the processor according to various embodiments may feed back to the caller that the user is in a conversationable state in various ways. For example, as shown in FIG. 8B , the processor emits a light emitted diode (LED) disposed on the outside of the electronic device in a specified pattern ( 821 ), and displays a dialog icon on the display ( 823 ). , it is possible to feed back to the caller that the user can hear the external sound through at least one of the output 825 of the specified sound effect.

Operations

815 and 817 of FIG. 8A are similar to

operations

443 and 435 of FIG. 4B , respectively, and thus detailed descriptions thereof will be omitted.

According to one embodiment of the present disclosure described above, the caller repeatedly calls the user because it is misunderstood as a state in which it is difficult to hear the external sound even though the user can hear the external sound by feeding back to the caller whether the user is in a conversational state. Or, you can relieve the discomfort of raising your voice.

9A is a flowchart illustrating a method of controlling audio output by changing a call word based on context information of an electronic device according to an embodiment of the present disclosure, and FIG. 9B is a flowchart illustrating an electronic device according to an embodiment of the present disclosure; It is a diagram illustrating an example of changing a call word based on context information.

Referring to FIGS. 9A and 9B , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may detect activation of a call word detection function in operation 901 . The call word detection function may be activated by a user through an input device (eg, a specified touch input or a specified key input), or may be automatically activated by a processor when an audio signal is output.

The processor according to various embodiments may collect context information in operation 903 . For example, the processor continuously or periodically collects contextual information about location (place), time, and/or movement (movement) while the call word detection function is active (or while an audio signal is output) can do. For example, the processor receives location information through a location information receiving module (eg, GPS, GNNS), through whether it is connected to a wireless router (eg, access point: AP) installed in a specific place (eg, home, company, department store). Information on whether the electronic device moves (moves) through place information and/or a sensor (eg, a gyro sensor) may be collected. According to an exemplary embodiment, the processor may receive context information from an external electronic device connected through short-range wireless communication.

In operation 905 , the processor according to various embodiments may determine whether a call word matched with the current context information exists. As a result of checking operation 905, if there is no call word matching the current context information, the processor may proceed to operation 909 . On the other hand, if it is determined in operation 905 that there is a call word matching the current context information, the processor may change the call word in operation 907 . For example, as shown in the diagram of identification code 931 of FIG. 9B , the processor may determine whether the current location (place) is identified as the home based on whether the electronic device is connected to the wireless router installed in the home or received location information. In this case, you can change the call word to "honey". As another example, the processor may change the call word to “excuse me” when it is confirmed that the electronic device is currently moving through the gyro sensor, as shown in the drawing of reference numeral 933 of FIG. 9B . As another example, the processor calls when it is confirmed that the current location is the company based on whether the electronic device and the wireless router installed in the company are connected or the received location information, as shown in the drawing of the identification code 935 of FIG. 9B , You can change the language to "Mr. Kim". As another example, the processor, as shown in the diagram of identification code 937 of FIG. 9B, based on whether the electronic device is connected to a wireless router installed in a department store, whether a BLE connection is made with a device installed in the department store, or received location information If it turns out that your current location is a department store, you can change the caller to "hello, excuse me, or may I help you."

According to an embodiment, the processor may give weight to the call word for each situation so as to preferentially check whether the call word corresponding to the current situation is included in the external sound.

In operation 909 , the processor according to various embodiments may determine whether the call word detection function is deactivated. If the call word detection function is not deactivated as a result of checking operation 909 , the processor may return to operation 903 . On the other hand, when the call word detection function is deactivated as a result of checking operation 909, the processor may end the procedure for adaptively changing the call word based on the context information according to an embodiment of the present disclosure.

Although not shown in FIG. 9A, after changing the call word, the processor acquires an external sound when outputting an audio signal, similar to that described in FIGS. 4A to 8B, and determines whether the changed call word is detected from the external sound. Based on the above, operations for controlling the output of the audio signal may be performed.

One embodiment of the present disclosure described above improves the efficiency (eg, speed and/or accuracy) of call word recognition by changing the call word for each context and detecting or preferentially detecting the call word matched (suitable) to the context. can do it

Referring to FIG. 10 , the processor of the electronic device (eg, the electronic device 100 ) according to an embodiment of the present disclosure may output an audio signal in operation 1001 and acquire an external sound in operation 1003 . Since

operations

1001 and 1003 are similar to

operations

401 and 403 of FIG. 4A , detailed descriptions thereof will be omitted.

According to various embodiments of the present disclosure, in operation 1005 , the processor may determine whether a specified keyword is detected from an external sound. If the keyword is not detected from the external sound as a result of checking operation 1005 , the processor may proceed to operation 1015 . On the other hand, when a keyword is detected from an external sound as a result of checking in operation 1005 , in operation 1007 , the processor may determine whether the detected keyword is a calling word calling the user.

When the detected keyword does not correspond to the call word as a result of checking in operation 1007 , in operation 1009 , the processor may execute a function corresponding to the keyword. For example, when the detected keyword corresponds to a voice command for controlling a specific function (eg, stopping music reproduction), the processor may perform a function corresponding to the voice command. According to an embodiment, the processor may transmit the detected keyword to the external electronic device. For example, when the detected keyword is ^{a keyword that calls a voice recognition service (eg, "Bixby TM} "), the external electronic device transmits the keyword and the external sound collected through the input device to the external electronic device, A recognition service can be performed.

As a result of checking operation 1007, if the detected keyword corresponds to the call word, the processor may perform

operations

1011, 1013, and 1015.

Operations

1011, 1013, and 1015 are similar to

operations

407, 409, and 411, and detailed descriptions thereof will be omitted.

11 is a block diagram illustrating the configuration of an electronic device and an external electronic device according to an embodiment of the present disclosure.

Referring to FIG. 11 , an electronic device 1101 (eg, the electronic device 100 ) according to an embodiment of the present disclosure includes a processor 1110 , a memory 1120 , a wireless communication module 1130 , and an output module 1140 . ), an input module 1150 , and a wear detection sensor 1160 .

According to various embodiments, the processor 1110 may control the audio output of the electronic device 1101 . For example, as described with reference to FIGS. 4A to 10 , the processor 1110 may control output of an audio signal when a designated call word is detected from an external sound while outputting the audio signal. The processor 1110 may include a call word detection module 1111 and an audio output control module 1113 .

According to various embodiments, the call word detection module 1111 detects a designated call word from the external sound by analyzing an external sound acquired through the input module 1150 while outputting an audio signal through the output module 1140 . can do. According to various embodiments, the call word detection module 1111 may perform speaker recognition and check whether the detected call word and the recognized speaker match. According to some embodiments, the processor 1110 may separately include a speaker recognition module.

According to various embodiments, the audio output control module 1113 may control the output of the audio signal in response to the call word detection module 1111 detecting the call word. For example, the audio output control module 1113 may control the output of the audio signal as described with reference to FIGS. 5 to 6C .

According to various embodiments, the memory 1120, when executed, may store instructions for controlling the output of the audio signal when the processor 1110 detects a designated call word while outputting the audio signal. According to various embodiments, the memory 1120 may include a calling word DB 1121 and a speaker recognition DB 1123 .

According to various embodiments, the call word DB 1121 may include at least one call word. According to an embodiment, in the call word DB, a speaker and at least one call word may be matched as shown in <Table 1> below, and situation information and at least one call word may be matched as shown in <Table 2>.

화자speaker	호출어call word
아내(wife)wife	여보(honey), 자기야(darling)Honey, darling
아들(son)son	아빠(papa), 아버지(father)papa, father
손님(guest)guest	XXX 씨(Mr. XXX)XXX (Mr. XXX)
친구(friend)friend	XXX 야(XXX)XXX hey (XXX)

상황 정보situation information	호출어call word
집(home)home	여보(honey), 자기야(darling), 아빠(papa), 아버지(father)Honey, darling, papa, father
사무실(office)office	XXX 씨(Mr. XXX)XXX (Mr. XXX)
가게(store)store	고객님("hello", "excuse me", "may I help you")Customer ("hello", "excuse me", "may I help you")
이동 중(moving)moving	"저기요(excuse me)""Excuse me"

<Table 1> and <Table 2> are only examples, and do not limit one embodiment of the present disclosure.

According to various embodiments, the speaker recognition DB 1123 may include reference data for recognizing a speaker corresponding to the detected call word. According to some embodiments, the memory 1120 of the electronic device 1101 may not include the speaker recognition DB 1123 . In this case, the electronic device 1101 may request speaker recognition to the external electronic device 1102 (eg, the external electronic device 200) through the wireless communication module 1130 and receive a result.

According to various embodiments, the wireless communication module 1130 may connect the external electronic device 1102 and a wireless communication channel (eg, Bluetooth). The wireless communication module 1130 may receive audio data from the external electronic device 1102 . The wireless communication module 1130 may transmit a signal requesting control of audio output (eg, pause of audio reproduction) to the external electronic device 1102 .

According to various embodiments, the output module 1140 may output an audio signal. The output module 1140 may be a speaker. The output module 1140 may include a first output module 1141 and a second output module 1142 to support stereo output. For example, the first output module 1141 may be a first speaker that outputs a first sound signal (eg, a left audio signal), and the second output module 1142 may output a second sound signal (eg, a right audio signal) signal) may be output from the second speaker.

According to various embodiments, the input module 1150 may acquire an external sound. For example, the input module 1150 may acquire an external sound while an audio signal is output through the output module 1140 . The input module 1150 may include a first input module (first microphone) 1151 and a second input module (second microphone) 1152 . According to some embodiments, the input module 1150 may include only one input module (eg, the first input module 1151 ) or may include three or more input modules.

According to various embodiments, the wear detection sensor 1160 may detect whether the electronic device 1101 is worn on the user's body (eg, ears). The wear detection sensor 1160 may include a temperature sensor that detects temperature or a touch sensor that detects contact. For example, the processor 1110 may recognize that the electronic device 1101 is worn on the body when a temperature (eg, 30 to 45 degrees) corresponding to the body temperature is sensed through the temperature sensor. As another example, the processor 1110 may recognize that the electronic device 1101 is worn on the body when a contact is detected through a contact sensor disposed at a position in contact with the body when the electronic device 1101 is worn. According to some embodiments, the electronic device 1101 may not include a wear detection sensor 1160 .

Although not shown, the electronic device 1101 may further include at least one module for collecting context information. For example, the electronic device 1101 may include a location information receiving module (eg, GPS, GNSS) for receiving location and time information, and a wireless router (eg, access point: It may further include a short-range wireless communication module (eg, Wi-Fi, BLE) connected to the AP), a gyro sensor for detecting movement (movement) of the electronic device, and the like.

The external electronic device 1102 (eg, the external electronic device 200 ) according to an embodiment of the present disclosure includes a processor 1170 , a memory 1180 , a wireless communication module 1190 , and an input module 1195 . can do.

According to various embodiments, the processor 1170 may control the overall operation of the external electronic device 1102 . For example, the processor 1170 may wirelessly communicate with the electronic device 1101 through the wireless communication module 1190 . The processor 1170 may transmit the audio data to the electronic device 1101 through the wireless communication module 1190 when playing a multimedia file including audio data in a state in which wireless communication is connected with the electronic device 1101 . The processor 1170 may include a call word detection module 1171 , an audio output control module 1173 , a call word registration module 1175 , and a control option setting module 1177 .

According to various embodiments, the call word detection module 1171 may detect a designated call word from an external sound acquired through the input module 1195 while transmitting audio data to the electronic device 1101 . The call word detection module 1171 may be the same as or similar to the call word detection module 1111 of the electronic device 1101 . According to some embodiments, the call word detection module 1171 may be omitted.

According to various embodiments, the audio output control module 1173 may control audio output. For example, the audio output control module 1173 may control (eg, pause) the audio output in a manner set by the control option setting module 1177 when the call word is detected, and detect the call word by the wireless communication module It can be transmitted to the electronic device 1101 through 1190 . According to some embodiments, the audio output control module 1173 may not be included in the external electronic device 1102 .

According to various embodiments, the call word registration module 1175 may control call word registration. For example, the call word registration module 1175 may register a voice received from the speaker as a call word and transmit the registered call word to the electronic device 1101 . A detailed description will be given later with reference to FIGS. 13A to 13D.

According to various embodiments, the control option setting module 1177 may set an audio output control method. For example, the control option setting module 1177 receives an operation (eg, pause, mute or decrease volume, or whether to provide feedback) to be performed by the electronic device 1101 when a call word is detected from the user, and receives the input from the user. It can be sent to (1101).

According to various embodiments, the control option setting module 1177 may obtain context information and transmit a call word matched to the context information to the electronic device 1101 . According to an embodiment, the control option setting module 1177 may transmit only context information to the electronic device 1101 .

According to various embodiments, the memory 1180 may include a call word DB 1181 and a speaker recognition DB 1183 . Since the call word DB 1181 and the speaker recognition DB 1183 are similar to the call word DB 1121 and the speaker recognition DB 1123 of the electronic device 1101 , a detailed description thereof will be omitted. According to some embodiments, the call word DB 1181 and the speaker recognition DB 1183 may not be included in the external electronic device 1102 .

According to various embodiments, the wireless communication module 1190 may connect the electronic device 1101 and a wireless communication channel (eg, Bluetooth). The wireless communication module 1190 may transmit audio data to the electronic device 1101 . The wireless communication module 1190 may receive a signal requesting control of audio output (eg, pause of audio reproduction) from the electronic device 1101 . According to some embodiments, the wireless communication module 1190 may transmit the detection of the call word to the electronic device 1101 . According to an embodiment, the wireless communication module 1190 may transmit context information to the electronic device 1101 .

According to various embodiments, the input module 1195 may acquire an external sound. For example, the input module 1195 may acquire an external sound while transmitting audio data to the electronic device 1101 . The input module 1195 may include at least one microphone. According to an embodiment, the input module 1195 is transmitting audio data to the electronic device 1101 and, in a situation in which an external sound can be acquired, may be activated to acquire an external sound. According to some embodiments, the input module 1195 may not be included in the external electronic device 1102 .

12 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present disclosure.

Referring to FIG. 12 , an electronic device 1201 (eg, the electronic device 100 ) according to another embodiment of the present disclosure may include a first sub-device 1201a and a second sub-device 1201b.

According to various embodiments, the first sub-device 1201a and the second sub-device 1201b may be wirelessly connected to support a stereo output. Each of the first sub-device 1201a and the second sub-device 1201b may operate as an independent device.

The first sub-device 1201a and the second sub-device 1201b may have the same or similar configuration to the electronic device 1101 of FIG. 11 . According to some embodiments, the second sub-device 1201b has some components overlapping with the first sub-device 1201a (eg, the second processor 1210b, the second memory 1220b, and the second input module 1250b). ), the wear detection sensor 1260b)) may not be included.

The sub-devices 1201a and 1201b of the electronic device 1201 described above may be wirelessly connected through the wireless communication modules 1230a and 1230b. According to various embodiments, the wireless communication module 1230a (a first wireless communication circuit) of the first sub-device 1201a connects a wireless communication channel with an external electronic device, and transmits an audio signal through the connected wireless communication channel. can receive The wireless communication module 1230b (second wireless communication circuit) of the second sub-device 1201b obtains (eg, sniffs) an audio signal transmitted from the external electronic device to the first wireless communication circuit. can When connected to the second sub-device 1201b, the first sub-device 1201a may provide information necessary for acquiring (sniffing) the audio signal to the second sub-device 1201b.

The electronic device 1201 is similar to the electronic device 1101 except that the two sub-devices 1201a and 1201b are wirelessly connected. Accordingly, the processors 1210a and 1210b of FIG. 12, the call word detection modules 1211a and 1211b, the audio output control modules 1213a and 1213b, the memories 1220a and 1220b, the call word DBs 1221a and 1221b, and the speaker recognition DB (1223a, 1223b), wireless communication module (1230a, 1230b), first output module (1240a), second output module (1240b), first input module (1250a), second input module (1250b), wear detection sensor A detailed description of (1260a, 1260b) will be omitted.

13A to 13D are exemplary views illustrating a method of registering a call word for each speaker according to an embodiment of the present disclosure.

13A to 13D , an external electronic device (eg, the external electronic devices 200 and 1102 ) according to an embodiment of the present disclosure may display a user interface for registering a call word on a display. For example, when executing a menu to register a call word, the external electronic device may display a call word registration UI on the display as shown in the reference numeral 1310 of FIG. 13A . The call word registration UI may include a speaker selection menu 1301 and a speaker addition menu 1302 for selecting one of speakers provided by default.

According to various embodiments of the present disclosure, when a speaker for registering a call word is selected, the external electronic device may display a screen requesting input of the call word on the display. For example, when a spouse is selected, the external electronic device may display on the display a screen requesting the spouse to input a call word by voice, as illustrated in

reference numerals

1310 and 1315 of FIG. 13A . As another example, the external electronic device may display, on the display, a screen requesting to input a call word with a male voice when a guest is selected, as shown in

reference numerals

1335 and 1340 of FIG. 13B . As another example, the external electronic device may display, on the display, a screen requesting the user to input a call word by voice when the user is selected, as shown in the

drawings

1355 and 1360 of FIG. 13C .

According to various embodiments, when a call word is input, the external electronic device may display a screen requesting to input the call word once again. In this case, the external electronic device may display the previously voiced call word on one area of the screen to induce the speaker to voice input the same call word as before. For example, the external electronic device may display, on the display, a screen requesting the spouse to re-enter the call word “honey” 1303 by voice, as shown in the diagram of identification code 1320 of FIG. 13A . As another example, the external electronic device may display on the display a screen requesting to re-enter the call word “Yonghoya” 1304 with a female voice, as shown in the figure of identification code 1345 of FIG. 13B . As another example, the external electronic device may display, on the display, a screen requesting the user to re-enter the call word “wait” 1305 by voice, as shown in reference numeral 1365 of FIG. 13C .

According to various embodiments, when registration of the call word is completed, the external electronic device may display a call word registration completion screen on the display. For example, the external electronic device may display a call word registration completion screen as shown in the drawing of the identification code 1325 of FIG. 13A , the drawing of the identification code 1350 of FIG. 13B , and the drawing of the identification code 1370 of FIG. 13C . have. The call word registration completion screen may include another call word registration menu 1306 for registering different keywords for the same speaker and another user registration menu 1307 for registering a call word for another speaker.

According to various embodiments of the present disclosure, at least one call word may be registered for each speaker through the call word registration method described above. For example, as shown in FIG. 13D , the user registers (sets) two call words (“Honey” and “Baby”) with respect to a spouse, and one call word (“ "Dad"), register two call words ("Abuji", "Dad") in relation to the second son, and three call words ("Yongpro-nim", "Yongho-nim", "Professional Dragon") can be registered.

In the above description, the call word is registered (set) through the external electronic device and transmitted to the electronic device. .

14 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;

Referring to FIG. 14 , in a network environment 1400 , an electronic device 1401 (eg, external electronic devices 200 and 1102 ) communicates with an electronic device 1402 through a first network 1498 (eg, a short-range wireless communication network). ), or may communicate with the electronic device 1404 or the server 1408 through the second network 1499 (eg, a remote wireless communication network). According to an embodiment, the electronic device 1401 may communicate with the electronic device 1404 through the server 1408 . According to an embodiment, the electronic device 1401 includes a processor 1420 (eg, processor 1270 ), a memory 1430 (eg, memory 1280 ), and an input module 1450 (eg, an input module 1295 ). )), sound output module 1455, display module 1460, audio module 1470, sensor module 1476, interface 1477, connection terminal 1478, haptic module 1479, camera module 1480 , a power management module 1488 , a battery 1489 , a communication module 1490 (eg, a wireless communication module 1290 ), a subscriber identification module 1496 , or an antenna module 1497 . In some embodiments, at least one of these components (eg, the connection terminal 1478 ) may be omitted or one or more other components may be added to the electronic device 1401 . In some embodiments, some of these components (eg, sensor module 1476 , camera module 1480 , or antenna module 1497 ) are integrated into one component (eg, display module 1460 ). can be

The processor 1420, for example, executes software (eg, a program 1440) to execute at least one other component (eg, hardware or software component) of the electronic device 1401 connected to the processor 1420. It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or computation, the processor 1420 converts commands or data received from other components (eg, the sensor module 1476 or the communication module 1490 ) to the volatile memory 1432 . may store the command or data stored in the volatile memory 1432 , and store the result data in the non-volatile memory 1434 . According to an embodiment, the processor 1420 includes the main processor 1421 (eg, a central processing unit or an application processor) or a secondary processor 1423 (eg, a graphics processing unit, a neural network processing unit ( a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 1401 includes the main processor 1421 and the sub-processor 1423 , the sub-processor 1423 uses less power than the main processor 1421 or is set to be specialized for a specified function. can The coprocessor 1423 may be implemented separately from or as part of the main processor 1421 .

The coprocessor 1423 may be, for example, on behalf of the main processor 1421 while the main processor 1421 is in an inactive (eg, sleep) state, or when the main processor 1421 is active (eg, executing an application). ), together with the main processor 1421, at least one of the components of the electronic device 1401 (eg, the display module 1460, the sensor module 1476, or the communication module 1490) It is possible to control at least some of the related functions or states. According to one embodiment, the coprocessor 1423 (eg, image signal processor or communication processor) may be implemented as part of another functionally related component (eg, camera module 1480 or communication module 1490). have. According to an embodiment, the auxiliary processor 1423 (eg, a neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 1401 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 1408). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The AI model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 1430 may store various data used by at least one component (eg, the processor 1420 or the sensor module 1476 ) of the electronic device 1401 . The data may include, for example, input data or output data for software (eg, a program 1440 ) and commands related thereto. The memory 1430 may include a volatile memory 1432 or a non-volatile memory 1434 .

The program 1440 may be stored as software in the memory 1430 , and may include, for example, an operating system 1442 , middleware 1444 , or an application 1446 .

The input module 1450 may receive a command or data to be used in a component (eg, the processor 1420 ) of the electronic device 1401 from the outside (eg, a user) of the electronic device 1401 . The input module 1450 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 1455 may output a sound signal to the outside of the electronic device 1401 . The sound output module 1455 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 1460 may visually provide information to the outside (eg, a user) of the electronic device 1401 . The display module 1460 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment, the display module 1460 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 1470 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 1470 acquires a sound through the input module 1450 or an external electronic device (eg, a sound output module 1455 ) directly or wirelessly connected to the electronic device 1401 . The electronic device 1402) (eg, a speaker or headphones) may output a sound.

The sensor module 1476 detects an operating state (eg, power or temperature) of the electronic device 1401 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to one embodiment, the sensor module 1476 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 1477 may support one or more specified protocols that may be used by the electronic device 1401 to directly or wirelessly connect with an external electronic device (eg, the electronic device 1402 ). According to an embodiment, the interface 1477 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 1478 may include a connector through which the electronic device 1401 can be physically connected to an external electronic device (eg, the electronic device 1402 ). According to an embodiment, the connection terminal 1478 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 1479 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 1479 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 1480 may capture still images and moving images. According to one embodiment, the camera module 1480 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 1488 may manage power supplied to the electronic device 1401 . According to an embodiment, the power management module 1488 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 1489 may supply power to at least one component of the electronic device 1401 . According to one embodiment, battery 1489 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 1490 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 1401 and an external electronic device (eg, the electronic device 1402, the electronic device 1404, or the server 1408). It can support establishment and communication through the established communication channel. The communication module 1490 may include one or more communication processors that operate independently of the processor 1420 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 1490 is a wireless communication module 1492 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 1494 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 1498 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 1499 (eg, legacy). It may communicate with the external electronic device 1404 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunication network such as a computer network (eg, LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 1492 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 1496 within a communication network, such as the first network 1498 or the second network 1499 . The electronic device 1401 may be identified or authenticated.

The wireless communication module 1492 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 1492 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 1492 uses various techniques for securing performance in a high frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 1492 may support various requirements specified in the electronic device 1401 , an external electronic device (eg, the electronic device 1404 ), or a network system (eg, the second network 1499 ). According to an embodiment, the wireless communication module 1492 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency (for URLLC realization) ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 1497 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 1497 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 1497 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 1498 or the second network 1499 is connected from the plurality of antennas by, for example, the communication module 1490 . can be selected. A signal or power may be transmitted or received between the communication module 1490 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 1497 .

According to various embodiments, the antenna module 1497 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 1401 and the external electronic device 1404 through the server 1408 connected to the second network 1499 . Each of the external electronic devices 1402 and 1404 may be the same as or different from the electronic device 1401 . According to an embodiment, all or part of the operations performed by the electronic device 1401 may be executed by one or more external

electronic devices

1402 , 1404 , or 1408 . For example, when the electronic device 1401 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 1401 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 1401 . The electronic device 1401 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 1401 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 1404 may include an Internet of things (IoT) device. The server 1408 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 1404 or the server 1408 may be included in the second network 1499 . The electronic device 1401 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

According to various embodiments of the present disclosure, an electronic device (eg, a sub device 1201a), a first speaker (eg, a first output module 1240a) for outputting a sound signal; a first microphone (eg, a first input module 1250a); a memory (eg, memory 1220a) for storing at least one call word; and a processor (eg, a processor 1210a) operatively connected to the first speaker, the first microphone, and the memory, wherein the processor is configured to output the audio signal through the first speaker. 1 Acquire an external sound through a microphone, check whether the at least one call word is detected from the acquired external sound, and respond to the detection of the at least one call word to the external sound obtained through the first microphone perform speaker recognition, check whether the recognized speaker and the detected call word match, and when the detected call word and the recognized speaker match, the user of the electronic device can recognize an external sound It can be set to control the output of the audio signal.

According to various embodiments, the electronic device further includes a wireless communication circuit (eg, a wireless communication module 1230a) that connects an external electronic device (eg, the second sub-device 1201b) and short-range wireless communication, and the processor outputs a first acoustic signal among the audio signals through the first speaker when the external electronic device and short-range wireless communication are connected, and outputs a second acoustic signal among the audio signals to a second speaker included in the external electronic device (eg : It may be set to control the external electronic device to output through the second output module 1240b).

According to various embodiments, the processor recognizes a direction in which the call word is generated when the call word is detected, and outputs the external sound through the first speaker and the second speaker to have a sense of space corresponding to the recognized direction can be set to

According to various embodiments, the processor receives an external sound acquired through a second microphone (eg, the second input module 1250b) included in the external electronic device through the wireless communication circuit, and the received external sound It may be configured to check whether the at least one call word is detected from

According to various embodiments of the present disclosure, in response to the at least one call word being detected through at least one of the first microphone and the second microphone, the processor may be configured to include an external sound acquired through the first microphone and the second microphone. Recognize a speaker's direction based on at least one of a difference in the size of the speakers or a reception time, and reduce or mute an output of an audio signal to the first speaker or the second speaker based on the recognized speaker's direction can be set to

According to various embodiments of the present disclosure, the electronic device may further include at least one module for collecting context information including at least one of a current location, a time, and whether to move. The processor may be configured to dynamically change the at least one call word based on the collected context information.

According to various embodiments, the processor may be configured to feed back to the recognized speaker that the user is in a state in which the external sound can be recognized, or to feed back to the user that the at least one call word has been detected.

According to various embodiments, the electronic device may further include at least one sensor (eg, a wear detection sensor 1260a) for detecting whether the electronic device is worn. The processor determines whether wearing of the electronic device is detected through the at least one sensor in response to the output of the audio signal, and when the wearing is detected, activates the first microphone to acquire the external sound can be

According to various embodiments of the present disclosure, an electronic device (eg, electronic device 1201) includes a first sub device (eg, first sub device 1201a) and a second sub device (eg, second sub device 1201b). )) may be included. The first sub device may include a first speaker (eg, a first output module 1240a) for outputting a first sound signal; a first microphone (eg, a first input module 1250a); a first wireless communication circuit (eg, a wireless communication module 1230a) for connecting the second sub-device and wireless communication; a memory (eg, memory 1220a) for storing at least one call word; and a processor (eg, a processor 1210a) operatively connected to the first speaker, the first microphone, the first wireless communication circuit, and the memory. The second sub device may include a second speaker (eg, a second output module 1240b) for outputting a second sound signal; a second microphone (eg, a second input module 1250b); and a second wireless communication circuit (eg, a wireless communication module 1230b) for connecting the first wireless communication circuit and wireless communication. The processor acquires an external sound through at least one of the first microphone or the second microphone while outputting an audio signal through the first speaker and the second speaker, and from the acquired external sound, the at least one Check whether a call word is detected, and, in response to the detection of the at least one call word, the direction of the speaker based on at least one of a difference in the size or reception time of external sounds acquired through the first microphone and the second microphone may be configured to reduce or mute the output of the audio signal to the first speaker or the second speaker based on the recognized direction of the speaker.

According to various embodiments, the processor may be set to control the output of the external sound to have a sense of space corresponding to the recognized direction of the speaker.

According to various embodiments, in response to the detection of the at least one call word, the processor performs speaker recognition on an external sound corresponding to the at least one call word, and the recognized speaker and the detected call word It may be configured to check whether a match is made, and to recognize the direction of the speaker when the detected call word and the recognized speaker match.

According to various embodiments of the present disclosure, a method for controlling an audio output of an electronic device (eg, the

electronic device

100 , 1101 , 1201 ) includes a plurality of speakers (eg, the first output module 1141 , 1240a ), a first 2 outputting an audio signal through the output modules 1142 and 1240b and the sound output device 1455); acquiring an external sound through at least one microphone (eg, first input modules 1151 and 1250a, second input modules 1152 and 1250b, and input device 1450) while outputting the audio signal; checking whether at least one designated call word is detected from the acquired external sound; performing speaker recognition on an external sound corresponding to the at least one call word in response to the detection of the at least one call word; checking whether the recognized speaker and the detected call word match; and controlling the output of the audio signal so that the user of the electronic device can recognize an external sound when the detected call word and the recognized speaker match.

According to various embodiments, the method may include: recognizing a direction in which the at least one call word is generated when the at least one call word is detected; and outputting the external sound through the plurality of speakers to have a sense of space corresponding to the recognized direction.

According to various embodiments of the present disclosure, the method may include feeding back to the recognized speaker that the user is in a state in which an external sound can be recognized; Alternatively, the method may further include at least one of feeding back to the user that the at least one call word has been detected.

According to various embodiments, the controlling of the output of the audio signal may include: pausing the reproduction of the audio signal; muting or reducing the volume of the audio signal to a specified level; an operation of turning off the noise canceling function; outputting the speaker's voice acquired through the at least one microphone through the plurality of speakers; comparing the levels of external sounds acquired through a plurality of microphones, and reducing or muting an audio output to a speaker disposed in the vicinity of the microphone having acquired an external sound having a relatively large size among the plurality of speakers; or decreasing the volume of the audio signal to a specified first level, increasing the volume of the external sound to a specified second level greater than the first level, and when the external sound is not detected for a specified time or longer, the audio It may include at least one of restoring the volume of the signal to the original level and blocking the output of the external sound.

According to various embodiments of the present disclosure, in the operation of acquiring the external sound, the wearing of the electronic device is detected through at least one sensor (eg, wearing detection sensors 1160, 1260a, 1260b) in response to the output of the audio signal. and activating the at least one microphone to acquire the external sound when the wearing is sensed. The performing of the speaker recognition may include performing the speaker recognition when the at least one call word is detected more than a specified number of times.

According to various embodiments, the method collects context information including at least one of the current location, time, or whether the user moves through at least one sensor (eg, the sensor module 1476) and/or wirelessly connected external electronic receiving context information from a device (eg, external electronic devices 200 and 1102 ); and dynamically changing the at least one call word based on the context information.

According to various embodiments of the present disclosure, an electronic device (eg, the

electronic devices

100 , 1101 , and 1201 ) may include a housing; a first speaker (eg, first output modules 1141 and 1240a) disposed in a first area of the housing and outputting a left sound signal; a second speaker (eg, second output modules 1142 and 1240b) disposed in the second area of the housing and outputting a right sound signal; at least one microphone included in the housing (eg, first input modules 1151 and 1250a, second input modules 1152 and 1250b, and input device 1450); a memory (eg, memories 1120, 1220a, 1220b, 1430) for storing at least one call word set for each speaker; and a processor (eg, processors 1110, 1210a, 1210b) operatively connected to the first speaker, the second speaker, the at least one microphone, and the memory. The processor outputs an audio signal through the first speaker and the second speaker, acquires an external sound through the at least one microphone while outputting the audio signal, and receives the at least one check whether a call word is detected, perform speaker recognition for an external sound corresponding to the call word in response to the detection of the at least one call word, and check whether the recognized speaker and the detected call word match; and when the detected call word and the recognized speaker match, the output of the audio signal may be controlled so that the user of the electronic device can recognize an external sound.

According to various embodiments, the processor is configured to recognize a direction in which the at least one call word is generated when the at least one call word is detected, and to control the output of the external sound to have a sense of space corresponding to the recognized direction can be

Electronic devices according to various embodiments disclosed in the present disclosure may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to an embodiment of the present disclosure is not limited to the above-described devices.

Various embodiments of the present disclosure and terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. In the present disclosure, "A or B", "at least one of A and B", "at least one of A or B," "A, B or C," "at least one of A, B and C," and "A , B, or C" each may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other components in question, and may refer to components in other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, for example, interchangeably with terms such as logic, logic block, component, or circuit. can be used A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present disclosure provide a storage medium (eg, a memory (eg, memory) readable by a machine (eg,

electronic devices

100, 1101, 1201, 1401, external electronic devices 200, 1102) 1120, 1180, 1220b), the internal memory 1436 or the external memory 1438) may be implemented as software (eg, a program 1440) including one or more instructions (eg, instructions) stored in the external memory 1438). For example, the processor (eg, the processor 1110 , 1170 , 1210b , 1420 ) of the device (eg, the

electronic device

100 , 1101 , 1201 , 1401 , or the external electronic device 200 or 1102 ) may store the device from the storage medium. Calling at least one of the stored one or more instructions, and executing it. This makes it possible for the device to be operated to perform at least one function according to the at least one command called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments of the present disclosure may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg, downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. . According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, omitted, or , or one or more other operations may be added.

Claims

In an electronic device,

a first speaker for outputting a sound signal;

a first microphone;

a memory for storing at least one call word; and

a processor operatively coupled to the first speaker, the first microphone, and the memory;

the processor

Acquire an external sound through the first microphone while outputting an audio signal through the first speaker,

Checking whether the at least one call word is detected from the acquired external sound,

In response to the detection of the at least one call word, performing speaker recognition for an external sound acquired through the first microphone,

Checking whether the recognized speaker and the detected call word match, and

an electronic device configured to control an output of the audio signal so that a user of the electronic device can recognize an external sound when the detected call word and the recognized speaker match.
The method of claim 1,

Further comprising a wireless communication circuit for connecting an external electronic device and short-range wireless communication,

the processor

outputting a first sound signal among the audio signals through the first speaker when the external electronic device and short-range wireless communication are connected,

An electronic device configured to control the external electronic device to output a second acoustic signal among the audio signals through a second speaker included in the external electronic device.
3. The method of claim 2,

the processor

The electronic device is configured to receive an external sound acquired through the second microphone included in the external electronic device through the wireless communication circuit, and to determine whether the at least one call word is detected from the received external sound.
4. The method of claim 3,

the processor

In response to the detection of the at least one call word through at least one of the first microphone and the second microphone, at least a difference in the amplitude or reception time of external sounds acquired through the first microphone and the second microphone recognizing the direction of the speaker based on one, and

An electronic device configured to provide a sense of space corresponding to the recognized direction of the speaker by controlling an output of an audio signal to the first speaker or the second speaker based on the recognized direction of the speaker.
The method of claim 1,

At least one module for collecting context information including at least one of a current location, time, or movement status,

the processor

an electronic device configured to dynamically change the at least one call word based on the collected context information.
The method of claim 1,

the processor

Feedback to the recognized speaker that the user is in a state in which the external sound can be recognized, or

an electronic device configured to feed back to the user that the at least one call word has been detected.
The method of claim 1,

At least one sensor for detecting whether the electronic device is worn,

the processor

In response to the output of the audio signal, it is confirmed whether wearing of the electronic device is detected through the at least one sensor,

an electronic device configured to activate the first microphone to acquire the external sound when the wearing is detected.
A method of controlling an audio output of an electronic device, the method comprising:

outputting an audio signal through a plurality of speakers;

acquiring an external sound through at least one microphone while outputting the audio signal;

checking whether at least one designated call word is detected from the acquired external sound;

performing speaker recognition on an external sound corresponding to the at least one call word in response to the detection of the at least one call word;

checking whether the recognized speaker and the detected call word match; and

and controlling an output of the audio signal so that a user of the electronic device can recognize an external sound when the detected call word and the recognized speaker match.
9. The method of claim 8,

recognizing a direction in which the at least one call word is generated when the at least one call word is detected; and

and outputting the external sound through the plurality of speakers to have a sense of space corresponding to the recognized direction.
9. The method of claim 8,

feedback to the recognized speaker that the user is in a state in which an external sound can be recognized; or

and at least one of feeding back to the user that the at least one call word has been detected.
9. The method of claim 8,

The operation of controlling the output of the audio signal is

pausing the reproduction of the audio signal;

muting or reducing the volume of the audio signal to a specified level;

an operation of turning off the noise canceling function;

outputting the speaker's voice acquired through the at least one microphone through the plurality of speakers;

comparing the levels of external sounds acquired through a plurality of microphones, and reducing or muting an audio output to a speaker disposed in the vicinity of the microphone having acquired an external sound having a relatively large size among the plurality of speakers; or

Decrease the volume of the audio signal to a specified first level, increase the volume of the external sound to a specified second level greater than the first level, and when the external sound is not detected for a specified time or longer, the audio signal A method comprising at least one of restoring the volume to the original level and blocking the output of the external sound.
9. The method of claim 8,

The operation of acquiring the external sound is

In response to the output of the audio signal, checking whether wearing of the electronic device is detected through at least one sensor, and when the wearing is detected, activating the at least one microphone to acquire the external sound, ,

The operation of performing the speaker recognition is

and performing the speaker recognition when the at least one call word is detected more than a specified number of times.
9. The method of claim 8,

collecting context information including at least one of a current location, time, and movement status through at least one sensor and/or receiving context information from an external electronic device connected via wireless communication; and

and dynamically changing the at least one call word based on the context information.
In an electronic device,

housing;

a first speaker disposed in a first area of the housing and configured to output a left sound signal;

a second speaker disposed in the second area of the housing and configured to output a right sound signal;

at least one microphone included in the housing;

a memory for storing at least one call word set for each speaker; and

a processor operatively coupled to the first speaker, the second speaker, the at least one microphone, and the memory;

the processor

output an audio signal through the first speaker and the second speaker;

acquiring an external sound through the at least one microphone while outputting the audio signal;

Checking whether the at least one call word is detected from the acquired external sound,

In response to the detection of the at least one call word, speaker recognition of an external sound corresponding to the call word is performed;

Checking whether the recognized speaker and the detected call word match, and

an electronic device configured to control an output of the audio signal so that a user of the electronic device can recognize an external sound when the detected call word and the recognized speaker match.
15. The method of claim 14,

the processor

Recognizing a direction in which the at least one call word is generated when the at least one call word is detected,

An electronic device configured to control the output of the external sound to have a sense of space corresponding to the recognized direction.