KR102342715B1 - System and method for providing supplementary service based on speech recognition - Google Patents

Abstract

A system for providing additional services based on voice recognition according to an embodiment includes: a client for receiving a voice input according to a user's utterance; and an apparatus for providing a speech recognition platform that works with the client to derive a user's speech intent corresponding to the speech input, and generates a plurality of response data based on the user's speech intent. based on the synchronization of the plurality of response data may be output to the outside.

Description

A system and method for providing additional services based on voice recognition

The present invention relates to a system and method for providing additional services based on voice recognition.

With the development of technology, various services to which voice recognition technology is applied have been introduced in many fields. Speech recognition technology can be said to be a series of processes that understand human voice and convert it into text information that a computer can handle. It can include a series of processes provided through the interface of

However, the general voice recognition service has a limitation in that the resolution is not high in information delivery due to the invisible characteristics of the user. For example, when notifying the user of the location of Seoul Station on the map, the visual-based interface can specify and inform the location at once, whereas the auditory-based interface informs the user of location information such as an address and explains If this is not the case, a lot of effort is required to transmit information, such as going through an auxiliary explanation of what is around.

The limitations of such voice recognition technology dilute the advantage of being able to easily perform multitasking with little effort and act as a factor hindering the use rate of the voice recognition service.

An embodiment is to provide a voice recognition-based supplementary service providing system and method capable of improving user experience by presenting a voice recognition platform in which a visual-based interface is implanted in an auditory-based interface.

The technical problem to be solved in the embodiment is not limited to the technical problem mentioned above, and another technical problem not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

According to an embodiment, a client receiving a voice input according to a user's utterance; and an apparatus for providing a speech recognition platform that works with the client to derive a user's speech intent corresponding to the speech input, and generates a plurality of response data based on the user's speech intent. Based on the synchronization of the plurality of response data to output to the outside, it is possible to provide an additional service providing system based on voice recognition.

The plurality of response data may include: first response data including audio data in an auditory form; and second response data including content data in a visual form.

The apparatus for providing the speech recognition platform may include: a first server that converts the speech input into query text and derives the user's intention to speak by using at least one of natural language processing and object name recognition; a second server for generating a response text by searching for the user's utterance intention based on a pre-established database; and a third server that performs a session connection with the client according to a prescribed protocol.

The second server may transmit the response text to the first server, and the first server may generate the first response data by converting the response text into a voice signal.

The second server may generate the second response data by extracting a key word from the response text, and may transmit the second response data to the third server.

The client may receive the first response data from the first server and receive the second response data from the third server while the session connection is maintained.

The response waiting time may be an output delay time between a first time point of receiving the voice input and a second time point of receiving all of the first and second response data.

The client may control the output of any one of the first and second response data in a standby state before the response waiting time elapses.

The client may store the voice input by mapping the first and second response data to each other.

The protocol may include Hyper Text Transfer Protocol Version 2 (HTTP/2).

According to at least one embodiment of the present invention, it is possible to provide the effect of multi-sensory integration to the user by presenting a voice recognition platform capable of delivering visual information together with hearing.

In addition, since a relay server can be built between the user terminal and the voice recognition platform to transmit visual information in a detour, the stability of the system can be improved. In addition, by controlling the user terminal so that the output timing of the auditory information and the visual information coincide, it is possible to reduce the sense of heterogeneity felt by the user.

Effects obtainable in this embodiment are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention pertains from the description below. .

1 is an exemplary diagram illustrating an operating environment of a system for providing additional services based on voice recognition according to an embodiment of the present invention.
FIG. 2 is a schematic block diagram of the system for providing additional services shown in FIG. 1 .
3 is a diagram for explaining an example of response text and additional information generated by the apparatus for providing a speech recognition platform according to an embodiment of the present invention.
4 is a flowchart illustrating a method of providing an additional service based on voice recognition according to an embodiment of the present invention.
5 is a diagram illustrating a response to a user's utterance implemented through a voice recognition application program installed in a client according to an embodiment of the present invention.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings. Since the embodiment may have various changes and may have various forms, specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the embodiment to the specific disclosed form, and it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the embodiment.

Terms such as “first” and “second” may be used to describe various elements, but these elements should not be limited by the terms. These terms are used for the purpose of distinguishing one component from another. In addition, terms specifically defined in consideration of the configuration and operation of the embodiment are only for describing the embodiment, and do not limit the scope of the embodiment.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary may be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, it is interpreted in an ideal or excessively formal meaning. doesn't happen

Hereinafter, an additional service providing system based on voice recognition according to an embodiment will be described with reference to the accompanying drawings.

1 is an exemplary diagram illustrating an operating environment of a system for providing additional services based on voice recognition according to an embodiment of the present invention.

Referring to FIG. 1 , the system 1 for providing additional services based on voice recognition may include a client 10 and a voice recognition platform providing apparatus 20 that are interlocked with each other through a network 30 .

The client 10 may receive user utterances containing specific commands or queries. For example, the client 10 executes an application program supporting the operation of the voice recognition service at the user's request, and based on the input device activated according to the execution of the program, a voice input according to the user's utterance (hereinafter, 'for convenience' user's uttered voice) may be received.

The client 10 includes various devices corresponding to the Internet of Things (IoT) such as smartphones, wearable terminals, artificial intelligence speakers, robot cleaners, set-top boxes, TVs, refrigerators, etc., and the various devices are operated by a communication network operator may be pre-subscribed and/or registered to a mobile communication service. However, this is exemplary and the scope of the present invention is not necessarily limited thereto.

The voice recognition platform providing apparatus 20 may perform a series of processes for providing additional services to the user based on voice recognition. For example, the apparatus 20 for providing a speech recognition platform recognizes the user's speech voice received from the client 10 to derive the user's speech intent, and mounts at least one information resource corresponding to the speech intent in the client 10 . It can support response to user's utterance by sending it to an output device.

The network 30 may serve to connect the client 10 and the apparatus 20 for providing a voice recognition platform. Such networks 30 include wired networks such as local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), integrated service digital networks (ISDNs), and/or Wideband Code Division Multiple Access (WCDMA), It may include a wireless communication network supporting mobile communication standards such as Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and 5G (Generation), but this is merely exemplary and the scope of the present invention is not necessarily However, the present invention is not limited thereto.

FIG. 2 is a schematic block diagram of the system for providing additional services shown in FIG. 1 .

Referring to FIG. 2 , the apparatus 20 for providing a voice recognition platform interworking with the client 10 searches for at least one information resource related to the intelligent voice recognition server 210 supporting the voice recognition service and the user's utterance intention. It may include a service providing server 230 and a relay server 250 performing a session connection with the client (10). Hereinafter, each component of the client 10 will be first described before describing the apparatus 20 for providing a voice recognition platform.

The client 10 may include a microphone 110 , a communication unit 120 , a speaker 130 , a display 140 , a memory 150 , and a control unit 160 . However, this is exemplary, and the client 10 may omit at least one of the above-described components or may additionally include other components.

The microphone 110 may receive a voice input according to a user's utterance. The microphone 110 may be driven according to the execution of a voice recognition application program, or may be controlled in a constantly driven state. Here, the voice recognition application program may be executed by a user's manipulation or a specified wake-up command.

The communication unit 120 may support communication between the client 100 and an external device. For example, the communication unit 120 may establish wired or wireless communication according to a prescribed protocol with the intelligent voice recognition server 210 and/or the relay server 250 . The communication unit 120 connects to the network 30 based on the wired or wireless communication, and thus at least one information resource involved in the operation of the intelligent voice recognition server 210 and/or the relay server 250 and the voice recognition service. can send and receive

The speaker 130 may output a voice signal generated inside the client 10 or received from the intelligent voice recognition server 210 . For example, the speaker 130 may receive the audible response utterance received from the intelligent voice recognition server 210 and output it to the outside under the control of the controller 160 .

The display 140 may output various types of additional information under the control of the controller 160 . The display 140 may output additional information in a visual form including text, graphic, image, video, or a combination thereof transmitted from the relay server 250 to at least one area, and the additional information is related to the user's utterance. It can be understood as at least part of the response. For example, the display 140 may be configured as a touch screen by forming a layer structure with a touch pad that receives a touch input by a user, and may output an interface related to the system setting of the client 100 . have.

The memory 150 may store at least one information resource involved in the operation of the voice recognition service or may store a command related to the functional operation of a component constituting the client 100 . Here, at least one information resource involved in the operation of the voice recognition service is a user uttered voice applied to the microphone 110 , a response uttered voice received from the intelligent voice recognition server 210 , and transmitted from the relay server 250 . It may include at least one of additional information. Also, the memory 150 may store at least one application program related to the operation of the client 10 , including the aforementioned voice recognition application program.

The controller 160 may be electrically connected to at least one component of the above-described client 100 to perform control, calculation, or data processing on the component. For example, the control unit 160 may control the speaker 130 to output an utterance voice, and may control the display 140 to output additional information. Alternatively, the controller 160 may store at least one information resource involved in the operation of the voice recognition service in the memory 150 or load it from the memory 150 and process it.

In this case, in relation to the provision of a response to the user's utterance, the controller 160 may synchronize and output the response uttered voice and additional information based on a response latency time. For example, the controller 160 controls the client 10 to be in a standby state when any one data of the response speech and the additional information is received, and when the other data is received, the controller 160 controls the response speech voice and additional information can be synchronized. Here, the standby state can be understood as waiting or suspending the output of any one data received by the client 10 during the response waiting time. In addition, the response waiting time may be defined as an output delay time between a point in time when a user's uttered voice is received and a time when all of the response uttered voice and additional information are received.

The voice recognition platform providing apparatus 20 may include an intelligent voice recognition server 210 , a service providing server 230 , and a relay server 250 .

The intelligent voice recognition server 210 supports a voice recognition service, and may include a voice recognition unit 211 , an intention analysis unit 212 , and a voice synthesis unit 213 .

The voice recognition unit 211 may recognize the user's uttered voice received from the client 10 and convert it into a query text. The voice recognition unit 211 uses an acoustic model including at least one piece of utterance or utterance-related information or a language model including at least one unit phoneme information to generate a query text for the utterance of the user. can be converted to Speech To Text (STT).

The intention determination unit 212 performs Natural Language Processing (NLP) and Named Entity Recognition (NER) on the query text received from the speech recognition unit 211 to derive the user's speech intention. can Here, natural language processing (NLP) refers to natural language understanding in which a query text is divided into grammatical units such as syntax, part-of-speech, and morpheme, and the meaning is determined by analyzing the linguistic characteristics of each grammatical unit. In addition, entity name recognition (NER) refers to an information retrieval technology for extracting entity names such as person, place, time, object, and organization contained in the query text and classifying the extracted entity name type.

The intention determination unit 212 may increase the accuracy of speech recognition by learning the user's language habit by applying an artificial intelligence (AI) algorithm (eg, deep learning, machine learning, etc.), and an intelligent speech recognition server ( 210) may request a response corresponding to the utterance intention to the service providing server 230 associated with the utterance. As an example, if the query text is [What are you doing in entertainment today], the intention determining unit 212 uses natural language processing and object name recognition to perform [entertainment program information request] corresponding to the 'time' object called [today] through the user's utterance. It can be derived as an intention and delivered to the service providing server 250 .

The voice synthesizer 213 may transmit, to the client 10 , the response utterance obtained by converting (Text To Speech, TTS) the response text in the natural language form generated by the service providing server 230 to be described later.

The service providing server 230 generates an appropriate response based on the user's utterance intention received from the intention determination unit 212 , and may include a search unit 231 and a database 232 .

The search unit 231 may obtain a response desired by the user by searching for the user's utterance intention through the knowledge management built in the database 232 . Here, the database 232 may be a knowledge-based database as a set of essential basic data for responding to a user's query. In the database 232, metadata of content related to each domain (eg, music, movie, broadcasting, weather, news, sports, transportation, finance, shopping, etc.) is classified and stored in advance, and the metadata is It may be provided by a third party vendor based on a subscription or affiliation with the recognition service. For example, in the case of a 'broadcast' domain, the metadata may include information about titles, characters, plots, broadcast times, ratings, broadcasting stations, etc. of related content.

The search unit 231 may extract metadata of a specific domain and each content from the database 232 based on the user's utterance intention, and may generate an appropriate response text in a natural language form using text summary and personalized recommendation technology. . For example, if the user's utterance intention is [entertainment program information request] broadcast [today], the search unit 231 includes [today's entertainment program, movies are good, acquaintance brother, what do you do when you play, immortal masterpieces, etc. ] can be output as response text.

In addition, the search unit 231 extracts a key word of the response text, and utilizes a search engine (not shown) provided inside and/or outside the service providing server 250 to provide additional information corresponding to the user's utterance intention. can create Here, the additional information may include content data in a visual form including text, graphic, image, video, or a combination thereof.

An example of the above-described response text and additional information will be first described below with reference to FIG. 3 .

3 is a diagram for explaining an example of response text and additional information generated by the apparatus for providing a speech recognition platform according to an embodiment of the present invention.

Referring to (a) of FIG. 3 , when a user's uttered voice (Q) of [Tell me today's weather] is transmitted from the client 10, the intelligent voice recognition server 210 derives the user's utterance intention (eg, ' 'Today's weather information request'), and the service providing server 230 may generate a response text (A) and additional information (B) corresponding thereto.

The search unit 231 may obtain the response text A stating [The current weather is sunny] using the text summary technology based on the metadata stored in the 'weather' domain on the database 232 .

In addition, the search unit 231 searches for the user's utterance intention in the database 232 and/or a search engine (not shown). Additional information (B) may be generated by combining numerical information (text) and an image representing weather conditions (sunny, cloud, rain, snow).

Referring to (b) of FIG. 3 , when a user's uttered voice Q is transmitted from the client 10 to [tell me the latest Tottenham match result], the intelligent voice recognition server 210 derives the user's utterance intention (eg ; 'request for information on the result of Tottenham's recent match'), and the service providing server 230 may generate a response text (A) and additional information (B) corresponding thereto.

Based on the metadata stored in the 'Sports' domain on the database 232, the search unit 231 responds to the text [Tottenham lost 0–1 against Newcastle yesterday] using text summarization and personalized recommendation technology. (A) can be obtained.

In addition, the search unit 231 searches a search engine (not shown) for a key word (eg, Tottenham, match result, yesterday) extracted from the response text (A) to obtain a sports league, stadium, match date and time, and match score. Additional information (B) can be generated by combining information such as text and an image in the shape of the emblems of both teams.

As shown in (a) to (b) of FIG. 3 , when additional information (B) in a visual form is provided together as a result of a response to a user's utterance, the user can intuitively understand the information, so that the voice recognition service User experience (UX) may be improved in the field. For example, according to an embodiment, since visual information is transmitted together with auditory information, a multi-sensory integration effect occurs, so that a faster and more accurate user experience can be provided beyond supplementing the auditory limitation.

Returning to FIG. 2 again, the above-described service providing server 230 may transmit each of the generated response text and additional information to the intelligent voice recognition server 210 and the relay server 250 , respectively.

The relay server 250 may relay the transmission of additional information between the service providing server 230 and the client 10 . To this end, the relay server 250 establishes communication with the client 10 according to a prescribed protocol, and establishes an infrastructure for supporting additional services based on voice recognition by performing a session connection with each other. can do. Here, as an example of the protocol, HTTP/2 (Hyper Text Transfer Protocol Version 2) may be applied, but the scope of the present invention is not necessarily limited thereto.

When the relay server 250 receives the session request message from the client 10 , session connection may be performed by allocating session information—eg, a session ID—to the client 10 and transmitting a session response message. In addition, the client 10 and the relay server 250 may maintain the session by periodically transmitting and receiving a status check message (eg, a ping) between each other.

The relay server 250 may receive additional information from the service providing server 230 and transmit it to the client 10 while the session with the client 10 is maintained. Accordingly, even if the specification of the client 10 does not conform to the communication standard supported by the service providing server 250 , the user can receive additional information in a detour through the relay server 10 .

Also, the relay server 250 may serve to extend the support range of data provided from the apparatus 20 for providing a voice recognition platform to the client 10 . That is, the apparatus 20 for providing a voice recognition platform according to an embodiment builds a separate relay server 250 in addition to the intelligent voice recognition server 210, so that the type, format, or transmission standard of data is different from each other. By dually allocating the transmission subject and the additional information, the intelligent voice recognition server 210 does not depend on the data transmission standard supported by various types or types of data - for example, text, graphics, images, video, audio. or a combination thereof- can be provided to the user.

For example, the apparatus 20 for providing a speech recognition platform allocates the intelligent speech recognition server 210 as a transmission subject of a response uttered voice composed of voice data in an auditory form, and a transmission subject of additional information composed of content data in a visual form Each relay server 250 can be assigned. As such, the reason for dually allocating the transmission subject according to the data type is that the data transmission standard supported by the intelligent voice recognition server 210 is extremely limited. In other words, even if there is additional information to be provided to the user when the voice recognition service is executed, if the standard to be transmitted from the intelligent voice recognition server 210 to the client 10 is not supported, the user may be provided with the additional information. no problem arises. For the above reasons, one embodiment can obtain the effect of providing various types of data to be expressed by the client 10 regardless of the specifications of the intelligent voice recognition server 210 through the construction of the relay server 250 . have.

Hereinafter, a method of providing an additional service based on voice recognition will be described with reference to FIG. 4 .

4 is a flowchart illustrating a method of providing an additional service based on voice recognition according to an embodiment of the present invention.

First, the relay server 250 may perform a session connection with the client 10 (S1). In step S1, the relay server 250 establishes communication with the client 10 according to a prescribed protocol, and when receiving a session request message from the client 10, the A session connection is established by sending a session reply message with a session ID. When the session connection is completed, the relay server 250 can maintain the session connection state with the client 10 by periodically transmitting and receiving a status check message.

Thereafter, the client 10 may receive a user utterance including a specific command or query ( S2 ). In step S2, the client 10 is activated as an application program supporting the operation of the voice recognition service is executed, and the program may be executed by a user's manipulation or a specified wake-up command.

In addition, the client 10 may transmit a voice input according to the user's utterance (hereinafter, referred to as 'user uttered voice') to the intelligent voice recognition server 210 (S3).

The intelligent speech recognition server 210 recognizes the user's spoken voice received from the client 10 and converts it into a query text (S4), and performs natural language processing and object name recognition on the query text to derive the user's speech intention. can be (S5). In steps S4 and S5, the intelligent voice recognition server 210 may learn the user's language habit based on an artificial intelligence (AI) algorithm in order to improve the accuracy of voice recognition or utterance intention.

Thereafter, the intelligent voice recognition server 210 may request a response corresponding to the user's utterance intention to the service providing server 230 (S6).

The service providing server 230 may acquire a response desired by the user by searching for the user's utterance intention through knowledge management built in the database (S7). For example, the service providing server 230 includes at least one of music, movie, broadcasting, weather, news, sports, transportation, finance, shopping, etc. in a specific domain on the database based on the user's intention to speak. It is possible to extract metadata of embedded related content and generate appropriate response text in natural language form using text summary and personalized recommendation technology. In addition, the service providing server 230 may extract a key word from the response text, and may generate additional information corresponding to the user's utterance intention by utilizing an internal and/or external search engine. Here, the additional information may include content data in a visual form including text, graphic, image, video, or a combination thereof so that the user can intuitively understand the response result.

Thereafter, the service providing server 230 may transmit the response text, which is at least a part of the response generated in step S7, to the intelligent voice recognition server 210 (S8).

In addition, the service providing server 230 may transmit additional information that is the remaining part of the response generated in step S7 to the relay server 250 (S9).

After step S8, the intelligent speech recognition server 230 converts the response text in the natural language form into a voice signal (Text To Speech, TTS) (S9), and generates a response speech in an auditory form corresponding to the converted voice signal. It can be transmitted to the client 10 (S11).

After step S9, the relay server 250 may transmit additional information to the client 10 maintaining the mutual session connection state (S12). Accordingly, even if the specification of the client 10 does not conform to the communication standard supported by the service providing server 250 , the user can receive additional information in a detour through the relay server 10 .

In addition, referring to steps S11 and S12, the voice recognition platform providing device 20 builds a separate relay server 250 in addition to the intelligent voice recognition server 210 to have different data types, formats, or transmission standards. By dually allocating the transmission subject of the response speech and the additional information, the intelligent speech recognition server 210 does not depend on the data transmission standard supported by various types or types of data - for example, text, graphics, images, video, audio, or a combination thereof - may be provided to the user. Accordingly, the support or service range of data provided from the apparatus 20 for providing a voice recognition platform to the client 10 may be expanded.

Thereafter, the client 10 may output a response by synchronizing the response speech received from each of the intelligent voice recognition server 210 and the relay server 250 with the additional information (S13). In step S13 , the client 10 may match the output timing of the response uttered voice with the additional information based on the response latency time. Here, the response waiting time can be understood as an output delay time between receiving the user speech utterance first point in time (t ₁₎ and the response voice utterance and all of the received additional information, the third time (t _3).

)가 발생될 수 있다. Each server 210, 250 for transmitting the response speech and additional information is physically or logically separated, and each server 210, 230 transmits data of different senses (auditory or visual) to the client 10. Therefore, a gap may occur at the point of time when each data is received from the perspective of the client 10 . The reason is that the data processing speed (or transmission speed) or the capacity of transmitted data between each server is different from each other. Therefore, the client 10 and a second point in time (t ₂₎ and a predetermined time period between the third time (t ₃₎ for receiving the additional information to receive a response utterance voice difference (

) may occur.

)를 두고 따로따로 구현되기 때문에 사용자 측면에서 음성인식 서비스에 대한 신뢰도를 저해하는 요인으로 작용할 수 있다. 특히, 각 데이터는 인간의 오감 중 서로 다른 감각(청각 또는 시각)에 의존하기 때문에 제공되는 응답에 대하여 사용자가 느끼는 이질감은 증대될 수밖에 없다.If the client 10 outputs a response immediately upon receiving any one of the response speech voice or additional information, each data is separated by a predetermined time difference (

) and implemented separately, it may act as a factor impeding the reliability of the voice recognition service from the user's point of view. In particular, since each data depends on different senses (hearing or sight) among human five senses, the sense of heterogeneity felt by the user in response to the provided response is inevitably increased.

For the above reasons, the client 10 according to an embodiment controls the output of the data to a standby state at the _{time point t 2} when any one of the response speech voice and the additional information is received, and the other data By synchronizing the response uttered voice and the additional information at the reception time t _{3 , the output time may be matched.} At this time, the client 10 maps and stores the user uttered voice, the response uttered voice, and additional information in the memory 150 (see FIG. 2 ), and when the response waiting time elapses, the response result from the memory 150 (see FIG. 2 ) can be loaded and processed.

An example of a response to a user's utterance implemented in the aforementioned client 10 will be described below with reference to FIG. 5 .

5 is a diagram illustrating a response to a user's utterance implemented through a voice recognition application program installed in a client according to an embodiment of the present invention.

Referring to FIG. 5 , in response to the user's utterance, the client 10 outputs a response utterance voice A through the speaker 130 (refer to FIG. 2 ) and simultaneously displays the display 140 (refer to FIG. 2 ). ) may display additional information (B).

Hereinafter, it is assumed that the client 10 collects the user's utterance voice Q of [what are you doing in entertainment today] from the user. However, this is only an example, and it is obvious to those skilled in the art that the scope of the user's spoken voice Q is not limited thereto.

When a user uttered voice (Q) of [What are you doing in entertainment today] from the user is input to the client 10, the client 10 responds to the uttered voice (A) generated by the voice recognition platform providing device (not shown) ) and additional information (B) can be received.

The response utterance voice (A) and additional information (B) received by the client 10 correspond to the user's utterance intention derived by the voice recognition platform providing device (not shown) - eg 'today's entertainment program' - response results may be included. As an example, the response speech (A) includes data in the form of an audible [there are good movies, acquaintances, what do you do when you play, immortal songs, etc. for today's entertainment program], and the additional information (B) is Thumbnail image for each content of 'Today's entertainment program' and visual form data including text containing information on title, broadcasting station, broadcast time, and audience rating of each content have.

The client 10 outputs the response uttered voice A through the speaker 130 (refer to FIG. 2) and outputs the additional information B through the display 140 (refer to FIG. 2), but the response uttered voice The user experience (UX) can be improved by matching the output timings of (A) and the additional information (B) and implementing them at the same time.

The method for providing additional services based on voice recognition according to the above embodiment may be produced as a program to be executed on a computer and stored in a computer-readable recording medium, and examples of the computer-readable recording medium include ROM, RAM. , CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

The computer-readable recording medium is distributed in a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described method can be easily inferred by programmers in the technical field to which the embodiment belongs.

Although only a few have been described as described above in relation to the embodiments, various other forms of implementation are possible. The technical contents of the above-described embodiments may be combined in various forms unless they are incompatible with each other, and may be implemented in a new embodiment through this.

On the other hand, the system and method for providing additional services based on voice recognition according to the above-described embodiment are various types of Internet of Things (IoT) such as smartphones, wearable terminals, artificial intelligence speakers, robot cleaners, set-top boxes, TVs, refrigerators, etc. available on the device.

It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (21)

a client receiving a voice input according to a user's utterance; and
A voice recognition platform providing apparatus that interworks with the client to derive a user's utterance intention corresponding to the voice input, and generates a plurality of response data based on the user's utterance intention;
the client,
Synchronize and output the plurality of response data to the outside based on the response waiting time,
The plurality of response data is
first response data including voice data in an auditory form; and
Including; second response data including content data in a visual form;
The voice data is a response text obtained by applying a text summarization technique based on the intention of the utterance into a voice signal,
The content data in the visual form is additional information obtained by combining an image obtained by searching for the intention of utterance in a database or a search engine to form additional information, a system for providing additional services based on voice recognition. delete According to claim 1,
The voice recognition platform providing device,
a first server that converts the voice input into a query text and derives the user's intention of speaking by using at least one of natural language processing and entity name recognition;
a second server for generating a response text by searching for the user's utterance intention based on a pre-established database; and
A system for providing additional services based on voice recognition, including; a third server for performing a session connection with the client according to a prescribed protocol. 4. The method of claim 3,
The second server transmits the response text to the first server,
The first server converts the response text into the voice data to generate the first response data, a system for providing additional services based on voice recognition. 5. The method of claim 4,
The second server,
extracting the key words of the response text to generate the second response data,
For transmitting the second response data to the third server, a system for providing additional services based on voice recognition. 6. The method of claim 5,
the client,
receiving the first response data from the first server;
A system for providing additional services based on voice recognition for receiving the second response data from the third server while the session connection is maintained. 7. The method of claim 6,
The response waiting time is
An additional service providing system based on voice recognition, which is an output delay time between a first time point of receiving the voice input and a second time point of receiving all of the first and second response data. According to claim 1,
the client,
Before the response waiting time elapses, the output of any one of the first and second response data is controlled in a standby state, a voice recognition-based supplementary service providing system. 7. The method of claim 6,
the client,
A system for providing an additional service based on voice recognition that maps the voice input and the first and second response data to each other and stores them. 4. The method of claim 3,
The protocol is an additional service providing system based on voice recognition, including HTTP/2 (Hyper Text Transfer Protocol Version 2). Client; And in the method for providing additional services based on the voice recognition of the system for providing additional services including a voice recognition platform providing apparatus,
In the apparatus for providing a voice recognition platform, receiving a voice input according to a user's utterance from the client;
deriving a user utterance intention corresponding to the voice input, and generating a plurality of response data based on the user utterance intention in the apparatus for providing a voice recognition platform;
receiving, at the client, the plurality of response data from the apparatus for providing a voice recognition platform; and
In the client, based on the response waiting time, synchronizing and outputting the plurality of response data to the outside;
The plurality of response data is
first response data including voice data in an auditory form; and
Including; second response data including content data in a visual form;
The voice data is a response text obtained by applying a text summarization technique based on the intention of the utterance into a voice signal,
The content data in the visual form is additional information obtained by combining an image obtained by searching for the intention of utterance in a database or a search engine, and is additional information based on voice recognition. delete 12. The method of claim 11,
The voice recognition platform providing apparatus includes first to third servers,
converting, at the first server, the voice input into a query text, and deriving the user's intention to speak by using at least one of natural language processing and entity name recognition;
generating, in the second server, a response text by searching for the user's utterance intention based on a pre-established database; and
In the third server, performing a session connection with the client according to a prescribed protocol; Containing, a method for providing additional services based on voice recognition. 14. The method of claim 13,
transmitting, by the second server, the response text to the first server; and
In the first server, converting the received response text into the voice data to generate the first response data; Containing, a method for providing additional services based on voice recognition. 15. The method of claim 14,
In the second server, extracting a key word of the response text, generating the second response data, and transmitting the second response data to the third server, a method for providing additional services based on voice recognition. 16. The method of claim 15,
in the client,
receiving the first response data from the first server; and
Receiving the second response data from the third server while the session connection is maintained; Containing, a method for providing additional services based on voice recognition. 17. The method of claim 16,
The response waiting time is
An output delay time between a first time point of receiving the voice input and a second time point of receiving all of the first and second response data, a method for providing additional services based on voice recognition. 12. The method of claim 11,
in the client,
Before the response waiting time elapses, controlling the output of any one of the first and second response data in a standby state; Containing, a method for providing additional services based on voice recognition. 17. The method of claim 16,
in the client,
and storing the voice input and the first and second response data by mapping each other. 14. The method of claim 13,
The protocol is a method for providing additional services based on voice recognition, including HTTP/2 (Hyper Text Transfer Protocol Version 2). A computer-readable recording medium in which an application program for realizing the method for providing additional services based on voice recognition according to any one of claims 11 and 13 to 20 is recorded through being executed by a processor.

Images