KR20180099163A - Apparatus for expressing interactions corresponding to a user's situation based on internet and method for the same - Google Patents

Abstract

An embodiment of the present invention provides an apparatus for expressing interactions. It includes a status information generation part for analyzing user daily information to generate user status information; a scenario selection part for selecting an interaction scenario corresponding to the user state information; an interaction determination part which determines an interaction object and interactions using the selected interaction scenario, requests the determination of the interaction object and the interaction from a server device for expressing interactions when the interaction object and the interaction cannot be determined, and substitutes the received interaction object and the interaction; and an interaction expression part for expressing the interaction object and the interaction through at least one output device of the user. It is possible to determine the interaction corresponding to the state of the user based on the information of the user.

Description

[0001] APPARATUS AND EXPRESSING INTERACTIONS CORRESPONDING TO A USER'S SITUATION BASED ON INTERNET AND METHOD FOR THE SAME [0002]

The present invention relates to an apparatus and a method for expressing an interaction corresponding to a user's situation on the Internet. More particularly, the present invention relates to an apparatus and a method for determining an interaction corresponding to a user's state using user's daily information collected from user's input devices and expressing the interaction through one or more output devices of the user.

Currently, there is a tendency that one person is increasingly socially. The increase in the number of single-person households can guarantee individual freedom and improve the quality of life, but it can lead to the problem that the indifference to others is spread and the life satisfaction is lowered due to loneliness. Accordingly, although there are various devices such as dolls capable of interacting with the user, there is a limitation in collecting the current user's situation and not responding appropriately.

As technologies such as smart phones, AI, and AR are developed, various types of input / output can be easily performed, and it becomes possible to analyze and judge the situation using only a simple device.

Korean Patent Laid-Open Publication No. 10-2015-0065523

The object of the present invention is to determine an interaction corresponding to a user's state based on information of a user collected through one or more user input devices (e.g., a smart phone, a tablet, etc.).

In particular, it is intended to convert the interaction corresponding to the user's state into a form preferred by the user and to express through one or more user output devices (e.g., AR device, TV, etc.).

According to an embodiment of the present invention, there is provided an information processing apparatus comprising: a status information generation unit for analyzing user daily information to generate user status information; A scenario selector for selecting an interaction scenario corresponding to the user state information; Determining interaction objects and interactions using the selected interaction scenarios and, if the interaction objects and interactions are not determined, determining interactions objects and interactions in the server device for expressing the interactions An interaction determining unit that replaces the received interaction object with the received interaction object; And an interaction expression unit for expressing the interaction object and the interaction through at least one or more output devices of the user. And a client device for expressing the interaction.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

Based on the user's information collected via one or more user input devices, an interaction corresponding to the user's state can be determined.

In particular, the interaction corresponding to the state of the user may be transformed into a form preferred by the user and expressed through one or more user output devices.

1 is a block diagram of a system for representing an interaction according to an embodiment of the present invention.
2 is a block diagram showing an example of the internal configuration of a client apparatus for expressing the interaction shown in FIG.
3 is a block diagram showing an example of the internal configuration of the server apparatus for expressing the interaction shown in FIG.
4 is an example of an augmented reality-based interactive image output to a user output device according to an embodiment of the present invention.
5 is another example of an augmented reality-based interactive image output to a user output device according to an embodiment of the present invention.
6 is an operational flowchart illustrating a method for representing an interaction performed by a client device for representing an interaction according to an embodiment of the present invention.
FIG. 7 is an operational flow diagram illustrating an example of steps for determining interaction objects and interactions shown in FIG. 6. FIG.
FIG. 8 is an operational flowchart showing an example of a step of converting the interaction with the interaction object shown in FIG. 6. FIG.
FIG. 9 is an operational flowchart showing an example of a step of generating the user status information shown in FIG.
FIG. 10 is an operation flowchart showing an example of a step of selecting an interaction scenario shown in FIG.

The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated and described in the drawings. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

However, the present invention is not limited to the embodiments described below, but all or some of the embodiments may be selectively combined and implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

1 is a diagram illustrating a configuration of a system 1 for expressing an interaction according to an embodiment of the present invention.

Referring to FIG. 1, a system for expressing an interaction according to an embodiment of the present invention includes a client device 100 for expressing an interaction, a server device 200 for expressing an interaction, Input devices 310_1 to 310_n, and user output devices 320_1 to 320_n.

The client device 100 for expressing the interaction is connected to each of the user input devices 310_1 to 310_n and the user output devices 320_1 to 320_n through a wired or wireless network or the like. In addition, the client device 100 for expressing the interaction and the server device 200 for expressing the interaction are connected through various communication technologies such as a cellular communication network, a wired / wireless network, and the like. At this time, a client device 100 for representing a plurality of interactions may correspond to one server device 200 for expressing an interaction.

The client device 100 for representing an interaction according to an embodiment of the present invention uses mutual user's daily information collected from at least one or more user input devices 310_1 to 310_n, You can determine the interaction with the action object. The interaction with the determined interactive object may be output using at least one or more user output devices 320_1 to 320_n. Here, the user's daily information includes information that can be collected from the user's daily life such as a user's facial expression, voice, and motion.

In detail, the client device 100 for expressing the interaction can analyze the collected user daily information to generate user status information, and can determine the point of time when the user interacts with the user using the generated user status information . When the user interacts with the user, the interaction scenario corresponding to the user state is selected, and the interaction object and the interaction corresponding to the user's behavior can be determined according to the selected scenario.

Here, the user state information is information that can determine the state of the user including the collection time information, the emotion state of the user, the life pattern, and the like. Interaction scenarios can include scenarios or settings that guide users and scenarios to proceed. The interaction object may include a dog, a spouse, and the like as an object with which the user interacts.

Also, in determining the point of time when the user interacts with the user, the predetermined dialog mode may be reflected. Here, the dialogue mode is a mode for determining a style of interacting with a user, and may include a mode in which the user interacts normally, or a mode in which the user tries to interact only when desired. Here, the user can change the conversation mode and set it.

At this time, when determining the interaction with the interactive object corresponding to the user's behavior according to the selected scenario, the client device 100 for expressing the interaction selects the scenarios selected by the server device 200 for expressing the interaction, And user behavior. And may receive interactions with the interacting objects determined at the server device 200 for expressing the interactions.

In an alternate embodiment, the client device 100 for expressing the interaction may convert the interaction with the determined interaction object to a predetermined mode of operation. Then, the interaction with the converted interactive object can be output using the user output devices 320_1 to 320_n.

Here, the operation mode is a mode for converting the interaction with the interaction object into various formats while maintaining the same content interaction, and may include a dog mode or a spouse mode. This allows the user to interact with the user in a preferred manner. Here, the user can set the operation mode by changing the operation mode.

For example, when the predetermined operation mode is the dog mode, the interactive object can be converted into a pet dog, and the interaction can be converted into a dog crying or the like.

In an alternative embodiment, the client device 100 for expressing the interaction may send user routine information to the server device 200 for expressing the interaction when analyzing the user's daily information and generating the user status information . The server device 200 for expressing the interaction can receive the user state information generated by analyzing the user's daily information.

Accordingly, when analyzing the user's daily information collected by the client device 100 for expressing the interaction, the server device 200 for expressing the interaction may be used to determine the user state corresponding to the user's daily information Information can be generated.

In an alternative embodiment, the client device 100 for expressing the interaction may send user state information to the server device 200 for expressing the interaction when selecting the interaction scenario from the user state information. The server apparatus 200 for expressing the interaction can receive the selected interaction scenario corresponding to the user status information.

Accordingly, when the client device 100 for expressing the interaction fails to select the interaction scenario corresponding to the user state information to interact with the user from the user state information, the server device 200 for expressing the interaction, The interaction scenario corresponding to the user state information can be selected.

In an alternate embodiment, the client device 100 for interacting with the server device 200 may be configured to interact with the interacting object to correspond to the interaction scenario and user behavior, Action scenarios and user actions. And, the server device 200 for expressing the interaction can receive an interaction with the determined interaction object based on the interaction scenario based on the user behavior.

Accordingly, when the client device 100 for expressing the interaction fails to determine the interaction with the interaction object corresponding to the user action based on the interaction scenario, the server device 200 for expressing the interaction, Can be used to determine interactions and interactions that correspond to user behavior based on the interaction scenario.

The server device 200 for representing an interaction according to an exemplary embodiment of the present invention stores information such as user daily information, user state information, interaction scenarios, and various interaction objects collected from various users Databases are available. Then, the client device for analyzing the user's daily information, selecting the interaction scenario, determining the interaction object and the interaction with the client device for expressing the interaction using the database is transmitted to the client device for expressing the interaction .

The user input devices 310_1 to 310_n may be a terminal such as a smart phone or a tablet, and may include a video recording module such as a camera or a voice recording module such as a microphone. Each of the user input devices 310_1 to 310_n may collect collectible user daily information together with time. Depending on the collected user input device, the collected daily information may be different in kind have.

In addition, the user input devices 310_1 to 310_n may include a user information collector for collecting user information, and an environment setting device for changing settings of the client device 100 for expressing the interaction. The user can change the setting of the user's daily information collection period, the operation mode, and the dialogue mode through the configuration unit.

At this time, the user input devices 310_1 to 310_n may periodically collect user's daily information according to a predetermined collection period, such as when the user has a request or when a specific response is received from the user. The collected user's daily information may be sent to the client device 100 for expressing the interaction and used to determine the time or content to interact with the user.

In particular, the user can change and set the user input devices 310_1 to 310_n to be used for collecting the user's daily information.

The user output devices 320_1 to 320_n may be a terminal such as a speaker, a TV, a monitor, and an AR device, and may include a video output module such as a display or a sound output module such as a speaker. At this time, the data formats of the interaction or interactive objects output from the different modules of the user output devices 320_1 to 320_n may be different.

In particular, the user can change and set the user output devices 320_1 to 320_n to express interaction with the interactive object.

For example, a sound output device such as a speaker outputs interactive voice or sound, and an image output device such as a TV or a monitor can output an interactive object image or an interactive voice caption. In particular, when a plurality of loudspeakers are provided with a sound output apparatus, the sound output apparatus may output a sound in the vicinity of the interactive object in association with the motion of the interactive object.

If the AR device is used as a user output device, the interactive objects may be output based on the augmented reality to be displayed as if they are included in the space of the actual user.

2 is a block diagram showing an example of the internal configuration of the client apparatus 100 for expressing the interaction shown in FIG.

2, a client apparatus 100 for representing an interaction according to an embodiment of the present invention includes a client communication unit 110, a daily information collecting unit 120, a status information generating unit 130, A storage unit 140, an interaction time determination unit 150, a scenario selection unit 160, an interaction determination unit 170, and an interaction expression unit 190.

In addition, the client apparatus 100 for representing an interaction according to an embodiment of the present invention may further include an interaction conversion unit 180.

In detail, the client communication unit 110 provides a communication interface necessary for transmitting a transmission / reception signal between the client apparatus 100 for expressing the interaction and the server apparatus 200 for expressing the interaction.

Here, the client communication unit 110 may be a device including hardware and software necessary for transmitting / receiving a signal such as a control signal or a data signal through a wired / wireless connection with another network device.

The routine information collecting unit 120 collects user daily information collected from at least one or more user input devices (see 310_1 to 310_n in FIG. 1). The user's daily information may include a user's facial expression, voice, operation, etc., and includes collection time information.

Here, the daily information collecting unit 120 may periodically collect user daily information every period of the predetermined collection period or whenever the user daily information is collected from the user input devices (see 310_1 to 310_n in FIG. 1) have. At this time, the user can change the user's daily information collection period and set it.

Particularly, in the case where the present invention is carried out at the user's home, when the user's daily information is newly collected from the user input devices (see 310_1 to 310_n in FIG. 1), such as when the user goes out and returns home, The information collecting unit 120 may collect user daily information.

The state information generating unit 130 analyzes user daily information collected by the daily information collecting unit 120 to generate user state information. The generated user state information may be stored in the state information storage unit 140 together with the analysis time information.

Here, the user state information may be information indicating the state of the user, and may include the emotion state of the user, the life pattern of the user, and the like. The contents of the interaction with the user can be changed according to the emotional state of the user or the life pattern of the user, and it is also possible to judge whether or not to proceed with the interaction with the user.

In an alternative embodiment, when the state information generator 130 analyzes the user's daily information and fails to generate user state information, the state information generator 130 may generate a user's daily routine Information can be transmitted. Then, the user state information generated by analyzing the user's daily information can be received from the server device (see 200 in FIG. 1) for expressing the interaction. In addition, the received user state information and analysis time information may be stored together in the state information storage unit 140.

The state information storage unit 140 stores the user state information and time information generated by the state information generation unit 130. Here, the user state information may be used together with the time information to determine a change in the user state.

The interaction time determination unit 150 determines a time to start the interaction with the user based on the user state information generated by the state information generation unit 130. [

Here, the interaction time determination unit 150 can periodically determine an interaction time point at every predetermined determination period or whenever the user state information is generated in the state information generation unit 130. [ At this time, the user can change the user daily information determination cycle and set it.

Accordingly, when it is determined that the current time is not the time for the current interaction, the user can wait for a predetermined period of time to determine the start time of the interaction or determine the start time of the interaction as the new user state information is generated.

The scenario selection unit 160 selects an interaction scenario corresponding to the user state information stored in the state information storage unit 140 when it is determined that the interaction time determination unit 150 starts the interaction with the user.

At this time, it is possible to select an interaction scenario by comparing the user state information and the change information of the user's emotional state or life pattern.

Here, the interaction scenario may be a content for improving the user's emotional state or a content suitable for the current life pattern of the user.

In an alternative embodiment, if the scenario selector 160 fails to select an interaction scenario corresponding to the user state information, the scenario selector 160 may select a user state (e.g., Information can be transmitted. Then, the server device (see 200 in FIG. 1) for expressing the interaction can receive the selected interaction scenario corresponding to the user status information.

The interaction determination unit 170 determines an interaction with the interaction object corresponding to the user behavior based on the interaction scenario selected by the scenario selection unit 160. [

For example, in the case that a user's mood is judged to be in a bad state, and a scenario that makes the user feel good is selected, the words "I do not feel well today" Action, and the interacting object can decide to build a careful and anxious expression.

Further, when the user is in the process of interacting with the user, when the information of the user's response or response is received, the interaction with the interaction object corresponding to the user response or response can be determined based on the selected interaction scenario . If the user's response is to terminate the interaction, then the corresponding interaction may be determined and the interaction with the current user may be terminated.

In addition, if the interaction determining unit 170 fails to determine the interaction with the interaction object corresponding to the user action based on the interaction scenario, the server apparatus for representing the interaction The user may transmit the interaction scenario and the user behavior information to the user. And, the server device (see 200 in FIG. 1) for expressing the interaction can receive the interaction with the determined interaction object based on the interaction scenario based on the user behavior.

The interaction transform unit 180 transforms the interaction object and interaction determined in the interaction determination unit 170 into an interaction with the interaction object corresponding to the predetermined operation mode. The mode of operation is a mode for providing a variety of expression forms as well as a shape for an interactive object, with appropriate interactions for an interactive object, even for the same interaction contents.

For example, if the user is male and the mode of operation is set to spouse mode, the interactive object may be changed to a female of similar age to the user, and the metabolism by the determined interaction may be referred to as " Can be changed by the interaction of dialogue such as "Honey, come on."

In an alternate embodiment, if the interaction transformer 180 fails to transform the interaction with the interaction object to correspond to a predetermined mode of operation, then the server device (see 200 of FIG. 1) Action objects, interaction and action mode information. And, the server device (see 200 in FIG. 1) for expressing the interaction can receive the interaction with the converted interaction object to correspond to the operation mode.

The interaction expression unit 190 expresses the interaction with the interaction object determined by the interaction determination unit 170 using preset user output devices (see 320_1 to 320_n in FIG. 1).

Here, the interactive object can be expressed by outputting the appearance of the object using the image information, and the metabolism among the interaction can be expressed by outputting the acoustic information. Expression information related to the interaction can be expressed by image, sound, vibration, etc., and desired interaction can be expressed using user output devices (refer to 320_1 to 320_n in FIG. 1) supporting each output.

For example, when the user sets the AR device and various audio devices as user output devices for interaction, the AR device can output an interactive target object based on augmented reality technology. The sound information for the interaction can be implemented by outputting sound using an audio device located close to the interaction target object in association with the position of the interaction target object.

In an alternate embodiment, the interaction expression unit 190 may interact with the transformed interacting object to correspond to a predetermined mode of operation in the interaction transform unit 180 using predetermined user output devices (320_1 to 320_n ) Can be used.

For example, when the user sets the audio device as an output device for user interaction with the audio device and the operation mode is set to the dog mode, the AR device outputs the image of the dog based on the augmented reality technique, This barking sound can be output by using AR device or audio device.

3 is a block diagram showing an example of the internal configuration of the server apparatus for expressing the interaction shown in FIG.

3, a server apparatus 200 for representing an interaction according to an exemplary embodiment of the present invention includes a server communication unit 210, a database 220, an information collection unit 230, and an interaction determination auxiliary unit 260 ) And the like.

In addition, the server device 200 for expressing the interaction according to an embodiment of the present invention may further include a status information generation assisting unit 240.

In addition, the server apparatus 200 for expressing an interaction according to an embodiment of the present invention may further include a scenario selection auxiliary unit 250. [

In addition, the server apparatus 200 for expressing an interaction according to an embodiment of the present invention may further include an interaction conversion assisting unit 270.

In detail, the server communication unit 210 provides a communication interface required to transmit a transmission / reception signal between the server apparatus 200 for expressing an interaction and the client apparatus 100 for expressing an interaction.

Here, the server communication unit 210 may be a device including hardware and software necessary for transmitting / receiving signals such as a control signal or a data signal through a wired / wireless connection with other network devices.

The database 220 stores information on accumulated interaction scenarios, interaction objects, interaction, user behavior, and correlation of each information from the client devices 100 for expressing various interactions .

The information collecting unit 230 may collect information on an interaction scenario, an interaction with a user, a user's action history, and the like from the client apparatus 100 for expressing various interactions and store the collected information in the database 220.

In an alternative embodiment, when the client device 100 for interactivity analyzes the user's daily information and fails to generate the user state information, the state information generation auxiliary unit 240 may request the generation of the user state information from the database 220 The user state information can be generated by analyzing the user's daily information received from the client device 100 for expressing the interaction using the information stored in the client device 100. [ Then, the generated user state information is transmitted to the client device 100 for expressing the interaction.

In an alternative embodiment, the scenario selection sub-unit 250 may include a database 220 that, when failing in selecting an interaction scenario corresponding to user state information in the client device 100 for interactivity, The interaction scenario corresponding to the user state information received from the client device 100 for expressing the interaction using the information stored in the interaction scenario can be selected. Then, the selected interaction scenario is transmitted to the client device 100 for expressing the interaction.

If the interaction determination auxiliary unit 260 fails to determine the interaction with the interaction object corresponding to the user action based on the interaction scenario in the client device 100 for expressing the interaction and instead requests the determination, The information stored in the database 220 can be used to determine interaction with the interaction scenario corresponding to the user behavior and the interaction object received from the client device 100 for representing the interaction. Then, it transmits the determined interaction object and the interaction to the client device 100 for expressing the interaction.

In an alternate embodiment, the interactive transformation assistant 270 may, in the event that the client device 100 for rendering the interaction fails to transform the interaction with the interaction object to correspond to the mode of operation and instead requests a transformation, The client device 100 may convert the interaction with the received interaction object corresponding to the operation mode received from the client device 100 for expressing the interaction using the information stored in the client device 220. [ Then, the client device 100 transmits the interaction with the converted interaction object to the client device 100 for expressing the interaction.

The server device 200 for expressing the interaction expresses the individual interaction because the scenario information, the interaction information, and the history information about the user's actions are accumulated in the client devices 100 for representing various interactions. As compared with the client apparatuses 100 for performing the authentication. Accordingly, the client device 100 can support more situations than the client device 100 for expressing the interaction and can support the client device 100 for expressing the interaction.

4 is an example of an augmented reality-based interactive image output to a user output device according to an embodiment of the present invention.

Referring to FIG. 4, when the user output device for expressing the interaction is the AR device, the interactive object 40a may be represented in the home of the user, have.

The example shown in FIG. 4 shows that the operation progress mode is set as the adult female mode or the wife mode, and the interaction progress object 40a is a young adult woman. Then, the interactions 40b when the user leaves the home and returns to the home are identified by expressing the pattern of the commute of the user so as to correspond to the young adult woman as the interaction progress object 40a.

The dialogue or dialogue interaction, such as interaction 40b, can be output as a sound using sound output devices, as letters through which the dialogue can be displayed through video output devices, or simultaneously with sound and text have.

5 is another example of an augmented reality-based interactive image output to a user output device according to an embodiment of the present invention.

Referring to FIG. 5, when the user output device for expressing the interaction is the AR device, the interactive object 50a may be represented in the home of the user, have.

In the example shown in FIG. 5, the operation mode is set to the dog mode or the dog mode, and the interaction progress object 50a is a puppy. The interactions 50b when the user leaves the home and returns to the home are identified by expressing the pattern of the commute of the user so as to correspond to the dog as the interaction progress object 50a.

The dialogue or dialogue interaction, such as interaction 50b, may be output as a sound using sound output devices, as letters through the image output devices, or simultaneously as sound and text have.

6 is an operational flowchart illustrating a method for representing an interaction performed by a client device for representing an interaction according to an embodiment of the present invention.

Referring to FIG. 6, a method for representing an interaction according to an exemplary embodiment of the present invention includes a client device (see 100 in FIG. 1) To 310_n) (S601).

Here, the collection of the user's daily information may be performed every predetermined collection period or at any time.

In addition, a method for representing an interaction according to an embodiment of the present invention includes analyzing collected user daily information to generate user state information (see 100 in FIG. 1) for expressing an interaction S603).

Here, the generated user state information may be stored together with time information.

In addition, a method for representing an interaction according to an embodiment of the present invention determines whether an interaction is started based on user state information (S 1005 in FIG. 1) ).

If it is determined in step S605 that the interaction is not started, the process returns to step S601 and the user routine information is collected again.

If it is determined in step S605 that the time for starting the interaction is correct, an interaction scenario corresponding to the user state information is selected (S607).

In addition, a method for representing an interaction in accordance with an embodiment of the present invention includes the steps of a client device (see 100 in FIG. 1) for representing an interaction, The object and the interaction are determined (S609).

In addition, a method for representing an interaction in accordance with an embodiment of the present invention includes the steps of a client device (see 100 in FIG. 1) for expressing an interaction to determine a determined interaction object and an interaction, Into an interactive object and an interaction (S611).

In addition, a method for representing an interaction according to an embodiment of the present invention includes a client device (see 100 in FIG. 1) for expressing an interaction, (320_1 to 320_n in FIG. 1) (S613).

In addition, a method for expressing an interaction according to an embodiment of the present invention determines whether a client device (see 100 in FIG. 1) for expressing an interaction has received a termination command from a user (S615). When the user is to transmit the termination command, a procedure for terminating the interaction with the user proceeds.

If it is determined in step S615 that the end command has not been received from the user, the flow returns to step S609 to determine the interaction object and the interaction corresponding to the user's action based on the interaction scenario.

If it is determined in step S615 that the end instruction is received from the user, the end interaction is converted into the end interaction corresponding to the predetermined operation mode (S617).

In addition, a method for representing an interaction according to an embodiment of the present invention includes a client device (see 100 in FIG. 1) for expressing an interaction, a predetermined user output device (see 320_1 to 320_n in FIG. 1) (S619). ≪ / RTI >

In an alternative embodiment, in the steps S601, S603, S605, S607, S609, S611, S613, S615, S617 and S619, the step of collecting user's daily information (S601) (S603) may be performed in parallel.

In an alternative embodiment, in the steps S601, S603, S605, S607, S609, S611, S613, S615, S617 and S619, the step of determining interaction with the interaction object (S609) The step of converting the interaction (S611) may be performed in parallel.

FIG. 7 is a flowchart illustrating an example of the step S609 of determining an interaction object and an interaction shown in FIG.

7, the determination step S609 of the interaction object and interaction shown in Fig. 6 is performed by the interaction determination unit (see 170 in Fig. 2) of the client apparatus (see 100 in Fig. 1) ) Attempts to determine interaction objects and interactions corresponding to user behavior based on the selected interaction scenario (S701).

The client device (see 100 in FIG. 1) for expressing the interaction can also determine a number of interacting objects and interactions, but can not determine that they correspond to the behavior or state of all users. Therefore, when the interaction object and the interaction can not be determined through the information of only the client device (see 100 in FIG. 1) for expressing the interaction, the server device (see 200 in FIG. 2) Intervention is needed.

In addition, the determination step (S609) of the interactive objects and the interaction shown in Fig. 6 is performed when the interaction determination unit (see 170 in Fig. 2) of the client apparatus It is determined whether the operation object and the interaction have been successfully determined (S703).

As a result of the determination in step S703, if the determination of the interaction object and the interaction is successful, the mutual determination unit (see 170 in FIG. 2) of the client apparatus (see 100 in FIG. 1) Action objects and interactions are used in subsequent steps.

As a result of the determination in step S703, if the determination of the interaction object and the interaction fails, the interaction scenario and the user behavior are transmitted to the server device (refer to 200 in FIG. 1) for expressing the interaction (S705).

In addition, the determination step (S609) of the interactive objects and the interaction shown in Fig. 6 is performed when the interaction determination unit (see 170 in Fig. 2) of the client apparatus (See FIG. 3) of the server device (see 200 in FIG. 1) for expressing the action (S707).

FIG. 8 is a flowchart illustrating an example of the step S611 of converting the interaction with the interactive object shown in FIG.

Referring to FIG. 8, the step of converting the interaction with the interaction object shown in FIG. 6 (S611) includes an interaction conversion unit (180 in FIG. 2) of the client apparatus (S801) attempts to convert the interaction with the determined interaction object into an interaction with the interaction object corresponding to the predetermined operation mode.

The client device (see 100 in FIG. 1) for expressing the interaction may also convert the interaction objects and interactions to correspond to the various modes of operation, but not all of the operation modes and the interaction objects and interactions none. Therefore, when the interactive object and the interaction can not be converted through the information of only the client device (see 100 in FIG. 1) for expressing the interaction, the server device (see 200 in FIG. 2) Intervention is needed.

In addition, the step of converting the interaction with the interactive object shown in Fig. 6 (S611) includes the step of converting the interaction conversion unit (see 180 in Fig. 2) of the client apparatus (see 100 in Fig. 1) It is determined whether or not the conversion of the interaction object and the interaction has succeeded (S803).

As a result of the determination in step S803, if the conversion of the interactive object and the interaction has succeeded, the result of the conversion in the interactive conversion unit (see 180 in FIG. 2) of the client apparatus (see 100 in FIG. 1) Interactive objects and interactions are used in subsequent steps.

As a result of the determination in step S803, if the conversion of the interaction object and the interaction fails, the operation mode and the interaction object and the interaction are transmitted to the server device (see 200 in FIG. 1) for expressing the interaction (S805 ).

6, the interactive conversion unit (see 180 in FIG. 2) of the client apparatus (see 100 in FIG. 1) for expressing the interaction is a mutual conversion unit (See FIG. 3) of the server device (see 200 in FIG. 1) for expressing the action (S807).

FIG. 9 is an operational flowchart illustrating an example of generating the user status information (S603) shown in FIG.

Referring to FIG. 9, the step S603 of generating the user state information shown in FIG. 6 includes a state information generating unit (see 130 in FIG. 2) of the client apparatus (see 100 in FIG. 1) , And attempts to generate user state information by analyzing the collected user daily information (S901).

Although the client device (see 100 in FIG. 1) for expressing the interaction can analyze various user daily information to generate user status information, it can not analyze daily information of all users. Therefore, when the user state information can not be generated through information of only the client device (see 100 in FIG. 1) for expressing the interaction, it is necessary to intervene the server device (refer to 200 in FIG. 2) Do.

The step S603 of generating the user state information shown in Fig. 6 includes a step of generating a state information generating unit (see 130 in Fig. 2) of the client apparatus (see 100 in Fig. 1) (S903). ≪ / RTI >

As a result of the determination in step S903, if the generation of the user state information is successful, the user state information generated in the state information generation unit (refer to 130 in FIG. 2) of the client apparatus (refer to 100 in FIG. 1) Are used in later steps.

If it is determined in step S903 that the generation of the user state information is unsuccessful, the user routine information is transmitted to the server device (refer to 200 in FIG. 1) for expressing the interaction (S905).

The step S603 of generating the user state information shown in FIG. 6 may also include a step S604 of generating the state information of the client device (see 100 in FIG. 1) (See FIG. 3) of the server device (see 200 in FIG. 1) for representing the user information (S907).

10 is an operation flowchart showing an example of a step S607 for selecting an interaction scenario shown in Fig.

Referring to FIG. 10, the step of selecting the interaction scenario shown in FIG. 6 (S607) includes a scenario selecting unit (see 160 in FIG. 2) of the client apparatus (see 100 in FIG. 1) An attempt is made to select an interaction scenario corresponding to the user state information (S1001).

Several interaction scenarios can also be selected in the client device (see 100 in FIG. 1) for expressing the interaction, but the corresponding interaction scenario can not be selected for all user status information. Therefore, when the interaction scenario can not be selected through information of only the client device (see 100 in FIG. 1) for expressing the interaction, it is necessary to intervene the server device (refer to 200 in FIG. 2) Do.

The step of selecting the interaction scenario shown in Fig. 6 (S607) includes a step of selecting a scenario selecting section (see 160 in Fig. 2) of the client device (see 100 in Fig. 1) It is determined whether or not the selection is successful (S1003).

As a result of the determination in step S1003, if the selection of the interaction scenario is successful, the interaction scenario selected in the scenario selection section (see 160 in FIG. 2) of the client device (see 100 in FIG. 1) It is used in the following step.

As a result of the determination in step S1003, if the selection of the interactive scenario fails, the user status information is transmitted to the server device (refer to 200 in FIG. 1) for expressing the interaction (S1005).

In addition, the step of selecting the interaction scenario shown in Fig. 6 (S607) may include the step of selecting a scenario (see 160 in Fig. 2) of the client device (see 100 in Fig. 1) (See 250 in FIG. 3) of the server apparatus (see 200 in FIG. 1) for receiving the interactive scenario (S1007).

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

Accordingly, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all ranges that are equivalent to or equivalent to the claims of the present invention as well as the claims .

1: System for expressing interaction
100: client device for expressing interaction
110: client communication unit 120: daily information collecting unit
130: status information generating unit 140: status information storing unit
150: Interaction time point determination unit 160: Scenario selection unit
170: interaction determination unit 180: interaction conversion unit
190: Interaction expression unit
200: server device for expressing interaction
210: server communication unit 220: database
230: information collecting unit 240: state information generating assistant
250: scenario selection sub-unit 260: interaction determination sub-
270: interactive conversion assistant
310_1 to 310_n: User input devices
320_1 to 320_n: User output devices

Claims (1)

A state information generation unit for analyzing user daily information and generating user state information;
A scenario selector for selecting an interaction scenario corresponding to the user state information;
Determining interaction objects and interactions using the selected interaction scenarios and, if the interaction objects and interactions are not determined, determining interactions objects and interactions in the server device for expressing the interactions An interaction determining unit that replaces the received interaction object with the received interaction object; And
An interaction expression unit for expressing the interaction object and the interaction through at least one or more output devices of the user;
A client device for communicating with the client device;

Images