JP3466143B2 - Distributed virtual environment computer system for learning foreign languages - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for acquiring communication ability in a foreign language in a distributed virtual environment realized by a computer network, and more particularly to a so-called avatar in a virtual space having a different feeling from everyday space. The present invention relates to a computer system for effectively learning the original communication function of the relevant language by allowing participants to interact with each other through.

[0002]

BACKGROUND OF THE INVENTION The most effective way to learn a foreign language is to put yourself in an environment where the foreign language is used on a daily basis. However, as a practical matter, achieving such an environment involves both economic and geographical difficulties. Moreover, it is extremely difficult to realize such an environment on a daily basis when a foreign language is acquired in a group of a certain size such as a school.

The most commonly used method in the conventional language learning is that one teacher repeatedly speaks a conversational sentence against many learners. It has been emphasized on many occasions that even if such a method is sufficient for understanding the usage of the language, it is not effective for acquiring the original communication ability of the language. Therefore, there has also been attempted a method of enhancing the learning effect by having the learners or groups of learners secondarily talk with each other in a foreign language. However, even in that case, it is often impossible to set the environment and situation that match the conversation content, such as having a shopping conversation in the classroom, so the language and the concept expressed by it are weakly linked. It was not possible to effectively acquire a foreign language because, for example, each had no own needs and became a mere list of words.

Recently, computer performance has been improved and CD-ROM (Compact Disc Read-Only Memory)
A learning system in which various stories are set in advance due to an increase in the storage capacity of a storage medium such as a DVD (Digital Video Disc), so that the learner's reaction is elicited by so-called interactive processing, and the development of the story is changed by the reaction. Is also provided.

[0005]

However, even in a system using a computer and a CD-ROM, the role of the learner is merely role play, and true communication with a purpose is not performed. After all, we can only expect learning within the range prepared in advance.

Rather, a large number of participants realize a place to live in a foreign language based on their thoughts and values,
Therefore, it is preferable that the foreign language can be efficiently acquired.

Therefore, an object of the present invention is to provide a place in which a plurality of participants can gain the experience of living in the foreign language by performing true communication for their own purpose by a computer system in a distributed virtual environment. It is to be.

Another object of the present invention is to provide a system for supporting each participant's experience of living in a foreign language through interaction between the participants, the relationship between language and the concept behind it. To provide a distributed virtual environment computer system for learning foreign languages.

[0009]

According to the first aspect of the present invention, on a computer network, a foreign language acquisition, which is connected to a computer network to realize a virtual place for living in a foreign language by a distributed virtual environment, is acquired. A distributed virtual environment computer system for a voice conversion means for converting a voice emitted by a user into an electric signal,
A voice reproducing means for converting an electric signal into a voice, an instruction generating means for detecting a physical movement of a user and generating an instruction for a distributed virtual environment, a display, a network, a voice converting means, a voice It is connected to a reproducing means, a display, and an instruction generating means, and receives information received via a network, information prepared in advance, necessary for realizing a virtual environment, and an instruction given from the instruction generating means. While maintaining a virtual environment,
A function of driving a display and a sound reproducing means to maintain the activity of a participant in a virtual environment, and giving to another computer system in response to an instruction given from an instruction generating means and an electric signal given from a voice converting means. And a computer programmed to realize a communication function for transmitting necessary information to a predetermined destination on the network, and a function for maintaining the activity of the participants in the virtual environment is realized by the distributed virtual environment computer system. Of the participant's avatar to the virtual environment that is displayed and the avatar of the user.
The instruction generating means includes means for displaying on the display the virtual environment from the viewpoints defined consecutively , and the instruction generating means recognizes the motion of the user's hand by image processing, and performs a predetermined motion on the avatar of the user. And means for generating an instruction to The computer is further programmed to display on the display a presentation specified by any of the participants in the virtual environment to be presented to other participants in the virtual environment. This computer also
The image representations are programmed to display on a display in a form as viewed from the perspective of different single <br/> constant the viewpoint of the user's avatar.

According to the first aspect of the invention, the instruction generating means recognizes the motion of the user's hand, recognizes the intention of the user, and generates an instruction for the action of the avatar according to the recognition. The user does not have to take his eyes off the display in order to give an instruction to the avatar, and can give the required instruction with a simple hand operation. In addition, participants will be animated on the display with avatars,
The voice conversion means and the voice reproduction means allow the participants to talk in a foreign language and form a community in which the participants can interact in the foreign language in a distributed virtual environment. The distributed virtual environment is an extension of the real world, but on the other hand, it is assumed that all the participants speak the same foreign language, so it is an environment distinct from the real world, and users are It has the property that it is easy to try to interact with others using only foreign languages. Therefore,
By using this distributed virtual environment computer system to participate in a community in a distributed virtual environment and to act in the same manner as reality by an avatar, learning of a foreign language can be performed more efficiently. Further, in this system, the presentation object designated by the participant in the virtual environment can be presented to other participants. In addition to communicating with other participants in the virtual space, the user recognizes the objects presented by the other participants and associates the knowledge in the foreign language acquired by communication with the objects presented. Will be easier to do. As a result, the concept behind the foreign language can be easily understood in the virtual space, and the foreign language can be efficiently acquired.
The virtual environment is a visual environment defined in relation to the user's avatar.
While the presentation is presented from a point, the presentation is for all participants a fixed viewpoint that is different from the viewpoint of each participant's avatar.
It is displayed as seen from . Unlike the real world, all participants can see the same presentation in the same way and communicate in that language in a foreign language. As a result, it is easier to relate the foreign language and the concept behind it in relation to the presentation,
It becomes possible to understand.

In the distributed virtual environment computer system for foreign language learning according to the second invention, in addition to the configuration of the first invention, the means for generating an instruction recognizes the motion of the user's hand. And means for generating an instruction for causing the user's avatar to have a predetermined facial expression.

According to the second aspect of the invention, the avatar can be made to have a predetermined expression by the motion of the user's hand.
The user can negotiate with other participants' avatars with his / her own avatar's expression without taking his eyes off the display, which facilitates communication with other participants in the virtual space. As a result, it is possible to easily communicate with other participants in a foreign language in the virtual space, and efficiently learn the foreign language.

In a distributed virtual environment computer system for foreign language acquisition according to a third aspect of the invention, in addition to the configuration of the first aspect of the invention, the means for generating an instruction recognizes the motion of the user's hand. And a means for generating an instruction for moving the viewpoint when the image of the virtual environment displayed on the display is generated.

According to the third invention, the viewpoint at the time of generating the image of the virtual environment can be moved by the motion of the hand of the user. The user does not take their eyes off the display,
The viewpoint of the virtual environment can be changed, and the positional relationship between other participants in the virtual space and their relative positions to themselves and the environment can be easily understood. Communication with other participants and recognition of things existing in the virtual environment become easier. As a result, it is possible to easily communicate with other participants in a foreign language in the virtual space, easily understand the concept behind the foreign language, and efficiently learn the foreign language. .

In a distributed virtual environment computer system for foreign language acquisition according to a fourth aspect of the invention, in addition to the configuration of the first aspect of the invention, the means for generating an instruction recognizes the motion of the user's hand. And a means for generating an instruction for changing the viewing angle when the image of the virtual environment displayed on the display is generated.

According to the fourth aspect of the invention, the viewing angle at the time of generating the image of the virtual environment can be changed by the motion of the hand of the user. The user does not take their eyes off the display,
The viewing angle of viewing the virtual environment can be changed, and the position and environment of other participants in the virtual space can be easily understood. Communication with other participants and recognition of things existing in the virtual environment become easier. As a result, it is possible to easily communicate with other participants in a foreign language in a virtual space, easily understand the concept behind the foreign language, and learn the foreign language efficiently. .

[0017]

[0018]

In a distributed virtual environment computer system for foreign language acquisition according to a fifth aspect of the invention, in addition to the configuration of the first aspect of the invention, the means for generating an instruction recognizes the motion of the user's hand. And a means for generating an instruction to switch the image displayed on the display between the image of the virtual environment and the enlarged display of the presentation.

According to the fifth aspect of the invention, the user can instruct the user to switch between the image of the virtual environment and the enlarged display of the presentation by keeping the eyes on the display by the action of the hand. Representations can be made to consider in more detail, it is easy to the association between knowledge and its presentation of a foreign language more obtained Communication. As a result, the concept behind the foreign language can be easily understood in the virtual space, and the foreign language can be efficiently acquired.

[0021] The sixth Distributed Virtual Environment computer system for foreign language acquisition in accordance with the invention of, in addition to the inventions of the configuration of the fifth, the computer, along with the avatars of the participants in the image of the presentation material in the virtual environment It is further programmed to display.

According to the sixth aspect , since the image of the presentation item is displayed together with the avatars of the participants, the virtual environment becomes more realistic and an environment suitable for learning a foreign language can be provided.

[0023]

[0024]

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION An English learning system according to an embodiment in which the present invention is applied to English learning in a university class will be described below. However, the present invention can be applied not only to acquisition of English but also to acquisition of foreign languages in general. In addition to foreign language acquisition, it can be applied to general acquisition systems where individuals can be expected to "grow" in some sense by participating in dialogue with other individuals. it can.

First, the features of the system of this embodiment will be briefly described. Regarding the matters described below,
This will be described in more detail later.

[What is a distributed virtual environment? ] A distributed virtual environment is a networked computer system in which people at each terminal share a virtual space on the network, gather in this space, work together, and communicate with each other. , Say the environment in which you make friends and live. As described below, the English acquisition system of this embodiment is realized by a distributed virtual environment.

In this system, each participant participates in a virtual place with an animation mainly in the shape of a person called an avatar. However, even if the figure is an avatar, it is the live humans (participants, learners) in front of each terminal who participate in the work. Each participant is responsible for his / her behavior in a virtual place even after returning to the real world.

[Structure of Distributed Virtual Environment] The distributed virtual environment is realized by the following mechanism. First, each terminal has virtual three-dimensional data. Each terminal uses this three-dimensional data and constructs a virtual field image from the viewpoint of its own avatar. At this time, animations of other avatars belonging to the same group are also drawn in the virtual space.

When the avatar in the field moves, data such as its position and posture is sent to another terminal. Each terminal determines the position and posture of a new avatar based on the sent data, and again constructs and draws the image of the field and each avatar according to the result. In this case, what is communicated between the terminals is not the video data itself for the image but a very small amount of data indicating the position, posture, etc. of the avatar. Therefore, communication traffic on the network is significantly limited, and three-dimensional images can be displayed and updated in real time.

[Participant's voice] In the foreign language acquisition system, it must be possible to hear the voices of other participants and to be able to send their own voices to other avatars. Therefore, it is assumed that voice communication is performed between the terminals in real time. Since the presence of voice must be realized in a virtual place in correspondence with the positional relationship between the speaker and the listener, the voice to be transmitted is monaural, but the terminal receiving the voice does not have a virtual presence. Left and right volume and time difference (phase difference) based on the positional relationship between the speaker and listener
Calculate and play. By performing such processing, even if only monaural sound is transmitted, it is possible to realize virtual space sound with a sense of presence.

[Concept of system construction] In the system of the present embodiment, the metaphysical aspects such as the thoughts, values, and feelings of each participant, and the content and voice of each participant speaking in the virtual space, It is up to the participants themselves. on the other hand,
The visual parts such as each participant's appearance (avatar) in the virtual space and the scene of the virtual space itself were made as simple as possible. This is because it is considered that the participants' appearance in the virtual space does not need to be detailed, as will be described later. This is also convenient for reducing the traffic on the network.

[Difficulty in acquiring a foreign language] The following are the difficulties in acquiring a foreign language in one's own country.・ I do not have the opportunity to use the foreign language I learned for true communication. This is the biggest difficulty in acquiring a foreign language.・ If you want to understand the target foreign language once it has been changed to your native language,
Communication of the expressed content is no different from what is done in the native language. The foreign language learned in this way is merely a language as knowledge, it is difficult to use it as a tool for true communication, and it cannot be acquired as a true foreign language. -To truly learn a language that can be used for communication, we need an environment in which the connection between a word and the concept behind the word is understood in the language. However, it is difficult to realize such an environment in the real world.

[Effect of Distributed Virtual Environment in Acquisition of Foreign Language] In the distributed virtual environment, the actual partner is not directly visible. Therefore, it is observed that if you get used to speaking in a foreign language in a distributed virtual environment a little, it becomes natural to speak in that foreign language in a semi-conditional manner when speaking with the other party. This tendency is the same as when a Japanese visits a foreign country, and has little to do with how familiar the Japanese is with the foreign language. In other words, a poor person speaks in a foreign language as if he were poor. This environment is useful as an environment for learning foreign languages.

In the virtual environment, since the user does not speak in his native language, it is rare to understand the conversation after it is converted into the native language. So the concept behind the words is
It becomes easy to understand by linking with the items presented on the spot. In the system of the embodiment described later, it is possible to show the presentation object in the virtual environment and use the characteristics of the virtual environment as described above to facilitate the understanding of the concept behind the expression of the language. ing.

[Meeting Master] Even if the content of the conversation is left to each participant, it may be necessary to determine the place in the virtual space where the participants meet or move all the participants to another place. is there. It is more convenient to have a person who arranges such a place selection and movement, and instructs each participant what kind of operation should be performed. Therefore, it is better to have a person who is the secretary of the avatar meeting in the virtual space. Such a person is called a meeting master here. The meeting master may be given special privileges to move other participants at one time to another location.

[System Configuration of Embodiment] The configuration of the English learning system will be described with reference to FIG. Intelligent hub 5 connected to the local area network (LAN) in the university where this system is installed
6, a server 54 that is connected to the intelligent hub 56 and manages various virtual spaces, which will be described later, and avatars that represent participants who are active in the virtual space, and a teacher terminal 60 that is connected to the intelligent hub 56. ,
First cluster 50, second cluster 52, third cluster 5 all connected to intelligent hub 56
8, a fourth cluster 62, and a fifth cluster 64. The first cluster 50, the second cluster 52, the third cluster 58, the fourth cluster 62, and the fifth cluster 64 all have similar configurations.

With reference to FIG. 2, for example, first cluster 50 includes client computer systems 72-80 connected to intelligent hub 56. All client computer systems 72-80 have the same configuration.

With reference to FIGS. 3 and 4, for example, the client computer system 72 is based on the information of the virtual space given from the server 54 and the information about the participants belonging to the same group among the groups provided in the virtual space. , A process of synthesizing a virtual space to be presented to the participant 102 and an image of the participant's avatar,
And an instruction given from the participant 102 to the server 54.
And a computer 90 that is connected to the computer 90 and that captures the movement of the hand 104 of the participant 102, analyzes and recognizes the meaning of the gesture, and outputs it as an instruction from the participant 102 to the virtual environment. 94
A monitor 92 connected to the computer 90 for displaying images of the virtual space and the avatar calculated by the computer 90, a keyboard 96 used by the participant 102 for giving some instructions to the system, and other participants. An earphone 98 for reproducing a voice transmitted from a person, a microphone 100 for converting a voice emitted by the participant 102 into an electric signal and giving the electric signal to the computer 90,
The mouse 106 is included.

The camera 94 uses an artificial retina chip. The artificial retina chip is a device having an imaging function of recognizing an image projected on a surface at high speed like a retina of a living being, and a function of processing an obtained image signal as needed. The artificial retina chip is a CCD (Charge-Coupled Devic), which has been used in the past, because it can detect the movement of the image by performing image processing at the same time as image detection on one chip
different from e).

By using the camera to give instructions from the participants to the system (client computer) in this way, the system can be operated without keeping an eye on the monitor 92. Moreover, since the movement of the hand, which is natural to humans, is used, it is possible to reduce the discomfort in the virtual environment.

Referring to FIG. 4, the computer 90 has a C
PU 110, memory 112, fixed disk 114
And a FD drive 116 in which a flexible disk (FD) 140 is mounted, and a CD-ROM (Compact).
Disc Read-Only Memory) 14
2 includes a CD-ROM drive 118, a sound board 120 to which the earphone 98 and the microphone 100 are connected, and a network adapter 130 for connecting the computer 90 to the network.

Since the configuration of server 54 and teacher terminal 60 shown in FIG. 1 is similar to that of client computer system 72, detailed description thereof will not be repeated here.

Referring to FIG. 5, the server management information 150 managed by the server 54 includes group information 152 regarding groups corresponding to “rooms” or “places” prepared in the virtual space, and virtual information about each room. It includes space-related information 154 including information on various environments and the shape of the avatar corresponding to each participant.

The group information 152 includes group-specific information 156 as many as the groups prepared in this system.

Referring to FIG. 6, for example, group-specific information 156 regarding the first group includes avatar information 170 that identifies an avatar belonging to the group, and a background of "place (for example, room)" given to this group. And background information 172 that identifies the item, and item information 174 that identifies the item existing in the place given to this group.
The contents of the background information 172 and the item specifying information 174 do not necessarily have to be incorporated in the system from the beginning. These pieces of information may be prepared by others so that they can be freely exchanged as needed. Note that another part of the same place can be shared by a plurality of groups.

The avatar information 170 includes information for specifying each of the avatars 1 to na belonging to this group. The background information 172 is information for individually specifying the background nb (mountain, sea, forest, etc.) from the information background 1 that constitutes the background of the place (outdoors may be assumed) given to this group. Including. The article information 174 includes articles 1 to nc (such as room doors, desks, chairs, and lighting) that are supposed to exist near the place given to this group.
It contains information to identify each individual. Each of these pieces of information includes a pointer into the avatar shape 162, the background information 158, and the article shape 160 shown in FIG. 5, and the avatar shape, the article shape, or the background shape specified by the pointer is given to this group. It is used to create a field in a virtual space.

Note that it is not necessary to prepare highly accurate avatar shapes, article shapes, background shapes, and the like. The reason is as follows.

The place where the participants meet and the atmosphere greatly affect the output of the gathering. But on the other hand,
Visualizing every detail of the scene is not so important. In any case, humans are not looking at the visible details of things, but rather looking at things according to the image already in their head. Therefore, even if the shape of an object in the virtual space is not detailed, a human being compensates for the missing portion.

However, there may be a scene where details of things are really needed. Therefore, in the present embodiment, as will be described later, a mechanism for presenting an article is prepared, and an image of the article is displayed.
It can be presented as something other than an avatar in the virtual space.

In addition, it is wasteful to try to make the avatar look like the person himself. Avatars can be distinguished from each other, and only need to be known by the person at a glance. A person's appearance is important, for example, when identifying him or her in unexpected places, but in a limited distributed virtual environment of participants, the appearance of the avatar is not important.

Referring to FIG. 7, individual avatar information 18
0 is the avatar identifier of this avatar, the client identifier to which this avatar is assigned, a pointer to the area within the avatar shape 162 in which the shape of this avatar is stored, and the position of this avatar in the virtual space. It includes information and posture information, and information about the presentation that this avatar presented to each avatar in the group. The shape of the avatar should be independent of the program so that it can be changed freely.

Referring to FIG. 8, client management information 190 managed by each client computer is information 200 regarding the position, viewpoint (POV) and viewing angle of the avatar corresponding to itself, and a list 202 of clients in the same group. , Shape data 204 of avatars in the same group, and coordinate and posture information 2 of avatars in the same group
06, and three-dimensional shape information 208 of the space. A list 202 of clients in the same group, shape data 204 of avatars in the same group, coordinates of avatars in the same group, posture information 206, and three-dimensional shape information 208 of space are transmitted from the server when this client logs in to the system. Is coming. These pieces of information are thereafter updated according to the information that this client receives from other clients.

For example, referring to FIG. 9, when a user (client) gives some instruction to his or her avatar and the position or posture of the avatar changes as a result, the client determines the position or posture of the avatar. The information indicating the change (hereinafter referred to as "movement information") is transmitted to the server 54 (220). When the server 54 receives the movement information, the server 54 transmits the movement information together with the client identifier to other clients belonging to the same group as the client (222). Each client can maintain the position information and the posture information of the avatars corresponding to the clients belonging to the same group based on the movement information transmitted from the server 54 from each client.

When there is a change in the participants such as a new participant joining the group or a part of the participants dropping out, the information is transmitted to each participant via the server.

In the case of learning a foreign language, it is necessary to transmit the voice uttered by one participant to the client of another participant and reproduce it as a voice. Since voice transmission requires a wide band, it is difficult for the server 54 to centrally manage it. Therefore, this system delivers audio as follows.

Referring to FIG. 10, each client first asks the server 54 to log in to the system and then periodically to a client belonging to the same group as itself (to an avatar existing in the same space as itself). Inquiry (230) of the corresponding client. In response to this inquiry, the server 54 returns a list of clients (avatars) in the same group as the client that issued the inquiry (avatar information 170 shown in FIG. 6) (23).
2). The client that receives this list updates the list 202 of the clients of the same group that it has with the received list. Then, until the next inquiry is made, the voice uttered by the participant is directly transmitted to all the clients listed in the client list 202 of the same group (234).

In order to perform the above processing, the server 54
Performs processing for managing the server management information 150 shown in FIGS. 5 to 7. Since the space related information 154 is fixed, only the group information 152 needs to be maintained. This information is relatively simple because it is only necessary to update it according to the information sent from each client and send the updated information to the required clients. Further, when the space-related information is completely changed, such as when the meeting place is moved to another place, the server 54 performs a process therefor and transmits necessary change information to each client.

The client visualizes the virtual space according to the information from the server or the same client and presents it to the participants, and updates the virtual space according to the instructions from the participants and notifies the server and the clients of the same group. Send. Hereinafter, this process will be described.

FIG. 11 is a functional block diagram of the client system 240 realized by the program operating on the client computer. Referring to FIG. 11, the client system 240 includes a keyboard 96,
An input processing unit 250 for processing an input from an input device such as the mouse 106 or the camera 94 to generate a message 252 to be transmitted to the server, and the coordinates of each avatar sent from the server 54 via the network 242. ,
The virtual space and the avatars of the same group existing in the virtual space are three-dimensionally visualized based on the rotation, the posture, the spatial information, the address information, etc., and presented to the participants using the monitor 92. A visual processing unit 254 that performs a process of generating a message 256 to be transmitted to the result server 54, and a voice-related message 262 transmitted from a client of the same group via the network 242 are processed to generate an environment in a virtual space. It is connected to an audio processing unit 260 for performing reproduction of sounds and the like and reproduction of real-time audio such as voice generated by an avatar, an input processing unit 250, a visual processing unit 254, an audio processing unit 260, and a network 242, and is mainly connected to a virtual unit. Update information about changes made to the space To perform the transfer and its holding, and a transport unit 264 that communicates with a respective client of the server 54 and the same group.

Referring to FIG. 12, the program running on the client first receives a message from server 54 or a client of the same group (250).
It is determined whether the message is audio-related or visual-related (252). If the message is audio-related, control proceeds to step 254; if the message is visual-related, control proceeds to step 256.
To, respectively.

In step 254, the program passes the message to the audio processor 260. Thereafter, control returns to step 250. The audio processing unit 260 generates a voice to be presented to the participant based on the received voice signal. The voice transmitted and received on this network as a message is monaural. At the time of reproduction, the audio processing unit 260 causes the information about the three-dimensional shape of the position of the avatar corresponding to the client in the virtual space, the position of the own avatar, and the position of the avatar corresponding to the client that has transmitted the message. Based on the above, the phase and magnitude of the sound wave are adjusted to produce stereo sound. As a result, it is possible to increase the realism of the virtual space while suppressing the traffic on the network.

On the other hand, when the message is determined to be visual-related, in step 256, the information necessary for visualizing the virtual space (each avatar in the field of view, the article, The position of the background, the size, etc.) are calculated, and the calculation result is held. By giving this calculation result to the visual processing unit 254 in step 258, the visual processing unit 254 visualizes the virtual space and displays it on the monitor.

Next, a process of inputting a command from a participant by moving the hand at the client will be described. As already mentioned, the camera 9 is used for input.
4 is used.

As shown in FIG. 13, first, the outline of the hand portion of the input image is extracted by image processing of the artificial retina chip of the camera 94 (270). The following is a program process for this contour information output from the camera 94 in real time.

In step 272, the shape of the hand and the moving direction are extracted using the difference between the input images. It is determined whether there is a change in this moving direction (274), and if there is a change, the process of step 272 is continuously performed. If there is no change, then it is determined whether the movement of the hand continues (276). When the movement is continued, it is continuously determined whether or not the movement is continued, and when the movement is completed, the process proceeds to step 278. In step 278, the combination of the movement directions of the hand detected by the processing of steps 272 to 276 and the shape of the hand detected from the contour image detected in step 270 are matched with a predetermined combination. , Recognize instructions from participants and determine how to operate the avatar.

FIG. 14 shows combinations of hand movement directions or detected hand shapes and corresponding avatar actions (actions instructed by participants). As shown in FIG. 14, the actions are an action depending on the moving direction of the hand (shown by a frame 280) and an action depending on the shape of the hand (frame 2).
Shown at 82. ) And roughly divided.

The action in the moving direction of the hand shown in the frame 280 is roughly divided into an action (indicated by a frame 290) determined by one movement of the hand in a single direction.
It is divided into an action determined by continuously moving the hand (shown by a frame 292) and an action determined by two consecutive hand movements (shown by frames 294 and 296). For example, if you move your hand upwards only once, you are instructed to walk.
If you move your hand up or down, you will be instructed to "run". If you move your hand to the right once and then move it upwards, you will be instructed to perform the "jump" action. The same applies to other actions.

The relationship between the movement of the hand and the action shown in FIG. 14 is merely an example, and it goes without saying that various other combinations are possible.

In this way, controlling the movement of the avatar by the movement of the hand can be easily performed as compared with using a keyboard if used. Moreover, it is not necessary to take your eyes off the screen when giving instructions. Therefore, the participants can always concentrate on the virtual space displayed on the monitor and can effectively learn a foreign language.

One feature of the system of this embodiment is that
This is that an image selected by one participant can be presented as a presentation object to other participants in the virtual space. As the presentation item, for example, a photograph, a chart, characters, etc. can be used. Whatever can be displayed as an image can be presented as a presentation item. In the present embodiment, a window different from the window in which the virtual space is displayed is opened on the monitor 92, and the presented item is displayed there.

This presentation item corresponds to a display such as a blackboard, a photograph, a picture, an OHP, an actual object or a model used in an ordinary conference. However, what this presentation is different from the real world presentation is that in the virtual world the presentation is presented to the avatar, but it is the participants who should understand its content. Therefore, the presentation needs to be expressed in the virtual world so that it can be clearly seen by the participants. Therefore, in the system of the present embodiment,
As will be described later, for example, a window different from the window displaying the virtual world is opened in each client system, and the presentation is displayed in that window. As a result, the presentation will be presented to all participants equally. Such a display can be realized even when the presentation object is placed in the same space as the avatar in the virtual space.
This will be described further below.

FIG. 15 shows a flowchart of a program for realizing a process in which a client presents a presentation target to clients in the same group (referred to as "presentation process"). With reference to FIG. 15, when the presentation process is activated, first, a process of selecting a presentation target is executed (310). This process is implemented, for example, by selecting, in a dialog, information that specifies the shape of the article to be presented, which is prepared in advance in the fixed disk 114 or another storage device on the network in a file format.

Subsequently, a process of opening a new window and displaying the copy of the presentation target selected in step 310 in this new window is performed. At this time, the pointer information to the presentation target is transmitted to the server 54, and further transmitted to all the clients in the same group via the server 54. Each client
When this pointer is received, the file pointed to by that pointer is read from the network,
Opens a new window for display.

Further, in step 314, it is determined whether or not any event such as an instruction to move the position of the presentation target, an instruction to change the posture of the presentation target, or the like has occurred. If any event occurs, the image of the presentation target changed by the event is calculated, only the window for the presentation object is image-processed, and the presentation target is displayed again (316). If there is no event, the process of this step is not performed.

Next, it is determined whether or not a request to close the window for presenting items has been input by an instruction by the movement of the hands of the participants, or by using the keyboard 96, the mouse 106 or the like (318). If there is no request, the control returns to step 314, and the processes of steps 314 to 318 are repeated thereafter. If there is a request, the presentation object window is closed in step 320, and the presentation process ends.

When the presentation item is displayed in a window different from the window for displaying the virtual space in this manner, it is easy for each client that the presentation item is different from the item in the virtual space. It has the advantage of being understandable. Also, the process of displaying the presentation target may be simple.

The presentation itself is owned by the participant who presented the presentation, and other participants cannot change the presentation. However, it is possible for other participants to add something to the presentation (addition of text, for example). When there is such an additional item, by delivering only the additional item to each client, each client can update the display corresponding to the additional recording.

In the system of the above embodiment, since a copy of the presentation item is displayed, even if each client changes the presentation item, the original item is not affected. Therefore, not only the postscript, but all the participants can also modify the presented item, register it as a new presented item by a new presenting entity, and manage it.

Now, how the virtual space is displayed in this system will be described. FIG.
Shows a display example of the virtual space. In this display example, the viewpoint (POV) for the displayed image is the viewpoint of the participant's avatar itself. In this case, there is a problem that the field of view presented to the participants becomes narrower with respect to the expansion of the virtual space. For example, in the example shown in FIG. 16, five avatars are gathered in this space in addition to the avatars of the participants themselves, but only four of them are displayed. Moreover, in this case, the participant's avatar itself will not be presented to the participant. It has been empirically known that it is very difficult for an avatar of oneself to feel the presence of something invisible in a virtual space.

Therefore, in the system of the present embodiment, as shown in FIG. 17, it is possible to set the viewpoint at the time of image display a little behind and above the avatar of the user, and as shown in FIG. The display system and the display system shown in FIG. 17 can be switched.

When the virtual space is displayed from the viewpoint shown in FIG. 17, the participants can check their avatars on the display screen. Therefore, it is possible to strengthen the feeling that the self is in the virtual space. Furthermore, since the viewpoint is placed behind and above, the field of view expands, and all avatars of the same group gathered in the virtual space can be confirmed on the display.

Such switching of the viewpoint does not change the coordinates of the avatar itself maintained by the client, but simply sets a predetermined viewpoint with respect to the viewpoint previously set based on the avatar and the line of sight at that time. This can be easily performed by performing a vector operation to obtain a new viewpoint.

Further, in the present embodiment, the viewpoint can be set further rearward as shown in FIG. By doing this, you can see not only the avatars that are gathering but also a relatively wide area around it, for example, when the number of avatars is large, even when the avatars are dispersed in a relatively wide area. You can check the whole group of avatars.

The viewing angle can be changed in addition to the switching of the viewpoint position. Furthermore, the change of the position of the viewpoint may be automatically switched according to the number of participants as well as according to the instructions of the participants. Further, the viewpoint position may be automatically determined so that all the participants are in the visual field according to the positions of the participants. The viewing angle may also be automatically switched depending on the distance between the viewpoint and the object.

In the system of the above-described embodiment, when a participant presents a presentation to another avatar,
Be sure to open the window for the presentation target. However, the present invention is not limited to such an embodiment. For example, an area for presenting a presentation is prepared in advance in the virtual space in the form of some article (eg, blackboard, whiteboard, canvas, table, table) in the virtual space, and the presentation is displayed there. You may leave it to do so.
Alternatively, the presentation items may be prepared in the form of a two-dimensional drawing, a two-dimensional photograph, a three-dimensional model, or a three-dimensional stereoscopic photograph, and they may be presented in the form of being leaned against a wall or spread on a desk.

In this case, it is usual to display a different image on each client according to the viewpoint of each avatar, as if the presented object exists in the virtual space.

However, in the virtual space, it is not necessary to display the presentation item as such. For example, the presentation of the presentation itself is performed in the virtual space, but the viewpoint of the display is
The same viewpoint can be used for all avatars. The actual whiteboard will look different depending on where the viewer is (eg front side or back side). However, in the virtual space, it is possible to see (display) the whiteboard in exactly the same way by a person on one side of the whiteboard in the virtual space and a person on the other side.

An example of such a display is shown in FIG. FIG. 19
In the example shown in FIG. 5, five avatars 332, 334, 336, 338 and 340 are present in the virtual space display window 330. A presentation object display area (referred to as “stand”) 342 is provided in the center of the area, and the presentation object is displayed there. Here, for example, the arrangement of each avatar viewed from the avatar 332 and the avatar 336.
The arrangement of each avatar seen from is different from each other. However, the stand 342 looks the same from the avatar 332 and the avatar 336.

It is also possible to switch between the display as shown in FIG. 19 and the enlarged display of the presentation as shown in FIG. 20, for example. In FIG. 20, the stand 342 is enlarged and displayed in the virtual space display window 330, and as a result, the presentation object 344 is displayed in a more understandable manner.

Such switching of display can be easily realized by using a normal display method only by changing the position of the viewpoint. FIG. 21 shows a flowchart of a program for switching the display and displaying the presented object.

Prior to the processing shown in FIG. 21, FIG.
It is assumed that the display has already been performed in the format shown in. First, in step 350, a process of selecting a presentation target is performed. This process is similar to the process performed in step 310 of FIG. Subsequently, in step 352, a process of moving the viewpoint to near the presentation object is performed, and the copy of the presentation target selected in step 350 is displayed in high resolution. The display result at this time is shown in FIG. The pointer to the entity file on the network, which is the presentation target, is transmitted to all the clients in the same group, and all clients access the entity file to perform the same processing as step 352.

Subsequently, in step 354, it is determined whether or not any event has occurred, such as enlarging display of the presentation target, rotation of the presentation target, and additional recording processing on the presentation target. If any event occurs, in step 356, the presentation item is enlarged, rotated, or additionally recorded in response to the event, and the updated presentation item is displayed. If there is no event, nothing happens.

Next, in step 358, it is determined whether or not the participant has instructed the end of the presentation object display mode. If the end instruction has not been issued, the control returns to step 354, and the processing of steps 354 to 358 is repeated. If the end instruction has been issued, a process of returning the viewpoint to the original position is performed in step 360, and the presentation process ends.

It should be noted that such switching of viewpoints is not performed by the participant's instruction, but by observing the line of sight of the participant and when the participant pays attention to a certain part in the image, a close-up display of the part is displayed. It may be performed automatically as is done. However, in this case, for example, a device that identifies the point of interest of the participant by image recognition is required.

As described above, according to the present embodiment, communication for foreign language learning is performed in the virtual space using the avatar. By using the virtual space to learn foreign languages in this way, participants have a different feeling from their daily lives even though they are extensions of their daily lives, and actively learn conversations with other participants. Will be done in. Therefore, it is possible to learn a foreign language more efficiently as compared with a case where conversation learning is simply performed while face-to-face with other participants in the classroom.

Also, since the action of the avatar can be specified in a simple format by using the hand and it is not necessary to move the line of sight from the monitor to another, it is possible to concentrate on the virtual space displayed on the monitor, and the foreign language Learning has become more efficient.

If the client computers in the same cluster are partitioned by partitions, etc., and the adjoining participants must belong to different groups, the adjoining participants do not pass through the virtual space. You can prevent talking directly.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope.

[Brief description of drawings]

FIG. 1 is a diagram showing an overall configuration of an English learning system according to an embodiment of the present invention.

2 is a more detailed diagram of a cluster of the system shown in FIG. 1. FIG.

FIG. 3 is a diagram showing the external appearance of a computer system used in this system.

FIG. 4 is a diagram showing a hardware configuration of a computer system used in this system.

FIG. 5 is a diagram showing server management information.

FIG. 6 is a diagram showing a configuration of individual group information in the server management information.

FIG. 7 is a diagram showing a configuration of individual avatar information among individual group information of server management information.

FIG. 8 is a diagram showing a structure of client management information.

FIG. 9 is a diagram showing a communication sequence between a server and a client regarding position information of an avatar and the like.

FIG. 10 is a diagram showing a communication sequence between a server and a client regarding voice communication.

FIG. 11 is a diagram showing a functional configuration of a client system.

FIG. 12 is a flowchart of a program executed for message processing in a client.

FIG. 13 is a flowchart of a program for inputting an instruction by a hand movement using a camera in a client.

FIG. 14 is a diagram showing an action depending on a combination of hand movements and the number of fingers.

FIG. 15 is a flowchart of a program that executes a process for presenting a presentation item.

FIG. 16 is a diagram showing a virtual space drawn from the viewpoint of an avatar.

FIG. 17 is a diagram showing a virtual space drawn from a viewpoint slightly behind and above the avatar.

FIG. 18 is a diagram showing a virtual space drawn from a viewpoint behind and above the avatar.

FIG. 19 is a diagram showing a monitor display when a presentation is drawn in a virtual space together with an avatar.

FIG. 20 is a diagram showing a monitor display when a presentation object is enlarged and displayed.

FIG. 21 is a flowchart of a process of switching between a method of displaying a presentation item together with an avatar and a method of enlarging and displaying only the presentation item.

[Explanation of symbols]

54 servers, 50 to 64 clusters, 72 to 80
Client computer system, 90 computer, 92 monitor, 94 camera, 96 keyboard, 98 headphones, 100 microphone, 106
Mouse, 140FD, 142 CD-ROM, 150
Server management information.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-7-255044 (JP, A) JP-A-6-27874 (JP, A) JP-A-8-314625 (JP, A) JP-A-11- 249779 (JP, A) JP 2000-172740 (JP, A) JP 8-9665 (JP, A) JP 2000-90285 (JP, A) JP 8-329282 (JP, A) JP 2001-338077 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 1/00 G06T 11/60-17/50 G06F 3/00 G09B 1/00-9/56 G09B 17 / 00-19/26 H04N 7/10-7/22

Claims (6)

(57) [Claims] 1. On a computer network,
A distributed virtual environment computer system for learning a foreign language, which is connected to a computer network to realize a virtual place for living in a foreign language in a distributed virtual environment. A voice converting means for converting, a voice reproducing means for converting an electric signal into a voice, and an instruction generating means for detecting a physical movement of the user and generating an instruction for the distributed virtual environment. A display, and a network connected to the network, the voice conversion unit, the voice reproduction unit, the display, and the instruction generation unit, information received via the network, prepared in advance, for realizing a virtual environment. The virtual environment is maintained by processing the necessary information and the instruction given from the instruction generating means, and Another computer in response to the function of driving the play and the voice reproducing means to maintain the activity of the participant in the virtual environment, and the instruction given from the instruction generating means and the electric signal given from the voice converting means. And a computer programmed to realize a communication function of transmitting information to be given to a system to a predetermined destination on the network, and a function of maintaining the activity of the participants in the virtual environment,
It is defined in relation to the animation display of the avatar of the participant in the virtual environment realized by the distributed virtual environment computer system , and the avatar of the user.
The instruction generation means recognizes the motion of the user's hand by image processing, and performs a predetermined motion on the avatar of the user. The computer includes a presentation item to be presented to other participants in the virtual environment specified by any participant in the virtual environment. The computer is further programmed to be displayed on the display, and the computer further displays the image of the presentation on the display from a certain viewpoint different from the viewpoint of the avatar of the user. A distributed virtual environment computer system for learning foreign languages. 2. The means for generating the instruction includes a means for recognizing a motion of the user's hand and generating an instruction for causing the avatar of the user to have a predetermined facial expression. A distributed virtual environment computer system for acquiring a foreign language according to claim 1. 3. The means for generating the instruction recognizes the movement of the user's hand and generates an instruction for moving the viewpoint when the image of the virtual environment displayed on the display is generated. The distributed virtual environment computer system for learning a foreign language according to claim 1, further comprising: 4. The instruction generating means recognizes the movement of the user's hand, and generates an instruction for changing the viewing angle when the image of the virtual environment displayed on the display is generated. A distributed virtual environment computer system for learning a foreign language according to claim 1, comprising: 5. The means for generating the instruction recognizes the motion of the user's hand and displays an image displayed on the display as an image of the virtual environment and an enlarged display of the presentation. The distributed virtual environment computer system for foreign language learning according to claim 1 , including means for generating an instruction for switching between. 6. The foreign language acquisition of claim 5, wherein the computer is further programmed to display an image of the presentation with the participant's avatar in the virtual environment. Virtual environment computer system for Linux.