JPH113249A - Operability evaluation support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate the experiment of evaluation while securing the correspondence between the speech data and the action history of an examine by preparing an input means recording part and a speech recording part. SOLUTION: The operating results of an operator are inputted to an operating system 108 of an information evaluation device 105 of a personal computer, etc., as the character and coordinate information. An event recording part 114 monitors the data transferred between the system 108 and a user interface program 107 and stores the operations of a keyboard 102 and a mouse 103, i.e., the occurrence of events as an operation history or an event file. A microphone 104 collects the speech of the operator, and these collected speech is stored in a speech recording part 115 via a sound driver 112 and the system 108.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an operability evaluation support apparatus for evaluating operability and supporting design in developing a user interface for an information processing apparatus using a window system or the like.

[0002]

2. Description of the Related Art With the spread of advanced information processing apparatuses such as personal computers (hereinafter abbreviated as PCs), not only the basic performances such as a processor and storage capacity, which are mainly hardware performances, but also Man & Software
There has been a demand for ease of use of a machine interface (hereinafter abbreviated as MMI). However, although an easy-to-use MMI is desired, the method of specifying the easy-to-use MMI is ambiguous, and the current situation is that it is being developed by repeated design through trial and error. What is this repetitive design?
First, the MMI was designed, then a prototype was produced, and then the prototype was used by the subjects to find problems, and the next design and development were performed to improve this point, and subsequent evaluations This is a way to check if the point has been resolved. In such repetitive design, design, prototype,
The shorter the evaluation loop, the better, and in actual development sites, there are time constraints such as delivery dates, and it is particularly desired that evaluation be performed quickly.

In this evaluation, the most important issue is to efficiently extract the problem of the target MMI. In a method in which an evaluation expert examines a prototype without using a subject and extracts a problem, heuristic evaluation (Heuristic Evaluation) and cognitive walkthrough (CognitiveW) are used.
methods (eg, J. Nielsen et al .: “Heuristi).
c Evaluation of UserInter
face ", CHI'90 Conference P
rosedings and C.I. Lewis et al .: “Test
ing a WalkthroughMonthodo
logy for Theory-Based Des
Ign of Walk-up-and-Use In
terfaces ", CHI'90 Conferen
ce Proceedings).

[0004] However, ultimately, the problem must be extracted by actually operating the user as a subject, and an experimental method called protocol analysis is usually used for this purpose.
This is done by letting the test subject operate the prototype machine while uttering his / her own thoughts, collecting the utterance contents obtained at this time as data, and then examining the thinking process during the operation against the action. It is to extract and analyze the problems of the prototype. This experiment method has the advantage that it is easy to grasp the needs of the user because the experiment can be performed by using the actual user as the subject, but it takes time to collect and listen to the utterance data, organize the data, and extract information. The disadvantage is that data analysis must be performed by experienced experts.

[0005] The outline of the experimental procedure is as follows. The experimenter uses a video camera or the like to photograph the subject using the prototype, and simultaneously collects data by recording the utterance of the subject using a sound recorder. Of course, the video and the uttered voice may be simultaneously recorded by the video camera. Subsequently, the utterance voice of the data is written down into a sentence, the sentence is divided according to the operation state of the subject and the operation state of the prototype, and the database is created by organizing the sentence according to each state. The database is then analyzed to check the subject's status by checking the utterance data for the part where the subject performed erroneous operation, and to find the part where utterance data exists, such as the operation being stuck or the operation not being understood. Then, the problem of the prototype is extracted by checking whether or not the problem is included in the state of the prototype at this time.

As described above, since the evaluation experiment must be performed quickly, a device that supports the operation of the evaluation experiment is desired. For example, Japanese Patent Application Laid-Open No.
No. 189480. This stores not only the utterance data of the operator (subject) but also the utterance data as the operator's question and the utterance data as the answer of the teacher who answers it in text, and stores this data in correspondence with the operation of the operator. It is attached and displayed to assist the experimenter in the analysis.

Further, the time at which the operation was performed must be detected. However, when a video camera or the like is used to record the operation history of the subject, the experimenter measures the recorded image while monitoring the recorded image. It is difficult to find a proper value, and the operation takes a long time. In order to solve this problem, a conversation history creation device and the like have been devised. For example, JP-A-2-41531 and JP-A-5-4059.
No. 3 publication. These are equipped with a mechanism to record the user's operation in the prototype itself and automatically collect data,
Later, by using the data to reproduce the operation history of the subject, it is possible to easily find the erroneous operation part.

[0008]

However, the above-mentioned prior art has the following problems when it is realized.

First, in the utterance data processing apparatus, the above-mentioned conventional example has a problem that the experimenter must write down the text while listening to the utterance voice data in advance and create the data, which requires a lot of trouble in creating the database. Was.

[0010] Further, when a device for recording a user's operation history is used by the prototype device itself, there is no problem in a case where the device alone collects only user operations such as a keyboard operation and a mouse operation. When used for analysis, utterance data of the subject is also recorded. At the time of analysis, it is difficult to correlate the utterance data of the subject with the motion of the subject, and there has been a problem that it is not possible to easily indicate which utterance was performed in which motion.

Therefore, the present invention solves the above problems,
An object of the present invention is to provide an operability evaluation support device that can easily perform an evaluation experiment while associating utterance data of a subject with a motion history of the subject.

[0012]

In order to achieve the above object, an operability evaluation support apparatus according to the present invention comprises an input device group such as a keyboard and a mouse, an event recording unit, a voice recording unit, and a processing unit. When the operator operates the device to be evaluated with the input device, the event recording unit records an event consisting of the time at which the operation was performed, the type of the input device, the operation content, and the like. Records the uttered voice of the operator, transmits the time information when the recording was performed to the event recording unit and records it as an event, and the processing unit displays and organizes the obtained information such as events. It is what was constituted.

Thus, an operability evaluation support device that can easily perform an evaluation experiment while associating the utterance data of the subject with the operation history of the subject is obtained.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention relates to an operability comprising at least an input means, an input means recording section for detecting an operation state of the input means and storing the information, and a voice recording section for recording voice. When the input means is operated, the input means recording unit records the time when the operation is performed, the identifier of the input means, and the operation method in association with each other, and records the voice. When a voice is input, the recording unit has an operation of recording the time when the voice input is performed and the voice in association with each other.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, FIG. 1 is a configuration diagram of an operability evaluation support device according to an embodiment of the present invention. Reference numeral 102 denotes a keyboard as input means, and reference numeral 103 denotes a mouse as input means, which are operated by an operator. The operation results of the operator are input from these devices as character information and coordinate information to the information processing apparatus 105 such as a personal computer.
In the information processing apparatus 105, an operating system 108 as software is operated. The signals from the keyboard 102 and the mouse 103 described above are input to the operating system 108 via the keyboard driver 110 and the mouse driver 111, respectively. By operating the application program 106 on the operating system 108, the information processing device 105 can be used as a device such as a word processor. However, the operation result of the keyboard 102 and the mouse 103 operated by the operator, that is, character information and coordinate information The application program 106 cannot be operated as it is. Therefore, the user interface program 107 is added to perform a process of converting the operation result into a command system that can be executed by the application program 106. Application program 1
06 from the operating system 108
Is converted into a video signal by the display driver 109 and displayed on the display 101, and presented to the operator as visible information.

Reference numeral 113 denotes an evaluation support program which is the main component of the present invention, which is also executed on the operating system 108 in principle. Here, the event recording unit 114 serving as an input unit recording unit includes the operating system 108 and the user interface program 1.
Monitor data between the keyboard 102 and the mouse 1
If an operation such as 03 has been performed (an event has occurred), this is indicated in the operation history (or event file).
Stored as Reference numeral 104 denotes a microphone,
Record the voice of the operator. The recorded voice is stored in the uttered voice recording unit 115 via the sound driver 112 and the operating system 108. Although omitted in the drawings for simplicity, the microphone 1
The output from the device 04 is converted into a digital signal by an A / D converter and then output to the sound driver 112.

When evaluating the operability of the prototyped program, the application program 106 and the user interface program 107 are replaced with the program to be evaluated and operated.

FIG. 2 shows an example of recording an operation history in an event file. Time is the time when the event occurred,
That is, the time at which the operation was performed. Here, the elapsed time from the start of execution of the application program 106 or the evaluation support program 113 is recorded as an integer in units of 1/10 second. Subsequently, the window ID is an identifier of a window on which an operation has been performed, and this number corresponds to an identification number assigned by the operation system 108 when the window is created. An event is an identifier of an operation device and its method, and expresses an operation by an operator in combination with a subsequent parameter.

For example, WM_MOUSEMOVE indicates that the mouse is moved, 0000 of the parameter indicates the state of the key when the mouse is moved, 440 indicates the value of the X coordinate of the movement destination, and 376 indicates the value of the Y coordinate of the movement destination. . Similarly, WM_LBUTTONDOWN indicates an event in which the left button of the mouse is pressed, and WM_LBUTTONUP indicates an event in which the right button of the mouse is pressed. The event ALOUD is an event indicating that the uttered voice of the operator has been input from the microphone 104, and the parameter 00000001 at this time is an identifier of a file in which the recorded uttered voice is recorded. Therefore, the event file allows the operator to perform
You can know when and what utterance was made. The time series of the data obtained by A / D conversion of the input voice may be recorded as the parameter of the event indicating that the uttered voice has been input. However, as described above, the input voice is stored as a separate file. If an identifier such as a file name is used and the address of the audio file is referred to by the identifier, the format of the event file does not need to be complicated.

After recording the event, a file is created by converting the audio content into a character string, the file name is stored as a parameter as an identifier, and the character string is displayed as appropriate. For example, the contents can be understood without reproducing the sound in the later analysis and listening to the contents. At this time, in order to convert the content of the voice to a character string, the experimenter listens to the voice and inputs it as a character string, but if a voice recognition device that performs automatic conversion from voice to a character string is available, Of course, you can use it.

Incidentally, if events are recorded and stored in chronological order, the operation history of the operator will be expressed as it is, which is effective for later analysis.

Regarding the recording of the utterance voice of the operator, the application program 106 is executed and recorded continuously from the start of the operation of the operator. As a result, one continuous voice file is generated. After that, the experimenter must transcribe the utterance contents into text while playing back the file, divide it into a series of units, add time information to each, and perform data reduction, which is a conventional protocol analysis It is almost the same as the work in the experiment. If the data can be collected by automatically dividing the voice into fixed lengths and adding time information to each voice, a significant reduction in work can be expected.

FIG. 3 is a flow chart showing the operation of the utterance voice recording unit 115 when collecting data while dividing the utterance voice of the operator. The processing will be described below with reference to the drawings.

First, as initialization of two variables, counter = 0 and T1 = 0 are executed (step 1, S1 in the figure).
And the same shall apply hereinafter). Subsequently, the amplitude value of the currently input voice is detected, and it is checked whether or not the amplitude value is equal to or larger than a predetermined value A (step 2). If it is equal to or greater than A, it is determined that the sound has been started, and the recording of the sound data in the file is started (step 3), and the time at this time is stored as T1 (step 4). If the amplitude value is less than A, repeat step 2 and wait for the sound to start. After step 4, it is checked again whether the amplitude value is equal to or greater than A (step 5), and while the counter is kept at 0 (step 6), recording of the sound is continued (step 7), and the amplitude value of the sound is smaller than A. Continue until it becomes. If the amplitude value of the sound becomes smaller than A, the sound may have been interrupted. Therefore, the value of the counter is increased by 1 (step 8), and it is checked whether or not the value of the counter is equal to or greater than T_OFF (threshold value for determining a sound separation) (step 9). Here, the value of the counter is a predetermined value T_OF.
If it is less than F, it is not determined that the sound has been interrupted yet,
Return to step 7 and continue voice recording.

If the state where the amplitude value of the voice is less than A continues in step 5, the value of the counter eventually becomes equal to or greater than T_OFF in step 9. At this time, it is determined that the voice has ended, and the voice recording ends (step 10). Here, an audio file having a certain time length is created, and it is checked whether or not the audio length of this file is equal to or greater than DUR (a threshold value for determining validity / invalidity of a predetermined file) (step 11). , DUR or more, the information of the sound start time T1 recorded in step 4 is added and the file is stored (step 12), and the entire process is terminated. If the voice length is less than the DUR in step 11, this file is discarded because there is a high possibility of the operator's stagnation, coughing, or noise around (step 13).

The various constants used in this operation are:
Change to an appropriate value depending on the environment in which the
It is desirable to adjust. That is, since the value of A is a threshold value for distinguishing between the background noise and the uttered voice of the operator, the value is higher than the input voice level when the operator is not speaking and lower than the level of the uttered voice of the operator. Set to be. T_OFF is a threshold value for determining an utterance break. If this value is too large, the utterance will not be divided. File tends to be divided every time. DUR is also a threshold value for eliminating stagnation and coughing out of the operator's utterance. If this value is too large, necessary utterance data is discarded. Conversely, if it is too small, unnecessary voice data is generated. A lot of data is stored, and the work of removing these data later becomes difficult. From a series of input audio information, a voice start position and an end position are detected,
The method of storing only the uttered voice of the operator is not limited to the method described above, and another method may be used.

If the recorded audio data is not divided as desired after the data has been recorded by the above-described method, that is, if the value of T_OFF is too large, the recorded audio data is recorded in the recorded audio data. Since it is also possible to update the file by re-executing the above method, first reduce the value of A, increase the value of T_OFF, decrease the value of DUR, record the audio data with a margin, and later It is also effective to split the file by re-executing the above method. For example, if one file A is divided into a file B and a file C for division, the audio start time of the file B is the same as the audio start time of the file A.
Although the value may be substituted as it is, the audio start time of the file C must be given a value delayed by the length of the file B from the audio start time of the file A.

Note that the file may not be divided automatically as described above, but may be designated by an operator operating an input device (not shown) such as a mouse.

FIG. 4 is a screen display of an embodiment for displaying an operator's operation history and speech voice data recorded by an information recording unit (not shown) recorded by the above-described method.

Reference numeral 401 denotes a display window, which is displayed on the display 101 as a display unit. Reference numeral 402 denotes a time axis, which indicates a lapse of time in a downward direction. 403, 40
4 is a display of a mouse move event, 405 is a display of a mouse left button down event, 406 is a display of a mouse left button up event, and 407 is a display of a speech event. Reference numeral 408 denotes an utterance character string. In the case of an utterance event, the utterance character string is displayed as shown in the figure. In this way, it is convenient to find out a problem or the like pointed out by the subject during the operation. Of course, if the screen is complicated, the character string is not displayed, and only an appropriate symbol (not shown) is displayed as an identifier to the address where the character string is recorded. In such a case, the character string may be displayed by accessing the address where the character string is stored. Similarly, an appropriate symbol (not shown) as an identifier for the address where the speech voice was recorded
May be displayed.

By designating the utterance character string 408 or the symbol with the mouse cursor 409, the address at which the voice is stored is accessed to obtain voice data, which is D / A converted, and a speaker (not shown) It is also possible to output more equivalent audio data. 4
Reference numeral 09 denotes a mouse cursor, and the operator operates the mouse cursor 409 in the window 401 using a mouse. A scroll bar 410 scrolls the display range up and down by designating the upper and lower triangles of the scroll bar with the mouse cursor 409.

As described above, according to the present embodiment, the reproduction of the operation history of the software and the sound generation of the operator can be obtained in synchronization with each other, so that the problem of the operability can be easily extracted. Also, the evaluation experiment itself can be easily performed.

[0033]

As described above in detail, according to the present invention, since the operation history of the operator can be easily performed, the problem of equipment operation can be easily extracted, and the time required for the operability evaluation experiment can be shortened. Development costs can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an operability evaluation support device according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram showing an example of an event file in which an operation history of the operability evaluation support device is recorded.

FIG. 3 is a flowchart showing the operation of an utterance voice recording unit of the operability evaluation support device.

FIG. 4 is a conceptual diagram of a screen configuration showing a display example of an operation history of an operator and speech voice data on a display unit of the operability evaluation support device.

[Explanation of symbols]

 Reference Signs List 101 display 102 keyboard 103 mouse 104 microphone 105 information processing device 106 application program 107 user interface program 108 operating system 109 display driver 110 keyboard driver 111 mouse driver 112 sound driver 113 evaluation support program 114 event recording unit 115 uttered voice recording unit

Claims (14)

[Claims] 1. An input means, an input means recording unit for detecting an operation state of the input means and storing the information, and a voice recording unit for recording a voice, the input means recording unit comprising: Is operated, the time at which the operation is performed, the identifier of the input means, and the operation method are recorded in association with each other, and when the voice is input, the voice input is performed. An operability evaluation support apparatus for recording the time and the voice in association with each other. 2. The time recorded in correspondence with the operation, the identifier of the input means and the operation method thereof, and the time recorded in correspondence with the voice input and the voice, The operability evaluation support device according to claim 1, wherein the recording is performed in the order of the respective times. 3. The operability evaluation support device according to claim 1, wherein a time at which the voice input is performed and an identifier for an address at which the voice is stored are recorded in association with each other. 4. The content of the voice is expressed as a character string,
The operability evaluation support device according to claim 1, wherein a time at which the voice input is performed and an identifier for an address at which the character string is stored are recorded in association with each other. 5. The voice recording section starts operation when the amplitude value of the input voice exceeds a predetermined value, and records the time at that time as the time at which the voice input was performed. The operability evaluation support device according to claim 1, wherein: 6. The operation of the sound recording unit is terminated when the state in which the amplitude value of the input sound falls below a predetermined value exceeds a predetermined time. The operability evaluation support device according to claim 1. 7. A dividing unit, wherein the dividing unit divides the voice into two or more voices in a time axis direction, and the divided voice includes a time corresponding to a leading time of each voice. 2. The operability evaluation support device according to claim 1, wherein a time when the voice input is performed is recorded in association with the voice. 8. A dividing unit, wherein the dividing unit divides the sound into two or more sounds in a time axis direction, and the divided sound has a time corresponding to a leading time of each sound. 4. The operability evaluation support device according to claim 3, wherein a time when the voice is input is recorded in association with an identifier of an address at which the voice is recorded. 9. A dividing unit, wherein the dividing unit divides the voice into two or more voices in a time axis direction, and the divided voice includes a time corresponding to a head time of each voice. 5. The operability evaluation support device according to claim 4, wherein a time when the voice input is performed is recorded in association with an identifier to an address where a character string expressing the content of the voice is recorded. 10. The operability evaluation support device according to claim 7, further comprising a designation unit for designating a position at which the division is performed. 11. The apparatus according to claim 1, wherein the dividing unit performs the division at a position where a state in which the amplitude value of the input voice falls below a predetermined value exceeds a predetermined time. 10. The operability evaluation support device according to any one of 7 to 9. 12. An input unit, a display unit, and an information recording unit, wherein the information recording unit includes a time when the operation of the input unit is performed, an identifier of the input unit, an operation method thereof,
The time when the input of the voice is performed and the voice are stored, and the display unit displays, in the order of the time, the identifier of the input unit, the operation method thereof, and the identifier to the address where the voice is recorded. Characteristic operability evaluation support device. 13. The information recording unit according to claim 1, further comprising: a time at which the operation is recorded correspondingly; an identifier of the input means; an operation method thereof; and a time at which the voice input recorded correspondingly is performed. And an address to a character string expressing the content of the voice, information is stored in the order of each time, the display unit, in the time order, the identifier of the input means and its operation method, the content of the voice 13. The operability evaluation support device according to claim 12, wherein an identifier to an address at which the expressed character string is recorded is displayed. 14. The operation according to claim 12, wherein the display unit displays, in the order of the time, an identifier of the input unit, an operation method thereof, and the character string expressing the content of the voice. Sex evaluation support device.