JP2010282083A - Incorrect recognition correction device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device etc. for efficiently correcting a speech recognition result. <P>SOLUTION: In the invention, a correction operation is composed of two operations of: (1) specifying an incorrect recognition part; and (2) correcting the specified incorrect recognition part. In the two operations, (1) specifying the incorrect operation is (2) performed by a person other than the corrector who corrects the specified incorrect part, and thereby, a burden of the corrector is reduced. For example, specifying the incorrect recognition part is performed by a hearing impaired person. The incorrect recognition part may be automatically specified by using reliability and a degree of closeness. Moreover, when the incorrect part is specified, a priority order of the incorrect recognition part is determined, and correction of the incorrect recognition part can be performed by the corrector in the order from the incorrect recognition part with higher priority. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for estimating a misrecognized portion that occurs during speech recognition and correcting the misrecognized portion.

Speech recognition means that a sound signal of speech spoken by a person is analyzed by a computer or the like and converted into character data. A certain degree of error is unavoidable in the conversion, and a misrecognized portion that is a part including a conversion error occurs.
Conventionally, in order to correct misrecognized parts, one corrector visually identifies the sentence as the voice recognition result, identifies the misrecognized parts, and corrects all the identified misrecognized parts in the specified order. (See, for example, Non-Patent Document 1 and Non-Patent Document 2.)

Masayuki Kobayashi, Shun Nishikawa, Yoshi Ishihara, “Real-time caption insertion system using speech recognition for the hearing impaired (1)”, IEICE Tech. 41-48 Satoshi Imai, “Research and Development for Information Barrier-Free in Broadcasting—Voice Recognition for Production of Live Subtitles”, Broadcast Technology, August 2008, p. 89-93

When one corrector performs the two tasks of identifying a misrecognized part and correcting the identified misrecognized part, the corrector is burdened, and the speech recognition rate is low and there are many misrecognized parts. Had the problem that corrections could not catch up.
Also, some misrecognized parts can be guessed from the context before and after, and do not require correction, but if all of the identified misrecognized parts are corrected in the order specified, The misrecognized portion that does not require correction is also corrected, and there is a problem that the correction efficiency is poor.

  In order to solve the above problem, the misrecognized location specifying unit acquires information about the misrecognized location in the sentence, the priority determining unit acquires information about the priority of the misrecognized location, and the display unit Displays the erroneous recognition location according to the priority.

By specifying a misrecognized portion by a person other than the corrector, the corrector can concentrate on correcting the misrecognized portion and reduce the burden.
The corrector can correct misrecognized portions in descending order of priority, and the correction efficiency is improved as compared with the prior art.

The functional block diagram of the example of the misrecognition correction apparatus by 1st embodiment. The functional block diagram of the example of the misrecognition correction apparatus by 2nd embodiment. The functional block diagram of the example of the misrecognition correction apparatus by 3rd embodiment. The flowchart of the example of the misrecognition correction method by 1st embodiment. The flowchart of the example of step S3 by 2nd embodiment. The flowchart of the example of step S3 by 2nd embodiment. The flowchart of the example of step S3 by 3rd embodiment. The image figure which illustrates the condition where this invention is used. Diagram for explaining the operation The figure for demonstrating a nodding operation | movement. The figure which shows the example of a display of the misrecognized location according to a priority. The figure which shows the example of a display of the misrecognized location according to a priority. The figure which shows the example of a display of the misrecognized location according to a priority. The figure which shows the example of a display of the misrecognized location according to a priority.

Hereinafter, embodiments of the present invention will be described in detail.
[First embodiment]
First, the situation where the present invention is used will be illustrated with reference to FIG. In this example, a workplace meeting is assumed in which a plurality of persons including persons with hearing impairments participate.

  There are three participants in the workplace meeting: speaker 11, speaker 12, and hearing impaired person 13. A corrector 14 who corrects a misrecognized portion is present at the workplace meeting. Microphones 21 and 22 are placed in front of the speakers 11 and 12, respectively, and the voices of the speakers 11 and 12 are collected. The microphones 21 and 22 are connected via a mixer 23 to a voice recognition PC 31 that performs voice recognition. The speech recognition PC 31, the correction PC 32 that the corrector 14 corrects, and the character output PC 33 that displays the corrected sentence of the misrecognized portion on the hearing impaired person 13 can transmit and receive necessary information. Connected to the network.

FIG. 1 is a functional block diagram of a misrecognition correction apparatus according to the first embodiment. FIG. 4 is a flowchart of the misrecognition correction method according to the first embodiment.
The voices of the speakers 11 and 12 are input to the microphones 21 and 22, respectively, and taken into the voice recognition PC 31 (step S1).
The voice recognition unit 311 of the voice recognition PC 31 recognizes the voices of the speakers 11 and 12 (step S2), and outputs a sentence as a voice recognition result to the correction PC 32 and the character output PC 33.

Information about misrecognized locations, which are locations including erroneous parts of the text, and information about the priority of each misrecognized location are acquired (step S3, step S4) and output to at least the correction PC 32. These pieces of information are generated by a person other than the corrector 14. A method for acquiring these pieces of information will be described later.
The misrecognized part is a part including an error part of a sentence, and may be an error part of the sentence itself (for example, a word) or a part including the error part (for example, a phrase, a line, or a paragraph). Also good.
The display unit 321 of the correction PC 32 displays a sentence that is a voice recognition result, a misrecognized portion, and a priority order. The corrector 14 uses the input unit 322 of the correction PC 32 to correct according to the priority order (step S5). For example, misrecognized portions are corrected in descending order of priority.

The corrected sentence is displayed on the display unit 331 of the character output PC 33 and is shown to the hearing impaired person 13 (step S6).
The display units 321 and 331 are display devices such as a CRT (Cathode Ray Tube) and a liquid crystal display. The input units 322 and 332 are input devices such as a keyboard and a mouse, for example.
There are several methods for obtaining information about misrecognized locations that are locations containing erroneous parts of text and information about priorities of each misrecognized location.

In the first embodiment, a person other than the corrector 14, for example, a hearing impaired person 13 specifies a misrecognized portion and determines its priority.
That is, the hearing-impaired person 13 looks at the sentence that is the voice recognition result displayed on the display unit 331, and specifies the erroneous recognition location by the input unit 332 (step S3). As described above, in the first embodiment, the information about the misrecognized portion is acquired by the input unit 332. That is, the input unit 332 corresponds to the misidentified part specifying unit in the claims.

The character output PC 33 displays the misrecognized part in the text displayed on the display unit 331 so that it can be distinguished from other parts. For example, the character color or background of the misrecognized portion is changed or italicized, or the character color or background or italic of the line or paragraph including the misrecognized portion is changed.
Information about the misrecognized portion is also sent to the correcting PC 32, and the correcting PC 32 distinguishes the misrecognized portion in the text displayed on the display unit 321 from other portions in the same manner as the character output PC 33. Display to be able to.

  In addition, when the hearing impaired person 13 specifies an erroneously recognized location, the hearing impaired person 13 may determine a priority that is an index indicating the degree of necessity of correction for the corrected recognized location (step S4). For example, when specifying a misrecognized location, the priority is given to any one of “(1) I want you to correct immediately”, “(2) I want you to correct” and “(3) I don't need to correct” 332 for input. Thus, in the first embodiment, information about the priority order is acquired by the input unit 332. That is, the input unit 332 corresponds to the priority order determination unit in the claims.

The character output PC 33 changes the display of the erroneous recognition location according to the priority order. For example, the color of the misrecognized portion is changed for each priority.
The information about the priority order is also sent to the correction PC 32, and the correction PC 32 changes the display of the erroneous recognition portion in accordance with the priority order in the same manner as the character output PC 33.

With reference to FIGS. 11 to 14, an example of displaying a misrecognized portion according to the priority order displayed on the display unit 321 of the correction PC 32 will be described. In this example, it is assumed that the priority is expressed in three stages: “(1) I want you to correct immediately”, “(2) I want you to correct”, and “(3) I don't need to correct”. In the examples of FIGS. 11 to 13, the misrecognized portions are all words, but the misrecognized portions may be sentence portions including words as in the example of FIG. 14.
In the example of FIG. 11, the display unit 321 displays three windows: a recognition result output window 71, a correction window 72, and a misrecognition standby window 73.

In the recognition result output window 71, a sentence that is a voice recognition result is displayed. The misrecognized part in the sentence displayed in the recognition result output window 71 is displayed in a different color so that it can be distinguished from other parts.
In the correction window 72, a misrecognized portion having the highest priority is displayed. The corrector 14 looks at the misrecognized portion displayed in the correction window 72 and inputs a correct word / phrase using the input unit 322 such as a keyboard. The misrecognized portion in the sentence displayed in the recognition result output window 71 is replaced with the input correct word / phrase.

It should be noted that a voice of a misrecognized portion displayed on the correction window 72 may be generated from a voice output unit (not shown) of the correction PC 32. As a result, the corrector 14 can hear the voice of the erroneously recognized portion, and the ease and certainty of correction increase.
In the misrecognition waiting window 73, all or a part of the specified misrecognition locations other than the misrecognition location with the highest priority are displayed in order of descending priority. For example, while the corrector 14 is correcting the erroneously recognized part “(1) -1 word”, the erroneously recognized part “(1) -4 word” having the priority “(1) I want to correct it immediately” is newly added. Is specified. In this case, the erroneous recognition location “(1) -4 words” is displayed below the erroneous recognition location “(1) -3 words” in the erroneous recognition standby window 73.

As shown in FIG. 12, the correction window 72 may display not only the misrecognized portion but also the characters before and after the portion. Thereby, the corrector 14 can easily grasp the context before and after, and the ease and certainty of correction increase.
A correction window 72 may be provided in the recognition result output window 71 as shown in FIG. Thereby, the corrector 14 can correct more intuitively. Further, as shown in FIG. 14, a line including a misrecognized portion may be displayed on the correction window 72.

  In this way, by specifying a misrecognized portion by a person other than the corrector (in this embodiment, a human), the corrector 14 can concentrate on correcting the misrecognized portion and reduce the burden. Further, the corrector 14 can correct misrecognized portions in descending order of priority, and the correction efficiency is improved as compared with the prior art. Furthermore, a hearing impaired person who is a party who particularly desires correction can identify a misrecognized portion and determine a priority order, thereby enabling more accurate and efficient correction.

  In the above example, the hearing-impaired person 13 has identified the erroneously recognized location, but any person other than the corrector 14 may identify the erroneously recognized location. For example, the speakers 11 and 12 may specify a misrecognized location. In this case, the speakers 11 and 12 input information about misrecognized portions from an input unit (not shown).

  Further, the person who specifies the misrecognized portion and the person who determines the priority order may be different persons. For example, the speakers 11 and 12 may specify a misrecognized location, and the hearing impaired person 13 may determine the priority order of the identified misrecognized location.

[Second Embodiment]
In the second embodiment, a computer, not a human, automatically specifies a misrecognized location using reliability and intimacy. About another part, it is the same as that of 1st embodiment. Hereinafter, in order to abbreviate | omit duplication description, it demonstrates centering on a different part from 1st embodiment.
FIG. 2 is a functional block diagram of the misrecognition correction apparatus of the second embodiment. FIG. 5 is a flowchart of step S3 in the second embodiment.

The speech recognition PC 31 further includes a misrecognition location specifying unit 313 and a priority order determining unit 314.
When the speech recognition unit 311 performs speech recognition in step S2, the speech recognition unit 311 calculates the reliability of each word constituting the sentence that is the speech recognition result. The calculated reliability is sent to the misrecognized location specifying unit 313 (step S31a).

  The reliability is an index indicating the degree of fit between the input speech and the speech recognition result. For example, when a voice of “voice” is input, if the voice recognition result is “voice”, the reliability is close to a perfect score, and if it is “school”, it is trusted. The degree is lowered. The reliability increases as it becomes closer to "voice" such as "mixed voice", "onsen", etc., but conversely, the reliability decreases as it moves away from "voice". A more specific definition of the reliability depends on the speech recognition engine employed by the speech recognition unit 311.

Misrecognized part identification unit 313 compares the first threshold value N ₁ reliability and given for each word (step S32). The first threshold value N ₁ is a constant which is appropriately determined according to performance required and specifications.
If the reliability is less than the first threshold value N _1, the misrecognized portion identifying unit 313 identifies as a recognition point erroneously a portion including the word or words that (step S33).
Reliability in the case of the first threshold value N ₁ or more, without doing anything erroneous recognition site identification unit 313 does not specify a recognition portion erroneous for that word.

The priority determining unit 314 determines a higher priority for a word having a lower reliability of each word (step S4). For example, it is assumed that predetermined threshold values Th ₁ ,..., Th _m ,..., Th _M satisfy the relationship Th ₁ <... <Th _m <... <Th _M , and priorities are determined in advance for each section delimited by the threshold values. deep. At that time, a higher priority is set for a section divided by a smaller threshold. For example, the priority of the section [Th _m−1 , Th _m ) is set higher than the priority of the section [Th _m , Th _{m + 1} ). The priority determination unit 314 determines whether the reliability of each word is included in any of the sections delimited by the threshold, and determines the priority corresponding to the section including the reliability as the priority of the word. .

Information on the identified erroneous recognition location and information on the priority order are sent to the display unit 312, the correction PC 32 and the character output PC 33 together with the text.
In this way, by specifying a misrecognized portion by a person other than the corrector (in this embodiment, a computer), the corrector 14 can concentrate on correcting the misrecognized portion and the burden is reduced. Further, the corrector 14 can correct misrecognized portions in descending order of priority, and the correction efficiency is improved as compared with the prior art.

Instead of the reliability, the familiarity may be used as a reference, and similarly to the case where the reliability is used as a reference, a misrecognized portion may be specified and the priority order may be determined.
The familiarity is an index indicating how familiar the word is, and is expressed in seven stages, for example, 1 (no familiarity) to 7 (the familiarity).

  FIG. 6 illustrates a flow of processing for specifying a misrecognized portion on the basis of closeness. When using intimacy, as shown by a broken line in FIG. 2, the intimacy storage unit 4 is connected to the misrecognized location specifying unit 313 and the priority determining unit 314 so that data can be read. And Intimacy storage unit 4 stores intimacy of each of a plurality of words.

The misrecognition location specifying unit 313 obtains the familiarity of each word constituting the sentence that is the speech recognition result by the speech recognition unit 311 with reference to the familiarity storage unit 4 (step S31b). Since the subsequent processing is the same as that based on the reliability, description thereof is omitted.
In the case where word familiarity is not stored in the closeness storage unit 4, closeness is as in the case smaller than the first threshold N _1, recognizing places erroneously the word or portion including the word May be specified.

[Third embodiment]
In the third embodiment, a computer, not a human, automatically specifies a misrecognized portion based on the action of a person who reads a sentence that is a voice recognition result. About another part, it is the same as that of 1st embodiment. Hereinafter, in order to omit redundant description, the description will be focused on parts different from the first embodiment.

FIG. 3 is a functional block diagram of the misrecognition correction apparatus according to the third embodiment. FIG. 4 is a flowchart of the process in step S3 of the third embodiment.
The motion information acquisition unit 5 is a device that can acquire information about human motion, such as a camera, a motion sensor, and a position sensor. In this example, it is assumed that the motion information acquisition unit 5 acquires motion information of a hearing impaired person. As long as it is a person who is reading a sentence that is a voice recognition result, operation information of a person other than the hearing impaired person may be acquired.

The motion information acquisition unit 5 acquires motion information of a hearing impaired person who is reading a sentence that is a voice recognition result displayed on the display unit 331 (step S35). The acquired operation information is sent to the misrecognized location specifying unit 333.
The misrecognition location specifying unit 333 detects whether or not the hearing impaired person reading the text is obscured based on the acquired motion information (step S36). The caulking is a gesture that tilts sideways from a normal face as shown in FIG. It is possible to detect whether or not the face is blurred based on the face angle and the change in the face angle.

  When it is detected that it is crisp, the misrecognition location specifying unit 333 specifies the text location newly displayed on the display unit 331 as the misrecognition location in the time zone detected as being stale (step) S33). For example, the line including the character newly displayed on the display unit 331 in the time zone in which it is detected that it is distorted is specified as the misrecognized portion. Information about the misrecognized portion is sent to at least the correction PC 32, and the misrecognized portion in the text displayed on the display unit 321 based on the information is marked with red, for example.

When it is detected that the page is not distorted, the misrecognition location specifying unit 333 does not specify the text location displayed on the display unit 331 in the time zone as the misrecognition location (step S34).
In addition, even if it is detected that it is not cautious in step S36, even if the misrecognition location specific | specification part 333 detects whether the hearing impaired person reading the sentence is nodding based on the acquired operation information Good (step S37). A nod is a gesture of lowering the head forward from a normal face as shown in FIG. Nodding is not limited to lowering the face once, but may be raised and lowered many times.

  When it is detected that it is nodding, the misrecognition location specifying unit 333 may specify the text location displayed on the display unit 331 in the time zone as a location that is correctly recognized, that is, a correct recognition location. Information about the correct recognition location is sent to at least the correction PC 32, and the correct recognition location in the text displayed on the display unit 321 based on the information is marked in blue, for example.

  In this way, by specifying a misrecognized portion by a person other than the corrector (in this embodiment, a computer), the corrector 14 can concentrate on correcting the misrecognized portion and the burden is reduced. Further, the corrector 14 can correct misrecognized portions in descending order of priority, and the correction efficiency is improved as compared with the prior art.

By marking the erroneously recognized part in red and marking the correctly recognized part in blue, the corrector 14 needs to make corrections centering on the part marked in red, reducing the fatigue of the corrector 14 It becomes possible to do.
If it is detected that the face is distorted, the priority determining unit 334 determines a higher priority as the face angle at that time increases, and sends at least information about the priority to the display unit 321. May be.

[Modifications, etc.]
The method for identifying the misrecognized portion and the method for determining the priority order described in the first to third embodiments can be combined as appropriate. For example, the erroneous recognition location may be specified by the method of the third embodiment, and the priority order may be determined by the method of the second embodiment.
It can be realized by a computer. In this case, the processing contents of the functions that the apparatus should have are described by a program. Then, by executing this program on a computer, each processing function in this apparatus is realized on the computer.

The program describing the processing contents can be recorded on a computer-readable recording medium. In this embodiment, these apparatuses are configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.
The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

Claims (10)

A speech recognition unit that recognizes the input speech and generates a sentence that is the speech recognition result;
A misrecognition location identifying unit for acquiring information about misrecognition locations in the sentence,
A priority determining unit for acquiring information on the priority of the erroneously recognized portion;
A display unit for displaying the misrecognized portion according to the sentence and the priority order;
An input unit for inputting corrections regarding the erroneously recognized parts by the corrector;
False recognition correction device including In the erroneous recognition correction apparatus according to claim 1,
The misrecognized part specifying part is a part to which information about a misrecognized part specified by a person different from the corrector is input.
False recognition correction device including In the erroneous recognition correction apparatus according to claim 1,
Using the index indicating the degree of fit between the input speech and the speech recognition result as the reliability, the speech recognition unit calculates the reliability of each word constituting the sentence when performing speech recognition,
The misrecognition location specifying unit specifies a word having the calculated reliability smaller than a predetermined threshold as an erroneous recognition location,
A misrecognition correction apparatus characterized by that. In the erroneous recognition correction apparatus according to claim 1,
An index indicating how familiar the word is as a familiarity, further including a familiarity storage unit in which the familiarity of each word is stored,
The misrecognition location specifying unit obtains the reliability of each word constituting the sentence by referring to the familiarity storage unit, and uses the word whose reliability is smaller than a predetermined threshold as the misrecognition location. Identify,
A misrecognition correction apparatus characterized by that. In the erroneous recognition correction apparatus according to claim 1,
An operation information acquisition unit for acquiring operation information about the operation of the person reading the sentence displayed on the display unit;
The misrecognition location specifying unit detects whether the person reading the text is stigmatized based on the operation information, and the text location displayed on the display unit in the time zone detected as stigmatized Is identified as a misrecognized location,
A misrecognition correction apparatus characterized by that. In the erroneous recognition correction apparatus according to any one of claims 1 to 5,
The priority determining unit is a unit to which information on priority determined by a person different from the corrector is input.
A misrecognition correction apparatus characterized by that. In the erroneous recognition correction apparatus according to any one of claims 1 to 5,
Using the index indicating the degree of fit between the input speech and the speech recognition result as the reliability, the speech recognition unit calculates the reliability of each word constituting the sentence when performing speech recognition,
The priority determining unit determines a higher priority for a word having a lower calculated reliability.
A misrecognition correction apparatus characterized by that. In the erroneous recognition correction apparatus according to any one of claims 1 to 5,
An index indicating how familiar the word is as a familiarity, further including a familiarity storage unit in which the familiarity of each word is stored,
The priority determining unit obtains the reliability of each word constituting the sentence with reference to the intimacy storage unit, and determines a higher priority for a word having a smaller reliability.
A misrecognition correction apparatus characterized by that. A speech recognition step for recognizing the input speech and generating a sentence that is the speech recognition result;
A step of identifying a misrecognized location that acquires information about a misrecognized location in the sentence,
A priority determining step for obtaining information on the priority of the erroneous recognition location;
A display step for displaying the misrecognized portion according to the sentence and the priority order;
An input step in which corrections made by the corrector regarding the erroneously recognized part are input;
Misrecognition correction method including.   A misrecognition correction program for causing a computer to function as each unit of the misrecognition correction apparatus according to claim 1.