JP2005084157A - Device and method for character recognition, character emphasizing method, display device, and mobile equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for character recognition that recognize characters in an image of a video signal united with an image, a display device which displays characters in the image with high visibility, mobile equipment equipped with the same, and a character emphasizing method of making high the visibility of the characters in the image. <P>SOLUTION: The character recognition device is provided with: a video reception part 31 which receives image data of a source image; an image buffer 32 which cuts out image data of a specified area on the display screen of a display part 28; and a contrast detection part 34 which extracts image data of a part corresponding to pixels whose lightness differences from adjacent pixels are equal to or larger than a threshold as image data of a character area from the image data cut out by the image buffer 32. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a character recognition device and method for recognizing characters arranged on an image, a character emphasis method for emphasizing characters arranged on an image, and a display device capable of emphasizing and displaying characters arranged on an image. And a portable device including the same.

  In portable devices such as mobile phones, PDAs, and portable televisions, the size of the housing is limited from the viewpoint of portability, and thus a display device having a small screen size must be applied.

  For this reason, when displaying a terrestrial television broadcast on a display device provided in a mobile device or playing a video by accessing the Internet network from a mobile device via wireless communication, the original video source is set to the size of the display screen. It must be reduced together.

  Therefore, even if the resolution of the display device included in the portable device is improved, it is difficult to display characters such as subtitles such as movies and news telops with high visibility.

  If a display device with a large screen is mounted on a portable device, it is easy to improve the visibility of these characters, but the portability is impaired.

  As described above, in a portable device to which a large-screen display device cannot be applied, it is difficult to improve both the visibility of characters displayed on the screen and the improvement of portability.

  Conventional techniques for improving the visibility of characters in a display device of a portable device include “LCD character enlargement display method” disclosed in Patent Document 1 and “LCD character enlargement display method” disclosed in Patent Document 2. There is.

  When the information displayed on the screen is only characters, by preparing several character fonts as in the inventions described in Patent Document 1 and Patent Document 2, only specific characters can be enlarged. It is possible to display. However, when the inventions described in Patent Document 1 and Patent Document 2 are applied when simultaneously displaying video and characters, such as movie subtitles and news telops, there is a possibility of being displayed on the screen. All of the characters with must be included as a font. In addition, after recognizing the characters in the video, it is necessary to identify which of the various fonts the character corresponds to, so it is very difficult to enlarge the characters in the video delivered in real time was difficult.

Further, as a conventional technique for enlarging and displaying characters in a video, there is a “DVD video playback device” disclosed in Patent Document 3. In the invention described in Patent Document 3, when a sub-picture display command is issued during DVD video playback, the sub-picture display area read from the disk is enlarged and then displayed by being combined with the main video and displayed. This improves the visibility of subtitles in a type of DVD video playback apparatus.
JP-A-6-012046 JP-A-6-012047 JP 2002-044590 A

  However, the invention described in Patent Document 3 cannot be applied unless the video is divided into a main video and a sub video. That is, it can be applied to the case of reproducing media in which video (main video) and subtitles (sub-video) are recorded in advance as separate video information, such as DVD video. It cannot be applied to the case where text is overlapped on the video to form a single video information, or when playing media such as movie film in which subtitles are directly written in the image. .

  As described above, conventionally, characters that are not distinguished from images cannot be recognized, such as subtitles displayed in an overlapping manner on a video, so that the characters cannot be highlighted and displayed.

  The present invention has been made in view of such problems, and a character recognition device and method for recognizing characters in an image that is a video signal integrated with the image, and a display device that displays characters on the image with high visibility. It is another object of the present invention to provide a character enhancement method that increases the visibility of characters on an image and a portable device including the same.

  In order to achieve the above object, the present invention provides, as a first aspect, means for receiving image data of an original image, image cutout means for cutting out image data for a predetermined area from the image data of the original image, and image cutout A character recognition apparatus comprising: character extraction means for extracting image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value from image data extracted by the means as image data of a character area It is.

Moreover, in order to achieve the said objective, this invention is equipped with the character recognition apparatus of the said 1st aspect of the said this invention as a 2nd aspect, and the display of the structure shown in either of the following 2-1 to 2-3. A device is provided.
2-1: a display device having display means for displaying information according to image data, color conversion means for converting image data in a character area into image data for displaying a designated color on the display means, and image clipping A first image data synthesizing unit that synthesizes the image data cut out by the image cutout unit and the image data converted by the color conversion unit so that an image corresponding to the image data cut out by the unit becomes a background; A second image data synthesizing unit that synthesizes the image data of the original image from which the image data for the region is cut out and the image data synthesized by the first image data synthesizing unit, and the second image data synthesizing unit. And a display means for displaying the image data on the display means.
2-2: A display device having display means for displaying information according to image data, wherein the character enlargement means for enlarging the image according to the image data of the character area, with the size of the predetermined area as an upper limit, and the image First image data synthesis for synthesizing the image data clipped by the image clipper and the image data of the character area expanded by the character zoomer so that the image corresponding to the image data clipped by the clipper is the background Means, second image data synthesizing means for synthesizing the image data of the original image from which image data for a predetermined area has been cut out, and image data synthesized by the first image data synthesizing means, and second image data A display device including: a display unit configured to display image data combined by the combining unit;
2-3: A display device having display means for displaying information according to image data, a color conversion means for converting image data in a character area into image data for displaying a designated color on the display means, and a predetermined area Character enlarging means for enlarging the image according to the image data of the character area with an upper limit of the size of the character area, and emphasis processing for causing at least one of the color conversion means and the character enlarging means to perform image processing on the image data of the character area And the image data cut out by the image cutout means and the image data of the character area subjected to the image processing by the enhancement processing means so that the image corresponding to the image data cut out by the image cutout means becomes a background. First image data synthesizing means, image data of an original image obtained by cutting out image data for a predetermined area, and image data synthesized by the first image data synthesizing means A second image data combining means for combining the display device having a means for displaying on the display unit the image data synthesized by the second image data synthesizing means.

  Moreover, in order to achieve the said objective, this invention provides the portable apparatus provided with the display apparatus which concerns on the said 2nd aspect of this invention as a 3rd aspect.

  In order to achieve the above object, according to a fourth aspect of the present invention, an image cutout step of cutting out image data of a predetermined area of the display screen of the display means from the image data of the original image, and a cutout in the image cutout step are as follows. And a character extracting step of extracting, from the image data, a portion of image data corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value as image data of a character region.

Moreover, in order to achieve the said objective, this invention provides the character emphasis method shown in either of the following 5-1 to 5-3.
5-1: Corresponding to pixels whose brightness difference with an adjacent pixel is greater than or equal to a threshold value from an image cutout step of cutting out image data of a predetermined area from image data of an original image and image data cut out in the image cutout step Character extraction step for extracting partial image data as image data of a character region, color conversion step for converting character region image data into image data for displaying a specified color on the display means, and an image cut out in the image cutout step A first image data synthesis step for synthesizing the image data cut out in the image cutout step and the image data converted in the color conversion step so that an image corresponding to the data becomes a background; and image data for a predetermined area Are combined in the first image data synthesis step with the image data of the original image from which Character enhancement method characterized by a second image data synthesizing step of synthesizing the image data.
5-2: Corresponding to pixels whose brightness difference with an adjacent pixel is greater than or equal to a threshold value from an image cut-out step of cutting out image data for a predetermined area from image data of an original image and image data cut out in the image cut-out step The character extraction step for extracting the image data of the portion as the image data of the character region, the character enlargement step for enlarging the image according to the image data of the character region up to the size of the predetermined region, and the image cutout step A first image data synthesizing step for synthesizing the image data cut out in the image cutout step and the image data of the character area enlarged in the character enlargement step so that an image corresponding to the image data becomes a ground; and a predetermined area Image data of the original image from which the image data of the first portion is cut out and the first image data synthesis step Character enhancement method characterized by a second image data synthesizing step of synthesizing the Oite synthesized image data.
5-3: Corresponding to a pixel whose brightness difference with an adjacent pixel is greater than or equal to a threshold value from an image cutout step of cutting out image data for a predetermined area from the image data of the original image and the image data cut out in the image cutout step Character extraction step for extracting partial image data as character region image data, color conversion processing for converting character region image data into image data for displaying a specified color on the display means, and upper limit on the size of the predetermined region An enhancement processing step that performs at least one of the character enlargement processing for enlarging the image according to the image data of the character region on the image data of the character region, and an image according to the image data cut out in the image cutout step The image data cut out in the image cutout step and the processing in the enhancement processing step A first image data synthesis step for synthesizing the image data of the applied character area, an image data of the original image from which image data for a predetermined area is cut out, and an image data synthesized in the first image data synthesis step And a second image data synthesizing step for synthesizing the two.

  According to the present invention, a character recognition device and method for recognizing characters in an image that is a video signal integrated with an image, a display device that displays characters on an image with high visibility, and a portable device including the same, In addition, it is possible to provide a character emphasis method that increases the visibility of characters on an image.

  When displaying a movie or news containing characters (such as subtitles) on a display device (such as a display device for a portable device) with a limited display screen size, the present invention usually uses white characters or white frame characters. The displayed character portion is recognized as a video, and further, the subtitle portion is displayed after at least one of enlargement and color correction. Thereby, even if the image and the character are not mutually independent video signals, the character can be recognized, and in a display device with a small display screen by performing emphasis processing (enlargement, color conversion, etc.) on the recognized character, Visibility of characters displayed together with video can be increased.

  FIG. 1 shows a technique for increasing the visibility of characters displayed together with video by the present invention. First, the original image 11 is divided at a boundary 12 designated by the user. Next, the character in the part (designated area) 12a where the character is displayed among the divided images is detected by image recognition. The character in the designated area 12a is subjected to at least one of color conversion and enlargement as an emphasis process, and the designated area 13 after the emphasis process and the video portion 12b of the divided image are combined and displayed. .

  By performing such processing, it is possible to display characters with high visibility along with video even in a display device with a small display screen.

  Hereinafter, a preferred embodiment of the present invention in which characters with high visibility are displayed together with video by the above-described method will be described.

  FIG. 2 shows the configuration of the display device according to the first embodiment of the present invention. The display device includes a video reception unit 31, a control unit 32, an image buffer 33, a contrast detection unit 34, a color conversion unit 35, an image enlargement unit 36, an image memory unit 37, and a display unit 38.

  The video receiving unit 31 is a functional unit that receives and demodulates an image signal input from outside the apparatus. For example, the video receiving unit 31 receives and demodulates an image signal distributed over the Internet network, or receives and demodulates a television broadcast. To do. The control unit 32 recognizes the demodulated image signal output from the image receiving unit 31 and controls the character enlargement process. The image buffer 33 cuts out a part (designated area) from the image to the image signal based on the control signal from the control unit 32. The contrast detection unit 34 extracts a white part corresponding to the caption from the part cut out by the image buffer 33. The color conversion unit 35 converts the color of the white portion extracted by the contrast detection unit 34 as corresponding to a caption. The image enlarging unit 36 generates an image in which only the portion extracted as corresponding to the caption by the contrast detecting unit 34 is enlarged in the designated direction. The image memory unit 37 is a memory in which information on an image to be displayed on the display unit 38 is written. The display unit 38 displays an image corresponding to the image information written in the image memory 37. As the display unit 38, a portable display device such as a small cathode ray tube, a liquid crystal display, and an organic EL display can be applied.

FIG. 3 shows an operation flow when the portable device according to the present embodiment displays information on the display unit 38.
When the original image is input, the control unit 32 confirms whether there is an enlargement request by the user (step S101).

  When there is an enlargement request (step S101 / Yes), the control unit 32 sets user settings (screen boundary line position (designated area size), contrast threshold value, designated color, and enlargement direction) set in advance by the user. (Setting) is read from a user setting storage unit (not shown) (step S102).

  After reading the user setting, the control unit 32 notifies the image buffer 33 of the position of the image boundary line and causes the image data in the designated area to be cut off. That is, the image buffer 33 stores only the designated area portion of the image data of the original image. When the image buffer 33 stores the image data of the designated area portion, the control unit 32 notifies the contrast detection unit 34 of the contrast threshold value and confirms whether or not a white portion exists in the designated area (step S103). ).

  When there is a white portion in the designated area (step S104 / Yes), the control unit 32 notifies the color conversion unit 35 of the designated color and converts the white portion in the designated area into the designated color (step S105). .

  When the color conversion unit 35 finishes the color conversion of the white portion, the control unit 32 notifies the image enlargement unit 36 of the enlargement direction setting. When the enlargement direction setting is the horizontal direction (step S106 / horizontal direction setting), the image enlargement unit 36 enlarges the color-converted portion in the horizontal direction so as to have substantially the same width as the designated area (step S107). When the enlargement direction setting is the vertical direction (step S106 / vertical direction setting), the image enlargement unit 36 enlarges the color-converted portion in the vertical direction so as to have the same height as the designated area (step S108). Alternatively, when the enlargement direction setting is an arbitrary direction (step S106 / arbitrary setting), the image enlargement unit 36 enlarges the color-converted portion according to a magnification set in advance by the user (step S109).

  The image enlargement unit 36 overwrites the image data in the image memory 37 with the image data of the designated area enlarged after the white portion is converted into the designated color (step S110). As a result, the image data in the image memory 37 has the characters in the designated area emphasized.

  The display unit 38 displays an image corresponding to the image data written in the image memory 37, so that characters in the designated area can be highlighted.

  If there is no portion recognized as a character in the designated area (No in step S104), the original image is displayed without performing processing such as enlargement or color conversion.

An operation of emphasizing characters on a video by the mobile device according to the present embodiment will be described.
The video receiver 31 demodulates the received video signal, and after receiving and demodulating information for one screen, writes the demodulated video information in the image memory 37 as image data. When the control unit 32 recognizes that the video receiving unit 31 has demodulated the information for one screen, the control unit 32 sends an instruction to the image buffer 33 and is designated by the user from the images corresponding to the demodulated video signal. Get the data for the area.

  The image data of the designated area fetched into the image buffer 33 is sent to the contrast detection unit 34, and the white portion is detected by comparing the contrast of the designated area with the contrast threshold value notified from the control unit 32. In other words, the contrast detection unit 34 detects a pixel whose contrast with an adjacent pixel is equal to or higher than a contrast threshold and exists in the designated area as a white portion. As a result, the outline of the character portion is recognized image-wise.

  The color conversion unit 35 converts the color of the part detected by the contrast detection unit 34 as a white part into a designated color notified from the control unit 32. The image data of the designated area in which the color of the white portion is converted to the designated color is sent from the color conversion unit 35 to the image enlargement unit 36. The image enlarging unit 36 performs vertical and horizontal directions on the part of the image data of the designated area acquired from the color conversion unit 35 based on the enlargement setting notified from the control unit 32 with respect to the part converted to the designated color. A process for enlarging the image is performed.

  The image enlarging unit 36 outputs the image data of the designated area obtained by enlarging the portion converted into the designated color to the image memory 37, and the image of the designated area of the image data for one screen written in the image memory 37. Overwrite part.

  The image memory unit 37 outputs the image data in which the designated area portion is overwritten to the display unit 38, and the image in which the white portion in the designated area is emphasized (the white portion in the designated area is converted into the designated color and enlarged). Image).

As described above, the display device according to the present embodiment can automatically recognize and expand and display subtitles and telops used when displaying images such as movies and news. Therefore, even if it is not possible to provide a large display screen from the viewpoint of portability, it is possible to increase the visibility of characters displayed together with the video.
In addition, since the white part on the video is used as it is, it is not necessary to prepare a character font or the like in advance. For this reason, it can be applied not only to normal fonts but also to subtitles and telops using all characters including handwritten characters and special characters (round characters, sumo characters, etc.).

FIG. 4 shows the configuration of a display device according to the second embodiment of the present invention. The display device includes a video reception unit 41, an image memory unit 47, a display unit 48, and a DSP unit 49.
The video receiving unit 41, the image memory unit 47, and the display unit 48 are the same as the video receiving unit 31, the image memory unit 37, and the display unit 38 in the first embodiment, respectively.

  The DSP unit 49 includes a control unit 42, an image buffer 43, a contrast detection unit 44, a color conversion unit 45, and an image enlargement unit 46. These have the same functions as the control unit 32, the image buffer 33, the contrast detection unit 34, the color conversion unit 35, and the image enlargement unit 36 in the first embodiment.

  In the display device according to the present embodiment, image processing for emphasizing and displaying characters in a video is recorded as a substantial program in the DSP unit 49, and the first processing is performed by software processing in the DSP unit 49. The same processing as in the embodiment is performed. That is, the image data for one screen demodulated by the video receiving unit 41 is taken into the DSP unit 49, and the control unit 42, the image buffer 43, the contrast detection unit 44, the color conversion unit 45, and the image enlargement in the DSP unit 49. The unit 46 performs the same operation as in the first embodiment, thereby converting the characters in the designated area into the designated color and writing the enlarged image data in the image memory 47. Thereby, the image which emphasized the character in a video | video can be displayed on the display part 48. FIG.

  In this embodiment, the case where the video receiving unit 41 and the image memory unit 47 are configured separately from the DSP unit 49 has been described as an example. However, the DSP unit 49 may include these.

  Each of the above embodiments is an example of a preferred embodiment of the present invention, and the present invention is not limited to this. For example, as the color conversion, in addition to the process of converting the white portion into the designated color, the process of enhancing the white color (for example, the color with the RGB value of 250, 250, 250 is changed to the complete RGB value of 255, 255, 255). Processing to convert to white) or processing to convert the color around the white part (when the surrounding white part is gray, the surroundings of the white part is converted to black, etc., and the contrast with the white part is increased. It is also possible to improve the visibility of the white part by applying a specific color set by the user (converted to a color that the user feels easy to see).

  In addition, for the enlargement setting, a setting for automatically displaying a specified area regardless of the size of the original white portion or a setting for enlarging with a fixed size can be applied.

  In each of the above embodiments, the example in which the white portion in the video is detected based on the contrast threshold has been described. However, when colored characters are displayed in the video mainly based on the white background, the contrast threshold is set. Based on the above, a colored portion may be detected as a character.

  In each of the above-described embodiments, the operation of performing both the color conversion process and the enlargement process on the white portion has been described. However, it is not always necessary to perform both of these processes, and only one of them may be performed. .

  Further, in each of the above embodiments, the image data for one screen is temporarily stored in the image memory, and the data of the designated area is overwritten with the image data obtained by performing the color conversion process and the enlargement process on the white part. There is no limit. For example, the designated area portion is cut out from the image data for one screen by the image buffer, only the image data of the video portion is written in the image memory, and the white area is subjected to the color conversion process and the enlargement process. The image data may be additionally recorded.

Furthermore, the present invention can be applied not only to a portable device to which a large-screen display device is difficult to apply, but also to any display device such as a television device for elderly people.
As described above, the present invention can be variously modified.

It is a figure which shows the method in which the portable apparatus which concerns on this invention emphasizes and displays the character in an image | video. It is a figure which shows the structure of the portable apparatus concerning the 1st Example which implemented this invention suitably. It is a figure which shows the flow of the process which the portable apparatus concerning a 1st Example emphasizes and displays the character in an image | video. It is a figure which shows the structure of the portable apparatus concerning the 2nd Example which implemented this invention suitably.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Original image 12 Boundary 12a Specified area 12b Video part 13 Specified area after emphasis processing 31, 41 Video receiving part 32, 42 Control part 33, 43 Image buffer 34, 44 Contrast detection part 35, 45 Color conversion part 36, 46 Image Enlargement unit 37, 47 Image memory unit 38, 48 Display unit 49 DSP unit

Claims (9)

Means for receiving image data of the original image;
Image cutout means for cutting out image data of a predetermined area from the image data of the original image;
A character recognition apparatus comprising: character extraction means for extracting, as image data of a character area, image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value from the image data cut out by the image cutout means. A display device comprising the character recognition device according to claim 1, comprising display means for displaying information according to image data,
Color conversion means for converting the image data of the character region into image data for displaying the designated color on the display means;
First image data composition for synthesizing the image data cut out by the image cut-out means and the image data converted by the color conversion means so that an image corresponding to the image data cut out by the image cut-out means becomes a background. Means,
Second image data synthesizing means for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized by the first image data synthesizing means;
And a display unit for displaying the image data combined by the second image data combining unit on the display unit. A display device comprising the character recognition device according to claim 1, comprising display means for displaying information according to image data,
Character enlarging means for enlarging an image according to image data of the character area, with the size of the predetermined area as an upper limit,
A first image that synthesizes the image data cut out by the image cutout unit and the image data of the character area enlarged by the character enlargement unit so that an image corresponding to the image data cut out by the image cutout unit becomes a background. Image data synthesis means;
Second image data synthesizing means for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized by the first image data synthesizing means;
And a display unit configured to display the image data combined by the second image data combining unit on the display unit. A display device comprising the character recognition device according to claim 1, comprising display means for displaying information according to image data,
Color conversion means for converting the image data of the character region into image data for displaying the designated color on the display means;
Character enlarging means for enlarging an image according to image data of the character area, with the size of the predetermined area as an upper limit,
Enhancement processing means for causing at least one of the color conversion means and the character enlargement means to perform image processing on the image data of the character area;
The image data cut out by the image cut-out means and the image data of the character area subjected to the image processing by the emphasis processing means are synthesized so that an image corresponding to the image data cut out by the image cut-out means becomes a background. First image data synthesis means;
Second image data synthesizing means for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized by the first image data synthesizing means;
And a display unit configured to display the image data combined by the second image data combining unit on the display unit.   A portable device comprising the display device according to claim 2. An image cutout step of cutting out image data for a predetermined area from the image data of the original image;
A character recognition method comprising: extracting, from the image data cut out in the image cut-out step, image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value as image data of a character region. An image cutout step of cutting out image data for a predetermined area from the image data of the original image;
A character extraction step of extracting, from the image data cut out in the image cutout step, image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value as image data of a character region;
A color conversion step of converting the image data of the character region into image data for displaying a designated color on a display means;
First image data composition for synthesizing the image data extracted in the image extraction step and the image data converted in the color conversion step so that an image corresponding to the image data extracted in the image extraction step serves as a background Steps,
And a second image data synthesis step for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized in the first image data synthesis step. Character emphasis method. An image cutout step of cutting out image data for a predetermined area from the image data of the original image;
A character extraction step of extracting, from the image data cut out in the image cutout step, image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value as image data of a character region;
A character enlargement step of enlarging an image corresponding to the image data of the character region with the size of the predetermined region as an upper limit;
First image data is combined with the image data extracted in the image extraction step and the image data of the character area expanded in the character expansion step so that an image corresponding to the image data extracted in the image extraction step serves as a background. Image data synthesis step;
And a second image data synthesis step for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized in the first image data synthesis step. Character emphasis method. An image cutout step of cutting out image data for a predetermined area from the image data of the original image;
A character extraction step of extracting, from the image data cut out in the image cutout step, image data of a portion corresponding to a pixel whose brightness difference with an adjacent pixel is equal to or greater than a threshold value as image data of a character region;
Color conversion processing for converting the image data of the character area into image data for displaying a designated color on a display unit, and a character for enlarging an image corresponding to the image data of the character area with the size of the predetermined area as an upper limit An emphasis processing step for performing at least one of enlargement processing on the image data of the character region;
The image data cut out in the image cutout step and the image data of the character area processed in the emphasis processing step are synthesized so that an image corresponding to the image data cut out in the image cutout step becomes a background. 1 image data synthesis step;
And a second image data synthesis step for synthesizing the image data of the original image from which the image data for the predetermined area is cut out and the image data synthesized in the first image data synthesis step. Character emphasis method.

Images