JP2006079288A - Writing face reproducing method, apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a writing face reproducing method which reproduces contents written by hand on a white board or the like, and to provide a writing face reproducing apparatus, and a writing face reproducing program therefor. <P>SOLUTION: The writing face reproducing method which reproduces an image of a writing face of the information script object described or written on the writing face of the information script object has; a 1st shooting step to shoot a 1st extent of the information script object; a 2nd shooting step to shoot a 2nd extent which is over the 1st one by approaching the information script object more than the 1st shooting step; a corresponding area detection step to make the predetermined areas of the 2nd extent and the 1st extent correspondent; and a corresponding area reproducing step which reproduces the predetermined area using the images of the 2nd extent. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a writing surface reproduction method, a writing surface reproduction device, and a writing surface reproduction program for reproducing an image of a writing surface of an information handwritten object.

The contents handwritten on the whiteboard at a meeting or the like are often needed for those who attended the meeting or distributed as minutes. Conventionally, a copy board that scans the entire screen of the whiteboard and converts the handwritten content of the whiteboard into image data, and a shooting system that captures images and stores them as image data have been proposed (for example, non-printing) (See Patent Document 1).
Visual education, IT education tool [searched on July 20, 2004], Internet <URL: http://www.eftel-takeo.co.jp/school/index32.html>

  However, it is inconvenient that the attendee needs to transfer the handwritten contents to a notebook or the like on a whiteboard that does not have a scanning function or a photographing function. Further, in the photographing system described in Non-Patent Document 1, since the digital camera is fixed to the conference room or the like, there is a disadvantage that the position of the whiteboard is also fixed.

  In addition, for example, a whiteboard may be photographed with a digital camera held by a person who is attending a meeting or the like, but it is difficult to distinguish characters from the photographed image due to the resolution of the digital camera or the like. There are many cases.

  In view of the above problems, an object of the present invention is to provide a writing surface reproduction method, a writing surface reproduction device, and a writing surface reproduction program for reproducing contents handwritten on a whiteboard or the like.

  In order to solve the above problem, the present invention provides a writing surface reproduction method for reproducing an image of a writing surface of an information handwritten object drawn or written on a writing surface of an information handwritten object. A first photographing step for photographing a range, a second photographing step for photographing one or more second ranges closer to the information handwriting object than the first photographing step, and the second range, And a corresponding area detecting step for taking correspondence with the predetermined area in the first range, and a corresponding area reproducing step for reproducing the predetermined area using an image in the second range.

  ADVANTAGE OF THE INVENTION According to this invention, the writing surface reproduction | regeneration method which reproduces | regenerates the content handwritten on the whiteboard etc. can be provided. The information handwritten object may be anything that can describe or write information, such as a whiteboard, flip chart, or blackboard.

  Moreover, in one form of the writing surface reproduction | regeneration method of this invention, a corresponding | compatible area | region detection step is the imaging distance information and focus with respect to the information handwritten object preserve | saved with image data in the 1st imaging | photography step and a 2nd imaging | photography step. A magnification calculation step for calculating the magnification of the second range image with respect to the first range image based on the distance, and the scale of the first range image according to the magnification calculated by the magnification calculation step The image reduction step for reducing the image of the second range, the image of the second range reduced by the image reduction step, and the image of the first range are compared one dot at a time, or a predetermined plurality of dots as one group. A reduced image comparison step.

  According to the present invention, by adjusting the scale of the image of the second range to the image of the first range, both can be compared, and then comparing one dot at a time or a predetermined plurality of dots as a group, Correspondence between the second range and the predetermined range of the first range can be taken.

  In the writing surface reproduction method of the present invention, a magnification setting step for setting a predetermined magnification based on the magnification of the image in the second range calculated by the magnification calculation step, and a predetermined magnification set by the magnification setting step And a magnification adapting step for enlarging the image in the first range and reducing the other image in the second range.

  According to the present invention, the first range image is enlarged and the second range image is reduced, so that the scales of the first and second range images can be adapted and the written image can be reproduced appropriately. .

  In the writing surface reproduction method of the present invention, the corresponding area reproduction step includes a contrast detection step for detecting the contrast of the image in the second range, and the second range having a low contrast based on the contrast in the contrast detection step. The predetermined area is reproduced in order from the first image.

  According to the present invention, it is possible to generate a reproduced image having a high contrast by reproducing the corresponding region in order from the second range image having a low contrast.

  In addition, the present invention provides a writing surface reproduction apparatus that reproduces an image of a writing surface of the information handwritten object drawn or written on the writing surface of the information handwritten object. Photographing means for photographing one or more second ranges narrower than one range; corresponding area detecting means for taking correspondence between the second range and the predetermined area of the first range; Corresponding area reproduction means for reproducing using an image in the range. ADVANTAGE OF THE INVENTION According to this invention, the writing surface reproduction | regeneration apparatus which reproduces | regenerates the content handwritten on the whiteboard etc. can be provided.

  Moreover, in one form of the writing surface reproducing device of the present invention, the corresponding area detecting means is based on the photographing distance information and the focal distance with the information handwritten object stored together with the image data by the photographing means. Magnification calculating means for calculating the magnification of the image in the second range relative to the image, and reducing the image in the second range in accordance with the scale of the image in the first range according to the magnification calculated by the magnification calculating means. Image reduction means, and reduced image comparison means for comparing the second range image reduced by the image reduction means and the first range image one dot at a time or a predetermined plurality of dots as a group. Features.

  Further, in one embodiment of the writing surface reproducing apparatus of the present invention, the magnification setting unit sets a predetermined magnification based on the magnification of the image in the second range calculated by the magnification calculating unit, and the magnification setting unit sets the magnification. And a magnification adapting means for enlarging the image in the first range and reducing the other image in the second range at a predetermined magnification.

  Further, in one form of the writing surface reproducing apparatus of the present invention, the corresponding area reproducing unit has a contrast detecting unit that detects the contrast of the image in the second range, and the contrast is detected based on the contrast detected by the contrast detecting unit. The predetermined area is reproduced in order from an image in the second range having a lower value.

  The present invention also provides a writing surface reproduction program that causes a computer to execute the writing surface reproduction method according to claims 1 to 4. ADVANTAGE OF THE INVENTION According to this invention, the writing surface reproduction | regeneration program which reproduce | regenerates the content handwritten on the whiteboard etc. can be provided.

  It is possible to provide a writing surface reproduction method, a writing surface reproduction device, and a writing surface reproduction program for reproducing the content handwritten on a whiteboard or the like.

  The best mode for carrying out the present invention will be described below with reference to the drawings. In the present embodiment, the content handwritten on the writing surface of the information handwritten object at a meeting or the like is reproduced (hereinafter, the content handwritten on the writing surface of the information handwritten object is referred to as a written image). The information handwritten object is mainly a whiteboard, a flip chart, a blackboard or the like, but may be any object as long as it can be handwritten. This embodiment will be described using a whiteboard. In addition, an apparatus for photographing an information handwritten object may have a photographing function such as a digital camera or a mobile phone with a camera, but in this embodiment, it is a mobile phone with a camera that is usually carried around. Preferably it is.

  FIG. 1 shows a functional block diagram of the writing surface reproducing apparatus. The photographing means 11 is a means for photographing a writing image on the whiteboard. The photographing means 11 converts an image formed on a solid-state image sensor such as a CCD or CMOS through a lens into a digital signal by an A / D converter. Thereafter, the image signal is subjected to image processing such as aperture correction, compression, and the like, and is stored in the memory as image data. The entire photographing image on the whiteboard and a range in which desired contents are handwritten are photographed by the photographing means 11. Hereinafter, an image in which the entire writing image is captured is referred to as an entire image, and an image in which a desired range of the writing image is captured is referred to as a close-up image. Note that the desired range is a range that needs to be recorded, such as due date and contact information, in the writing image of the whiteboard. In addition, since the whole image and the close-up image are in a relative relationship, the whole image may not be an image of the entire writing image, and the whole image only needs to include the close-up image.

  Along with the image data, the distance to the whiteboard and the focal length are recorded. The image data may be recorded in any format such as EXIF (Exchangeable Image File Format), JPEG, BMP, etc. However, since these pieces of information are recorded together with the image data, it is preferable to record in the EXIF format. . Further, it is preferable that the photographing means 11 has an autofocus function so that the distance to the whiteboard is accurately recorded.

  The corresponding area detecting unit 14 is configured to include a magnification calculating unit 12, an image reducing unit 13, and a reduced image comparing unit 9, and detects an area corresponding to the close-up image in the entire image.

  The magnification calculation means 12 calculates the magnification of each close-up image with respect to the entire image based on the distance to the whiteboard and the angle of view (focal length). That is, the angle of view is calculated from the focal length recorded together with the image data. Based on the calculated angle of view and the distance to the whiteboard, the size of the written image being photographed on the whiteboard is obtained using a trigonometric function.

  The magnification calculation means 12 calculates the magnification of each close-up image by obtaining the size of the handwritten image for each image and comparing the size of the handwritten image of each close-up image with the size of the entire image. Therefore, the magnification in the present embodiment refers to the ratio of the written image of the entire image and the written image of each close-up image. Note that if the writing image is skewed (distorted) because the digital camera has an inclination with respect to the whiteboard, the magnification is calculated after correcting the distortion by a known correction method.

  The image reduction means 13 reduces each close-up image according to the scale of the entire image based on the magnification of each close-up image. Thereby, the size of the handwritten image of each close-up image is reduced to the size of the handwritten image of the entire image.

  The reduced image comparison means 9 performs so-called image pattern recognition and detects which region of the entire image corresponds to each close-up image. The detection method may compare the whole image and the close-up image one dot at a time or a predetermined plurality of dots as a group, or may extract and compare feature points of a written image.

  The magnification setting means 15 sets an appropriate magnification for reproducing the whole image as each close-up image based on the average magnification of each close-up image. It is preferable that the appropriate magnification is set so that the entire image and each close-up image can be enlarged or reduced so that the reproduced image looks beautiful.

  The magnification adapting means 8 enlarges the whole image to the magnification set by the magnification setting means 15 and reduces each close-up image. The magnification adapting means 8 adapts the magnification of the whole image and each close-up image so that the sizes of the written images are equal.

  The corresponding area reproduction unit 16 reproduces (replaces) the area corresponding to the close-up image in the entire image enlarged by the magnification adaptation unit 8 with each close-up image reduced by the magnification adaptation unit 8. At this time, if the handwritten images of the close-up images overlap, the close-up images with low contrast detected by the contrast detecting means 7 are reproduced in order so that the close-up images with high contrast are reproduced. The reproduced handwritten image is stored in the memory by the storage unit 17.

  FIG. 2 is a schematic hardware configuration diagram of the writing surface reproducing apparatus. The schematic hardware configuration diagram of FIG. 2 shows a photographing device 21, a CPU 22, an input / output control unit 23, a memory card controller 24, a communication device 25, a main storage device 26, a flash ROM 27, and a display device that are mutually connected by a bus. 28 is configured.

  The photographing device 21 provides photographing means 11 including a lens and an image sensor. The CPU 22 comprehensively controls processing performed by the writing surface reproducing apparatus. More specifically, the program stored in the flash ROM 27 is executed. The input / output control unit 23 includes a numeric keypad, a scroll button, and the like, and is used for inputting various operation instructions from the user. The memory card controller 24 provides an input / output function with the memory card 29 for storing photographed image data and storing a program. The communication device 25 is an interface for connecting to the Internet, a LAN, a wireless base station, or the like. The main storage device 26 is a storage area for temporarily storing an operating system, programs, and image data.

  A writing surface reproduction program is installed in the flash ROM 27. The writing surface reproduction program causes the CPU 22 to execute the photographing unit 11, the corresponding region detection unit 14, the magnification setting unit 15, the magnification adaptation unit 8, the corresponding region reproduction unit 16, and the storage unit 17 of FIG. The program may be stored in advance in the flash ROM 27, or may be downloaded via the memory card 29 or a network. The display device 27 displays a written image of the whiteboard acquired by the photographing device 21, displays a photographed image, various windows and data necessary for operation, and the like.

  Note that the imaging device 21 may be a separate body. In this case, the image data of the photographed image is acquired by the communication device 25 or the memory card 29, and the photographed image of the whiteboard is reproduced.

  Next, a flow of processing for reproducing a handwritten image handwritten on the writing surface of the whiteboard will be described with reference to an embodiment. FIG. 3 is a flowchart showing the flow of the writing surface reproduction method.

(Step S1)
First, the photographing means 11 makes a close-up of the entire written image of the whiteboard and the range in which the desired content of the written image is handwritten. Thereby, the whole image and the close-up image are obtained. The close-up image is photographed in close-up so that the written image on the whiteboard can be determined according to the number of pixels of the image sensor. Note that the entire image capturing corresponds to the first capturing step in the claims, and the close-up image capturing corresponds to the second capturing step.

  FIG. 4 shows a written image of the photographed whiteboard. FIG. 4A shows a written image of the whole image in which the entire whiteboard is photographed. In FIG. 4A, a region surrounded by a dotted line indicates a close-up region. 4B is a handwritten image obtained by close-up of the region 31, FIG. 4C is a hand-written image obtained by close-up of the region 32, FIG. 4D is a hand-written image obtained by close-up of the region 33, and FIG. Indicates handwritten images of close-up areas 34 respectively.

  Each image data is stored in the EXIF format, and the distance to the whiteboard, the focal length, and the like are stored together with each image data as shown in FIG. It is preferable to assign a file name or an extension of each image data so that the entire image can be associated with each close-up image.

(Step S2)
Subsequently, if there is distortion in the written image of the whole image or each close-up image, it is corrected. When the distortion is corrected, the magnification calculation means 12 calculates the magnification of the writing image of each close-up image with respect to the writing image of the whole image based on the distance to the whiteboard and the angle of view (focal length) recorded together with the image data. Is done.

  FIG. 5 shows a handwritten image of a close-up image with distortion corrected and magnification calculated. The magnification of the entire image is x1, the magnification of the region 31 is x10, the magnification of the region 32 is x8, the magnification of the region 33 is x9, and the magnification of the region 34 is x7.

(Step S3)
The image reduction means 13 reduces the handwritten image of each close-up image to the scale of the handwritten image of the entire image. FIG. 6 shows a handwritten image of an image obtained by reducing each close-up image at a magnification with respect to the entire image. That is, each close-up image in FIG. 5 is reduced in accordance with the magnification calculated in step S2. 6 (b) is reduced to 1/10, FIG. 6 (c) is reduced to 1/8, FIG. 6 (d) is reduced to 1/9, and FIG. 6 (e) is reduced to 1/7. The rendered image is shown.

  Next, the reduced image comparison unit 9 compares the handwritten image of each close-up image with the written image of the whole image one dot at a time or a predetermined plurality of dots, thereby detecting a corresponding region corresponding to the close-up image in the whole image. The The corresponding characters may be detected by OCR (Optical Character Reader) of the photographed characters.

(Step S4)
The magnification setting means 15 sets the magnification of a reproduced image obtained by reproducing the entire image areas 31 to 34 using each close-up image. The magnification setting means 15 obtains the average magnification of each close-up image and sets the close-up image having a magnification close to the average magnification as the reproduction image magnification. The magnification of the reproduced image may be obtained by including the entire image in the average calculation, or the average magnification itself may be used.

  The magnification adapting means 8 reduces or enlarges the entire image and each close-up image in accordance with the set magnification of the reproduced image. FIG. 7 shows a handwritten image of an enlarged / reduced close-up image. In FIG. 7, as an example, the magnification of FIG. 7D obtained by close-up of the region 33 is set as the magnification of the reproduced image. Since the magnification of the area 33 is × 9, the images of the area 32 and the area 34 with a smaller magnification are enlarged, and the image of the area 31 with a larger magnification is reduced. The whole image is enlarged to x9 in accordance with the magnification of the reproduced image.

  When each image is enlarged / reduced, the corresponding area reproduction means 16 reproduces (replaces) the corresponding area of all the images obtained in step S3 with the handwritten image of each close-up image.

(Step S5)
At the time of replacement, if the handwritten image of the close-up image has an overlapping portion, the contrast of the overlapped close-up image is compared, and the close-up image with high contrast is replaced with the entire image. FIG. 8 shows an example of a close-up image having an overlapping portion. As shown in FIG. 8, when the region 32 and the region 33 have an overlapping portion 35, the higher contrast of the regions 32 and 33 is replaced with the entire image. For example, when the contrast of the region 32 is higher than that of the region 33, a close-up image with high contrast can be used by replacing the region 32 with the entire image after replacing the region 33 with the entire image. . Note that the contrast of each close-up image is calculated before replacing the close-up image without determining whether the handwritten image of the close-up image has an overlapped portion, and the close-up image is replaced with a low-contrast image. Good. By replacing a low-contrast close-up image, it is finally possible to reproduce a high-contrast close-up image.

(Step S6)
A reproduction image obtained by reproducing each close-up image is stored and output by a predetermined method. Only the reproduced image may be stored, or the entire image and each close-up image may be stored in association with the reproduced image.

  As described above, according to the present embodiment, it is possible to easily record a written image of a whiteboard even on a whiteboard having no scanning function or photographing function. In particular, since mobile phones with cameras are prevalent, it is possible to record a written image on a whiteboard without any special preparation.

It is a functional block diagram of a writing surface reproducing | regenerating apparatus. It is a schematic hardware block diagram of a writing surface reproduction | regeneration apparatus. It is a flowchart figure which shows the flow of the writing surface reproduction | regeneration method. The written image of the photographed whiteboard is shown. It is a handwritten image such as a close-up image in which distortion is corrected and magnification is calculated. It is a handwritten image of an image obtained by reducing each close-up image at a magnification relative to the whole image. This is a handwritten image of an enlarged / reduced close-up image. It is an example of the handwritten image of the close-up image which has an overlap part.

Explanation of symbols

8 Magnification Adaptation Unit 9 Reduced Image Comparison Unit 11 Imaging Unit 12 Magnification Calculation Unit 13 Image Reduction Unit 14 Corresponding Region Detection Unit 15 Magnification Setting Unit 16 Corresponding Region Reproducing Unit 17 Storage Unit 31-34 Region

Claims (9)

In a writing surface reproduction method for reproducing an image of the writing surface of the information handwriting object handwritten on the writing surface of the information handwriting object,
A first photographing step of photographing a first range of a writing surface of the information handwriting object;
A second imaging step of imaging one or more second ranges of the writing surface of the information handwritten object closer to the first imaging step;
A corresponding area detecting step for taking a correspondence between the second range and a predetermined area of the first range;
A corresponding area reproduction step of reproducing the predetermined area using the image in the second range;
A method for reproducing a writing surface, comprising: The corresponding area detecting step includes an image of a first range based on shooting distance information and a focal length with respect to the information handwritten object stored together with image data in the first shooting step and the second shooting step. A magnification calculating step for calculating a magnification of an image in a second range with respect to
An image reduction step of reducing the image of the second range according to the scale of the image of the first range according to the magnification calculated by the magnification calculation step;
A reduced image comparison step of comparing the second range image reduced by the image reduction step and the first range image one dot at a time or a predetermined plurality of dots as a group;
The writing surface reproducing method according to claim 1, wherein: A magnification setting step for setting a predetermined magnification based on the magnification of the image in the second range calculated by the magnification calculation step;
A magnification adapting step of enlarging the first range image and reducing the other second range image to the predetermined magnification set by the magnification setting step;
The writing surface reproducing method according to claim 2, wherein: The corresponding region reproduction step includes a contrast detection step of detecting a contrast of the image in the second range;
2. The writing surface reproduction method according to claim 1, wherein the predetermined area is reproduced in order from an image in a second range having a low contrast based on the contrast in the contrast detection step. In a writing surface reproduction device for reproducing an image of a writing surface of the information handwriting object handwritten on the writing surface of the information handwriting object,
An imaging means for imaging the first range of the writing surface of the information handwriting object and one or more second ranges narrower than the first range;
Corresponding area detecting means for taking a correspondence between the second range and the predetermined area of the first range;
Corresponding area reproduction means for reproducing the predetermined area using the image in the second range;
A writing surface reproducing apparatus comprising: The corresponding area detection unit calculates a magnification of the image in the second range with respect to the image in the first range based on the shooting distance information and the focal length with the information handwritten object stored together with the image data by the shooting unit. A magnification calculation means for calculating;
Image reduction means for reducing the image of the second range in accordance with the scale of the image of the first range according to the magnification calculated by the magnification calculation means;
Reduced image comparison means for comparing the second range image reduced by the image reduction means and the first range image one dot at a time or a predetermined plurality of dots as a group;
The writing surface reproducing apparatus according to claim 5, comprising: A magnification setting means for setting a predetermined magnification based on the magnification of the image in the second range calculated by the magnification calculation means;
Magnification adapting means for enlarging the image in the first range to a predetermined magnification set by the magnification setting means and reducing the image in the other second range;
The writing surface reproducing apparatus according to claim 2, comprising: The corresponding area reproduction means includes contrast detection means for detecting a contrast of the image in the second range;
6. The writing surface reproducing apparatus according to claim 5, wherein the predetermined area is reproduced in order from an image in the second range having a low contrast based on the contrast detected by the contrast detecting means.   A writing surface reproduction program for causing a computer to execute the writing surface reproduction method according to claim 1.

Images