JP4148867B2 - Handwriting processor - Google Patents

Description

  The present invention relates to a handwriting input device that is used as input means of an information processing device such as a computer, a portable information terminal, or a word processor, and converts handwriting data that an operator writes on an input screen into information intended by the writer.

  2. Description of the Related Art Conventionally, there is an on-line character recognition device that displays a recognition result as a character code or a character code string from handwritten information about characters or a character string input by handwriting via a coordinate input device such as a coordinate input tablet. A character string input in an environment in which character breaks are not explicitly given is recognized while appropriately cutting out characters (see, for example, Patent Document 1). In addition, recognition processing is also performed when writing information for each character is obtained (see, for example, Patent Document 2).

  The recognition technique is based on the premise that the handwriting is all letters. For example, when it is desired to write not only characters but also figures and gestures representing commands, for example, these recognition techniques cannot be used as they are. These methods are used after classifying the type of handwriting by some method and extracting only the character part from the handwriting.

  When the user wants to input a mixture of characters and figures, several methods have been proposed. First, there is a method of providing an input mode corresponding to each of a character, a figure, and a gesture, and switching the mode for performing handwriting input after designating the input mode. When the user wants to input characters, the user writes the characters after specifying the character input mode by some operation (pressing a mode selection button or the like). After that, if the user wishes to write a figure, the user must perform an operation for changing to the graphic mode. In this case, since the user has to execute an operation for switching the input mode each time the type of handwriting to be input changes, the operation is complicated. In addition, there is a post-selection method in which the user specifies the type of handwriting after entering the handwriting, but in this method, the user performs the operation of specifying the type of handwriting for each of the multiple images, so input in advance Compared to the method of writing after switching the mode, the work may be complicated.

  Therefore, if there is an automatic determination method for automatically determining what handwriting is input by the user, it is not necessary to perform a complicated operation for switching the input mode. Conventionally, as a method for automatically discriminating characters and figures, there are a method for discriminating characters and figures from the handwriting shape and size, and a method for discriminating characters and figures from the length of handwriting. However, with these methods, it is difficult to completely distinguish between characters and graphics. For example, it is difficult to accurately determine the user's intention as to whether the character is “mouth” or a rectangle. In the method of switching by the user, the user's intention can be surely reflected through the user's operation, whereas in the automatic determination method, the determination of the input handwriting may be erroneous. If a determination error occurs, an operation for correcting the determination error occurs. On the contrary, the user's work efficiency may be deteriorated.

  The same applies to graphics and gestures. In order to perform processing suitable for graphics and graphics for gestures, it is first necessary to appropriately classify the type of handwriting. However, in the case of a gesture and a figure, as in the case of a character and a figure, when a figure similar to a gesture is input, there is a possibility that the input figure is determined as a gesture. Furthermore, in the case where characters and gestures coexist, it is possible to avoid discrimination errors to some extent by assigning handwriting having a shape not similar to characters as gestures. However, it is not always easy to use because users have to write gestures with the intention of distinguishing them from the shape of characters.

Japanese Unexamined Patent Publication No. 2002-203208 ([0028] to [0046], FIG. 1) JP-A-9-269974 ([0018] to [0024], FIG. 1) JP-A-8-180135 “Research on English Signature Identification by Tree Matching”, IEICE Technical Report D-2, Vol. J75-D-2, no. 1, pp 31-38, January 1992

  In the prior art, on devices that perform character input, figure input, and operation to a pen computer etc. based on user handwriting input, handwriting attributes (for example, gestures meaning characters, figures, operations (commands), etc.) In the case of inputting while setting by the method, there is a problem that the work is complicated. On the other hand, even when the handwriting attribute is automatically determined, there is a possibility of erroneous determination in the prior art, and accordingly, there is a problem that the user has to correct.

  The present invention has been made in consideration of the above problems, and determines objects on the screen located at the start and end points of handwriting information input by the user, and determines handwriting attributes based on the determined objects. Accordingly, an object of the present invention is to provide a handwriting processing device that eliminates the complexity of attribute setting and can easily input a handwriting intended by a user.

In order to achieve the above object, a handwriting processing apparatus according to the present invention includes handwriting acquisition means for acquiring handwriting information expressed by a coordinate series from when an indicator touches a coordinate input unit until it leaves, and the coordinate input A display unit for displaying the handwriting information, the coordinates of the start point and the end point of the handwriting information are obtained , and a handwriting, a figure, and a figure representing a region, an icon, a character, and a character are obtained. Each of the start point and the end point is based on an object database in which an object including any one of a handwriting representing an image, a handwriting representing a signature, and a combination of these and the position information of the object is stored in advance. An object determination unit that determines which object of the display unit is located on the basis of the object determined by the object determination unit; Attribute characters of the handwriting information, graphics, whether it is a kind of the gesture, or attributes of the handwriting information character, whether it is a kind of the gesture, or attributes of the handwriting information graphics, gesture Which type, or handwriting determination means for determining whether the attribute of the handwriting information is a character, a figure, a gesture, or a signature, and the attribute of the handwriting information in the handwriting determination means is a gesture against the determined handwriting information, it determines the type of the gesture from the determination object in said object determining unit, to execute a command corresponding to the type of the gesture to an object that is determined by the object determination unit And a gesture processing means.

In the handwriting processing apparatus according to the present invention, the gesture processing unit may include the object determined by the object determination unit with respect to the handwriting information in which the attribute of the handwriting information is determined as a gesture by the handwriting determination unit, and the object It determines the type of the gesture information from the shape of the handwriting information, characterized by comprising the gesture processing means for executing a command corresponding to the type of the gesture.

  As described above, according to the present invention, the user feels the complexity of changing the attributes of the handwriting by determining the input handwriting based on the positional relationship between the position of the handwriting and the object on the screen. Since it is possible to easily write whether the handwriting to be input is a character, a graphic, or a gesture depending on the input position, it is possible to input as intended.

  In addition, the user can input the handwriting without being aware of the shape of other types of handwriting. Therefore, it is possible to provide an input environment that is easy for the user to use.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram according to an embodiment of the present invention. A coordinate point sequence based on a handwriting handwritten by a user with an instruction (for example, a pen, a finger, etc., not shown) is input to the handwriting acquisition unit 2 in time series from the tablet 1 that performs coordinate input. The handwriting acquisition unit 2 stores the input handwriting in the handwriting buffer 3. As shown in FIG. 2, the handwriting buffer 3 additionally stores a time series (handwriting information) of handwriting coordinate data obtained from the tablet 1 for each stroke (stroke number). The handwriting display unit 4 displays the handwriting information in the handwriting buffer on the display unit 5. The coordinates of the tablet 1 and the coordinates of the display unit 5 correspond to each other, and the display-integrated tablet in which the tablet 1 is installed on the screen of the display unit 5 may be used.

  The object determination means 6 obtains the coordinates of the start point and the end point of the handwriting information in the handwriting buffer 3 and uses the object DB (database) 7 to determine which object on the display unit 5 screen each point is located on. Make a decision. As shown in FIG. 3A, the object DB 7 uses the number, type (area, icon, character (column), figure, image, etc.) and vertex of the polygon as coordinates for each object represented by a polygon. The position is recorded in advance. As shown in FIG. 3B, as the type of object, the position of a character (column) or figure may be represented by a vertex of a rectangle that circumscribes it, and the position of the figure may be a polygon of the displayed polygon. It may be represented by each vertex.

A method for determining whether a coordinate point is located within a polygon will be described with reference to FIG. When the start point or the end point of the handwriting information is P and the vertexes of the polygon to be determined are Q _{1 to} Q _n−1 , the angle formed by Q _i PQ _{i + 1} is α _i (however, including the sign) . As shown in FIG. 4 (a), when the outside of the closed curve, Shigumaarufa _i is smaller than 360 degrees, in that case determines that outside the polygon. On the other hand, as shown in FIG. 4B, when it is inside the closed curve, Σα _i is 360 degrees, and in this case, it is determined that it is inside the polygon.

  Object types include areas, icons, characters (columns) (text), handwriting representing characters (columns), figures, handwriting representing figures, images, handwriting representing signs, and the like. These may be combined into one object.

  The handwriting determination means 8 searches the handwriting determination DB 9 as shown in FIG. 5 using the determined object (number or type) of the start point and the end point as keys, and determines the attribute of the handwriting to be applied. To do. The attributes of the handwriting determined based on the handwriting determination DB may be classified into three types of characters, graphics, and gestures, or may be further classified. In FIG. 5, characters and gestures are each classified into a plurality of types. Characters 1 and 2 in the figure are distinguished by character type (for example, character 1 is all character types, character 2 is only hiragana), and character attributes are distinguished (for example, character 1 is an arbitrary character string, and character 2 is a date). And so on. In addition, the handwriting attributes may be classified into two types of characters, graphics, and gestures. In addition to the classification, the signature handwriting used for personal authentication may be classified into four categories: characters, graphics, gestures, and signatures. good.

  By the way, the object determination means 6 may determine only one of the start point and the end point. For example, FIG. 6 shows the handwriting determination DB 9 when only the start point is used, and the object (number or type) where the start point is located is searched for as a key. Moreover, the object determination means 6 may use the position of the polygon surrounding a handwriting for determination. For example, if the handwriting is a rectangle that circumscribes the handwriting, as shown in FIG. 7, the circumscribed rectangle has the maximum value and the minimum value (Xmin, Ymin, Xmax, Ymax) of the x coordinate and the y coordinate for all coordinate points of the handwriting. It can be determined by seeking.

  Furthermore, the object determination means 6 may determine an object where all coordinate points from the start point to the end point of the handwriting are located. A handwriting determination process is performed based on the position information. The handwriting determination DB 9 used by the handwriting determination means 8 is as shown in FIG. 8, for example. At this time, a search is performed by combining the keys in Table 1.

  The object determination means 6 may further determine an object surrounded by handwriting. At this time, the handwriting determination unit 8 searches the handwriting determination DB 9 by adding “an object surrounded by handwriting” to Table 1. The method for detecting the object surrounding the handwriting is as follows.

1. A closed curve determination process is performed to determine whether the handwriting forms a closed curve. When the handwriting is composed of a plurality of closed curves, the process 2 is performed for all the closed curves.
2. For all objects in the screen of the display unit, polygon points constituting the object are acquired from the object DB 7 and it is determined whether all the points constituting the object are inside the closed curve obtained in 1 (described above). (See description of FIG. 4). When all the points are inside, it is determined that the object is surrounded by the closed curve formed by the handwriting.

Closed curve determination processing is to check whether or not L _i and L _j (i <j) intersect when each line segment of a broken line is expressed by Li = P _i P _{i + 1} . i]... P [j + 1] is determined to constitute a closed curve. For example, as shown in FIG. 9, line segments L [2] and L [7] intersect, and P [2]... P [8] form a closed curve.

  In this embodiment, position information such as a start point and an end point is obtained from input handwriting information, and object determination processing is performed based on the position information. However, the following deformation processing is performed on handwritten handwriting. The object determination process may be performed using the handwritten information.

  For example, the handwriting handwritten on the tablet 1 is deformed by the smoothing process so that the handwriting is approximated and smooth. The input handwriting information is (X [1], Y [1]), (X [2], Y [2]),. . . (X [n], Y [n])

  And Further, the handwritten handwriting (FIG. 10A) may be determined by handwriting with a reduced number of points (FIG. 10B). You may perform an object determination process using the handwriting information which approximated the handwriting with the B-spline curve.

  As described above, according to the configuration of the first embodiment, it is possible to easily write whether the handwriting to be input is a character, a graphic, or a gesture depending on the input position without feeling the complexity of changing the attribute of the handwriting. You can input as you want.

Next, FIG. 11 is a configuration diagram according to another embodiment. The same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.
The handwriting classification means 10 classifies the attributes of the handwriting determined by the handwriting determination means 8 for each attribute. A corresponding handwriting processing means is performed for each classified handwriting information.

  For the handwriting information determined to be a character (string), the character handwriting processing means 11 performs character handwriting processing. In this character handwriting process, for example, character recognition is performed at the timing when a gesture corresponding to a character recognition command is input, and the character string (text) as a result of character recognition is replaced with the original handwriting information and stored in the handwriting buffer 3. The accumulated handwriting information in the handwriting buffer 3 is displayed on the display unit 5 through the handwriting display means 4. For the character recognition processing, Patent Document 2 or the like may be used.

  For the handwriting information determined to be a graphic, the graphic handwriting processing means 12 performs graphic handwriting processing. This graphic handwriting process may be data in which handwriting information is first stored in the handwriting buffer 3 or may be shaped by a graphic recognition process described in Patent Document 3 or the like.

  For the handwriting information determined to be a signature, the signature handwriting processing means 13 performs a signature handwriting process. In the signature handwriting process, handwriting information is stored in the handwriting buffer 3 as it is and used later for visual confirmation by the writer. Or you may perform a signature collation process (for example, nonpatent literature 1) with respect to handwriting information.

  For the handwriting information determined to be a gesture, the gesture handwriting processing means 14 performs gesture handwriting processing. In this gesture handwriting process, a command corresponding to the gesture determined by the handwriting determination unit 8 is executed based on the gesture DB 15 in which a command corresponding to the gesture type is stored in advance. As shown in FIG. 12, the gesture may execute a command corresponding to the type of gesture of the determination result, or may be further classified using shape information or the like as shown in FIG. .

  In FIG. 13, the handwriting information determined as a gesture is subdivided depending on whether it is rotating (forming a closed curve). Whether the handwriting information is rotated may use the closed curve determination method described in FIG. If the character handwriting erase command is assigned to the determined gesture, the handwriting information stored in the handwriting buffer 3 is erased. A plurality of commands may be assigned to the gesture.

  As described above, according to the configuration of the second embodiment of the present invention, it is possible to input handwriting without being conscious of the shape of other types of handwriting and to automatically convert intended handwriting information. An easy input environment can be provided.

Furthermore, FIG. 14 is a configuration diagram according to another embodiment. The same components as those in the first and second embodiments are denoted by the same reference numerals, and the subsequent description is omitted.
The determined handwriting information or conversion information obtained by converting the determined handwriting information is added to the object DB 7 by the object adding means 16 as a new object.
Thereby, handwritten information once input by handwriting or converted information obtained by converting handwriting can be used as an object, so that it is possible to provide an input environment that is easier for the user to use in situations where it is used repeatedly.

Specific operations of the above-described first to third embodiments will be described with reference to FIGS.
For example, a case where a handwritten comment on image (map) data is handwritten will be described.
The handwriting determination DB 9 is set as shown in FIG. At this time, if (1) the handwriting written around the image A (map) in FIG. 15B is determined (2) the handwriting is determined, the part of the arrow is determined to be a figure and the part “here is home” is determined to be the character. The When the recognition process is performed on the handwriting of the character part, “here is home” is converted into a text display as shown in (3), and when the recognition process (shaping) is performed on the handwriting of the graphic part, as shown in (4), Arrows are converted into a formatted display.

  When it is desired to handwrite a map from the beginning, the handwriting determination DB is set as shown in FIG. For example, when it is desired to handwrite a map in the region 1, the map is handwritten by handwriting (2) in the state (1) in FIG. When an arrow and a comment (3) are entered on the handwritten map and the handwriting is determined, the map and the arrow are determined to be figures and the “here is home” is determined to be a character. When the recognition process is performed on the handwriting of the character part, “here is home” is converted into the text display (4), and the handwriting of the graphic part. When the recognition process (shaping) is performed, the map and the arrow are converted into a shaped display (5).

  In the case of the sign field, if the handwriting determination DB shown in FIG. 8 is set such that “a part of the handwriting is in region 1” as shown in FIG. Thus, even if the handwriting protrudes from the sign column (area 1), it can be determined that the handwriting is a sign.

  In the case of a gesture, for example, in a transfer gesture that moves directly from region to region, the command in the gesture DB 15 is set to “transfer handwriting and objects in region A where the start point is located to region B where the end point is located”. . Therefore, as shown in FIG. 18, when the handwriting in the area A is to be transferred to the area B, it can be realized by an intuitive operation of writing a line from the area A to the area B.

  In a transfer gesture that moves to an area in another window using an icon, the handwriting determination DB shown in FIG. 19A is set, and the command of the gesture DB transfers the gesture 1 to the area 1. The gesture 2 is set to be transferred to the area 2. Therefore, the handwriting and the like in the region A can be transferred to the region 1 by the handwriting of the gesture in which the start point is located in the region A and the end point is located in the icon 1. This is particularly useful when another window cannot be pointed directly with the pen. (For example, it is out of reach / on a different screen where the pen device cannot be used)

  If the gesture determination process of the present invention as described above is used, there may be an operation similar to a drag and drop operation performed by operating a mouse on a PC. However, all operations based on handwriting are suitable for devices such as pen computers, various operations other than drag and drop can be realized with gesture functions suitable for handwriting input, characters, figures, etc. Other than the fact that other handwriting input can be realized in the same mode, it has an excellent feature not found in mouse-based operations.

  In addition, by using the determination method relating to the gesture of the present invention, the type of gesture is associated with an object positioned at the start point or end point of the input handwriting, so that the writer can perform an intuitive command operation.

The figure which shows the structure which concerns on Example 1 of this invention. The figure which shows the data accumulate | stored in a handwriting buffer. The figure which shows an example of a data structure of object DB. The figure explaining the method to determine inside / outside of a closed curve. The figure which shows an example of a data structure of handwriting determination DB. The figure which shows the other example of a data structure of handwriting determination DB. The figure explaining the method of determining the position of the polygon surrounding a handwriting. The figure which shows the other example of a data structure of handwriting determination DB. The figure explaining the closed curve determination method. The figure explaining the thinning-out process of handwriting information. The figure which shows the structure which concerns on Example 2 of this invention. The figure which shows an example of a data structure of gesture DB. The figure which shows the other example of a data structure of gesture DB. The figure which shows the structure which concerns on Example 3 of this invention. The figure which shows an example of operation | movement of this invention. The figure which shows the other example of operation | movement of this invention. The figure which shows the other example of operation | movement of this invention. The figure which shows the other example of operation | movement of this invention. The figure which shows the other example of operation | movement of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Display part 2 ... Handwriting acquisition means 3 ... Handwriting buffer 4 ... Handwriting display means 5 ... Display part 6 ... Object determination means 7 ... Object DB
8 ... Handwriting determination means 9 ... Handwriting determination DB
DESCRIPTION OF SYMBOLS 10 ... Handwriting classification means 11 ... Character handwriting processing means 12 ... Graphic handwriting processing means 13 ... Sign handwriting processing means 14 ... Gesture handwriting processing means 15 ... Gesture DB
16 ... Object addition means

Claims (2)

Handwriting acquisition means for acquiring handwriting information represented by a coordinate series until the indicator touches the coordinate input unit and then leaves;
A display unit having coordinates associated with the coordinate input unit and displaying the handwriting information;
The coordinates of the start point and the end point of the handwriting information are acquired, and the region, icon, character, handwriting representing the character, figure, handwriting representing the figure, image, handwriting representing the signature, or a combination thereof are included. An object determination means for determining on which object of the display unit each of the start point and the end point is based on an object database stored in advance in association with the object and the position information of the object ;
Based on the determined object by said object determining means, attribute characters of the handwriting information, graphics, whether it is a kind of the gesture, or, if the attribute of the handwriting information character, is any type of gesture Or handwriting determination means for determining whether the attribute of the handwriting information is a figure or a gesture, or whether the attribute of the handwriting information is a character, a figure, a gesture, or a signature ;
For the handwriting information in which the attribute of the handwriting information is determined to be a gesture by the handwriting determination unit, the type of the gesture is determined from the object determined by the object determination unit, and the command corresponding to the type of the gesture is A handwriting processing apparatus comprising gesture processing means for executing an object determined by the object determination means. The gesture processing means determines the type of gesture information from the object determined by the object determination means and the shape of the handwriting information with respect to the handwriting information determined by the handwriting determination means as the gesture information attribute. and, handwriting processing apparatus according to claim 1, characterized by including the gesture processing means for executing a command corresponding to the type of the gesture.

Images