JP6691597B2 - Information processing apparatus, information processing program, and information processing method - Google Patents

Description

  The present invention relates to an information processing device, an information processing program, and an information processing method, and more particularly, to an information processing device, an information processing program, and an information processing method for displaying characters handwritten using a touch panel on a display.

  The minutes creation support device disclosed in Patent Document 1 converts the input voice of a speaker into character information, and further extracts feature information from the input voice. Then, the character information is classified based on the extracted characteristic information, and the minutes are created by assigning arbitrary speaker identification information to the classified character information.

JP, 2010-60850, A

  However, in the background art described above, when the voice is not properly input, it is necessary to correct the character information by the input operation of the user. In addition, in this background technology, since all the input voices are included in the minutes, it is necessary to extract necessary items from the minutes once created and newly create minutes according to a predetermined format, which is troublesome. Is. Furthermore, this background art creates a minutes using voice, and it is not assumed to create a minutes according to a predetermined format from the minutes written on the electronic blackboard.

  Therefore, a main object of the present invention is to provide a novel information processing device, information processing program, and information processing method.

  Another object of the present invention is to provide an information processing device, an information processing program, and an information processing method capable of easily creating a new document in accordance with a predetermined document format using the written contents. is there.

  An information processing apparatus according to a first aspect of the present invention is an information processing apparatus that displays a handwritten image on a display device. The information processing apparatus corresponds to an input detection unit that detects an input instruction from a user, a predetermined document format, and various types of meetings. A storage unit that stores format data having a display field that displays the content of the item, and a range that specifies a range including at least a part of the handwritten image displayed on the display device according to the input instruction detected by the input detection unit. When the range including at least a part of the handwritten image is designated by the designating unit and the range designating unit, a menu display unit for displaying an item selection menu capable of selecting a plurality of items included in the format and a menu display unit are displayed. From the item selection menu, select one item according to the input instruction detected by the input detection means. Content storage means for storing a handwritten image included in the range designated by the item selection means and range designation means in the storage means as the content corresponding to one item selected by the item selection means, and the storage means by the content storage means The handwriting image stored in the item 1 is displayed in a display field for displaying the content of one item selected by the item selecting unit, and the handwriting image is used to create a new document conforming to the format.

  According to the first aspect of the present invention, it is possible to easily create a new document in accordance with the format of a predetermined document having a display field for displaying the contents of various items regarding the proceedings by using the handwritten image.

  The information processing apparatus according to the second aspect of the invention further includes an editing unit that adds, changes, or deletes an item included in the format according to the input instruction detected by the input detection unit.

  According to the second aspect, it is possible to edit the items included in the format according to the desired specifications. Therefore, it is easy to use.

  In the information processing device of the third invention, the editing means adds, changes or deletes the selectable items of the item selection menu when the items included in the format are added, changed or deleted.

  According to the third invention, since the selectable items of the item selection menu are edited in accordance with the editing of the format, the selected handwritten image can be easily associated with the desired item even if the format is changed. .

  The information processing apparatus according to the fourth aspect further includes size changing means for changing the size of the display field of the handwritten image in the format according to the size of the handwritten image included in the range designated by the range designating means. .

  The information processing apparatus according to the fifth aspect of the invention further includes adjusting means for adjusting the layout of the printed page when the size of the display field in the format is changed by the size changing means.

  In the information processing apparatus according to the sixth aspect of the invention, when one display field extends over two pages according to the layout of the printed page, the adjusting means inserts page break information at the head position of the display field.

  According to the fourth to sixth inventions, the user can create a new document conforming to the format using the handwritten image without worrying about the layout of the printed page.

  In the information processing device of the seventh invention, when the handwriting image included in the range designated by the range designating means by the content storing means is stored in the storing means, the handwritten image is displayed on the display device in a distinguishable manner. A mode changing means is further provided.

  According to the seventh aspect, the user can surely recognize the stored handwritten image.

  According to an eighth aspect of the present invention, information for displaying a handwritten image on a display device is provided, which includes a storage unit that stores format data corresponding to a predetermined document format and has a display column that displays the contents of various items regarding the proceedings. An information processing program executed by a processing device, comprising: a processor of the information processing device, an input detection unit that detects an input instruction from a user, and handwriting displayed on a display device according to the input instruction detected by the input detection unit. When a range specifying unit that specifies a range including at least a part of an image and a range including at least a part of a handwritten image are specified by the range specifying unit, an item selection menu capable of selecting a plurality of items included in the format is displayed. Input detection from the menu display means for selecting Storage means for selecting one item according to the input instruction detected by the step, and handwritten images included in the range designated by the range designating means as contents corresponding to one item selected by the item selecting means. The handwriting image is used by displaying the content storage means to be stored in, and the handwritten image stored in the storage means by the content storage means in the display column for displaying the content of one item selected by the item selection means. An information processing program that functions as a creation means for creating a new document that conforms to the specified format.

  A ninth invention is an input detecting means for detecting an input instruction by a user, and a storage means for storing format data corresponding to a predetermined document format and having a display column for displaying contents of various items related to proceedings. An information processing method of an information processing apparatus for displaying a handwritten image on a display device, comprising: (a) at least a part of the handwritten image displayed on the display device according to an input instruction detected by an input detection unit. A step of specifying a range to be included, (b) a step of displaying an item selection menu capable of selecting a plurality of items included in the format when a range including at least a part of the handwritten image is specified in step (a), (c) ) One from the item selection menu displayed in step (b) according to the input instruction detected by the input detection means. An item selecting step, (d) a step of storing the handwritten image included in the range specified in step (a) in the storage means as contents corresponding to one item selected in step (c), (e) step The handwritten image stored in the storage means in (d) is displayed in the display field for displaying the content of one item selected in step (c), thereby creating a new document conforming to the format using the handwritten image. An information processing method including steps.

  In the eighth and ninth inventions as well, similar to the first invention, a handwritten image is used to create a new document in accordance with a predetermined document format having a display field for displaying the contents of various items related to the proceedings. Can be easily created.

  According to the present invention, since the handwritten image included in the predetermined range designated by the user is simply stored in association with the selected item, a new document conforming to a predetermined document format can be easily used by using the handwritten image. Can be created.

  The above-mentioned objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

FIG. 1 is a block diagram showing the electrical configuration of the information processing apparatus of the first embodiment. 2A is an illustrative view showing an example of a touch screen displayed on the display, and FIG. 2B is an illustrative view showing a second example of the touch screen displayed on the display. FIG. 3 is an illustrative view showing one example of the format of the minutes. FIG. 4A is an illustrative view showing a third example of the touch screen displayed on the display, and FIG. 4B is an illustrative view showing a fourth example of the touch screen displayed on the display. FIG. 5A is an illustrative view showing a fifth example of the touch screen displayed on the display, and FIG. 5B is an illustrative view showing a sixth example of the touch screen displayed on the display. FIG. 6 is an illustrative view showing an example of the created minutes. FIG. 7 is an illustrative view showing one example of a memory map of the RAM shown in FIG. FIG. 8 is an illustrative view showing one example of a data structure of the minutes data shown in FIG. 7. FIG. 9 is a flowchart showing a part of an example of the drawing control processing of the CPU shown in FIG. FIG. 10 is a flowchart showing another part of the drawing control processing of the CPU shown in FIG. 1 and sequel to FIG. 9. 11 is a flowchart showing another part of the drawing control processing of the CPU shown in FIG. 1 and sequel to FIG. FIG. 12 is a flowchart showing another part of the drawing control processing of the CPU shown in FIG. 1 and sequel to FIG. 11. FIG. 13 is a flowchart showing the range designation processing of the CPU shown in FIG. FIG. 14 is an illustrative view showing one example of a touch screen displayed on the display in the second embodiment. FIG. 15 is an illustrative view showing one example of the minutes created in the second embodiment. FIG. 16 is an illustrative view showing one example of the layout of the printed page of the created minutes. FIG. 17 is an illustrative view showing another example of the layout of the printed page of the created minutes. FIG. 18 is a flowchart sequel to FIG. 11 showing a part of the drawing control processing of the CPU in the second embodiment. FIG. 19 is an illustrative view showing one example of a touch screen displayed on the display in the third embodiment. FIG. 20 is a flowchart showing a part of the drawing control processing of the CPU in the third embodiment and sequel to FIG. 11.

<First embodiment>
FIG. 1 is a block diagram showing the electrical configuration of the information processing apparatus 10 of the first embodiment.

  Referring to FIG. 1, an information processing device 10 according to the first embodiment of the present invention includes a CPU 12. A RAM 14, a touch panel control circuit 16, and a drawing control circuit 18 are connected to the CPU 12 via a bus 30. A touch panel 20 is connected to the touch panel control circuit 16, and a display 22 is connected to the drawing control circuit 18.

  Although illustration is omitted, the information processing apparatus 10 is also provided with a non-volatile memory such as an HDD and a ROM, and is connected to the CPU 12 via the bus 30.

  In the first embodiment, the information processing device 10 is applied to various information devices or electronic devices such as an electronic blackboard, a tablet terminal, a smart phone, and a PC.

  In the first embodiment, the case where the touch panel 20 is used as an example of the input means will be described. However, as the input means other than the touch panel 20, for example, a pointing device such as a computer mouse, a touch pad, or a pen tablet is used. Good. In addition, the information processing apparatus 10 may be provided with a hardware key such as an operation panel, or a hardware keyboard may be connected as another input unit.

  Returning to FIG. 1, the CPU 12 controls the entire information processing apparatus 10. The RAM 14 is used as a work area and a buffer area of the CPU 12.

  The touch panel control circuit 16 applies a necessary voltage and the like to the touch panel 20, detects a touch operation (touch input) within the touch effective range of the touch panel 20, and outputs touch coordinate data indicating the position of the touch input to the CPU 12. Output to.

  The touch panel 20 is a general-purpose touch panel, and an arbitrary type such as a capacitance type, an electromagnetic induction type, a resistance film type, an infrared type can be used. In the first embodiment, as the touch panel 20, a capacitance type touch panel is provided on the display surface of the display 22.

  The drawing control circuit 18 includes a GPU, a VRAM and the like, and under the instruction of the CPU 12, the GPU uses the handwriting input data 332 and the image generation data 334 (see FIG. 9) stored in the RAM 14 to display the display 22. Screen data for displaying a screen (touch screen 100 described later) is generated in the VRAM, and the generated screen data is displayed on the display 22. As the display 22, for example, an LCD or an EL (Electro-Luminescence) display can be used.

  In the information processing apparatus 10 having such a configuration, in the handwriting input mode, the user uses the touch panel 20 to input characters, figures, symbols and the like (hereinafter, sometimes referred to as “characters”) by handwriting (touch input). Then, the touch panel control circuit 16 detects the touch input and outputs the touch coordinate data 330 corresponding to the touch position to the CPU 12. The CPU 12 draws (displays) a handwritten character or the like on the display 22 based on the touch coordinate data output from the touch panel control circuit 16. That is, under the instruction of the CPU 12, handwritten characters and the like are drawn in the VRAM in the drawing control circuit 18, and image data (included in the screen data) corresponding to the image including the handwritten characters and the like drawn in the VRAM. Is output to the display 22. Therefore, an image including characters handwritten by the user is displayed on the display 22. Hereinafter, in this specification, an image including a handwritten character will be referred to as a “handwritten image” on the assumption that the user handwrites the character.

  FIG. 2A is an illustrative view showing an example of the touch screen 100 (initial screen) before handwriting is started. That is, FIG. 2A is an example of the touch screen 100 displayed on the display 22 by executing the drawing application, and shows the initial touch screen 100 when the handwriting input mode is started. Further, FIG. 2B is a second example of the touch screen 100 displayed on the display 22, and shows the touch screen 100 in a state where the minutes are handwritten (blackboard).

  As shown in FIG. 2A, for example, at the beginning of the handwriting input mode, a touch screen 100 of a blank page on which a handwritten image is not displayed is displayed on the display 22. However, when the stored image data of the handwritten image is read, the touch screen 100 including the handwritten image is displayed. As described above, the touch panel 20 is provided on the display surface of the display 22. The same applies hereinafter.

  Further, as shown in FIG. 2A, an image (icon) of a button (function button) exhibiting a predetermined function is displayed at the left end portion of the touch screen 100. In the first embodiment, the minutes edit button 102 and the range designation button 104 are displayed. The minutes editing button 102 is a button for editing items of the minutes and entering texts of some items. The range designation button 104 is a button for designating the range of the handwritten image to be selected.

  Although illustration is omitted, on the touch screen 100, attribute buttons for selecting drawing attributes such as line type, line width (thickness), and line color, and other functions (open new page, open image) Icons such as buttons for selecting (insert), save, undo, etc. are also displayed.

  For example, the user operates the touch panel 20 with the dedicated touch pen 110. However, the user can also operate with fingers. Operations (inputs) using the touch panel 20 include taps (short press), slides (drag), flicks, and long touches (long press). In the first embodiment, these are referred to as “touch input” or simply. Collectively referred to as "input". Further, the change from the state where the touch panel 20 is not touched to the touch state is called touch-on (pen down), and the change from the state where the touch panel 20 is touched to the non-touch state is touch-off (pen down). Up). In response to continuous touch input, that is, input by sliding or flicking, the touch panel 20 outputs touch coordinate data corresponding to the current touch position in a cycle shorter than a predetermined cycle. For example, the predetermined cycle is one to several frames, and one frame is 1/30 second, 1/60 second or 1/120 second.

  In a general electronic blackboard, which is an example of the information processing device 10, when a user handwrites a character or the like, the handwritten character or the like is displayed (drawn) on the touch screen 100. That is, a handwritten image including handwritten characters and the like is displayed on the display 22 (touch screen 100). At this time, the handwritten image is drawn (displayed) according to the set attribute information (line type, line color, line width).

  For example, in a meeting or a meeting, as shown in FIG. 2B, the content of the proceedings is written on the board using the information processing device 10. Therefore, the written contents can be printed as the minutes as they are or distributed (delivered) by e-mail.

  However, when the content of the proceedings is written on the board using the information processing device 10, the written content is often disorderly and handwritten in pieces. Therefore, if the content written on the blackboard is used as it is as the minutes, it becomes very difficult to see. For this reason, it is necessary to create the minutes again based on the written contents, which is troublesome.

  In order to avoid such an inconvenience, in the first embodiment, the minutes can be easily created by using (according to) a predetermined minutes format from the handwritten (blackboard) contents. The details will be described below.

  FIG. 3 is an illustrative view showing one example of the format of the minutes. In the first embodiment, data (format data) regarding the format of a predetermined document (here, minutes) created in advance is stored in the memory (for example, HDD) of the information processing device 10. As shown in FIG. 3, the format of the minutes is a display column (hereinafter referred to as “item column”) that displays various items regarding the proceedings (discussion contents, deliberation items, voting results, etc.), and these various types. It is composed of a display column (hereinafter, referred to as "content column") for displaying the content of the item.

  Further, as shown in FIG. 3, in the item columns, “meeting name”, “holding date”, “holding place”, “participant”, “deliberation content”, “conclusion”, “homework matter” and “remarks” Is described (displayed). Further, as described above, the content column is provided corresponding to each item column. However, in the minutes format, each content column is blank. Further, in the format of the minutes, the size (vertical and horizontal length) of each of the item column and the content column is set to a predetermined size. In the format of the minutes shown in FIG. 3, each item column is set to the same size, and similarly, each content column is set to the same size. For example, if you print the minutes on a paper of a predetermined size (B5, A4, B4, etc.), set the size of the item and content fields so that all the items and content fit on one sheet of paper. To be done.

  4 (A), 4 (B), 5 (A) and 5 (B) are other examples of the touch screen 100 displayed on the display 22, showing how to create a minutes. 1 shows a touch screen 100 for explaining the above. Further, FIG. 6 is an illustrative view showing an example of the minutes created by using a handwritten image in accordance with the format of the minutes.

  Briefly, the user selects desired content (all or part of the handwritten image) from the content written on the blackboard (all of the handwritten image). Next, the user selects (designates) an item in which the selected desired content should be described (displayed). Then, the selected desired content is stored in the memory (for example, the RAM 14) in association with the selected (designated) item. Such processing is repeated, desired contents are stored in association with each item, and minutes are created. The details will be described below.

  As shown in FIG. 4A, when the user touches the range designation button 104, the mode for designating the range is set. Subsequently, as shown in FIG. 4B, the user uses the touch pen 110 to drag diagonally so as to surround desired contents. At this time, the frame 106 is designated (displayed) according to the drag. However, the shape and size of the frame 106 are determined by the touch-on position when starting dragging and the touch-off position when ending dragging. When the frame 106 is designated, the handwritten image surrounded by the frame 106 is selected as the desired content. That is, the handwritten images of the points and lines included in the frame 106 are recognized as one group.

  For example, as shown in FIG. 4B, a two-dimensional coordinate system (screen coordinate system) is set on the display 22, and the lower left vertex of the display surface of the display 22 is the origin O (0,0). Is set. Further, the position and size of the frame 106 are determined by a point P1 (x1, y1) which is an upper left apex and a point P2 (x2, y2) which is a lower right apex.

  The characters in the coordinate system shown in FIG. 4B and the points P1 and P2 are only used for internal processing of the information processing apparatus 10 and are not displayed on the display 22.

  Further, the coordinate system does not need to be limited to the mode of the first embodiment, and can be set arbitrarily.

  Furthermore, in the first embodiment, the frame 106 is specified by dragging, but the frame 106 may be specified by tapping two diagonal points of the frame 106.

  As described above, when the frame 106 is designated (displayed on the touch screen 100) so as to surround the handwritten image, as shown in FIG. 5A, an arbitrary action including the minutes is created near the frame 106. A menu (hereinafter, referred to as “action selection menu”) 120 for selection is displayed.

  Although omitted in FIG. 5A, in the action selection menu 120, other arbitrary actions such as copy, cut, and delete are displayed in a selectable manner.

  When the minutes creation is touched (selected) in the action selection menu 120 shown in FIG. 5A, the items in the minutes are selected instead of the action selection menu 120 as shown in FIG. 5B. The item selection menu 130 is displayed.

  However, when Cancel is selected in the action selection menu 120, the action selection menu 120 is erased (hidden), the frame 106 is erased, and desired contents (all or part of the handwritten image) are selected. It will be canceled.

  When another action is selected, the selected desired content is copied, cut, deleted, or the like.

  As can be seen from FIG. 5B, the item selection menu 130 displays a list of some of the items described in the item column shown in the format of the minutes in FIG.

  Each item included in the item selection menu 130 is associated with each item in the corresponding minutes format, and when one item is selected from the item selection menu 130, the desired content selected earlier is selected. (All or part of the handwritten image) is determined as the content displayed in the content column corresponding to the item column in which the associated item in the minutes format is described.

  Therefore, when an appropriate item (here, “meeting name”) is selected from the items displayed in the item selection menu 130, the handwritten image recognized as a group in the frame 106 is selected by the selected item ( Here, it is stored in the RAM 14 in association with the “meeting name”).

  The item selection menu 130 in the first embodiment does not include the date and time of the event, the place of the event, and the participants in the minutes format shown in FIG. Since it is considered that the contents of these items are usually hardly written on the blackboard, the user separately inputs them in the first embodiment. However, this is only an example, and the item selection menu 130 may include the date and time of the event, the place of the event, and the participants, like other items.

  As described above, the user repeatedly performs the operation of selecting desired content (all or part of the handwritten image) from the written content (all of the handwritten image) and selecting an appropriate item from the item selection menu 130. By doing so, the minutes can be created according to the minutes format using the handwritten image. An example of the minutes created is shown in FIG. For example, when the user issues an instruction to display the minutes, the minutes instructed to be displayed are displayed on the display 22.

  However, in the first embodiment, the date and time of the event, the place of the event, and the contents of the participants are text-input using a keyboard or a computer mouse (hereinafter referred to as “keyboard or the like”) connected to the information processing apparatus 10. For example, when the minutes editing button 102 is touched on the touch screen 100, the format of the minutes is displayed on the display 22 (touch screen 100). In the format of the minutes, the content column corresponding to the item for text input is selected, and the content to be displayed (described) in the selected content column is text input. When the content to be displayed in the content field is input as text, the data of the character string is stored in the RAM 14 in association with the item described in the item field corresponding to the content field, as in the case of a handwritten image. . Further, in the header and footer, information related to the minutes can be input as text. In the example of the minutes shown in FIG. 6, the date when the minutes are created is input as text in the header. As the footer, a page number or the like can be input as text. When inputting a header or footer, for example, inputting a header or footer is selected from a menu (not shown).

  In the first embodiment, the date and time of the event, the event place, and the contents of the participants are input by text using the keyboard or the like, but may be handwritten using the touch pen 110.

  In addition, it is also possible to input texts about contents other than the date and time of the event, the place of the event, and the contents of the participants.

  Also, items in the minutes format can be edited (added, changed, deleted). For example, as described above, in the format of the minutes displayed on the display 22, a desired item is selected, the selected item is changed or deleted, an item display column is added, and the added item A desired item may be added (described) in the display field. Such editing work is performed using a keyboard or the like connected to the information processing device 10. As described above, when the item in the minutes format is edited, the item included (displayed) in the item selection menu 130 is also edited.

  The above-described operation of the information processing device 10 is realized by the CPU 12 executing the information processing program stored in the RAM 14. In the first embodiment, as an example of the information processing program, a program (drawing control program) for a drawing application that draws and saves handwritten characters is executed. Specific processing will be described later using a flow chart.

  FIG. 7 shows an example of the memory map 300 of the RAM 14 shown in FIG. As shown in FIG. 7, the RAM 14 includes a program storage area 302 and a data storage area 304. A drawing control program is stored in the program storage area 302. The drawing control program includes an input detection program 310, an image generation program 312, a display program 314, a range designation program 316, and a minutes creation program 318.

  The input detection program 310 is a program for acquiring the touch coordinate data 330 indicating the touch input position on the touch panel 20 output from the touch panel control circuit 16 and storing the touch coordinate data 330 in the data storage area 304. However, the input detection program 310 is also a program for detecting an input from the keyboard of the hardware connected to the information processing apparatus 10 or the operation panel or the operation button of the hardware provided in the information processing apparatus 10.

  The image generation program 312 uses touch coordinate data (handwriting input data 332 described below) detected according to the input detection program 310 or uses image generation data 334 described below to display a screen corresponding to the touch screen 100. It is a program for generating data. Specifically, when the image generation program 312 is executed, the GPU draws screen data corresponding to the touch screen 100 in the VRAM in the drawing control circuit 18 under the instruction of the CPU 12.

  The display program 314 is a program for outputting the screen data generated according to the image generation program 312 to the display 22. Therefore, the touch screen 100 is displayed on the display 22.

  The range designation program 316 is a program for designating a range in the display area of the display 22 according to a touch input by the user. In the first embodiment, the range of the rectangular frame determined based on the touch-on position and the touch-off position when dragging is the designated range.

  The minutes preparation program 318 is a program for storing in the RAM 14 the image data corresponding to the handwritten image within the designated range defined by the range designation program 316 in association with the designated (selected) item. Further, the minutes creating program 318 inputs texts about some items of the minutes according to the user's operation, edits items included in the minutes format, and items included in the minutes format. It is also a program for editing the items of the item selection menu 130 in accordance with the editing of.

  Although illustration is omitted, the drawing control program also includes programs for selecting and executing various functions.

  The data storage area 304 stores touch coordinate data 330, handwriting input data 332, image generation data 334, minutes data 336, frame data 338, and the like.

  The touch coordinate data 330 is the current (in the current frame) touch coordinate data detected (acquired) according to the input detection program 310. When the touch coordinates indicated by the touch coordinate data 330 is included in the display area of the button (range specification button 104 or the like) displayed on the touch screen 100, the function (action) assigned to the button is executed, or the attribute is executed. (Attribute information) is set. If the touch coordinates indicated by the touch coordinate data 330 are not included in the display area of the button displayed on the touch screen 100, it is determined that the touch coordinates are included in the handwritten characters, and the handwriting input data is determined. It is stored as 332.

  The handwriting input data 332 is data on characters handwritten by touch input, and is managed for each locus (point or line) from touch-on to touch-off. In the handwriting input data 332, the touch coordinate data 330 detected for each frame from touch-on to touch-off is stored (arranged) in time series.

  The image generation data 334 is data such as polygon data and texture data for generating screen data corresponding to various screens such as the touch screen 100. The image generation data 334 also includes image data for various buttons (102, 104, etc.) and menus (120, 130, etc.) displayed on the touch screen 100.

  The minutes data 336 is data about the minutes, and includes format data 336a and each content data 336b. Specifically, as shown in FIG. 8, the format data 336a is data regarding the format of the minutes as shown in FIG. Each content data 336b is data about the content stored in association with each item of the minutes.

  FIG. 8 shows an example of the data structure of the minutes data 336. In the data structure shown in FIG. 8, the format data 336a corresponds to data regarding the size of the character string for each item, each item column, and each content column. However, in FIG. 8, the item column and the content column are collectively described as “display column”. Further, in the data structure shown in FIG. 8, each content data 336b corresponds to the data of the content described in correspondence with each item of the format data 336a and other data. However, in FIG. 8, the symbol "+" means that the items on the left side correspond to the items on the right side. Further, as the other data, data on the contents such as headers and footers and the display positions thereof are applicable.

  Returning to FIG. 7, the frame data 338 is data for defining the position and range of the frame 106. As described above, the frame data 338 is coordinate data of the point P1 that is the upper left apex of the frame 106 and the point P2 that is the lower right apex of the frame 106.

  In the data storage area 304, other data necessary for executing the drawing control program is stored, and a timer (counter) or register necessary for executing the drawing control program is provided.

  9 to 12 are flowcharts showing the drawing control processing of the CPU 12 shown in FIG. For example, the drawing application is started, and the handwriting input mode is selected by the user's instruction or by the default setting. Then, as shown in FIG. 10, the CPU 12 starts the drawing control process and executes the initial process in step S1. In the initial process of step S1, the CPU 12 erases the touch coordinate data 330 in the data storage area 304, reads the image generation data 334 from the non-volatile memory, and then displays the initial touch screen 100 on the display 22. indicate. That is, under the instruction of the CPU 12, the drawing control circuit 18 generates (draws) the screen data of the initial touch screen 100 as shown in FIG. 2A and outputs it to the display 22. However, when the user gives an instruction to read the stored handwritten image data, the touch screen 100 including the handwritten image is displayed on the display 22.

  In the next step S3, it is determined whether or not the touch is on. Here, the CPU 12 determines whether or not the current (current frame) touch coordinate data 330 is stored in the data storage area 304.

  If "NO" in the step S3, that is, if it is not touch-on, the process returns to the same step S3. On the other hand, if “YES” in the step S3, that is, if touch-on, it is determined in a step S5 whether the minutes edit button 102 is touched. Here, the CPU 12 determines whether the touch position indicated by the acquired touch coordinate data 330 indicates the display area of the minutes edit button 102. Hereinafter, the same applies to the case of determining whether or not the button displayed on the touch screen 100 has been touched.

  If "NO" in the step S5, that is, if the minutes editing button 102 is not touched, the process proceeds to a step S19 shown in FIG. On the other hand, if “YES” in the step S5, that is, if the minutes editing button 102 is touched, the minutes format is displayed on the display 22 in a step S7, and it is determined whether or not the item is edited in a step S9. to decide. However, when the minutes edit button 102 or the action selection menu 120 is first touched (selected), the minutes format data 336a is read from the HDD and stored in the RAM 14. Further, in step S7, under the instruction of the CPU 12, the drawing control circuit 18 displays the minutes format on the display 22 using the minutes format data 336a read out to the RAM 14.

  If "YES" in the step S9, that is, if the item is edited, the format of the minutes in which the item is edited is stored in a step S11. Here, the CPU 12 stores (overwrites) the format data 336a of the minutes in which the item is edited in the RAM 14. In the next step S13, the items of the item selection menu 130 are edited in accordance with the editing of the format, and the process proceeds to step S19. In step S13, the CPU 12 edits the notation of the item selection menu 130 in the data of the item selection menu 130 included in the image generation data 334 according to the editing of the minutes format, and updates the data.

  On the other hand, if "NO" in the step S9, that is, if the item is not edited, it is determined in a step S15 whether or not the content is text input. If "NO" in the step S15, that is, if the text is not input, the process directly returns to the step S15. However, the CPU 12 may end the editing of the minutes when the editing of the minutes is canceled or when the editing of the item and the text input of the contents are not executed even after a predetermined time elapses.

  On the other hand, if “YES” in the step S15, that is, if the content is text-input, in a step S17, the data of the character string text-input in association with the item (included in each content data 336b) is stored in the RAM 14. Store and proceed to step S19.

  As shown in FIG. 10, in step S19, it is determined whether the range designation button 104 has been touched. If “YES” in the step S19, that is, if the range designation button 104 is touched, the process proceeds to a step S31 shown in FIG. On the other hand, if “NO” in the step S19, that is, if the range designation button 104 is not touched, it is determined whether another button is touched in a step S21. That is, the CPU 12 determines whether any other button except the minutes edit button 102 and the range specification button 104 has been touched.

  If "YES" in the step S21, that is, if a button other than the minutes editing button 102 and the range specification button 104 is touched, a process corresponding to the other button is executed in a step S23, and a step S3 is performed. Return to. For example, in step S23, the CPU 12 changes the line attribute information (line type, line color, line width), copies a handwritten image, or pastes the copied image according to a user operation (instruction). , Specify a range of a handwritten image, and save image data of the handwritten image (handwritten input data 332) in a non-volatile memory.

  On the other hand, if “NO” in the step S21, that is, if any button other than the minutes editing button 102 and the range specification button 104 is not touched, it is determined that the touch input is for handwriting, and a step S25 is performed. , Touch coordinates are stored as handwriting input. That is, in step S25, the CPU 12 stores the touch coordinate data 330 according to the touch input in the RAM 14 as the handwriting input data 332. Then, in step S27, the handwritten image is displayed according to the touch input, and the process proceeds to step S29. In step S27, under the instruction of the CPU 12, the drawing control circuit 18 draws the screen data of the touch screen 100 including the handwritten image in the VRAM using the handwriting input data 332 and the image generation data 334, and outputs the screen data to the display 22. Therefore, during handwriting input, a line is drawn by a plurality of points according to the touch input, and a handwritten image such as a character is drawn by one or a plurality of lines.

  Then, in a step S29, it is determined whether or not the touch is off. Here, the CPU 12 refers to the data storage area 304 and determines whether or not the touch coordinate data 330 of the current frame is stored. If "NO" in the step S29, that is, if the touch-on state continues, the process returns to the step S17. Therefore, of the handwritten input data 332, the data on the line being drawn is updated, and the line is displayed (drawn) according to the touch input. On the other hand, if “YES” in the step S29, that is, if the touch is off, the process returns to the step S3.

  As shown in FIG. 11, in step S31, a range designation process (see FIG. 13) described later is performed, and in step S33, the action selection menu 120 is displayed as shown in FIG. In S35, it is determined whether the action selection menu 120 has been touched.

  If "NO" in the step S35, that is, if the action selection menu 120 is not touched, the process returns to the same step S35. However, when the touch screen 100 other than the action selection menu 120 is touched, the range designation (selection of the handwritten image) may be canceled and the process may return to step S31 or step S3.

  On the other hand, if “YES” in the step S35, that is, if the action selection menu 120 is touched, it is determined in a step S37 whether the minutes preparation is selected (touched). This determination is similar to the case of determining whether or not the button has been touched. The same applies when another menu is selected.

  If "YES" in the step S37, that is, if the minutes creation is selected, the process proceeds to a step S47 shown in FIG. On the other hand, if "NO" in the step S37, that is, if the minutes preparation is not touched, it is determined whether or not the cancel is touched in a step S39. If "NO" in the step S39, that is, if the cancel is not touched, the process according to the other action as described above is executed in a step S41, and the process proceeds to the step S43.

  On the other hand, if “YES” in the step S39, that is, if Cancel is touched, the action selection menu 120 is erased in a step S43, the frame 106 is erased in a step S45, and the step S3 shown in FIG. Return to. However, in step S43 or S45, the CPU 12 cancels the range designation (selection of the handwritten image).

  As shown in FIG. 12, in step S47, the action selection menu 120 is erased, and in step S49, the item selection menu 130 is displayed as shown in FIG. 5B. Then, in step S51, it is determined whether any item is selected (touched) from the item selection menu 130.

  If "NO" in the step S51, that is, if the item is not touched, the process returns to the same step S51. However, when the touch screen 100 other than the item selection menu 130 is touched, the range designation (selection of the handwritten image) may be canceled and the process may return to step S31 or step S3.

  On the other hand, if “YES” in the step S51, that is, if any item is touched from the item selection menu 130, in a step S53, the item is associated with the selected item and designated in the step S31. The handwritten image within the range is stored. Here, the CPU 12 stores a copy of the image data of the handwritten image (included in the handwriting input data 332) within the specified range in the RAM 14 as a part of each content data 336b in association with the selected item.

  Then, in step S55, the item selection menu 130 is erased, in step S57, the frame 106 is erased, and the drawing control process ends.

  The minutes (minutes data 336) are created by repeating the range designation and the item selection. Although not described in detail, for example, the created minutes data 336 is stored in the HDD in association with the image data of the handwritten image.

  FIG. 13 is a flowchart of the range designation process of step S31 shown in FIG. As shown in FIG. 13, when starting the range designation processing, the CPU 12 determines in step S81 whether a start point has been designated. That is, the CPU 12 determines whether or not the touch coordinate data is not output from the touch panel 20 to the touch coordinate data output state.

  If “NO” in the step S81, that is, if the start point is not designated, the process directly returns to the step S81. On the other hand, if “YES” in the step S81, that is, if the starting point is designated, the coordinates of the designated starting point are stored in a step S83. Here, the coordinate data of the starting point (point P1) of the frame data 338 is stored.

  Then, in step S85, it is determined whether the end point is designated. Here, the CPU 12 determines whether the state in which the touch coordinate data is output from the touch panel 20 has changed to the state in which the touch coordinate data is not output.

  If “NO” in the step S85, that is, if the end point is not designated, the frame 106 defined by the start point and the current touch position is displayed on the touch screen 100 in a step S87, and the process returns to the step S85.

  On the other hand, if “YES” in the step S85, that is, if the end point is designated, the coordinates of the designated end point are stored in a step S89. Here, the coordinate data of the end point (point P2) of the frame data 338 is stored. Therefore, the frame data 338 (frame 106) is determined (designated). Then, in step S91, the frame 106 defined by the start point and the end point is displayed on the touch screen 100, and the process returns to the drawing control process.

  According to the first embodiment, since the handwritten image within the predetermined range designated by the user is stored in association with the item selected from the menu, the handwritten image is used to follow the format of the minutes. Recording can be easily created.

  Further, in the first embodiment, the minutes are created according to the format of the minutes by using the handwritten image, but the present invention is not limited to this. For example, an abstract summarizing the content (handwritten image) written on a blackboard in a school class may be created. In such a case, the format of the abstract will include items such as "class theme", "important points", and "homework" in the item column, and a content column will be provided in association with each of these items. To be Then, as in the case of the first embodiment, the handwritten image included in the designated range is stored in association with the item selected from the menu. In this way, the handwritten image is used to create the abstract according to the format of the abstract.

In the first embodiment, when the minutes are created, the handwritten images in the frame 106 are grouped and stored in association with the selected item, but the present invention is not limited to this. . For example, the handwritten images in the frame 106 may be grouped and then subjected to character recognition to be converted into text characters or character string data and then stored.
<Second embodiment>
The information processing apparatus 10 of the second embodiment is the same as the information processing apparatus 10 of the first embodiment except that the display field in the format of the minutes can be enlarged. However, duplicate description will be omitted.

  Briefly described, in the second embodiment, when the handwritten image included in the specified range does not fit in the content field, the content field and the corresponding item field (hereinafter, the content field and the item field are collectively referred to as “display field”). It may be called)) is expanded. Further, when the display column extends over two pages due to the enlargement of the display column, the layout of the printed page is adjusted.

  FIG. 14 is an illustrative view showing one example of the touch screen 100 displayed on the display 22 in the second embodiment. As can be seen from FIG. 14, the second embodiment is the same as the touch screen 100 shown in FIGS. 4 and 5 of the first embodiment except that the number of blackboard contents is increased, and thus the duplicate description is omitted. To do. Also in the second embodiment, as described in the first embodiment, the minutes according to the format of the minutes are created using the handwritten image.

  FIG. 15 is an illustrative view showing one example of the minutes created in the second embodiment. In the minutes shown in FIG. 15, the size of the item column for homework items and the corresponding content column, that is, the display column is enlarged. This is because the handwritten image selected as the content of the homework item is larger than the size of the content column corresponding to the item column of the homework item in the minutes format. For example, the height of the selected handwritten image is compared with the height of the content field in which the handwritten image should be displayed. If the height of the handwritten image is larger than the height of the content field, the minutes In the format, the vertical width of the content field is increased to accommodate the handwritten image, and the corresponding item field is increased accordingly. That is, the display field including the content field for displaying the handwritten image is enlarged according to the size of the selected handwritten image.

  However, the vertical width (size) of the selected handwritten image is not the vertical width (size) displayed on the display 22, but the vertical width (size) corresponding to the paper size of the minutes to be created. ) Is the vertical width (size) when converted (changed). Therefore, the second embodiment can be applied to the information processing apparatus 10 having various displays 22 having different sizes of the display surface or the display area.

  Further, in the second embodiment, the size of the display field is increased in the display field of the format of the minutes corresponding to the format data 336a included in the minutes data 336.

  Further, in the second embodiment, when the vertical width of the selected handwritten image is larger than the vertical width of the content field, the vertical width of the display field is increased, but the invention is not limited to this. There is no. For example, when the horizontal width of the selected handwritten image is larger than the horizontal width of the content field, the horizontal width of the content field may be increased. Further, the size may be changed for both the vertical width and the horizontal width. However, when the width of the content field is increased, the size of the corresponding item field is not changed.

  However, when the widths of the content columns are increased, the widths of all the content columns are increased.

  FIG. 16 is an illustrative view showing one example of the layout of the printed page of the minutes created in the second embodiment. In addition, FIG. 17 is an illustrative view showing another example of the layout of the printed page of the minutes created in the second embodiment.

  In addition, it is expected that the minutes will be printed for distribution to related parties. In addition, as described above, when the vertical width of the display field is enlarged, the minutes may be printed on a plurality of pages. For example, as shown in FIG. 16, when the minutes are printed on two pages, one display field may be divided into two pages. That is, one display field may be printed across page breaks.

  For this reason, in the second embodiment, when one display field crosses a page break, the page break information is moved to the head position of the display field (head position in the printing direction).

  Normally, the page break information is determined according to the paper size and the size of the margin, and is set at the end position (the end position in the printing direction) of each page excluding the margin.

  Further, in FIG. 16, the layout of the print page is shown for the sake of easy understanding, but the layout is not limited to this, and the page break information may be moved when the minutes are displayed in the edit mode. Good.

  Specifically, as shown in FIG. 16, since the homework item display field extends across page breaks, the page break information is moved to the beginning position of the homework item display field. By moving the page break information in this way, as shown in FIG. 17, the end position of the first page becomes the end position of the conclusion display field, and the start position of the second page becomes the homework item display field. Is the start position of. Therefore, when the minutes are printed on a plurality of pages, they can be easily read.

  The above-described operation of the information processing apparatus 10 is realized by the CPU 12 executing the information processing program stored in the RAM 14, as in the first embodiment. However, in the second embodiment, the print page adjusting program is further stored in the program storage area 302 described in the first embodiment.

  The print page adjustment program is a program for moving a page break to the head position of the display column when one display column in the minutes spans two pages.

  Further, in the second embodiment, the minutes creating program 318 is also a program for vertically expanding the display field including the content field when the handwritten image does not fit in the content field.

  Hereinafter, the drawing control process of the second embodiment will be specifically described. However, the drawing control process is the same as the drawing control process of the first embodiment shown in FIGS. Omit it. Further, the same reference numerals are attached to the same processes as the drawing control process of the first embodiment.

  FIG. 18 is a flowchart showing a part of the drawing control processing of the CPU 12 in the second embodiment and sequel to FIG.

  As can be seen from FIG. 18, in the second embodiment, in the drawing control process shown in FIG. 12, the processes of steps S101, S103, S105, and S107 are added between steps S51 and S53.

As shown in FIG. 18, when determining that any item is touched in step S51, the CPU 12 determines “YES”, and determines in step S101 whether the handwritten image fits in the vertical direction of the content column. Here, the CPU 12 determines whether or not the vertical width of the handwritten image is smaller than the vertical width of the content field corresponding to the selected item in the minutes format.
If “NO” in the step S101, that is, if the handwritten image does not fit in the vertical direction of the content field, the item field and the content field (display field) are vertically expanded in a step S103, and the process proceeds to step S105.

  On the other hand, if "YES" in the step S101, that is, if the handwritten image fits in the vertical direction of the content column, it is determined in a step S105 whether the display column spans two pages. Here, the CPU 12 determines whether or not one display column crosses page breaks.

  If "NO" in the step S105, that is, if the display column does not extend over two pages, the process proceeds to a step S53. On the other hand, if “YES” in the step S105, that is, if the display field spans two pages, the information of page break is moved to the head of the display field in a step S107, and the process proceeds to the step S53.

  According to the second embodiment, when the item column and the content column in the minutes format are expanded, the page layout is adjusted by inserting a page break with an item that is well separated. That is, the user can create the minutes without worrying about the layout of the printed page.

  In the second embodiment, when the display column extends over two pages, the position of the page break is moved to the start position of the display column, but the minutes may be reduced to fit within one page. You may

Further, as described above, the content column may be expanded in the horizontal direction, and when the minutes run off the page, the entire minutes may be reduced to fit within the page.
<Third embodiment>
The information processing apparatus 10 of the third embodiment is the same as the information processing apparatus 10 of the first embodiment, except that the handwritten image already selected as the content to be recorded in the minutes is displayed in a distinguishable manner. Therefore, duplicate description will be omitted.

  FIG. 19 is an illustrative view showing one example of the touch screen 100 displayed on the display 22 in the third embodiment.

  As shown in FIG. 19, in the third embodiment, when the handwritten image is stored in the memory in association with the selected item, the color in the frame 106 is changed to a color different from the background and the handwritten image. . That is, the frame 106 is given a predetermined color (the inside of the frame 106 is filled with a predetermined color).

  The above-described operation of the information processing device 10 is realized by the CPU 12 executing the information processing program stored in the RAM 14 shown in FIG. 7. However, in the third embodiment, the minutes creating program 318 means that when a handwritten image within a specified range defined by the range specifying program 316 is stored in the RAM 14, a predetermined color (for example, red) is displayed in the frame 106. Is different from the first embodiment in that.

  In addition, in the touch screen 100 shown in FIG. 19, not only is the frame 106 given a predetermined color, but the selected handwritten image is bordered, but the border may not be bordered.

  FIG. 20 is a flowchart showing a part of the drawing control processing of the CPU 12 in the third embodiment and sequel to FIG. 11. Hereinafter, the drawing control process of the third embodiment will be described, but the description of the same contents as those described in the first embodiment will be omitted.

  As can be seen from FIG. 20, in the third embodiment, in the drawing control process shown in FIG. 12, the process of step S201 is added between steps S55 and S57.

  As shown in FIG. 20, in step S55, the item selection menu 130 is erased, in step S201, a color different from the handwritten image and the background is added to the frame 106, and the process proceeds to step S57.

  According to the third embodiment, for a handwritten image that has already been selected to be entered in the minutes column of the minutes, the inside of the frame 106 enclosed when the handwritten image is selected is colored differently from the background and the handwritten image. Since the mark is added (filled with color), the user can easily know the handwritten image already selected as the content corresponding to the item of the minutes.

  Although the frame 106 is colored when the handwritten image is selected in the third embodiment, the present invention is not limited to this. Alternatively, the frame 106 when a handwritten image is selected may be colored with a color that indicates that the frame 106 has been selected. Alternatively, the frame 106 when a handwritten image is selected may be displayed in a blinking manner. Alternatively, the already selected handwritten image may be displayed in gray out or may be displayed in blinking. That is, the display mode of the handwritten image and the frame 106 is changed so that the handwritten image is already selected as the content corresponding to the minutes item. However, both the frame 106 and the change of the display mode in the frame 106 and the change of the display mode of the selected handwritten image may be executed at the same time.

  Needless to say, the third embodiment is also applicable to the second embodiment.

  Further, in the above-described embodiment, the edit of the item and the change of the size of the display field are reflected only in the format of the minutes corresponding to the format data 336a included in the minutes data 336. At the same time, the original format data stored in the HDD may be reflected.

  Furthermore, the screen configurations and the like mentioned in each of the above-described embodiments are examples, and can be changed as appropriate according to the actual product. Further, when the same effect is obtained, the order of the steps shown in the flowchart may be appropriately changed.

10 ... Information processing device 12 ... CPU
14 ... RAM
16 ... Touch panel control circuit 18 ... Drawing control circuit 20 ... Touch panel 22 ... Display 100 ... Touch screen 102 ... Minutes edit button 104 ... Range designation button 106 ... Frame 110 ... Touch pen

Claims (9)

An information processing device for displaying an image on a display device,
Input detection means for detecting an input instruction from the user,
A storage unit that stores format data corresponding to a predetermined document format and having a display column that displays the contents of various items regarding the proceedings.
Range designating means for designating a range including at least a part of the image displayed on the display device according to the input instruction detected by the input detecting means,
For a range specified by the range specifying means, from a plurality of items included in the format, an item specifying means for specifying one item, and an image included in the range specified by the range specifying means, An information processing apparatus, comprising: creating means for creating a new document conforming to the format using the image by displaying in the display field displaying the content of the one item specified by the item specifying means.   The information processing apparatus according to claim 1, further comprising an editing unit that adds, changes, or deletes an item included in the format according to an input instruction detected by the input detection unit. The item designating means designates the one item by selecting one item from the selectable items as a plurality of items selectable in the format,
The information processing apparatus according to claim 2, wherein the editing unit adds, changes, or deletes the selectable items together when an item included in the format is added, changed, or deleted.   4. The size changing means for changing the size of the display field of the image in the format according to the size of the image included in the range designated by the range designating means. An information processing apparatus according to claim 2. When the size of the display section in the format has been changed by said size changing means further comprises adjustment means for adjusting the layout of the printed page, claim 4 Symbol placing of the information processing apparatus.   6. The information processing apparatus according to claim 5, wherein when one of the display columns extends over two pages, the adjusting unit inserts page break information at the head position of the display column. When the image included in the range specified by the previous above range designating means is stored in said storage means, further comprising a display form changing means for displaying on said display device the image in an identifiable manner, claim 7. The information processing device according to any one of 1 to 6. Information that is executed by an information processing device that stores format data that corresponds to a predetermined document format and that has a display column that displays the contents of various items related to the proceedings and that displays an image on the display device. A processing program,
The processor of the information processing device,
Input detection means for detecting an input instruction from the user,
Range designating means for designating a range including at least a part of the image displayed on the display device according to the input instruction detected by the input detecting means,
For a range specified by the range specifying means, from a plurality of items included in the format, an item specifying means for specifying one item, and an image included in the range specified by the range specifying means, An information processing program for displaying the content of the one item designated by the item designating unit in the display field, thereby causing the image to function as a creating unit for creating a new document conforming to the format. . An input detecting means for detecting an input instruction from a user, and a storage means for formatting a predetermined document and storing format data having a display column for displaying the contents of various items concerning the proceedings are provided, and the image is displayed on the display device. An information processing method of an information processing device for displaying, comprising:
(A) a step of designating a range including at least a part of the image displayed on the display device in response to an input instruction detected by the input detection means,
(B) an item specifying step of specifying one item from a plurality of items included in the format with respect to the range specified in step (a),
(C) By using the image by displaying an image included in the range specified in step (a) in the display column that displays the content of the one item specified in step (b). An information processing method, comprising the step of creating a new document conforming to the format.

Images