JP6818795B2 - Information processing equipment, information processing methods and computer programs - Google Patents

Description

Embodiments of the present invention relate to information processing devices, information processing methods and computer programs.

In the meter reading work of equipment installed in office buildings or factories, inspectors read the meters that show the operating status of those equipment and maintain the equipment. In recent years, equipment that can be connected to a network and can store and manage data on a server is being introduced. However, there are only a few such equipments, and many equipments that do not support network connection remain. Therefore, the visual meter reading work of the inspector for those equipments is still performed.

Human error is inevitable in the meter reading work by the inspector. Therefore, a method has been devised in which an image including the meter reading value of the meter is taken and an image recognition technique is applied to the taken image to automatically read the meter reading value .

Japanese Unexamined Patent Publication No. 8-7078

An object to be solved by the present invention is to provide an information processing device, an information processing method, and a computer program capable of inputting the results of inspection work more efficiently regardless of the type of meter .

The information processing apparatus of the embodiment uses image data related to the meter reading value of the meter and the two-dimensional code related to the meter, which are acquired by the imaging unit. The information processing device has a learning result storage unit, a meter reading value generation unit, and a meter reading value input unit. The learning result storage unit stores the learning result for acquiring the numerical value of the meter from the image data related to the meter reading value of the meter acquired by the imaging unit according to the type of the meter. The meter reading value generation unit generates a meter reading value from the image data of the meter based on the learning result of the learning result storage unit. The meter reading value input unit inputs the meter reading value to the input item related to the two-dimensional code acquired as image data by the imaging unit.

FIG. 6 is a system configuration diagram showing a configuration example of the inspection support system 1 of the first embodiment. The functional block diagram which shows the functional structure of the information processing apparatus 100 of 1st Embodiment. The functional block diagram which shows the functional structure of the upper server 200 of 1st Embodiment. It is a figure which shows a specific example of the form data table of 1st Embodiment. The figure which shows a specific example of the form image of 1st Embodiment. The figure which shows a specific example of the form input image of 1st Embodiment. The figure which shows a specific example of the reading of the two-dimensional bar code of 1st Embodiment. The figure which shows a specific example of reading the meter reading value of 1st Embodiment. The figure which shows a specific example of the form input image which input the meter reading value of 1st Embodiment. The figure which shows a specific example of the form input image which input the meter reading value of 1st Embodiment. The figure which shows a specific example of the form input image which input the meter reading value and the evidence image of 1st Embodiment. The flowchart which shows the process flow of the generation of the meter reading value input of 1st Embodiment. The functional block diagram which shows the functional structure of the information processing apparatus 100a of 2nd Embodiment. The functional block diagram which shows the functional structure of the upper server 200a of the 2nd Embodiment. The functional block diagram which shows the functional structure of the information processing apparatus 100b of 3rd Embodiment. The figure which shows a specific example of the form image of the modification of embodiment. The figure which shows a specific example of the form image corresponding to the multimeter of the modification of the embodiment.

Hereinafter, the information processing apparatus, the information processing method, and the computer program of the embodiment will be described with reference to the drawings.

(First Embodiment)
FIG. 1 is a system configuration diagram showing a configuration example of the inspection support system 1 of the first embodiment. The inspection support system 1 includes an information processing device 100, a host server 200, and a plurality of meters 300. The information processing apparatus 100 and the host server 200 are connected to each other so as to be able to communicate with each other via the network 400.

The information processing device 100 is configured by using a device such as a smartphone, a tablet computer, smart glasses, a smart head, or a personal computer. The information processing device 100 includes a processor, a memory, an auxiliary storage device, and the like connected by a bus. The information processing device 100 is operated by an inspector. The inspector may be, for example, a person who inspects the meter 300 or a person who aggregates the inspection results. The inspector may be any person who operates the information processing apparatus 100. The information processing device 100 inputs the numerical value indicated by the meter 300 as the meter reading value into the form data. The form data is data in which numerical values (meter reading values) indicated by a plurality of meters 300 can be input.

The host server 200 is configured by using a device such as a personal computer, a server, a workstation, or an industrial computer. The host server 200 includes a processor, a memory, an auxiliary storage device, and the like connected by a bus. The host server 200 generates a recognition engine for enabling the information processing apparatus 100 to input the numerical value indicated by the meter 300. The generated recognition engine may be stored in the information processing device 100. The host server 200 may be realized by a cloud computing system.

The meter 300 is, for example, a digital type having a 7-segment display displaying a number from 0 to 9, and has a configuration showing a numerical value (meter reading value) of at least one digit or more according to the operating state of the equipment. The meter 300 is not limited to a digital type having a 7-segment display. For example, the meter 300 is, for example, a mechanical type using a number wheel in which numbers from 0 to 9 are described, and is configured to show a multi-digit numerical value (meter reading value) by rotating according to the amount used. It may be configured by a needle type in which a scale and a character indicating a unit are displayed on a scale board.

The network 400 may be, for example, a wired communication network such as the Internet, or a wireless communication network such as a wireless LAN (Local Area Network) and a mobile phone communication network. The network 400 may be any communication network as long as communication is possible between the information processing device 100 and the host server 200.

FIG. 2 is a functional block diagram showing a functional configuration of the information processing apparatus 100 of the first embodiment. The information processing device 100 executes a form data generation program to control the communication unit 101, the input unit 102, the display unit 103, the imaging unit 104, the form definition storage unit 105, the learning result storage unit 106, the form data storage unit 107, and the control unit. It functions as a device including the unit 108.

The communication unit 101 is a network interface. The communication unit 101 communicates with the host server 200 via the network 400. The communication unit 101 may communicate by a communication method such as wireless LAN, wired LAN, Bluetooth (registered trademark) or LTE (Long Term Evolution) (registered trademark).

The input unit 102 is configured by using an input device such as a touch panel, a mouse, and a keyboard. The input unit 102 may be an interface for connecting the input device to the information processing device 100. In this case, the input unit 102 generates input data (for example, instruction information indicating an instruction to the information processing device 100) from the input signal input in the input device, and inputs the input data to the information processing device 100.

The display unit 103 is an output device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, and an organic EL (Electro Luminescence) display. The display unit 103 may be an interface for connecting the output device to the information processing device 100. In this case, the display unit 103 generates a video signal from the video data and outputs the video signal to the video output device connected to itself.

Although not shown in FIG. 2, the image pickup unit 104 includes, for example, an optical system, an image sensor, and an image processing substrate that processes an image pickup signal output from the image pickup element, and image data in a predetermined format based on the image pickup signal. Is generated and output. The imaging unit 104 captures an identification image in which the identification information of the meter 300 attached to the meter 300 itself or the periphery of the meter 300 is embedded. The imaging unit 104 generates identification image data including the identification image. The identification image is, for example, a two-dimensional bar code. The imaging unit 104 captures a numerical value (meter reading value) indicated by the meter 300. The imaging unit 104 photographs the appearance of the meter 300 as an evidence image that serves as evidence for meter reading. The imaging unit 104 generates meter image data including the appearance of the meter 300. When the imaging unit 104 is activated, the live view is displayed on the display unit 103. The live view is a function that allows an inspector to take a picture while checking the subject taken by the image pickup unit 104. The inspector can take a picture while checking the live view displayed on the display unit 103. For example, when the inspector photographs the appearance of the meter 300 as an evidence image, the inspector sets the meter 300 so that the meter reading value of the meter 300 can be discriminated while checking the live view displayed on the display unit 103. Shooting is performed by adjusting the distance from the information processing device 100.

The form definition storage unit 105 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form definition storage unit 105 stores the form definition information. The form definition information is data that defines input items to the form and the form format. As the form definition information, different form definition information may be defined depending on the place to be inspected or the meter 300. Therefore, the data of the form definition information may be different.

The learning result storage unit 106 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 106 stores the learning result. The learning result is used to acquire a numerical value that can be input to the form data from the image data indicating the numerical value of the photographed meter 300. The learning result is recorded when the form data generation program is introduced into the information processing apparatus 100. The learning result may be updated when the form data generation program is updated. The learning result may be recorded according to the type of the meter 300. The type of the meter 300 may be distinguished by, for example, a digital type, a mechanical type, a needle type, etc., and a manufacturer, model, year, etc. May be distinguished by.

The form data storage unit 107 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form data storage unit 107 stores the form data table. The form data table holds form data. Each value held in the form data is held based on the input items defined in the form definition information. A form image is generated based on the data held by the form data and the form definition information associated with the form data.

FIG. 4 is a diagram showing a specific example of the form data table of the first embodiment. The form data table has form data. The form data has each value such as an inspection target, a date and time, a form definition information, a first meter, a first meter image, a second meter, a second meter image, a third meter, and a third meter image. The inspection target represents the target to be inspected by the inspector. The date and time represent the date and time when the form data was generated. The form definition information represents the form definition information associated with the form data. The first meter represents the numerical value of the first meter when the associated form definition information has the first meter as an input item. When the first meter is photographed, the first meter image represents the storage location of the image data captured by the first meter or the name of the image data. The second meter represents the numerical value of the second meter when the associated form definition information has the second meter as an input item. When the second meter is photographed, the second meter image represents the storage location of the image data captured by the second meter or the name of the image data. The third meter represents the numerical value of the third meter when the associated form definition information has the third meter as an input item. When the third meter is photographed, the third meter image represents the storage location of the image data captured by the third meter or the name of the image data.

In the example shown in FIG. 4, in the form data at the top of the form data table, the value to be inspected is "8F air conditioning", the date and time value is "2017.12.4 14:23:10", and the value of the form definition information is. "Def_1", the value of the 1st meter is "120", the value of the 1st meter image is "8f_air_meter1.jpg", the value of the 2nd meter is "110", the value of the 2nd meter image is "8f_air_meter2.jpg", The value of the third meter is "-", and the value of the third meter image is "-". Therefore, according to the form data at the top of the form data table, the inspected place is "8F air conditioning", the form data is generated at "2017.12.4 14:23:10", and "def_1" is used as the form definition information. It can be seen that is used. Then, it can be seen that the form definition information has the first meter, the first meter image, the second meter and the second meter image as the input items, and does not have the third meter and the third meter image. The form data table shown in FIG. 4 is only a specific example. Therefore, the form data table may be configured in a manner different from that shown in FIG. For example, the form data table may be configured to hold the name of the inspector, or may have an entire image in which multiple meters hold an image taken in one, or taken. It may have information indicating the shooting time of the image.

Returning to FIG. 2, the description of the information processing apparatus 100 will be continued. The control unit 108 controls the operation of each unit of the information processing device 100. The control unit 108 is executed by a device including, for example, a processor and a memory. By executing the form generation program, the control unit 108 executes the form data generation unit 109, the form image generation unit 110, the image acquisition unit 111, the input item determination unit 112, the meter reading value generation unit 113, the form data input unit 114, and the abnormality. It functions as a determination unit 115 and an error processing unit 116.

The form data generation unit 109 acquires the form definition information from the form definition storage unit 105 based on the selection instruction of the form definition information. The form data generation unit 109 generates form data based on the acquired form definition information. The selection instruction may be received via the input unit 102, or may be embedded in the two-dimensional bar code.

The form image generation unit 110 generates a form image based on the form format defined by the acquired form definition information and the generated form data. The form image is displayed on the display unit 103. The form image includes each value held by the form data and an instruction input image associated with the input unit 102. The input unit 102 receives a predetermined instruction when the instruction input image displayed on the display unit 103 is touch-operated by the inspector. The predetermined instruction may be the activation of the imaging unit 104, the generation of the form input image, or the recording of the form data. The predetermined instruction may be any instruction as long as it is a predetermined instruction.

The image acquisition unit 111 activates the image pickup unit 104. The image acquisition unit 111 acquires the live view displayed on the display unit 103 from the image pickup unit 104. The image acquisition unit 111 outputs the live view to the input item determination unit 112 or the meter reading value generation unit 113.

The input item determination unit 112 identifies the two-dimensional bar code from the live view. The input item determination unit 112 determines the input item into which the meter reading value is input based on the identification information embedded in the specified two-dimensional bar code image. The identification information includes information necessary for the input item determination unit 112 to identify the input item included in the form data. The identification information includes, for example, information such as "first meter". The identification information may include, for example, information that identifies the form definition information, or may include a threshold value for meter reading. The identification information is not limited to the two-dimensional bar code. For example, the identification information may be embedded in a barcode, embedded in an IC tag such as an RFID (Radio Frequency Identifier), or recorded in a storage device included in the meter.

The meter reading value generation unit 113 identifies an image showing the meter reading value from the live view. The meter reading value generation unit 113 generates a meter reading value based on an image showing the specified meter reading value. The meter reading value generation unit 113 generates a meter reading value based on, for example, an image showing the meter reading value and a learning result recorded in the learning result storage unit 106. The meter reading value generation unit 113 may generate a meter reading value by using an existing image recognition technique such as OCR (Optical Character Recognition). The meter reading value generation unit 113 acquires the time when the meter reading value is generated as the meter reading time. The meter reading value generation unit 113 may be configured to read out the generated meter reading value from the speaker of the information processing device 100 by voice.

The form data input unit 114 inputs the generated meter reading value to the input item determined among the input items of the form data. When the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 is photographed, the form data input unit 114 inputs the file name of the photographed image data and the photographed shooting time into the form data. The form data input unit 114 records the input form data in the form data storage unit 107. The form data input unit 114 may transmit the form data to the upper server 200. The form data input unit 114 may record the PDF file generated based on the form image in the form data storage unit 107, or may be configured to transmit the PDF file to the upper server 200.

The abnormality determination unit 115 determines whether or not the input input value satisfies a predetermined condition. When the predetermined condition is satisfied, the abnormality determination unit 115 instructs the error processing unit 116 to perform error processing. If the predetermined condition is not satisfied, the abnormality determination unit 115 does not instruct the error processing unit 116 to perform error processing. The predetermined condition is, for example, the difference between the meter reading time, which is the time when the image showing the meter reading value is acquired, and the shooting time when the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 is taken. If it exceeds the threshold value of, it may be determined that a predetermined condition is satisfied. The predetermined condition may be any condition as long as it is a predetermined condition. The predetermined threshold value may be a threshold value embedded in the two-dimensional bar code or a threshold value included in the form definition information. The predetermined threshold value may be any value as long as it is a preset threshold value.

When the error processing unit 116 receives the error processing instruction from the abnormality determination unit 115, the error processing unit 116 executes the error processing. The error processing may be, for example, a processing that prompts the inspector to re-enter the meter reading value in the input item. The error processing may be any processing as long as it is a processing specified in advance.

FIG. 3 is a functional block diagram showing a functional configuration of the upper server 200 of the first embodiment. By executing the learning program, the upper server 200 executes the communication unit 201, the input unit 202, the display unit 203, the user information storage unit 204, the teacher data storage unit 205, the learning result storage unit 206, the form data storage unit 207, and the control unit. It functions as a device equipped with 208.

The communication unit 201 is a network interface. The communication unit 201 communicates with the information processing device 100 via the network 400. The communication unit 201 may communicate by a communication method such as wireless LAN, wired LAN, Bluetooth or LTE.

The input unit 202 is configured by using an input device such as a touch panel, a mouse, and a keyboard. The input unit 202 may be an interface for connecting the input device to the upper server 200. In this case, the input unit 202 generates input data (for example, instruction information indicating an instruction to the upper server 200) from the input signal input in the input device, and inputs the input data to the upper server 200.

The display unit 203 is an output device for a CRT display, a liquid crystal display, an organic EL display, or the like. The display unit 203 may be an interface for connecting the output device to the host server 200. In this case, the display unit 203 generates a video signal from the video data and outputs the video signal to the video output device connected to itself.

The user information storage unit 204 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The user information storage unit 204 stores user information. The user information is information about a person who uses the meter 300. The user information includes, for example, a user identifier that identifies the user.

The teacher data storage unit 205 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The teacher data storage unit 205 stores teacher data. The teacher data is image data taken by the meter 300 used by the user. The teacher data may include image data taken from the front, image data taken from an angle, and the like. The teacher data may be stored in association with the user identifier.

The learning result storage unit 206 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 206 stores the learning result. The learning result is generated by the learning unit 209 based on the image data stored in the teacher data storage unit 205. The learning result may be associated with the user identifier and generated for each user.

The form data storage unit 207 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form data storage unit 207 stores the form data table. The form data table holds form data. Form data transmitted from the information processing device 100 is recorded in the form data storage unit 207.

The control unit 208 controls the operation of each unit of the host server 200. The control unit 208 is executed by a device including, for example, a processor and a memory. The control unit 208 functions as the learning unit 209 by executing the learning program.

The learning unit 209 generates a learning result based on the teacher data. The learning unit 209 generates a learning result from the teacher data by machine learning such as deep learning. The learning result may be generated for each type of the meter 300, or a single learning result may be generated regardless of the type of the meter 300.

FIG. 5 is a diagram showing a specific example of the form image of the first embodiment. The form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. The form image displayed on the display unit 103 includes an input item image 131, an input item image 132, an input item image 133, and an instruction input image 134.

The input item image 131, the input item image 132, and the input item image 133 are images that display the meter reading values held in the form data. The input item of the form data is associated with the input item image 131, the input item image 132, and the input item image 133. According to FIG. 5, the input item image 131 is associated with the input item “first meter”, the input item image 132 is associated with the input item “second meter”, and the input item image 133 is associated with the input item “third meter”. .. When the meter reading value is input to the input item of the form data, the meter reading value input to the input item image associated with the input item is displayed. For example, when "100" is input to the input item "first meter" of the form data, "100" is displayed in the input item image 131.

The instruction input image 134 is an image that receives an instruction to generate a form input image. By touching the instruction input image 134, an instruction to generate a form input image is received by the input unit 102. The touch operation is an act in which the user touches or pushes the instruction input image 134 displayed on the display unit 103. When the input unit 102 receives the form input image generation instruction, the form image generation unit 110 generates the form input image. The display unit 103 displays the generated form input image. The display unit 103 may display the form input image so as to be superimposed on the form image, or the display unit 103 may discard the form image and display the form input image.

FIG. 6 is a diagram showing a specific example of the form input image of the first embodiment. The form image generation unit 110 generates a form input image based on the form definition information and the form data. The display unit 103 displays the generated form input image. The form input image displayed on the display unit 103 includes an input area image 141, an input area image 142, an input area image 143, an instruction input image 144, and an instruction input image 145.

The input area image 141, the input area image 142, and the input area image 143 are areas for holding the values input to the form data. Input items of form data are associated with the input area image 141, the input area image 142, and the input area image 143. According to FIG. 6, the input area image 141 is associated with the input item “first meter”, the input area image 142 is associated with the input item “second meter”, and the input area image 143 is associated with the input item “third meter”. ..

The instruction input image 144 is an image that receives an activation instruction of the imaging unit 104. When the instruction input image 144 is touch-operated, the start instruction of the imaging unit 104 is received by the input unit 102. When the input unit 102 receives the generation instruction of the image pickup unit 104, the image acquisition unit 111 activates the image pickup unit 104. The display unit 103 displays the live view when the imaging unit 104 is activated.

The instruction input image 145 is an image that receives an input instruction to the form data. By touching the instruction input image 145, an input instruction to the form data is received by the input unit 102. When the input unit 102 receives an input instruction to the form data, the form data input unit 114 inputs a value to the form data. For example, when "100" is input as the meter reading value in the input area image 141, "100" is input in the input item "first meter" of the form data.

FIG. 7 is a diagram showing a specific example of reading a two-dimensional bar code according to the first embodiment. FIG. 7 includes a meter 300 and a display unit 103 of the information processing device 100. The meter 300 includes a region 301 and a region 302. Region 301 includes a two-dimensional bar code attached to the meter 300. The area 302 includes the meter reading value represented by the meter 300. The display unit 103 includes the area 137. Region 137 includes a two-dimensional bar code image of region 301 of the meter 300 imaged by the imaging unit 104.

The input item determination unit 112 identifies the two-dimensional bar code displayed in the live view. The input item determination unit 112 determines the input item into which the meter reading value is input based on the identification information embedded in the specified two-dimensional bar code image. The blowout 137a represents the identification information embedded in the two-dimensional bar code included in the area 137. According to the blowout 137a, it can be seen that the "first meter" is embedded as the identification information. The input item determination unit 112 determines the input item as the "first meter".

FIG. 8 is a diagram showing a specific example of reading the meter reading value of the first embodiment. FIG. 8 includes a meter 300 and a display unit 103 of the information processing device 100. The meter 300 includes a region 301 and a region 302. Region 301 and region 302 are the same as in FIG. The display unit 103 includes the area 138. Region 138 includes an image showing the meter reading value of region 302 of the meter 300 imaged by the imaging unit 104.

When the input item determination unit 112 determines the input item, the reading of the meter reading value is started. The meter reading value generation unit 113 specifies an image showing the meter reading value displayed in the live view. The meter reading value generation unit 113 generates a meter reading value based on an image showing the specified meter reading value. When the meter reading value generation unit 113 generates the meter reading value, the generated time is acquired.

FIG. 9 is a diagram showing a specific example of a form input image in which the meter reading value of the first embodiment is input. When the meter reading value is generated, the form image generation unit 110 displays the meter reading value on the input area image. The meter reading value is displayed in the input area image associated with the input item. The form input image displayed on the display unit 103 includes an input area image 141a, an input area image 142a, an input area image 143a, an instruction input image 144, and an instruction input image 145. The instruction input image 144 and the instruction input image 145 are the same as those in FIG. 7.

The input area image 141a, the input area image 142a, and the input area image 143a are areas for holding the values input to the form data. According to FIG. 9, the input area image 141a holds “12332”, the input area image 142a holds “678”, and the input area image 143a holds “90000”. According to FIG. 9, the input area image 141 is associated with the input item “first meter”, the input area image 142 is associated with the input item “second meter”, and the input area image 143 is associated with the input item “third meter”. .. Therefore, when the instruction input image 145 is touch-operated, the form data input unit 114 inputs the values held in the input area image 141a, the input area image 142a, and the input area image 143a into the form data.

FIG. 10 is a diagram showing a specific example of a form input image in which the meter reading value of the first embodiment is input. When the instruction input image 145 is touch-operated, the form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. In FIG. 10, since the meter reading value is input to the form data, the meter reading value is displayed in the input item image. The form image displayed on the display unit 103 includes an input item image 131a, an input item image 132a, an input item image 133a, and an instruction input image 134. The instruction input image 134 is the same as in FIG.

The input item image 131a, the input item image 132a, and the input item image 133a are images that display the meter reading values input to the form data. The input item of the form data is associated with the input item image 131a, the input item image 132a, and the input item image 133a. According to FIG. 10, the input item image 131a is associated with a "first meter" as an input item, the "first meter" is input with "12332" as a meter reading value, and the input item image 132a is an input item. The "second meter" is associated, the "second meter" is associated with "678" as the meter reading value, and the input item image 133a is associated with the "third meter" as the input item, and the "third meter" is associated. It can be seen that "90000" is input as the meter reading value.

FIG. 11 is a diagram showing a specific example of a form input image in which the meter reading value and the evidence image of the first embodiment are input. As shown in FIG. 11, the form input image may be configured so that the evidence image is also displayed. Evidence images are taken after meter readings have been generated. Specifically, when the meter image is included as an input item of the form definition information, the form data input unit 114 displays a screen for confirming whether or not to capture the appearance of the meter 300 after the meter reading value is generated. It is displayed on 103. When the image pickup unit 104 receives an instruction to capture the appearance of the meter 300 via the input unit 102, the image pickup unit 104 captures an image of the meter 300. The form data input unit 114 inputs the photographed image data and the photographed shooting time into the form data.

According to FIG. 11, the form images displayed on the display unit 103 are the input item image 131a, the input item image 132a, the input item image 133a, the input item image 131b, the input item image 132b, the input item image 133b, and the instruction input image 134. including. The input item image 131a, the input item image 132a, the input item image 133a, and the instruction input image 134 are the same as those in FIG.

The input item image 131b, the input item image 132b, and the input item image 133b are images representing evidence images held in the form data. The input item of the form data is associated with the input item image 131b, the input item image 132b, and the input item image 133b. According to FIG. 11, the input item image 131b is associated with the "first meter image" as the input item, the input item image 132b is associated with the "second meter image" as the input item, and the input item image 133b is associated with the input item image 133b. , It can be seen that the "third meter image" is associated with the input item.

FIG. 12 is a flowchart showing a flow of processing for generating a meter reading value input according to the first embodiment. The form data generation unit 109 generates form data based on the acquired form definition information (step S101). The form image generation unit 110 generates a form image based on the form format and the form data defined by the acquired form definition information (step S102). The input unit 102 receives an instruction to generate a form input image by touching the instruction input image 134 included in the form image (step S103). The form image generation unit 110 generates a form input image (step S104). The input unit 102 receives an activation instruction of the imaging unit 104 by touch-operating the instruction input image 144 included in the form input image (step S105). The image acquisition unit 111 activates the image pickup unit 104 (step S106). The live view is displayed on the display unit 103.

The input item determination unit 112 identifies the two-dimensional bar code displayed in the live view (step S107). The input item determination unit 112 determines the input item into which the meter reading value is input based on the identification information embedded in the image of the specified two-dimensional bar code (step S108). The meter reading value generation unit 113 specifies an image showing the meter reading value displayed in the live view (step S109). The meter reading value generation unit 113 generates a meter reading value based on an image showing the specified meter reading value (step S110). When the meter reading value generation unit 113 generates the meter reading value, the meter reading value generation unit 113 acquires the meter reading time. The form image generation unit 110 displays the generated meter reading value on the input area image (step S111).

The form data input unit 114 receives an instruction as to whether or not to photograph the appearance of the meter 300 (step S112). The form data input unit 114 determines whether or not an instruction to image the meter 300 has been received (step S113). When an instruction to image the meter 300 is received (step S113: YES), the imaging unit 104 photographs the appearance of the meter 300 and acquires image data according to the user's operation (step S114). The imaging unit 104 acquires the shooting time. If the instruction to take a picture of the meter 300 is not accepted (step S113: NO), the process proceeds to step S115.

The form image generation unit 110 determines whether or not values have been input to all the input area images (step S115). If there is an input area image for which no value has been input (step S115: NO), the process proceeds to step S107. When the values are input to all the input area images (step S115: YES), the form data input unit 114 determines whether or not the input instruction to the form data has been accepted (step S116). Specifically, it is determined whether or not the instruction input image 145 is touch-operated in the form input image. If the input instruction to the form data is not accepted (step S116: NO), the process proceeds to step S116. When the input instruction to the form data is received (step S116: YES), the form data input unit 114 inputs a value to the form data. The value may be, for example, a meter reading value or a meter image (step S117). The form image generation unit 110 generates a form image based on the form definition information and the form data (step S118). Since the meter reading value is input to the form data, the meter reading value is displayed in the input item image.

The abnormality determination unit 115 determines whether or not the difference between the meter reading time and the photographing time is smaller than a predetermined threshold value (step S119). A predetermined threshold value may be included in the two-dimensional bar code or may be given in advance. When it is smaller than a predetermined threshold value (step S119: YES), the form data input unit 114 records the form data in the form data storage unit 107 (step S120). If it is not smaller than a predetermined threshold value (step S119: NO), the error processing unit 116 performs error processing (step S121). In this flowchart, the error processing is processing without recording the form data in the form data storage unit 107.

In the information processing apparatus 100 configured in this way, the input item determination unit 112 reads the two-dimensional bar code to acquire the identification information embedded in the two-dimensional bar code. The identification information is associated with an input item into which a meter reading value is input. The meter reading value generation unit 113 generates the meter reading value of the meter 300. By inputting the meter reading value in the input item associated with the identification information, the form data input unit 114 does not input the meter reading value in the wrong input item, so that the result of the inspection work is input more efficiently. can do

(Second Embodiment)
Next, the inspection support system 1 of the second embodiment will be described. The inspection support system 1 in the second embodiment is different from the first embodiment in that the information processing device 100a is provided in place of the information processing device 100 and the upper server 200a is provided in place of the upper server 200. Other than that, the configuration is the same. Hereinafter, the points different from the first embodiment will be described.

FIG. 13 is a functional block diagram showing a functional configuration of the information processing apparatus 100a of the second embodiment. The information processing device 100a is different from the information processing device 100 in that the control unit 108a is provided instead of the control unit 108 and the learning result storage unit 106 is not provided, but the other configurations are the same.

The control unit 108a controls the operation of each unit of the information processing device 100a. The control unit 108a is executed by a device including, for example, a processor and a memory. By executing the form generation program, the control unit 108a executes the form data generation unit 109, the form image generation unit 110a, the image acquisition unit 111a, the input item determination unit 112, the form data input unit 114, the abnormality determination unit 115, and the error processing. It functions as a unit 116.

The form image generation unit 110a generates a form image based on the form format and the form data defined in the form definition information. The form image is displayed on the display unit 103. The form image includes each value held by the form data and an instruction input image associated with the input unit 102. The input unit 102 receives a predetermined instruction when the instruction input image displayed on the display unit 103 is touch-operated by the inspector. The predetermined instruction may be the activation of the imaging unit 104, the generation of the form input image, or the recording of the form data. The predetermined instruction may be any instruction as long as it is a predetermined instruction. When the form image generation unit 110a receives the meter reading value from the upper server 200, the form image generation unit 110a displays the meter reading value on the input area image.

The image acquisition unit 111a activates the image pickup unit 104. The image acquisition unit 111a acquires the live view displayed on the display unit 103 from the image pickup unit 104. The image acquisition unit 111a outputs the live view to the input item determination unit 112. The image acquisition unit 111a transmits the live view to the upper server 200a.

FIG. 14 is a functional block diagram showing a functional configuration of the upper server 200a of the second embodiment. The upper server 200a is different from the upper server 200 in that the control unit 208a is provided instead of the control unit 208, but the other configurations are the same.

The control unit 208a controls the operation of each unit of the host server 200a. The control unit 208a is executed by a device including, for example, a processor and a memory. The control unit 208a functions as the learning unit 209 and the meter reading value generation unit 210 by executing the learning program.

The meter reading value generation unit 210 identifies an image showing the meter reading value from the live view. The meter reading value generation unit 210 generates a meter reading value based on an image showing the specified meter reading value. The meter reading value generation unit 210 generates a meter reading value based on, for example, an image showing the meter reading value and a learning result recorded in the learning result storage unit 206. The meter reading value generation unit 210 acquires the time when the meter reading value is generated as the meter reading time. The meter reading value generation unit 210 transmits the meter reading value and the meter reading time to the information processing device 100a.

In the information processing apparatus 100a configured in this way, the input item determination unit 112 reads the two-dimensional bar code to acquire the identification information embedded in the two-dimensional bar code. The identification information is associated with an input item into which a meter reading value is input. The image acquisition unit 111a transmits the live view to the upper server 200a. The meter reading value generation unit 210 of the upper server 200a identifies an image showing the meter reading value from the live view and generates the meter reading value. The meter reading value generation unit 210 transmits the meter reading value to the information processing device 100a. By inputting the meter reading value in the input item associated with the identification information, the form data input unit 114 does not input the meter reading value in the wrong input item, so that the result of the inspection work is input more efficiently. can do. Further, by generating the meter reading value on the upper server 200a, it is not necessary to generate the meter reading value on the information processing apparatus 100a. Therefore, even when the information processing device 100a does not have the function of generating the meter reading value from the image showing the meter reading value, the meter reading value can be input to the form data. Further, the learning result used for generating the meter reading value is generated by the upper server 200a. Therefore, the information processing apparatus 100a can always acquire the meter reading value generated by using the latest learning result, and the recognition accuracy can be improved.

(Third Embodiment)
Next, the inspection support system 1 of the third embodiment will be described. The inspection support system 1 in the third embodiment is different from the first embodiment in that the information processing device 100b is provided instead of the information processing device 100 and the host server 200 is not provided, but the other configurations are the same. Is. Hereinafter, the points different from the first embodiment will be described.

FIG. 15 is a functional block diagram showing a functional configuration of the information processing apparatus 100b according to the third embodiment. The information processing device 100b includes a form input application. The form input application is a program for the information processing device 100b to input the numerical value indicated by the meter 300 into the form data as the meter reading value. The information processing device 100b is different from the information processing device 100 in that the control unit 108b is provided instead of the control unit 108, but the other configurations are the same.

The control unit 108b controls the operation of each unit of the information processing device 100b. The control unit 108b is executed by a device including, for example, a processor and a memory. By executing the form input application, the control unit 108b executes the form data generation unit 109, the form image generation unit 110a, the image acquisition unit 111a, the input item determination unit 112, the form data input unit 114b, the abnormality determination unit 115, and the error processing. It functions as a unit 116 and a start unit 117.

The activation unit 117 activates the form input application. When the form input application is started, the activation unit 117 receives a selection instruction of the form definition information. The activation unit 117 receives a selection instruction for form definition information via the input unit 102. The activation unit 117 outputs the selection instruction of the received form definition information to the form data generation unit 109.

The form data input unit 114b inputs the generated meter reading value to the input item determined among the input items of the form data. When the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 is photographed, the form data input unit 114b inputs the file name of the photographed image data and the photographed shooting time into the form data. The form data input unit 114 records the input form data in the form data storage unit 107. The form data input unit 114b may record the PDF file generated based on the form image in the form data storage unit 107.

In each of the above embodiments, the form data generation unit 109, the form image generation unit 110, the image acquisition unit 111, the input item determination unit 112, the meter reading value generation unit 113, the form data input unit 114, the abnormality determination unit 115, and the error processing unit 116 Is a software function unit, but may be a hardware function unit such as an LSI.

According to at least one embodiment described above, by having the form data generation unit 109, the input item determination unit 112, the meter reading value generation unit 113, and the form data input unit 114, the result of the inspection work can be input more efficiently. can do.

(Modified example of embodiment)
FIG. 16 is a diagram showing a specific example of a form image of a modified example of the embodiment. The form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. The form image displayed on the display unit 103 includes an input item image 131, an input item image 132, an input item image 133, an instruction input image 134, a capture instruction image 131c, a capture instruction image 132c, and a capture instruction image 133c. .. Here, since the input item image 131, the input item image 132, the input item image 133, and the instruction input image 134 are the same as those in the first embodiment, the description thereof will be omitted.

The capture instruction image 131c, the capture instruction image 132c, and the capture instruction image 133c are images that receive the meter reading value acquisition instruction. When any one of the capture instruction image 131c, the capture instruction image 132c, and the capture instruction image 133c is touch-operated, the meter reading value acquisition instruction is received by the input unit 102. When the input unit 102 receives the meter reading value acquisition instruction, the image acquisition unit 111 activates the image pickup unit 104. The live view is displayed on the display unit 103.

The meter reading value generation unit 113 identifies an image showing the meter reading value displayed in the live view. The meter reading value generation unit 113 generates a meter reading value based on an image showing the specified meter reading value. When the meter reading value generation unit 113 generates the meter reading value, the meter reading value generation unit 113 acquires the generated time. The form image generation unit 110 displays the generated meter reading value on the input item image associated with the touch-operated import instruction image. In the case of FIG. 15, the input item image 131 is associated with the capture instruction image 131c, the input item image 132 is associated with the capture instruction image 132c, and the input item image 133 is associated with the capture instruction image 133c.

In the information processing apparatus 100 configured in this way, the meter reading value is input based on the image showing the imaged meter reading value. Therefore, it is possible to reduce the input error of the meter reading value for the form data.

The form image generation unit 110 may be configured not to generate a form image and a form input image. In this case, when the form definition information is selected, the instruction input image 144 is displayed on the display unit 103. When the instruction input image 144 is touch-operated, the imaging unit 104 is activated. The live view is displayed on the display unit 103. The user takes an image with the image pickup unit 104 in the order of the two-dimensional bar code of the meter 300 and the meter reading value of the meter 300. When the two-dimensional bar code is imaged, the input item determination unit 112 acquires the identification information embedded in the two-dimensional bar code and determines the input item. When the meter reading value of the meter 300 is imaged, the meter reading value generation unit 113 generates the meter reading value. When the meter reading value is generated, the form data input unit 114 inputs the meter reading value to the determined input item. In the information processing apparatus 100 configured in this way, the user can continuously input the meter reading value without checking the form image. Therefore, the meter reading value can be input to the form data more quickly.

The abnormality determination unit 115 may be configured to issue an alarm when there is an input item in which no value has been input in the form data. With such a configuration, it is possible to prevent the input omission of the meter reading value.

The error processing unit 116 may perform predetermined error processing when the input item determination unit 112 cannot determine the input item of the form data based on the identification information. For the predetermined error processing, for example, a warning message such as "This is a meter that cannot be entered in the form" may be displayed, or identification information embedded in a two-dimensional bar code such as "This is the second meter" is displayed. You may.

The form data input unit 114 may be configured to record in the form data storage unit 107 when all the values are input in the form data input items. In this case, the form data can be recorded in the form data storage unit 107 faster than the inspector inputs the instruction for recording the form data. Therefore, the inspector can carry out the inspection work more efficiently.

FIG. 17 is a diagram showing a specific example of a form image corresponding to a multimeter of a modified example of the embodiment. FIG. 17 includes a multimeter 300a and a display unit 103 of the information processing device 100. The multimeter 300a is a meter that displays meter reading values of different items when a button or switch attached to the meter is pressed. The multimeter 300a may be configured to display meter reading values of different items when an instruction is received from the outside by a communication means such as infrared communication.

The multimeter 300a includes a region 301, a region 302a and a button 303. Region 301 includes a two-dimensional bar code attached to the meter 300. Region 302a includes the meter reading value represented by the multimeter 300a. The button 303 is a switching button for changing the meter reading value represented by the multimeter 300a to the meter reading value of another item. When the button 303 is pressed, the meter reading value included in the area 302a is changed to the meter reading value representing another item. For example, an example will be described in which the multimeter 300a can display the meter reading values of three items of current, voltage and electric power. When the meter reading value of the current is displayed in the area 302a and the button 303 is pressed, the meter reading value of the voltage is displayed. Next, when the button 303 is pressed, the meter reading value of the electric power is displayed. Next, when the button 303 is pressed, the meter reading value of the current is displayed. In this way, when the button 303 is pressed, the meter reading value is displayed in the order of current → voltage → power. The items that can be displayed by the multimeter 300a are not limited to the current, voltage, and meter reading values of electric power. For example, the amount of water or gas used may be displayed.

The display unit 103 includes a region 137 and a region 142b. Region 137 includes a two-dimensional bar code image of region 301 of the meter 300 imaged by the imaging unit 104. The input item determination unit 112 identifies the two-dimensional bar code displayed in the live view. The input item determination unit 112 determines the input item into which the meter reading value is input based on the identification information embedded in the specified two-dimensional bar code image. The blowout 137b represents the identification information embedded in the two-dimensional bar code included in the region 137. According to the outlet 137b, it can be seen that "second meter_ + 3" is embedded as the identification information. The input item determination unit 112 determines the input item to be the “second meter”. In addition, "+3" is embedded as identification information. Therefore, the input item determination unit 112 determines that three meter reading values are input with respect to the input item "second meter".

Region 142b includes a plurality of input region images with respect to the second meter. "Second data" is associated with the three input area images included in the area 142b as input items. Therefore, the inspector presses the button 303 to switch the meter reading value displayed in the area 302a to image the area 302a. The meter reading value generation unit 113 generates a meter reading value from an image showing the imaged meter reading value. The form data input unit 114 sequentially inputs the meter reading value to the input area image associated with the second data.

In the information processing apparatus 100 configured in this way, even if the inspection target is the multimeter 300a, the result of the inspection work can be efficiently input.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention as well as the invention described in the claims and the equivalent scope thereof.

1 ... Inspection support system, 100 ... Information processing device, 101 ... Communication unit, 102 ... Input unit, 103 ... Display unit, 104 ... Imaging unit, 105 ... Form definition storage unit, 106 ... Learning result storage unit, 107 ... Form data Storage unit, 108 ... Control unit, 109 ... Form data generation unit, 110 ... Form image generation unit, 111 ... Image acquisition unit, 112 ... Input item determination unit, 113 ... Meter reading value generation unit, 114 ... Form data input unit, 115 ... Abnormality determination unit, 116 ... Error processing unit, 200 ... Upper server, 201 ... Communication unit, 202 ... Input unit, 203 ... Display unit, 204 ... User information storage unit, 205 ... Teacher data storage unit, 206 ... Learning result storage Unit, 207 ... Form data storage unit, 208 ... Control unit, 209 ... Learning unit, 300 ... Meter, 400 ... Network, 100a ... Information processing device, 108a ... Control unit, 110a ... Form image generation unit, 111a ... Image acquisition unit , 200a ... Upper server, 208a ... Control unit, 210 ... Meter reading value generation unit, 100b ... Information processing device, 108b ... Control unit, 114b ... Form data input unit, 117 ... Activation unit

Claims (7)

An information processing device that inputs the meter reading value of a meter that indicates the operating status of equipment into a form .
Based from the image data according to the meter reading value area of the plurality of types of the meter in the learning result that is used to obtain a meter reading value of the meter, meter reading from the image data according to the meter reading value region of the meter that is acquired by the imaging unit A meter reading value generator that generates a value,
In the input field of the form that is determined by the two-dimensional code according to the meter, which is acquired as image data by the imaging unit, and input unit to enter the meter reading value generated by the meter reading value generating unit,
Have a,
The input unit inputs the image data captured by the meter or the file name thereof and the time when the image data is acquired into the form data together with the meter reading value.
Information processing device. The learning result is generated according to the type of the meter .
The information processing device according to claim 1 . An information processing device that inputs the meter reading value of a meter that indicates the operating status of equipment into a form.
Based on the learning result used to acquire the meter reading value of the meter from the image data related to the meter reading value region of the meter, the meter reading value is generated from the image data related to the meter reading value region of the meter acquired by the imaging unit. Meter reading value generator and
An input unit for inputting the meter reading value generated by the meter reading value generating unit into the input item of the form determined by the two-dimensional code related to the meter acquired as image data by the imaging unit.
Have,
The input unit inputs the image data captured by the meter or the file name thereof and the time when the image data is acquired into the form data together with the meter reading value.
Information processing device. It is an information processing method that inputs the meter reading value of the meter that shows the operating status of equipment into the form .
It relates to the meter reading area of the meter acquired by the imaging unit based on the learning result used by the information processing apparatus to acquire the meter reading value of the meter from the image data related to the meter reading area of a plurality of types of the meter. A meter reading value generation step that generates a meter reading value from image data,
The information processing apparatus, the input field of the form to a more determined in the two-dimensional code according to the meter, which is acquired as image data by the imaging unit, to enter the meter reading value generated in the meter reading value generating step and input step,
Have a,
In the input step, the image data captured by the meter or the file name thereof and the time when the image data is acquired are input to the form data together with the meter reading value .
Information processing method. It is an information processing method that inputs the meter reading value of the meter that shows the operating status of equipment into the form.
From the image data related to the meter reading area of the meter acquired by the imaging unit based on the learning result used by the information processing device to acquire the meter reading value of the meter from the image data related to the meter reading area of the meter. Meter reading value generation step to generate meter reading value and
An input step in which the information processing device inputs the meter reading value generated in the meter reading value generation step into the input item of the form determined by the two-dimensional code related to the meter acquired as image data by the imaging unit. When,
Have,
In the input step, the image data captured by the meter or the file name thereof and the time when the image data is acquired are input to the form data together with the meter reading value.
Information processing method. A computer program for causing a computer to function as the information processing device according to claim 1 . A computer program for causing a computer to function as the information processing device according to claim 3 .

Images