JP2022135893A - Optical information reader - Google Patents

Abstract

To preventing the load of inferring processing from causing a prolong of a processing time while increasing the accuracy of reading by inferring processing of a machine learning.SOLUTION: The optical information reading device has a processor including: an inferring processing unit for performing inferring processing of inputting a read image into a neural network and generating an ideal image for the read image; and a decoding processing unit of executing first decoding processing of decoding the read image and second decoding processing of decoding the ideal image generated by the inferring processing unit. The processor executes the inferring processing and the first decoding processing in parallel, and also executes the second decoding processing after the inferring processing is completed.SELECTED DRAWING: Figure 16

Description

The present invention relates to an optical information reader that optically reads information.

BACKGROUND ART In recent years, so-called traceability, which enables, for example, traceability of a distribution route of an article from a manufacturing stage to a consumption stage or a disposal stage, has been emphasized, and code readers for the purpose of this traceability have become popular. Code readers are also used in various fields other than traceability.

Generally, a code reader photographs a code such as a bar code or a two-dimensional code attached to a workpiece with a camera, extracts the code contained in the obtained image by image processing, binarizes it, and decodes it. It is configured to be able to read information, and is also called an optical information reader because it is a device for optically reading information.

As disclosed in Patent Document 1, for example, this type of optical information reading device learns a model structure showing the relationship between a code image acquired by a visual sensor of a robot and an ideal code image. There is known a machine equipped with a machine learning device that Patent Document 1 describes restoring an image suitable for reading by applying learning results from a machine learning device to an image of a code acquired by a visual sensor during operation.

Japanese Patent No. 6683666

By the way, image restoration by machine learning, that is, inference processing of an ideal image imposes a heavy computational load on a processor and takes a long time to process. Therefore, if the read image is not difficult to read, that is, if the read image is in good printing condition, it is not necessary to input it to the neural network and execute the inference processing of the ideal image, and the read image is immediately subjected to decoding processing. Considering the processing time, this method is preferable. On the other hand, it is not easy to determine in advance from the read image whether inference processing by machine learning is necessary or not, and serial execution of decoding processing and inference processing by machine learning takes time.

The present invention has been made in view of this point, and its purpose is to improve the reading accuracy by inference processing of machine learning, but to prevent the processing time from becoming long due to the load of the inference processing. is to suppress

In order to achieve the above object, one aspect of the present disclosure can presuppose an optical information reading device that reads a code attached to a work. The optical information reading device includes a camera that captures a code and generates a read image, and a storage unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to the read image generated by the camera. an inference processing unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating an ideal image corresponding to the read image; , a decoding processing unit for executing a first decoding process for decoding the read image generated by the camera and a second decoding process for decoding the ideal image generated by the inference processing unit. The processor executes inference processing by the inference processing unit and first decoding processing by the decoding processing unit in parallel, and after the inference processing by the inference processing unit is completed, the second decoding by the decoding processing unit. configured to do the work.

According to this configuration, when the read image generated by the camera during the operation of the optical information reading device is not suitable for reading the code, such as when the print condition of the code is poor, the read image generated by the camera An ideal image is generated by inputting an image to the inference processing unit and executing inference processing. On the other hand, if the read image generated by the camera during the operation of the optical information reading device is suitable for reading the code, such as when the code printing condition is good, the read image generated by the camera is inferred. First decoding processing is performed in the decoding processing unit without being input to the processing unit. Since the inference processing and the first decoding processing are executed in parallel, executing the decoding processing for another read image that does not require the inference processing while the inference processing is being executed for the read image that requires the inference processing; Alternatively, inference processing can be executed for another read image that requires inference processing while decoding processing is being executed for a read image that does not require inference processing. This shortens the processing time. Note that the first decoding process and the inference process may be performed on the same read image.

Further, after the inference processing by the inference processing unit is completed, the second decoding processing is executed to decode the ideal image, so the reading accuracy is enhanced.

In another aspect of the present disclosure, the decoding processing section may have a core that executes the first decoding process and the second decoding process in parallel in separate threads.

In another aspect of the present disclosure, the decoding processing unit may be configured with multi-cores capable of executing the first decoding process and the second decoding process with different cores.

According to this configuration, since the core that executes the first decoding process and the core that executes the second decoding process are different cores, the first decoding process and the second decoding process can be executed in parallel. time is reduced.

In another aspect of the present disclosure, the decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image generated by the camera, and extracts the read image by the extraction process. and a general-purpose core for executing the first decoding process for decoding the partial image extracted from the inside. The inference processing unit has an inference processing dedicated core that inputs a partial image extracted from a read image by extraction processing and executes inference processing for generating an ideal image corresponding to the read image.

According to this configuration, the general-purpose core executes the code candidate region extraction process and the first decoding process of decoding the partial image extracted by the extraction process, and the inference process is performed by the inference process different from the general-purpose core. Inference processing can be accelerated because it can be executed on a dedicated inference processing core.

In another aspect of the present disclosure, the general-purpose core executes second decoding processing for decoding an ideal image generated by the inference processing performed by the inference processing dedicated core.

In another aspect of the present disclosure, the general-purpose core includes a first core that executes the extraction process, a second core that executes the first decoding process, and a third core that executes the second decoding process. have.

According to this configuration, the extraction process, the first decoding process, and the second decoding process can all be executed by separate cores. In particular, since the second core and the third core can be specialized for decoding processing, the processing speed is further increased.

In another aspect of the present disclosure, when an ideal image is generated by the inference processing performed by the inference processing dedicated core, the general-purpose core performs the second decoding processing for decoding the ideal image prior to the first decoding processing. to execute.

That is, when an ideal image is generated by the inference processing, the processing time of the first decoding processing for the read image generated by the camera takes a long time, and the image is considered to be difficult to read. Therefore, when an ideal image is generated, the ideal image is decoded prior to the first decoding process for the read image generated by the camera, and as a result, a highly accurate reading result can be obtained in a short time. .

In another aspect of the present disclosure, when a plurality of the partial images are extracted as a result of the extraction processing by the general-purpose core, the inference processing dedicated core performs inference processing on the plurality of partial images according to a predetermined order. can be executed. Further, when a plurality of partial images are extracted as a result of the extraction processing, the general-purpose core can execute the first decoding processing on the plurality of partial images according to a predetermined order.

That is, when one read image is generated by photographing a workpiece, a plurality of code candidate areas may exist in the one read image. In this case, multiple partial images are extracted. Inference processing can be performed on a plurality of partial images, for example, in descending order of the priority of the inference processing. In addition, the first decoding process can be executed on a plurality of partial images, for example, in descending order of decoding process priority. The priority may be based on a condition for determining a predetermined order, and for example, processing may be performed starting with partial images that are more likely to be codes.

In another aspect of the present disclosure, the processor is configured to end the inference processing by the inference processing unit when decoding is successful in the first decoding processing.

That is, the first decoding process is a decoding process for a read image that has not undergone inference processing. Successful decoding of the read image means that the read image is not difficult to read, and no inference processing is required. In this case, even if the inference process is in progress, by ending the inference process, it is possible to early start the inference process for another read image that is input next.

In another aspect of the present disclosure, the general-purpose core performs a first extraction process of extracting the code candidate region under a first predetermined condition, and extracts the code candidate region with higher precision than the first predetermined condition. It is possible to execute a second extraction process of extracting under a second predetermined condition that enables and the first decoding process of decoding a partial image extracted from the read image by the first extraction process. Further, the inference processing dedicated core can execute inference processing for inputting a partial image extracted from the read image by the second extraction processing and generating an ideal image corresponding to the read image.

According to this configuration, the general-purpose core can perform the first extraction process with relatively low precision and the second extraction process with relatively high precision. Since the accuracy of the first extraction process is low, high-speed extraction is possible. , the reading result can be output in a short time.

On the other hand, a read image for which decoding fails in the first decoding process is an image that is difficult to read and may be difficult to extract in the first extraction process. In this case, the partial image extracted by the second extraction process capable of extracting with higher accuracy is subjected to inference processing by the inference processing core to generate the ideal image. Reading accuracy is improved by subjecting the generated ideal image to the second decoding process.

That is, the first decoding process can output the reading result of the read image that is easy to read at high speed, and the second decoding process can output the reading result of the read image that is highly likely to fail in decoding in the first decoding process. It is possible to reduce the frequency of decoding failures by enabling reading even with respect to

In another aspect of the present disclosure, the optical information reading device stores a partial image extracted from the read image by the extraction process, and stores a memory that can be accessed by both the general-purpose core and the inference process-dedicated core. I have.

According to this configuration, the extraction result of the extraction process can be shared between the general-purpose core and the inference process-dedicated core, so that the processing speed can be increased. The memory is composed of a RAM or the like, and the general-purpose core and the dedicated inference processing core can be configured to access the same RAM.

In another aspect of the present disclosure, the general-purpose core and the inference processing dedicated core may be mounted on the same substrate. In this case, the dedicated inference processing core can be composed of an IC, FPGA, or the like dedicated to inference processing by a neural network.

In another aspect of the present disclosure, an image restoration unit that inputs a read image to a neural network and performs an inference process to generate a restored image obtained by restoring the read image, and a first decoding process for the read image generated by the camera. and a second decoding processing unit for executing second decoding processing on the restored image generated by the image restoration unit. The second decoding processing section extracts a code candidate region from the read image and sends a trigger signal to the image restoration section to execute inference processing. When receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs the partial image corresponding to the code candidate area to the neural network, and executes inference processing to generate a restored image by restoring the partial image. do. The second decoding processing unit can determine grid positions indicating each cell position of the code based on the restored image, and decode the restored image based on the determined grid positions. That is, the first decoding processing unit decodes a normal read image that is not subjected to inference processing, the image restoration unit generates a restored image by inference processing, and the restored image restored by inference processing By having the second decoding processing unit execute the decoding of , it is possible to construct a processing circuit suitable for each of them and to increase the processing speed. In addition, since the image restoration section only needs to restore the partial image corresponding to the code candidate area, the processing speed can be further increased.

In another aspect of the present disclosure, processing speed can be increased by executing inference processing by the image restoration unit and first decoding processing by the first decoding processing unit in parallel. In addition, since at least a part of the period during which the second decoding processing unit is executing the second decoding processing has a pause period during which the inference processing by the image restoration unit is not executed, the load on the image restoration unit can be reduced, Heat generation can be suppressed. In this case, the second decoding processing section may be configured with a plurality of cores.

In another aspect of the present disclosure, the first decoding processing section can execute the first decoding process in a shorter time than the time required for the second decoding process. Further, the setting unit can switch whether or not to execute the second decoding process, that is, whether or not to execute the restored image decoding process. Since the second decoding process is for decoding the restored image, it takes longer than the first decoding process for decoding the normal read image. On this premise, if it is set to execute the second decoding process, it is possible to set a decoding timeout time (decoding limit time) longer than the time required to execute the second decoding process. On the other hand, if it is set not to execute the second decoding process, a decode timeout period shorter than the time required to execute the second decoding process can be set. When it exceeds, it is possible to quickly shift to the decoding process of the next captured image.

The decoding processing unit according to another aspect of the present disclosure can execute the second decoding process on the restored image in parallel with the first decoding process on the read image generated by the camera. processing speed can be increased. When the setting unit has set not to execute the second decoding process, the decoding process part can allocate processing resources used for the second decoding process to the first decoding process. The processing resource is, for example, a memory usage area, a core constituting a multi-core CPU, or the like. When the second decoding process is set not to be executed, the memory area and the core can be used for the first decoding process, so that the first decoding process can be further speeded up.

A decoding processing unit according to another aspect of the present disclosure extracts a code candidate area in which a code is likely to exist from a read image. The image restoration unit inputs a partial image corresponding to the code candidate area to the neural network and generates a restored image by restoring the partial image. The decoding processing unit determines a grid position based on the restored image generated by the image restoration unit, and decodes the restored image based on the determined grid position. That is, since it often takes a long time to generate a repaired image, by extracting a code candidate region based on, for example, a feature value indicating code-likeness before generating a repaired image, Save time. Also, since the grid position can be determined with high accuracy based on the restored image, reading performance is improved.

In another aspect of the present disclosure, tuning processing is performed to determine the optimum imaging conditions and decoding conditions and to set the size of the code to be read. The decoding processing unit can determine the size of the extracted image including the code candidate area from the read image based on the size of the code set in the tuning process. By enlarging/reducing the extracted image to an image of a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range. Restoration effect can be stably drawn out. In addition, although the size of the code to be read may vary, since the size of the code that is finally input to the neural network can be fixed, the balance between high speed processing and ease of decoding can be achieved. can keep.

A camera according to another aspect of the present disclosure receives light that passes through the first polarizing plate and is reflected from the code through the second polarizing plate. to generate a low-contrast read image. By using the polarizing plate, the specular reflection component of the workpiece is removed, so that a read image can be acquired with reduced influence of the specular reflection component. For example, a polarizing plate is suitable for an image with many regular reflection components, such as when an image of a metal workpiece is captured, but a decrease in the amount of light may darken the read image and lower the contrast. In such a case, reading performance is improved by converting the image into a high-contrast restored image through neural network inference processing.

In another aspect of the present disclosure, the tuning execution unit can set a plurality of imaging conditions and code conditions including a first imaging condition and code condition and a second imaging condition and code condition. The decode processing unit inputs the read image generated under the first imaging condition and the code condition to the neural network to generate the restored image, and if the decode processing of the restored image fails, the first imaging condition and the code condition are generated. A read image generated under second imaging conditions and code conditions different from the code conditions is input to the neural network to generate a restored image, and the restored image can be decoded.

As described above, the inference processing by the logic processing unit and the decoding processing by the decoding processing unit can be executed in parallel. It is possible to suppress the processing time from increasing due to the load of the inference processing while improving the accuracy.

It is a figure explaining the time of operation of a stationary optical information reader. 1 is a perspective view of a stationary optical information reader; FIG. 1 is a block diagram of an optical information reader; FIG. 1 is a perspective view of a handheld optical information reader; FIG. It is a figure explaining each part comprised by a processor. 1 is a conceptual diagram of a neural network; FIG. 4 is a flow chart showing an example of a basic procedure during learning of a neural network; A pair of a defective image and an ideal image is shown, where (A) is an example of a defective image and (B) is an example of an ideal image. 1 is a conceptual diagram of a convolutional neural network used for image transformation; FIG. 4 is a flow chart showing an example of a procedure of a tuning process performed when setting the optical information reading device; 7 is a flow chart showing an example of a decoding processing procedure before determination of a reduction ratio and an enlargement ratio; 7 is a flow chart showing an example of a decoding process procedure after determination of a reduction rate and an enlargement rate; (A) shows an example of a read image generated by a camera, and (B) shows an example of an image after execution of an image processing filter. (A) shows an example of an image after reduction processing, and (B) shows an example of an image after code region extraction. FIG. 10 is a diagram showing an example in which an inference process is performed on a read image by a neural network; FIG. 10 is a flow chart showing a first example of parallel execution of decode processing and inference processing for a read image; FIG. FIG. 11 is a sequence diagram showing sequence example 1 of the first example; FIG. 11 is a sequence diagram showing sequence example 2 of the first example; FIG. 12 is a sequence diagram showing sequence example 3 of the first example; FIG. 10 is a flow chart showing a second example of parallel execution of decoding processing and inference processing for a read image; FIG. FIG. 12 is a sequence diagram showing a sequence example of the second example; FIG. 11 is a flow chart showing a third example of parallel execution of decoding processing and inference processing for a read image; FIG. FIG. 12 is a sequence diagram showing a sequence example of the third example; FIG. 10 is a task sequence diagram when reading is successful in the first decoding process; FIG. 10 is a task sequence diagram when reading is successful in the second decoding process; FIG. 10 is a task sequence diagram when reading is successful in the first decoding process and the second decoding process; FIG. 10 is a task sequence diagram when reading is successful twice in the second decoding process; FIG. 10 is a reading sequence diagram during continuous shooting; FIG. 10 is a sequence diagram of decoding processing during continuous shooting; It is a figure which shows an example of the decoding processing procedure which concerns on other embodiment. 4 is a time chart of decoding processing by a first decoding processing unit and a second decoding processing unit; 4 is a time chart of decoding processing by a decoding processing unit; 4 is a flowchart showing an example of first decoding processing and second decoding processing; It is a figure which shows the image example of each process. FIG. 10 is a diagram for explaining a case in which a plurality of neural networks are used to handle black-and-white inversion of a code; FIG. 10 is a diagram for explaining a case in which one neural network is used to handle black/white inversion of a code; 4 is a graph showing the relationship between contrast and matching level; 7 is a flowchart illustrating an example of matching level calculation processing; FIG. 4 is a diagram showing an example of a user interface displayed on the display unit; It is a figure which shows the example using an imaging device with an AI chip.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

(Stationary Optical Information Reader)
FIG. 1 is a diagram schematically showing operation of a stationary optical information reader 1 according to an embodiment of the present invention. In this example, a plurality of works W are conveyed in the direction of arrow Y in FIG. Such an optical information reader 1 is installed. The optical information reader 1 is a code reader configured to photograph a code attached to a work W, decode the code contained in the photographed image, and read information. In the example shown in FIG. 1, the optical information reader 1 is of a stationary type. When the stationary optical information reader 1 is operated, it is fixed to a bracket or the like (not shown) so that the optical information reader 1 does not move. The stationary optical information reader 1 may be used while being held by a robot (not shown). Alternatively, the code of the work W in a stationary state may be read by the optical information reader 1 . The operating time of the stationary optical information reading device 1 is the time when the code of the work W conveyed by the conveying belt conveyor B is sequentially read.

A code is attached to the outer surface of each work W. As shown in FIG. Codes include both barcodes and two-dimensional codes. Examples of two-dimensional codes include QR code (registered trademark), micro QR code, data matrix (Data code), Veri code, Aztec code, PDF417, and Maxi code. and so on. Two-dimensional codes include stack type and matrix type, but the present invention can be applied to any two-dimensional code. The code may be attached by printing or stamping it directly on the work W, or by attaching it to the work W after being printed on a label.

The optical information reader 1 is wired to a computer 100 and a programmable logic controller (PLC) 101 via signal lines 100a and 101a, respectively. A communication module may be incorporated in the PLC 101 to wirelessly connect the optical information reader 1 with the computer 100 and the PLC 101 . The PLC 101 is a control device for sequence-controlling the transport belt conveyor B and the optical information reader 1, and a general-purpose PLC can be used. As the computer 100, a general-purpose or dedicated computer, a portable terminal, or the like can be used.

During operation, the optical information reader 1 receives a read start trigger signal that defines the start timing of code reading from the PLC 101 via the signal line 101a. Then, the optical information reader 1 picks up and decodes the code based on this reading start trigger signal. After that, the decoded result is transmitted to the PLC 101 via the signal line 101a. As described above, when the optical information reader 1 is operated, the input of the reading start trigger signal and the output of the decoding result are performed between the optical information reader 1 and the external control device such as the PLC 101 via the signal line 101a. Repeatedly. The reading start trigger signal input and the decoding result output may be performed via the signal line 101a between the optical information reader 1 and the PLC 101 as described above, or other signals (not shown) may be input. You can go through the line. For example, a sensor for detecting the arrival of the work W may be directly connected to the optical information reader 1, and a reading start trigger signal may be input to the optical information reader 1 from the sensor.

As shown in FIG. 2, the optical information reader 1 includes a box-shaped housing 2, a polarizing filter attachment 3, an illumination section 4, a camera 5, a display section 6, a power connector 7, and a signal line connector. 8 are provided. Further, the housing 2 is provided with an indicator 9, an aimer light irradiation section (light irradiation section) 10, and operation buttons 11 and 12. The indicator 9, the aimer light irradiation section 10 and the operation buttons 11 and 12 are also provided. It is a component of the optical information reader 1 .

The housing 2 has a shape elongated in a predetermined direction, but the shape of the housing 2 is not limited to the illustrated shape. A polarizing filter attachment 3 is detachably attached to the front outer surface of the housing 2 . Inside the housing 2, an illumination unit 4, a camera 5, an aimer light irradiation unit 10, a processor 20, a storage device 30, a ROM 40, a RAM 41, and the like are accommodated. The processor 20 , storage device 30 , ROM 40 and RAM 41 are also components of the optical information reader 1 .

A lighting section 4 is provided on the front side of the housing 2 . The illumination unit 4 is a part for illuminating at least the code of the work W by irradiating light toward the front of the optical information reading device 1 . As also shown in FIG. 3, the lighting unit 4 includes a first lighting unit 4a made up of a plurality of light emitting diodes (LEDs: Light Emission Diodes), a second lighting unit 4b made up of a plurality of light emitting diodes, and a first lighting unit 4b made up of a plurality of light emitting diodes. and an illumination drive unit 4c including an LED driver or the like for driving the unit 4a and the second illumination unit 4b.The first illumination unit 4a and the second illumination unit 4b are individually driven by the illumination drive unit 4c, The lighting drive unit 4c is connected to the processor 20, and is controlled by the processor 20. The first lighting unit 4c can be turned on and off. One of 4a and the second lighting unit 4b may be omitted.

As shown in FIG. 2, a camera 5 is provided in the central portion of the front side of the housing 2 . The direction of the optical axis of the camera 5 substantially coincides with the direction of light emitted by the illumination section 4 . The camera 5 is a part that photographs the code and generates a read image. As shown in FIG. 3, the camera 5 includes an imaging device 5a for receiving reflected light from a code attached to the work W and illuminated by the illumination unit 4, an optical system 5b having a lens and the like, an AF and a module (autofocus module) 5c. Light reflected from the coded portion of the workpiece W is incident on the optical system 5b. image.

The imaging device 5a is an image sensor comprising a light-receiving device such as a CCD (charge-coupled device) or CMOS (complementary metal oxide semiconductor) that converts a code image obtained through the optical system 5b into an electric signal. The image sensor 5a is connected to a processor 20, and the electric signal converted by the image sensor 5a is input to the processor 20 as read image data. The AF module 5c is a mechanism that performs focusing by changing the position and refractive index of a focusing lens among the lenses that make up the optical system 5b. The AF module 5 c is also connected to and controlled by the processor 20 .

As shown in FIG. 2, a display portion 6 is provided on the side surface of the housing 2. As shown in FIG. The display unit 6 is composed of, for example, an organic EL display, a liquid crystal display, or the like. The display unit 6 is connected to the processor 20, and can display, for example, the code imaged by the imaging unit 5, the character string that is the result of decoding the code, the reading success rate, the matching level, and the like. The reading success rate is the average reading success rate when the reading process is executed multiple times. The matching level is a reading margin indicating how easily a successfully decoded code can be read. This can be obtained from the number of error corrections that occur during decoding, and can be represented by a numerical value, for example. The fewer error corrections, the higher the matching level (reading margin), while the more error corrections, the lower the matching level (reading margin).

A power cable (not shown) for supplying power to the optical information reader 1 from the outside is connected to the power connector 7 . Signal lines 100 a and 101 a for communicating with the computer 100 and PLC 101 are connected to the signal line connector 8 . The signal line connector 8 can be composed of, for example, an Ethernet connector, a serial communication connector such as RS232C, a USB connector, or the like.

An indicator 9 is provided on the housing 2 . The indicator 9 is connected to the processor 20 and can consist of a light emitter such as a light emitting diode. The operating state of the optical information reader 1 can be notified to the outside by the lighting state of the indicator 9 .

A pair of aimer light irradiation units 10 are provided on the front side of the housing 2 so as to sandwich the camera 5 therebetween. As shown in FIG. 3, the aimer light irradiation unit 10 includes an aimer 10a composed of a light emitting diode or the like, and an aimer driving unit 10b for driving the aimer 10a. The aimer 10a irradiates light (aimer light) toward the front of the optical information reading device 1 to indicate the imaging range and the center of the visual field of the camera 5, the guideline of the optical axis of the illumination unit 4, and the like. . Specifically, the aimer 10a irradiates visible light of a color (for example, red or green) different from ambient light toward the range of the imaging field of view of the camera 5, and the surface irradiated with the visible light is visible to the naked eye. Form a visible landmark. The mark may be various figures, symbols, characters, or the like. The user can also set the optical information reader 1 by referring to the light emitted from the aimer 10a.

As shown in FIG. 2, the side surface of the housing 2 is provided with operation buttons 11 and 12 that are used when setting the optical information reader 1 and the like. The operation buttons 11 and 12 include, for example, select buttons and enter buttons. In addition to the operation buttons 11 and 12, for example, touch panel type operation means may be provided. The operation buttons 11 and 12 are connected to a processor 20 so that the processor 20 can detect the operation states of the operation buttons 11 and 12. FIG. By operating the operation buttons 11 and 12, it is possible to select one of a plurality of options displayed on the display unit 6 and to confirm the selected result.

(hand-held optical information reader)
In the above example, the optical information reader 1 is of a stationary type, but the present invention is applicable to devices other than the stationary optical information reader 1 . FIG. 4 shows a handheld optical information reader 1A, and the present invention can also be applied to a handheld optical information reader 1A as shown in this figure.

A housing 2A of the hand-held optical information reader 1A has a vertically elongated shape. The orientation of the optical information reader 1A when used is not limited to the illustrated orientation, and can be used in various orientations. is doing.

A display portion 6A is provided on the upper portion of the housing 2A. The display section 6A is configured in the same manner as the display section 6 of the stationary optical information reader 1. FIG. A lower portion of the housing 2A is a grip portion 2B for a user to grip during operation. The grip part 2B is a part that a general adult can hold with one hand, and its shape and size can be freely set. By holding the grip portion 2B, the optical information reader 1A can be carried and moved. In other words, the optical information reader 1A is a portable terminal device, and can be called a handy terminal, for example.

Similar to the stationary optical information reader 1, the hand-held housing 2A also accommodates an illumination unit, a camera, an aimer light irradiation unit, a processor, a storage unit, a ROM, a RAM, etc. (not shown). . The optical axis of the illumination unit, the optical axis of the camera, and the optical axis of the aimer light irradiation unit are directed obliquely upward from the vicinity of the upper end of the housing 2A. A buzzer (not shown) is also provided in the handheld housing 2A.

A plurality of operation buttons 11A and trigger keys 11B are provided on the grip portion 2B and its vicinity. The operation button 11A is the same as the operation button 11 of the stationary optical information reader 1 and the like. When the user presses the trigger key 11B with the tip (upper end) of the optical reader 1A facing the workpiece W, the tip of the optical reader 1A emits an aimer beam. When the user adjusts the direction of the optical reader 1A while visually observing the aimer light reflected on the surface of the work W, and aligns the aimer light with the code to be read, the code is automatically read and decoded. . When the reading is completed, the buzzer emits a completion notification sound.

As an example of use of the handheld optical information reader 1A, there is an example of using the handheld optical information reader 1A for picking work in a distribution warehouse. For example, when shipping ordered products from a distribution warehouse, the necessary products are picked from the product shelves in the product warehouse. This picking operation is performed by a user holding an order slip with a code written thereon, after moving to the product shelf, while comparing the code of the order slip with the code attached to the product or the product shelf. In this case, the hand-held optical information reader 1A alternately reads the code of the order slip and the code attached to the product or the product shelf.

(Processor configuration)
The following description is common to the stationary optical information reader 1 and the handheld optical information reader 1A, and can be applied to either apparatus 1 or 1A unless otherwise specified. As shown in FIG. 3, the processor 20 is a multi-core processor having multiple cores that are central processing units. Specifically, the processor 20 includes a first general-purpose core 21, a second general-purpose core 22, a third general-purpose core 23, a fourth general-purpose core 24, and a dedicated core 25 dedicated to inference processing by a neural network. The first to fourth general-purpose cores 21 to 24 and dedicated core 25 are so-called system-on-a-chip (SoC, SOC) and are mounted on the same substrate. Note that the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 may not be SoC, and in that case, they may not be mounted on the same substrate. included. In this embodiment, the case where the number of general-purpose cores is four will be described, but the number of general-purpose cores is not limited to this, and may be one, or any number of two or more (for example, six cores, eight cores, etc.). may be

A RAM 41 as the same memory is connected to the first to fourth general-purpose cores 21 to 24 and the dedicated core 25, and all of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 have the same RAM 41. is accessible. Further, the processor 20 is connected with a ROM 40 as the same memory, and all of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 can access the same ROM 40 as well.

The first to fourth general-purpose cores 21 to 24 are so-called general-purpose processors, for example, AF control, lighting control, camera control, extraction processing for extracting code candidate regions, decoding processing for read images, various filter processing for read images, and the like. is the part that executes A specific processing example of the first to fourth general-purpose cores 21 to 24 will be described later.

On the other hand, the dedicated core 25 is a dedicated inference processing core for executing inference processing for generating an ideal image corresponding to the read image using a neural network, and exceeds the sum-of-products operation required for processing by the neural network. Specialized for fast execution. The dedicated core 25 includes, for example, an IC, FPGA, and the like. Since the read image can be restored to an image suitable for decoding by applying the learning result of the neural network and performing inference processing, generating an ideal image by inference processing is also called restoration of the read image. In this case, the dedicated core 25 is a part that tries to restore the read image using a neural network.

As shown in FIG. 5, the processor 20 operates an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, an expansion unit 20i, and inference processing. A part 20j is configured. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decoding processing unit 20f, the extraction unit 20g, the reduction unit 20h, and the expansion unit 20i perform calculations of the first to fourth general-purpose cores 21-24. It is a part configured by processing. On the other hand, the inference processing unit 20j is a part configured by the dedicated core 25, and is also called an image restoration unit because it is a part that executes inference processing for generating a restored image obtained by restoring a read image.

(Configuration of AF control unit)
The AF control unit 20a is a unit that controls the AF module 5c shown in FIG. 3, and is configured to be able to focus the optical system 5b by conventionally known contrast AF or phase difference AF. The AF control unit 20a may be composed of cores serving as the decoding processing unit 20f and the extraction unit 20g among the first to fourth general-purpose cores 21 to 24, or cores other than the cores serving as the decoding processing unit 20f and the extraction unit 20g. core.

(Configuration of imaging control section)
The imaging control section 20b is a unit that adjusts the gain of the camera 5, controls the amount of light of the illumination section 4, and controls the exposure time (shutter speed) of the imaging device 5a. Here, the gain of the camera 5 is an amplification factor (also called magnification) when the brightness of the image output from the image sensor 5a is amplified by digital image processing. The amount of light of the illumination unit 4 can be changed by controlling the first illumination unit 4a and the second illumination unit 4b separately. The gain, the amount of light of the illumination unit 4 and the exposure time are imaging conditions of the camera 5 . The imaging control unit 20b may be composed of cores serving as the decoding processing unit 20f and the extraction unit 20g among the first to fourth general-purpose cores 21 to 24, or cores other than the cores serving as the decoding processing unit 20f and the extraction unit 20g. core. Note that the AF control unit 20a and the imaging control unit 20b may be configured with the same core, or may be configured with different cores.

(Configuration of filter processing section)
The filter processing unit 20c is a portion that executes an image processing filter on the read image, and may be composed of cores serving as the decoding processing unit 20f and the extraction unit 20g, or may be the decoding processing unit 20f and the extraction unit 20g. A core other than the core may be used. A core that configures the filter processing unit 20c may be a DSP core.

The filter processing unit 20c executes a noise removal filter that removes noise included in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, and the like. The image processing filters executed by the filter processing unit 20c are not limited to noise removal filters, contrast correction filters, and averaging filters, and may include other image processing filters.

The filter processing unit 20c is configured to perform an image processing filter on a read image before execution of inference processing, which will be described later. Further, the filter processing unit 20c is configured to perform an image processing filter on a read image before enlargement or reduction, which will be described later. Note that the filter processing unit 20c may be configured to execute an image processing filter on the read image after executing the inference processing, or execute the image processing filter on the read image after enlargement or reduction. It may be configured as

(Configuration of Tuning Execution Unit)
The tuning execution unit 20d shown in FIG. 5 changes the imaging conditions of the camera 5 and the decoding conditions of the decoding process, repeats the imaging and decoding processes, and makes it easier to read the code calculated under each imaging condition and decoding condition. Based on the matching level indicating the size, the optimum imaging condition and decoding condition are determined, and a tuning process for setting the size of the code to be read is executed. Specifically, when setting the optical information reader 1 or 1A, the tuning execution unit 20d sets the imaging conditions such as the gain of the camera 5, the light amount and exposure time of the illumination unit 4, and the image processing conditions in the filter processing unit 20c. is changed to set various conditions (tuning parameters) so that the conditions are suitable for decoding. The image processing conditions in the filter processing unit 20c include image processing filter coefficients (filter strength), switching of image processing filters when there are a plurality of image processing filters, combination of different types of image processing filters, and the like. Appropriate imaging conditions and image processing conditions differ depending on the influence of external light on the workpiece W during transportation, the color and material of the surface to which the code is attached, and the like. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets processing by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

The tuning execution unit 20d is configured to be able to acquire the size of the code included in the read image generated by the camera 5. FIG. When acquiring the size of the chord, first, the tuning execution unit 20d searches for the chord based on the feature quantity indicating the likeness of the chord. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. The PPC, code type, code size, etc. are included in the code parameters or code conditions, so the tuning execution unit 20d is configured to be able to acquire the code parameters or code conditions of the searched code.

The two-dimensional code is constructed by randomly arranging a plurality of white modules and black modules. For example, QR code model 2 has modules from 21×21 to 177×177. In the case of the stationary optical information reader 1, by reading the code to be read in advance, the number of modules and PPC are limited, and the code size (pixel size) is limited to the number of modules x PPC. .

PPC is a code parameter that indicates how many pixels (picture elements) constitute one module among the modules that constitute the code. After identifying one module, the tuning execution unit 20d can obtain the PPC by counting the number of pixels that constitute one module.

The code type is the type of code such as QR code, data matrix code, Vericode, for example. Since each chord has its own characteristics, the tuning execution unit 20d can determine the chord type by, for example, the presence or absence of a finder pattern.

Also, the code size can be calculated from the number of modules arranged in the vertical or horizontal direction of the code and the PPC. The tuning execution unit 20d can obtain the code size by calculating the number of modules arranged vertically or horizontally in the code, calculating the PPC, and multiplying the number of modules arranged vertically or horizontally by the PPC.

(Structure of decoding processing unit)
The decoding processing unit 20f is a part that decodes black and white binarized data. Decoding can use a table showing the correspondence of the encoded data. Furthermore, the decoding processing unit 20f checks whether the decoded result is correct according to a predetermined check method. If an error is found in the data, correct data is calculated using the error correction function. The error correction function differs depending on the type of code.

In this embodiment, the decoding processing unit 20f decodes a code included in an ideal image after inference processing, which will be described later, but can also decode a code included in a read image before inference processing. The decoding processing unit 20f is configured to write the decoding result obtained by decoding the code into the decoding result storage unit 40b of the ROM 40 shown in FIG.

(Structure of extraction unit)
The extraction unit 20g is a part that extracts a code candidate area that is highly likely to contain a code from the read image generated by the camera 5. FIG. A chord candidate region can be extracted based on a feature amount indicating chord-likeness. In this case, the feature amount indicating chord-likeness serves as information for specifying a chord. For example, the extraction unit 20g can acquire a read image and search for a code in the acquired read image based on a feature amount indicating code-likeness. Specifically, it is searched for whether or not there is a portion having a predetermined or more feature amount indicating code-likeness in the acquired read image, and as a result, a portion having a feature amount indicating code-likeness is searched. If possible, an area including that part is extracted as a code candidate area. The code candidate region may include regions other than code, but includes at least a portion that is more likely to be code than a predetermined number. Note that the code candidate region is a region in which there is a high possibility that a code exists, and therefore, there may be a case where the code is not included in the region.

The extraction unit 20g can also use the positional information of the aimer beam emitted from the aimer beam irradiation unit 10 as information for identifying the code, and extract the code candidate area. In this case, the extraction unit 20g identifies a portion corresponding to the mark formed by the aimer light irradiation unit 10 from the read image, and extracts the identified portion as a code candidate area. That is, as described above, since the aimer light is used to indicate the imaging range and the center of the field of view of the camera 5, the mark formed by the aimer light should be aligned with the center of the field of view of the camera 5 in advance. It is possible. In particular, in the case of the hand-held optical information reader 1A, the user aligns the aimer light with the code to be read, so there is a high probability that the mark formed by the aimer light overlaps the code. That is, by aligning the portion corresponding to the mark formed by the aimer light irradiation unit 10 with the center of the field of view of the camera 5, when the extraction unit 20g extracts an area including the center of the field of view of the camera 5, the area is a code. is likely to exist. In this case, the portion corresponding to the mark formed by the aimer light irradiation unit 10 does not have to be completely aligned with the center of the field of view of the camera 5, and may be slightly shifted in the vertical or horizontal direction of the field of view. Also, since the code has a predetermined size, the region extracted by the extraction unit 20g is not only the center of the field of view of the camera 5, but also a region of a predetermined size including the center of the field of view.

The extraction unit 20g can also specify the central portion of the read image and extract the central portion as the code candidate area. For example, by obtaining in advance the center of the field of view of the camera 5 as information for specifying the code, the portion that will be the center of the field of view of the camera 5 can be specified on the read image. In particular, in the case of the hand-held optical information reader 1A, since the central portion specified on the read image corresponds to the mark formed by the aimer light irradiation section 10, the code may exist in the central portion. This is an area with high potential.

The extraction unit 20g can be configured to accept a user's designation of a predetermined portion from the read image, and can extract the designated portion as a code candidate region. That is, when the user designates an area of arbitrary size at an arbitrary position (coordinates) on the read image, the extraction unit 20g extracts a predetermined area from the read image based on the information on the coordinates and size of the position. Accepts part specification. The extraction unit 20g extracts the received portion as a code candidate region. For example, in the case of a stationary optical information reader 1, as shown in FIG. The work W does not always exist in the center of the width direction of the belt conveyor B, but the work W may exist at the end. In this case, the conveying belt conveyor B By specifying the part corresponding to the end of , it is possible to accurately extract the area where the code is likely to exist. In addition, there are cases where the code is displayed near the edge of a large work W, away from the center. By doing so, it is possible to accurately extract the region where the code is likely to exist.

(Configuration of storage device and ROM)
The storage device 30 shown in FIG. 3 can be composed of a readable/writable storage device such as an SSD (Solid State Drive). The storage device 30 can store various programs, setting information, image data, and the like.

The ROM 40 includes an image data storage section 40a, a decoding result storage section 40b, a parameter set storage section 40c, and a neural network storage section 40d. The image data storage section 40a is a section that stores the read image generated by the camera 5. FIG. The decoding result storage unit 40b is a part that stores the decoding result of the code executed by the decoding processing unit 20f. The parameter set storage section 40c is a section that stores various conditions set as a result of tuning executed by the tuning execution section 20d and various conditions set by the user. The neural network storage unit 40d is a part that stores the structure and parameters of a trained neural network, which will be described later.

At least part of the storage units 40a, 40b, 40c, and 40d may be provided in a portion other than the ROM 40, and may be provided in the storage device 30, for example.

(Inference processing using neural networks)
The optical information reader 1, 1A has an ideal image generating function of generating an ideal image by inferring the read image acquired by the camera 5 using a neural network. In this embodiment, instead of performing machine learning in the optical information reader 1, 1A, the optical information reader 1, 1A holds in advance the structure and parameters of the neural network that has already completed learning, and An example configured to execute inference processing using a neural network configured with a retained structure and parameters will be described, but not limited to this, machine learning with the optical information reading device 1, 1A may be performed to change the structure and parameters of the neural network.

As shown in FIG. 6, the neural network has an input layer to which input data (image data in this example) is input, an output layer to output output data, and an intermediate layer provided between the input layer and the output layer. have. For example, a plurality of intermediate layers can be provided, thereby forming a multi-layered neural network.

(Neural network learning)
First, the basic procedure for neural network learning will be described based on the flow chart shown in FIG. Learning of the neural network can be performed using a computer prepared for learning other than the optical information reading devices 1 and 1A, but may be performed using a general-purpose computer other than for learning. Also, the learning may be performed by the optical information reader 1, 1A. A conventionally well-known method may be used for the learning method of the neural network.

In step SA1 after the start, data of defective images prepared in advance are read. A defective image is an image having a portion inappropriate for code reading, and can be exemplified by the image shown on the left side of FIG. The inappropriate part is, for example, a dirty part, a colored part, or the like. After that, the process advances to step SA2 to read the ideal image data prepared in advance. The ideal image is an image suitable for reading the code, and can be exemplified by the image shown on the right side of FIG. A defective image and an ideal image are paired, and a plurality of such pairs are prepared. When learning is performed by the optical information reader 1, 1A, a defective image and an ideal image are input to the optical information reader 1, 1A.

At step SA2, a loss function is calculated to find the difference between the defective image and the ideal image. At step SA3, the parameters of the neural network are updated by reflecting the difference obtained at step SA2.

At step SA4, it is determined whether or not the machine learning completion condition is satisfied. The machine learning completion condition can be set based on the difference obtained in step SA2. For example, if the difference is a predetermined value or less, it can be determined that the machine learning completion condition is satisfied. If it is determined YES in step SA4 and the machine learning completion condition is satisfied, the machine learning is terminated. On the other hand, if it is determined NO in step SA4 and the machine learning completion condition is not satisfied, the process returns to step SA1, another defective image is read, and then the process proceeds to step SA2 to pair with the other defective image. Load another ideal image that is

In this manner, a learned neural network can be generated in advance by performing machine learning on a plurality of defective images and a plurality of ideal images corresponding to the plurality of defective images. By holding the structure and parameters of the trained neural network in the optical information reader 1, 1A, the trained neural network can be constructed in the optical information reader 1, 1A. As in this example, machine learning is performed outside the optical information reader 1, 1A, and the structure and parameters of the neural network obtained as a result are held in the optical information reader 1, 1A. Inference processing by a neural network becomes possible while realizing a reduction in size and weight of the information reading devices 1 and 1A. Incidentally, if the processor 20 with a sufficiently high arithmetic processing capability is installed, there is no problem even if the optical information reader 1, 1A is used for learning.

FIG. 9 is a conceptual diagram of a convolutional neural network (CNN) configured as described above. "Convolution" extracts the features of the input image. This layer consists of convolution operations such as image processing filters. The weight of the filter is called a kernel, and the feature quantity is extracted according to the kernel. A convolutional layer generally has a plurality of different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can take, for example, 3×3, 5×5, and so on.

In "pooling", the reduction process is performed to combine the reactions of each kernel. "Deconvolution" reconstructs an image from feature values using a deconvolution filter. "Unpooling" sharpens the reaction value by performing expansion processing.

The structure and parameters of the trained neural network are stored in the neural network storage unit 40d of the ROM 40 shown in FIG. The neural network structure includes the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, and the like. The parameters of the neural network are the parameters set in step SA3 of the flow chart shown in FIG.

Further, the neural network storage unit 40d can store the structures and parameters of a plurality of neural networks having different numbers of layers or different numbers of image processing filters. In this case, the structures and parameters of a plurality of neural networks and code conditions can be stored in the neural network storage unit 40d in association with each other. For example, the structure and parameters of the neural network for constructing the first neural network are associated with a first code condition corresponding to the first neural network. When making the association, the number of layers or filters of the first neural network can be prespecified optimally by the first code condition, especially the first PPC, so this relationship be associated so that Similarly, the structure and parameters of a second neural network, different from the first neural network, are associated with a second code condition, different from the first code condition.

When the structures and parameters of a plurality of neural networks are stored in the neural network storage unit 40d, the tuning execution unit 20d operates as follows when setting the optical information reader 1, 1A. That is, when setting the code conditions included in the read image generated by the camera 5, the tuning execution unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 40d and operates them. Identify as a neural network to use from time to time. For example, when the tuning execution unit 20d sets the first code condition as the code condition included in the read image, the structure and parameters of the first neural network associated with the first code condition are set to the neural network. Read from the storage unit 40d. Then, the structure and parameters of the first neural network are specified as the neural network to be used during operation of the optical information reader 1, 1A. As a result, inference processing can be executed on the read image with the optimal neural network structure and parameters for the code conditions, so that reading accuracy can be improved.

In learning the neural network, it is also possible to machine-learn a defective image and an ideal image in which the pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of a module can be represented, for example, by the number of pixels (PPC) of one module that constitutes a code, and machine learning of a neural network may be performed only on defective images and ideal images in which the PPC is within a specific range. .

The specific range includes a pixel resolution with a high inference processing effect on the read image and excludes a pixel resolution with little improvement in the inference processing effect on the read image, and includes a code composed of a plurality of modules. can be set on the premise of inference processing for As a result, the processing speed can be improved as a neural network structure specialized for inference processing for images containing codes.

The specific range of the number of pixels can be, for example, 4 PPC or more and 6 PPC or less, but it may be less than 4 PPC or more than 6 PPC. In addition, for example, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale fluctuations from the image used for learning, and it is also possible to optimize the trained neural network structure. .

(inference processing)
As shown in FIG. 5, the processor 20 of the optical information reader 1, 1A is provided with an inference processing section 20j composed of a dedicated core 25. FIG. The inference processing unit 20j reads the structure and parameters of the neural network stored in the neural network storage unit 40d of the ROM 40, and configures the neural network in the optical information reading device 1, 1A with the read structure and parameters of the neural network. . The neural network configured in the optical information reader 1, 1A is the same as the trained neural network learned according to the procedure shown in the flow chart of FIG.

The inference processing unit 20j inputs the read image generated by the camera 5 to the neural network to generate an ideal image corresponding to the read image according to the structure and parameters of the neural network stored in the neural network storage unit 40d. Execute the process. Since the ideal image generated by the inference processing unit 20j is an image generated by the inference processing, it is not necessarily the same as the ideal image used during the learning described above. The generated image is also called an ideal image.

The filter processing unit 20c is configured to perform an image processing filter on the read image generated by the camera 5 before the inference processing unit 20j inputs the read image to the neural network. As a result, appropriate image processing can be performed prior to execution of inference processing by the neural network, resulting in more accurate inference processing.

The inference processing unit 20j may perform inference processing on all the read images to be decoded by the decoding processing unit 20f, or the inference processing unit 20j may perform inference processing only on some of the read images to be decoded by the decoding processing unit 20f. 20j may perform inference processing. For example, the decoding processing unit 20f executes decoding processing on the read image before executing the inference processing, and determines whether or not the decoding is successful. If the decoding is successful, it means that the read image does not require inference processing, and the result is output as it is. On the other hand, if the decoding fails, the inference processing unit 20j executes the inference processing, and the decoding processing unit 20f executes the decoding processing on the generated ideal image.

(Input partial image)
The image input to the neural network by the inference processing unit 20j may be the entire read image generated by the camera 5. 20, the computational load becomes heavy, and the processing speed may be lowered. Specifically, in a fully convolutional network (FCN) type neural network as shown in FIG. Since it is proportional to the square of the size of the code, the amount of computation is also proportional to the square of the size of the code.

On the other hand, it can be said that there are no cases where the code exists in the entire read image generated by the camera 5. During normal operation, the code exists only in a partial area of the read image, and only in that area. Reading accuracy can be improved if inference processing can be executed by a neural network.

Therefore, the image input to the neural network by the inference processing unit 20j can be a part of the read image generated by the camera 5, that is, a partial image containing the code. Specifically, a partial image corresponding to the code candidate region extracted by the extraction unit 20g is stored in the image data storage unit 40a and made accessible to the inference processing unit 20j. By inputting the partial image to the neural network, the inference processing unit 20j performs inference processing on the partial image according to the structure and parameters stored in the neural network storage unit 40d. Then, the decoding processing unit 20f executes decoding processing on the generated ideal image. Further, as will be described later, when decoding processing is executed without executing inference processing on a partial image, the decoding processing unit 20f is made accessible to the image data storage unit 40a. The decode processing unit 20f reads the stored partial image and executes the decoding process.

Since there are almost no cases where the code exists in the entire read image as described above, the size of the partial image corresponding to the code candidate area is smaller than the size of the read image. As a result, the size of the image to be input to the neural network is reduced, so that the computational load on the processor 20 is reduced, and the processing speed is increased. In addition, since the partial image corresponds to an area where the code is highly likely to exist, it is possible to create an image containing the information required for reading, that is, the entire code. Since the image containing the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

The inference processing unit 20j can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The inference processing unit 20j makes the input size of the partial image to be input to the neural network larger than the code size obtained by the tuning execution unit 20d by a predetermined amount.

That is, inputting the partial image to the neural network can reduce the computational load, but on the other hand, if, for example, the code is rotated or the position of the code cannot be detected precisely, the code in the partial image cannot be detected. It is also possible that part of the data is missing and the decoding process fails. In this embodiment, since the input size to the neural network is not the same as the code size obtained by the tuning execution unit 20d, but is larger than the code size by a predetermined amount, it is possible to prevent a part of the code from being missing in the partial image. . By making the code size larger than the code size by a predetermined amount, it is possible to give an upper limit to the input size to the neural network, thereby preventing the computational load from becoming heavy. The "predetermined amount" means, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d. , or 2.0 times or more, or 3.0 times or more.

When the inference processing unit 20j determines the input size of the partial image to be input to the neural network, the number of pixels in one module constituting the code acquired by the tuning execution unit 20d and the number of modules arranged in the vertical or horizontal direction of the code. can be determined based on

Also, the tuning execution unit 20d can set the code conditions as described above. The inference processing unit 20j can also set the size of the read image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the inference processing unit 20j acquires the code size among the code conditions set by the tuning execution unit 20d, a partial image having a size larger than the acquired code size by a predetermined amount is input to the neural network. If the code size is large, the size of the partial image to be input to the neural network will be large, while if the code size is small, the size of the partial image to be input to the neural network will be small. In other words, the size of the partial image input to the neural network can be changed according to the code conditions.

(reduction/enlargement function)
Here, in order to fully capture the features of the code in the convolutional neural network as shown in FIG. 9 and create an ideal image suitable for decoding, it is necessary to extract feature quantities within a range that covers a certain number of modules. be. For example, if one feature value obtained from the neural network is a value extracted from a range of 6×6 modules, and one module is an image captured with 20 pixels, it is aggregated from a range of 120×120 pixels. A neural network must be designed to calculate feature values that In other words, the wider the pixel range to be covered, the deeper the layer of the neural network must be. For example, to aggregate the feature values from the 120×120 pixel range described above, six convolution layers are required. Become.

However, since the code is composed of white modules and black modules arranged randomly, the relationship between pixel values over a wide range is sparse. The treatment effect is hardly improved. Such features are referred to herein as narrow range features.

In addition, the arrangement of fixed module patterns such as the finder pattern included in the code and the rectangular shape of the entire code can be wide-range features. Inference processing can be performed satisfactorily using only narrow-range features without using A wide range feature can be defined as a feature having a fixed shape over a wide range.

Therefore, when learning the neural network shown in FIG. 7, an image in which the PPC of the code is within a specific range can be used. As a result, the processing speed can be improved as a neural network structure specialized for inference processing for images containing codes, but there is a concern that the inference processing effect for read images in which the PPC is outside the specific range will be reduced. When learning the neural network, an image in which the PPC of the code is outside the specific range may be used.

The optical information reading apparatuses 1 and 1A of the present embodiment are provided with functions for reducing/enlarging the read image so that the PPC of the read image is within a specific range. As shown in FIG. 5, the processor 20 includes a reducing section 20h and an enlarging section 20i. The reduction unit 20h is a unit that generates a read image that is reduced so that the pixel resolution of the modules that constitute the code in the read image generated by the camera 5 is within the above-described specific range. is a portion for generating a read image that is enlarged so that the pixel resolution of the modules constituting the code in the read image generated by the camera 5 is within the above-described specific range.

The reducing unit 20h and the enlarging unit 20i, for example, determine whether the PPC of the code in the read image generated by the camera 5 is outside a specific range (4 PPC or more and 6 PPC or less). While shrinking or enlarging is not executed, if it is outside the specified range, shrinking or enlarging is executed. By reducing or enlarging, the PPC of the code in the partial image that the inference processing unit 20j inputs to the neural network falls within a specific range.

The inference processing unit 20j inputs the read image, which has been enlarged or reduced within a specific range, to the neural network configured with the structure and parameters stored in the neural network storage unit 40d, and according to the structure and parameters, Perform inference processing on the read image. Then, the decoding processing unit 20f executes decoding processing on the generated ideal image.

The extracting unit 20g searches for codes in the read image enlarged by the enlarging unit 20i or the read image reduced by the reducing unit 20h, and designates an area including the searched code as a code candidate area where there is a high possibility that the code exists. It may be configured to extract. In this case, the inference processing unit 20j inputs the partial image corresponding to the chord candidate region extracted by the extraction unit 20g to the neural network configured with the structure and parameters stored in the neural network storage unit 40d, thereby Perform inference processing on the read image according to the structure and parameters.

When setting the optical information reading apparatus 1, 1A, the tuning execution unit 20d generates a plurality of read images (enlarged images) with different enlargement ratios and a plurality of read images (reduced images) with different reduction ratios. can be generated. The tuning execution unit 20d inputs a plurality of generated read images (enlarged images or reduced images) to a neural network, executes inference processing for each read image, and executes decoding processing for each generated ideal image. , to obtain the reading margin indicating the readability of the code. The tuning executing unit 20d specifies the enlargement ratio or reduction ratio of the read image with the obtained reading margin higher than a predetermined value as the enlargement ratio or reduction ratio to be used during operation. When the tuning execution unit 20d specifies the enlargement ratio or reduction ratio, the reduction unit 20h and the enlargement unit 20i read at the enlargement ratio or reduction ratio specified by the tuning execution unit 20d during operation of the optical information reader 1 or 1A. Enlarge or reduce the image.

That is, for example, if the specific range has a certain extent, it is conceivable that the reading margin will change even within the specific range if the enlargement ratio or the reduction ratio is changed. Since the enlargement ratio or reduction ratio of the read image during operation can be specified so that the read margin obtained by the tuning execution unit 20d is higher than a predetermined value, the processing speed and reading accuracy are improved.

After obtaining the reading slack for each read image, the tuning executing unit 20d selects the enlargement ratio or reduction ratio of the read image with the highest reading slack among the plurality of obtained reading slacks as the enlargement ratio to be used during operation. Or it can be specified as a reduction ratio. Thereby, the processing speed and reading accuracy can be further improved.

Further, by utilizing the fact that the read image can be reduced and enlarged by the reduction unit 20h and the enlargement unit 20i, the types of parameter sets can be increased. A plurality of parameter sets with different reduction ratios by the reduction unit 20h and a plurality of parameter sets with different enlargement ratios by the expansion unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 40c of the ROM 40. FIG. For example, when a plurality of reduction ratios such as 1/2, 1/4, and 1/8 can be set by the reduction unit 20h, a parameter set with a reduction ratio of 1/2, a parameter set with a reduction ratio of 1/4, and a reduction ratio A parameter set with a rate of 1/8 can be stored in the parameter set storage unit 40c. In addition, when the enlargement ratio by the enlargement unit 20i can be set multiple times, such as 2 times, 4 times, and 8 times, a parameter set for the enlargement ratio of 2 times, a parameter set for the enlargement ratio of 4 times, and a parameter set for the enlargement ratio of 8 times. A parameter set can be stored in the parameter set storage unit 40c. Then, any one parameter set can be applied from the plurality of parameter sets stored in the parameter set storage unit 40c.

(inference processing filter)
The inference processing unit 20j may be a part that executes an inference processing filter that executes inference processing using a neural network. The inference processing filter performs inference processing according to the structure and parameters stored in the neural network storage unit 40d by inputting the read image to the neural network configured with the structure and parameters stored in the neural network storage unit 40d. This filter performs the same function as the inference processing function described above.

When making the inference processing filter executable, a setting unit 20e (shown in FIG. 5) for setting the inference processing filter can be provided. The setting unit 20e is configured to be able to receive the setting of the inference processing filter by the user. The setting unit 20e generates and displays a user interface that allows the user to select one of "application of the inference processing filter" and "non-application of the inference processing filter" when setting the optical information reader 1 or 1A, for example. It can be configured to be displayed on the unit 6 and accept selection by the user. Therefore, the inference processing unit 20j applies the inference processing filter set by the setting unit 20e to the read image to perform the inference processing on the read image.

The setting unit 20e may be included in the tuning execution unit 20d, or may be configured separately from the tuning execution unit 20d. When the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the inference processing filter during tuning. Parameters relating to the inference processing filter can include "application of inference processing filter" and "non-application of inference processing filter". “Applying the inference processing filter” means setting the inference processing filter to be executable, and “non-applying the inference processing filter” means not setting the inference processing filter.

A parameter related to the inference process filter is also information related to the inference process filter, and in this case, a parameter set including information related to the setting of the inference process filter can be stored in the parameter set storage unit 40c. Since the parameters related to the inference processing filter include "application of the inference processing filter" and "non-application of the inference processing filter", the parameter set stored in the parameter set storage unit 40c is set so that the inference processing filter can be executed. and a second parameter set that does not set the inference processing filter. During operation of the optical information reader 1, 1A, one parameter set selected from the first parameter set and the second parameter set is applied. When the first parameter set is applied, the ideal image on which inference processing has been performed by the inference processing filter is subjected to decoding processing by the decoding processing unit 20f, while the second parameter set is applied. In this case, the decode processing unit 20f performs the decoding processing on the read image for which the inference processing is not performed.

The parameter sets stored in the parameter set storage unit 40 c also include items for setting code conditions included in the read image generated by the camera 5 . In the items for setting the code conditions, the PPC, code type, code size, etc. acquired by the tuning execution unit 20d are set. For example, one parameter set includes PPC, code type, and code size as items for setting code conditions, gain of camera 5 as imaging conditions, light amount of illumination unit 4, exposure time, and filter processing unit 20c. An image processing filter type as an item of an image processing filter to be applied, a parameter of the image processing filter, and an application item of an inference processing filter can be included. For each of these items, the values set by the tuning execution unit 20d may be used as they are, or the user may arbitrarily change them.

(Example of tuning process procedure)
An example of the procedure of the tuning process performed by the tuning executing section 20d when setting the optical information reader 1, 1A will be specifically described with reference to the flowchart shown in FIG. At step SB1 after the start of the flow chart shown in FIG. 10, the tuning executing section 20d controls the illumination section 4 and the camera 5 to cause the camera 5 to generate a read image, and the tuning executing section 20d acquires the read image. At this time, the presence or absence and type of an image processing filter to be executed before the decoding process, and the decoding process parameters for applying the inference process by the neural network are set to arbitrary parameters. Next, in step SB2, the tuning executing section 20d causes the decoding processing section 20f to execute decoding processing on the acquired read image.

After the decoding process, the process proceeds to step SB3, and the tuning execution section 20d determines whether or not the decoding process of step SB2 was successful. If NO is determined in step SB3 and the decoding process in step SB2 fails, that is, if the code cannot be read, the process advances to step SB4 to change the decoding process parameter to another parameter. Execute the decoding process again. If the decoding process fails with all the decoding process parameters, this flow is ended and the user is notified.

On the other hand, if YES is determined in step SB3 and the decoding process in step SB2 succeeds, the process proceeds to step SB5, where the tuning execution unit 20d determines code parameters (PPC, code type, code size, etc.). When the tuning execution unit 20d determines the code parameters, the neural network structure and parameters associated with the code parameters and stored in the neural network storage unit 40d are also determined (step SB6). At step SB6, a neural network is configured with the structure and parameters read from the neural network storage unit 40d.

At step SB7, the inference processing unit 20j inputs the read image to the neural network configured at step SB6 to execute inference processing, and the decoding processing unit 20f executes decoding processing on the generated ideal image. do. After that, the process proceeds to step SB8, and the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores it.

At Step SB9, it is determined whether or not the decoding process has been completed with all the decoding process parameters. If NO is determined in step SB9 and execution of the decoding process has not been completed with all the decoding process parameters, the process proceeds to step SB10, changes the decoding process parameters to other parameters, and then proceeds to step SB7.

On the other hand, when it is judged as YES in step SB9 and execution of the decoding process is completed with all the decoding process parameters, the process proceeds to step SB11. At step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest reading margin from all the decoding process parameters, and determines the selected decoding process parameter as the parameter to be applied during operation. Incidentally, in the tuning process, the imaging conditions and the like are also set to appropriate conditions.

(Decoding processing procedure before determination of reduction ratio and enlargement ratio)
Next, an example of the decoding processing procedure before determining the reduction rate and the enlargement rate will be specifically described with reference to the flowchart shown in FIG. The processing specified in the flowchart shown in FIG. 11 can be executed in step SB2 of the flowchart shown in FIG.

At step SC1 after the start of the flowchart shown in FIG. 11, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter executed by the filter processing unit 20c. After that, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the read image.

Next, at step SC3, an image pyramid is created. The image pyramid consists of an original read image, an image reduced to 1/2 of the original read image, an image reduced to 1/4 of the original read image, an image reduced to 1/8 of the original read image, and so on. It is configured. Reduction of the read image is performed by the reduction unit 20h. The image pyramid consists of an original read image, an image obtained by enlarging the original read image by two times, an image obtained by enlarging the original read image by four times, an image obtained by enlarging the original read image by eight times, and so on. You can also Enlargement of the read image is performed by the enlarging section 20i. One of the reduced image and the enlarged image may be omitted.

After creating the image pyramid, the process advances to step SC4 to select an arbitrary read image from among the plurality of read images forming the image pyramid. This selected image may include the original scanned image that has not been scaled down or scaled up. At step SC5, the extraction unit 20g extracts a code candidate area in which a code is likely to exist from the read image selected at step SC4. After that, in step SC6, the inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted in step SC5 to the neural network and executes inference processing. The size of the partial image input to the neural network is determined by the code conditions described above.

After executing the inference processing and generating the ideal image, the process proceeds to step SC7 to execute the contour of the code and the positioning processing of the modules constituting the code. After the positioning process, the process proceeds to step SC8 to determine whether each module constituting the code is white or black. After determining black and white, the process proceeds to step SC9, and the decoding processing unit 20f executes decoding processing on the generated ideal image. In decoding processing, for example, a method of restoring a character string from a 0-1 matrix of modules can be adopted.

Thereafter, the process proceeds to step SC10, and the tuning execution unit 20d determines whether or not the decoding process of step SC9 was successful. If NO is determined in step SC10 and the decoding process in step SC9 fails, the process proceeds to step SC11 to determine whether or not all the read images forming the image pyramid have been selected. If the determination in step SC11 is NO and all the read images forming the image pyramid have not been selected, the process proceeds to step SC4 to select another reduced read image or another enlarged image forming the image pyramid. The scanned image is selected, and the process proceeds to step SC5.

On the other hand, if YES is determined in step SC11 and all the read images forming the image pyramid have been selected, this flow is terminated and the conditions for successful decoding are stored.

(Decode processing procedure after determination of reduction ratio and enlargement ratio)
Next, an example of the decoding processing procedure after determining the reduction rate and the enlargement rate will be specifically described with reference to the flowchart shown in FIG. The processing specified in the flowchart shown in FIG. 12 can be executed during operation of step SB7 of the flowchart shown in FIG. 10 and the optical information reader 1, 1A.

At step SD1 after the start of the flow chart shown in FIG. 12, the filter processing section 20c executes the image processing filter selected at step SC1 of the flow chart shown in FIG. 11 on the read image. At this time, when the filter processing unit 20c executes the averaging filter on the read image as shown in the upper part of FIG. 13, the image after execution of the image processing filter as shown in the lower part of FIG. 13 is obtained. .

After that, the process advances to step SD2 to execute reduction/enlargement processing of the read image as necessary. Specifically, when the PPC of the code in the read image after execution of the image processing filter is within a specific range, reduction or enlargement is not executed, and when it is outside the specific range, reduction or enlargement is not executed. Execute and make sure the PPC is within a certain range. The upper part of FIG. 14 shows an example of a read image after reduction processing.

Next, in step SD3, the extracting unit 20g extracts code candidate areas in which codes are likely to exist from the read image reduced or enlarged in step SD2. The lower part of FIG. 14 shows an example of the chord candidate area extraction image. If the image is not reduced or enlarged in step SD2, the original read image is extracted in step SD3.

Then, in step SD4, the inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 to the neural network and executes inference processing. FIG. 15 shows an example of a read image before execution of inference processing and an example of an ideal image after execution of inference processing. After the ideal image is generated, this flow ends through steps SD5 to SD7. Steps SD5-SD7 are the same as steps SC7-SC9 in the flow chart shown in FIG.

(Two kinds of decoding processing)
As described above, in the present embodiment, the decode processing unit 20f decodes the read image generated by the camera 5 without the inference processing unit 20j executing the inference processing on the read image. In some cases, the inference processing unit 20j performs inference processing on the read image generated by the inference processing unit 20j to generate an ideal image, and the decoding processing unit 20f performs decoding processing on the generated ideal image. The former case, i.e., decoding the read image, is referred to as the first decoding process, and the latter case, i.e., decoding the ideal image generated by the inference process, is referred to as the second decoding process. .

The first decoding process and the second decoding process are both executed by the decoding processing section 20f, but may be executed by different decoding processing sections. That is, since the processor 20 has the dedicated core 25 and the first to fourth general-purpose cores 21 to 24, the inference processing by the inference processing unit 20j configured by the dedicated core 25 and the first to fourth general-purpose The first decoding processing by the decoding processing section 20f constituted by at least one of the cores 21 to 24 can be executed in parallel at high speed. Furthermore, the processor 20 can execute the second decoding process by the decoding processing section 20f after the inference processing by the inference processing section 20j is completed. A specific description will be given below.

(first example)
FIG. 16 is a flow chart showing a first example of parallel execution of the first decoding process and the inference process. At step SE1 after the start of this flow chart, the processor 20 acquires a read image. After that, in step SE2, the decoding processing section 20f executes the first decoding process, that is, the decoding process for the read image. After the first decoding process, the process proceeds to step SE3 to determine whether or not the reading is successful. If the reading is not successful, the process returns to step SE2 and executes the first decoding process again. If reading is not successful even after a predetermined number of times (time) has elapsed, a time-out occurs and the decoding process is terminated. On the other hand, if it is determined in step SE3 that the reading has succeeded, the decoding process is terminated.

In parallel with proceeding from step SE1 to step SE2, the process advances to step SE4. At step SE4, the inference processing unit 20j performs inference processing on the read image generated by the camera 5 to generate an ideal image. The generation of this ideal image and the first decoding process of step SE2 are executed in parallel, and are different read images. After the ideal image is generated in step SE4, the process proceeds to step SE5, where the second decoding process, that is, the decoding process of the ideal image generated in step SE4 is performed by the decoding processing unit 20f. After the second decoding process, the process proceeds to step SE6 to determine whether or not the reading is successful. If the reading is not successful, the process returns to step SE5 to execute the second decoding process again. If reading is not successful even after a predetermined number of times (time) has elapsed, a time-out occurs and the decoding process is terminated. On the other hand, if it is determined in step SE6 that the reading has succeeded, the decoding process is terminated.

The optical information reader 1 may be provided with a function to cancel steps SE4 to SE6. Steps SE4 to SE6 are steps for executing the inference processing and the second decoding processing. Since the inference processing in particular takes time, canceling steps SE4 to SE6 can speed up the processing. As a method of canceling steps SE4 to SE6, for example, there is a method by user's operation. For example, one of the parameters stored in the parameter set storage unit 40c may include a selection parameter as to whether or not to execute inference processing. By changing this selection parameter by the user, it is possible to switch between a mode in which inference processing is executed and a mode in which inference processing is not executed. The selection parameter may be, for example, a selection parameter for "speed priority mode". When the 'speed priority mode' is selected by the user, the machine is operated in a mode in which inference processing is not executed, and when the 'speed priority mode' is not selected, it is operated in a mode in which inference processing is executed. It should be noted that the inference processing may be automatically canceled when it is determined that the inference processing is unnecessary based on the information obtained by tuning.

FIG. 17A is a sequence diagram showing sequence example 1 of the first example. As shown in this figure, while at least one general-purpose core among the first to fourth general-purpose cores 21 to 24 is executing the first decoding process, the dedicated core 25 executes the inference process. The general-purpose core executes the second decoding process when the first decoding process ends and the inference process ends.

FIG. 17B is a sequence diagram showing sequence example 2 of the first example. In this example, the first general-purpose core 21 executes the first decoding process and the second decoding process in parallel in separate threads. Specifically, the dedicated core 25 executes the inference processing while the first general-purpose core 21 executes the first decoding processing. When the inference processing ends, the first general-purpose core 21 executes the second decoding processing in a different thread from the first decoding processing without waiting for the end of the first decoding processing.

FIG. 17C is a sequence diagram showing sequence example 3 of the first example. In this example, the first general-purpose core 21 executes the first decoding process, and the second general-purpose core 22 executes the second decoding process. Specifically, the dedicated core 25 executes the inference processing while the first general-purpose core 21 executes the first decoding processing. When the inference processing ends, the second general-purpose core 22 executes the second decoding processing without waiting for the end of the first decoding processing. As a result, the first decoding process and the second decoding process can be executed by different general-purpose cores, so that the processing speed is further increased.

(Second example)
FIG. 18 is a flowchart showing a second example in which the first decoding process and the inference process are executed in parallel. This second example shows a case where a plurality of code candidate regions are extracted. In step SF1 after the start of this flow chart, the extracting unit 20g executes an extracting process of extracting code candidate areas in which codes are likely to exist from the read image generated by the camera 5. FIG. As a result, a candidate region group consisting of a plurality of code candidate regions is extracted. A plurality of code candidate areas are extracted from one read image, and in fact, there may be areas where no code is included.

After step SF1, the processing is divided into the processing by the first decoding sequence processing section and the processing by the second decoding sequence processing section. In step SF2, the first decoding sequence processing section selects one code candidate area from the code candidate area group. At this time, an order is given to the regions with the highest probability that the code exists in descending order, and the region with the highest probability that the code exists is preferentially selected. At step SF3, the decoding processing unit 20f performs the first decoding process on the code candidate area selected at step SF2. After the first decoding process, the process proceeds to step SF4 to determine whether or not the reading is successful. If it is determined in step SF4 that the reading has succeeded, the decoding process is terminated.

On the other hand, if the reading is not successful, the process returns to step SF2, and only one code candidate area for which the first decoding process has not been executed is selected from the code candidate area group according to the above order. After that, the process proceeds to step SF3, and the decoding processing unit 20f executes the first decoding process on the code candidate area selected in step SF2. If reading is not successful again, the process returns to step SF2. If reading is not successful, this is repeated and the decoding processing unit 20f executes the first decoding process on all code candidate areas. Even if the reading is not successful, a time-out occurs and the decoding process is terminated.

On the other hand, in step SF5, the second decoding sequence processing section selects one code candidate area from the code candidate area group, as in step SF2. In step SF6, the inference processing unit 20j performs inference processing on the code candidate region selected in step SF5 to generate an ideal image. The generation of this ideal image and the first decoding process in step SF3 are executed in parallel. After the ideal image is generated in step SF6, the process proceeds to step SF7, where the second decoding process, that is, the decoding process of the ideal image generated in step SF6 is performed by the decoding processing unit 20f. After the second decoding process, the process proceeds to step SF8 to determine whether or not the reading is successful. If it is determined in step SF8 that the reading has succeeded, the decoding process is terminated.

On the other hand, if the reading is not successful, the process returns to step SF5, and only one code candidate area for which inference processing has not been executed is selected from the code candidate area group according to the above order. Then, after proceeding to step SF6 to generate an ideal image, the process proceeds to step SF7. In step SF7, a second decoding process is executed. If reading is not successful again, the process returns to step SF5. If the reading is not successful, this is repeated to generate the ideal image and execute the second decoding process for all code candidate areas. The decoding process is terminated even if none of the readings are successful.

FIG. 19 is a sequence diagram showing a sequence example of the second example. In this example, both the first general-purpose core 21 and the second general-purpose core 22 execute extraction processing. As a result, even if there are a plurality of code candidate regions, the extraction process can be completed quickly. In this example, a case where chord candidate regions 1 to 8 are extracted will be described.

After the extraction process, the first general-purpose core 21 executes the first decoding process on the code candidate area 1 (first decoding process 1). Also, the second general-purpose core 22 executes the first decoding process for the code candidate area 2 (first decoding process 2). The first decoding process 1 and the first decoding process 2 are executed in parallel. Also, the dedicated core 25 executes inference processing on the code candidate region 5 while the first decoding processing 1 and the first decoding processing 2 are being executed (inference processing 1).

After completing the first decoding process 1, the first general-purpose core 21 executes the first decoding process on the code candidate area 3 (first decoding process 3). Further, when the first decoding process 2 ends, the second general-purpose core 22 executes the first decoding process on the code candidate area 4 (first decoding process 4). When the dedicated core 25 completes the inference processing 1 during execution of the first decoding processing 3 and the first decoding processing 4, the dedicated core 25 performs inference processing on the code candidate region 7 (inference processing 2).

When the first decoding process 3 ends, the first general-purpose core 21 performs the first decoding on the ideal image (image corresponding to the code candidate region 5) generated by the inference process 1, because the inference process 1 has already ended. 2 decoding processing is executed (second decoding processing 5). On the other hand, when the first decoding process 4 ends, the second general-purpose core 22 executes the first decoding process on the code candidate region 6 because the inference process 2 has not ended (first decoding process 6).

The dedicated core 25 ends the inference processing 2 while the second decoding processing 5 and the first decoding processing 6 are being executed. When the first decoding process 5 ends, the first general-purpose core 21 performs the first decoding on the ideal image (image corresponding to the code candidate region 7) generated by the inference process 2, because the inference process 2 has already ended. 2 decoding processing is executed (second decoding processing 7). On the other hand, after completing the first decoding process 6, the second general-purpose core 22 executes the first decoding process on the code candidate area 8 (first decoding process 8). In this way, when the ideal image is generated by the inference processing by the dedicated core 25, the first general-purpose core 21 and the second general-purpose core 22 perform the next first decoding when the currently executing first decoding processing is completed. It is configured to execute the second decoding process with priority over the next first decoding process without executing the process.

Among the plurality of general-purpose cores 21 to 24, for example, the first general-purpose core 21 executes the extraction process, the second general-purpose core 22 executes the first decoding process, and the third general-purpose core 23 executes the second decoding process. may Further, the fourth core 24 may perform imaging control of the camera 5 and control of the lighting section 4 . By allocating a plurality of types of processing to the general-purpose cores 21 to 24 in this manner, the processing speed can be further increased.

(Third example)
FIG. 20 is a flowchart showing a third example of parallel execution of the first decoding process and the inference process. In this third example, when a plurality of code candidate regions are extracted, after the step of determining whether or not the code candidate region has been decoded, if it is determined that the decoding has not been completed, the first code candidate region is extracted. I am trying to do the decoding process.

In this flowchart, after starting, the processing is divided into the processing by the first decoding sequence processing section and the processing by the second decoding sequence processing section. In step SG1 of the first decoding sequence processing section, the extraction section 20g executes a first extraction process of extracting a code candidate area in which a code is likely to exist from the read image generated by the camera 5. FIG. On the other hand, in step SG7 of the second decoding sequence processing section, the extraction section 20g executes a second extraction process of extracting a code candidate area in which a code is likely to exist from the read image generated by the camera 5. FIG.

The first extraction process of step SG1 and the second extraction process of step SG7 have different extraction conditions. Processing is executed (step SG4), and inference processing is executed for the code candidate regions extracted in the second extraction processing (step SG14). The reason why the extraction conditions are changed between the first extraction process and the second extraction process is that the first decoding sequence processing unit can quickly capture and read an image that is easy to read. This is because, in the second decoding sequence processing section, there is a high possibility that the first decoding sequence processing section will fail, and the aim is to accurately capture and accurately read an image that is difficult to read.

For example, the first extraction process extracts the code candidate region under a first predetermined condition that enables high-speed extraction, while the second extraction process extracts the code candidate region with higher accuracy than the first predetermined condition. is extracted under a second predetermined condition that enables In other words, since the accuracy of extraction is lower under the first predetermined condition than under the second predetermined condition, high-speed extraction processing is possible. is a condition that enables

As specific condition settings, the extraction threshold for the first predetermined condition is set to be lower than the extraction threshold for the second predetermined condition, and the number of extraction conditions constituting the first predetermined condition is set to be lower than the second predetermined condition. For example, the number of extraction conditions is set to be less than the number of constituent extraction conditions. Alternatively, the second predetermined condition may be configured by combining another extraction condition with the extraction condition that configures the first predetermined condition.

After high-speed extraction processing is executed in step SG1, the process proceeds to step SG2 to select one chord candidate region from the chord candidate region group. After that, the process proceeds to step SG3, and it is determined whether or not the code candidate area selected in step SG2 is already decoded. If YES is determined in step SG3 and the code candidate area has already been decoded, the process returns to step SG2, another code candidate area is selected from the code candidate area group, and the process proceeds to step SG3.

If it is determined YES in step SG3, it means that the code candidate area selected in step SG2 has not been decoded yet. The processing unit 20f executes the first decoding process. After the first decoding process, the process proceeds to step SG5 to determine whether or not the reading is successful. If it is determined in step SG5 that the reading has succeeded, the process proceeds to step SG6 to determine whether or not the reading has been completed. If the reading is completed in the first decoding process in step SG5, this flow is terminated. If the reading is not completed, the process returns to step SG2 to select another code candidate area from the code candidate area group. is selected and the process proceeds to step SG3. Also, if the reading is not successful in step SG5, the process returns to step SG2.

On the other hand, when proceeding to step SG8 via step SG7 of the second decoding sequence processing section, one code candidate area is selected from the code candidate area group. Steps SG9 to SG12 are steps similar to steps SG3 to SG6.

Further, when proceeding to step SG13 through step SG7, one chord candidate area is selected from the chord candidate area group. In step SG14, the inference processing unit 20j performs inference processing on the code candidate region selected in step SG13 to generate an ideal image. The generation of this ideal image and the first decoding process of step SG4 and/or step SG10 are executed in parallel. After generating the ideal image in step SG14, the process proceeds to step SG15 to execute the second decoding process. After the second decoding process, the process proceeds to step SG16 to determine whether or not the reading is successful. If it is determined in step SG16 that the reading has succeeded, it is determined in step SG17 whether or not the reading has been completed. If reading has been completed by the first decoding processing in steps SG4 and SG10 or by the second decoding processing in step SG15, this flow ends. Another code candidate area is selected from the area group, and the process proceeds to step SG14. Also, if the reading is not successful in step SG16, the process returns to step SG13.

FIG. 21 is a sequence diagram showing a sequence example of the third example. In this example, the first general-purpose core 21 executes the relatively high-speed first extraction process, and the second general-purpose core 22 executes the relatively high-precision second extraction process. It is longer than the time for one extraction process. Also, in this example, the case where code candidate regions 1 to 9 are extracted by the first extraction process and the second extraction process will be described.

After the first extraction process, the first general-purpose core 21 executes the first decoding process on the code candidate region 1 (first decoding process 1). After completing the first decoding process 1, the first general-purpose core 21 executes the first decoding process on the code candidate area 2 (first decoding process 2). After the second extraction process, the second general-purpose core 22 executes the first decoding process on the code candidate area 3 (first decoding process 3). The first decoding process 2 and the first decoding process 3 are executed in parallel. After the second extraction process, the dedicated core 25 extracts the code candidate area 6 (the code candidate area extracted in the second extraction process) while the first decoding process 2 and the first decoding process 3 are being executed. Inference processing is executed with respect to (inference processing 1).

Further, when the first decoding process 2 ends, the first general-purpose core 21 executes the first decoding process on the code candidate area 4 (first decoding process 4). After completing the first decoding process 3, the second general-purpose core 22 executes the first decoding process on the code candidate area 5 (first decoding process 5). When the dedicated core 25 completes the inference processing 1 during the execution of the first decoding processing 4 and the first decoding processing 5, the dedicated core 25 performs the inference processing on the code candidate region 8 (inference processing 2).

When the first decoding process 4 ends, the first general-purpose core 21 performs the first decoding on the ideal image (image corresponding to the code candidate region 6) generated by the inference process 1, because the inference process 1 has already ended. 2 decoding processing is executed (second decoding processing 6). On the other hand, when the first decoding process 5 ends, the second general-purpose core 22 executes the first decoding process on the code candidate region 7 because the inference process 2 has not ended (first decoding process 7).

The dedicated core 25 ends the inference processing 2 while the second decoding processing 6 and the first decoding processing 7 are being executed. When the second decoding process 6 ends, the first general-purpose core 21 performs the first decoding on the ideal image (image corresponding to the code candidate region 8) generated by the inference process 2, because the inference process 2 has already ended. 2 decoding processing is executed (second decoding processing 8). On the other hand, after completing the first decoding process 7, the second general-purpose core 22 executes the first decoding process on the code candidate area 9 (first decoding process 9).

(Description of task sequence)
FIG. 22 is a task sequence diagram when reading is successful in the first decoding process. This diagram shows a control task, a first decoding task, a second decoding task, and an inference processing task. When the control task issues a read execution instruction 1 to the first decoding task, the first decoding process is started. do. When reading is successful in the first decoding process, read success notification 2 is issued to the control task. When the control task receives the read success notification 2, it issues a processing end instruction 3 to the first decoding processing task, and the first decoding processing task issues a processing end notification 4 to the control task.

Also, when the control task issues a read execution instruction 5 to the second decoding task, the second decoding task issues an inference start instruction 6 to the inference process task. When the inference processing task is completed, the inference processing task issues an inference completion notification 7 to the second decoding processing task. After that, the second decode processing task issues the next inference start instruction 8 to the inference processing task. After that, since the processing end notification 4 is issued to the control task, the control task issues the processing end instruction 9 to the second decoding processing task. The second decode processing task issues an inference end instruction 10 to the inference processing task, and the inference processing task issues an inference end notification 11 to the second decode processing task. The second decoding task then issues a process end notification 12 to the control task to end the reading. In other words, the processor 20 is configured to end the inference processing in the middle even if the inference processing by the inference processing unit 20j is not completed when decoding is successful in the first decoding processing. This allows the next inference process to start early.

FIG. 23 is a task sequence diagram when reading is successful in the second decoding process. The control task issues a read execute command 1 to the first decoding task and a read execute command 2 to the second decode processing task. The second decode processing task issues an inference start instruction 3 to the inference processing task. When the inference processing task is completed, the inference processing task issues an inference completion notification 4 to the second decoding processing task. After that, the second decode processing task issues the next inference start instruction 5 to the inference processing task.

The second decoding task receives the inference completion notification 4 and starts the second decoding process. When the second decoding process succeeds in reading and the second decoding task issues a reading success notification 6 to the control task, the control task issues a process end instruction 7 to the second decoding process task. At this time, the control task also issues a processing end instruction 8 to the first decoding processing task.

When the second decode processing task receives the processing end instruction 7, it issues an inference end instruction 9 to the inference processing task, and the inference processing task issues an inference end notification 10 to the second decode processing task. Also, the first decoding processing task issues a processing end notification 11 to the control task. Also, the second decoding processing task issues a processing end notification 12 to the control task. In this example, the inference processing by the inference processing unit 20j can be terminated when decoding is successful in the second decoding processing.

FIG. 24 is a task sequence diagram when reading is successful in the first decoding process and the second decoding process. The control task issues a read execute command 1 to the first decoding task and a read execute command 2 to the second decode processing task. The second decode processing task issues an inference start instruction 3 to the inference processing task. The first decoding task starts the first decoding process. When reading is successful in the first decoding process and the first decoding process task issues a read success notification 4 to the control task, the control task issues a process end instruction 5 to the second decoding process task. The second decoding processing task issues a processing end notification 6 to the control task.

On the other hand, the inference processing task issues an inference completion notification 7 to the second decoding processing task when the inference processing is completed. After that, the second decode processing task issues the next inference start instruction 8 to the inference processing task. The second decoding task receives the inference completion notification 7 and starts the second decoding process. When the second decoding process succeeds in reading and the second decoding task issues a reading success notification 9 to the control task, the control task issues a process end instruction 10 to the second decoding task.

When the second decode processing task receives the processing end instruction 10, it issues an inference end instruction 11 to the inference processing task, and the inference processing task issues an inference end notification 12 to the second decode processing task. After receiving the inference end notification 12, the second decoding processing task issues a processing end notification 13 to the control task. In this example, if the decoding is successful in both the first decoding process and the second decoding process, the inference process by the inference processor 20j can be terminated.

FIG. 25 is a task sequence diagram when reading is successful twice in the second decoding process. The control task issues a read execute command 1 to the first decoding task and a read execute command 2 to the second decode processing task. The second decode processing task issues an inference start instruction 3 to the inference processing task. When the inference processing task is completed, the inference processing task issues an inference completion notification 4 to the second decoding processing task. After that, the second decode processing task issues the next inference start instruction 5 to the inference processing task. The second decoding task receives the inference completion notification 4 and starts the second decoding process. The second decoding process succeeds in reading, and the second decoding task issues a read success notification 6 to the control task.

Thereafter, the inference processing task completes the second inference processing and issues a second inference completion notification 7 to the second decoding processing task. The second decoding task receives the second inference completion notification 7 and starts the second second decoding process. The second decoding process also succeeds in reading, and the second decoding task issues a read success notification 8 to the control task. During this time, the first decoding task does not successfully read, so the first decoding task does not issue a read success notification.

The control task issues a processing end instruction 9 to the second decoding processing task. When the second decode processing task receives the processing end instruction 9, it issues an inference end instruction 10 to the inference processing task, and the inference processing task issues an inference end notification 11 to the second decode processing task. After receiving the inference end notification 11, the second decoding processing task issues a processing end notification 12 to the control task. Further, when the control task issues a process end instruction 13 to the first decoding task, the first decoding process task issues a process end notification 14 to the control task. In this example, if decoding is successful in both the first second decoding process and the second second decoding process, the inference process by the inference processor 20j can be terminated.

(Continuous shooting with camera)
As shown in FIG. 1, when the optical information reader 1 is used at a site where a plurality of workpieces W are continuously delivered, the code passes in front of the camera 5 in sequence at certain time intervals. For this reason, it is necessary for the camera 5 to continuously capture images at predetermined short intervals to sequentially obtain the read images, and for the optical information reader 1 to finish reading within a predetermined time.

FIG. 26 is a reading sequence diagram when photographing and decoding are repeated for a certain period of time after receiving a reading start trigger signal. The time for one decoding process is set in advance, and this time is set as the decoding timeout time. In addition, the total reading limit time is set in advance separately from the decode timeout time, and this is set as the tact timeout time. In addition, when you want to forcibly end the decoding process, the timing when the decoding process is successful (timing 1), the timing when the decoding timeout time has passed after the reading is started (timing 2), and the takt timeout after the reading is started (timing 2). There are three points of time (timing 3) when time has passed. For example, if the response speed to a forced termination instruction of the reading process is set to about 10 msec, it may be preferable to allow the decoding process to respond within about the same time.

On the other hand, the inference processing is executed by the dedicated core 25 and may require a processing time of several tens to several hundred milliseconds. It may be preferable to have either or both functions to initiate processing.

FIG. 27 is a sequence diagram of decoding processing during continuous shooting. This decode processing sequence diagram is similar to the sequence diagram shown in FIG. 19, but shows an example in which the inference processing is interrupted when one of the above timings 1 to 3 arrives during the inference processing.

(Action and effect of the embodiment)
As described above, according to this embodiment, when the optical information reader 1 or 1A is in operation, inputting the read image generated by the camera 5 to the neural network generates an ideal image through inference processing. Since decoding processing is executed on the ideal image after this inference processing, reading accuracy is enhanced.

Further, when the read image is suitable for reading the code, the read image generated by the camera 5 is decoded without inference processing. Since the above-mentioned inference processing and decoding processing for decoding without inference processing are executed in parallel, during execution of inference processing for a read image that requires inference processing, another read image that does not require inference processing It is possible to perform decoding processing, or to perform inference processing for another read image that requires inference processing while decoding processing is being performed for a read image that does not require inference processing. This shortens the processing time.

(Other embodiments)
FIG. 28 is a diagram for explaining an example of a processing procedure during operation of the optical information reader 1 according to another embodiment. In this embodiment, the processor 20 includes a preprocessing unit 200, a postprocessing unit 201, a first decoding processing unit 202, and a second decoding processing unit 203 in addition to the inference processing unit (image restoration unit) 20j. is configured. The preprocessing section 200 is a section that executes a noise removal filter, a contrast correction filter, an averaging filter, etc. on the read image generated by the camera 5, similarly to the filter processing section 20c.

The first decoding processing unit 202 is a part that performs decoding processing on the read image generated by the camera 5, and the decoding processing by the first decoding processing unit 202 is also called first decoding processing. When the processor 20 has a plurality of cores (N cores), the first decoding processing unit 202 is composed of the cores 0 to i. The first decoding processing unit 202 may be configured with one core.

The second decoding processing unit 203 is a part that performs decoding processing on the restored image generated by the inference processing unit 20j, and the decoding processing by the second decoding processing unit 203 is also called second decoding processing. When the processor 20 has N cores, the second decoding processing unit 203 is composed of the cores i+1 to N−1. The second decoding processing unit 203 may be composed of one core.

As shown in FIG. 29, when the first decoding processing unit 202 is composed of core 0 to core i, each of core 0 to core i has a code in the read image generated by the camera 5. After extracting the code candidate regions with high probability, the code outline and positioning of the modules that compose the code are performed, and then each module that composes the code is white or black. is determined and decode processing is executed. A processing result is output to the post-processing unit 201 .

On the other hand, as shown in FIG. 29, when the second decoding processing unit 203 is composed of core i+1 to core N−1, each of core i+1 to core N−1 may detect the presence of a code from the read image. A code candidate region with a high value is extracted as a repair code region, and a trigger signal is sent to the inference processing unit 20j so as to execute inference processing. In FIG. 29, it is described that one task is executed by two cores, but one task may be executed by any one or more cores.

Specifically, when the cores N-2 and N-1 constituting the second decoding processing unit 203 extract the repair code candidate region, they output a trigger signal (trigger 1) to the inference processing unit 20j, It causes the inference processing unit 20j to execute inference processing. When the inference processing unit 20j receives the trigger signal (trigger 1) sent from the core N-2 and the core N-1, the part corresponding to the repair code candidate region extracted by the core N-2 and the core N-1 An image is input to a neural network, and an inference process is executed to generate a restored image obtained by restoring the partial image (image restoration 1). The restored image generated in image restoration 1 is sent to core N-2 and core N-1. Core N-2 and core N-1 determine the grid position indicating each cell position of the code based on the restored image generated in the image restoration 1, and execute decoding processing of the restored image based on the determined grid position. do.

On the other hand, when the cores i+1 and i+2 constituting the second decoding processing unit 203 extract the repair code candidate region, they output a trigger signal (trigger 2) to the inference processing unit 20j, and the inference processing unit 20j Let the inference process run. When the inference processing unit 20j receives the trigger signal (trigger 2) sent from the core i+1 and the core i+2, it inputs a partial image corresponding to the repair code candidate region extracted by the core i+1 and the core i+2 to the neural network, An inference process is executed to generate a restored image obtained by restoring the partial image (image restoration 2). The restored image generated in image restoration 2 is sent to core i+1 and core i+2. Core i+1 and core i+2 determine grid positions indicating the positions of each cell of the code based on the restored image generated in image restoration 2, and decode the restored image based on the determined grid positions.

Thereafter, in the same manner, image restoration 3 is executed by trigger signals (trigger 3) sent from core N-2 and core N-1, and image restoration 3 is performed by trigger signals (trigger 4) sent from core i+1 and core i+2. Restoration 4 is executed, image restoration 5 is executed by a trigger signal (trigger 5) sent from core N-2 and core N-1, and image restoration is performed by a trigger signal (trigger 6) sent from core i+1 and core i+2. Restoration 6 is executed, and image restoration 7 is executed by a trigger signal (trigger 7) sent from core N-2 and core N-1.

As shown in this time chart, the inference processing by the inference processing unit 20j and the first decoding processing by the first decoding processing unit 202 are executed in parallel. For example, while the first decoding processing unit 202 continuously performs code region extraction, grid positioning, and decoding processing, the inference processing by the inference processing unit 20j is executed once or multiple times. Further, while the first decoding processing unit 202 continues the code region extraction, grid positioning, and decoding processing, the second decoding processing unit 203 extracts the recovery code region, outputs the trigger signal, grid positioning, and performs the second decoding. Processing is performed. In this way, the inference processing unit 20j can specialize in restoration of the partial image corresponding to the restoration code candidate area, so that the inference processing speed can be increased.

At least part of the period during which the second decoding processing section executes the second decoding processing has a rest period during which the inference processing is not executed by the inference processing section 20j. That is, as shown in FIG. 29, after the image restoration 1 of the inference processing unit 20j is completed, the second decoding processing unit 203 starts the second decoding processing. A predetermined period is provided before the start, and this predetermined period is a rest period during which the inference processing unit 20j does not perform image restoration. This idle period is a period in which the inference processing unit 20j does not perform inference processing. Note that the grid positioning processing time may be long or short depending on the restoration image to be processed. Although FIG. 29 shows an example in which the processing of each core is alternately executed, the processing of each core is not necessarily alternately executed depending on the processing time of grid positioning. In other words, trigger generation and grid positioning may be executed in order from the cores that are idle for processing.

As described above, the setting unit 20e is configured so that the user can select one of "application of the image restoration filter" and "non-application of the image restoration filter". Applying the image restoration filter means performing image restoration, and not applying the image restoration filter means not performing image restoration. It also corresponds to a part for setting whether or not the processing unit 203 executes the second decoding process. Since this setting is performed by the user, the setting unit 20e is configured to be able to receive the operation by the user. The setting form is not limited to the application or non-application of the image restoration filter, and may be, for example, the application or non-application of inference processing (restoration processing) or the application or non-application of processing by AI. The operation by the user can be, for example, the operation of a button or the like displayed on the user interface screen.

The optical information reading device 1 is operated after performing a tuning process for predetermining imaging conditions such as exposure time and gain, and settings such as an image processing filter. For example, when the workpiece W is a label, the code is clearly printed, so if appropriate conditions are set by the tuning process, a high reading rate can be achieved. However, in the case of workpieces W such as metal and resin, although optimal conditions are set for the tuning process, it may become difficult to read due to changes in the read image due to weak ambient light, etc. Due to the design, there may be cases where reading is not possible due to ultra-low contrast.

By executing the image restoration algorithm executed by the inference processing unit 20j of this example in parallel with the case where the decoding process is performed without image restoration, it is possible to read the work W such as metal or resin while maintaining the conventional reading performance. Reading performance can be improved and reading stability can be enhanced. In addition, the work W that cannot be read by conventional optical design and algorithms can be read by applying the image restoration algorithm.

To give a specific example, in the process of reading a code printed on a sticker attached to a box in a certain factory, if there are no restrictions on the installation conditions of the optical information reader 1 and optimal installation is possible, Since stable operation is possible even with conventional reading performance, priority can be given to takt time by not applying the image restoration filter.

Also, in a certain factory, there is a process to read codes clearly printed on metal, and although the optimum conditions are determined in the tuning process at the time of reading setting, when actually operating, scratches on the metal surface, In some cases, reading may become unstable due to variations in reading position/angle, changes in the brightness of the surrounding environment, etc. In such a case, by applying an image restoration filter to improve the image quality, it becomes resistant to the above-described variations in conditions, and the reading performance is improved.

Also, some workpieces cannot be read by the conventional algorithm, but in such cases, applying an image restoration filter may make it possible to read them. For example, even if the code is severely damaged or the contrast is not clear, by applying an image restoration filter, the contrast and flaws can be restored and the code can be read.

Since the second decoding process executed when the image restoration filter is applied executes the decoding process on the restored image, it takes longer time than the first decoding process which executes the decoding process of the normal read image. is often long. Therefore, the first decoding processing section 202 can execute the first decoding processing in a shorter time than the time required for the second decoding processing by the second decoding processing section 203 . In this example, on the premise of this, the setting unit 20e can change the decode timeout time, which is the time limit for the decoding process.

Specifically, when the setting unit 20e is set to execute the second decoding process, the setting unit 20e is configured to be able to set a decoding timeout time longer than the time required to execute the second decoding process. When it is set that the second decoding process is not to be executed, a decoding timeout period shorter than the time required to execute the second decoding process can be set. As a result, during operation, when executing the second decoding process, it is possible to set a decoding timeout time longer than the time normally required for executing the second decoding process. can run. On the other hand, if it is set not to execute the second decoding process, generally only the first decoding process that can be decoded in a shorter time than the second decoding process is executed. A corresponding short decode timeout period can be set. By changing the decode timeout time in this way, it is possible to set the decode timeout time corresponding to each of the first decoding process and the second decoding process.

29, the first decoding processing unit 202 and the second decoding processing unit 203 are separated, but as shown in FIG. 30, the first decoding processing unit 202 and the second decoding processing unit 203 are integrated. may be the decode processing unit 210 . The decoding processing unit 210 performs a second decoding process on the restored image in parallel with the first decoding process on the read image generated by the camera 5 . The part that executes the decoding process in this way can be arbitrarily divided on the hardware, or can be integrated. Hereinafter, an example in which the part that executes the decoding process is integrated into one will be described, but the present invention is not limited to this, and can be similarly applied to a case where the part is divided into a plurality of parts.

When the setting unit 20e sets not to execute the second decoding process, the decoding processing unit 210 allocates processing resources used for the second decoding process to the first decoding process, and executes the second decoding process. Then, it is configured to speed up the first decoding process compared to the case where it is set. The resources for processing are, for example, memory usage areas, cores constituting a multi-core CPU, and the like. Specifically, when it is set not to execute the second decoding process, the number of cores in charge of the first decoding process is increased compared to when it is set to execute the second decoding process. Further, when it is set not to execute the second decoding process, the memory area used in the first decoding process is expanded compared to when it is set to execute the second decoding process. Increasing the number of cores and expanding the memory area may be used in combination, or only one of them may be performed. As a result, the performance of the multi-core CPU can be maximized and the speed of the first decoding process can be increased.

The decoding processing unit 210 extracts a code candidate area in which a code is likely to exist from the read image prior to image restoration. That is, there is a problem that if the image to be input to the neural network is large during image restoration using the neural network, the computation time becomes long. Also, from the viewpoint of speeding up the processing, it is desirable to reduce the number of decoding attempts as much as possible. On the other hand, in this example, a heat map-based search method is adopted instead of a line search-based method in order to ensure that the code candidate region can be extracted even if it takes some time. For example, the decoding processing unit 210 quantifies the feature amount of the code, generates a heat map in which the magnitude of the feature amount is assigned to each pixel value, and displays code candidates that are highly likely to exist on the heat map. Extract regions. As a specific example, there is a method of acquiring a characteristic portion (eg, finder pattern, etc.) of a two-dimensional code in an area that is relatively hot (having a large characteristic amount) in the heat map. When a plurality of characteristic portions are acquired, they can be extracted with priority and stored in the RAM 41, the storage unit 30, or the like.

After the decoding processing section 210 extracts the code candidate area, it outputs a partial image corresponding to the extracted code candidate area to the inference processing section 20j. The inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit 210 to the neural network, and performs inference processing to generate a restored image by restoring the partial image. Thereby, the calculation time can be shortened.

Further, the decoding processing unit 210 extracts a code candidate area having a size that can contain the code to be read set by the tuning process described above and having a high possibility that the code exists in the read image. The image may be reduced or enlarged to a predetermined size and then input to a neural network, and the restored image restored by the neural network may be decoded. By enlarging/reducing the extracted image to an image of a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range. The image restoration effect can be drawn out stably. In addition, although the size of the code to be read may vary, since the size of the code that is finally input to the neural network can be fixed, the balance between high speed processing and ease of decoding can be achieved. can keep.

Flowcharts of the first decoding process and the second decoding process will be described in detail below with reference to FIG. At step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding processing section 210. FIG. At step SE2, a first decoding process is executed without performing image restoration on the read image acquired at step SE1. If the first decoding process succeeds, the process advances to step SE4 to execute end processing, that is, decoding result output processing. If the first decoding process fails in step SE2, the process returns to step SE2 and the first decoding process is executed again. When a predetermined decode timeout time elapses, the first decoding process for the read image is stopped.

On the other hand, in the route from step SE1 to step SE5, the decode processing section 210 extracts the restoration code area from the read image. Specifically, a code candidate area having a size that can contain the code to be read set by the tuning process and having a high possibility that the code exists in the read image is extracted. At this time, if a plurality of restoration code regions are extracted, they are temporarily stored as candidate regions R1, R2, . An example of the extracted repair code region is shown in FIG. After that, the process proceeds to step SE6 and sets the value of k to 1.

At step SE7, the partial image corresponding to the candidate area Rk is resized. The resizing process is a process of reducing or enlarging the partial image corresponding to the restoration code area extracted in step SE5 to a predetermined size. With this resizing process, the size of an image to be input to a neural network, which will be described later, can be set to a predetermined size in advance. The example shown in FIG. 32 shows the case where the size of the partial image to be input to the neural network is 256 pixels×256 pixels, but the size of the partial image is not limited to this, and the processing speed and the like are not limited to this. You can resize it to any size you want.

At step SE8, it is determined whether or not the black/white reversal setting is "both". In other words, image inpainting is only a mapping mathematically, and it is difficult to realize white-to-black, black-to-white, black-to-black, and white-to-white mappings with a single neural network. can be expensive. Therefore, black-and-white reversal processing is executed based on the property so that the image is printed in black on a white background. Depending on how the neural network is trained, the image may be printed in white on a black background. FIG. 32 shows an example of black-and-white reversal processing. Details of the black-and-white reversal processing will be described later.

If YES is determined in step SE8 and the white/black inversion setting is set to "both", the process proceeds to step SE9 to input the resized image to the inference processing unit 20j to restore the image. After that, the process proceeds to step SE10, and the decoding processing unit 210 executes the second decoding process on the restored image. If the second decoding process succeeds in step SE10, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE10, the process proceeds to step SE11, and black and white reversal processing is performed on the resized image. After that, in step SE11, the image after black-and-white reversal processing is input to the inference processing unit 20j to execute image restoration (see FIG. 32). Next, proceeding to step SE13, the decoding processing unit 210 performs the second decoding processing on the restored image. If the second decoding process succeeds in step SE13, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE13, the process proceeds to step SE14. At step SE14, it is determined whether or not the candidate area extracted at step SE5 is still present.

On the other hand, if NO is determined in step SE8 and the black/white inversion setting is not "both", the process proceeds to step SE15 to determine whether or not the black/white inversion setting is "OFF". If NO is determined in step SE15 and the black/white inversion setting is not "OFF", the process proceeds to step SE16 to invert the black/white of the resized image. After that, in step SE17, the image after black-and-white inversion processing is input to the inference processing unit 20j to execute image restoration. Next, proceeding to step SE18, the decoding processing unit 210 performs the second decoding processing on the restored image. If the second decoding process succeeds in step SE18, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE18, the process proceeds to step SE14. At step SE14, it is determined whether or not there are still candidate areas, and if there are no candidate areas, the process proceeds to step SE4 to execute end processing.

If it is determined in step SE14 that there is still a candidate area, the process proceeds to step SE19, adds 1 to the value of k, and proceeds to step SE7. At step SE7, since the value of k is increased by one from the previous flow, the partial image corresponding to the next candidate region Rk is similarly resized. Step SE8 and subsequent steps are as described above. That is, if the decoding is successful in either the first decoding process or the second decoding process, the decoding result is output at the time of success, and the decoding process is not performed for the other candidate areas, and the read image of the next work W is picked up. and perform similar processing.

The post-processing unit 201 shown in FIG. 32 executes a process of synthesizing a partial image that has not been restored after resizing and a partial image that has been restored by the inference processing unit 20j after resizing. That is, in image restoration using a neural network, even though the code before restoration can be visually confirmed by humans, the image restoration may not go well or the cells may become rounded after the image restoration. For these reasons, if a partial image that has not been restored and a partial image that has been restored are superimposed at an appropriate ratio, subsequent decoding processing may be facilitated. The ratio of superimposing the partial image that has not been image-repaired and the partial image that has been image-repaired may be a predetermined fixed value or may be a value other than the fixed value. Based on the degree of stability, it may be determined dynamically at the time of reading or optimized by pre-evaluation (tuning). For example, in the preliminary evaluation, the user may be allowed to adjust the ratio using a scale bar or the like so that the adjustment can be made while presenting the synthesized image to the user.

(Bank switching function)
In the present embodiment, for example, the parameter set storage unit 30c can store a parameter set including parameters that configure the imaging conditions of the camera 5 and parameters that configure the processing conditions in the decoding processing unit 210. It is configured. A parameter set can be called a bank. A plurality of banks are provided, and different parameters are stored in each bank. For example, in the parameter set storage unit 30c, a plurality of imaging conditions and code conditions including the first imaging condition and code condition set by the tuning execution unit 20d and the second imaging condition and code condition are separately stored. is stored as a parameter set of

In this optical information reading device 1, from among the plurality of parameter sets stored in the parameter set storage unit 35c, one parameter set including the first imaging condition and the code condition is used to obtain the second imaging condition and the code condition. and vice versa. The parameter set switching can be performed by the processor 20, by the user, or can be configured to be performed by a switching signal from an external control device such as the PLC 101. FIG. When the user switches the parameter set, the user may operate a parameter set switching unit incorporated in the user interface, for example. By enabling the parameter set switching unit, the parameter set of the bank is used during operation of the optical information reader 1, and by disabling the parameter set switching unit, the parameter set of the bank is used. The set is not used when the optical information reader 1 is operated. That is, the parameter set switching unit is for switching from one parameter set to another parameter set.

When the decoding processing unit 210 of this example fails the second decoding processing for the restored image while operating using one parameter set, the decoding processing unit 210 switches to another parameter set, and then the image is captured under another condition. Image restoration and second decoding processing are performed on the read image. For example, when the decoding processing unit 210 detects that the second decoding process has failed, it switches to a parameter set other than the currently used parameter set, then executes image restoration and the second decoding process. When it detects that the second decoding process has failed, it switches to another parameter set that has not been used before, and then executes image restoration and second decoding process. When the parameter set is switched, not only the imaging condition of the camera 5 but also the code size to be read may be changed. In this case, the size when extracting the code candidate area where the code is likely to be present also changes. It will be. As a result, the reduction ratio or enlargement ratio of the image input to the neural network also changes, and the result of image restoration also changes.

(Supports black and white inversion)
Next, correspondence to black/white inversion of codes will be described with reference to FIGS. 33 and 34. FIG. FIG. 33 illustrates a case where two neural networks, namely, a neural network trained with a code printed in black on a white background and a neural network trained with a code printed in white on a black background, are used to handle black-and-white inversion of a code. It is a diagram. FIG. 33A inputs the read image (image on the left end) to a neural network trained with a code printed in black on a white background and a neural network trained with a code printed in white on a black background, and outputs decoding results from each neural network. This method is called a parallel execution method. Since the read image is an image of a black-printed code on a white background, decoding is successful in the restored image by a neural network trained with a black-printed code on a white background, but the decoded image is successfully decoded by a neural network trained with a white-printed code on a black background. Decoding fails on the restored image.

FIG. 33B first inputs the read image (image at the left end) only to the neural network trained with the white-on-black code, and attempts to decode the restored image. Only when the decoding process fails, the read image is input to a neural network that has been trained with black-on-white code to generate a restored image. This method is a method of using another network after a failure.

FIG. 33C is a method of selecting a neural network for inputting a read image based on user settings. The case shown on the left side is a case where the read image is set to be an image of a black-printed code on a white background. In this case, the network selection module selects a neural network trained with black-on-white code. Then, the read image (image at the left end) is input to a neural network that has been trained with black-printed codes on a white background. On the other hand, the case shown on the right side is a case where the read image is set to be an image of a white-printed code on a black background. In this case, the network selection module selects a neural network trained with white-on-black code. Then, the read image (image at the left end) is input to a neural network that has been trained with white-printed code on a black background.

FIG. 34 is a diagram for explaining a case where only one neural network is used to handle black/white inversion of a code. FIG. 34A shows a case where a black-and-white inverted image is generated by subjecting the read image (image at the left end) to black-and-white inversion processing. An image without white-black reversal processing and an image with black-white reversal processing are input to a neural network that has been trained with a code for black printing on a white background, respectively. Since the read image is an image of a white-printed code on a black background, the decoding process fails with the restored image by the neural network trained with the black-printed code on a white background. , the decoding process succeeds.

FIG. In the example shown above in 34B, the read image (leftmost image) is an image of a code printed in white on a black background, so if it is input to a neural network trained with a code printed in black on a white background, the decoding process fails. After that, a black-and-white reversed image is generated and input to a neural network that has learned black-on-white code, and the decoding process succeeds.

FIG. In the example shown below 34B, the read image (leftmost image) is an image of a code printed in black on a white background, so if it is input to a neural network trained with a code printed in white on a black background, the decoding process fails. After that, a black-and-white reversed image is generated and input to a neural network that has learned a white-on-black code, and the decoding process succeeds.

FIG. 34C is a method for determining whether or not to execute black/white reversal processing based on user settings. The case shown on the upper side is a case where the read image is set to be an image of a black-printed code on a white background. In this case, the black-and-white reversal processing module does not perform the black-and-white reversal processing, but directly inputs to the neural network that has learned the code for black printing on a white background. On the other hand, the case shown on the lower side is a case where the read image is set to be an image of a white-printed code on a black background. In this case, the black-and-white reversal processing module performs the black-and-white reversal processing. Then, the black-and-white reversed image is input to a neural network that has been trained with black-printed code on a white background.

(Relationship between contrast and matching level)
The matching level is used, for example, as a score during tuning, or for managing print quality/reading performance during operation. For example, the matching level is defined as a positive integer value from 0 to 100, and if it is 50 or higher, it can be designed so that the read rate is 100% when a read rate test is performed. Also, matching level 0 is the lowest value when reading is impossible, and matching level 1 is the lowest value when reading is possible.

FIG. 35 is a graph showing the relationship between the contrast and the matching level. The solid line shows the relationship between the contrast of the image restored by the inference processing unit 20j (restored image) and the matching level. shows the relationship between the contrast of an image (read image) without image restoration and the matching level. As is clear from this graph, the matching level of the repaired image is generally higher than the matching level of the read image.

(Matching level calculation process)
The matching level is set for each algorithm, generally one algorithm per code type. However, in this embodiment, two algorithms (an image restoration algorithm and an image restoration algorithm not executed) are executed with one code type by installing the image restoration function. Of these two algorithms, the result of the one whose decoding process succeeds earlier is output, so if the decoding process succeeds alternately with the two algorithms, the value of the matching level may become unstable. . In addition, since the image can be easily read by executing the image restoration, the matching level is likely to be higher than the existing one in the matching level calculation method similar to the conventional one. For these reasons, in the present embodiment, the weighted sum of the matching levels of the two algorithms is calculated, and the value of the matching level is adjusted. A specific example will be described below based on the flowchart shown in FIG.

In step SF1 after the start, the read image is input to the decode processing section 210. FIG. In step SF2, the decoding processing unit 210 executes the first decoding process without restoring the image. At step SF3, a matching level A (MLA) is calculated by the first decoding process. On the other hand, in step SF4, the read image is input to the inference processing section 20j and image restoration processing is executed. After that, the process proceeds to step SF5, and the decode processing unit 210 executes the second decoding process on the restored image. At step SF6, a matching level B (MLB) is calculated by the second decoding process. If the decoding is not successful in step SF2, MLA=0, and if the decoding is not successful in step SF5, MLB=0.

In step SF7, the values of MLA and MLB are adjusted using, for example, an averaging function. Further, even if the decoding is successful in either step SF2 or SF5, a process of weighting the value of MLA is performed.

(Example of user interface)
FIG. 37 is a diagram showing an example of the user interface 300 displayed on the display unit 6. As shown in FIG. The processor 20 generates the user interface 300 and causes the display unit 6 to display it. The user interface 300 includes a first image display area 301 in which an image (live view image) currently captured by the camera 5 is displayed, a second image display area 302 in which a read image is displayed, and an inference processing unit 20j. A third display area 303 is provided in which the restored image restored by is displayed. Accordingly, the read image and the restored image can be presented to the user in a form that enables comparison. Also, the user can grasp what kind of image the image on which the decoding process is executed or the image after the decoding process is executed.

(Example of image sensor with AI chip)
FIG. 38 is a diagram showing an example using an AI chip-equipped image sensor 5a, and the image sensor 5a of the camera 5 can be configured as shown in each figure. FIG. 38A is a diagram showing a case in which an AI chip for performing image restoration is provided in the imaging element 5a and packaged, and the read image is output to the processor 20 after image restoration. Also, FIG. 38B is a diagram showing a case in which an AI chip for executing restoration code area extraction and image restoration is provided in the imaging device 5a and packaged. , the partial image corresponding to the region is output to the processor 20 after image restoration. Also, FIG. 38C is a diagram showing a case in which an AI chip for extracting a repair code area is provided in the imaging device 5a and packaged. The partial image to be processed is output to the processor 20 .

Also, FIG. In the example shown in 38A, the read image acquired by the imaging element 5a may be output to the processor 20, and the AI chip may perform image restoration, and the restored image may be output to the processor 20. FIG. FIG. 38B, FIG. The same is true for the example shown in 38C.

(Polarizing filter attachment 3)
In the present embodiment, as shown in FIG. 2, a first illumination section 4a and a second illumination section 4b, which are composed of a plurality of light-emitting diodes, are provided as light sources for generating illumination light for illuminating at least the cord. The polarizing filter attachment 3 includes a first polarizing plate 3a that passes light of a first polarization component out of the light generated by the light emitting diodes that constitute the second lighting unit 4b, and a second polarization plate that is substantially perpendicular to the first polarization component. and a second polarizing plate 3b for passing light of two polarization components. The first polarizing plate 3a and the second polarizing plate 3b can be provided in the hatched areas in FIG. The second polarizing plate 3b is arranged to cover the optical system 5b of the camera 5 from the light incident side.

Therefore, the camera 5 receives the light that passes through the first polarizing plate 3a and is reflected from the surface (code) of the workpiece W via the second polarizing plate 3b. As a result, the regular reflection component of the workpiece W is removed, and the camera 5 generates a read image with a lower contrast than when the first polarizing plate 3a and the second polarizing plate 3b are not interposed. The processor 20 acquires this low-contrast read image. By inputting the low-contrast read image into the neural network, the processor 20 converts it into a restored image with a higher contrast than before the input, and executes decoding processing on the restored image.

That is, by using the polarizing plates 3a and 3b, the specular reflection component of the workpiece W is removed, so that a read image with reduced influence of the specular reflection component can be acquired. For example, the polarizing plates 3a and 3b are suitable for images with many specular reflection components, such as when an image of a metal workpiece is captured, but the read image may become dark due to a decrease in the amount of light, resulting in a decrease in contrast. In such a case, reading performance is improved by converting the image into a high-contrast restored image through neural network inference processing.

The above-described embodiments are merely examples in all respects and should not be construed in a restrictive manner. Furthermore, all modifications and changes within the equivalent scope of claims are within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the optical information reader according to the present invention can be used, for example, to read a code attached to a work.

1 stationary optical information reading device 1A handheld optical information reading device 5 camera 20 processor 20f decoding processing unit 20j inference processing unit (image restoration unit)
21 to 24 1st to 4th general-purpose cores 25 Dedicated cores (dedicated cores for inference processing)
40 ROMs
40d neural network storage unit 41 RAM

Claims (20)

An optical information reader that reads a code attached to a work,
a camera that captures the code and produces a read image;
a storage unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to the read image generated by the camera;
an inference processing unit that inputs a read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit and executes inference processing for generating an ideal image corresponding to the read image; a processor having a decoding processing unit that executes a first decoding process for decoding the read image generated by the camera and a second decoding process for decoding the ideal image generated by the inference processing unit;
The processor executes inference processing by the inference processing unit and first decoding processing by the decoding processing unit in parallel, and after the inference processing by the inference processing unit is completed, the second decoding by the decoding processing unit. An optical information reader configured to perform processing. The optical information reader according to claim 1,
The optical information reading device, wherein the decoding processing section has a core that executes the first decoding processing and the second decoding processing in parallel in separate threads. The optical information reader according to claim 1,
The optical information reading device, wherein the decoding processing section is composed of multi-cores capable of executing the first decoding processing and the second decoding processing with different cores. In the optical information reader according to any one of claims 1 to 3,
The decoding processing unit performs an extraction process for extracting a code candidate area in which a code is likely to exist from the read image generated by the camera, and extracts a partial image extracted from the read image by the extraction process. Having a general-purpose core that executes the first decoding process for decoding,
The inference processing unit receives a partial image extracted from the read image by extraction processing, and has an inference processing dedicated core that executes inference processing for generating an ideal image corresponding to the read image. Information reader. In the optical information reader according to claim 4,
The general-purpose core is an optical information reader that executes second decoding processing for decoding an ideal image generated by the inference processing performed by the inference processing dedicated core. The optical information reader according to claim 4 or 5,
The general-purpose core has a first core that executes the extraction process, a second core that executes the first decoding process, and a third core that executes the second decoding process for optical information reading. Device. In the optical information reader according to any one of claims 4 to 6,
The general-purpose core is an optical information reader that executes the second decoding process of decoding the ideal image prior to the first decoding process when an ideal image is generated by the inference processing performed by the dedicated inference processing core. . In the optical information reader according to any one of claims 5 to 7,
When the plurality of partial images are extracted as a result of the extraction processing by the general-purpose core, the inference processing dedicated core performs inference processing on the plurality of partial images according to a predetermined order,
The general-purpose core is an optical information reading device configured to, when a plurality of partial images are extracted as a result of the extraction processing, perform the first decoding processing on the plurality of partial images in accordance with a predetermined order. In the optical information reader according to claim 8,
The optical information reading device, wherein the processor terminates the inference processing by the inference processing unit when decoding is successful in the first decoding processing. In the optical information reader according to any one of claims 4 to 9,
The general-purpose core performs a first extraction process for extracting the code candidate region under a first predetermined condition, and a second predetermined condition for enabling extraction of the code candidate region with higher accuracy than the first predetermined condition. and the first decoding process for decoding the partial image extracted from the read image by the first extraction process,
The inference processing dedicated core receives a partial image extracted from the read image by the second extraction processing, and performs inference processing for generating an ideal image corresponding to the read image. In the optical information reader according to any one of claims 4 to 10,
An optical information reader comprising a memory that stores a partial image extracted from the read image by the extraction process and that can be accessed by both the general-purpose core and the inference process core. The optical information reader according to any one of claims 4 to 11,
The general-purpose core and the inference processing dedicated core are mounted on the same substrate,
The inference processing dedicated core is an optical information reading device composed of a dedicated core dedicated to inference processing by a neural network. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding processing unit that executes a first decoding process on the read image generated by the camera;
a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit;
The second decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, and sends a trigger signal to the image restoration unit to execute inference processing,
When receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the second decoding processing unit to the neural network, and restores the partial image to the neural network. Execute an inference process that generates a repaired image that is a repaired image,
The second decoding processing unit determines a grid position indicating each cell position of the code based on the restored image, and executes decoding processing of the restored image based on the determined grid position. 14. The optical information reader according to claim 13,
The inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are executed in parallel, and at least part of the period during which the second decoding processing unit is executing the second decoding processing is the image restoration. An optical information reader having a rest period during which inference processing by the unit is not performed. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding processing unit that executes a first decoding process on the read image generated by the camera;
a second decoding processing unit that performs a second decoding process on the restored image generated by the image restoration unit;
a setting unit that sets whether to execute the second decoding process,
The first decoding processing unit is capable of executing the first decoding processing in a time shorter than the time required for the second decoding processing,
When the setting unit is set to execute the second decoding process, the setting unit sets a decoding timeout period longer than the time required to execute the second decoding process, and executes the second decoding process. An optical information reader configured to be able to set a decoding time-out period shorter than the time required for executing the second decoding process when it is set not to do so. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding process on the read image generated by the camera; and a decoding processing unit that, in parallel with the first decoding process, performs a second decoding process on the restored image generated by the image restoration unit;
a setting unit that sets whether to execute the second decoding process,
When the setting unit sets not to execute the second decoding processing, the decoding processing unit allocates processing resources used for the second decoding processing to the first decoding processing, and performs the second decoding processing. An optical information reader that speeds up the first decoding process compared to when the process is set to be executed. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
The decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image,
The image restoration unit inputs a partial image corresponding to the code candidate region extracted by the decoding processing unit to the neural network, and performs an inference process for generating a restored image obtained by restoring the partial image,
The decoding processing unit determines a grid position indicating each cell position of a code based on the restored image, and decodes the restored image based on the determined grid position. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
Repeating the imaging and decoding processes by changing the imaging conditions of the camera and the decoding conditions of the decoding process, and based on the matching level indicating the readability of the code calculated under each imaging condition and decoding condition, a tuning execution unit that determines optimum imaging conditions and decoding conditions and executes tuning processing for setting the size of a code to be read;
The decoding processing unit
A code candidate area having a size capable of containing the code to be read set by the tuning process and having a high possibility that the code exists in the read image is extracted, and the extracted image is subjected to a predetermined An optical information reading device that reduces or enlarges the size of the image, inputs it to the neural network, and decodes the restored image that has been restored by the neural network. The optical information reader according to any one of claims 1 to 12,
a plurality of light sources that generate illumination light that illuminates at least the code;
a first polarizing plate for passing light of a first polarization component out of the light generated by the light source;
a second polarizing plate that passes light of a second polarization component substantially orthogonal to the first polarization component,
The camera receives light that has passed through the first polarizing plate and is reflected from the code via the second polarizing plate, and removes specular reflection components of the work, thereby obtaining the first polarizing plate and the second polarizing plate. Generating a low-contrast read image compared to the case where two polarizing plates are not interposed,
The optical information reading device, wherein the processor inputs the low-contrast read image to the neural network to convert it into a restored image with a higher contrast than before the input, and executes decoding processing on the restored image. The optical information reader according to any one of claims 1 to 12,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
The imaging conditions of the camera are changed to repeat imaging and decoding processing, and the imaging conditions and the code of the code to be read are calculated based on the matching level indicating the readability of the code calculated under each imaging condition. a tuning execution unit capable of setting a plurality of conditions,
The storage unit stores a plurality of imaging conditions and code conditions including a first imaging condition and code condition and a second imaging condition and code condition set by the tuning execution unit,
The decoding processing unit inputs to the neural network a read image generated under the first imaging condition and the code condition among the plurality of reading conditions stored in the storage unit to generate a restored image, optical information for generating a restored image by inputting the read image generated under the second imaging condition and the code condition into the neural network when decoding of the restored image fails, and executing decoding of the restored image; reader.

Images