WO2024218987A1

WO2024218987A1 - Programming device, setting adjustment method, and program

Info

Publication number: WO2024218987A1
Application number: PCT/JP2023/016009
Authority: WO
Inventors: 順一郎吉田
Original assignee: ファナック株式会社
Priority date: 2023-04-21
Filing date: 2023-04-21
Publication date: 2024-10-24

Abstract

The present invention is a programming device comprising: a program generation part that provides a function to generate an image processing program for executing image processing of an image captured by a visual sensor; and a setting part that receives settings related to the image processing. The setting part displays, in a display screen, a plurality of adjustment purposes related to the image processing in a selectable manner, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes so that the priority corresponding to the selected one adjustment purpose can be recognized.

Description

PROGRAMMING DEVICE, SETTING ADJUSTMENT METHOD, AND PROGRAM

This disclosure relates to a programming device, and a setting adjustment method and program executed on the programming device.

There is known an image processing system that processes an image of an object captured by a camera and detects and inspects the object. In this regard, Patent Documents 1 and 2 describe examples of the configuration of an image processing device that can present a setting screen for setting measurement conditions.

JP 2008-123034 A JP 2016-120565 A

The image processing programs used in image processing devices such as those described above tend to have many setting items so that they can be used for as wide a variety of objects as possible. For this reason, it is generally a difficult task that requires skill for the operator of such image processing programs to find the necessary setting items according to the purpose and set them appropriately. There is a demand for technology that makes it possible to easily and efficiently program the setting items and adjustments of image processing program settings.

One aspect of the present disclosure is a programming device that includes a program creation unit that provides a function for creating an image processing program that executes image processing on an image captured by a visual sensor, and a setting unit that accepts settings related to the image processing, and the setting unit displays a plurality of adjustment purposes related to the image processing on a display screen in a selectable manner, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a manner that allows the user to recognize the priority according to the one selected adjustment purpose.

These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the present invention illustrated in the accompanying drawings.

1 is a diagram illustrating a configuration of an image processing system including a programming device according to an embodiment. FIG. 2 is a functional block diagram of a programming device and an image processing device. FIG. 11 is a diagram showing an example of a setting screen serving as a user interface for programming. 13A and 13B are diagrams showing an example of an image processing program that is set and displayed in a processing flow display area. 5 is a diagram showing examples of setting items associated with each functional module constituting an image processing program. FIG. 13A and 13B are diagrams showing examples of adjustment purposes displayed selectably in an adjustment purpose selection area. FIG. 13 is a diagram showing an example of a display mode of setting items displayed in a setting area when an adjustment purpose "increase processing speed" is selected. 13 is a diagram showing an example in which functional modules including setting items to which priorities according to adjustment purposes have been assigned are highlighted in a processing flow display area in which the processing flow of an image processing program is displayed. FIG. 4 is a flowchart showing a basic operation of a process for adjusting a setting item of an image processing program.

Next, an embodiment of the present disclosure will be described with reference to the drawings. In the drawings, similar components or functional parts are given similar reference symbols. The scale of these drawings has been appropriately changed to facilitate understanding. Furthermore, the form shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the form shown.

FIG. 1 is a diagram showing the configuration of an image processing system 100 including a programming device according to one embodiment. As shown in FIG. 1, the image processing system 100 includes a programming device 10, an image processing device 20 connected to the programming device 10, and a visual sensor 30 and a lighting device 31 connected to the image processing device 20. The image processing system 100 is configured to capture an image of an object 90 using the visual sensor 30, and to perform various image processing such as detection and inspection of the object 90 on the captured image. In the image processing system 100, the programming device 10 provides a function for creating various programs related to the capture of images by the visual sensor 30 and image processing of the captured images. The image processing device 20 is responsible for the function of controlling the visual sensor 30 and the lighting device 31 and performing image processing based on commands from the programming device 10.

In this specification, programming an image processing program includes creating a program using command statements, function modules, icons, etc., as well as completing the function of the image processing program by setting and adjusting configuration items.

The programming device 10 can be any information processing device, such as a tablet terminal. The programming device 10 may have a hardware configuration as a general computer having a processor 11, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 12, a display unit 13, an operation unit 14, an input/output interface, a network interface, etc. (see FIG. 2). The display unit 13 may be a flat panel display (such as a liquid crystal display). The operation unit 14 may be a touch panel input device integrated with the display.

The visual sensor 30 may be a two-dimensional camera that captures grayscale or color images, or a stereo camera or three-dimensional sensor that can obtain distance images. The image processing device 20 holds a model pattern of the object, and can perform image processing to detect the object by pattern matching between the object in the captured image and the model pattern.

The image processing device 20 may be configured as a device dedicated to image processing, or may be configured as a general-purpose information processing device such as a PC (personal computer). The image processing device 20 may have a hardware configuration as a general computer having a processor 21, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 22, an input/output interface, a network interface, etc. (see FIG. 2). The image processing device 20 may further have a display unit and an operation unit as hardware components.

In FIG. 1, an example is shown in which the image processing device 20 is configured as a device separate from the programming device 10, but the functions of the image processing device 20 may be built into the programming device 10.

FIG. 2 shows a functional block diagram of the programming device 10 and the image processing device 20.

As shown in FIG. 2, the programming device 10 includes a program creation unit 111, a program execution unit 112, a setting unit 113, a processing time acquisition unit 114, and a status analysis unit 115. These functional blocks may be realized by the processor 11 of the programming device 10 executing software. FIG. 2 also shows the memory unit 12 of the programming device 10. The memory unit 12 may be configured, for example, from a non-volatile memory or a hard disk drive.

The program creation unit 111 provides a function for creating various programs, such as programs related to capturing images by the visual sensor 30 and image processing of the captured images. For example, the program creation unit 111 displays a program creation screen for creating programs and accepts user input for programming. The program creation unit 111 may be configured to accept programming based on input of text-based commands, as well as programming based on functional modules or icons. The program creation unit 111 also has a function for registering the created programs in the storage unit 12.

The program execution unit 112 provides a function for executing the created program. The program execution unit 112 sends commands based on the program to the image processing device 20 to perform image capture and image processing.

The setting unit 113 displays various setting items related to image processing, and provides a function for accepting input of setting values for the setting items. The setting unit 113 has a function for displaying selectable adjustment purposes for image processing so that an operator can efficiently set parameters for image processing, and for displaying setting items related to the selected adjustment purpose in response to the adjustment purpose selected by the operator. Furthermore, the setting unit 113 has a function for displaying one or more setting items related to one adjustment purpose selected from multiple adjustment purposes, so that the priority according to the selected adjustment purpose can be recognized. This allows the operator to recognize the priority according to the adjustment purpose for the setting items related to the selected adjustment purpose, and allows efficient setting.

For each adjustment purpose, information associating the setting items related to the adjustment purpose with the priority of the setting items related to the adjustment purpose according to the adjustment purpose (hereinafter also referred to as priority information) may be set in advance and stored in the storage unit 12, for example. As an example, the priority information has a data structure as shown in Table 1 below. Note that in the priority information shown in Table 1 below, adjustment purpose A is associated with setting items A1 and A2, which are related setting items, and the priority of those setting items A1 and A2 according to adjustment purpose A. Also, in the priority information shown in Table 1 below, adjustment purpose B is associated with setting items B1 and B2, which are related setting items, and the priority of those setting items B1 and B2 according to adjustment purpose B.

The setting unit 113 can use such priority information to display one or more setting items related to one adjustment purpose selected from multiple adjustment purposes in such a way that the priority according to the selected adjustment purpose can be recognized.

The processing time acquisition unit 114 can acquire from the processing time measurement unit 123 the processing time actually required for each functional unit related to image processing when image processing is executed in the image processing device 20. For example, the processing time acquisition unit 114 can acquire the time required for processing to search for the angle of an object as one functional unit in pattern matching for detecting the object.

The status analysis unit 115 cooperates with the image processing device 20 and has the function of acquiring and analyzing the status related to the execution of the image processing program.

The storage unit 12 stores programs, various settings related to image processing programs, priority information, etc.

As shown in FIG. 2, the image processing device 20 includes a visual sensor control unit 121, an image processing unit 122, and a processing time measurement unit 123. These functional blocks may be realized by the processor 21 of the image processing device 20 executing software. FIG. 2 also shows the memory unit 22 of the image processing device 20. The memory unit 22 may be configured, for example, from a non-volatile memory or a hard disk drive.

The visual sensor control unit 121 controls the visual sensor 30 in accordance with commands sent from the programming device 10. The image processing unit 122 has the function of executing various image processing operations on the image captured by the visual sensor 30. The processing time measurement unit 123 provides a function of measuring the actual processing time for each function corresponding to each setting item when image processing is performed by the image processing unit 122. The function of the processing time measurement unit 123 can be realized, for example, by incorporating a function in the image processing program that obtains the processing start time and processing end time for each function from an internal timer and calculates the processing time.

The storage unit 22 stores various information such as image data, various setting information related to image processing, and model data of the object.

FIG. 3 shows an example of a setting screen 200 as a user interface for programming. The setting screen 200 also functions as a program creation screen. The setting screen 200 is provided by the program creation unit 111 and the setting unit 113. The program creation unit 111 and the setting unit 113 display the setting screen 200 on the display unit 13. The program creation unit 111 and the setting unit 113 also accept input to the setting screen 200 via the operation unit 14.

As shown in FIG. 3, the setting screen 200 includes an image display area 211, an adjustment purpose selection area 212, a processing flow display area 213, and a setting area 214.

The image display area 211 is an area for displaying the image captured by the visual sensor 30. For example, the programming device 10 (program execution unit 112) executes an image processing program and acquires an image G of the object 90 captured by the visual sensor 30 from the image processing device 20, and displays it in the image display area 211. The worker can check the image used for image processing (inspection, etc.) on the image display area 211. Image G is an example of an image showing three objects 90 in the situation shown in Figure 1.

The adjustment purpose selection area 212 is an area for displaying a list of adjustment purposes related to the image processing program and accepting the selection of an adjustment purpose. The setting unit 113 displays a list of multiple selectable adjustment purposes in the adjustment purpose selection area 212. The setting unit 113 accepts a user operation to select one adjustment purpose from the list of adjustment purposes.

The processing flow display area 213 is an area for setting and displaying the processing flow of the image processing program. The program creation unit 111 can provide a function that enables an operator to perform programming by arranging icons representing functional modules in the processing flow display area 213 along the processing flow. An operator can perform programming by selecting a desired functional icon from a list of functional module icons prepared in advance and arranging it in the processing flow display area 213.

The setting area 214 is an area for displaying setting items related to the image processing program and for inputting or adjusting the setting items. The setting unit 113 has a function for displaying setting items related to a functional module selected in the processing flow display area 213 and accepting settings for those setting items. Furthermore, the setting unit 113 has a function for displaying, in the setting area 214, one or more setting items related to an adjustment purpose selected in the adjustment purpose selection area 212, so that the priority according to the selected adjustment purpose can be recognized.

FIG. 4 shows an example of an image processing program P1 that is set and displayed in the processing flow display area 213. The image processing program P1 is a program for inspecting an object. As shown in FIG. 4, the image processing program P1 includes an imaging function M1, a pattern matching function M2, a length measurement function M3, and a pass/fail determination function M4.

The imaging function M1 corresponds to a function of capturing an image of an object using the visual sensor 30. The pattern matching function M2 corresponds to a function of detecting an object from the captured image by pattern matching using a model pattern of the object. The length measurement function M3 corresponds to a function of measuring the length of an inspection target portion on an object recognized by pattern matching. An example of the length measurement function M3 is a process of measuring the length between opposing sides when the object is box-shaped. The pass/fail judgment function M4 corresponds to a function of making a pass/fail judgment on the length measured by the length measurement function M3.

The operator can perform programming by arranging the functional modules in the process flow display area 213 in order and setting and adjusting the necessary setting items for each functional module. The setting unit 113 has a function of displaying the setting items of the functional module selected in the process flow display area 213 in the setting area 214 and accepting settings for those setting items.

Referring to FIG. 5, an example of setting items associated with each functional module constituting the image processing program P1 is shown. The setting unit 113 displays in the setting area 214 the setting items associated with the functional module selected in the processing flow display area 213 and accepts the settings. Information associating each functional module with its related setting items is stored in advance in the storage unit 12.

As shown in Fig. 5, the imaging function M1 includes the following setting items related to it: camera selection, exposure time, illumination intensity, number of images taken, imaging area, and image reduction ratio. The screen of the setting items displayed in the setting area 214 in response to the selection of the imaging function M1 is referred to as a setting item screen 301. When the imaging function M1 is selected on the process flow display area 213, the setting unit 113 has a function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Fig. 5 shows an example in which the setting values are input for these setting items as follows:
Camera selection: Camera 1
Exposure time: 200 ms
Lighting Intensity: 100%
Number of shots: 1 Imaging area: 1280 x 1024 pixels
Image reduction ratio: 1/2

The pattern matching function M2 includes, as its associated setting items, model setting, score threshold, contrast threshold, size threshold, angle threshold, upper limit of detection number, and detection time limit. The screen of setting items displayed in the setting area 214 in response to the selection of the pattern matching function M2 is referred to as a setting item screen 302. When the pattern matching function M2 is selected on the process flow display area 213, the setting unit 113 has a function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the setting values are input for these setting items as follows:
Model Settings: Set score threshold: 75%
Contrast threshold: 50
Size Threshold: 90 to 120%
Angle threshold: -30 to +30 deg
Detection limit: 5
Detection time limit: 30 seconds

The length measurement function M3 includes input, measurement region setting, contrast threshold, and length threshold as its associated setting items. The screen of setting items displayed in the setting area 214 in response to the selection of the length measurement function M3 is referred to as a setting item screen 303. When the length measurement function M3 is selected on the process flow display area 213, the setting unit 113 has a function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the following setting values are input for these setting items:
Input: Pattern match Measurement area setting: Already set Contrast threshold: 50
Length threshold: 100 to 100 pixels

The pass/fail judgment function M4 includes input, judgment criteria, and judgment as its associated setting items. The screen of setting items displayed in the setting area 214 in response to the selection of the pass/fail judgment function M4 is referred to as the setting item screen 304. When the pass/fail judgment function M4 is selected in the process flow display area 213, the setting unit 113 has a function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the following setting values are input for these setting items.
Input: Length measurement Criteria: 50>input>25
・Judgment: Pass if within range

In this way, the program creation unit 111 and the setting unit 113 can provide basic programming functions, such as setting an image processing program in the processing flow display area 213, displaying setting items in the setting area 214, and accepting setting input.

Next, we will explain the function of the setting unit 113 to display one or more setting items related to one adjustment purpose selected in the adjustment purpose selection area 212 so that the priority according to the selected adjustment purpose can be recognized.

6 shows an example of selectable adjustment purposes displayed in the adjustment purpose selection area 212. The adjustment purposes displayed here are set based on the results of a prior analysis of what kind of adjustments workers involved in image processing generally want to make. In this example, the multiple adjustment purposes are as follows:
1. "Standard"
2. "I want to speed up processing."
3. "I want to take measures against false positives."
4. "I want to take measures against undetected viruses."
5. "I want to improve detection accuracy."
6. "I want to limit adjustments."
Contains:

The adjustment purpose "standard" is an item that is selected when the operator desires standard parameter settings for the image processing program. When the adjustment purpose "standard" is selected, the setting unit 113 may, for example, set predetermined standard values as values for each setting item for the image processing program set in the processing flow display area 213, and display them in the setting area 214.

The adjustment objective "I want to increase the processing speed" is an item that is selected when the operator wishes to increase the processing speed of the image processing program. When the adjustment objective "I want to increase the processing speed" is selected, the setting unit 113 displays one or more setting items related to this adjustment objective in the setting area 214 so that the priority according to this adjustment objective can be recognized.

The adjustment purpose "I want to take measures against false detection" is an item that is selected when the operator wishes to improve false detections in the image processing program. Here, false detection refers to an event in which the image processing program detects an object that is not the target object as the target object. When the adjustment purpose "I want to take measures against false detection" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

The adjustment purpose "I want to take measures against non-detection" is an item that is set when the operator wishes to improve the non-detection of the image processing program. Here, "non-detection" refers to an event in which the image processing program is unable to detect an object that should have been detected. When the adjustment purpose "I want to take measures against non-detection" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

The adjustment purpose "I want to improve detection accuracy" is an item that is set when the operator desires more accurate detection. When the adjustment purpose "I want to improve detection accuracy" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

The adjustment purpose "I want to limit adjustment" is an item that is selected when the operator wishes to perform settings in a state in which the setting items to be adjusted are limited and only the necessary setting items are subject to adjustment. When the adjustment purpose "I want to limit adjustment" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

FIG. 7 shows an example of the display mode of the setting items displayed in the setting area 214 when the adjustment objective "I want to speed up processing speed" is selected when image processing program P1 is set in the processing flow display area 213. Here, we consider a situation in which standard values are set in advance for each setting item, and the worker selects this adjustment objective "I want to speed up processing speed" in order to speed up processing speed. As shown in the example in FIG. 7, the setting area 214 displays, in a recognizable manner, the setting items that should be given priority in accordance with the adjustment objective "I want to speed up processing speed" out of the setting items related to each functional module of image processing program P1.

As shown in FIG. 7, when the imaging function M1 is selected in the processing flow display area 213, the setting item screen 301 displayed in the setting area 214 is in the display mode indicated by the reference symbol 301a in FIG. 7. Of the setting items of the imaging function M1, camera selection, exposure time, lighting intensity, number of times of imaging, imaging area, and image reduction ratio, the exposure time, number of times of imaging, imaging area, and image reduction ratio are related to the processing speed. The imaging area is a setting item that specifies the area of the imaging area that is to be acquired as an image. By setting the imaging area to a small value, the processing time can be shortened. The image reduction ratio is a setting item that specifies the degree to which the image is compressed. Here, the smaller the image reduction ratio is set to, the smaller the image will be, and the shorter the time required for image processing. Therefore, in the setting item screen 301a related to the imaging function M1, of the setting items of the camera selection, exposure time, lighting intensity, number of times of imaging, imaging area, and image reduction ratio, the exposure time, number of times of imaging, imaging area, and image reduction ratio are highlighted. Highlighting may be done, for example, by changing the size, weight, color, font, etc. of the text.

Furthermore, among these four setting items, exposure time, number of captures, capture area, and image reduction ratio, exposure time and image reduction ratio are particularly closely related to the processing speed of image processing, so among these four setting items, "exposure time" and "image reduction ratio" have the highest priority. Therefore, among the above four setting items, "exposure time" and "image reduction ratio" are displayed with even more emphasis. Further emphasis of "exposure time" and "image reduction ratio" may be achieved, for example, by changing the size, thickness, color, font, etc. of the characters.

As shown in FIG. 7, when the pattern matching function M2 is selected in the processing flow display area 213, the setting item screen 302 displayed in the setting area 214 is in the display mode indicated by the reference symbol 302a in FIG. 7. Among the setting items of the pattern matching function M2, namely, model setting, score threshold, contrast threshold, size threshold, angle threshold, upper limit of number of detections, and detection time limit, the size threshold and angle threshold are related to the processing speed. In the pattern matching, a process is performed to search the image for a feature that is the same as the taught feature (model feature). At this time, the search is performed while rotating the model feature and changing its size. The "angle threshold" specifies the allowable angle range for the feature to rotate during this search. For example, if the angle threshold is set to ±0 degrees, the rotation of the feature is not taken into consideration, and the search becomes faster. The "size threshold" specifies the allowable range of the feature size during this search. For example, if the size threshold is set to 100%, only features of the same size as the model feature are searched, and the processing becomes faster. In this way, among the setting items of the pattern matching function M2, the size threshold and angle threshold are related to the processing speed. Therefore, on the setting item screen 302a for the pattern matching function M2, these two setting items, the size threshold and the angle threshold, are highlighted compared to the other setting items.

In this way, the operator can recognize which setting items should be prioritized and their priorities according to the adjustment objective, such as "I want to speed up processing," and can efficiently make appropriate settings according to the adjustment objective.

In addition, when the processing time of the entire image processing program is taken into consideration, if there is a difference in effectiveness between the setting items to be emphasized for the imaging function M1 and the setting items to be emphasized for the pattern matching function M2, the degree of emphasis may be set by further taking into consideration the difference in effectiveness. For example, when the processing time of the entire image processing program is taken into consideration, the degrees of effectiveness (priority) of improving the processing speed for the exposure time, the number of images captured, the imaging area, the image reduction rate, the size threshold, and the angle threshold are assumed to be in the following order:

(Exposure time, image reduction ratio)>(Number of images taken, image area)>(Size threshold, angle threshold)

In this case, the setting unit 113 may display the exposure time and image reduction ratio with the strongest emphasis, then the number of times images are captured and the captured area with the second strongest emphasis, and the size threshold and angle threshold with the third strongest emphasis.

For the length measurement function M3, there is no setting item that should be given priority in accordance with the adjustment purpose of "I want to speed up the processing speed," so no special emphasis is given on the setting item screen 303 displayed in association with the length measurement function M3. For the pass/fail determination function M4, there is no setting item that should be given priority in accordance with the adjustment purpose of "I want to speed up the processing speed," so no special emphasis is given on the setting item screen 304 displayed in association with the pass/fail determination function M4.

The

setting item screens

301a, 302a, 303, and 304 shown in FIG. 7 may be displayed simultaneously in window format in the setting area 214 in response to the selection of "I want to increase the processing speed" in the adjustment objective selection area 212 when the image processing program P1 is set in the processing flow display area 213.

In addition to the above-mentioned example, examples of the manner in which the setting items to be preferentially set are displayed in a recognizable manner include the following:
- Emphasize the setting items that should be set with priority by underlining, adding a border or other image.
- The priority level is indicated by the number of marks and displayed next to the setting item.
- Hide or weaken the display of settings other than those that should be set as a priority.
The term "weak display" used here means displaying text with a lowered emphasis by lightening the text color, reducing the text size, etc.

If the adjustment purpose is "to take measures against erroneous detection," the following setting items may be displayed in an emphasized manner as setting items that should be preferentially set in accordance with this adjustment purpose.
- "Score threshold" for pattern matching function
- "Contrast threshold" for pattern matching function
・"Model Settings" for the Pattern Matching Function
- "Size threshold" for pattern matching function
・"Exposure time" of the imaging function

The pattern match setting "Score threshold" specifies the threshold for the degree of match calculated for the target on the image in pattern matching. Increasing the score threshold of pattern matching has the effect of reducing false positives. The pattern match setting "Contrast threshold" specifies the contrast threshold for the background of the features on the image (e.g., the contour of the target). Increasing the contrast threshold of pattern matching makes it possible to subject only clearly visible features to pattern matching, which has the effect of reducing false positives. The pattern match setting "Model setting" is a setting item for teaching model features. For example, re-executing model setting and increasing model features (e.g., increasing the number of edge points that represent model features) is a measure to reduce false positives. Reducing the size threshold of pattern matching is effective in reducing false positives. In addition, for example, increasing the exposure time of the imaging function so that unnecessary features are not captured on the image has the effect of reducing false positives.

Of the above five setting items that are effective in countering false positives, the "score threshold" and "contrast threshold" are particularly effective in reducing false positives, and therefore the setting unit 113 may assign a higher priority to the "score threshold" and "contrast threshold" of the above five setting items than to the other setting items. Therefore, the setting unit 113 may display the "score threshold" and "contrast threshold" of the above five setting items with even greater emphasis.

In this way, even when the adjustment objective is "to take measures against false positives," the operator can recognize the setting items that should be prioritized according to the adjustment objective and their priorities, and can efficiently make appropriate settings according to the adjustment objective.

When the adjustment purpose is "to take measures against undetected problems," the following setting items may be displayed with emphasis as setting items that should be preferentially set according to this adjustment purpose.
- "Score threshold" for pattern matching function
- "Contrast threshold" for pattern matching function
- "Size threshold" for pattern matching function
・"Exposure time" of the imaging function

Lowering the "score threshold" of the pattern matching function has the effect of reducing undetected cases. Lowering the "contrast threshold" of the pattern matching function has the effect of reducing undetected cases. Increasing the width of the "size threshold" of the pattern matching function has the effect of reducing undetected cases. Adjusting the "exposure time" of the imaging function so that the necessary features of the object are revealed has the effect of reducing undetected cases.

Of the above four setting items that are effective in preventing non-detection, the "score threshold" and "contrast threshold" are particularly effective in reducing non-detection, and therefore the setting unit 113 may assign a higher priority to the "score threshold" and "contrast threshold" than to the other setting items among the above four setting items. Therefore, the setting unit 113 may display the "score threshold" and "contrast threshold" among the above four setting items that are effective in preventing non-detection with even more emphasis.

In this way, even when the adjustment objective is "to take measures against undetected problems," the operator can recognize the setting items that should be prioritized according to the adjustment objective and their priorities, and can efficiently make appropriate settings according to the adjustment objective.

When the adjustment purpose is "to improve detection accuracy," the following setting items may be displayed in an emphasized manner as setting items that should be preferentially set in accordance with this adjustment purpose.
・"Model Settings" for the Pattern Matching Function
- "Contrast threshold" for pattern matching function
・Image reduction ratio for the imaging function

For example, re-teaching the model features in the "Model Settings" of the pattern matching function so that only stable features are included as model features is effective in improving detection accuracy. Increasing the "Contrast Threshold" of the pattern matching function has the effect of improving detection accuracy. Increasing the "Image Reduction Ratio" of the image capture to a value closer to 1 increases the image resolution, which is effective in improving detection accuracy.

Of the above three setting items that are effective in terms of improving detection accuracy, "Model setting" and "Contrast threshold" are particularly effective in improving detection accuracy, so the setting unit 113 may assign a higher priority to "Model setting" and "Contrast threshold" than to the other setting items among these three setting items. Therefore, the setting unit 113 may display "Model setting" and "Contrast threshold" with even more emphasis among the above three setting items that are effective in terms of improving detection accuracy.

The adjustment purpose "I want to limit adjustment" is selected when it is desired to limit the setting items so that only necessary setting items are set. For example, in a situation where an operator changes only the pass/fail judgment criteria in the daily operation of the image processing system 100, it may be desired to operate only this setting item and exclude other setting items from the operation targets. In this case, for example, in response to the selection of the adjustment purpose "I want to limit adjustment",
- Only the "judgment criteria" for pass/fail judgment will be displayed as a configurable setting item.
- Only the "criteria" for pass/fail judgment will be highlighted.
- The settings other than the "Judgment Criteria" for pass/fail judgment will be displayed weakly (grayed out, etc.),
Examples of such displays are possible.
Such an operation can also be considered as an example of an operation for displaying setting items related to one selected adjustment purpose so that the priority according to the selected adjustment purpose can be recognized.

When the adjustment purpose "I want to limit adjustments" is selected, there may be two or more setting items that are highlighted as configurable setting items. When there are two or more setting items that can be set in relation to the adjustment purpose "I want to limit adjustments", a priority according to this adjustment purpose may be added to those two or more setting items.

This adjustment purpose "I want to limit adjustments" allows an inexperienced worker to take over the setting know-how of an experienced worker as priority information. For example, if priority information is created so that setting items that do not need to be adjusted are not set when an image processing program is executed, an inexperienced worker can select the adjustment purpose "I want to limit adjustments" to make adjustments while excluding setting items that do not need to be adjusted from the settings.

The setting unit 113 may have a function to display a functional module in the process flow display area 213 so that the user can recognize it when there is a setting item with a priority assigned according to the selected adjustment purpose for a certain functional module. FIG. 8 shows an example in which a functional module including a setting item with a priority assigned according to the adjustment purpose is highlighted in the process flow display area 213 in which the process flow of the image processing program P1 is displayed. In this example, when the adjustment purpose is "to increase the processing speed", the "imaging function M1" and the "pattern matching function M2" include setting items with a priority assigned, and these "imaging function M1" and "pattern matching function M2" are highlighted. In this example, the processing blocks of the imaging function M1 and the pattern matching function M2 are highlighted by thickening the borders. Other methods for highlighting the processing blocks in the process flow display area 213 (for example, highlighting by character size or color) may be used.

The setting unit 113 may display, with a stronger emphasis, functional blocks including setting items that should be set with higher priority among the functional blocks that should be highlighted in the processing flow display area 213. FIG. 8 shows an example in which the processing block of the imaging function M1 is displayed with more emphasis than the processing block of the pattern matching function M2 because the imaging function M1 includes setting items with a higher priority than the pattern matching function M2. Here, the processing block of the imaging function M1 is displayed with a thicker frame than the processing block of the pattern matching function M2.

In the above-described embodiment, the setting unit 113 sets the priority of one or more setting items related to the selected adjustment purpose according to preset priority information. The setting unit 113 may have a function of switching the priority set according to the priority information according to various situations related to the setting or execution of the image processing program. Note that switching the priority may include assigning a priority to a setting item to which a priority has not been assigned (i.e., a setting item to which a priority has been assigned zero), or canceling the priority of a setting item to which a priority has been assigned. In this regard, the setting unit 113 may have one or more of the following functions (F1) to (F3).
(F1) A function for switching the priority of setting items based on the actual processing time for each function of the image processing program.
(F2) A function for switching the priority of each setting item in the image processing program according to the current setting value of the setting item.
(F3) A function for switching the priority of setting items according to the execution status of the image processing program.

An example of the above function (F1) will be described. As an example, it is assumed that the selected adjustment purpose for the above-mentioned image processing program P1 is "to increase the processing speed." In this case, the processing time acquisition unit 114 can acquire the processing time per function when the image processing program P1 is executed from the image processing device 20 (processing time measurement unit 123). For example, it is assumed that the processing time acquisition unit 114 has determined that angle search is dominant in pattern matching (or in the entire image processing) from the measured values of processing time per function, such as angle search and size search in pattern matching. In this case, the setting unit 113 may set the "angle threshold" of the pattern matching function M2 as the setting item with the highest priority, or may switch the priority of the "angle threshold" to a value higher than the current value.

In this way, the setting unit 113 determines, from the measured value of the processing time for each function,
- Set the priority of the setting item that corresponds to the function that takes the longest time to process to the highest priority,
- Raise the priority of settings that correspond to functions that take longer to process than other functions, or
The priority of a setting item corresponding to a function whose processing time is shorter than that of other functions may be lowered.

A first example of the above function (F2) will be described. When the purpose of adjustment is "to take measures against false detection," the setting items to which priority has been assigned by the priority information include the pattern matching score threshold and the contrast threshold. Setting the score threshold high is effective in taking measures against false detection, but if the current score threshold is already set to a high value, there is no room left for adjusting the score threshold. In such a situation, the setting unit 113 may lower the priority of the score threshold, or cancel the assignment of priority to the score threshold. That is, the setting unit 113 may lower the emphasis level of the score threshold, or may not perform emphasis display. In addition, in a situation where there is no room left for adjusting the score threshold, the importance of adjusting the contrast threshold increases. Therefore, the setting unit 113 may, for example, switch the priority of the contrast threshold to a higher value, so that the priority of the contrast threshold becomes higher than the priority of the score threshold.

A first example of the function (F3) will be described. The purpose of adjustment is to "take measures against non-detection." For example, in a situation where the captured image is dark overall, the setting unit 113 may assign priority to the exposure time and the illumination intensity so that they are adjusted. The situation where the captured image is dark overall may be determined, for example, by the state analysis unit 115 acquiring and analyzing an image obtained by executing an image processing program from the image processing device 20 (image processing unit 122). The state analysis unit 115 may determine the degree of brightness of the entire image based on, for example, a histogram of the luminance values of each pixel of the captured image. In this case, when the exposure time has already reached the settable upper limit, the setting unit 113 may additionally apply the function (F2) described above, assign priority only to the illumination intensity, and display only the illumination intensity in an emphasized manner.

A second example of the above function (F3) will be described. Assume that it is found that the detection result of the object's position is unstable even though detection is performed repeatedly without moving the object. In this case, the setting unit 113 may assign a priority to the setting item of the model setting or switch the priority of the model setting to a higher value in order to prompt re-registration of the model of the pattern matching function for the adjustment purpose of "improving detection accuracy". The state of stability of the detection result of the object's position can be realized by the state analysis unit 115 acquiring the detection result of the object when the image processing program is executed multiple times from the image processing device 20 (image processing unit 122) and analyzing the detection position of the object based on the detection results. For example, in a situation where the position where the object is placed is constant, when the variation in the detection position of the object over multiple times is equal to or greater than a certain value, the state analysis unit 115 may determine that the detection result of the object's position is unstable.

This function allows the priority state (degree of emphasis) of the setting items displayed in the setting area 214 to be switched to contents that reflect the settings and execution status of the image processing program.

FIG. 9 is a flowchart showing the basic operations of a process (setting adjustment method) for adjusting the setting items of an image processing program provided by the programming device 10. This setting item adjustment process is executed under the control of the processor 11 of the programming device 10. First, the setting unit 113 displays a list of adjustment objects selectable in a first area (adjustment object selection area 212) on the setting screen 200 (step S1).

Then, the setting unit 113 displays one or more setting items related to the selected adjustment purpose in the second area (setting area 214) so that the priority according to the selected adjustment purpose can be recognized (step S2). The setting unit 113 then accepts adjustment operations for the one or more setting items in the second area (setting area 214) (step S3). With this configuration, as described above, the operator can easily and efficiently make adjustments that match the adjustment purpose.

In the above embodiment, the setting unit 113 displays in the setting area 214 the setting items that should be preferentially set according to the adjustment purpose in a recognizable manner. In addition to this function, the setting unit 113 may have a function of displaying guidance information for the setting items that should be preferentially set. Information that associates the setting items with the guidance information may be stored in advance in the storage unit 12, and the setting unit 113 may be configured to display the guidance information based on this information.

For example, when the adjustment purpose is "increasing the processing speed," the setting item screen 301a shown in FIG.
- Display a message saying "We recommend shortening the exposure time" as guidance information next to the setting item "Exposure time".
- Display the message "We recommend you reduce the image further" as guidance information next to the setting item "Image reduction ratio".
Examples include:

If the purpose of the adjustment is to "prevent false positives,"
- Regarding the score threshold, "We recommend that you set the score threshold higher."
- Regarding contrast threshold, "We recommend that you increase the contrast threshold."
Such a message may be displayed as guidance information.

If the purpose of the adjustment is to "take measures against undetected errors,"
- Regarding the score threshold, "We recommend lowering the score threshold."
- Regarding contrast threshold, "We recommend lowering the contrast threshold."
Such a message may be displayed as guidance information.

If the purpose of adjustment is to "improve detection accuracy,"
・Regarding model settings, "We recommend re-training to increase model features."
- Regarding image reduction ratio, "We recommend that you set the image reduction ratio as close to 1 as possible."
Such a message may be displayed as guidance information.

Such guidance information may be constantly displayed next to the setting item, for example in a balloon display format, or may be displayed in a pop-up format when the cursor is placed over the setting item. By displaying guidance information in this manner, the worker can adjust the setting items more easily and efficiently.

As described above, according to this embodiment, the operator can easily and efficiently make settings according to the adjustment purpose by simply selecting the adjustment purpose when programming the image processing program.

The functional allocation in the functional block diagram of the programming device 10 and image processing device 20 shown in FIG. 2 is an example, and various examples of functional allocation are possible. For example, there may be a configuration in which some of the functions of the programming device 10 shown in FIG. 2 are arranged in the image processing device 20. There may also be a configuration in which the entire functions of the image processing device 20 are arranged in the programming device 10 (i.e., a configuration in which the programming device 10 and the image processing device 20 are integrated).

The image processing system 100 can also be applied to a robot system configured to detect an object using a visual sensor and handle it using a robot. In this case, the functions of the image processing device 20 and the programming device 10 may be incorporated into the robot control device. Alternatively, the functions of the image processing device 20 and the entire functions of the programming device 10 may be incorporated into a teaching device connected to the robot control device.

The functional blocks in the functional block diagram of the programming device shown in FIG. 2 may be realized by one or more processors of the programming device executing various software stored in a storage device, or may be realized by a configuration mainly consisting of hardware such as an ASIC (Application Specific Integrated Circuit). The functional blocks in the functional block diagram of the image processing device shown in FIG. 2 may be realized by one or more processors of the image processing device executing various software stored in a storage device, or may be realized by a configuration mainly consisting of hardware such as an ASIC (Application Specific Integrated Circuit).

The programs for executing various processes such as the setting item adjustment process of FIG. 9 in the above-mentioned embodiment can be recorded on various computer-readable recording media (for example, semiconductor memory such as ROM, EEPROM, flash memory, magnetic recording media, and optical disks such as CD-ROM and DVD-ROM).

Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible to these embodiments without departing from the gist of the present disclosure, or without departing from the spirit of the present disclosure derived from the contents described in the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-mentioned embodiments, the order of each operation and the order of each process are shown as examples, and are not limited to these. The same applies when numerical values or formulas are used to explain the above-mentioned embodiments.

The following additional notes are provided regarding the above embodiment and modifications.
(Appendix 1)
A programming device (10) comprising: a program creation unit (111) that provides a function for creating an image processing program that executes image processing on an image captured by a visual sensor (30); and a setting unit (113) that accepts settings related to the image processing, wherein the setting unit (113) displays a plurality of adjustment purposes related to the image processing in a selectable manner on a display screen, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a manner that allows a priority according to the selected adjustment purpose to be recognized.
(Appendix 2)
The programming device (10) described in Appendix 1, wherein the setting unit (113) displays one or more setting items to which a priority has been assigned according to the selected adjustment purpose in a manner that is more emphasized than setting items to which the priority has not been assigned.
(Appendix 3)
The programming device (10) according to claim 1 or 2, wherein the setting unit (113) displays one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned such that a setting item having a higher priority corresponding to the selected adjustment purpose is more strongly highlighted.
(Appendix 4)
The programming device (10) according to any one of appendices 1 to 3, further comprising a memory unit (12) that stores priority information that associates, for each of the multiple adjustment purposes, at least one setting item related to the adjustment purpose with a priority for the at least one setting item according to the adjustment purpose, and the setting unit (113) assigns a priority corresponding to the selected adjustment purpose to one or more setting items related to the selected adjustment purpose based on the priority information.
(Appendix 5)
The setting unit (113) switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on the setting or execution state of the image processing program.
(Appendix 6)
The programming device (10) described in Appendix 5, wherein the setting unit (113) switches the priority assigned to one or more setting items related to the selected adjustment objective based on the priority information, based on current setting values of the one or more setting items related to the selected adjustment objective.
(Appendix 7)
The programming device (10) described in Appendix 6, wherein the setting unit (113) lowers the priority or cancels the priority of a setting item that is determined to have no room for adjustment based on current setting values of one or more setting items related to the selected adjustment purpose.
(Appendix 8)
The programming device (10) described in Appendix 5 further comprises a processing time measurement unit (123) capable of measuring the processing time on a functional basis corresponding to each setting item when the image processing program is executed, and the setting unit (113) switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on the measured value of the processing time on a functional basis corresponding to each setting item when the image processing is executed.
(Appendix 9)
The programming device described in Appendix 5, further comprising a state analysis unit (115) that determines the state of the image captured by the visual sensor (30) or the stability state of the detection result of the object included in the image processing as the execution state of the image processing program.
(Appendix 10)
The programming device (10) according to any one of appendices 1 to 9, wherein the setting unit (113) displays a setting screen including a first area displaying the plurality of adjustment objectives and a second area displaying one or more setting items related to the selected adjustment objective, and a display state of the second area is switched according to a priority corresponding to the selected adjustment objective.
(Appendix 11)
The programming device (10) according to any one of appendices 1 to 10, wherein the plurality of adjustment purposes include one or more of: increasing processing speed, taking measures against erroneous detection, taking measures against non-detection, improving detection accuracy, or limiting adjustment items.
(Appendix 12)
The programming device (10) according to any one of appendices 1 to 11, wherein the setting unit (113) additionally displays guidance information corresponding to the selected adjustment purpose for at least one of the one or more setting items displayed so that the priority can be recognized.
(Appendix 13)
A method for adjusting settings related to image processing in a programming device (10), comprising: displaying a plurality of adjustment objectives related to the image processing in a selectable manner in a first area on a display screen; displaying one or more setting items related to one adjustment objective selected from the plurality of adjustment objectives in a second area on the display screen so that a priority according to the selected one adjustment objective can be recognized; and accepting adjustment operations for the one or more setting items in the second area.
(Appendix 14)
A program for causing a computer processor to execute the steps of: displaying a plurality of adjustment purposes related to image processing in a selectable manner in a first area on a display screen; displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that a priority according to the selected one adjustment purpose can be recognized; and accepting adjustment operations for the one or more setting items in the second area.

REFERENCE SIGNS LIST 10 Programming device 11 Processor 12 Memory unit 13 Display unit 14 Operation unit 20 Image processing device 21 Processor 22 Memory unit 30 Visual sensor 31 Lighting device 90 Target object 100 Image processing system 111 Program creation unit 112 Program execution unit 113 Setting unit 114 Processing time acquisition unit 115 Status analysis unit 121 Visual sensor control unit 122 Image processing unit 123 Processing time measurement unit

Claims

a program creation unit that provides a function for creating an image processing program for executing image processing on an image captured by the visual sensor;
a setting unit that accepts settings related to the image processing,
The setting unit, on a display screen,
A plurality of adjustment purposes related to the image processing are displayed in a selectable manner,
A programming device that displays one or more setting items related to one adjustment objective selected from the plurality of adjustment objectives in such a way that a priority according to the selected one adjustment objective can be recognized.
The programming device according to claim 1, wherein the setting unit displays one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned in a more emphasized manner than setting items to which the priority has not been assigned.
The programming device according to claim 1 or 2, wherein the setting unit displays one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned such that the setting item with a higher priority corresponding to the selected adjustment purpose is more strongly emphasized.
a storage unit that stores priority information that associates, for each of the plurality of adjustment purposes, at least one setting item related to the adjustment purpose with a priority for the at least one setting item according to the adjustment purpose;
4. The programming device according to claim 1, wherein the setting unit assigns a priority to one or more setting items related to the selected adjustment purpose based on the priority information, the priority being in accordance with the selected adjustment purpose.
The programming device according to claim 4, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on the setting or execution state of the image processing program.
The programming device according to claim 5, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on the current setting value of the one or more setting items related to the selected adjustment purpose.
The programming device according to claim 6, wherein the setting unit lowers the priority or cancels the priority of a setting item that is determined to have no room for adjustment based on the current setting values of one or more setting items related to the selected adjustment purpose.
a processing time measurement unit capable of measuring a processing time for each function corresponding to each setting item when the image processing program is executed,
6. The programming device according to claim 5, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on a measured value of processing time per function corresponding to each setting item when the image processing is executed.
The programming device according to claim 5, further comprising a state analysis unit that determines the state of the image captured by the visual sensor or the stability state of the detection result of the object included in the image processing as the execution state of the image processing program.
the setting unit displays a setting screen including a first area displaying the plurality of adjustment purposes and a second area displaying one or more setting items related to the selected one adjustment purpose,
10. The programming device according to claim 1, wherein a display state of the second area is switched according to a priority corresponding to the selected one adjustment purpose.
The programming device according to any one of claims 1 to 10, wherein the multiple adjustment purposes include one or more of: increasing processing speed, taking measures against erroneous detection, taking measures against non-detection, improving detection accuracy, or limiting adjustment items.
The programming device according to any one of claims 1 to 11, wherein the setting unit additionally displays guidance information corresponding to the selected adjustment purpose for at least one of the one or more setting items displayed so that the priority can be recognized.
1. A method for adjusting image processing settings in a programming device, comprising:
displaying a plurality of adjustment purposes related to the image processing in a selectable manner in a first area on a display screen;
displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that a priority according to the one selected adjustment purpose can be recognized;
A setting adjustment method, comprising: accepting an adjustment operation for the one or more setting items in the second area.
The computer processor
a step of displaying a plurality of adjustment purposes related to image processing in a selectable manner in a first area on a display screen;
a step of displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that a priority according to the one selected adjustment purpose can be recognized;
a step of accepting an adjustment operation for the one or more setting items in the second area; and a program for executing the step.