WO2022188410A1

WO2022188410A1 - Automatic device assembly control method and apparatus based on manipulator

Info

Publication number: WO2022188410A1
Application number: PCT/CN2021/124934
Authority: WO
Inventors: 陈新; 寇慧; 许俊嘉; 许藤
Original assignee: 佛山隆深机器人有限公司
Priority date: 2021-03-09
Filing date: 2021-10-20
Publication date: 2022-09-15
Also published as: CN112621765B; CN112621765A

Abstract

Disclosed in the present invention are an automatic device assembly control method and apparatus based on a manipulator. The method comprises: obtaining a device installation instruction of a device to be installed; performing visual recognition processing on device parts, on a worktable, of said device, so as to obtain a device part recognition result, and a placement pose of each device part on the worktable; performing coordinate positioning processing on the device parts, on the worktable, of said device, so as to obtain the coordinate position of each device part on the worktable; marking the device parts, which have been subjected to visual recognition, with numbers, so as to obtain device parts marked with numbers; forming a device assembly strategy; and on the basis of the device assembly strategy, controlling a manipulator to perform, on the worktable and by using the device parts of said device, automatic assembly. In the embodiments of the present invention, a device assembly strategy can be formed according to an arrangement situation, on a worktable, of device parts of a device to be installed, thereby realizing efficient and accurate assembly.

Description

A method and device for automatic assembly control of equipment based on manipulator

technical field

The invention relates to the technical field of automatic control, and in particular, to a method and device for automatic assembly control of equipment based on a manipulator.

Background technique

The current automatic assembly is generally in the form of a production line. The automatic assembly production line equipment is generally large in structure, and requires a large area and a large space during installation, and if a small number of assembled products are used , and starting the entire production line will result in a waste of resources; or, when a small amount of equipment products need to be assembled, the manual method is generally used, which requires a lot of human resources and the assembly efficiency is not high.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies of the prior art, and the present invention provides a method and device for automatic assembly control of equipment based on a manipulator, which can realize the formation of equipment according to the placement of the equipment parts of the equipment to be installed on the workbench. Assembly strategies to achieve high-efficiency and accurate assembly.

In order to solve the above technical problems, an embodiment of the present invention provides a method for controlling automatic assembly of equipment based on a manipulator, and the method includes:

Obtain the equipment installation instructions for the equipment to be installed;

Perform visual recognition processing on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera set above the worktable, and obtain the identification results of the equipment parts and the placement posture of each equipment part on the worktable;

Perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

Mark the visually identified equipment parts based on the equipment installation instructions, and obtain the marked equipment parts;

Forming an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

Based on the equipment assembly strategy, the robot is controlled to perform automatic assembly on the workbench using the equipment components of the equipment to be installed.

Optionally, the device installation manual includes a schematic diagram of the installation sequence and a schematic diagram of the installation components involved in each schematic diagram in the schematic diagram of the installation sequence.

Optionally, the obtaining a device installation manual of the device to be installed includes:

Based on the model data of the equipment to be installed, the equipment installation instructions of the equipment to be installed are obtained by matching in the instruction database; or,

When the device installation manual of the device to be installed cannot be matched in the manual database, the device installation manual of the device to be installed is obtained based on the user input.

Optionally, the visual recognition processing is performed on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera disposed above the worktable, and the identification result of the equipment parts and the identification results of each equipment part on the worktable are obtained. Placement poses, including:

constructing a workbench plane coordinate system with workbench image pixels collected by the visual recognition camera on the workbench plane;

When the workbench senses the equipment parts on which the equipment to be installed is placed based on the gravity sensing sensor device, start the visual recognition camera to collect images of the equipment parts on the workbench;

Separate the image foreground and background of the collected image of equipment parts to obtain a foreground image;

Perform target segmentation processing on each equipment component in the foreground image to obtain a target image of each equipment component;

Based on the target image of each equipment component and the equipment component in the equipment installation manual, the similarity target is matched to obtain the equipment component identification result;

The coordinate position of the target image of each equipment component in the workbench plane coordinate system is obtained, and the placement attitude of each equipment component on the workbench is obtained based on the coordinate position of each equipment component.

Optionally, when the equipment components of the equipment to be installed are placed on the workbench, two adjacent equipment components do not block each other.

Optionally, the coordinate positioning process is performed on the equipment components of the equipment to be installed on the workbench, and the coordinate position of each equipment component on the workbench is obtained, including:

The coordinates of the equipment parts of the equipment to be installed on the workbench plane coordinate system constructed by the workbench are obtained, and the coordinate position positioning process is performed to obtain the coordinate position of each equipment part on the workbench.

Optionally, marking the visually identified equipment parts based on the equipment installation instructions, and obtaining the marked equipment parts, including:

Obtain a schematic diagram of the installation components involved in each schematic diagram in the installation sequence diagram;

According to the schematic diagram of the installation components involved in each schematic diagram of the installation sequence, the visually identified equipment components are labeled to obtain the labeled equipment components.

Optionally, the equipment assembly strategy is formed based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components, including:

Obtain the position of the manipulator relative to the plane coordinate system of the worktable and obtain the schematic diagram of the installation sequence in the equipment installation manual;

According to the position of the manipulator relative to the plane coordinate system of the worktable, an equipment assembly strategy is formed by using the schematic diagram of the installation sequence, the placement posture of each equipment component on the workbench, and the labeled equipment components.

Optionally, the control of the manipulator based on the equipment assembly strategy to automatically assemble using the equipment parts of the equipment to be installed on the workbench includes:

A target tracking camera is arranged on the manipulator;

Based on the equipment assembly strategy, the assembled current target components of the equipment components are obtained, and based on the target tracking camera, the assembled current target components are controlled in real time according to the equipment assembly strategy. Automatic assembly.

In addition, an embodiment of the present invention also provides a robot-based device automatic assembly control device, the device comprising:

Obtaining module: used to obtain equipment installation instructions for the equipment to be installed;

Visual recognition module: It is used to visually recognize the equipment parts of the equipment to be installed on the workbench based on the visual recognition camera set above the workbench, and obtain the identification results of the equipment parts and the identification of each equipment part on the workbench. put posture;

Coordinate position positioning module: used to perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

Labeling module: used to label the visually identified equipment parts based on the equipment installation instructions, and obtain the labeled equipment parts;

Assembly strategy forming module: used to form an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

The automatic assembly module is used to control the robot to perform automatic assembly using the equipment parts of the equipment to be installed on the workbench based on the equipment assembly strategy.

In the embodiment of the present invention, an equipment assembly strategy can be formed according to the arrangement of the equipment components of the equipment to be installed on the workbench, so as to achieve high-efficiency and accurate assembly; at the same time, the workbench is miniaturized, and only One set of manipulators, and all assembly is completed by one set of manipulators; suitable for automatic assembly and production of small batches of equipment, reducing labor costs.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic flowchart of a method for controlling automatic assembly of equipment based on a manipulator in an embodiment of the present invention;

FIG. 2 is a schematic structural composition diagram of a robot-based device automatic assembly control device in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of a method for controlling automatic assembly of equipment based on a manipulator in an embodiment of the present invention.

As shown in Figure 1, a method for automatic assembly control of equipment based on a manipulator, the method includes:

S11: Obtain the device installation manual of the device to be installed;

In the specific implementation process of the present invention, the device installation manual includes a schematic diagram of the installation sequence and a schematic diagram of the installation components involved in each schematic diagram in the schematic diagram of the installation sequence.

Further, obtaining the equipment installation instructions of the equipment to be installed includes:

Based on the model data of the equipment to be installed, the equipment installation instructions of the equipment to be installed are obtained by matching in the instruction database; or, when the equipment installation instructions of the equipment to be installed cannot be matched in the instruction database, the equipment installation instructions of the equipment to be installed are obtained based on user input. installation instruction.

Specifically, the device installation manual includes at least a schematic diagram of the installation sequence and a schematic diagram of the installation components involved in each schematic diagram in the schematic installation sequence; and they are all formed in the form of images to facilitate subsequent processing and identification; There are equipment installation instructions for a certain number of equipment models in the database; when obtaining the equipment to be installed, the user needs to obtain the equipment model data of the equipment to be installed, and use the equipment model data of the equipment to be installed to match in the manual database, so as to obtain The device installation manual of the device to be installed; when the device installation manual of the device to be installed cannot be matched in the manual database, the user needs to manually input the device installation manual, and also need to input the device model data of the device to be installed. The device installation manual of the device to be installed, and the device installation manual of the device to be installed input by the user and the corresponding device model data are stored in the manual database.

S12: Perform visual recognition processing on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera set above the worktable, and obtain the identification results of the equipment parts and the placement posture of each equipment part on the worktable;

In the specific implementation process of the present invention, the visual recognition processing is performed on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera disposed above the worktable, and the identification results of the equipment parts and the identification results of each equipment part are obtained. The placement posture on the workbench includes: constructing a workbench plane coordinate system with workbench image pixels collected by the visual recognition camera on the workbench plane; When the equipment parts of the equipment are to be installed, start the visual recognition camera to collect the images of the equipment parts on the workbench; separate the image foreground and background of the collected equipment parts images to obtain a foreground image; Perform target segmentation processing on each equipment component in the device to obtain a target image of each equipment component; perform similarity target matching based on the target image of each equipment component and the equipment components in the equipment installation instructions to obtain equipment components Recognition result; obtain the coordinate position of the target image of each equipment component in the workbench plane coordinate system, and obtain the placement posture of each equipment component on the workbench based on the coordinate position of each equipment component .

Further, when the equipment components of the equipment to be installed are placed on the workbench, two adjacent equipment components do not block each other.

Specifically, when the equipment parts of the equipment to be installed are placed on the workbench, the two adjacent equipment parts do not block each other; before the visual recognition processing, first identify the two adjacent equipment through the practical recognition camera Whether the parts do not block each other; when there is blockage, the manipulator can be controlled to move the equipment parts that block each other so that there is no mutual blockage, and an alarm can be sent to the user to let the user move the blocked equipment parts; This facilitates operations such as visual identification of subsequent equipment components.

Generally, the plane coordinate system of the workbench image needs to be established. In this embodiment, the visual recognition camera collects the image information of the workbench on the workbench plane, and according to the pixel structure of the image information of the workbench Construct the plane coordinate system of the workbench; at the same time, a gravity sensor device is installed on the workbench, and the gravity sensor device is used to sense whether the equipment parts to be installed are placed on the workbench, and the gravity sensor device senses that the gravity of the workbench changes. When considering that the equipment parts to be installed may be placed on the workbench, it is necessary to start the visual recognition camera to collect the images of the equipment parts on the workbench; and after confirming that the equipment parts to be installed are placed on the workbench, the The collected image of equipment parts is separated from the foreground and background of the image to obtain a foreground image; then, the target segmentation process is performed on each equipment part in the foreground image, so as to obtain the target image of each equipment part; after each equipment part is obtained After the component target image, the similarity between the target image of each equipment component and the equipment components in the equipment installation instructions is calculated by the similarity algorithm, and the calculation result with the highest similarity is taken as the equipment component identification result; then each equipment component is obtained. The coordinate position of the component target image in the workbench plane coordinate system, and the placement attitude of each equipment component on the workbench is obtained according to the coordinate position of each equipment component; specifically, through the coordinate system of each equipment component in this coordinate system The coordinate position occupied in the table is used to estimate the placement attitude of the corresponding equipment components on the workbench.

S13: Perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

In the specific implementation process of the present invention, performing coordinate positioning processing on the equipment components of the equipment to be installed on the workbench to obtain the coordinate position of each equipment component on the workbench includes: according to the to-be-installed equipment The equipment components of the equipment perform coordinate position positioning processing on the coordinates of the workbench plane coordinate system constructed by the workbench to obtain the coordinate position of each equipment component on the workbench.

Specifically, it is necessary to obtain the coordinates of the workbench plane coordinate system constructed by the equipment components of the equipment to be installed on the workbench, and then determine the coordinate position according to the coordinates of the workbench plane coordinate system constructed by the equipment components of the equipment to be installed on the workbench. The positioning process is performed to obtain the coordinate position of each equipment component on the worktable.

S14: Mark the visually identified equipment parts based on the equipment installation instructions, and obtain the marked equipment parts;

In the specific implementation process of the present invention, the step of labeling the visually recognized equipment components based on the equipment installation instructions, and obtaining the labeled equipment components includes: obtaining the installation related to each schematic diagram in the installation sequence schematic diagram. Schematic diagram of components; according to the schematic diagram of the installation components involved in each schematic diagram of the installation sequence, label the equipment components after visual identification, and obtain the labeled equipment components.

Specifically, it is necessary to extract the schematic diagrams of the installation components involved in each schematic diagram of the installation sequence schematic diagrams in the equipment installation manual; then, the installation components extraction processing of the schematic diagrams of the installation components involved in each schematic diagram has been performed in sequence to form an orderly Target installation parts, and then carry out corresponding labeling according to the equipment parts after visual identification of the ordered target installation parts, and obtain the labelled equipment parts.

S15: Form an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

In the specific implementation process of the present invention, forming an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components includes: obtaining the The position of the manipulator relative to the plane coordinate system of the worktable and the schematic diagram of the installation sequence in the equipment installation instructions; according to the position of the manipulator relative to the plane coordinate system of the worktable, the schematic diagram of the installation sequence is used, and each equipment component is placed on the worktable. The placement posture and the labeled equipment components form the equipment assembly strategy.

Specifically, by obtaining the position of the manipulator relative to the plane coordinate system of the worktable and obtaining the schematic diagram of the installation sequence in the equipment installation manual, that is, the position information of the manipulator is first obtained through the manipulator, and the position information is coupled to the plane coordinate system of the worktable to determine the relative position of the manipulator. The position of the plane coordinate system of the workbench is obtained by extracting the content in the equipment installation manual to obtain the schematic diagram of the installation sequence; then the position of the manipulator relative to the plane coordinate system of the workbench is based on the schematic diagram of the installation sequence, the placement attitude and label of each equipment component on the workbench The latter equipment parts form an equipment assembly strategy, which is the optimal strategy of the planned assembly path for robot assembly.

S16: Based on the equipment assembly strategy, the robot is controlled to perform automatic assembly on the workbench using the equipment parts of the equipment to be installed.

In the specific implementation process of the present invention, the controlling the manipulator to perform automatic assembly on the workbench using the equipment parts of the equipment to be installed based on the equipment assembly strategy includes: arranging a target tracking camera on the manipulator; The equipment assembly strategy obtains the assembled current target components of the equipment components during assembly, and based on the target tracking camera, automatically controls the posture of the manipulator on the workbench in real time for the assembled current target components according to the equipment assembly strategy. assembled.

Specifically, a target tracking camera is set on the manipulator, and the user of the target tracking camera tracks the target in real time. Because the manipulator moves in real time, the relationship between the manipulator and the target changes in real time. In the process of change, errors may occur. , resulting in poor accuracy of the manipulator during equipment installation, resulting in low installation efficiency, which may lead to product damage or failure to complete automatic installation. Therefore, it is necessary to perform real-time tracking of the target and correct the movement path of the manipulator in time; then assemble according to the equipment. The strategy obtains the assembled current target components of the equipment components during assembly, and automatically assembles the assembled current target components according to the equipment assembly strategy in real time according to the target tracking camera.

Embodiment 2

Please refer to FIG. 2 . FIG. 2 is a schematic structural diagram of a control device for automatic assembly of equipment based on a manipulator according to an embodiment of the present invention.

As shown in Figure 2, a robot-based automatic assembly control device for equipment, the device includes:

Obtaining module 21: used to obtain equipment installation instructions of the equipment to be installed;

Visual recognition module 22: used to perform visual recognition processing on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera set above the worktable, and obtain the identification results of the equipment parts and the identification results of each equipment part on the worktable the placement posture;

Coordinate position positioning module 23: used to perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

Labeling module 24: used to label the visually identified equipment parts based on the equipment installation instructions, and obtain the labeled equipment parts;

Assembly strategy forming module 25: used to form an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

The automatic assembly module 26 is used to control the robot to perform automatic assembly on the workbench using the equipment parts of the equipment to be installed based on the equipment assembly strategy.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: Read-only memory (ROM, ReadOnly Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

In addition, a method and device for automatic assembly control of equipment based on a manipulator provided by the embodiments of the present invention are described in detail above. Specific examples should be used in this paper to illustrate the principles and implementations of the present invention. The description is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. , the contents of this specification should not be construed as limiting the invention.

Claims

A method for automatic assembly control of equipment based on a manipulator, characterized in that the method comprises:

Obtain the equipment installation instructions for the equipment to be installed;

Perform visual recognition processing on the equipment parts of the equipment to be installed on the worktable based on the visual recognition camera set above the worktable, and obtain the identification results of the equipment parts and the placement posture of each equipment part on the worktable;

Perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

Mark the visually identified equipment parts based on the equipment installation instructions, and obtain the marked equipment parts;

Forming an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

Based on the equipment assembly strategy, the robot is controlled to perform automatic assembly on the workbench using the equipment components of the equipment to be installed.
The method for controlling automatic assembly of equipment according to claim 1, wherein the equipment installation manual includes a schematic diagram of an installation sequence and a schematic diagram of the installation components involved in each schematic diagram in the schematic diagram of the installation sequence.
The device automatic assembly control method according to claim 1, wherein the obtaining the device installation manual of the device to be installed comprises:

Based on the model data of the equipment to be installed, the equipment installation instructions of the equipment to be installed are obtained by matching in the instruction database; or,

When the device installation manual of the device to be installed cannot be matched in the manual database, the device installation manual of the device to be installed is obtained based on the user input.
The device automatic assembly control method according to claim 1, wherein the visual recognition processing is performed on the equipment components of the equipment to be installed on the workbench based on the visual recognition camera disposed above the workbench, and the equipment components are obtained. Identify the results and the placement attitude of each equipment component on the workbench, including:

constructing a workbench plane coordinate system with workbench image pixels collected by the visual recognition camera on the workbench plane;

When the workbench senses the equipment parts on which the equipment to be installed is placed based on the gravity sensing sensor device, start the visual recognition camera to collect images of the equipment parts on the workbench;

Separate the image foreground and background of the collected image of equipment parts to obtain a foreground image;

Perform target segmentation processing on each equipment component in the foreground image to obtain a target image of each equipment component;

Based on the target image of each equipment component and the equipment component in the equipment installation manual, the similarity target is matched to obtain the equipment component identification result;

The coordinate position of the target image of each equipment component in the workbench plane coordinate system is obtained, and the placement attitude of each equipment component on the workbench is obtained based on the coordinate position of each equipment component.
The equipment automatic assembly control method according to claim 1 or 4, wherein when the equipment parts of the equipment to be installed are placed on the workbench, two adjacent equipment parts do not block each other.
The equipment automatic assembly control method according to claim 4, wherein the coordinate positioning process is performed on the equipment components of the equipment to be installed on the workbench, and the coordinates of each equipment component on the workbench are obtained. location, including:

The coordinate position positioning process is performed according to the coordinates of the equipment parts of the equipment to be installed on the workbench constructed by the workbench plane coordinate system, and the coordinate position of each equipment part on the workbench is obtained.
The method for controlling automatic assembly of equipment according to claim 2, wherein the visually identified equipment components are labeled based on the equipment installation instructions, and the labeled equipment components are obtained, including:

Obtain a schematic diagram of the installation components involved in each schematic diagram of the installation sequence;

According to the schematic diagram of the installation components involved in each schematic diagram of the installation sequence, the visually identified equipment components are labeled to obtain the labeled equipment components.
The method for automatic assembly control of equipment according to claim 4, wherein said equipment components are formed based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components. Equipment assembly strategy, including:

Obtain the position of the manipulator relative to the plane coordinate system of the worktable and obtain the schematic diagram of the installation sequence in the equipment installation manual;

According to the position of the manipulator relative to the plane coordinate system of the worktable, an equipment assembly strategy is formed by using the schematic diagram of the installation sequence, the placement posture of each equipment component on the workbench, and the labeled equipment components.
The control method for automatic assembly of equipment according to claim 1, wherein the control of the manipulator based on the equipment assembly strategy to perform automatic assembly using the equipment parts of the equipment to be installed on the workbench comprises:

A target tracking camera is arranged on the manipulator;

Based on the equipment assembly strategy, the assembled current target components of the equipment components are obtained, and based on the target tracking camera, the assembled current target components are controlled in real time according to the equipment assembly strategy. Automatic assembly.
A control device for automatic assembly of equipment based on a manipulator, characterized in that the device comprises:

Obtaining module: used to obtain the equipment installation manual of the equipment to be installed;

Visual recognition module: It is used to visually recognize the equipment parts of the equipment to be installed on the workbench based on the visual recognition camera set above the workbench, and obtain the identification results of the equipment parts and the identification of each equipment part on the workbench. put posture;

Coordinate position positioning module: used to perform coordinate positioning processing on the equipment components of the equipment to be installed on the workbench, and obtain the coordinate position of each equipment component on the workbench;

Labeling module: used to label the visually identified equipment parts based on the equipment installation instructions, and obtain the labeled equipment parts;

Assembly strategy forming module: used to form an equipment assembly strategy based on the equipment installation instructions, the placement posture of each equipment component on the workbench, and the labeled equipment components;

The automatic assembly module is used to control the robot to perform automatic assembly using the equipment parts of the equipment to be installed on the workbench based on the equipment assembly strategy.