KR102268579B1 - Apparatus and method for controlling painting robot using motion pattern of painting worker - Google Patents

Abstract

Disclosed are an apparatus and a method for controlling a painting robot using a motion pattern of a painting worker. The method for controlling the painting robot comprises: a step of acquiring a two-dimensional depth image photographing the painting worker by using a depth camera; a step of converting the acquired two-dimensional depth image into a three-dimensional voxel image; a step of generating physical data of the painting worker by using the converted three-dimensional voxel image; a step of acquiring glove sensor data from a glove sensor worn by the painting worker; a step of generating hand data of the painting worker by using the acquired glove sensor data; a step of engaging the physical data with the hand data and generating motion data of the painting worker; a step of learning the motion pattern of the painting worker by using the generated motion data, and generating the motion pattern data by learning the motion pattern; and a step of controlling the painting robot by using the generated motion pattern data. The present invention aims to provide an apparatus and a method for controlling a painting robot using a motion pattern of a painting worker, which are able to lower the possibility of errors in a painting process.

Description

Apparatus and method for controlling painting robot using motion pattern of painting worker

The present invention relates to an apparatus and method for controlling a painting robot using a motion pattern of a painting worker.

In general, since the car body painting process in the automobile production process requires a high degree of rust prevention and aesthetics, it has a complex process compared to general painting, Medium and top coat processes are basic. That is, the car body painting process is to prevent corrosion of the car body after the car body is completed, and to enhance the marketability by improving the aesthetics as well as sound insulation.

This car body painting process completely removes dirt and oils and fats adhering to the inner and outer plates of the car body, and at the same time performs a pre-treatment process of forming a phosphate film that improves the corrosion resistance of the steel plate and adhesion to the electrodeposition paint. Then, the car body that has passed through the drying furnace of the pre-treatment process is fully immersed in the electrodeposition paint, and the electrodeposition paint is applied uniformly to the inside of the car body as well as the outer plate. Next, after the process of applying the sealer to the joints, etc., an intermediate process for imparting chipping resistance, smoothness, weather resistance, etc. of the paint and an intermediate drying process for drying the intermediate paint in a drying tank are performed. Next, after forming a coating film through a topcoat base and a clear process that gives aesthetic glossiness, a topcoat drying process of drying the topcoat in a drying furnace is performed again.

After that, when the painting on the vehicle body is completed, an exterior inspection is performed to inspect the exterior of the vehicle body through a visual inspection. In this exterior inspection process, dust, dirt, scratches, etc. on the painted surface are inspected. If it is necessary to correct the body painting through visual inspection, repaint it through the topcoat base and clear process, or correct it through a correction process such as polishing work, and then put it into the decoration factory.

As described above, since the painting process is a complicated process, there is a high possibility of occurrence of defects for various reasons. Accordingly, there is a problem in that a high level of technology for detecting defects is required, and a lot of manpower is required to handle them.

Republic of Korea Patent Publication No. 10-1584080 (2016.01.05)

An object of the present invention is to provide an apparatus and method for controlling a painting robot using the motion pattern of a painting operator, which acquires and learns the motion pattern of a skilled painting operator, and controls the painting robot using the learned motion pattern.

According to one aspect of the present invention, a painting robot control method performed by a painting robot control apparatus using a motion pattern of a painting worker is disclosed.

A painting robot control method according to an embodiment of the present invention includes acquiring a two-dimensional depth image obtained by photographing the painting worker using a depth camera, and converting the obtained two-dimensional depth image into a three-dimensional voxel image. converting, generating body data of the painting worker using the converted three-dimensional voxel image, obtaining glove sensor data from a glove sensor worn by the painting worker, using the obtained glove sensor data to generate the painting worker's hand data, combining the body data and the hand data to generate motion data of the painting worker, learning the motion pattern of the painting worker using the generated motion data, , generating motion pattern data through the motion pattern learning and controlling the painting robot using the generated motion pattern data.

The generating of the body data includes extracting the body of the painting worker from the 3D voxel image, setting each joint as a key point in the extracted body, and estimating the position of each set key point. The body data including three-dimensional coordinates of each key point are generated.

In the generating of the hand data, the hand data including three-dimensional coordinates of each joint of the hand is generated using the glove sensor data.

In the generating of the motion data, the motion data in which the body and the hand are combined is generated by matching the three-dimensional coordinates of the body and the hand of the painting worker.

According to another aspect of the present invention, an apparatus for controlling a painting robot using a motion pattern of a painting worker is disclosed.

A painting robot control apparatus according to an embodiment of the present invention includes a depth camera, a glove sensor, a memory for storing instructions, and a processor for executing the instructions, wherein the instructions photograph the painting worker using the depth camera. obtaining a two-dimensional depth image, converting the obtained two-dimensional depth image into a three-dimensional voxel image, and generating body data of the painting worker using the converted three-dimensional voxel image Step, obtaining glove sensor data from the glove sensor worn by the painting worker, generating hand data of the painting worker using the obtained glove sensor data, combining the body data and the hand data Generating the motion data of the painting worker, learning the motion pattern of the painting worker using the generated motion data, generating motion pattern data through the motion pattern learning, and using the generated motion pattern data A painting robot control method including the step of controlling the painting robot using

An apparatus and method for controlling a painting robot using a motion pattern of a painting worker according to an embodiment of the present invention is a painting process by acquiring and learning the motion pattern of a skilled painting operator and controlling the painting robot using the learned motion pattern. It is possible to lower the possibility of defects in the painting process and to perform the painting process with a minimum of manpower.

1 is a flowchart illustrating a painting robot control method performed by an apparatus for controlling a painting robot using a motion pattern of a painting worker according to an embodiment of the present invention.
2 to 7 are views for explaining a method of controlling a painting robot using a motion pattern of a painting worker according to an embodiment of the present invention.
8 is a diagram schematically illustrating the configuration of a painting robot control apparatus using a motion pattern of a painting worker according to an embodiment of the present invention.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a method for controlling a painting robot performed by a painting robot control apparatus using a motion pattern of a painting worker according to an embodiment of the present invention, and FIGS. 2 to 7 are motions of a painting operator according to an embodiment of the present invention It is a diagram for explaining a method of controlling a painting robot using a pattern. Hereinafter, a method of controlling a painting robot using a motion pattern of a painting worker according to an embodiment of the present invention will be described with reference to FIG. 1 , with reference to FIGS. 2 to 7 .

In step S110, the painting robot control device acquires a two-dimensional depth image of the painting worker using the depth camera.

Here, the depth camera measures the distance to the object and outputs it as an image. For example, the depth camera may acquire a depth image of a captured scene using a time-of-flight (TOF) method. The depth image may be acquired as shown in FIG. 2B . FIG. 2A shows an actual image obtained by a general camera, and FIG. 2B shows a depth image obtained by a depth camera.

In step S120, the apparatus for controlling the painting robot converts the obtained 2D depth image into a 3D voxel image.

For example, referring to FIG. 3 , a 2D depth image is a 3D voxel image including shapes of a polygonal mesh, a point cloud, and a volumetric pixel (Voxel) using pixel data. can be converted to That is, the 3D voxel image may be formed in the form of a polygon mesh by connecting a set of 3D point groups including spatial information. In addition, the 3D voxel image may be generated by defining a 3D area through spatial division or the like, and converting input 3D data into a standardized voxel space through the defined 3D area.

In step S130, the painting robot control apparatus generates body data of the painting worker by using the converted 3D voxel image.

That is, referring to FIGS. 4 and 5 , the painting robot control apparatus extracts the body of the painting worker from the three-dimensional voxel image, sets each joint as a key point from the extracted body, and sets the position of each key point. By estimating the three-dimensional coordinates of each key point can be generated as body data. Through this, the apparatus for controlling the painting robot may generate body data including information such as the position, posture, movement, and angle of the body from the 3D voxel image.

In step S140, the painting robot control device acquires the glove sensor data from the glove sensor worn by the painting worker. Here, the glove sensor includes a motion sensor mounted on each part set in advance, and can measure the movement of the hand using this. For example, the glove sensor data may be a measurement of hand movement.

In step S150, the painting robot control device generates hand data of the painting worker by using the acquired glove sensor data.

That is, referring to FIG. 6 , the painting robot control apparatus may generate three-dimensional coordinates of each joint of the hand as hand data by using the glove sensor data. Through this, the painting robot control apparatus may generate hand data including information such as movement and angle of the hand from the glove sensor data.

In step S160, the painting robot control device generates motion data of the painting worker by combining the generated body data and hand data.

That is, referring to FIG. 7 , the apparatus for controlling the painting robot may generate motion data in which the body and the hand are combined by matching the three-dimensional coordinates of the body and the hand.

In step S170, the painting robot control device learns the motion pattern of the painting worker using the generated motion data, and generates the motion pattern data through the motion pattern learning.

In step S180, the painting robot control apparatus controls the painting robot using the generated motion pattern data.

For example, each joint of the painting robot may be matched with a preset joint among the joints of the body and hand of the painting worker. And, the motion pattern data may be generated for each painting object. Therefore, the painting robot control device controls the movement of the painting robot based on the joints of the painting robot according to the painting target based on the motion pattern data generated through learning, so that the painting robot can use the same motion pattern as a skilled painting worker. It can be controlled to perform the painting operation.

8 is a diagram schematically illustrating the configuration of a painting robot control apparatus using a motion pattern of a painting worker according to an embodiment of the present invention.

Referring to FIG. 8 , a painting robot control apparatus using a motion pattern of a painting worker according to an embodiment of the present invention includes a processor 10 , a memory 20 , a depth camera 30 , a glove sensor 40 , and a communication unit ( 50) and an interface unit 60 .

The processor 10 may be a CPU or a semiconductor device that executes processing instructions stored in the memory 20 .

Memory 20 may include various types of volatile or non-volatile storage media. For example, the memory 20 may include ROM, RAM, or the like.

For example, the memory 20 may store instructions for performing the painting robot control method using a motion pattern of a painting worker according to an embodiment of the present invention.

The depth camera 30 measures the distance to the object and outputs it as an image. For example, the depth camera may acquire a depth image of a captured scene using a time-of-flight (TOF) method.

The glove sensor 40 includes a motion sensor mounted on each part set in advance, and can measure the movement of the hand using this.

The communication unit 50 is a means for transmitting and receiving data with other devices through a communication network.

The interface unit 60 may include a network interface and a user interface for accessing a network.

On the other hand, the components of the above-described embodiment can be easily grasped from a process point of view. That is, each component may be identified as a respective process. In addition, the process of the above-described embodiment can be easily understood from the point of view of the components of the apparatus.

In addition, the technical contents described above may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The above-described embodiments of the present invention have been disclosed for the purpose of illustration, and various modifications, changes, and additions will be possible within the spirit and scope of the present invention by those skilled in the art having ordinary knowledge of the present invention, and such modifications, changes and additions should be regarded as belonging to the following claims.

10: Processor
20: memory
30: depth camera
40: glove sensor
50: communication department
60: interface unit

Claims (5)

In the painting robot control method performed by the painting robot control device using the painting worker's motion pattern,
obtaining a two-dimensional depth image of the painting worker by using a depth camera;
converting the obtained 2D depth image into a 3D voxel image;
generating body data of the painting worker by using the converted 3D voxel image;
acquiring glove sensor data from the glove sensor worn by the painting worker;
generating hand data of the painting worker using the acquired glove sensor data;
generating motion data of the painting worker by combining the body data and the hand data;
learning the motion pattern of the painting worker using the generated motion data, and generating motion pattern data through the motion pattern learning; and
and controlling the painting robot using the generated motion pattern data.
According to claim 1,
The step of generating the body data comprises:
Extracting the body of the painting worker from the 3D voxel image, setting each joint as a key point in the extracted body, estimating the position of each set key point, and including 3D coordinates of each key point A painting robot control method, characterized in that generating the body data.
According to claim 1,
The step of generating the hand data includes:
A painting robot control method, characterized in that by using the glove sensor data to generate the hand data including the three-dimensional coordinates of each joint of the hand.
According to claim 1,
The step of generating the motion data comprises:
The painting robot control method, characterized in that by matching the three-dimensional coordinates of the body and the hand of the painting worker to generate the motion data in which the body and the hand are combined.
In the painting robot control device using the motion pattern of the painting worker,
depth camera;
glove sensor;
memory to store instructions; and
A processor that executes the instructions,
The command is
obtaining a two-dimensional depth image of the painting worker using the depth camera;
converting the obtained 2D depth image into a 3D voxel image;
generating body data of the painting worker by using the converted 3D voxel image;
obtaining glove sensor data from the glove sensor worn by the painting worker;
generating hand data of the painting worker using the acquired glove sensor data;
generating motion data of the painting worker by combining the body data and the hand data;
learning the motion pattern of the painting worker using the generated motion data, and generating motion pattern data through the motion pattern learning; and
A painting robot control apparatus, characterized in that performing a painting robot control method comprising the step of controlling the painting robot using the generated motion pattern data.

Images