KR101801279B1 - Surgical robot system, control method thereof, and recording medium thereof - Google Patents

Abstract

In order to prevent an accident or a harm caused by an unintentional malfunction of a surgical operator during surgery while minimizing disturbance to a surgical operation of the operator, the present invention includes a slave robot including a robot arm that has a multi- A user operation section for generating a control signal for operating the arm; And a reaction force control unit for performing a force feedback function to apply a predetermined reaction force to deform or increase the reaction force detected by the robot arm to a certain degree with respect to an operation of the user operation unit And provides a robot system.

Description

Technical Field [0001] The present invention relates to a surgical robot system, a control method thereof, and a recording medium on which the robot system,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a surgical robot system, a control method thereof, and a recording medium on which the robot arm is controlled. More particularly, And a force feedback function for applying a predetermined reaction force to deform or increase the reaction force detected by the robot arm to a certain degree with respect to the robot arm, a control method thereof, and a recording medium on which the robot system is recorded.

The surgical robot refers to a robot having a function of substituting surgical operation performed by a surgeon. Such a surgical robot has an advantage that it can perform accurate and precise operation compared to a human and can perform a remote operation.

Surgical robots currently being developed around the world include bone surgery robots, laparoscopic surgery robots, and stereotactic robots. Here, a laparoscopic surgical robot is a robot that performs minimally invasive surgery using laparoscopic and small surgical instruments.

Laparoscopic surgery is an advanced surgical technique that inserts a laparoscope, which is an endoscope for inserting a 1 cm hole in the umbilicus and looks into the abdomen. A recent laparoscope is equipped with a computer chip to provide a sharper, more magnified image than is seen with the naked eye, and the laparoscopic surgical instruments specially designed by looking at the screen through the monitor have developed enough to perform any operation. In addition, laparoscopic surgery has the advantage of being less complicated than open surgery, initiating treatment within a very short time after surgery, and maintaining the physical strength and immune function of the surgical patients, have.

On the other hand, the surgical robot system generally comprises a master robot and a slave robot. When the operator manipulates a steering lever (for example, a steering wheel) provided on the master robot, the surgical tool held by the robot arm is coupled to the robot arm of the slave robot.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

The present invention relates to a robot arm that applies a predetermined reaction force to deform or enhance a reaction force detected by a robot arm with respect to an operation of a user's operating unit when the robot arm operates within a predetermined predetermined area or performs predetermined predetermined operations A surgical robot system that performs a force feedback function, a control method thereof, and a recording medium on which the robot system is recorded.

The present invention relates to a slave robot including a robot arm having multiple degrees of freedom and being driven, a user operation unit for generating a control signal for operating the robot arm, And a reaction force control unit for performing a force feedback function to apply a predetermined reaction force to deform or increase the reaction force detected by the robot arm to a certain degree with respect to an operation of the user operation unit And provides a robot system.

In the present invention, it is preferable that the reaction force control unit is configured to operate the robot arm such that the robot arm operates within a predetermined area, or the robot arm moves in, out, or approaches a predetermined area, It is possible to perform a force feedback function of applying the predetermined reaction force to the operation of the user's operating unit.

Here, the predetermined region may be the entire operation site being operated.

Here, the reaction force control unit may be configured to control the operation of the user's operating unit so that the gravity direction on the screen displayed on the display member displaying the image of the surgical site, or the gravity direction Mode can be performed.

Here, the reaction force control unit may perform a high frictional force mode in which an additional reaction force is applied to the operation of the user's operating unit in the direction opposite to the traveling direction of the robot arm.

Here, the reaction force control unit may set the reaction force applied to the user's operating unit to be greater than a predetermined reference value for the acceleration operation of the robot arm, and set the reaction force applied to the user's operation unit to a predetermined reference value Or less than the reference value.

In the present invention, the slave robot may further include a surgical endoscope for scanning a surgical site.

Here, the predetermined region may include at least a part of at least one region in which the tissue and the organ calculated from the image of the surgical site photographed from the surgical endoscope are located.

Here, the master robot further includes a display member for displaying an image photographed through the surgical endoscope, and the predetermined region may be any region selected by a user input on the display member.

Here, the reaction force control unit may be configured to control the operation of the user's operating unit so that the gravity direction on the screen displayed on the display member displaying the image of the surgical site, or the gravity direction Mode can be performed.

Here, the reaction force control unit may perform a high frictional force mode in which an additional reaction force is applied to the operation of the user's operating unit in the direction opposite to the traveling direction of the robot arm.

Here, the reaction force control unit may control the robot arm such that when the robot arm operates in the calculated or selected predetermined area, or when the robot arm moves in, out, or approaches the selected predetermined area, So as to apply an additional reaction force.

Here, the reaction force control unit may control the operation of the user manipulation unit to apply different reaction forces to the plurality of regions when the region includes a plurality of regions.

In the present invention, when the robot arm is separated from a work area of the robot arm extracted from a trajectory of a position and velocity of the robot arm for a predetermined period of time, It is possible to control to apply a predetermined additional reaction force to the operation of the operation portion.

In the present invention, the reaction force control section may control the robot to apply a predetermined additional reaction force to the operation of the user operation section when the acceleration of the robot arm is equal to or greater than a predetermined reference value.

In the present invention, the display member may include a stereoscopic display device.

According to another aspect of the present invention, there is provided a control method of a surgical robot system including a master robot and a slave robot, the method comprising: displaying an image photographed through a surgical endoscope of the slave robot on a display member of the master robot; Whether or not the robot arm of the slave robot operates within a predetermined area or whether the robot arm is pulled in, drawn out, or accessed for a predetermined area, or whether the robot arm performs predetermined predetermined operations ; And when the robot arm operates within a predetermined area or when the robot arm moves in, out, or approaches a predetermined area, or when the robot arm performs predetermined operations, And performing a force feedback function to apply a predetermined reaction force to deform or increase the reaction force detected by the robot arm to a certain degree with respect to the operation of the operation unit.

In the present invention, the step of applying a predetermined reaction force to the operation of the user operating unit may include a step of applying a predetermined reaction force to the operation of the user's operating unit in accordance with the actual gravity direction, the gravity direction on the screen displayed on the display member, And may be a high gravity mode that applies an additional reaction force in the arbitrarily set imaginary gravity direction.

In the present invention, the step of applying a predetermined reaction force to the operation of the user's operating unit may include a step of applying a reaction force to the manipulation of the user's operating unit in a high frictional force mode that applies an additional reaction force in the direction opposite to the advancing direction of the robot arm .

In the present invention, the predetermined region may include at least a part of at least one region in which the tissue and organs calculated from the image of the surgical site photographed from the surgical endoscope are located.

In the present invention, the predetermined area may be one or more arbitrary areas selected by user input on the display member.

Here, the step of applying a predetermined reaction force to the operation of the user's operation unit may include: operating the robot arm within the calculated or selected predetermined area; , It is possible to control so as to apply a predetermined additional reaction force to the operation of the user operation portion.

Here, the step of applying a predetermined reaction force to the operation of the user operating unit may control the operation of the user operating unit to apply different reaction forces to the plurality of areas when the area includes a plurality of areas have.

In the present invention, the step of applying a predetermined reaction force to the manipulation of the user manipulation unit may include the step of: applying a predetermined reaction force to the manipulation of the robot manipulator from the manipulation area of the robot arm extracted from a trajectory When the arms are spaced apart from each other by a predetermined distance, it is possible to perform control so as to apply a predetermined additional reaction force to the operation of the user operation portion.

In the present invention, the step of applying a predetermined reaction force to the operation of the user's operation unit may be controlled so as to apply a predetermined additional reaction force to the operation of the user's operation unit when the acceleration of the robot arm is a predetermined reference value or more have.

According to another aspect of the present invention, a program of instructions executable by a digital processing apparatus to perform a control method of a surgical robot system described in any one of the above is tangibly embodied and readable by a digital processing apparatus A recording medium on which a program is recorded is provided.

According to the present invention, it is possible to obtain an effect of preventing an accident or a harm caused by an unintentional malfunction of a surgical operator during surgery while minimizing disturbance to a surgical operation of the operator.

1 is a plan view showing the entire structure of a surgical robot system according to an embodiment of the present invention.
2 is a perspective view showing a master robot of the surgical robot system of FIG.
3 is a block diagram schematically showing the configuration of a master robot and a slave robot according to an embodiment of the present invention.
FIG. 4 is a view showing a screen display mode in which an image photographed by a laparoscope is output through a display member. FIG.
FIG. 5 is a diagram showing a screen display mode in which a tissue and a long-term area are superimposed and displayed on the display member of FIG. 4. FIG.
FIG. 6 is a view showing a screen display mode in which a user selection area is superimposed on a display member of FIG. 4; FIG.
7 is a flowchart schematically showing a control method of a surgical robot system according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art to which the present invention pertains. The present invention may be embodied in many different forms and is not limited to the embodiments described herein.

Herein, the present invention is a technical idea that can be used universally for surgeries using surgical endoscopes (for example, laparoscopic, thoracoscopic, arthroscopic, non-rigid, etc.), but in describing the embodiments of the present invention, For convenience, a laparoscope is used as an example.

FIG. 1 is a plan view showing an overall structure of a surgical robot system according to an embodiment of the present invention, and FIG. 2 is a perspective view showing a master robot of the surgical robot system of FIG.

1 and 2, the surgical robot system 1 includes a slave robot 200 for performing surgery on a patient lying on the operating table, a master robot 100 for allowing the operator to remotely operate the slave robot 200, . The master robot 100 and the slave robot 200 are not necessarily separated from each other by a separate device that is physically independent and may be integrated into one unit.

The master robot 100 includes an operation lever 110 and a display member 120 and the slave robot 200 includes a robot arm 210 and a laparoscope 220.

In detail, the master robot 100 is provided with an operation lever 110 so that the operator can grasp and operate the operator's hands, respectively. The operation lever 110 may be implemented by two or more handles as illustrated in FIGS. 1 and 2. An operation signal according to a manipulation of a handle of a surgical operator may be transmitted to a slave robot 200 via a wired or wireless communication network And the robot arm 210 is controlled. That is, the operation of the robot arm 210, such as the movement, rotation, cutting, etc., of the robot arm 210 can be performed by operating the handle of the operator.

For example, the operator can manipulate the slave robot arm 210, the laparoscope 220, and the like using a handle-shaped operation lever. Such an operation lever may have various mechanical configurations according to its operation mode, and may include a master handle for operating operations of the slave robot arm 210 and the laparoscope 220 and the like, and a master robot Such as various input devices such as a joystick, a keypad, a trackball, and a touch screen, which are added to the slave robot 100, for operating the robot arm 210 and / or other surgical equipment of the slave robot 200 . Here, the operation lever 110 is not limited to the shape of the handle, and can be applied without any limitation as long as it can control the operation of the robot arm 210 through a network such as a wired or wireless communication network.

An image photographed through the laparoscope 220 is displayed as an image image on the display member 120 of the master robot 100. In addition, the display member 120 may have a touch screen function to provide a function of selecting a predetermined region requiring reaction force control in an image photographed through the laparoscope 220. Further, the display member 120 may be provided as a stereoscopic display device so that an observer can feel a three-dimensional feeling of life and realism.

Here, the display member 120 may be constituted by one or more monitors, and information necessary for operation may be displayed on each monitor individually. Although FIGS. 1 and 2 illustrate the case where the display member 120 includes three monitors, the number of monitors can be variously determined according to the type and kind of information required to be displayed.

Meanwhile, the slave robot 200 may include one or more robot arms 210. Generally, a robot arm has a function similar to a human arm and / or wrist, and means a device capable of attaching a predetermined tool to a wrist part. In the present specification, the robot arm 210 may be defined as a concept including both upper and lower components such as upper arm, lower arm, wrist, and elbow, and a surgical instrument coupled to the wrist. The robot arm 210 of the slave robot 200 may be configured to be driven with multiple degrees of freedom. For example, the robot arm 210 may include a surgical instrument inserted into a surgical site of a patient, a swinging drive unit that rotates the surgical instrument in a yaw direction according to a surgical position, a pitch direction , A pitch driving part for rotating the surgical instrument, a feed driving part for moving the surgical instrument in the longitudinal direction, a rotation driving part for rotating the surgical instrument, and a surgical instrument driving part installed at the end of the surgical instrument for cutting or cutting the surgical lesion. . However, the configuration of the robot arm 210 is not limited thereto, and it should be understood that these examples do not limit the scope of the present invention. Here, a detailed description of the actual control process, such as rotation and movement of the robot arm 210 in the corresponding direction by operating the operation lever 110 by the operator, will be omitted.

The slave robot 200 may be used more than once to perform a surgery on a patient and the laparoscope 220 for displaying a surgical site through a display member 120 as an image image may be implemented as an independent slave robot 200 It is possible. Further, as described above, the embodiments of the present invention can be widely used in surgeries in which a variety of surgical endoscopes other than laparoscopes (e.g., thoracoscopic, arthroscopic, non-circumferential, etc.) are used.

3 is a block diagram schematically showing the configuration of a master robot and a slave robot according to an embodiment of the present invention.

3, the master robot 100 includes an image input unit 130, a screen display unit 140, a user operation unit 150, an operation signal generation unit 160, a reaction force control unit 170, and a control unit 180 do. The slave robot 200 includes a robot arm 210 and a laparoscope 220.

Here, the surgical robot system 1 according to an embodiment of the present invention may be configured to limit the operation of the robot arm 210 of the slave robot 200 in operation to a predetermined range, The present invention is characterized in that the safety during operation is improved by applying a predetermined reaction force in which the reaction force is deformed or augmented to a certain degree.

In detail, in a surgical robot system having a master-slave structure, a malfunction which may cause tissue or organ damage due to unintentional movement of a slave robot in operation is caused by two main causes. The first cause is due to errors in the system itself such as control and communication, and the second cause is the inadvertent or inadvertent malfunction of the operator who is manipulating the slave robot through the master robot. will be. In the former case, technical considerations for enhancing the safety of the system itself (for example, duplication of control systems, automatic correction of communication errors, implementation of countermeasures against electromagnetic noise, etc.) may be improved. On the contrary, in the latter case, since the operation operation intended by the practitioner and the unintended operation due to the carelessness of the operator are mixed during the operation of the practitioner, it is possible to prevent malfunction Implementation of a mechanism to do so has been requested.

In order to prevent the unintentional movement of the operator, a virtual fixture concept and force feedback concept for telepresence have been proposed.

First, the virtual fixed wall does not actually exist but virtually sets the restricted area or the restricted path, and restricts the movement of the robot arm so as not to deviate from the set area or the path, and gives a virtual reaction force to obstruct the movement. For example, a virtual wall (i.e., a virtual fixture) may be set at the interface and random stiffness (i.e., a weight for reaction force) may be arbitrarily set so that the robot arm never crosses a virtual wall, or It is possible to control the robot arm in such a manner as to allow the robot arm to push the wall to some extent but to allow the robot arm to be operated only by giving a force greater than the actual force. If the weight is 1 or an identity matrix, the robot is unconditionally followed by a speed command designated as in the general control. If the weight matrix is an identity matrix, The effect of giving an arbitrary restriction condition to the movement in a specific direction in space is obtained to obtain the virtual fixed wall effect.

On the other hand, in a robot system of a master-slave structure or a telemanipulation structure, a reaction force felt by a slave robot at a remote location is felt on the side of the master robot, or in a wider sense, The senses are collectively referred to as telepresence. In general, the main purpose of such remote display control is to maximize "transparency," ie, to make the user feel closest to the remote feel. In the case of a surgical robot, the remote display technique is implemented with the aim of transmitting the reaction force received by the slave robot to the master robot in a transparent manner as much as possible when the slave robot hits the tissue or peripheral organs during operation.

The concept of so-called force-feedback control can be applied in order to implement such a remote display technique. The basic concept of force-feedback control is to feedback the reaction force value sensed by a sensor or the like and the present position value of the attachment of the robot arm to the side of the master robot in the attachment of the robot arm of the slave robot, Taking into account its own kinematic structure, the reaction force felt at the most distal end of the user interaction is controlled to be as close as possible to the feedback value from the slave robot. Generally, the general feedback control structure when a robot and an object are present together is extended, and a concept of a so-called bilateral control structure in which a local system and a remote environment of a remote place are connected to each other to become a feedback control structure . Here, the sensor for measuring the reaction force value may be attached to the robot arm of the slave robot, or may be installed at a position spaced apart from the attachment of the robot arm for easy installation, May be measured.

However, the virtual fixed wall concept and the force feedback concept for preventing unintentional movement of the operator have a certain limit.

In detail, the virtual fixed wall concept is focused on induction of motion of a robot, and it mainly aims at prevention of invasion (protection purpose) in a specific area or prevention of departure from a specific area (induction purpose). However, in robotic surgery, it is necessary to set a fixed restricted area such as an absolute invasion restricted area or a departure-avoiding area. However, in order to perform a safe operation in the area where the operation is performed and the robot arm's operation path, There is a need to be able to arbitrarily set an operation restriction area, an operation restriction range, an operation to be restricted, and the like.

In addition, force feedback technology is aimed at transparent remote presence, and includes techniques for control stability, convergence guarantee, and so on. However, in the surgical robot, not only the transparent force feedback but also the virtual environment of the augmented reality type based on the spatial environment information of various actual surgery sites related to the above-mentioned purpose of the safety mechanism, Feedback control, which includes reaction force control for interaction with the force feedback mechanism.

That is, in order to adaptively and efficiently cope with the situation and necessity during the surgical operation by expanding from the implementation of the space limitation area arbitrarily set by the user or the transparent feedback on the reaction force detected by the robot arm of the slave robot, And the interaction with the augmented reality space in which the three - dimensional spatial information is recognized.

For this purpose, the surgical robot system 1 according to an embodiment of the present invention is configured to operate the robot arm such that the robot arm operates within a predetermined area, or the robot arm moves in, out, or approaches a predetermined area, And performs a force feedback function to apply a predetermined reaction force to the operation of the user's operation unit when the arm performs predetermined predetermined operations. Here, the predetermined reaction force refers not to the actual reaction force sensed by the robot arm, but to a reaction force that deforms or augments the reaction force sensed by the robot arm to a certain degree. The reaction force that is deformed or augmented to a certain degree is given to the user, And to improve the safety during surgery by drawing attention of the user.

Referring again to FIG. 3, each component of the master robot 100 will be described in detail.

The image input unit 130 receives an image photographed through a camera provided in the laparoscope 220 of the slave robot 200 through a wired or wireless communication network.

The screen display unit 140 outputs visual images corresponding to the images received through the image input unit 130 as visual information. In addition, when biometric information of the subject is input from the slave robot 200, the screen display unit 140 can output more information corresponding to the biometric information. The screen display unit 140 may display the patient's related image data (for example, an X-ray image, a computed tomography (CT) image, a magnetic resonance imaging (MRI) A reconstructed three-dimensional image data or a numerical model based on computed tomography (CT) image data). Here, the screen display unit 140 may be implemented in the form of a display member (see 120 in FIG. 2), and an image processing process for causing the received image to be output as an image image through the screen display unit 140 may be performed by a control unit 180, a reaction force control unit 170, or an image processing unit (not shown).

Here, the screen display unit 140 (i.e., the display member 120 of FIG. 2) may be provided as a stereoscopic display device. In detail, the stereoscopic display device applies depth information to a two-dimensional image by applying a stereoscopic technique, and uses an image display device (not shown) to allow the observer to feel three- Quot; Herein, the surgical robot system 1 according to an embodiment of the present invention includes a stereoscopic display device as a screen display unit, thereby providing a more realistic virtual environment to a user.

The user manipulation unit 150 is a means for allowing an operator to manipulate the position and function of the robot arm 210 of the slave robot 200. 2, the user operation unit 150 may be formed in the form of a handle-like operating member (see 110 in FIG. 2), but the shape is not limited thereto and may be modified into various shapes for achieving the same purpose . Further, for example, a part of the finger may be formed in a handle shape and the other part may be formed in a different shape such as a clutch button, and a finger insertion tube or insertion More rings may be formed.

The user operation unit 150 may provide a predetermined reaction force to the operation input by the user under the control of the reaction force control unit 170 and the control unit 180 connected thereto. For example, when the user grasps the user manipulation part 150 and the user manipulation part 150 is pushed in a certain direction, the user manipulation part 150 provides a reaction force of a certain magnitude in a direction opposite to the direction in which the user pushes the manipulation part 150 . As a result, the user must exert a greater force than usual in order to operate the user's operating unit 150, and this can draw attention to the possibility of malfunction of the user.

Or the user operating unit 150 may emit a predetermined alarm message for the operation input by the user under the control of the reaction force control unit 170 and the control unit 180 connected thereto. For example, when the user manipulates the user's operation unit 150 while the user grasps the user's operation unit 150 and the robot arm 210 enters the predetermined restricted area, the user's operation unit 150 may generate a predetermined alarm the user can divert an alarm message to the user and thereby alert the user of the possibility of malfunction.

The operation signal generating unit 160 generates an operation signal corresponding to the operation of the user operation unit 150 by the operator in order to move the robot arm 210 and / or the laparoscope 220, And transmits it to the slave robot 200. The operation signal can be transmitted and received through a wired or wireless communication network as described above.

The reaction force control unit 170 controls the reaction force of the robot arm 210 so that the robot arm 210 is operated within a predetermined area or the robot arm is moved in, out, or approach a predetermined region, or the robot arm performs predetermined operations A force feedback function for applying a predetermined reaction force to the operation of the user operation unit 150 is performed. Specific functions, various detailed configurations, and the like of the reaction force control unit 170 will be described in detail with reference to related drawings hereinafter.

The control unit 180 controls the operation of each component so that the above-described functions can be performed. The control unit 180 may convert the image input through the image input unit 130 into an image to be displayed through the screen display unit 140. The control unit 180 may also transmit the image input from the image input unit 130 to the reaction force control unit 170 and execute the operation implementation signal generated by the reaction force control unit 170. [

Hereinafter, the configuration and functions of the reaction force control unit 170 will be described in more detail. As described above, the reaction force control unit 170 controls the reaction force of the robot arm 210 so that the robot arm 210 is operated within a predetermined area or the robot arm is drawn in, drawn out, or approached to a preset predetermined area, And performs a force feedback function to apply a predetermined reaction force to the operation of the user operation unit 150 when performing a predetermined operation.

First, the reaction force control unit 170 performs a force feedback function to apply a predetermined reaction force to the operation of the user operation unit 150 when the robot arm 210 operates within a predetermined predetermined area. Here, the force feedback function refers to an operation lever (see 110 in Fig. 2) and a signal from a tactile sensor (not shown) provided at the end of the robot arm 210 or a predetermined distance therefrom And a reaction force is applied to the operation of the same user operation unit 150 to thereby reproduce the feeling when the doctor performs the operation with the hand. In other words, this means a function of returning the result of the operation back to the information of the force to the side of operating the mechanism, or a system using the function. Such a force feedback function can be used to perform a manual operation The same feeling can be reproduced.

Here, the predetermined region may be the entire surgical site. That is, the reaction force control unit 170 can control the robot arm 210 to apply a predetermined reaction force to the operation of the user operation unit 150 with respect to all operations of the robot arm 210. When the predetermined region is the entire operation region, the user can set the high gravity mode or the high friction mode.

The dual high gravity mode means that the surgical tool feels heavier than it actually is, thus allowing the operator to take more careful action. At this time, the additional reaction force applied to the user manipulation unit is limited to the direction of gravity, and the magnitude of the reaction force depends on the weight and posture of the surgical tool.

Here, the gravitational direction in the high gravity mode may be the actual gravity direction (generally, vertically below the display member 120). Alternatively, the gravitational direction in the high gravity mode may be the gravitational direction on the screen displayed on the display member 120. In detail, the actual gravity direction and the gravity direction recognized by the user through the display member may be different from each other. For example, when the user rotates the laparoscope 220 that captures an image of the surgical site, the screen displayed on the display member 120 will also rotate, and therefore, The directions of gravity on the screen can be different from each other. In this case, a predetermined gravitational direction display mark is displayed on the display member 120, the gravitational direction display mark is rotated together with the screen displayed on the display member 120, and the additional reaction force The user operation unit 150 may be controlled so as to be provided with the user. Further, the user may desire to arbitrarily set the direction of the additional reaction force in the high gravity mode in a direction other than the direction of the actual gravity (for example, the opposite direction of gravity). Thus, the gravitational direction in the high gravity mode herein can be defined as including both the actual gravitational direction, the gravitational direction on the screen displayed on the display member 120, and the imaginary gravitational direction arbitrarily set by the user There will be.

On the other hand, the high frictional force mode is a function that gives a feeling of being in a shallow underwater space or treating an object in soft mud. That is, the operation mode is a mode in which the operator performs a more careful operation with a lot of force, and the additional reaction force applied to the user manipulation unit becomes a reaction force in the three-dimensional direction with respect to the movement direction of the operation, Or any function. Here, the concept of frictional force must be distinguished from the inertial force proportional to the speed. It is a concept that makes the deceleration easy and makes it difficult to accelerate, and restricts only the acceleration movement for agile but aggressive operation. In other words, in the high frictional force mode, the reaction force applied to the user manipulation part 150 is set to be larger than the reference value in the acceleration operation of the robot arm 210, and the reaction force applied to the user manipulation part 150 Is equal to or smaller than the reference value.

On the other hand, the predetermined area may be automatically recognized by the master robot or may be a specific area designated by the user.

First, the control unit 180 may automatically specify the operation region by calculating positions and boundaries of various tissues and organs from the image of the surgical site photographed from the laparoscope 220. [

5, which is a diagram showing a screen display mode in which an image photographed by a laparoscope is output through a display member, an image photographed through a camera provided in the laparoscope 220 of the slave robot 200 And is output to the display member 120 through the image input unit 130 of the master robot 100. 6, the control unit 180 of the master robot 100 performs image processing on the image, automatically calculates the positions and boundaries of various tissues and organs, and outputs it to a display member (not shown) 120).

At this time, the control unit 180 may control the reaction force control unit 170 to apply a predetermined reaction force to the operation of the user operation unit 150 for all the calculated tissue and organ area.

Alternatively, the control unit 180 may display only the tissues and organs to which the user wants to apply a reaction force larger than the actual one of the tissue and organ parts displayed on the display member, in a state where all the tissues and organ parts calculated on the display member are displayed, Touch, or the like, and controls the reaction force control unit 170 to apply a predetermined reaction force to the operation of the user operation unit 150 only for the selected tissue or organ area.

7, the user arbitrarily draws a three-dimensional solid figure DL such as a sphere, a cube, or a cylinder on the display member so that only the area inside the drawn figure can be manipulated by the operation of the user manipulation part 150 The reaction force control unit 170 may be controlled to apply a predetermined reaction force.

On the other hand, when the master robot automatically recognizes or controls the user operation unit 150 to apply a predetermined reaction force to a predetermined area designated by the user, the user may select a high gravity mode, a high friction mode, Can be set. Here, the high gravity mode and the high frictional force mode are the same modes as described above.

The drainage enhancement mode is a mode in which a value obtained by increasing or decreasing the reaction force measured or estimated by the slave robot is fed back to the master robot. In other words, there is no limitation in the operation when moving a space without work (i.e., the robot does not have any additional reaction force to the user's manipulation part) and interacts with the object (for example, , It is a mode for attracting the operator's attention by requiring a lot of operation force than the actual operation. Furthermore, even in such drainage enhancement mode, the weight of the reaction force may be differently assigned to a specific region or a specific tissue region or organ, so that the effect can be emphasized only on a region or region requiring careful manipulation.

The reaction force control unit 170 performs a force feedback function to apply a predetermined reaction force to the operation of the user operation unit 150 when the robot arm 210 performs predetermined predetermined operations. This is defined as the operation restriction mode.

In detail, the operation restriction mode is a mode for restricting the operation of the robot arm 210 according to the path of the operation of the robot arm 210. In the case where the robot arm 210 operates in a predetermined area as described above, the reaction force is calculated on the basis of the space in which the robot arm 210 works, but in this mode, And applies a limiting reaction force to the robot arm 210 based on information extracted from its own path. That is, the control unit 180 controls a trajectory for a recent predetermined time (approximately several tens of milliseconds to several seconds) of the attached position and speed of the moving robot arm 210 according to the user's manipulation, Observe the change process of the attachment of the arm 210. Then, after grasping the area where the main work is currently performed, when the work area is far from the work area, it is given a reaction force which is doubled according to the distance from the work area, or a reaction force that arbitrarily restricts the acceleration when the operation is rapidly accelerated The reaction force control unit 170 may be controlled. According to the present invention as described above, the operator concentrates on the area displayed on the display member 120, and induces an efficient and stable work operation without a rapid deceleration in terms of speed.

According to the present invention, it is possible to obtain an effect of preventing an accident or a harm caused by an unintentional malfunction of a surgical operator during surgery while minimizing disturbance to a surgical operation of the operator. Further, by using the operation restriction mode of the present invention, it is possible to obtain the effect of training to improve the operative operation of the operator in a more efficient form.

Hereinafter, a method of controlling a surgical robot system according to an embodiment of the present invention will be described with reference to related drawings. 7 is a flowchart schematically showing a control method of a surgical robot system according to an embodiment of the present invention.

7, an image captured through the laparoscope 220 of the slave robot 200 is displayed on the display member 120 of the master robot 100. In the control method of the surgical robot system according to the embodiment of the present invention, (Step S110), the robot arm 210 is operated within a predetermined area, or the robot arm moves in, out, or approaches a predetermined area, or the robot arm moves to a predetermined motion (Step S120). When the robot arm 210 is operated within a preset predetermined area or performs predetermined predetermined operations, it is determined whether or not a predetermined reaction force And performing a force feedback function to apply a force feedback function (step S 130).

First, an image photographed through the laparoscope 220 of the slave robot 200 is displayed on the display member 120 of the master robot 100 (step S110). A screen display mode in which an image photographed by the laparoscope 220 is output through the display member 120 is shown in FIG.

Next, it is determined whether the robot arm 210 is operating within a predetermined area or whether the robot arm is pulled in, pulled out, or approached for a predetermined area, or whether the robot arm performs predetermined predetermined operations (Step S120). As a result of the determination, when the robot arm 210 operates within a predetermined area, or when the robot arm moves in, out, approaches, or performs a preset predetermined operation, A force feedback function for applying a predetermined reaction force to the operation of the operation unit 110 is performed in step S130.

Here, the predetermined area may be the entire operation site, or a predetermined area may be automatically recognized by the master robot or may be a specific area designated by the user.

The reaction force control unit 170 can control the robot arm 210 to apply a predetermined reaction force to the operation of the user operation unit 150 when the predetermined region is the entire operation region. When the predetermined region is the entire operation region, the user can set the high gravity mode or the high friction mode. Here, since the high gravity mode and the high frictional force mode have been described in detail above, detailed description thereof will be omitted here.

On the other hand, when the preset predetermined area is automatically recognized by the master robot or is a specific area designated by the user, the reaction force control unit 170 controls the robot arm 210 to move only the operation performed within the specific area by the user operation unit 150 so that a predetermined reaction force is exerted on the operation of the control device. In the case where the preset predetermined area is automatically recognized by the master robot or is a specific area designated by the user, the user can set a high gravity mode, a high friction mode, or a multiple reinforcement mode. Here, since the high gravity mode, the high frictional force mode, and the multiple reinforcement mode have been described in detail above, detailed description thereof will be omitted here.

At this time, the control unit 180 automatically calculates the positions and boundaries of various tissues and organs from the images of the surgical site photographed from the laparoscope 220, and controls the operation of the user manipulation unit 150 The reaction force control unit 170 may be controlled to apply a predetermined reaction force. Or all tissues and long-term areas calculated on the display member are displayed, the user selects only the tissues and organs of the tissue and long-term areas displayed on the display member, The reaction force control unit 170 may be controlled so as to apply a predetermined reaction force to the operation of the user manipulation unit 150 only for the selected tissue or organ region. A reaction force control unit 170 for applying a predetermined reaction force to the operation of the user operating unit 150 only for the area inside the drawn figure by drawing a three-dimensional solid figure such as a sphere, a cube, .

The reaction force control unit 170 may perform a force feedback function to apply a predetermined reaction force to the operation of the user operation unit 150 when the robot arm 210 performs predetermined predetermined operations. This operation mode is defined as an operation restriction mode. Since the operation restriction mode has been described in detail above, detailed description thereof will be omitted here.

Finally, it is determined whether a termination signal is received from the user operation unit 110 (step S140). If the termination signal is not received from the user operation unit 110, steps S110 to S130 are performed again.

According to the present invention, it is possible to obtain an effect of preventing an accident or a harm caused by an unintentional malfunction of a surgical operator during surgery while minimizing disturbance to a surgical operation of the operator. Further, by using the operation restriction mode of the present invention, it is possible to obtain the effect of training to improve the operative operation of the operator in a more efficient form.

Although the present invention has been described with reference to the limited embodiments, various embodiments are possible within the scope of the present invention. It will also be understood that, although not described, equivalent means are also incorporated into the present invention. Therefore, the true scope of protection of the present invention should be defined by the following claims.

1: Surgical robot system 100: Master robot
110: Operation lever 120: Display member
200: Slave robot 210: Robot arm
220: Laparoscopic

Claims (26)

A slave robot including a robot arm driving with multiple degrees of freedom,
A user operation unit for generating a control signal for operating the robot arm; And
And a reaction force control unit for performing a force feedback function to apply a predetermined reaction force to deform or increase the reaction force detected by the robot arm to a certain degree with respect to the operation of the user operation unit,
The reaction force control unit controls the operation of the user's operating unit in a gravity mode in which an additional reaction force is applied in an actual gravity direction, a gravity direction on a screen displayed on a display member for displaying an image of a surgical site, Wherein the master robot comprises: The method according to claim 1,
The reaction force control unit may control the robot arm such that the robot arm operates within a predetermined area or the robot arm moves in, out, or approaches a predetermined area, or the robot arm performs predetermined operations And performs a force feedback function to apply the predetermined reaction force to the operation of the user operation unit. 3. The method of claim 2,
Wherein the predetermined region is the entire surgical site to be operated. delete The method of claim 3,
Wherein the reaction force control unit performs a high frictional force mode in which an additional reaction force is applied to the operation of the user's operation unit in a direction opposite to the traveling direction of the robot arm. The method of claim 3,
Wherein the reaction force control unit applies a reaction force applied to the user's operating unit to an accelerating operation of the robot arm larger than a predetermined reference value and sets a reaction force applied to the user's operating unit to a predetermined reference value, Wherein the high frictional force mode is performed in a state in which the robot is in a high frictional force mode. 3. The method of claim 2,
Wherein the slave robot further comprises a surgical endoscope for scanning a surgical site. 8. The method of claim 7,
Wherein the predetermined region includes at least a part of at least one region in which a tissue and organ calculated from an image of a surgical site photographed from the surgical endoscope are located. 8. The method of claim 7,
Wherein the master robot further comprises a display member for displaying an image photographed through the surgical endoscope,
Wherein the predetermined area is an arbitrary area selected by user input on the display member. 10. A method according to any one of claims 8 to 9,
The reaction force control unit controls the operation of the user's operating unit in a gravity mode in which an additional reaction force is applied in an actual gravity direction, a gravity direction on a screen displayed on a display member for displaying an image of a surgical site, Wherein the surgical robot system comprises: 10. A method according to any one of claims 8 to 9,
Wherein the reaction force control unit performs a high frictional force mode in which an additional reaction force is applied to the operation of the user's operation unit in a direction opposite to the traveling direction of the robot arm. 10. The method of claim 9,
The reaction force control unit may control the robot arm such that when the robot arm operates in the selected predetermined area or when the robot arm moves in, out or approaches the selected predetermined area, a predetermined additional reaction force is applied to the operation of the user's operation unit And a control unit for controlling the operation of the robot. 10. A method according to any one of claims 8 to 9,
Wherein the reaction force control unit performs control so that when the region includes a plurality of regions, an additional reaction force different from each other is applied to the operation of the user operation unit. 3. The method of claim 2,
When the robot arm is separated from a work area of the robot arm extracted from a trajectory of a position and a speed of the robot arm for a predetermined time interval by a predetermined distance or more, So as to apply a predetermined additional reaction force. 3. The method of claim 2,
Wherein the reaction force control unit performs control so as to apply a predetermined additional reaction force to the operation of the user operation unit when the acceleration of the robot arm is equal to or greater than a predetermined reference value. delete delete delete delete delete delete delete delete delete delete delete

Images