KR102274175B1 - Surgical navigation apparatus and the method thereof - Google Patents

Abstract

The present invention relates to a surgical navigation device and a method therefor, wherein the surgical navigation device according to the present invention registers a first image including an object marker attached to a surgical object and a second image of a preoperative surgical object to perform the surgery. an object matching unit for deriving a correlation with respect to a position and posture between an object marker attached to the object and the surgical object; a reference position storage unit configured to set and store a reference position with respect to at least one reference point of the operation object based on the correlation between the object marker derived from the object matching unit and the operation object; a position calculator for receiving the position and posture information of the object marker, and calculating the position of the reference point of the surgical object based on the derived correlation; and a marker deformation determining unit that determines that the object marker is deformed when the position of the reference point calculated by the position calculating unit deviates from the reference position. Accordingly, it is possible to easily detect whether the object marker is modified in real time.

Description

SURGICAL NAVIGATION APPARATUS AND THE METHOD THEREOF

The present invention relates to a surgical navigation device and method therefor, and more particularly, to a surgical navigation device and method for detecting whether a marker attached to a surgical object is deformed.

Recently, with the development of medical technology, navigation surgery using a robot and computer system has been actively introduced, and this surgery is also applied in the field of artificial joint surgery.

In the case of the knee joint, when pain or behavioral disorders due to trauma, various infections, or diseases come, orthopedic surgery is used to treat the entire or part of the joint using replacement surgery. wear and tear, and partial knee joint replacement surgery is performed.

Among the robots used for orthopedic joint surgery, there is a robot that automatically performs the entire surgical process, and these surgical robots automatically cut bones along a pre-planned path without human intervention.

When performing knee joint surgery using a conventional orthopedic surgical robot, the surgical route is determined in advance, an optical marker is mounted on the surgical object, and the position and posture of the optical marker are tracked with an optical sensor, and the patient position and The operation is carried out by monitoring the position of the robot.

On the other hand, it often occurs that the marker mounted on the surgical object is deformed by an external force or the like during surgery. As such, when the marker is deformed, a problem arises in estimating the position and posture of the surgical object. In order to solve this problem, conventionally, an additional marker or instrument is attached to the surgical object to check whether the object is deformed.

However, in this prior art, since markers or instruments need to be further attached to the surgical object, for example, bone, it becomes a burden on the patient's bone damage.

Republic of Korea Patent Publication No. 2015-0119102 (2015.10.23)

The present invention has been proposed to solve the above problems, and to provide a surgical navigation device and method capable of easily detecting whether a marker attached to a surgical object is deformed during surgery.

In addition, the present invention has been proposed to solve the above problems, and it is to provide a surgical navigation device and method for easy recovery when a marker is deformed.

In a surgical navigation device for achieving the above object, a first image including a target marker attached to a surgical object and a second image related to an object prior to surgery are matched to match the object marker attached to the surgical object and the surgery an object matching unit deriving a correlation regarding positions and postures between objects; a reference position storage unit for setting and storing a reference position with respect to at least one reference point of the operation target; a position calculator for receiving position and posture information of the object marker from a tracker, and calculating a position of the reference point of the surgical object based on the derived correlation; and a marker deformation determining unit that determines that the object marker is deformed when the position of the reference point calculated by the position calculating unit deviates from the reference position.

In a surgical navigation device for achieving the above object, a first image including a target marker attached to a surgical object and a second image related to an object prior to surgery are matched to match the object marker attached to the surgical object and the surgery an object matching unit deriving a correlation regarding positions and postures between objects; a reference position storage unit for calculating and storing a reference positional relationship with respect to a changeable position of the object marker using at least one reference point of the surgical object as a reference position; and a marker deformation determining unit that receives the position and posture information of the object marker from the tracker, and determines that the object marker is deformed when the position and posture of the object marker deviates from the reference position relationship.

In addition, based on the image including the robot marker attached to the surgical robot, a correlation with respect to the position and posture between the robot marker and the surgical object, and a correlation regarding the position and posture between the robot marker and the reference point are derived. a robot matching unit; and deriving a correlation regarding the position and posture before and after deformation of the object marker based on a change in the correlation between the object marker and the robot marker before and after deformation of the object marker, and correlation before and after deformation of the object marker It further includes a recovery unit for resetting the correlation between the object marker and the operation object based on the.

In a surgical navigation method for achieving the above object, a first image including an object marker attached to a surgical object is matched with a second image about a surgical object before surgery, and the object marker attached to the operation object and the surgery deriving a correlation regarding positions and postures between objects; setting and storing a reference position with respect to at least one reference point of the surgical object; receiving position and posture information of the subject marker from a tracker, and calculating a position of the reference point of the surgical subject based on the derived correlation; and determining that the object marker is deformed when the calculated position of the reference point deviates from the reference position.

As described above, according to the present invention, it is possible to easily detect whether a marker is deformed in real time by setting a reference position with respect to a reference point on a surgical object and tracking the position of the reference point during surgery.

Also, according to the present invention, when the object marker is deformed, it can be easily recovered using the robot marker.

1 schematically illustrates a surgical navigation system according to an embodiment of the present invention.
2 is a control block diagram of a surgical navigation device according to an embodiment of the present invention.
3 is a view for explaining the operation of the control unit of the surgical navigation device according to an embodiment of the present invention.
4 is for explaining the operation of the marker deformation determining unit according to an embodiment of the present invention.
5 is a block diagram of a control unit of the surgical navigation device according to an embodiment of the present invention.
6 is for explaining the operation of the robot matching unit according to an embodiment of the present invention.
7 is for explaining the operation of the recovery unit according to an embodiment of the present invention.
8 is a flowchart illustrating a method of determining whether an object marker is deformed by a surgical navigation device according to an embodiment of the present invention.
9 is a flowchart illustrating a method of determining whether an object marker is deformed by a surgical navigation device according to another embodiment of the present invention.
10 is a flowchart illustrating a method for determining and recovering whether an object marker is deformed by a surgical navigation device according to another embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. However, detailed descriptions of well-known functions or configurations that may obscure the gist of the present invention in the following description and accompanying drawings will be omitted. Also, it should be noted that throughout the drawings, the same components are denoted by the same reference numerals as much as possible.

The terms or words used in the present specification and claims described below should not be construed as being limited to their ordinary or dictionary meanings, and the inventor is appropriate as a concept of terms for describing his invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention based on the principle that it can be defined in Accordingly, the embodiments described in this specification and the configurations shown in the drawings are only the most preferred embodiment of the present invention, and do not represent all of the technical spirit of the present invention. It should be understood that there may be equivalents and variations.

Hereinafter, a surgical navigation device according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4 .

1 schematically illustrates a surgical navigation system according to an embodiment of the present invention. Referring to FIG. 1 , a surgical navigation system according to an embodiment of the present invention includes object markers 10 and 20 attached to surgical objects 1 and 2 , a surgical robot 30 , a tracker 40 , and a surgical navigation system. device 100 .

The surgery object (1, 2) refers to a target for surgery, and in the embodiment of the present invention, the operation object is a knee joint of the femur (1) (Femur) and the tibia (2) (Tibia), the object marker (10, 20) will be described as an example that each is attached to the femur (1) and the tibia (2).

The surgical robot 30 is for joint surgery and includes a robot base and an arm, and a cutting tool may be located in an end effector of the arm. A robot marker 31 is attached to the base of the surgical robot 30 .

For the object markers 10 and 20 and the robot marker 31, optical markers may be used, and 3 or 4 branch-shaped bars in different directions are formed based on the center point, and each high bar is formed at the end of the bar. A reflective ball marker may be formed.

The tracker 40 is for tracking the position and posture of the robot marker 31 attached to the surgical robot 30 and the object markers 10 and 20 attached to the surgical objects 1 and 2, and three-dimensional spatial coordinates. The position and posture of the marker on the upper body are sensed and transferred to the surgical navigation device 100 to be described later. In the embodiment of the present invention, the tracker 40 will be described as an example implemented as an optical tracking system (OTS).

The surgical navigation device 100 performs registration of the surgical object and the surgical robot 30, receives a signal input from the tracker 40, and determines whether the object markers 10 and 20 are deformed. 1, it may be implemented including a computer (processor) and a display. 1 shows that the surgical navigation device 100 is separated from the surgical robot 30 and implemented as a separate device, but in some cases, the computer (processor) of the surgical navigation device is installed in the surgical robot 30, The display of the surgical navigation device may be connected to the tracker 40 and installed together. In this case, the surgical robot 30 is connected to the tracker 40 to receive the position/posture information of the markers, process it and provide it to the display.

2 is a control block diagram of the surgical navigation device 100 according to an embodiment of the present invention. Referring to FIG. 2 , the surgical navigation device 100 according to an embodiment of the present invention includes a signal receiving unit 110 , a user input unit 120 , a display unit 130 , a memory unit 140 , and a control unit 150 . include

The signal receiving unit 110 is for receiving a signal from the outside, for example, an HDMI (High Definition Multimedia Interface) connector 11 for connection with an external device, a D-sub connector, or an Internet network. It may include a communication module for connecting to a wired / wireless network including. The signal receiver 110 may include a wired/wireless communication module for interworking between the tracker 40 and the surgical robot 30 .

The user input unit 120 receives a command from a user and transmits it to the control unit 150 to be described later, and may include various input means such as a keyboard, a mouse, and a button.

The display unit 130 is for displaying an image on a screen, and includes a liquid crystal display (LCD) panel, a light emitting diode (LED) panel, an organic light emitting diode (OLED) panel, and the like. can be implemented as

The memory unit 140 may store various OSs, middleware, platforms, and various applications of the surgical navigation device 100 , and may store program codes and signal-processed image signals, audio signals, and data. The memory unit 140 stores a surgical target image, for example, a CT image of a patient, about the surgery target 1 and 2 acquired before surgery.

The controller 150 is in charge of overall control of the surgical navigation device 100 according to a user command input through the user input unit 120 or an internal program. The control unit 150 may be implemented by including a program code for signal processing and control and a processor executing the program. The controller 150 performs image registration and position tracking using the position/posture information received from the tracker 40 through the signal receiver 110 , and detects whether the object markers 10 and 20 are deformed. Also, when the object markers 10 and 20 are deformed, the controller 150 performs recovery.

Referring to FIG. 2 , the control unit 150 includes an object matching unit 151 , a reference position storage unit 153 , and a marker deformation determining unit 155 .

The object registration unit 151 includes a first image (optical image) including the object markers 10 and 20 obtained from the tracker 40 and a second image of the patient's surgical objects 1 and 2 taken before surgery. (eg, 3D CT image) is matched to derive a correlation regarding the position/posture between the object markers 10 and 20 (bone marker) attached to the surgical object 1 and 2 and the surgical object 1 and 2 . After attaching the object markers 10 and 20 to the surgical objects 1 and 2, the positions/positions of the surgical objects 1 and 2 and the object markers 10 and 20 are recognized using a probe. The object registration unit 151 receives the optical image regarding the position/position indicated by the probe through the tracker 40 , and performs registration with the patient's 3D data, for example, a CT image, which is pre-stored in the memory unit 140 . A correlation with respect to the position/posture between the markers 10 and 20 and the surgical subject 1 and 2 may be derived.

In the present specification, the term 'surgical object' is used in a broad sense of a surgical target, such as the femur 1 and the tibia 2, and indicates a specific position or surgical site of the surgical object 1, 2, such as the implant origin or joint center. It is also used in the meaning of consultation.

3 is a diagram for explaining the operation of the control unit 150 according to an embodiment of the present invention. Referring to FIG. 3 , the surgical target is a knee joint of the femur 1 and the tibia 2 , and the target markers 10 and 20 are attached to the femur 1 and the tibia 2 .

In FIG. 3 , FMC denotes a coordinate system of the femur marker 10 based on the positions and postures of the target markers 10 and 20 of the femur 1 , and HC denotes the hip joint center of the femur 1 . , means a coordinate system based on the position and posture of HJC). The object registration unit 151 provides a correlation between the position of the femur marker 10 and the position/posture of the center point of the hip joint of the femur 1 through image registration, for example, the hip of the femur 1 with respect to the femur marker 10 . ^{A transformation matrix ( F} T _H ) can be derived as a coordinate transformation relationship of the joint center. Accordingly, the position and posture information of the femur marker 10 is obtained from the tracker 40, and the transformation matrix derived from the object matching unit 151 to the obtained position and posture information of the femur 1 ( ^F T _H ) By multiplying by , the position and posture of the center (HJC) of the hip joint of the femur 1 can be derived.

In FIG. 3 , TMC denotes a coordinate system of the tibia marker 20 , and AC denotes a coordinate system having an ankle joint center (AJC) of the tibia 2 as an origin. In the same way, the object registration unit 151 is a correlation with respect to the position/posture of the ankle joint center (AJC) with respect to the tibia marker 20 attached to the tibia 2 through image registration, for example, a transformation that is a coordinate transformation relationship. Derive a matrix ^T T _{A .} Accordingly, the position and posture information regarding the tibial marker 20 is obtained from the tracker 40, and the transformation matrix ( ^T T _{A) derived from the surgical subject matching unit 151 to the position and posture information of the tibial marker 20} ) to derive the position and posture of the center of the ankle joint (AJC) of the tibia (2).

The object matching unit 151 derives correlations with respect to positions/postures of a number of points, such as the implant origin, in addition to the center of the hip joint of the femur 1 and the center of the ankle joint of the tibia 2 described above. Accordingly, if the position/position of the marker is tracked by the tracker 40 , the positions and positions of a plurality of points of the object may be derived.

When the registration of the surgery objects 1 and 2 is completed, the robot is moved and arranged close to the operation objects 1 and 2 to prepare for surgery. At this time, the surgical object (1, 2) is fixed. For example, in FIG. 3, the hip joint center and the ankle joint center are fixed to prevent physical movement, and the femur (1) and the tibia (1) and the tibia ( 2) may be in a moving or stationary state.

The reference position storage unit 153 is based on the correlation between the object markers 10 and 20 derived from the object registration unit and the operation objects 1 and 2, at least one reference point of the operation objects 1 and 2 Set the reference position and save it. Here, the at least one reference point refers to a point serving as a reference point for movement in the surgery objects 1 and 2, and may be set differently depending on the type of the operation object 1 and 2 or the type of operation. In this embodiment, the center of the hip joint is set as a reference point in the femur (1) and the position and posture of the point are stored as the reference position after fixing the surgical object (1, 2), and in the tibia (2), the center of the ankle joint is the standard It is set as a point and the position and posture of the corresponding point are saved as the reference position. Here, the reference positions of the reference points of the surgical objects 1 and 2 stored in the reference position storage unit 153 refer to positions and postures based on the coordinate system of the tracker 40 .

The position calculating unit 154 receives the position and posture information of the object markers 10 and 20 from the tracker 40 , and based on the correlation derived from the object matching unit 151 , Calculate the position of the reference point. Referring to FIG. 3 , the position and posture of the femoral marker 10 are tracked to calculate the position of the center of the hip joint, and the position and posture of the tibial marker 20 are tracked to calculate the position of the center of the ankle joint. The position calculating unit 154 calculates the position of the reference point continuously at a predetermined period during the operation. In this case, the position calculator 154 calculates the position and posture of the reference point based on the coordinate system of the tracker.

When the position of the point calculated by the position calculating unit 154 deviates from the reference position, the marker deformation determining unit 155 determines that the object markers 10 and 20 are deformed, and when the same as the reference position, determines the normal state. do. Referring to FIG. 3 , the femur 1 is rotatable with the center of the hip joint as a reference point, and the tibia 2 is rotatable with the center of the ankle joint as a reference point. Accordingly, the femoral marker 10 may be located on the spherical surface of A1, and the tibial marker 20 may be located on the spherical surface of A2. Since the correlation regarding the position and posture between the object markers 10 and 20 and the object does not change, even if the object markers 10 and 20 move during surgery, the positions of the reference points, for example, the hip joint center and the ankle joint center, are fixed at the time of initial fixation. must have the same reference position. The marker deformation determination unit 155 determines whether the position of the reference point calculated by the position calculation unit 154 is the same as the reference position stored in the reference position storage unit 153, and if the same is normal, it is not the same. It is judged that the deformation of the marker has occurred.

FIG. 4 is for explaining the operation of the marker deformation determining unit 155 according to an embodiment of the present invention, and illustrates a case in which the object markers 10 and 20 are deformed due to an external force or the like during surgery. In the femur 1, it is assumed that the femoral marker 10 is rotated (rotation). The position calculator 154 obtains the position and posture information of the femur marker 10 from the tracker 40 and uses the correlation ^F T _H calculated by the object matching unit 151 to obtain a reference point, for example, the hip. Calculate the position of the joint center. The femur marker 10 rotates to the right from the origin position of the existing FMC coordinate system and moves to the posture of the FMC' coordinate system, and accordingly, the hip joint center is moved from the existing HJC position to the HJC' position. The marker deformation determining unit 155 determines whether the HJC position, which is the position of the reference point previously stored in the reference position storage unit 153, and the HJC' position, which is the position of the currently tracked reference point, are the same. As shown in FIG. 4 , since the reference position HJC and the current position HJC' of the reference point are different from each other, the marker deformation determining unit 155 may determine that the marker is deformed due to an external force or the like.

In the tibia (2), it was assumed that the tibial marker 20 had a translation in the vertical direction. The position calculator 154 obtains the position and posture information of the tibia marker 20 from the tracker 40 , and uses the correlation ( ^T T _A ) calculated by the object matching unit 151 to obtain a reference point, for example, an ankle Calculate the position of the joint center. The tibial marker 20 moves in parallel downward from the origin of the existing TMC coordinate system to the origin of the TMC' coordinate system, and accordingly, the center of the ankle joint is moved from the existing AJC position to the AJC' position. The marker deformation determining unit 155 determines whether AJC, which is the position of the reference point pre-stored in the reference position storage unit 153, and AJC', which is the position of the currently tracked reference point, are the same. As shown in FIG. 4 , since the reference position AJC and the current position AJC' of the reference point are different, the marker deformation determining unit 155 may determine that the marker is deformed due to an external force or the like.

The surgical navigation device according to an embodiment of the present invention may further include a GUI generator 156 . The GUI generating unit 156 may generate a message informing that the marker deformation is detected to be displayed on the display unit 130 . The user can know that the marker is deformed through the message displayed on the screen.

Hereinafter, a recovery method when marker deformation occurs in the surgical navigation device according to an embodiment of the present invention will be described with reference to FIGS. 5 to 7 . 5 is a block diagram of the control unit 150 of the surgical navigation device according to an embodiment of the present invention. As shown in FIG. 5 , the surgical navigation device according to this embodiment further includes a robot matching part 152b as the matching part 152 , and may further include a recovery part 157 .

The robot matching unit 152b is a correlation with respect to the position and posture between the robot marker 31 and the surgical object 1 and 2, and the correlation between the position and posture between the robot marker 31 and the reference point through registration. derive The robot matching unit 152b first attaches a marker to the robot arm and the robot base, and tracks the position and posture of the marker through the tracker 40 while moving the robot arm. The robot registration is performed by deriving a correlation between the posture and the position/posture of the robot marker 31 . When the surgical preparation is complete, the surgical robot 30 is placed in the operable area, and the robot marker 31 and the object markers 10 and 20 and the surgical object installed on the base of the surgical robot 30 through the tracker 40 ( 1, 2) to derive the correlation regarding the position and posture. Here, the robot matching unit 152b may derive a correlation regarding the position and posture between the registration and surgery object, the object marker, and the robot marker based on the coordinate system of the surgical robot 30 or the coordinate system of the robot marker 31. have. 6 is for explaining the operation of the robot matching unit 152b according to an embodiment of the present invention. Referring to FIG. 6 , RM means the position/position of the robot marker 31 , and RMC is a coordinate system based on the position of the robot marker 31 and serves as a reference for the position and posture of the robot marker 31 . The robot matching unit 152b is configured to correlate the positions and postures between the robot marker 31 and the object markers 10 and 20 based on the position and posture information of the markers obtained from the tracker 40 ( ^R T _F , ^{R ).} T _T ) is derived. At this time, the robot matching unit 152b is a robot marker based ^{on the correlations (F} T _H , ^T T _A ) regarding the position and posture between the object markers 10 and 20 and the reference point calculated by the object matching unit 151 . ^{(31) and the correlation (R} T _H , ^R T _A ) regarding the position and posture between the surgical subjects 1 and 2 may be derived.

For example, the correlation regarding the position and posture between the robot marker 31 and the reference position (HJC) of the center of the hip joint is ^R T _H = ^R T _F x ^F T _H , and the reference of the robot marker 31 and the center of the ankle joint The correlation regarding the position and the posture between the positions (AJC) can be derived as ^R T _A = ^R T _T x ^T T _{A .} Here, ^R T _F is the transformation matrix of the femoral marker 10 for the robot marker 31 , ^R T _T is the transformation matrix of the tibia marker 20 for the robot marker 31 ^{, R} T _{H is the robot marker 31} ) of the hip joint center transformation matrix, ^R T _A means the ankle joint center transformation matrix for the robot marker 31 .

^{The correlations R} T _H , ^R T _A with respect to the position and posture between the robot marker 31 and the reference position of the reference point calculated by the robot matching unit 152b are stored in the reference position storage unit 153 . ^{In addition, the correlation ( R} T _F , ^R T _T ) regarding the position and posture between the robot marker 31 and the object markers 10 and 20 calculated by the robot matching unit 152b is stored in the reference position storage unit ( 153) is stored. On the other hand, the reference position storage unit 153 but stores the reference position of the reference point of the surgical object (1, 2), the position based on one of the coordinate system of the base of the robot marker 31 or the surgical robot 30, and You can save your posture information. For recovery, position and posture information of a reference point based on a fixed third position that does not move is required. Since the tracker 30 is likely to be moved during surgery, in this embodiment, the position and posture information of the reference point for recovery is a coordinate system based on the fixed surgical robot 30 without moving, that is, the surgical robot 30 ), the position and posture information of the reference position based on the coordinate system of the base or the coordinate system of the robot marker 31 is stored and used during recovery. In addition, in this embodiment, the marker deformation determining unit 155 calculates the position of the reference point based on the coordinate system of the base of the robot marker 31 or the surgical robot 30, and the value stored in the reference position storage unit 153 and By comparison, it is possible to determine whether the marker is modified. In another example, the marker deformation determination unit 155 calculates the position of the reference point based on the coordinate system of the tracker 40 and stores the value stored in the reference position storage unit 153 (eg, the reference position based on the coordinate system of the tracker) and It is also possible to determine whether the marker is modified by comparison.

The recovery unit 157 is located before and after deformation of the object markers 10 and 20 based on a change in the correlation between the object markers 10 and 20 and the robot marker 31 before and after deformation of the object markers 10 and 20 . and deriving a correlation regarding posture, and resetting the correlation between the object markers 10 and 20 and the surgical object 1 and 2 based on the correlation before and after deformation of the object markers 10 and 20 .

7 is for explaining the operation of the recovery unit 157 according to an embodiment of the present invention. Referring to FIG. 7 , as in FIG. 4 , rotation occurs in the femur marker 10 in the femur 1 , and in the tibia 2 , the tibia marker 20 moves in parallel in the vertical direction. It is assumed that this has occurred. The marker deformation determining unit 155 determines that the position of the reference point calculated by the position calculating unit 154, for example, the hip joint center position (HJC') and the ankle joint center position (AJC') is the reference position, such as HJC and AJC. It is determined whether or not it is the same as, and it is determined that deformation has occurred in both the femur marker 10 and the tibia marker 20 . When the marker modification determining unit 155 detects that the marker has been modified, the recovery unit 157 correlates the position and posture between the object markers 10 and 20 and the surgical object 1 and 2 based on the changed marker. Re-establish the relationship.

The correlation between the object markers 10 and 20 and the robot marker 31 after deformation of the femoral marker 10 can be calculated from the position/position obtained through the tracker 40, and in FIG. 7, the femoral marker 10 It is represented by ^{the transformation matrix R} T _F' of the tibial marker 20 with respect to the robot marker 31 after the deformation of . As described above, the reference position storage unit 153 stores the transformation matrix of the femoral marker 10 with respect to the robot marker 31 before the marker deformation occurs.

The recovery unit 157 uses the transformation matrix of the femur marker 10 with respect to the robot marker 31 before and after the deformation of the femoral marker 10 is used, and the femoral marker 10 before and after the deformation of the femoral marker 10 occurs. The correlation regarding the change of position and posture can be derived as follows.

[Equation 1]

^F T _F' = inv( ^R T _F )x ^R T _{F' … … … … …} (One)

= inv( ^R T _H x inv( ^F T _H ))x ^R T _{F' … … … … …} (2)

(Here, ^F T _F' is a transformation matrix related to the change in the position and posture of the femoral marker 10 before and after the deformation of the femoral marker 10, ^R T _F' is the robot marker after the deformation of the femoral marker 10 occurs. (31) is the transformation matrix of the femoral marker 10, ^R T _F is the transformation matrix of the femoral marker 10 with respect to the robot marker 31 before deformation of the femur marker 10, ^R T _H is the femoral marker ( The transformation matrix of the hip joint center (reference position) for the robot marker 31 before the deformation of 10), ^F T _H , is the hip joint center (reference position) for the femur marker 10 before the deformation of the femoral marker 10 occurs. ), and inv means an inverse function.) At this time, when the ^R T _F value is stored in the reference data storage unit 153 , the recovery unit 157 (1) When using an equation and the ^R T _{H and} ^F T _H ^{values are stored in the reference data storage unit 153 , the F} T _F' value may be calculated using the equation (2).

^{The recovery unit 157 is a femoral marker after deformation of the femur marker 10 based on the correlation (F} T _F' ) regarding the change in the position and posture of the femoral marker 10 before and after the deformation of the femoral marker 10 occurs. It can be reset by deriving a correlation between (10) and the hip joint center (HJC).

[Equation 2]

^F' T _H = inv( ^F T _F' )x ^F T _H

(Here, ^F' T _H means the transformation matrix of the hip joint center (HJC) with respect to the femur marker 10 after deformation of the femur marker 10)

In the same way, the recovery unit 157 uses the transformation matrix of the tibial marker 20 with respect to the robot marker 31 before and after the deformation of the tibial marker 20, and the tibia before and after the deformation of the tibial marker 20 occurs. A correlation regarding a change in the position and posture of the marker 20 may be derived as follows.

[Equation 3]

^T T _T' = inv( ^R T _T )x ^R T _{T' … … … … … … … … …} (3)

= inv( ^R T _A x inv( ^T T _A ))x ^R T _{T' … … … … … … … … … … …} (4)

(Here, ^T T _T' is a transformation matrix related to a change in the position and posture of the tibial marker 20 before and after deformation of the tibial marker 20, ^R T _T' is a robot marker after the deformation of the tibial marker 20 occurs. The transformation matrix of the tibial marker 20 with respect to (31), ^R T _T is the transformation matrix of the tibial marker 20 with respect to the robot marker 31 before the deformation of the tibial marker 20, ^R T _A is the tibial marker ( The transformation matrix of the angle joint center (reference position) for the robot marker 31 before the deformation of 20), ^T T _A is the angle joint center (reference position) for the tibial marker 20 before the deformation of the tibial marker 20 occurs. ) of the transformation matrix.) At this time, when the ^R T _T value is stored in the reference data storage unit 153 , the recovery unit 157 (3) ^{Equation is used and when R} T _{A and} ^T T _A values are stored in the reference data storage unit 153 , ^{the T} T _T' value may be calculated using Equation (4).

^{The recovery unit 157 is a tibial marker after deformation of the tibial marker 20 based on the correlation (T} T _T' ) regarding the change in the position and posture of the tibial marker 20 before and after the deformation of the tibial marker 20 occurs. It can be reset by deriving a correlation between (20) and the center of the ankle joint (AJC).

[Equation 4]

^T' T _A = inv( ^T T _T' )x ^T T _A

(Here, ^T' T _A means the transformation matrix of the center of the ankle joint (AJC) with respect to the tibial marker 20 after the deformation of the tibial marker 20)

The correlation between the object markers 10 and 20 reset by the recovery unit 157 and the surgical objects 1 and 2 is stored in the object matching unit 152a and the robot matching unit 152b, and the position calculating unit 154 ) may track the position and posture of the surgery object 1 and 2 based on the correlation between the newly set object markers 10 and 20 and the operation object 1 and 2 .

If the robot marker 31 is deformed by an external force or the like, it is preferable to separately provide a restoration marker for checking the robot marker 31 .

In the above-described embodiment, the reference position of the reference point on the object markers 10 and 20 is stored, the position of the object markers 10 and 20 is tracked during surgery, and the change of the reference point is confirmed therefrom. , 20) was determined, but according to another embodiment of the present invention, a reference position relationship with respect to the transformable positions of the object markers 10 and 20 as the reference position of the reference point is calculated, and the reference position storage unit calculates the reference position relationship. (153) can also be stored.

The marker deformation determining unit 155 obtains position and posture information of the object markers 10 and 20 from the tracker 40 , and the position and posture information of the object markers 10 and 20 are stored in the reference position storage unit 153 . By determining whether the reference positional relationship is satisfied, it is possible to determine whether the marker is deformed. Referring to FIG. 3 , the object markers 10 and 20 may be located only on the spherical surfaces of A1 and A2 based on the reference position of the reference point. For example, the spherical surfaces of A1 and A2 may be the reference positional relationship.

In the above-described embodiment, the position and posture of the reference point are calculated from the position and posture information of the object markers 10 and 20, whereas in this embodiment, the position and posture information of the object markers 10 and 20 is the reference positional relationship. There is a difference in that it checks that you are satisfied.

8 is a flowchart illustrating a method of determining whether the object markers 10 and 20 are deformed by the surgical navigation device according to an embodiment of the present invention. A description that overlaps with the above-described embodiment will be omitted. Referring to FIG. 8 , in the surgical navigation method according to an embodiment of the present invention, the tracker 40 is first through a process of recognizing the positions of the object markers 10 and 20 and the surgical objects 1 and 2 using a probe. ), the first image of the marker acquired by ) and the second image of the surgical object 1 and 2 acquired in advance before surgery are matched (S10). Through such a registration process, a correlation between the object markers 10 and 20 and the surgical object 1 and 2 is derived (S11).

At this time, a reference position with respect to a reference point, which is a reference point for movement of the object, among a plurality of points of the operation object 1 and 2 is set and the corresponding position value is stored (S12). In the present embodiment, it will be described that the surgical subjects 1 and 2 include the femur 1 and the tibia 2 , and the reference point includes the center of the hip joint and the center of the ankle joint. In addition, the reference position of the reference point is stored as a position value based on the coordinate system of the tracker 40 .

During surgery, the position and posture of the object markers 10 and 20 are acquired through the tracker 40 (S13), and the reference of the object markers 10 and 20 calculated in the above-described registration process and the operation object 1, 2 The position and posture of the reference point are derived using the correlation between the position and posture of the point (S14). The correlation between the object markers 10 and 20 attached to the surgery objects 1 and 2 and the reference point is not changed by the movement of the tracker 40 or the movement of the operation objects 1 and 2 .

The marker deformation determining unit 155 determines whether the position of the reference point is the same as the reference position stored in the reference position storage unit 153 (S15). If they do not match, it is determined that the object markers 10 and 20 are deformed (S16), and if they match, it is determined that the object markers 10 and 20 are in a normal state (S17).

9 is a flowchart illustrating a method of determining whether the object markers 10 and 20 are deformed by the surgical navigation device according to another embodiment of the present invention. A description that overlaps with the above-described embodiment will be omitted.

Referring to FIG. 9 , in the surgical navigation method according to another embodiment of the present invention, the tracker 40 is first through a process of recognizing the positions of the object markers 10 and 20 and the surgical objects 1 and 2 using a probe. ), the first image of the marker acquired by ) and the second image of the surgical object 1 and 2 acquired in advance before surgery are matched (S20). Through such a registration process, a correlation between the object markers 10 and 20 and the surgical object 1 and 2 is derived ( S21 ).

At this time, the reference positional relationship with respect to the changeable positions of the object markers 10 and 20 is calculated based on the position of the reference point, which is the reference point for the movement of the object, as the reference position among the plurality of points of the operation object 1 and 2 . It is stored in the location storage unit 153 (S22). Referring to FIG. 3 , the spherical surfaces of A1 and A2 may be a reference positional relationship.

During surgery, the position and posture of the object markers 10 and 20 are acquired through the tracker 40 (S23), and the position and posture of the object markers 10 and 20 are stored in the reference position storage unit 153. Reference positional relationship It is determined whether or not is satisfied (S24). If the positions and postures of the object markers 10 and 20 are out of the reference positional relationship, it is determined that the object markers 10 and 20 are deformed (S25), and if the reference positional relationship is satisfied, the object markers 10, 20) is determined to be in a normal state (S26).

10 is a flowchart illustrating a method of determining whether the object markers 10 and 20 are deformed and a recovery method by the surgical navigation device according to another embodiment of the present invention.

Referring to FIG. 10 , in the surgical navigation method according to an embodiment of the present invention, the tracker 40 through a process of recognizing the positions of the object markers 10 and 20 and the surgical objects 1 and 2 using a probe. The first image of the marker acquired by , and the second image of the surgical object 1 and 2 acquired in advance before surgery are matched (S30). On the other hand, a correlation with respect to the position and posture of the robot and the robot marker 31 is derived through the matching of the robot. In addition, the robot is placed by moving it to the operating area, and the surgery objects 1 and 2 are fixed. A correlation between the position and posture between the robot marker 31 and the object and the correlation between the position and posture between the robot marker 31 and the surgical object 1 and 2 are derived through the tracker 40 ( S31 ).

At this time, a reference position with respect to a reference point, which is a reference point for movement of the object, among a plurality of points of the operation object 1 and 2 is set and the corresponding position value is stored (S32). Here, the reference position with respect to the reference point includes position and posture information based on at least one of the coordinate system of the tracker 40 , the coordinate system of the robot marker 31 , and the coordinate system of the base of the surgical robot 30 . In this embodiment, the reference position of the reference point based on the coordinate system of the robot marker 31 or the surgical robot 30 may be used during recovery, and the determination of whether the object marker is deformed is determined by the coordinate system of the tracker 40, the robot marker Any one of the coordinate system of (31) and the coordinate system of the surgical robot 30 may be used as a reference. ^{In addition, the correlation ( R} T _H , ^R T _A ) regarding the position and posture between the robot marker 31 and the reference position of the reference point calculated by the robot matching unit 152b, the robot marker 31 and the object marker ( ^{10 and 20), the correlations R} T _F and ^R T _T with respect to the positions and postures are stored in the reference position storage unit 153 during recovery and used during recovery.

The position and posture of the object markers 10 and 20 are acquired through the tracker 40 during the operation (S33), and the reference points of the object markers 10 and 20 calculated in the above-described process and the operation objects 1 and 2 The position and posture of the reference point are calculated using the correlation between the position and the posture of ( S34 ).

The marker deformation determining unit 155 determines whether the position of the reference point is the same as the reference position stored in the reference position storage unit 153 (S35). If they do not match, it is determined that the object markers 10 and 20 are deformed ( S36 ), and recovery is performed by the recovery unit 157 . On the other hand, when the position of the reference point is the same as the reference position, it is determined that the object markers 10 and 20 are in a normal state (S39)

The recovery unit 157 uses the robot marker 31 to determine the correlation regarding the position and posture between the object markers 10 and 20 and the surgical object 1 and 2 based on the changed object markers 10 and 20 . Reset. The recovery unit 157 determines the positions and postures of the object markers 10 and 20 based on a change in the correlation between the object markers 10 and 20 and the robot marker 31 before and after deformation of the object markers 10 and 20 occurs. A correlation is derived (S37), and based on this, the object markers 10 and 20 after deformation of the object markers 10 and 20 and the surgical objects 1 and 2, such as the surgical objects 1 and 2, are derived. can be reset by deriving a correlation between the reference points of ( S38 ).

The correlation between the object markers 10 and 20 reset by the recovery unit 157 and the surgical objects 1 and 2 is stored in the object matching unit 152a and the robot matching unit 152b, and the position calculating unit 154 ) may track the position and posture of the surgery objects 1 and 2 based on the correlation between the newly set object markers 10 and 20 and the surgery objects 1 and 2 .

1: Femur 2: Tibia
10: femur marker 20: tibia marker
30: surgical robot 31: robot marker
40: tracker 100: surgical navigation device
110: signal receiving unit 120: user input unit
130: display unit 140: memory unit
150: control unit 151, 152a: object matching unit
152: matching part 152b: robot matching part
153: reference position storage unit 154: position calculation unit
155: marker deformation determination unit 156: GUI generation unit
157: recovery unit

Claims (10)

In the surgical navigation device,
A first image including an object marker attached to a surgical object and a second image of a pre-surgery object are matched to derive a correlation with respect to the position and posture between the object marker attached to the operation object and the operation object matching part;
a reference position storage unit for setting and storing a reference position with respect to at least one reference point of the operation target;
a position calculator for receiving position and posture information of the object marker from a tracker, and calculating a position of the reference point of the surgical object based on the derived correlation;
a marker deformation determining unit that determines that the object marker is deformed when the position of the reference point calculated by the position calculating unit deviates from the reference position;
A robot for deriving a correlation with respect to the position and posture between the robot marker and the surgical object, and a correlation regarding the position and posture between the robot marker and the reference point, based on an image including the robot marker attached to the surgical robot matching part; and
Based on the change in the correlation between the object marker and the robot marker before and after the deformation of the object marker, a correlation with respect to the position and posture of the object marker before and after deformation is derived, and in the correlation before and after deformation of the object marker Surgical navigation device, characterized in that it comprises a recovery unit for resetting the correlation between the object marker and the operation object based on the operation.
In the surgical navigation device,
A first image including an object marker attached to a surgical object and a second image of a pre-surgery object are matched to derive a correlation with respect to the position and posture between the object marker attached to the operation object and the operation object matching part;
a reference position storage unit for calculating and storing a reference positional relationship with respect to a changeable position of the object marker using at least one reference point of the surgical object as a reference position;
a marker deformation determining unit that receives position and posture information of the object marker from a tracker and determines that the object marker is deformed when the position and posture of the object marker deviates from the reference position relationship;
A robot for deriving a correlation with respect to the position and posture between the robot marker and the surgical object and a correlation regarding the position and posture between the robot marker and the reference point based on an image including the robot marker attached to the surgical robot matching part; and
Based on the change in the correlation between the object marker and the robot marker before and after the deformation of the object marker, a correlation with respect to the position and posture of the object marker before and after deformation is derived, and in the correlation before and after the deformation of the object marker Surgical navigation device, characterized in that it comprises a recovery unit for resetting the correlation between the object marker and the operation object based on the operation.
delete 3. The method of claim 1 or 2,
the surgical subject includes at least one of a femur and a tibia;
The reference point is a surgical navigation device, characterized in that it includes at least one of the center of the hip joint and the center of the ankle joint.
5. The method of claim 4,
The reference position of the at least one reference point of the surgical object is a position based on at least one of the coordinate system of the tracker, the coordinate system of the surgical robot, and the coordinate system of the robot marker.
In the surgical navigation method,
Deriving a correlation with respect to the position and posture between the object marker attached to the operation object and the operation object by matching the first image including the object marker attached to the operation object with the second image about the operation object before surgery ;
setting and storing a reference position with respect to at least one reference point of the surgical object;
Deriving a correlation with respect to the position and posture between the robot marker and the surgical object and a correlation regarding the position and posture between the robot marker and the reference point based on an image including the robot marker attached to the surgical robot ;
receiving position and posture information of the subject marker from a tracker, and calculating a position of the reference point of the surgical subject based on the derived correlation;
determining that the object marker is deformed when the calculated position of the reference point deviates from the reference position;
deriving a correlation with respect to the position and posture of the object marker before and after deformation based on a change in the correlation with respect to the position and posture between the object marker and the robot marker before and after deformation of the object marker; and
and resetting a correlation regarding a position and posture between the object marker and the reference point based on a correlation before and after deformation of the object marker.
In the surgical navigation method,
matching the first image including the object marker attached to the surgical object with the second image of the preoperative surgical object;
deriving a correlation with respect to a position and posture between an object marker attached to the operation object and the operation object;
calculating and storing a reference positional relationship with respect to a changeable position of the object marker using at least one reference point as a reference position;
Deriving a correlation with respect to the position and posture between the robot marker and the surgical object and a correlation regarding the position and posture between the robot marker and the reference point based on an image including the robot marker attached to the surgical robot ;
receiving position and posture information of the object marker from a tracker; and
determining that the object marker is deformed when the position and posture of the object marker deviates from the reference position relationship;
deriving a correlation with respect to the position and posture of the object marker before and after deformation based on a change in the correlation with respect to the position and posture between the object marker and the robot marker before and after deformation of the object marker; and
and re-establishing a correlation with respect to a position and posture between the object marker and the point based on a correlation before and after deformation of the object marker.
delete 8. The method of claim 6 or 7,
The surgical navigation method further comprising the step of generating and outputting a warning message when the deformation of the object marker occurs.
8. The method of claim 6 or 7,
the surgical subject includes at least one of a femur and a tibia;
the reference point includes at least one of a hip joint center and an ankle joint center;
The reference position of the reference point is a position based on at least one of the coordinate system of the tracker, the coordinate system of the surgical robot, and the coordinate system of the robot marker.

Images