KR20000011134A - Stereotactic surgical procedure apparatus and method - Google Patents

Abstract

PURPOSE: An apparatus for stereo tactic surgical procedure is provided to permit accurate computer-based planning of the insertion point and angle of approach of a needle, a drill, a screw, a nail, a wire or other surgical instruments into the body of a patient. CONSTITUTION: An apparatus and method are provided for coordinating two fluoroscope images (62, 86), which permit accurate computer-based planning of the insertion point and angle of approach of a needle, drill, screw, nail, wire or other surgical instrumentation into the body of a patient, and subsequently guide the surgeon in performing the insertion in accordance with the plan.

Description

Stereotactic Surgical Method and Device

The present invention is Grant No. awarded from the National Science Association. It is made with the support of the US Government under DMC-8857854, and the US Government has certain rights in the invention.

The present invention relates to an apparatus and method for designing and guiding insertion of an object into a human body along a linear track. More specifically, the present invention relates to a method and apparatus for adjusting two captured fluoroscope images such that only two-dimensional images can be used to design an effective three-dimensional path.

In medical practice, needles, drills, screws, nails, wires, or other devices are often inserted into the human body. In some cases, both the angle and the location of such a device can be very important, for example, drilling holes to fasten screws along the axis of the spine. In other cases, positioning the peaks of such devices may be the most important, for example placing a biopsy needle on a prospective tumor. In another case, the purpose may be to define a point rather than a line, for example, to treat the tumor with a target for radiotherapy. There are many other examples, especially in the field of orthopedic surgery.

The invention also relates to improvements in treatments that penetrate the skin. Determining a linear path for inserting an instrument through the skin into the human body is more difficult than in the case of an open surgical technique, but it would be required to reduce damage or impact of skin tissue.

Fluoroscopy techniques are commonly used to assist medical practice. Continuous fluorescence in surgical operations is undesirable because the doctor's hand is exposed to radioactive material. In addition, if fluorescence is performed continuously or intermittently, the resultant image is two-dimensional, but the doctor must insert a medical device in consideration of three-dimensional space.

The method and apparatus of the present invention include capturing and storing two independent fluorescence images of the human body, which are taken in two different directions. Typically, but not always, they can be anterior / rear (A / P) images taken from the front and back of the patient and sagittal images taken laterally from the side. These two fluorescence images are displayed on two adjacent computers. The surgeon uses a trackball or other computer input device to specify the insertion point or insertion path on the monitor.

And the doctor uses the insertion of a medical instrument, the mechanical positioning device is used for positioning the guide device. This positioning device may be an actively computer-controlled manipulator, such as a robot, or may be a manually operated instrument device numerically set in accordance with an output from a computer.

The method and apparatus of the present invention, regardless of where fluorescence is performed, develops the geometrical relevance of the projections associated with each of the two received fluorescence images to create a three dimensional workspace around or inside the patient's body. The two images then become coordinated pairs, which allows to achieve a three-dimensional design, which would have required a computed tomography (CT) scan without this method.

Acquiring and displaying two images that are nearly orthogonal may be required to enable the physician to design three dimensions, but the two images are not necessary. In some cases it is also possible to use a single collected image, especially if the physician has adjusted the beam axis of the fluorescence to the intended trajectory. Also, two or more images may be acquired and coordinated, which is more desirable.

Some other methods for stereotactic or robotic surgery designed on computer screens displaying medical images have been described by other technicians and will be listed below. Some background discussion is mentioned before describing the prior art. The method and apparatus of the present invention constitute a technique called coordinated fluoroscopy. Coordinated fluorescence is a technique for registration and surgical design. It enables registration based on the fluorescence images required by themselves without requiring any additional measuring device. It does not require three-dimensional images, such as computed tomography (CT), and does not require two fluorescence images to be obtained from a right-angle fluorescence pose, three-dimensional surgery based on fluorescence photographs from two angles. Enable design

REGISTRATION

Registration is the most important step in an image-guided surgical system. Registration is a determination regarding the correspondence between the image points at which the surgical plan is prepared and the points in the workspace near (and inside) the patient.

It is common to register with the aid of a global positioning system, which is usually optical, that can measure the three-dimensional coordinates of a marker that exists somewhere in large space. Coordinated fluorescence eliminates the need for such an expensive and inconvenient device and induces direct registration from the resulting fluorescence image itself. Coordinated fluorescence uses a "registration artifact" obtained at a fixed position relative to the patient when one or more images are obtained from different angles (poses). There is no need to limit the fluorescence poses from which these various images are obtained, for example, there is no need for them to be at right angles, nor does there need to be a mechanism for fluorescence to measure pose angles. Instead, the pose information is subsequently extracted from the image. An important advantage of the invention is that it is possible to obtain fluorescence images using fluorescence poses of the physician's own choice, depending on the circumstances in which they are needed.

The registration construct includes a variety of features (fiducials) designed to easily identify on one fluorescence image. The embodiment described herein uses eight small steel spheres implemented in the light emission matrix. These reference points are known in the context of coordinate systems fixed to the artificial structure by design or measurement.

From the two-dimensional position of the projection of these reference points in one fluorescent image, one can derive a general three-dimensional point somewhere near the artificial structure into the projected image on the image. This forms a registration with the image and workspace company. Several images may be registered respectively with respect to the same registration structure, thus leading all the images to the registration with each other.

Recognition of the geometric projection as described above is not possible from the fluorescence image itself, since it is very nonlinear and distorted. You must first map or compensate for these distortions. When comparing the present invention with the prior art, the need for distortion compensation should be recognized.

SURGICAL PLANNING

It is also an important step in surgery that is guided by surgical design images. Design of three dimensional surgical behavior can be expected to be performed on a three dimensional data set, which can be reconstructed from computed tomography (CT) data. However, doctors are used to designing on two-dimensional images, radiographs and fluorescence images. In fact, even when CT data is available, it is done on individual two-dimensional CT "pieces" rather than taking the way by three-dimensional reconstruction.

The coordinates of the tip of the line segment representing the intended screw, biopsy needle, or drilled hole are, of course, three-dimensional, as well as the coordinates of a single point in the human body indicating the current location of the tumor or fragment. In surgical design, such points may be embodied on one two-dimensional image, or may be embodied on each of several two-dimensional images. Each such two-dimensional image is a projection of the same three-dimensional space.

It is necessary to convert the two-dimensional coordinates of the materialized points for each of several images into three-dimensional coordinates that can be used to guide the tool along the intended trajectory or to some point in the human body. To do so, you must have knowledge of the geometric relationship of the projection that created the image.

Without such geometric knowledge, the points embodied on one image and the embodied points separately on another image virtually do not coincide with any one point in the human body. This is because the point embodied on one two-dimensional image is a projection of a line in space. The point implied in three dimensions is the intersection of two such lines, implied by the points embodied in each image. Two such lines can have independent angles of inclination and no intersection. Similarly, line segments for the intended behavior cannot be selected independently in two images, otherwise they will generally not match well defined three-dimensional line segments.

In coordinated fluorescence, geometric projection, which involves two images into one three-dimensional coordinate system, is formed before the start of the design. Therefore, the points selected by the physician on two or more images can be limited by software such that they exactly match the points on a well-defined three-dimensional image. In fact, when the physician adjusts the intended point or line segment to one image, the point or line segment displayed in the other image (s) also continues to update and adjust. It is not possible to draw "random" points or line segments independently on an image. In other words, only software can form points or line segments that match well-defined three-dimensional point or line segments.

The advantages of designing on geometrically coordinated images as described above are summarized in the following three ways.

1) Once the physician has selected one point or line segment on two images, the three-dimensional point or line segment to which the selections correspond is fully defined and ready for execution.

2) As can be seen from the CT slice, the axial picture is difficult to obtain by fluorescence. The most easily visible angle in the axial direction is known as the transverse angle, which is costly to select and implement under fluorescence. In coordinated fluorescence, the transverse angle is implicitly specified by the physician selecting a line segment on two images. This can help the physician visualize and design the transverse angle for medical practice.

3) In the conventional fluorescence test, the image expansion due to the dispersion of the beam is not well known, which makes it difficult to maintain accuracy in the anatomical distance measurement. In coordinated fluorescence, the actual simultaneous length of the intended line segment can be determined by software. This is useful for selecting an appropriate slope length or for other purposes.

Lavalle et al., Grenoble, France, have developed a system for spinal surgery that uses computed tomography as an image source. The CT data are combined into a three dimensional data set, which can be freely manipulated again on the orthogonal plane. The surgical design proceeds simultaneously on three mutually orthogonal planes. Registration is performed by using an optical tracking device to digitize any surface points of the spine, matching those surface points with a CT data set.

Nolte, Bern, Switzerland, developed a spinal system very similar to Ravel et al. The difference in registration is that the optical tracking device is used to digitize certain anatomical issues rather than the general surface contours. The features are manually pointed out in the CT data, which makes the coincident operation possible.

Blood of the British High Wycombe. P. Finlay has developed a fluorescence system for femoral (hip) fractures. The requirement for accuracy in this procedure is not very large and compensation of fluorescence distortion is not required. The lack thereof makes it impossible to identify geometric projections from the image as performed in the present invention. Instead, two fluorescence distortion compensations are required to be orthogonal and the C-arms should not be moved along the floor between the two images. Registration is performed by recognizing the various features of the surgical instrument that appears in the image, and by highlighting the marker wire that appears in the photograph of the fluorescence. Potamianos, London, UK, has developed a system for renal biopsy and similar soft tissue medical practice. It includes a digitizing mechanical arm that has a biopsy needle attached and can be moved manually by a physician. There is no surgical plan per se, but instead a superimposed line segment representing the current position of the needle is displayed on the (corrected) fluorescence image collected as the needle is manually moved in and near the patient.

Phillips of Hugh, UK, has developed an orthopedic system. This uses an optical tracking device as well as fluorescence. Registration is done by using fluorescence with light emitting diodes and tracking them with an optical tracking device. Surgical design software uses specific to surgical operations and provides medical findings rather than merely displaying trajectories as in the present invention. For placement of the spinal cord nails, for example, the surgeon outlines the target hole of the prosthetic spinal cord and the software calculates the trajectory through them.

US Pat. No. 4,750,487 (Zanetti) discloses a stereotactic frame for a patient. Front / back fluorescence is obtained, where the crosshairs are attached to the frame. By measuring the placement of the crosshairs from a given target, a frame movement is made that keeps the two in a straight line. This invention does not facilitate the three-dimensional stereotactic which the present invention considers.

US Pat. No. 5,078,140 to Kwoh describes neurosurgical stereotactic and robotic systems.

In the present invention, a method of designing a stereotactic surgical procedure for inserting a surgical tool along a linear trajectory into a human body using a fluoroscope to produce an image of the human body is provided. The method involves positioning a registration artifact comprising a plurality of baselines near the human body, displaying an image of the patient's human body and the registration artifact on a computer monitor, and registering the registration artifact displayed on the first monitor. Receiving input from a user or an automatic algorithm to identify a two-dimensional baseline coordinate of the image, and registering the image by creating a geometric model having parameters and projecting the three-dimensional coordinates to the image points, and a known reference point Their projection of the three-dimensional coordinates numerically optimizes the parameters of the geometric model so that they best fit the recognized two-dimensional coordinates of the image.

In addition, the method may further comprise displaying, on an angle different from the first image, a second image taken from the patient's human body and the registered artificial structure on a second computer monitor, identifying two-dimensional baseline coordinates displayed on the second computer monitor. Receiving input of a user or an automatic calculation method, and registering a second image by generating a geometric model having parameters and projecting the three-dimensional coordinates to the image points, and of the three-dimensional coordinates of known reference points. Numerically optimizing the parameters of the geometric model such that the projection best fits the recognized two-dimensional coordinates of the second image.

In addition, the method further includes receiving user input to select an insertion point for the surgical instrument on the computer monitor, whether one or more images are obtained. In the case of two images, it also represents the trajectory of the surgical instrument and draws segments, to select the position, length, and angle of the virtual guidewire, known as the projected guidewire, on the image (s). Receive user input. When there are two images, the projected guidewires are constrained to geometrically match the same three-dimensional segment in space, which is known as a virtual guidewire.

The method also allows the user to move any end of the projected guidewire by correcting the virtual guidewire onto which the projected guidewire (s) is projected, and by redrawing the projected guidewires in accordance with the calibrated virtual guidewire. Receiving input.

The method further includes receiving user input to change the length of the virtual guidewire, and redrawing the projected guidewire (s) in accordance with the calibrated virtual guidewire. There is also a special case of zero length, which is designed as a virtual target point rather than a virtual guidewire.

In addition, the method may be adapted to receive user input to change the angle (s) of the sagittal, transverse, or coronal of the virtual guidewire, or to direct the virtual guidewire based on the new angle. Updating and redrawing the projected guidewire (s) to match the calibrated virtual guidewire.

The method further includes generating an output to adjust the coordinates of the tool guide such that the projection of the guide axis in the image coincides with the input point displayed on the computer monitor.

In addition, the method can generate an output to adjust the coordinates of the instrument guide to match the virtual guidewire, or to control the position in the body of the surgical instrument to be inserted, so that the position of the guide along its axis is one of the virtual guidewires. And generating an output for adjusting the representation of the tool guidance to be offset by a predetermined distance from the endpoint.

The method further includes transmitting the coordinates to a robot or other automatic mechanical device, or displaying the coordinates so that an operator can manually adjust the mechanical device.

1 is a schematic diagram of an stereotactic surgical device of the present invention for coordinates an image from fluorescence, designing a medical behavior in a linear orbit, and controlling the robot to control the medical behavior in a linear orbit;

Figure 2 is a perspective view of the guide to the registered artificial structure and tool of the present invention,

3A is a sample screen play of a user interface including a front / rear (A / P) displayed on a first computer monitor with many buttons and input fields taken by fluorescence and needed to perform a program;

3B is a sample screen play that includes a skull image displayed on a second computer monitor with many buttons and input fields taken by fluorescence and needed to perform a program,

3C is a flow diagram relating to the steps performed by a computer during a main program loop;

4 is a flow chart of the steps performed by a computer to obtain an A / P image from fluorescence;

5 is a flow chart of the steps performed by a computer to obtain a skull image from fluorescence;

6 is a flow chart of the steps performed by the computer and the user to select or identify A / P reference points from the A / P image displayed in FIG. 3A;

7 is a flow chart of the steps performed by the computer and the user to select or identify A / P reference points from the A / P image displayed on the skull image of FIG. 3B;

8 is a flowchart of steps performed by a computer to register an A / P image;

9 is a flowchart of steps performed by a computer to register a skull image,

10 is a flowchart of steps performed by a computer to change the transverse angle of the virtual guidewire;

11 is a flow chart of the steps performed by a computer to vary the length of the virtual guidewire used in fluorescence passing;

12 is a flow chart of the steps performed by a computer to change the skull angle of the virtual guidewire;

13 is a flow chart of the steps performed by the computer to change the approach angle of the robot;

14 is a flow chart of the steps performed by a computer to move the robot shown in FIG. 1 to a designed position and orientation;

15 is a flow chart of the steps performed by the computer to move the distal actuator of the robot along the axis of the tool guide;

FIG. 16 is a flow chart of the steps performed by the computer when the computer accepts user input based on a cursor in the A / P image area of FIG. 3A;

FIG. 17 is a flow chart of the steps performed by the computer when the computer receives user input based on a cursor in the skull image area of FIG. 3B;

FIG. 18 is a flow chart of the steps performed by the computer when the computer accepts user input based on a cursor in the robot control area of FIGS. 3A-B;

Referring to the drawings, FIG. 1 illustrates a stereotactic system 10 for linear trajectory medical intervention using calibrated and coordinated fluoroscopy. The apparatus and method of the present invention utilize an image from a fluoroscopy device 12, such as a standard C-arm, which generates a fluoroscopy image or x-ray image of the body on the operating table 14. Imaging arm 16 is movable to obtain an anterior / rear (A / P) image and a sagittal or lateral image of the body.

The robot 18 is disposed adjacent to the operating table 14. For example, the robot is a PUMA-560 robot. The robot 18 includes a movable arm assembly 20 having an end flange 22. The alignment or adjustment member 24 is connected to the end flange 22 of the robot 18.

The adjusting member 24 is best shown in FIG. 2. The adjusting member 24 is transmissive and visually transparent except for eight opaque spheres or reference points 26 and openings 30 holding the tool guide 28 through the adjusting member 24. . Initially, the adjustment member 24 is disposed approximately in the field of view of the fluoroscopy apparatus over the relevant site of the body 32. Thus, the reference point 26 appears as clear dots on the A / P image and the sagittal image as described below. The shape of the adjusting member is set so that the image dots from the reference point 26 are sensitive to angle deviations without overlapping each other. The robot arm 20 can adjust the adjusting member 24 in three dimensions with respect to the X-axis 34, the Y-axis 36, or the Z-axis 38 shown in FIG. 1.

The coordinated perspective control system of the present invention is controlled by a computer 40, which includes a microprocessor 42, an internal RAM 44, and a hard disk drive 46. Computer 40 is connected to two graphics monitors 48 and 50. The first graphics monitor 48 displays the sagittal image obtained by the C-arm 12. The computer 40 also includes a serial communication port 52 connected to the controller 53 of the robot 18. Computer 40 is also coupled to C-arm 12 to receive an image from C-arm 12 via image acquisition card 54. The computer 40 is also connected to an input device 56, which is a keyboard with a trackball control 58, for example. The trackball input control unit 58 controls the cursors on both monitors 48 and 50.

Displays on monitors 48 and 50 are shown in FIGS. 3A and 3B. According to FIG. 3B, the sagittal image is displayed in the area 62 on the monitor 48. All eight reference points 26 should appear in the sagittal image area 62. Otherwise, the means 24 or the C-arm 12 must be adjusted. As will be described in detail below, the computer 40 displays the upper entry point 64 and the lower point 66 of the projected guidewire 68. The projected guidewire 68 is a line segment displayed on the sagittal image area indicating the position of the instrument to be inserted during the stereotactic procedure. Aim line 70 is also displayed in sagittal image area 62.

Various user select buttons are displayed on the monitor 48. The surgeon or operator can access these selections by moving the cursor over the button and clicking or by using the appropriate function keys (F1, F2, etc.) on the keyboard. The selection button displayed on the monitor 48 includes a button 72 (function F2) for acquiring a sagittal image, a button 74 (F4) for selecting a trial reference point and a button 76 (F6) for registering a sagittal image. ). In addition, a button 78 (F10) for selecting a sagittal image is provided, a button 80 (F8) for setting a screw length is provided, and a button 82 (F12) for moving the robot along the axis of the tool guide. This is provided. Finally, the display screen includes a robot control area 84. The operator can move the cursor and click in the robot control area 84 to control the robot 18 as described below.

According to FIG. 3A, the A / P image displayed on the display screen of the monitor 50 is shown. The A / P image is displayed in area 86 of the screen. Again, all eight reference points 26 must appear within the A / P image area 86. The insertion point of the virtual guidewire is shown at position 88 and the lower point is disposed at position 90. Guidewire projection to the A / P image is shown by line segment 92.

Computer 40 also displays various selection buttons on monitor 50. The button 94 (F1) is provided to acquire an A / P image. Button 96 (F3) is provided to select an A / P reference point. Button 98 (F5) is provided to register the A / P image. Button 100 (F7) is provided to set the crossing angle of the virtual guidewire, and button 102 (F9) is provided to set the approach angle to the robot. Button 104 (F11) is provided to move the robot. Computer 40 also displays robot control area 84. The operator can move the cursor and click in the robot control area 84 to control the robot 18 as described below.

The present invention allows selecting the point at which the surgical instrument is inserted by moving the upper point of the projected guidewire 88 in the A / P image area 86. The operator can also adjust the bottom point of the projected guidewire 90 to specify the crossing angle and the sagittal angle. In addition, the operator adjusts the upper point of the projected guidewire 64 to specify the position on the aiming line, and adjusts the lower point of the projected guidewire 66 to adjust the sagittal angle and the transverse angle in the sagittal image area 62. Can be specified. Thus, the surgeon can select the desired position and orientation of the surgical instrument to the body.

The computer 40 is programmed with software for correcting spatial distortion from the optics of the perspective inspection apparatus 12. The system of the present invention enables stereotactic surgery using only a pair of two-dimensional fluorescence images on adjacent monitors 48 and 50. Use of a CT slice to globally specify the position of the surgical instrument is not required. The computer 40 forms a direct geometric relationship between the A / P and sagittal images despite image distortion and random or free positioning of the C-arms to form the A / P and sagittal images. The improved system of the present invention can form such precise geometric relationships at sub-millimeter precision.

Once the sagittal image and the A / P image are registered, the point or line selected by the surgeon on either the A / P image or the sagittal image is immediately displayed on the computer 40 as a corresponding projection on the other image. Thus, by using the sagittal image on the monitor 48 and the A / P image on the monitor 50, the surgeon can stereoscopically plan the linear trajectory without the CT scan slice. Thus, the present invention can be performed without expensive CT scan devices that cost more than $ 1 million.

The operation of the software controlling the system of the present invention is shown in detail in FIGS. 3C-18.

All signs, subscripts and mathematical laws, equations and explanations are included in the appended appendix. For all of the flowcharts shown in FIGS. 4-18, the reference refers to the appendices and areas [1] to [15] shown in the appendix.

The main program starts at block 110 of FIG. 3C. Computer 40 creates a parent window at block 112 and draws a button on the main window as shown in block 114. Computer 40 then creates a sagittal child window on monitor 48 as shown in block 116. Computer 40 also creates an A / P child window on monitor 50 as shown in block 118. Computer 40 then determines whether the button or key was pressed at block 120. If not pressed, computer 40 waits as shown in block 122 and then returns to block 120 to wait for a button or key to be pressed.

When the button or key is pressed at block 120, computer 40 determines whether the A / P image acquisition button 94 or F1 key has been pressed at block 124. If pressed, computer 40 proceeds to block 166 of FIG. If not pressed, the computer 40 determines whether the sagittal image acquisition button 94 or the F3 key has been pressed at block 126. If pressed, computer 40 proceeds to block 200 of FIG. If it is not pressed, the computer 40 determines whether the A / P reference point selection button 96 or the F3 key has been pressed at block 128. If pressed, computer 40 proceeds to block 234 of FIG. 6. If the button 96 or the F3 key was not pressed at block 128, the computer 40 determines whether the sagittal reference point selection button 74 or the F4 key has been selected as shown at block 130. If so, computer 40 proceeds to block 276 of FIG. If not, computer 40 proceeds to block 132.

In block 132, the computer 40 determines whether the A / P image registration button 98 or the F5 key has been pressed. If pressed, computer 40 proceeds to block 324 of FIG. 8. If not pressed, the computer 40 determines whether the sagittal image registration button 76 or the F6 key has been pressed as shown in block 134. If pressed, computer 40 proceeds to block 350 of FIG. 9. If not pressed, computer 40 proceeds to block 136.

From block 136, computer 40 determines whether the traverse angle button 100 or the F7 key was pressed as shown in block 138. If pressed, computer 40 proceeds to block 376 of FIG. If not pressed, the computer 40 determines whether the screw length button 78 or the F10 key was pressed as shown in block 142. If pressed, computer 40 proceeds to block 388 of FIG. 11. If not pressed, the computer 40 determines whether the sagittal angle button 78 or the F10 key has been pressed as shown in block 142. If pressed, computer 40 proceeds to block 400 of FIG. 12. If not pressed, computer 40 determines whether the approach angle button 102 or F9 key has been pressed as shown in block 144. If pressed, computer 40 proceeds to block 412 of FIG. 13. If not pressed, computer 40 proceeds to block 146.

In block 146, the computer 40 determines whether the robotic move button 104 or F11 key has been pressed. If pressed, computer 40 proceeds to block 422 of FIG. 14. If not pressed, the computer 40 determines whether the robotic move button 82 or the F12 key along the axis was pressed as shown in block 148. If pressed, computer 40 proceeds to block 452 of FIG. If it is not pressed, the computer 40 determines whether the A / P image area of the monitor 50 is selected by clicking as shown in block 150 when the cursor is in the A / P image area 86. . If so, computer 40 proceeds to block 476 of FIG. If not selected, the computer 40 determines whether the sagittal image area has been selected by placing and clicking the cursor on the sagittal image area 62 on the monitor 48.

From block 154, the computer 40 moves the cursor and clicks on the robot control area 84 on the monitor 48 or the robot control area 106 on the monitor 50 to control the robot control area 54 or 106. ) Is selected. If robot control has been selected, then computer 40 proceeds to block 536 of FIG. 18. If no robot control has been selected, the computer 40 proceeds to block 158 to determine if the "Q" key was pressed indicating the operator's intention to abort the main program. When the "Q" button is pressed, computer 40 is freed from all allocated memory as shown in block 160 and terminates the main program as shown in block 162. If the "Q" button is not pressed at block 154, computer 40 proceeds to block 122 and waits for another button or key to be pressed.

Various functions performed by the system of the present invention will be described. When the A / P image acquisition button 94 or the F1 key is pressed, the computer 40 proceeds to block 166 of FIG. 4. The computer 40 then determines whether the image acquisition card is in the pass-through mode at block 168. Button 94 and the F1 key are toggle buttons. When the button 94 or F1 key is initially pressed, the card is in pass-through mode and the image from the C-arm 12 is sent directly to the monitor 50. It is shown in the A / P image area 86 on the monitor 50 whether the image by the C-arm is acquired. Thus, if the card is not in the pass mode at block 168, the pass mode is set at block 170 by pressing button 94 or the F1 key. The computer 40 then returns to block 172 to wait for the next command. If the button 94 or F1 key is pressed again after the image acquisition card in the computer 40 has entered the pass-through mode, block 174 preserves the live image and captures the A / P image. The captured image is displayed on monitor 50 at block 176. Next, at block 178, computer 40 disables buttons F11, F12, and F5 to dimm, and enables button 96 and key F3 to lighten. In other words, after the A / P image is captured, computer 40 allows the operator to select an A / P reference point via button 96 or key F3.

Computer 40 then allocates a NULL tool at block 180. The NULL tool of the robot is a three dimensional arrangement of the end flange 22 of the robot 18. In other words, the end flange 22 forms a three-dimensional position with respect to the robot without depending on the particular surgical tool that may be attached to the end flange 22. Computer 40 determines whether a NULL tool has been properly assigned at block 182. If it has not been assigned, then computer 40 generates an error message at block 184 stating that "Tool not assigned." Next, the computer 40 waits for the next command at block 186. If the NULL tool has been properly assigned at block 182, computer 40 obtains the current position of the end flange from robot controller 53 at block 188. Computer 40 then determines whether the sagittal image is displayed on monitor 48 at block 190. If not displayed, computer 40 sends a " sagittal image acquisition " message at block 192 and returns to block 194 to wait for the next command. If the sagittal image is displayed at block 190, computer 40 transmits a “reference point selection” message at block 196. Computer 40 then returns to block 198 to wait for the next command.

When the sagittal image acquisition button 72 or the F2 key is pressed, the computer 40 proceeds to block 200 of FIG. 5. Computer 40 then determines in block 202 whether the image acquisition card is in a pass mode. Button 72 and the F2 key are toggle buttons. If the card is not in the pass mode at block 202, set the pass mode at block 204 by pressing button 72 or the F2 key.

Computer 40 then returns to block 206 to wait for the next command. Computer 40 then returns to block 206 to wait for the next command. If the button 72 or the F2 key is pressed again after the image acquisition card in the computer 40 has entered the pass mode, block 208 preserves the live image and captures the sagittal image. The captured image is displayed on the monitor 48 at block 210. In block 212 the computer 40 then disables and dims buttons F11, F12, and F6, and enables and brightens button 74 and key F3. In other words, after the sagittal image is captured, the computer 40 allows the operator to select the sagittal reference point via the button 74 or the key F4.

Computer 40 then assigns a NULL tool at block 214. Computer 40 determines in block 216 whether the NULL tool has been properly assigned. If not, computer 40 generates a "tool not assigned" error message at block 218. Computer 40 then waits for the next command at block 220. If the NULL tool is properly assigned at block 216, computer 40 obtains the current position of end flange 22 from robot controller 53 at block 222. Computer 40 then determines whether an A / P image is displayed on monitor 50 at block 224. If not displayed, computer 40 sends a " A / P image acquisition " message at block 226 and returns to block 228 to wait for the next command. If the A / P image is displayed at block 224, computer 40 sends a “reference point selection” message at block 230. Computer 40 then returns to block 232 to wait for the next command.

When the A / P reference point selection button 96 or the F3 key button is pressed, the computer 40 proceeds to block 234 of FIG. 6. First, computer 40 determines at block 236 whether an A / P image has been displayed on monitor 50. If not displayed, generate a "A / P image acquisition" error message at block 238. Computer 40 then returns to block 240 to wait for the next command.

If the A / P image is displayed at block 236, computer 40 displays a rectangular cursor on the display screen of monitor 50 at block 242. The computer 40 then resets the reference point number to zero at block 244. The computer 40 then waits for a trackball button to be clicked by the operator at block 246. When the trackball button is clicked on the reference point shadow, computer 40 generates a beep at block 248. Computer 40 then performs edge detection around the mouse cursor coordinates selected at block 250. Such edge detection is performed using the gradient reference method developed by John Canny and described in the paper referenced in section [1] of the appendix. The paper is incorporated herein by reference.

The computer 40 then determines whether at least three edge pixels are found during the edge detection step shown in block 252. If no edge pixel is found, the computer 40 generates a "access to reference point" error message at block 254. Computer 40 then returns to block 246 to wait for the mouse button to be clicked again. If at least three edge pixels were not found at block 252, computer 40 maps the edge pixels to their corrected image coordinates using equation [13] of the appendix appended at block 256.

The computer 40 then finds the center of the reference point shadow generated by the reference point 26 using the corrected edge pixels as described in equation [14] of the appendix. This step is shown at block 258. Computer 40 then proceeds to block 262 of FIG. From block 262, computer 40 draws a circle around the center of the reference point shadow.

Computer 40 then determines whether all eight reference points 26 have been placed in the A / P image at block 264. If not, computer 40 returns to block 246 in FIG. 6 and waits for a mouse button to be clicked on another reference point shadow.

If all eight reference points are placed at block 264, computer 40 stores image coordinates of all reference points in computer memory at block 268. Computer 40 then enables and brightens A / P image registration button 98 and the F5 key at block 270. The computer 40 then sends a "Register A / P Image" message at block 272.

Computer 40 then proceeds to block 274 at entry1 location in FIG. The computer 40 does not wait for the operator to press a button to move to the Entry 1 position of FIG. 8.

When the sagittal reference point selection button or the F4 key button is pressed, the computer 40 proceeds to block 276 of FIG. 7. First, computer 40 determines whether a sagittal image is displayed on monitor 48 at block 278. If not displayed, computer 40 generates a "thalamic image acquisition" error message at block 280. Computer 40 then returns to block 282 to wait for the next command.

If the sagittal image is displayed at block 278, computer 40 displays a rectangular cursor on the display screen of monitor 48 at block 290. The computer 40 then resets the number of reference points placed at block 292 to zero. The computer 40 then waits for the trackball button to be clicked by the operator at block 294. When the trackball button is clicked, computer 40 generates a dial tone at block 296. Computer 40 then performs edge detection around the trackball cursor coordinates selected at block 298. Edge detection is performed using the slope reference method developed by John Canny as described in the paper referenced in section [1] of the appendix.

Computer 40 then determines whether at least three edge pixels are found during the edge detection step shown in block 300. If no edge pixel is found, the computer 40 generates a "retrieval reference point" error message at block 302. The computer 40 then returns to block 294 to wait for the trackball button to be clicked again. If at least three edge pixels have not been found at block 300, computer 40 maps the edge pixels to their corrected image coordinates using equation [13] of the appendix appended at block 304.

The computer 40 then finds the center of the reference point shadow generated by the reference point 26 using the corrected edge pixels as described in equation [14] of the appendix. This step is shown in block 306. Computer 40 then proceeds to block 310. From block 310, computer 40 draws a circle around the center of the reference point shadow.

Computer 40 then determines in block 312 if all eight reference points 26 have been placed in the sagittal image. If not, computer 40 returns to block 294 and waits for the trackball button to be clicked again.

If all eight reference points are placed at block 312, computer 40 stores image coordinates of all reference points in computer memory at block 316. The computer 40 then enables and brightens the sagittal image registration button 76 and the F6 key at block 318. The computer 40 then sends a "sagittal image registration" message at block 320.

The computer 40 then proceeds to block 322 to the entry2 location in FIG. 9. The computer 40 does not wait for the operator to press the button to move to the entry 2 position of FIG. 9.

When the A / P image registration button 98 or F5 key is pressed, the computer 40 proceeds to block 324 of FIG. First, computer 40 determines whether all A / P image reference points have been found at block 326. If not found, the computer 40 generates a "not all reference points selected" error message at block 328. Computer 40 then returns to block 330 to wait for the next command.

If all A / P reference points have been found at block 326, the computer 40 proceeds to block 332. As discussed above, the computer 40 also proceeds from block 274 of FIG. 6 to block 332 after all reference points have been selected.

At block 332, computer 40 first retrieves all two-dimensional coordinates of the A / P reference point center. The computer 40 then reads data from the three-dimensional coordinate file of the center of the reference point 26 at block 334. The three-dimensional coordinates of the reference point 26 are obtained using a coordinate measuring machine (CMM). Thus, the data provides information regarding the actual position of the reference point 26. In general, the coordinates by these CMMs are obtained from the manufacturer of the adjusting member 24.

The computer 40 then optimizes the parameters of the geometric model that projects the three-dimensional coordinates to the corresponding image points. The optimized model is encapsulated in the registration matrix described in section [3]. Optimization is performed by minimizing the difference between the model projection of the three-dimensional coordinates read at block 334 and the model projection of the two-dimensional coordinates read at block 334. The Levenberg-Market method is used for optimization as described in equation [2] of the appendix appended to block 336. Computer 40 then forms the registration matrix described in the appended section [3]. This step is shown in block 338.

Next, computer 40 determines whether the sagittal closure image has been written as described in block 340. If not, computer 40 generates a message of " launch sagittal image execution " as described in block 342. < Desc / Clms Page number 10 > The computer 40 then returns to wait for the next command as described in block 344.

If the sagittal stitched image has been registered at block 340, computer 40 generates a display message of “selection entry point” as described at block 346. The computer 40 then returns to wait for the next command as described in block 348.

If register sagittal image button 76 or F6 is pressed, computer 40 proceeds to block 350 of FIG. The computer 40 first determines whether all sagittal closure reference points are found as described in block 352. If not, computer 40 generates an error message that "not all reference points have been selected" as described in block 354. Computer 40 then returns to wait for the next command as described in block 356.

If all sagittal reference points are found at block 352, computer 40 proceeds to block 358. As discussed above, the computer 40 automatically advances from block 322 to block 358 in FIG. 7 after all reference points have been selected.

At block 358, computer 40 first calls all two-dimensional coordinates of the sagittal reference point center. Next, computer 40 reads data from a file of three-dimensional coordinates with respect to the center of reference point 26 as described in block 360. The coordinates of the reference point 26 are obtained using a coordinate measuring machine (CMM). Thus, this informational information provides information relating to the actual location of the reference point 26. Typically, these coordinates are obtained from the manufacturer of the registration artifact 24.

Next, the computer 40 adds a pit between the three-dimensional coordinates read in block 360 and the two-dimensional coordinates read in block 358 as described in block 362 [2]. Optimize using the Levenberg-Marquardt method described by The computer 40 then constructs a registration matrix as described in section [4] of the appendix. This step is described in block 364.

Next, computer 40 determines whether the A / P image has been registered as described in block 366. If not, the computer 40 generates a message of "execute A / P registration" as described in block 368. Computer 40 returns to wait for the next command as described in block 370.

If the A / P image has been registered at block 366, computer 40 generates a message of "selection entry point" as described at block 372. Computer 40 returns to wait for the next command as described in block 374.

If the cross angle button 100 or the F7 key is pressed, the computer 40 proceeds to block 376 of FIG. 10. The traversing angle is the angle determined using the right hand rule with respect to the X axis 34 of FIG. 1. To adjust the traverse angle, the operator sets the position of the cursor with the entry field button 101 of FIG. 3A as described in block 378 of FIG. 10. The operator then enters a numerical value for the crossing angle as described in block 380. The computer 40 then reads the new crossing angle and updates the direction of the virtual guidewire using the equations described in section [6] of the appendix. This step is described in block 382. The computer 40 then uses the equations described in section [7] of the appended appendix as described in block 384 to the A / P image area of the sagittal image area 62 based on the new cross angle. Redraw the virtual guide and eye projection 92 on (86, 68). The computer 40 then returns to wait for the next command as described in block 386.

When the thread length button 80 or the F8 key is pressed, the computer 40 proceeds to block 388 of FIG. 11. The cursor is then placed on the entry field 81 of FIG. 3B as described in block 390. The operator then enters a numerical value for the new thread length as described in block 392. The computer 40 reads the new screw length and updates the length of the virtual guidewire using the equation described in section [11] of the appendix. This step is described in block 394. Next, the computer 40 redraws the guidewire 92 projected on the A / P image area 86 using the equation described in section [7] of the appended appendix to the sagittal image area 62. Redraw the projected guidewire 68. These steps are described in block 396. The computer 40 then returns to wait for the next command as described in block 398.

If the sagittal closure button 78 or the F10 key is pressed, the computer 40 proceeds to block 400 of FIG. 13 to adjust the sagittal closure angle. The sagittal closure angle is the angle to the Y-axis 36 of FIG. 1 using the right hand rule.

The cursor is located in the entry field 79 of FIG. 3B as described in block 402. The operator then enters a numerical value for the sagittal closure angle, as described in block 404. The computer 40 reads the value of the new sagittal closure angle and updates the orientation of the virtual guidewire using the equations described in section [10] of the appendix.

This step has been described at block 406. The computer 40 then draws the protruding guide wire 92 back into the A / P image area 62 using the equation developed in section [7] of the appendix and sagittal image the protruding guide wire 68. Redraw in area 62. This step is described at block 408. Computer 40 then returns to wait for the next command as indicated at block 410.

When the approach angle button 102 or the F9 key is pressed, the computer 40 proceeds to block 412 of FIG. The approach angle is the angle taken with respect to the Z axis 38 of FIG.

As shown at block 414, the cursor is located in the entry field 103 of FIG. 1. The operator enters a new value for the approach angle as shown at block 416. Computer 40 reads the new approach angle as shown at block 418. The computer 40 then waits for the next command, as shown in block 420.

To plan a linear trajectory in space, only two angles are needed, and for this particular procedure the transverse and sagittal angles are used. The approach angle allows the surgeon to control the movement of the robot. In other words, the approach angle is not used to plan linear trajectories.

When the robot movement button 104 or the F11 key is pressed, the computer 40 proceeds to block 422 of FIG. 14. Computer 40 calls the approach angle from memory as shown in block 424. The computer 40 then calls the sagittal angle, traversal angle and three-dimensional coordinates of the virtual guide wire as shown in block 426. The computer 40 then calculates the planned position and orientation using the equation in section [12] of the appendix. This step is described at block 428. Computer 40 then reads data from a file related to the end effector of the particular surgery used during the procedure, as indicated at block 430. Such data includes three-dimensional data from a coordinate calculation device (CMM).

In block 434 the computer 40 determines whether the surgical end effector has been properly assigned. If not, the computer 40 generates an error message "No Surgical End Effect is Assigned" as indicated at block 436. Computer 40 then returns to wait for the next command as indicated at block 438.

If the surgical end effect has been properly assigned at block 434, the computer is directed through the serial communication port 50 to the robot controller 53 to move the robot to the planned position and direction as indicated at block 440. Send the command. Computer 40 determines at block 444 whether a NULL end effector has been properly assigned. If not, computer 40 generates an error message " No NULL End Effector is Assigned " as shown in block 446. Computer 40 then returns to block 448 to wait for the next command. If the NULL end effector was not properly assigned at block 444, the computer returns to wait for the next command as indicated at block 450.

If the “move robot along axis” button 82 of FIG. 3B is selected, the computer 40 proceeds to block 452 of FIG. 15. Computer 40 has already moved the robot in the proper direction during the step of FIG. Accordingly, the step of FIG. 21 is designed to move the robot along the tool guide axis defined by the tool guide 28 of FIG. 2. The tool guide axis is generally moved toward and away from the body on the table 14 along the tool guide axis. Computer 40 determines whether a screw adjustment program named "robot axis movement" in block 454 is processed. This screw adjustment program runs on its own until it is stopped. If the program has not been started at block 454, the computer 40 starts this program as shown at block 456. Computer 40 returns to block 458 to wait for further instructions. If the screw adjustment program starts at block 454, the computer 40 determines whether the page up button was pressed at block 460. If not, the computer 40 determines whether the page down button was pressed at block 462. If not, computer 40 returns to block 464 to wait for the next command.

If the page up button is pressed at block 460, computer 40 determines whether the page up button is still pressed at block 466. If not, computer 40 returns to wait for the next naming, as indicated at block 468. If the page up button is still depressed at block 466, computer 40 sends a VAL command from communication port 50 to robot controller 53, so that the robot can guide the positive tool as shown at block 470. Move on the axis. The positive tool guide axial direction is the direction of elevation away from the patient. Computer 40 returns to block 466.

If the page down button was pressed at block 462, computer 40 determines whether the page down button is still pressed at block 472. If not, the computer 40 returns to block 468 to wait for the next command. If the page down button is still pressed at block 472, the computer sends a VAL command to cause the robot to move in the negative tool guide direction as shown at block 474. The negative tool guide axial direction is a downward direction towards the patient. Computer 40 returns to block 472.

In other words, the control step of FIG. 15 allows the operator to move the robot along the tool guide axis. If the robot is moving in the positive or negative direction, the movement continues until the page up or page down is released. The entire robot moves to keep the end effector 24 and tool guide 28 in the same direction along the planned axis. In other words, the end effector 24 of the robot 18 is held in a direction of 45 ° with respect to the Z axis 38 of FIG. 1. VAL is a program control language for the PUMA-560 controller 53. It is understood that other robots, controllers and programming languages may be used in accordance with the present invention.

If the cursor is on the A / P image area 86 of FIG. 3A, the computer 40 proceeds to block 476 of FIG. 16. The computer 40 waits for the trackball to be clicked in the A / P image area 86 as shown in block 478. When the trackball is clicked at block 478, the computer 40 determines whether both the A / P image and the sagittal image are registered as described at block 480. If not registered, computer 40 does nothing and returns to block 482 to wait for the next command.

Once the A / P and sagittal images are registered at block 480, computer 40 determines whether the protruding guide wire is drawn as shown at block 484. If not, the computer 40 assumes that the operator intends to draw a protruding guidewire. Accordingly, computer 40 draws the crosshairs in trackball coordinates U and V as described at block 486. The computer 40 then draws a curve representing the aiming line on the sagittal image using the equation in section [5] of the appendix as described in block 488. Due to the distortion in the image a curve representing the aiming line is drawn. If an x-ray aiming line is taken, its image is curved due to the distortion inherent in the fluorescence image intensifier. This is because the curve must be drawn to represent the aiming line. If the aiming line indicator 70 is drawn on the sagittal area 62 of FIG. 3B, the computer 40 returns to wait for the next command as described at block 490.

If the protruding guide wire is already drawn at block 484, computer 40 determines whether the trackball coordinates are within five pixels from the upper point 88 in the A / P image. This step has been described at block 492. If the cursor coordinates are within five pixels from the upper point 88, the computer 40 clears the protruding guidewires as described in block 494 and returns to wait for the next command as described in block 496. do.

If the trackball cursor coordinates are not within five pixels from the upper point 88 at block 492, the computer 40 is within the five pixels of the lower point 90 as shown at block 498. Decide whether or not to do so. Otherwise, computer 40 returns to wait for the next command, as indicated at block 490. If the trackball cursor coordinates are within five pixels from the lower point 90 at block 498, the computer 40 determines whether the trackball is clicked again as shown at block 500. If so, the computer returns to block 490 to wait for the next command. Otherwise, computer 40 updates the transverse angle or sagittal angle as shown in block 502 based on the movement of the trackball. The traverse angle value is increased when the trackball moves up. The traverse angle value is decreased when the trackball moves downward. The sagittal angle value is increased when the trackball moves to the right. The sagittal angle value is decreased when the trackball moves left. Incrementing factor is 0.1 ° per pixel. The equations for these steps are developed in section [6] of the appendix.

After the transverse and / or sagittal angles have been updated at block 502, computer 40 and guide wire 92 protruding within A / P image area 86 using the equation in section [7] of the appendix; The guide wire 68 protruding in the sagittal image region 62 is drawn again. The computer then returns to block 500.

If the cursor is on the sagittal image of FIG. 3B, computer 40 proceeds to block 506 of FIG. 17. Computer 40 determines whether the aiming line is drawn at block 508. If not, the computer returns to block 510 to wait for the next command. If the line of aiming is drawn at block 508, the computer 40 uses the equation in section [8] of the appendix to guide wire 92 and sagittal image area 62 protruding within the A / P image area 86. Draw the guide wire 68 protruding therein. This step is described in block 512. At block 513 the computer 40 also checks if the robot has been initialized, and if so, the computer 40 at block 515 “Move Robot” and “Move Along Drill” Axis) "button and the F11 and F12 keys are enabled and lit. The computer 40 then waits for the trackball to be clicked within the sagittal image area 62 as shown at block 514. If the robot has not been initialized, the computer 40 waits for the trackball to be clicked within the sagittal image area 62 as shown in block 514. The computer 40 then determines whether the trackball cursor coordinates are located within five pixels from the upper point 64 as shown in block 516. If not, the computer 40 determines whether the trackball cursor coordinates are within five pixels from the bottom point 66 as shown in block 518. If not, computer 40 returns to block 520 to wait for the next command.

If the trackball coordinates are at five pixels of the upper point 64 at block 516, the computer 40 determines whether the trackball is clicked again at block 522. If so, return to block 524 to wait for the next command. Otherwise, the computer 40 updates the position of the virtual guide wire 68 by moving the guide wire along the aiming line in the same direction as the trackball movement. The rate of increase is 0.1 mm per pixel. This step is described at block 526. The computer uses the expression developed in section [9] of the appendix to update the virtual guidewire location. The computer 40 then uses the equation developed in section [7] of the appendix to redraw and also the A / P image of the guide wire 68 protruding in the sagittal image area 62 as shown in block 528. Guide wire 92 protruding in region 86 is redrawn. Computer 40 then returns to block 522 again.

If the trackball coordinates are located within five pixels from the bottom point 66 at block 518, then computer 40 determines whether the trackball is clicked again at block 530. If so, computer 40 returns to block 524 to wait for the next command. Otherwise, the computer 40 assumes that the operator wants to adjust the position of the lower point 66. Accordingly, the computer updates the sagittal angle and / or the transverse angle based on the movement of the trackball as indicated at block 532. The traversing angle is increased when the trackball moves up. The traverse angle is reduced when the trackball moves downward. The sagittal angle is increased when the trackball moves to the right. The sagittal angle is reduced when the trackball moves to the left. The rate of increase is 0.1 ° per pixel. The computer uses the equation developed in section [10] of the block for these steps, as shown in block 532. The computer 40 then uses the equation developed in section [7] of the appendix as shown in block 534 to redraw and A / P image the guide wire 68 protruding in the sagittal image area 62. The guide wire 92 protruding in the area 86 is redrawn. Computer 40 then returns to block 530.

Once the robot control region 84 of FIGS. 3A and 3B is selected, the computer 40 proceeds to block 536 of FIG. 18. Computer 40 then displays a menu that provides a user selection at block 538. The first selection unit is a "robot initialization" selection unit. Computer 40 determines whether the robot initialization menu item has been selected at block 540. In such a case, the computer 40 opens the serial communication port 52 for communication with the robot controller 53 as shown in block 542. Computer 40 transmits a VAL program language command required to initialize robot controller 53 as shown in block 544. Computer 40 determines whether the VAL has been properly initialized at block 546. If the VAL was not properly initialized, computer 40 sends a message that the VAL was not initialized, as indicated at block 548. Computer 40 returns at block 550.

If the VAL is properly initialized at block 546, computer 40 transmits a preset HOME position and START position to robot controller 53, as indicated at block 552. The HOME position and the START position are two positions located within the working space of the robot. In addition, the computer 40 initializes a preset NULL end effector and a SURGICAL end effector as shown in block 554. In other words, computer 40 transmits a description of the exact arrangement of the particular surgical command to be used. The controller 53 is thus programmed to move the robot to these positions. During surgery, computer 40 may instruct controller 53 to move to a particular HOME or START position. In addition, the controller 53 recognizes a command for a particular surgical end effector, which is initialized in step 554. The robot speed is then set at a very slow speed, as shown at block 556. For example, the robot speed is set to 5 out of 256 speeds. The computer 40 then checks whether a virtual guide wire was planned, and if so, the computer moves "robot movement" 104 and "robot movement along the drill axis" 82, as indicated at block 557.5. Enable and light the button and the F11 and F12 keys. The computer 40 then returns to wait for the next command as indicated at block 559.

If the virtual guidewire was not planned, the computer 40 returns to wait for the next command as indicated at block 558.

If a selection named “Move to a predefined location” has been selected from the pop-up menu 538 and the robot has already been initialized as shown at block 560, then computer 40 may advance in advance as shown at block 562. Displays a dialog box with a selection to move the robot to the defined position. In other words, a dialog box with a selection for moving the robot to the HOME position or the START position is displayed. The operator can select one of these selections at block 562. The computer 40 sends a VAL command to the controller 53 to move the robot 18 to a specific location as shown in block 564. Computer 40 then returns to block 568 to wait for the next command.

If computer 40 has determined whether a selection of "assigned and predefined tools" is selected from the pop-up menu 538 and the robot has already been initialized as shown in block 570, then computer 40 may select block ( A dialog box with a selection unit for assigning a predefined tool set during the initialization step 554 is displayed. This step is described at block 574. In other words, computer 40 displays a dialog box for assigning a NULL end effector or a SURGICAL end effector at block 574. Computer 40 then returns to block 578 to wait for the next command. If an assigned and predefined end-effector item has not been selected or the robot has not been initialized at block 570, computer 40 returns to block 572 to wait for the next command.

Although the invention has been described in detail with reference to certain preferred embodiments, variations and modifications are possible within the spirit and scope of the invention as defined and described in the appended claims.

Appendix

[1] J. Canny; "A Computational Approach to Edge Detection"; IEEE Transactions on Pattern Analysis Machine Intelligence; Vol 8, Nov. 1986, pp. 679-698.

[2] Mathematics is used to implement the Levenberg-Marquardt optimization method.

Where x _i = [x, y, z] is the three-dimensional coordinates of the reference point, (u, v) is the two-dimensional coordinates with respect to the center of the reference point, and a = [ø, θ, ψ, t _x , t _y , t _z ] are six parameters that define six degrees of free attitude.

[3] Once the pits are formed, a homogeneous transformation matrix corresponding to the optimized parameters a = [ø, θ, ψ, t _x , t _y , t _z ] is constructed as follows.

[4] Once pitting is performed, a homogeneous transformation matrix corresponding to the optimized parameters a = [[o], [theta], [phi], t _x , t _y , t _z ] is constructed as follows.

[5] The line of sight is calculated in the following manner. The viewing line is defined by (0, 0, 0) and (u _c , v _c , f) in CCS.

Note: (u _c , v _c ) is the measured equation of (u, v). See [13]

Because of the interference distortion in the fluoroscopy, the viewing line is drawn as a curved image. This is done by drawing multiple lines through them without measuring 50 points on the line defined by (u ₁ , v ₁ ) and (u ₂ , v ₂ ) as in [15].

[6] Reproduce that the virtual guidewire is a three-dimensional object defined by (0 _wt , 0 _wt , 0 _wt ) and (0 _wb , 0 _wb , thread length _wb ). β = β + 0.1 * (# pixels moved by trackball).

[7] For (Vx _wt , Vy _wt , Vz _wt ) and (Vx _wb , Vy _wb , Vz _wb ), the projection of the virtual guidewire is drawn with A / P and sagittal image using the following equation.

Because of the distortion in the fluoroscopy, the projected guidewires are drawn in a curve. It does not measure 20 points on the line defined by (u _at , v _at ) and (u _ab , v _ab ), as in [15], but instead of (u _st , v _st ) and (u _sb , v _sb ). Using to draw multiple lines on the A / P image and sagittal image through them.

[8] To draw a virtual guidewire projection, two points (0, 0, 0) and (0, 0, thread length) in the WCS are placed on the line of sight with the top point (0, 0, 0). Is converted to The viewing guidewire is initially set at 30 mm. The projected guidewire is drawn using the following method.

Early;

Depth = 0.2

Screw length = 30mm

α = 0, β = 0

(tx, ty, tz) is constrained to lie on the line of sight defined by (LSx _w1 , LSy _w1 , LSz _w1 ) and (LSx _w2 , LSy _w2 , LSz _w2 ). therefore,

T consists of the following transformations:

T = Trans (tx, ty, tz) Rot (y, α) Rot (x, β) or

To draw the projected guides of the image, the points (Vx _wt , Vy _wt , Vz _wt ) and (Vx _wb , Vy _wb , Vz _wb ) are used together.

[9] Reproduce that the virtual guidewire is a three-dimensional object defined by (0 _wt , 0 _wt , 0 _wt ) and (0 _wb , 0 _wb , thread length _wb ).

Depth = Depth + 0.1 * (# pixel moved by trackball)

[10] Reproduce that the virtual guidewire is a three-dimensional object defined by (0 _wt , 0 _wt , 0 _wt ) and (0 _wb , 0 _wb , thread length _wb ).

α = α + 0.1 * (# pixels limited by trackball)

[11] Reproduce that the virtual guidewire is a three-dimensional object defined by (0 _wt , 0 _wt , 0 _wt ) and (0 _wb , 0 _wb , thread length _wb ).

[12]

Given the following matrix, i.e.

Is determined, the remaining two vectors that are complete [FP] are determined.

Since the PUMA 560 robot uses the Euler equation to specify the direction, the inverse of [FP] is determined in the following way.

Euler equation = Rot (z, ø) Rot (y, θ) Rot (z, ψ). Therefore, from [i]

Is obtained.

By adding the PUMA specific offset to? and?, the final position and direction are set.

Final posture = (ø + 90, θ-90, ψ, tx, ty, tz).

The measured coordinates (x, y) of the edge pixels (u, v) are determined using the fourth order polynomial as follows.

The set of parameters a and b is previously determined using a measurement program.

[14] The center of the reference point shadow is found by applying a cyclic equation to the edge pixels using a pseudo inversion method.

or

A = BP

Using pseudo inversion

Once P is set, the center of the reference points h and k is determined as follows.

[15] The unmeasured (distorted) coordinates (u, v) correspond to the measured coordinates (x, y) and are determined using a fourth order polynomial as follows.

The set of parameters a and b is predetermined using a separate measurement program.

Claims (32)

In the method of designing stereotactic surgery using a fluoroscopic to generate an image of the human body, Positioning a registration artificial structure having a plurality of reference points adjacent to the human body at a known position relative to the known coordinate frame of the artificial structure; Displaying on the computer monitor an image taken from the patient's human body and the registered artificial structure; Receiving an input identifying two-dimensional coordinates for a reference point of the registered artifact displayed on the image; Registering the image by forming a geometric model with parameters, the model projecting the three-dimensional coordinates within the image point and geometrically such that the projection of the known three-dimensional coordinates of the reference points coincides with the same two-dimensional coordinates in the image. And numerically optimizing the parameters of the model. The method of claim 1, Displaying a second image taken from the patient's human body and a registered artificial structure and having a different angle from that of the first image; Receiving an input for identifying two-dimensional coordinates for a reference point of a registered artificial structure displayed on the second image; Forming a geometric model with parameters to register the second image, the model projecting three-dimensional coordinates within an image pointer, wherein the projection of known three-dimensional coordinates of reference points is the same in the second image. And numerically optimizing the parameters of the geometric model to match two-dimensional coordinates. 3. The method of claim 2, further comprising receiving user input to select a point on the first image, the point partially representing a virtual guidewire. 4. The method of claim 3, further comprising receiving an input specifying a position, a length, and an angle of the virtual guidewire. 5. The method of claim 4, further comprising drawing the projected guidewire segment such that the projected guidewire is a virtual guidewire projected on the image. 6. The method according to claim 5, wherein the guide wires projected on one image are corrected by drawing two guide wires projected on each image according to the corrected virtual guide wires. Receiving user input to move either end. 6. The method of claim 5, further comprising calibrating user input that changes the length of the virtual guidewire, and redrawing the two guidewires projected on each image in accordance with the calibrated virtual guidewire. Characterized in that the method. 6. The method of claim 5, receiving user input to change the sagittal closure angle of the virtual guidewire, updating the orientation of the virtual guidewire based on the new sagittal closure angle, and according to the calibrated virtual guidewire. And redrawing the two guidewires projected on each image. 6. The method of claim 5, wherein the device receives a user input for adjusting a crossing angle of the virtual guidewire, updates the direction of the virtual guidewire based on the new crossing angle, and wherein the respective virtual guidewire is in accordance with the calibrated virtual guidewire. And redrawing the two guidewires projected on the image. 6. The method of claim 5, wherein a user input for adjusting the head angle of the virtual guide wire is received, the direction of the guide wire is updated based on the new head angle, and the respective images according to the calibrated virtual guide wire. And redrawing the two guidewires projected onto it. 6. The method of claim 5, further comprising generating an output that adjusts the coordinates of the tool guide such that the coordinate axis is aligned with the virtual guidewire. 12. The method of claim 11, further comprising generating an output that adjusts the coordinates of the tool guide such that the position of the guide along the coordinate axis is offset by a distance from one end point of the virtual guidewire. 12. The method of claim 11, further comprising transmitting the coordinates to an automatic machine. 12. The method of claim 11, further comprising displaying the coordinates so that an operator can manually adjust the mechanism. 12. The method of claim 11, wherein the registration artifact comprises a tool guide. 3. The method of claim 2, further comprising receiving an input for selecting a point on the first image, wherein the point partially represents a target point for the surgical instrument. 17. The method of claim 16, further comprising drawing a target point projected on the first image and another target point projected on the second image such that the projected target point is a virtual target point projected on the image. How to feature. 18. The target projected on one image according to claim 17, wherein the projected two target points are corrected and the targets projected on any one image are redrawn by projecting the two target points projected in each image according to the corrected virtual target points. Receiving user input to move the point. 19. The method of claim 18, further comprising generating an output that adjusts the coordinates of the tool guide such that the coordinate axis intersects the virtual target point. 20. The method of claim 19, further comprising generating an output that adjusts the coordinates of the tool guide such that the position of the guide along the coordinate axis is offset by the predetermined distance from the virtual target point. 20. The method of claim 19, further comprising transmitting the coordinates to an automatic device. 20. The method of claim 19, further comprising displaying the coordinates so that an operator can manually adjust the mechanism. 20. The method of claim 19, wherein the registration artifact comprises a tool guide. The method of claim 1, further comprising receiving an input for selecting a point on the first image, the point partially representing a virtual guidewire representing the trajectory of the surgical instrument into the human body. Way. 25. The method of claim 24, further comprising generating an output that adjusts the coordinates of the tool guide such that the coordinate axis is aligned with the virtual guidewire. 27. The method of claim 25, further comprising transmitting the coordinates to an automatic mechanism. 27. The method of claim 25, further comprising displaying the coordinates by which an operator can manually adjust the mechanism. 27. The method of claim 25, wherein the registration artifact comprises a tool guide. In the device for designing stereotactic surgery using a fluoroscopic to generate an image of the human body, Means for positioning a registration artificial structure having a plurality of reference points adjacent to the human body; Means for displaying an image taken from the human body and the reference point; Means for identifying two-dimensional coordinates of the reference point in an image; Means for registering an image for the reference point artificial structure; And means for receiving an input for selecting and adjusting the virtual guidewire or target point while the guidewire or target point is superimposed on the image for display. In a device for designing stereotactic surgery to insert a surgical instrument in a linear orbit using a fluoroscopic to generate an image of the human body, A registration artificial structure located adjacent to the human body and having a plurality of reference points located in known three-dimensional coordinates associated with the known coordinate frame; Means for displaying at least one image taken from the human body and the artificial structure on at least one computer monitor; Means for identifying two-dimensional coordinates for the reference point in each image; Means for numerically optimizing a parameter of a geometrical model, said model projecting three-dimensional coordinates into an image point such that the known three-dimensional coordinates of a reference point coincide with the same two-dimensional coordinates in said image. . 31. The apparatus of claim 30, further comprising: means for receiving user input for selecting the position, length, and angle of the virtual guidewire, and means for displaying the guidewire segment projected on each registered image representing the position of the virtual guidewire; Apparatus further comprising a. 31. The apparatus of claim 30, further comprising a tool guide and means for generating an output for adjusting the coordinates of the tool guide.