[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2023096679A1 - Commande d'espace d'image pour outils endovasculaires - Google Patents

Commande d'espace d'image pour outils endovasculaires Download PDF

Info

Publication number
WO2023096679A1
WO2023096679A1 PCT/US2022/040118 US2022040118W WO2023096679A1 WO 2023096679 A1 WO2023096679 A1 WO 2023096679A1 US 2022040118 W US2022040118 W US 2022040118W WO 2023096679 A1 WO2023096679 A1 WO 2023096679A1
Authority
WO
WIPO (PCT)
Prior art keywords
medical
medical instrument
catheter
medical image
plane
Prior art date
Application number
PCT/US2022/040118
Other languages
English (en)
Inventor
David James BELL
Jake Anthony SGANGA
Gregory Kahn
Original Assignee
Remedy Robotics, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/810,102 external-priority patent/US11690683B2/en
Application filed by Remedy Robotics, Inc. filed Critical Remedy Robotics, Inc.
Priority to CA3239010A priority Critical patent/CA3239010A1/fr
Priority to EP22765314.4A priority patent/EP4437557A1/fr
Priority to AU2022395958A priority patent/AU2022395958A1/en
Publication of WO2023096679A1 publication Critical patent/WO2023096679A1/fr

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/37—Leader-follower robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/06—Devices, other than using radiation, for detecting or locating foreign bodies ; Determining position of diagnostic devices within or on the body of the patient
    • A61B5/061—Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body
    • A61B5/062—Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body using magnetic field
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B17/00234—Surgical instruments, devices or methods for minimally invasive surgery
    • A61B2017/00292—Surgical instruments, devices or methods for minimally invasive surgery mounted on or guided by flexible, e.g. catheter-like, means
    • A61B2017/003—Steerable
    • A61B2017/00318—Steering mechanisms
    • A61B2017/00323—Cables or rods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/301—Surgical robots for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/376—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
    • A61B2090/3966—Radiopaque markers visible in an X-ray image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/32—Surgical robots operating autonomously

Definitions

  • the present application is directed to control systems for endovascular and other intraluminal tools or medical instruments, such as catheters.
  • the devices, systems, and methods described herein can be included in or used in conjunction with robotic medical systems configured to facilitate control and operation of the medical instrument.
  • Endovascular medical procedures are common.
  • a tool or medical instrument that is generally configured as a long, thin, flexible body is inserted into and navigated through a lumen or other cavity of the body.
  • the tools or medical instruments are articulable or controllable, for example, using one or more pull wires, to allow an operator to navigate the tool or medical instrument within the body. Such navigation is often accomplished through deflection (for example, bending) of the distal tip of the tool or medical instrument.
  • Some tools or medical instruments are configured for manual control, for example, using knobs or levers mounted on a proximally located handle of the tool or medical instrument.
  • the tools or medical instruments can be configured for robotic control, for example, control by a robotic medical system.
  • an operator can use the robotic medical system (for example, a controller, user interface, and/or the like) to robotically control the tool or medical instrument.
  • control inputs are provided with respect to a plane of a two-dimensional medical image such as an X-ray.
  • control inputs can be provided to adjust a heading of an instrument within the plane of the medical image, adjust an incline of the instrument into or out of the plane of the medical image, and/or to insert or retract the medical instrument.
  • Providing a control scheme in which control inputs are provided by a user with respect to the plane of the medical image can advantageously facilitate intuitive and natural control of the instrument.
  • image space control because control inputs are provided with respect to the plane of a two-dimensional medical image.
  • Articulating the instrument either to adjust the heading of the instrument within the two-dimensional plane of the medical image or to adjust the incline of the instrument into or out of the plane of the medical image, typically requires an accurate understanding of the current roll angle of the instrument about its longitudinal axis.
  • the control scheme described herein allows for determination or estimation of the current roll angle of the instrument based on an appearance of one or more radio-opaque fiducials that are included on the medical instrument, and which are visible within the medical image.
  • computer vision can be used to analyze a two-dimensional medical image to determine the position and/or orientation (including, in some examples, current roll) of a catheter based on radio-opaque markers that are included on the catheter.
  • five degrees of freedom for the catheter can be determined: two positional degrees of freedom (e.g., x and y position) and three degrees of freedom relating to orientation (e.g., heading, incline, and roll).
  • Various example configurations for the radioopaque markers are disclosed. The use of other configurations for the radio-opaque markers is also possible, and this disclosure should not be limited to only the disclosed configurations.
  • the devices, systems, and methods described herein can provide several notable advantages over existing technologies.
  • position and orientation can be determined using minimally sophisticated medical imaging (e.g., single plane X-ray). Suitable medical imaging devices are widely available, allowing the devices, systems, and methods described herein to be widely available. Additionally, position and orientation can be determined without the additional hardware that is often required by other systems. For example, existing systems often determine position and orientation using electromagnetic sensors and electromagnetic field generators. Such systems are cumbersome and relatively inaccurate, requiring precise registrations with other forms of data (e.g., medical imaging data, computer models, robotic movement data, etc.) in order to be useful. With the principals described herein, the need for such systems can be avoided. Finally, the devices, systems, and methods of the present application can allow for safer and more precise control of a catheter.
  • minimally sophisticated medical imaging e.g., single plane X-ray
  • Suitable medical imaging devices are widely available, allowing the devices, systems, and methods described herein to be widely available.
  • position and orientation can be determined without the additional hardware that is often required by other systems. For example, existing systems often determine position
  • control schemes described herein can facilitate natural and intuitive control of an instrument, allowing the user to provide inputs with respect to the plane of the medical image that the user is currently viewing, and in some cases, without requiring the user to actively consider the current roll of the instrument.
  • a computer-implemented system comprises at least one processor and at least one electronic storage medium, the electronic storage medium storing instructions configured to cause the at least one processor to: receive, from a medical imaging device, a two-dimensional medical image including a view of at least a distal portion of a medical instrument, the distal portion of the medical instrument including one or more fiducials positioned thereon, the one or more fiducials being radio-opaque and visible in the medical image; detect, within the medical image, a two-dimensional appearance of the one or more fiducials; and based on the two-dimensional appearance of the one or more fiducials, determine at least one of: a roll angle of the distal portion of the medical instrument, and an incline of the distal portion of the medical instrument.
  • the system may include one or more of the following features in any combination: (a) wherein the at least one processor is configured to detect the two- dimensional appearance of the one or more fiducials based on a computer vision algorithm; (b) the at least one processor is configured to detect the two-dimensional appearance of the one or more fiducials using a neural network; (c) wherein the at least one processor is further configured to determine both of the roll angle of the distal portion of the medical instrument and the incline of the distal portion of the medical instrument; (d) wherein the at least one processor is further configured to determine the incline with respect to an image plane of the two-dimensional medical image; (e) wherein the one or more fiducials are configured such that the two-dimensional appearance of the fiducials within the medical image is visually distinguishable for different roll angles and different inclines of the medical instrument; (f) wherein the one or more fiducials are configured such that the two-dimensional appearance of the fiducials within the medical image is visually distinguishable for different roll
  • a method includes: receiving, from a medical imaging device, a two-dimensional medical image including a view of at least a distal portion of a medical instrument, the distal portion of the medical instrument including one or more fiducials positioned thereon, the one or more fiducials being radio-opaque and visible in the medical image; detecting, within the medical image, a two-dimensional appearance of the one or more fiducials; and based on the two-dimensional appearance of the one or more fiducials, determining at least one of: a roll angle of the distal portion of the medical instrument, and an incline of the distal portion of the medical instrument.
  • the method may include one or more of the following features in any combination: (a) wherein detecting the two-dimensional appearance of the one or more fiducials is based on a computer vision algorithm; (b) detecting the two-dimensional appearance of the one or more fiducials using a neural network; (c) determining both of the roll angle of the distal portion of the medical instrument and the incline of the distal portion of the medical instrument; (d) determining the incline with respect to an image plane of the two-dimensional medical image; (r) wherein the one or more fiducials are configured such that the two-dimensional appearance of the fiducials within the medical image is visually distinguishable for different roll angles and different inclines of the medical instrument; (f) wherein the one or more fiducials are configured such that the two-dimensional appearance of the fiducials within the medical image is visually distinguishable for different roll angles and different inclines of the medical instrument for incremental changes of less than 5 degrees, less than 10 degrees, less than 15 degrees, less than 20 degrees,
  • a computer-implemented system includes at least one processor and at least one electronic storage medium storing instructions configured to cause the at least one processor to: display, on a graphical user interface, a two-dimensional medical image including a view of at least a distal end of a medical instrument, the distal end including one or more fiducials positioned thereon and that are visible in the medical image; determine, based on the one or more fiducials in the medical image, a roll estimate of a current roll angle of the medical instrument; receive a user input from a user input device, the user input comprising at least one of: a heading command to change a heading of the medical instrument within a plane of the medical image, or an incline command to change an incline of the medical instrument into or out of the plane of the medical image; based on the roll estimate and the user input, generate one or more motor commands configured to cause a robotic system coupled to the medical instrument to move the robotic medical instrument according to the user input; and cause the robotic
  • the system may include one or more of the following features in any combination: (a) wherein the one or more motor commands comprise pullwire commands configured to actuate one or more pullwires of the medical instrument; (b) wherein the roll estimate is determined based on a two-dimensional appearance of the one or more fiducials in the medical image; (c) wherein the at least one processor is configured to determine the roll estimate based on a computer vision analysis of the one or more fiducials in the medical image; (d) wherein the processor is further configured to display, on the graphical user interface, a plurality of sample images, each sample image comprising a shape corresponding to a sample two-dimensional projection of the one or more fiducials onto a plane at a different roll angle, and receive a sample selection on the user input device, wherein the sample selection comprises an indication of a sample image that most closely corresponds to the two-dimensional appearance of the one or more fiducials in the medical image, wherein the roll estimate is determined based on the sample selection; (e)
  • a method which includes: displaying, on a graphical user interface, a two-dimensional medical image including a view of at least a distal end of a medical instrument, the distal end including one or more fiducials positioned thereon and that are visible in the medical image; determining, based on the one or more fiducials in the medical image, a roll estimate of a current roll angle of the medical instrument; receiving a user input from a user input device, the user input comprising at least one of: a heading command to change a heading of the medical instrument within a plane of the medical image, or an incline command to change an incline of the medical instrument into or out of the plane of the medical image; based on the roll estimate and the user input, generating one or more motor commands configured to cause a robotic system coupled to the medical instrument to move the robotic medical instrument according to the user input; and cause the robotic medical system to move the robotic medical system based on the one or more motor commands.
  • the method may include one or more of the following features in any combination: (a) wherein the one or more motor commands comprise pullwire commands configured to actuate one or more pullwires of the medical instrument; (b) wherein the roll estimate is determined based on a two-dimensional appearance of the one or more fiducials in the medical image; (c) wherein determining the roll estimate is based on a computer vision analysis of the one or more fiducials in the medical image; (d) displaying, on the graphical user interface, a plurality of sample images, each sample image comprising a shape corresponding to a sample two-dimensional projection of the one or more fiducials onto a plane at a different roll angle, and receiving a sample selection on the user input device, wherein the sample selection comprises an indication of a sample image that most closely corresponds to the two-dimensional appearance of the one or more fiducials in the medical image, wherein the roll estimate is determined based on the sample selection; (e) wherein the heading command to change the heading of the medical instrument
  • FIGs. 1A-1D illustrate an example coordinate system for three- dimensional image pose estimation.
  • FIG. 2A illustrates a side view of a distal end of an embodiment of an endovascular catheter.
  • FIG. 2B is a fluoroscopic image illustrating an endovascular catheter, such as that shown in FIG. 2A, navigating through a vascular network of a patient, according to an embodiment.
  • FIGs. 3A-3F illustrate an embodiment of a marker on a distal end of a catheter at different orientations, according to an example.
  • FIG. 4 illustrates an example model of a vascular network of a patient.
  • FIG. 5A is an example fluoroscopic image of a catheter navigating through the vasculature.
  • FIG. 5B is another example fluoroscopic image of a catheter navigating through the vasculature.
  • FIGs. 6A-6F illustrate determination of out-of-plane angle based on detection of a shape of the marker within a medical image, according to some examples.
  • FIGs. 7A-7C illustrate an example of a semicircular marker for a catheter configured to allow for, among other things, determination of the sign of the incline of the catheter.
  • FIG. 7D illustrates an additional example of a catheter including different marker types that can be configured to allow for, among other things, determination of the sign of the incline of the catheter.
  • FIGs. 8A-8B illustrate example fluoroscopic images of a catheter including an example of a non-circumferential ring configured to allow for determination of, among other things, both the sign and magnitude of the incline of the catheter.
  • FIG. 8C illustrates example two-dimensional appearances of radio-opaque markers at various heading and roll positions.
  • FIG. 9A illustrates examples of projections of three-dimensional generated catheters onto real world two-dimensional X-ray images, according to some examples.
  • FIGs. 9B-9C illustrate an example prediction of a trained deep neural network for predicting a position of a body of a catheter body, according to an example.
  • FIG. 9D illustrates an example determination of a centerline of a catheter.
  • FIG. 9E illustrates an example determination of a position and heading of a catheter.
  • FIG. 10 is a block diagram of an embodiment of a computer system configured to implement features described herein.
  • FIGs. 11A and 11B are side and top views of an embodiment of a catheter that includes markers configured to allow for determination of, among other things, a catheter roll angle.
  • FIGs. 12A-12D show the catheter and markers at various roll positions.
  • FIGs. 13A and 13B are perspective and side views of another embodiment of a catheter that includes markers configured to allow for determination of a catheter roll angle.
  • FIGs. 14A and 14B are perspective and top views of another embodiment of a catheter that includes markers configured to allow for determination of a catheter roll angle.
  • FIGs. 15A-15H illustrate determination of roll angle based on detection of a phase of a sinusoid of a helical fiducial within a medical image, according to some examples.
  • FIG. 16A is a perspective view of another embodiment of a catheter that includes a radio-opaque one and one quarter roll helix fiducial configured to allow for determination of a catheter roll angle.
  • FIGs. 16B-16E show fluoroscopic images of a catheter for determination of a catheter roll angle, according to some examples.
  • FIG. 16F illustrates the two-dimensional appearance of a radio-opaque marker at different roll positions.
  • FIGs. 17A-17B illustrate an embodiment of a catheter that includes radio opaque braids can be used to determine roll, according to some examples.
  • FIGs. 18A-18B illustrate another embodiment of a catheter where radio opaque markers or fiducials can be used to detect planarity.
  • FIG. 19 illustrates another embodiment of a catheter that includes markers can be used to determine whether the catheter is angled into or out of the image plane, according to some examples.
  • FIGs. 20A-20D illustrate an embodiment of a graphical user interface for providing image space control of a medical instrument.
  • FIGs. 21A-21B illustrate two example embodiments for roll estimate determination.
  • FIGs. 22A-22B illustrate an embodiment of a user input device for providing image space control of a medical instrument.
  • FIG. 23 illustrates another embodiment of a user input device for providing image space control of a medical instrument.
  • FIG. 24 is a flowchart depicting an example user space control method.
  • This application describes devices, systems, and methods for detecting or determining the position and/or orientation of endovascular or other intraluminal tools or medical instruments, such as catheters.
  • the term “pose” is used herein to refer to the position and orientation of a catheter.
  • determination of pose can be made based on a two-dimensional medical image, such as a single plane X-ray image, and one or more radio-opaque markers included on a catheter.
  • Computer vision models can be employed to detect the radio-opaque markers in the two-dimensional medical image and to determine the pose of the catheter therefrom.
  • the pose can be defined by five degrees of freedom for the catheter.
  • the five degrees of freedom can include two positional degrees of freedom (e.g., x and y position) and three degrees of freedom relating to orientation (e.g., heading, incline, and roll).
  • the pose can comprise greater (e.g., six) or fewer (e.g., four or fewer) degrees of freedom.
  • the pose of an instrument can be defined in many different ways. While this application primarily describes examples of pose in terms of x, y, and z for position, and heading, incline, and roll for orientation, other methods for describing or defining the pose (e.g., alternative coordinate systems, alternative naming conventions, etc.) are possible, and the principles of this application extend to all methods for defining pose. Further, in some embodiments, the methods and systems of this application may be used to determine one, more than one, or all elements of pose.
  • This application also describes devices, systems, and methods for controlling endovascular or other intraluminal tools or medical instruments, such as catheters, wherein control inputs are provided with respect to a plane of a two-dimensional medical image.
  • control inputs are provided with respect to a plane of a two-dimensional medical image.
  • a user can provide control inputs to change a heading of an instrument within the plane of the medical image and/or to change an incline of the instrument into or out of the plane of the medical image.
  • a computer system can determine appropriate motor commands to cause the desired movement/articulation of the instrument based on the control inputs and an estimate of a current roll of the instrument.
  • the estimate of the current roll of the instrument can be determined based on an appearance of one or more radio-opaque markers or fiducials that are included on a distal end of the image and which are visible within the medical image. In some instances, the estimate of current roll is determined by the system based on a computer vision analysis of the medical image. In some instances, the estimate of the current roll is determined and input by the user based on a user- identified appearance of the fiducials.
  • This type of control system is referred to herein as “image space control” because control inputs are provided with respect to the plane of the image (e.g., adjust heading within the plane or adjust inclination into or out of the plane).
  • This type of control system is intuitive as the user may provide such inputs while viewing the medical image. That is, the user can provide control inputs relative to the current appearance of the instrument within a medical image and without, in some embodiments, needed to specifically understand which pull wires of the instrument need to be actuated to achieve a desired motion.
  • the principals described herein can be applicable to robotic medical procedures, for example, where the catheter is robotically controlled by a robotic medical system that is configured to insert, retract, roll, and/or articulate the catheter based on inputs received from a physician or in an autonomous or semi- autonomous manner. In some instances the principals of this disclosure may also be applicable to manually controlled catheters.
  • the medical instrument or tool is an endovascular catheter configured to navigate within the vasculature of the patient. These examples, however, should not be construed as limiting of the principles of the disclosure.
  • the devices, systems, and methods for detecting or determining position and/or orientation described herein have application in other contexts.
  • the principles described herein can be useful with other endoluminal, endoscopic, or laparoscopic tools, instruments, procedures and/or the like.
  • a primary example related to an endovascular catheter is provided.
  • an endovascular catheter or catheter may also be applied to other endoluminal, endoscopic, and/or laparoscopic tools or the like. Additionally, it should be realized that while this application provides several example configurations for tools or medical instruments that include specific configurations of radio-opaque markers, other configurations of radio-opaque markers can also be used.
  • FIG. 2B provides an example X-ray image of a catheter 100 navigating through an aortic arch of a patient. From FIG. 2B alone, however, a person would have difficulty understanding the exact pose of the catheter 100. For example, does the catheter 100 lie completely within the plane of the image, or is it inclined into or out of the plane of the image? Highly skilled physicians may be able to make educated guesses with respect to these questions based on their understanding of human anatomy. Still, however, uncertainty exists, which increases the risk of damage to the patient during a medical procedure.
  • catheters include pullwires that can be actuated (e.g., tensioned or pulled) to cause deflection of a distal tip of the catheter. See, for example, the catheter 100 of FIG. 2A described below.
  • catheters include four pullwires, each configured to cause deflection of the catheter in one of four cardinal directions.
  • one pullwire can be associated with deflecting the tip of the catheter up, one pullwire can be associated with deflecting the tip down, one pullwire can be associated with deflecting the tip right, and one pullwire can be associated with deflecting the tip down.
  • one pullwire to actuate to cause a given deflection requires an understanding of the current roll position of the catheter. For example, if the distal tip of the catheter is rolled by 90 degrees, actuating the pullwire generally associated with an upward deflection of the tip would instead cause the catheter to articulate (possibly unexpectedly) to the right or left. Unintended articulation can frustrate navigation and cause injury to a patient.
  • the distal tip of a catheter often rolls (in unexpected ways) as the catheter is navigated through complex anatomy, such as through a vascular network of a patient, even if roll inputs are not provided at the proximal (e.g., external) end of the catheter.
  • the systems, methods, and devices provide for accurate determination of the pose of a catheter (including its roll) based on detection of radio-opaque fiducials included on the catheter.
  • detection of the radio-opaque fiducials is achieved using computer vision analysis of a two- dimensional medical image of the catheter.
  • the methods and systems described herein can also be used with biplane imaging systems to determine six degree of freedom pose estimates of the catheter. In such cases, determination of incline may (in some instances) be determined from the biplane images, while roll angle can be determined based on computer vision analysis of the radio-opaque fiducials included on the catheter.
  • FIGs. 1A-1D illustrate an example coordinate system for three- dimensional image pose estimation or determination.
  • the term “pose” is used herein to refer to the different combinations of positions and/or orientations of an endovascular tool, intraluminal tool, medical instrument and/or the like.
  • Position can refer to, for example, an x, y, and z position (for example, a location within three-dimensional space).
  • position can refer to the x and y position within the two-dimensional image plane of a medical image, such as an X-ray.
  • Orientation can be represented in many different ways.
  • orientation is referred to using three Euler angles: (1) heading, which can be a measure of angulation or articulation about the z-axis and/or where the device is pointing in the image plane; (2) incline, which can be a measure of rotation about the x-axis and/or where the device is pointing out of the image plane (e.g., pitch); and (3) roll which can be a measure of rotation about the longitudinal or central axis of the catheter.
  • heading which can be a measure of angulation or articulation about the z-axis and/or where the device is pointing in the image plane
  • incline which can be a measure of rotation about the x-axis and/or where the device is pointing out of the image plane (e.g., pitch)
  • roll which can be a measure of rotation about the longitudinal or central axis of the catheter.
  • FIGs. 1A-1D provide additional illustrations.
  • FIG. 1A represents two- dimensional position within the x-y plane of the medical image.
  • FIG. IB illustrates the heading angle, measured about the z-axis, which generally corresponds to the apparent heading or direction of the catheter within the x-y plane of the image.
  • FIG. 1C illustrates the incline angle, measured about the x-axis, which generally corresponds to a measurement of the angle into or out of the plane of the image.
  • FIG. ID illustrates the roll of the catheter, measured about the catheter’s longitudinal or central axis.
  • the methods and systems described herein are also applicable to determination of one, more than one, or all elements of pose, regardless of the manner in which pose is defined (e.g., regardless of coordinate system, nomenclature, etc.).
  • the use of computer vision algorithms and unique radio-opaque markings or fiducials can be included on the catheter that may be used to quantitatively estimate the endovascular tool’s five-dimensional pose (for example, two-dimensional position (e.g., see FIG. 1A), as well as its heading (e.g., FIG. IB), incline (e.g., FIG. 1C), and roll (e.g., FIG. ID).
  • five-dimensional pose for example, two-dimensional position (e.g., see FIG. 1A), as well as its heading (e.g., FIG. IB), incline (e.g., FIG. 1C), and roll (e.g., FIG. ID).
  • radio-opaque markings can preserve the readily visible dimensions, for example, the position of the endovascular tool on the image plane (the x and y position) and the direction the endovascular tool points in the image plane (the heading), and further, the use of unique radio-opaque markings can further allow the sign of the incline dimension, the degree of incline dimension, and the roll to be quantitatively estimated or determined. While many different radio-opaque materials may be used, in some embodiments, it may be preferable to use platinum, tungsten, and/or gold because of their superior X-ray attenuation properties. In some embodiments, the radioopaque material may be a piece of wire, metal, radio-opaque ink and/or the like that is place within a layer of the catheter or other tool. Examples are provided below.
  • FIG. 2A illustrates a distal end of an embodiment of an endovascular catheter 100.
  • the catheter 100 includes a long, thin, and flexible body 101 that extends to a distal end 103.
  • the body 101 can be configured to be navigated through the patient’s vasculature.
  • a channel may be formed through the body 101 such that other tools or instruments can be passed through the catheter 100 and gain access to the patient’s anatomy through an opening that can be included on the distal end 103 of the catheter.
  • one or more tools can be integrated directly into the catheter 100 itself.
  • the catheter 100 or a distal portion thereof can be configured to be articulable or deflectable.
  • the catheter 100 includes one or more pull wires 105 that extend on, in, or through the body 101 and attach at or near the distal end 103. Actuation (e.g., pulling or shortening) the pull wires 105 at or near their proximal end can be configured to cause a distal portion of the catheter 100 to deflect or articulate.
  • the catheter 100 can be configured for oneway, two-way, three-way, or four-way deflection, although other configurations providing different degrees of articulation or deflection are also possible.
  • the catheter 100 can be configured such that control thereof (e.g., control of the deflection of the distal portion of the catheter 100) can be accomplished manually or robotically.
  • control thereof e.g., control of the deflection of the distal portion of the catheter 100
  • the body 101 and pull wires 105 extend proximally to a handle located on a proximal end of the catheter 100 (not shown).
  • the handle can be configured to by operated by hand (e.g., manually) to actuate the pull wires 105.
  • the handle can include one or more manual inputs such as levers, buttons, dials and/or the like that allow a user to manually actuate the pull wires 105 to cause deflection of the distal portion of the catheter 100.
  • the body 101 and pull wires 105 extend proximally to a base located on a proximal end of the catheter 100 (not shown).
  • the base can be configured to connect to and be operated by a robotic medical system to actuate the pull wires 105.
  • the base can include one or more robotic inputs configured to engage with robotic outputs or actuators on the robotic medical system. In some embodiments, other methods and configurations for manual and robotic control may be used.
  • FIG. 2A also illustrates that, in some embodiments, the catheter 100 can include a marker (or fiducial) 107.
  • the marker 107 is positioned at the distal end 103 of the catheter 100.
  • the marker 107 can be configured to facilitate identification of the distal end 103 of the catheter 100 in a medical image captured during a medical procedure.
  • the marker 107 can be radio-opaque so as to be readily identifiable within a fluoroscopic image.
  • the marker 107 comprises a radio-opaque ring positioned on the distal end 103 of the catheter 100.
  • the marker 107 comprises a radio-opaque material
  • it can be more easily identified within, for example, a fluoroscopic image such as an X-ray. Identification of the distal end 103 within a medical image during a procedure can greatly facilitate navigation and control of the catheter 100.
  • FIG. 2B provides an example medical image of a catheter 100, such as the catheter 100 of FIG. 2A, navigating within the vasculature of the patient.
  • the image of FIG. 2B is a fluoroscopic X-ray image.
  • both blood vessels 109 of the patient’s vasculature, as well as the catheter 100 itself, are visible in the image.
  • the marker 107 included at the distal end 103 of the catheter 100 helps a user (or a computer-vision algorithm) viewing the image to identify the distal end 103 of the catheter 100. Because the marker 107 comprises a radio-opaque material, it shows up well within the image.
  • a physician or other operator attempts to guide the distal end 103 of the catheter 100 to a specific location, such as, for example a treatment site.
  • a specific location such as, for example a treatment site.
  • a mechanical thrombectomy is a mechanical thrombectomy.
  • a large vessel occlusion (LVO) stroke occurs when a blood clot lodges in at least one of the internal carotid, proximal middle cerebral artery, proximal anterior cerebral artery, basilar artery, or vertebral artery.
  • LVO large vessel occlusion
  • Such a clot can partially or completely occlude downstream blood supply to brain tissue resulting in neuronal infarction and subsequent neurological impairment or death.
  • a physician gains access to the patient’s vasculature and inserts a catheter, such as catheter 100.
  • the catheter 100 is guided to the obstruction using, for example, one or more medical images similar to the one shown in FIG. 2B.
  • tools are passed through the working channel of the catheter 100 to remove the obstruction.
  • the catheter 100 may include an articulable portion that is actuated via pull wires.
  • the catheter 100 may move unexpectedly to the left if the catheter 100 is rolled 180 degrees such that the right most pull wire 105 is positioned on the left side of the catheter 100.
  • the physician may estimate the roll position of the catheter 100 based on the roll position of the proximal end of the catheter 100.
  • the roll position at the proximal end of the catheter 100 may not (and often does not) directly correspond to the roll position at the distal end 103 of the catheter 100.
  • the roll position at the distal portion of the catheter 100 may not (and often does not) directly correspond to the roll position at the distal end 103 of the catheter 100.
  • a catheter 100 can include a marker 107 (or a plurality of markers) positioned on the distal end 103 thereof.
  • the marker 107 comprises a ring positioned on the distal end 103.
  • the marker 107 can be radio-opaque such that it can be easily identifiable within a medical image, such as, for example, the medical image shown in FIG. 2B.
  • the shape of the marker(s) 107 within the two- dimensional image can be analyzed to determine the out of the plane angle or incline of the catheter 100.
  • the two-dimensional projection of the three-dimensional shape of the marker 107 can be analyzed to determine the out of the plane angle of the catheter 100.
  • the term “incline,” as referred to herein, can refer to the degree of angulation of the catheter towards or away from the X-ray source (e.g., into or out of the imaging plane), for example, as shown in FIG. 1C.
  • a positive degree of angulation indicates that the catheter is angled towards the X-ray emitter, with a maximum incline of +90-degrees.
  • a negative degree of angulation indicates that the catheter is angled away from the X-ray emitter, with a maximum incline of -90-degrees.
  • the ring-shaped marker 107 presents or appears as a rectangle having a straight edge at the distal most tip within the two-dimensional plane of FIG. 2A. With this orientation, it can be determined that the distal portion of the catheter 100 lies in the plane of the FIG. 2A (e.g., in the illustrated orientation, the catheter 100 is not curving into or out of the plane of FIG. 2A).
  • FIGs. 3A-3F illustrate the marker 107 in different orientations.
  • the distal most end of the marker 107 is illustrated with a double line.
  • the marker 107 comprises a ring shape that can be positioned on the end of a catheter, such as the catheter 100 of FIG. 2A.
  • the marker 107 is generally pointed up and to the left.
  • the distal end of the marker 107 presents as an ellipse within the plane of FIG. 3A. Because the entire ellipse is visible, it can be determined that the marker 107 is pointed out of the plane of the page.
  • FIG. 3F In contrast, consider FIG. 3F. In FIG. 3F, within the plane of the page, one can see that the marker 107 is pointed straight up. Considering the distal edge of the marker 107, one can only see half of the ellipse (the other half being blocked from view). From this, it can be determined the marker 107 of FIG. 3F is angled into the page.
  • FIG. 3B presents a similar example to FIG. 3F, except, as shown in FIG. 3B, the entire ellipse is visible. From this it can be determined that the marker is angled out of the plane of FIG. 3B.
  • FIG. 3C presents an example where the marker 107 is pointed up and to the right within the plane of FIG. 3C and angled slightly out of the page based on the shape of the ellipse.
  • FIG. 3D illustrates another example where the marker 107 is angled out of the plane of the page. Comparing FIGs. 3B and 3D, one can see that the length of the minor axis of the ellipse is shorter in FIG. 3D as compared to FIG. 3B. This indicates that the degree of the angle out of the page in FIG. 3D is less than the degree of the angle out of the page of FIG. 3B.
  • FIG. 3E illustrates an example similar to FIG. 2A, in which the marker is pointed upwards, within the plane of the page. At this orientation, the minor axis has decreased to zero, indicating that the marker 107 is pointed upwards within the plane of the page.
  • a computer system such as or which can include a computer vision system, can be used to (1) detect the shape (e.g., the visible, ellipse, circle, curve, or line) created by the distal end of the ring-shaped marker 107 within the two- dimensional image, and (2) extract or determine the out of the plane angle of the marker 107 or the tool incline from the detected shape.
  • the computer system may utilize artificial intelligence or machine learning to perform such functionality.
  • a neural network can be trained to detect the shape created by the distal end of the ring-shaped marker 107 within the two-dimensional image, and extract or determine the out of the plane angle of the marker 107 from the detected shape.
  • computer vision can be used to define the orientation of the tool along the z- axis.
  • the out of the plane angle of the catheter 100 can be important in improving and/or facilitating navigation through a luminal network, such as the vasculature.
  • the vasculature of a patient will not lie within a single plane. This is apparent considering the model example vasculature provided in FIG. 4.
  • the arch of the aorta does move slightly from left to right, but it moves or arches predominately from front to back.
  • a tool navigating through the arch of the aorta will be moving significantly out of the plane of the imaging device.
  • the angle of the catheter 100 out of the page can be determined as described above.
  • FIGs. 5 A and 5B provide additional example medical images of a catheter, such as the catheter 100 of FIG. 2A, navigating through vasculature. These figures illustrate how the catheter 100 moves at an angle into and out of the plane of the medical image based on the portion of the anatomy through which the catheter 100 passes.
  • FIG. 5A which illustrates the catheter 100 navigating through a portion of the carotid artery
  • the distal end 103 of the catheter 100 lies generally within the plane of the medical image, and thus the distal end 103 of the marker 107 does not form an ellipse.
  • FIG. 5B illustrates the catheter 100 navigating through a portion of the aortic arch. As shown, the distal end 103 of the marker 107 does forms an ellipse which indicates that the catheter 100 is moving with at an angle that is moving out of the plane of FIG. 5B.
  • FIGs. 6A-6F provide additional examples of detecting the shape (e.g., the visible, ellipse, circle, curve, or line) created by the distal end of the ring-shaped marker 107 within the two-dimensional image and extracting or determining the out of the plane angle of the marker 107 from the detected shape.
  • FIGs. 6A-6C illustrate three example images
  • FIGs. 6D-6F illustrate the same images with the shape of the ring-shaped marker 107 that has been detected and highlighted.
  • other methods or mechanisms may also be employed for determining the out of the plane angle of the catheter 100.
  • a degree of angulation may be presumed for each vessel through which the catheter 100 passes, for example, based on the general or average angulation of that vessel across the population.
  • the angulation of a vessel can be determined based on a CT scan.
  • an additional medical image at a different orientation than the first medical image e.g., a lateral angiographic view
  • the medical imager can be moved so as to gain an understanding of the out of the plane angulation.
  • the magnitude of the incline of endovascular and other intraluminal tools can, in some embodiments, be determined by analysis of the elliptical shape of the tool mouth (or a marker, such a ring-shaped marker, included thereon).
  • the sign of the degree of angulation is not always readily identifiable solely from analysis of the marker 107 because the elliptical shape may look the same when imaged in two dimensions for both positive angulation and negative angulations of the same degree of incline.
  • an additional method may be desired.
  • FIG. 7A illustrates one method for determining the sign of the incline of catheter 100
  • catheter 100 can include a semicircular marker 110 positioned on the distal end 103 thereof.
  • the semicircular marker 110 can be radio-opaque such that it can easily be identifiable within a medical image.
  • the semicircular marker 110 extends around one half of the distal end 103 of the catheter 100. In other embodiments, other portions can be used (e.g., 14, etc.).
  • additional fiducials such as helical fiducial 111 may also be used.
  • the position of the marker 110 within the two-dimensional image can be analyzed to determine the sign of the incline of the catheter 100.
  • the sign of the incline of catheter 100 can be determined by whether the semicircular marker 110 appears to be above or below the distal end 103 in the two-dimensional image.
  • FIGS. 7B-7C illustrate the appearance of the marker 110 on a catheter tip on an X-ray at different inclines.
  • the catheter itself is not shown in these figures.
  • the semicircular marker 110 in combination with helical fiducial 111 will produce a different appearance in the two-dimensional image when inclined at the same angle, but at different signed inclines (e.g., whether into or out of the plane of the image).
  • the magnitude of the incline could be determined to be 45-degrees using a method such as the ring-shaped marker method described above.
  • the semicircular marker 110 in combination with helical fiducial 111 could then be used to determine the signed incline.
  • the catheter tip is inclined at negative 45-degrees
  • FIG. 7C the catheter tip is inclined at positive 45-degrees.
  • a computer system such as a computer vision system, can be used to (1) detect the position of the semicircular marker 110 in combination with helical fiducial 111 within the two-dimensional image, and (2) extract or determine the sign of the tool incline from the detected position.
  • the computer system may utilize artificial intelligence or machine learning to perform such functionality.
  • a neural network can be trained to detect the position of the semicircular marker 110 in combination with helical fiducial 111 within the two- dimensional image, and extract or determine the sign of the tool incline from the detected position.
  • the machine learning algorithm does not hardcode the aforementioned approach. Instead, the machine learning algorithm trains a deep neural network to directly predict the incline angle from the input of the X-ray image.
  • FIG. 7D illustrates an additional example of a catheter 100 including a helical marker 111 and additional markers 114.
  • the additional markers 114 include a semi-circular portion that extends partially around the radius of the catheter 100 and a tail portion that extends along longitudinally along the catheter.
  • FIGs 8A-8B illustrate provide an additional example of determination of the sign and magnitude of incline of catheter 100, where the catheter 100 can include one or more non-circumferential rings 113.
  • the non-circumferential rings are semi-circular, extending part way around the catheter 100.
  • the non-circumferential rings 113 can be radio-opaque such that it can easily be identifiable within a medical image. The appearance of the non-circumferential rings 113 can be analyzed to determine the sign and magnitude of the incline of the catheter 100.
  • the sign and magnitude of the incline of catheter 100 can be determined by the unique appearance of the non-circumferential rings 113 in the two-dimensional image at varying degrees of incline, both positive and negative.
  • non-circumferential rings 113 are arranged in an asymmetrical design. That is, in some embodiments, the non-circumferential rings 113 are each positioned at a different rotational position around the catheter 100. In the illustrated embodiments, the non-circumferential rings are positioned at 90-degree offsets. In some embodiments, non-circumferential rings 113 are multiple ellipses offset from each other.
  • FIG. 8C illustrates how an example arrangement of non-circumferential rings 113 positioned on a distal end of a catheter may provide a unique appearance at different inclination and roll angles. Images are provided at positive, neutral (i.e., zero), and negative inclinations, as well as at different roll positions provided in 30-degree increments. As shown, each of the 36 different illustrated positions provides a unique appearance. By detected, for example, using computer vision, this appearance within a medical image, the incline (including its sign) and the roll of the catheter can be determined. While FIG. 8C illustrates how the radio-opaque markers provide different two-dimensional appearances for different roll positions at 30-degree increments and for different positive, neutral (zero), and negative inclines, the illustrated increments are not intended to be limiting.
  • the radio-opaque markers provide unique or visually distinguishable two-dimensional appearances at all different roll or incline positions. In some embodiments, the radio-opaque markers provide unique or visually distinguishable two-dimensional appearances at different roll or incline positions within increments of about, at least, or at most 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 7.5 degrees, 10 degrees, 12.5 degrees, 15 degrees, 17.5 degrees, 20 degrees, 25 degrees, 30 degrees, or 40 degrees.
  • the radio-opaque markers are configured with a three-dimensional shape that, when viewed within the two-dimensional plane of a two- dimensional medical imaging device, provides a unique or visually distinguishable appearance that can be distinguished at the different incremental roll or incline angles listed above.
  • the above listed increments can be considered minimum resolutions for the system or the minimum change in roll or incline that is detectable by the system.
  • FIG. 9A illustrates examples of projections of three-dimensional generated catheters onto real world two-dimensional X-ray images.
  • FIGs. 9B-9C illustrate an example prediction of the trained deep neural network for predicting where the catheter body is.
  • position can refer to translation of endovascular and/or other intraluminal tools along the x, y, and z directions.
  • only the two-dimensional (x and y) position is used instead of determining the three-dimensional position of the instrument (z, y, and z). This can be because estimating the z position (i.e., depth) may require calibrating the X-ray camera to obtain determine intrinsic parameters thereof. While this is feasible, it would likely (1) add a burdensome pre-operative image calibration step, (2) may not help that much because motion planning is primarily done in two-dimensions, and (3) we the C-arm can be rotated to obtain a Z position estimate.
  • the system may be configured to predict where the full tool body is, and then from this tool body we the two-dimensional tip location can be extracted. This approach may be beneficial because the tool body provides a very strong training signal for learning deep neural network segmentation models. That is, in some instances, it may be easier for a neural network of computer vision algorithm to detect the body of a catheter and then extract the location of the tip from there. In some embodiments, catheter kinematics are further used refine this estimate.
  • a deep neural network can be used to estimate the two- dimensional centerline position of the catheter based on one or more images of the catheter navigating within the body.
  • FIG. 9D illustrates an example, in which the neural network has identified the catheter within the image, superimposed its estimated centerline onto the image, and highlighted the catheter. Once the centerline of the catheter has been identified within the image, the two-dimensional position can be directly extracted by computing the most distal position along the centerline. Similarly, the heading of the catheter can also be directly extracted from the body estimate by computing the vector of the tip of the body line.
  • FIG. 9E illustrates an example in which the distal tip position and heading angle have been determined and the image has been updated to include a highlight identifying the position and an arrow indicating the heading.
  • a machine learning algorithm for estimating the position of a catheter and/or other tool may use the following approach.
  • the image generation procedure is modified by drawing the catheter on top of tangible X-ray images (e.g., as shown in FIG. 9B). This process may have the advantage of training the deep neural network with realistic noise and occlusions that would be seen in actual X-rays, making the system robust to real world conditions.
  • the two-dimensional x and y position is estimated.
  • radio-opaque markings may be added to the tool body, such as, for example, a full-length helix, to assist with the identification.
  • the three- dimensional x, y, and z position may be estimated instead.
  • estimation of the Z position may require calibration of the X-ray camera to obtain its camera intrinsic, which requires an additional step of a pre-operation image calibration.
  • the two- dimensional position estimation will be preferable so the pre-operation image calibration step does not need to be completed and because the z position may not be necessary because motion planning is primarily conducted in two-dimensions.
  • a z position estimate can be obtained by rotating the C-arm.
  • the two-dimensional x and y position of the full tool body may be predicted.
  • the two-dimensional x and y position location of the tool tip (such as, for example, the distal end 103 of catheter 100) can be determined.
  • catheter kinematics may be used to further refine the position estimate.
  • heading can refer to a measure of angulation or articulation about the z-axis and/or where the device is pointing in the image plane.
  • the deep neural network prediction of the catheter body position may be used. Based on the prediction of the two-dimensional x and y position of the catheter tip, a second position located on the catheter body may be determined. The second position may be an infinitesimal distance from the tool tip in a direction along the catheter body. The heading angle of the catheter may then be calculated using trigonometry based on the x and y position of the tool tip and the second position along the catheter body.
  • radio-opaque markers can be placed at the distal tip 103 of a catheter 100 to improve the visibility of the catheter 100 in a medical image (see FIG. 2 A and 2B). In some embodiments, these markers are symmetric about the tools axis and thus do not provide any information related to the roll angle of the catheter. As described in this section, the degree of rotation of a tool about its centerline may be informed by the addition of radio-opaque rotation fiducials. Numerous configurations of radio-opaque rotation fiducials can be utilized to determine the degree of tool rotation, provided the configurations result in a unique X-ray appearance of the tool at differing degrees of rotation.
  • markers 108 that are rotationally asymmetric can be utilized on the catheter 100 and configured to provide information from which the roll angle of the catheter 100 can be determined.
  • a computer vision algorithm can be used to detect and analyze the positions of the markers 108 to determine the rotation or roll of the catheter 100 about its longitudinal axis. As described above, the rotation of the catheter 100 determines how an actuation of a given pull wire will move the catheter 100. Thus, an accurate understanding of the roll angle of the catheter 100 can be important for successful navigation of the catheter 100 through the vasculature.
  • FIGs. 11A and 11B illustrate side and top views respectively of a distal portion of a catheter 100 that has been configured with markers 108 from which the roll angle of the catheter 100 can be determined.
  • the catheter 100 includes markers 1O8A-1O8D configured as radio-opaque fiducials (e.g., dots) placed at different distances from the distal tip 103, at differing angles/arcs from the center of the tool.
  • radio-opaque fiducials e.g., dots
  • the markers may be other radioopaque fiducials, such as, for example squares, circles, diamonds, and/or the like.
  • a first marker 108A is positioned a first distance 2D (twice the diameter of the tool) from the distal tip 103 and at a 45-degree rotational offset with respect to a first pull wire 105.
  • a second marker 108B is positioned an additional distance 2D from the distal tip 103 (4D) and at a 135-degree rotational offset with respect to the first pull wire 105.
  • a third marker 108C is positioned an additional distance 2D from the distal tip 103 (6D) and at a 225-degree rotational offset with respect to the first pull wire 105.
  • a fourth marker 108D is positioned an additional distance 2D from the distal tip 103 (8D) and at a 315-degree rotational offset with respect to the first pull wire 105.
  • the rotational offset of first marker 108A from the first pull wire may be, for example, 0-degrees, 5-degrees, 10-degrees, 15-degrees, 20-degrees, 25-degrees, 30-degrees, 35-degrees, 40-degrees, 45-degrees, 50-degrees, 55-degrees, 60-degrees, 65-degrees, 70- degrees, 75-degrees, 80-degrees, 85-degrees, 90-degrees, and/or the like, with the second marker 108B, third marker 108C, and fourth marker 108D being an additional 90-degrees, 180 degrees, and 270-degrees respectively rotationally offset from first marker 108 A.
  • the distance 2D is approximately 5mm.
  • each marker 108 is positioned at a longitudinal distance that is twice the diameter D of the catheter 100 below the marker 108 above it (or below the distal tip 103 for the first marker). This need not be the case in all embodiments, and other spacings are possible.
  • each marker 108 is positioned at 90-degree offsets and between the adjacent pull wires 105. Again, this need not be the case in all embodiments and other spacings are possible.
  • the catheter 100 is illustrated as transparent, allowing all four markers 108 to be visible. It should be appreciated, however, that in the illustrated example, markers 108A and 108D are on the back side of the catheter 100 (the side facing away relative to the orientation of the image) and thus would not be visible if the catheter were not illustrated as transparent.
  • FIGs. 12A-12D provide various examples of the catheter 100 at different orientations. In each of FIGs. 12A-12D, both a model of the catheter and a corresponding image of the catheter are shown. FIGs. 12A-12D illustrate that for each roll position, the appearance of the markers 1O8A-1O8D is different, thus allowing for roll determination.
  • determining roll from the markers 1O8A-1O8D can be accomplished as follows: (1) the locations of the markers 1O8A-1O8D within an image can be determined, in some embodiments, this is accomplished through computer vision or a neural network that has been trained to identify the markers 1O8A-1O8D; (2) the centerline of the catheter 100 can be determined using, for example, computer vision or a neural network; (3) the distance (with appropriate sign, positive or negative) between each of the markers 1O8A-1O8D and the centerline can be determined; (4) the signed distance between the markers and the centerline can be used to determine the roll angle using geometry principles.
  • markers 1O8A-1O8D can be advantageous for various reasons. For example, because each marker 1O8A-1O8D is placed at a different distance from the distal tip 103, they can each be uniquely identified by that distance. This can allow the machine learning model and/or loss to be much more robust and/or generalizable. Use of four markers 1O8A-1O8D may also prevent against or eliminate angle aliasing, for example, allowing for full determination of roll between 0 and 360 degrees.
  • markers 1O8A-1O8D are equidistantly spaced along the longitudinal axis, the markers 1O8A-1O8D can be analyzed to determine out-of-of plane rotations. Further, use of four markers 1O8A-1O8D can allow for determination of the centerline of the catheter 100. Use of markers 1O8A-1O8D and four pull wires 105 can also facilitate control and verification.
  • increasing the axial distance between the markers 1O8A-1O8D can increase the signal to noise ratio, for example, allowing the markers 108A- 108D to be more easily identified within the medical image.
  • more than four markers may be used to determine the roll of the catheter 100. In some embodiments, less than four markers may be used to determine the roll of the catheter 100.
  • FIGs. 13A and 13B illustrate another embodiment of markers 115 that can be used to determine roll angle of a catheter 100.
  • FIG. 13A is a perspective view of a distal end of a catheter 100 including markers thereon.
  • FIG. 13B is a side view of the markers 115.
  • multiple ring-shaped markers 115 are included.
  • Each ring-shaped marker 115 can include a through hole formed therethrough. When the ring 115 is rotated such that the holes align, there will be a visible gap in the ring. When the ring 115 is rotated such that the holes do not align, the ring will appear solid.
  • This distinction may be detectable through a centroid detection algorithm or a computer vision algorithm trained for this type of device.
  • the hole need not be circular, the hole may be, for example, any polygonal shape.
  • multiple rings 115 may be stacked at set rotations, such that the rotation of the tool can be determined by identifying which of the rings 115 presents a visible gap.
  • radio-opaque sleeves or other features can be coupled to the pull wires.
  • a computer vision algorithm can be configured to either detect the features at opposite sides of the catheter or to detect when the features overlap. In either case, these features would allow the computer vision system to assess if the tool is oriented with each pull wire in plane.
  • FIGs. 14A and 14B illustrate another embodiment of catheter 100 that includes the use of two markers 117 and 119 of different lengths oriented at 90 degrees with respect to each other. Because the markers 117 and 119 are of different length, the relative position of the markers 117 to marker 119 can be determined, allowing roll angle to be reliably computed using only two markers.
  • FIGs. 15A-15H illustrate another embodiment of a configuration that may be used to determine the roll of the catheter 100.
  • a radio-opaque helix fiducial 121 is coupled to catheter 100.
  • the radio-opaque helix fiducial 121 is approximately sinusoidal such that the phase of the sinusoid can be used to determine the degree of roll.
  • the phase of the sinusoid may be determined by the location of the radio-opaque helix fiducial 121 closest to the distal end of the catheter 100 and by the location of the radio-opaque helix fiducial 121 furthest from the distal end of the catheter 100, near the base of the cylindrical band.
  • FIGs. 15A-15H illustrate X-ray images of catheter 100 at different degrees of roll and different degrees of incline.
  • FIG. 16A illustrates another embodiment of a configuration that may be used to determine the roll of the catheter 100.
  • a radio-opaque one and one quarter roll helix fiducial 123 is coupled to catheter 100, where the helix is made slightly longer than one complete revolution, such that the roll helix fiducial 123 is approximately 1.25 times the articulation length.
  • the degree of roll of catheter 100 can be determined because the roll helix 123 takes on a different appearance depending on the degree of roll.
  • the roll helix fiducial can comprise other lengths greater than or less than one and one quarters.
  • the roll helix 123 takes on a distinct appearance.
  • the articulation length is approximately two centimeters.
  • the embodiments illustrated in FIGs. 16A-16E also include examples of the non-circumferential markers discussed above.
  • FIG. 16F illustrates the two-dimensional appearance (e.g., as within the plane of medial image) of the helical fiducial 123 of FIG. 16A and different roll positions in 30-degree increments. As shown, each roll position provides a unique appearance which can be used to determine roll, for example, by a computer vision, neural network, or machine learning system. While FIG. 16F illustrates how the radio-opaque markers provide different two-dimensional appearances for different roll positions at 30-degree increments, the illustrated increments are not intended to be limiting.
  • the radio-opaque markers provide unique or visually distinguishable two-dimensional appearances at all different roll positions. In some embodiments, the radio-opaque markers provide unique or visually distinguishable two- dimensional appearances at different roll positions within increments of about, at least, or at most 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 7.5 degrees, 10 degrees, 12.5 degrees, 15 degrees, 17.5 degrees, 20 degrees, 25 degrees, 30 degrees, or 40 degrees. The above listed increments can be considered minimum resolutions for the system or the minimum change in roll that is detectable by the system.
  • the roll angle determined based on the markers of any of these embodiments can be used by a motion planning algorithm to determine how it will move the catheter.
  • the algorithm can be configured to rotate the catheter until the radio-opaque identifiers aligns with the imaging plane.
  • the algorithm can measure the rotation of the tool using the radio-opaque identifiers and update which pull wires it uses to execute a maneuver.
  • FIGs. 17A and 17B illustrate an embodiment where radio opaque braids 125 can be used to determine roll.
  • the way in which the radio opaque braids 125 are wound around the catheter 100 may be used to detect out-of-plane deflection.
  • FIGs. 18A and 18B illustrate that radio opaque markers or fiducials 127 can be used to detect planarity.
  • the distance observed in a single plane can be used to provide information regarding a tool’s degree of out-of-plane deflection.
  • the shorter distance between the two fiducials 127 indicates bending out of plane, whereas the longer distance indicates the tool is more planar.
  • FIG. 19 by adding markers on opposite sides of the catheter that can be distinguished from one another, one can determine whether the catheter is angled into or out of the image plane.
  • a double dot 131 is shown on one side and a single dot 133 on the other side.
  • the first image is a side view
  • the second shows the device head on (from x-ray view).
  • the double dot 131 above the single dot 133.
  • the single dot 133 is above the double dot 131.
  • One may need to know the roll of the device to use this information for example, to know which one is in front and which one is behind).
  • markers With at least two pairs of front/back markers 90 degrees apart and the roll is known, one could determine this deflection for any roll value.
  • alternative markers could be used, such as shapes, for example circles, squares, donuts, lines, and or the like.
  • markers such as a solid ring and/or a partial ring(s) and/or the like may be used.
  • Unexpected motion of the distal end of a catheter can jeopardize the safety of endovascular or other procedures.
  • the term “unexpected motion,” as referred to herein, describes any movement or behavior of the distal end of a catheter that is not predicted based on the movement and/or control of the proximal end of the catheter.
  • An example of unexpected motion may be a poor torque response where an unexpected roll motion occurs at the distal end of a catheter.
  • the unexpected roll motion may occur when the rotation at the distal end of the catheter does not correlate to the rotation at the proximal end. For example, sometimes when the proximal end of the catheter is rotating, the distal end of the catheter may be rotating at a slower rate or may not be rotating at all.
  • the proximal end of the catheter continues to rotate and the distal end rotates at a slower rate or does not rotate at all, torque builds up in the catheter. If the proximal end of the catheter continues to rotate, eventually the torque in the catheter may cause the distal end of the catheter to rotate very quickly to catch up with the rotation imposed on the proximal end of the catheter and the quick rotation may be characterized as unexpected motion which could cause harm to the patient. For example, the unexpected motion at the distal end could cause damage to or a tear in a vessel wall.
  • the degree of rotation of the distal end of a tool about its centerline may be informed by the addition of radio-opaque rotation fiducials.
  • unexpected motion at the distal end of a catheter may be prevented by tracking and comparing the rotation rate and roll of the distal and proximal ends of a catheter.
  • the comparison may be performed at discrete steps. In some embodiments, the comparison may be performed continuously.
  • rotation at the proximal end of a catheter may be prevented when there is a difference of more than a specific number of degrees of rotation between the proximal and distal ends of the catheter. By preventing further rotation after the difference in rotation is calculated at a specific amount, the system may prevent unexpected motion (for example, snapping and/or whipping of the distal end) to increase the safety of the procedure.
  • a computer vision system may be used to identify the fiducials to model how far the distal catheter tip has rotated in relation to how far the motors controlling rotation at the proximal end of the catheter have moved.
  • this method may be paired with other safety information such as, for example, force detection and/or the like.
  • a similar method may be applied to detect discrepancies in expected advancement and retraction of the catheter as compared to actual advancement and retraction of the catheter. Use of this method may be used to identify potential obstructions to catheter motion. For example, using the methods described herein to determine the actual position a catheter tip can be compared to the expected position of a catheter tip based on how the catheter was controlled.
  • the automated identification of a tool, tool tip and/or tool orientation may be used to control movement of a decoupled imaging source to maintain optimal viewing.
  • movement of an angiography system could be precisely controlled and/or centered on the tool in question without manual manipulation.
  • movement of a camera could be precisely controlled and/or centered on the tool in question without manual manipulation. Being able to maintain optimal viewing in this manner may have the advantage of significantly better imaging for the surgeon.
  • automated tool tracking may improve the speed and efficiency of procedures, for example, the procedure would be faster and more efficient because the surgeon does not have to put down the tools and move the camera intermittently.
  • automated tool tracking may reduce the number of personal required in the operation room because no one needs to control the camera.
  • Automated tool tracking may be most useful for interventional and/or surgical applications where the source of input imaging is decoupled from the navigating and/or interventional tool. In these procedures, generally an assistant must manually track the surgeon's tools which may result in both lag and imprecision.
  • the automated tool tracking system may be used to maintain the tool tip halfway across and one third up the image screen for the entire procedure instead of the current method of having the physician put down the tools and manually readjust the screen to focus on the tool.
  • the automated tool tracking system may be used to enable an external camera source to be automatically adjusted to maintain the tools in the center of the image, thus improving the focus and positioning of the camera.
  • the various technologies disclosed herein related to determination of position and/or orientation determination can be used to facilitate the treatment of various diseases and other conditions where a robotic or manual device is advanced through an intraluminal (e.g., intravascular) network of a subject to reach the site of intravascular pathology (e.g., thrombosis, embolus, occlusion, aneurysm, rupture, bleeding, dissection, etc.).
  • intraluminal e.g., intravascular
  • the systems, devices, and methods described herein can be used to facilitate one or more endovascular purposes, surgeries, and/or treatments.
  • the systems, processes, and methods described herein can be used for one or more of removal of intravascular blockage/reestablishment of perfusion; treatment of vessel wall injury (aneurysm and/or dissection); treatment of bleeding: aneurysm rupture/trauma; and/or the like.
  • the systems, devices, and methods described herein can be used to treat vascular trauma.
  • the systems, devices, and methods described herein can be used to facilitate neurovascular applications and/or treatments, such as for example to treat subarachnoid hemorrhage, aneurysm, arteriovenous malformation, and/or the like.
  • the systems, devices, and methods described herein can be used for cardiovascular applications and/or treatments, such as for example to treat myocardial infarction, coronary artery disease, pacemaker insertion, and/or the like.
  • the systems, devices, and methods described herein can be used for aortic applications and/or treatments, such as for example to treat aortic dissection, aortic aneurysm, and/or the like.
  • the systems, devices, and methods described herein can be used for peripheral emboli applications and/or treatments. In some embodiments, the systems, devices, and methods described herein can be used for vascular trauma applications and/or treatments. In some embodiments, the systems, devices, and methods described herein can be used for venous applications and/or treatments.
  • This section describes devices, systems, and methods for controlling medical instruments, such as catheters, wherein user inputs are provided with respect to a plane of a two-dimensional medical image.
  • a user such as a physician that is controlling a robotic medical instrument or a medical instrument that is coupled to a robotic system, can provide user inputs to change a heading of the instrument within the plane of the medical image (e.g., as shown in FIG. IB) and/or to change an incline of the instrument into or out of the plane of the medical image (e.g., as shown in FIG. 1C).
  • This type of control system can be referred to herein as “image space control” because the user inputs are provided with respect to the plane of the image (e.g., adjusting the heading within the plane or adjusting the inclination into or out of the plane).
  • This type of control system is intuitive as the user may provide such inputs while viewing the medical image which includes at least a representation of a distal portion of the instrument. That is, the user can provide control inputs relative to the current appearance of the instrument within a medical image.
  • many articulable medical instruments or catheters use one or more pull wires to control articulation (bending or deflection) of a distal tip of the instrument.
  • some catheters include four pullwires, each configured to cause deflection of the catheter in one of four directions: one pullwire can be associated with deflecting the tip of the catheter up, one pullwire can be associated with deflecting the tip down, one pullwire can be associated with deflecting the tip right, and one pullwire can be associated with deflecting the tip down.
  • knowing which pullwire to actuate to cause a given deflection requires an understanding of the current roll position of the catheter. If the distal tip of the catheter is rolled by 90 degrees, actuating the pullwire generally associated with an upward deflection of the tip would instead cause the catheter to articulate (possibly unexpectedly) to the right or left.
  • a computer system can determine appropriate motor commands to cause the desired movement/articulation of the instrument based on user provided control inputs and an estimate of a current roll of the instrument.
  • the estimate of the current roll of the instrument can be determined based on an appearance of one or more radio-opaque markers or fiducials that are included on a distal end of the image and which are visible within the medical image.
  • the estimate of current roll is determined by the system based on a computer vision analysis of the medical image.
  • the estimate of the current roll is determined and input by the user based on a user-identified appearance of the fiducials.
  • the user can provide natural and intuitive inputs with respect to the current position and orientation of the instrument within a medical image, and the system can determine appropriate motor commands (e.g., commands for actuating one or more of the pullwires of the instrument) to cause the desired motion.
  • this can allow the user to control the catheter in one or more of the following three directions: forward and back (insertion), left and right (heading), and/or into and out of the image (incline). These directions move with respect the plane of the image regardless of how the X- ray is moved or how the catheter is rolled in the body.
  • This control mode is intuitive and provides a large advancement over the current standard of care, which requires the user to frequently guess and check which way the catheter will move on screen. Using these controls, the user can easily access tricky vessels and ensure safe navigation of the instrument through the vessels in an atraumatic fashion.
  • a user may provide user inputs in various ways. For example, in some embodiments, the user can specify desired targets for insertion, heading, and/or incline. Once specified, the system can determine the appropriate motor commands for causing the instrument to move from its current position and orientation to the desired position and orientation. Providing such absolute targets (e.g., desired targets for insertion, heading, and/or incline) may advantageously provide some resiliency and safety in the event in a lag in communication between the user and the robotic medical system. This can be advantageous for situations wherein the user is remotely located from the robotic system and patient and communication occurs of a computer network, such as the internet.
  • a user may provide user inputs that are indicated relative to the current position or orientation of the instrument. For example, a user can specify that the instrument adjust its heading to the right relative to the current heading of the instrument, While such a system may be less tolerant to high latency and communication lag, it still allows user to navigate in a simple and intuitive manner.
  • the roll estimate is determined automatically the system based on the appearance of one or more fiducials on the instrument in the image.
  • the roll estimate may be determined with assistance from the user. For example, the user may select or provide a roll estimate by comparing the current appearance of the one or more fiducials in the image to one or more sample images representative of the appearance of the one or more fiducials at different roll angles.
  • Another advantage to image space control is that it continues to function even if the imaging device, for example, a C-arm of an X-ray machine, is moved. This is because control inputs are provided with respect to the plane of the image. If the imaging device is moved, the plane will move also, and control inputs will be provided with respect to the moved plane.
  • FIGs. 20A-20D illustrate an embodiment of a graphical user interface 200 for providing image space control of a medical instrument.
  • the graphical user interface is configured to display a two-dimensional medical image 202, such as an X-ray.
  • the medical image 202 includes a view of a distal end of a medical instrument, such as a catheter 204.
  • the catheter 204 includes one or more fiducials 206 positioned thereon.
  • the fiducials 206 are visible within the medical image 202.
  • the fiducials 206 can be configured as described above in order to allow for vision-based determination of the position and orientation (including roll) of the medical instrument.
  • the graphical user interface 200 may also include a user input device 208.
  • the user input device 208 is configured to receive user inputs from a user that are provided with respect to the two-dimensional medical image 202.
  • the user input device 208 includes features for allowing a user to input insert commands (e.g., to advance or retract the instrument 204), heading commands (e.g., to alter the heading of the medical instrument 204 within the plane of the medical image 202, for example, to the right or left of the instrument’s current heading), and incline commands (e.g., to alter the incline of the medical instrument 204 into or out of the plane of the medical image 202.
  • the user input device 208 may include other options as well.
  • the user input device 208 includes options to inject contrast, confirm an entered movement, and to relax the catheter.
  • the user input device 208 is illustrated as a component of the graphical user interface 200, this need not be the case in all embodiments.
  • the user input device 208 can comprise a handheld control.
  • the user input device 208 allows the user to provide user inputs for controlling the instrument 204 with respect to the current configuration of instrument as shown in the two-dimensional medical image 202.
  • the user may input a desired heading for the medical instrument via the heading input of the user input device.
  • the user can input a desired heading by selecting a target point on the wheel.
  • the desired heading is shown at about 355 degrees with a highlighted circle.
  • the current heading is also shown on the wheel at about 270 degrees as a lighter circle.
  • the user may also select a desired inclination using the incline slider, if desired.
  • the robotic system can determine appropriate motor commands to cause the instrument 204 to move to the desired heading and incline.
  • FIG. 20C shows the instrument 204 after movement.
  • FIG. 20D illustrates that, by using the insert arrows of the user input device 208 the user can command forward and backward motion of the instrument 208.
  • the graphical user interface 200 and user input device 208 of FIGs. 20A- 20D provide only one example of how these features may be configured. Additional examples are discussed below with reference to FIGs. 22A-23, and further embodiments will be apparent to those of ordinary skill in the art upon consideration of this disclosure.
  • the system determines the roll of the instrument automatically, for example, using computer vision analysis of the appearance of one or more of the fiducials in the image as discussed above.
  • the system may determine the roll of the instrument based upon a user input.
  • FIGs. 21A-21B illustrate two example embodiments for roll estimate determination.
  • the user is presented with both the medical image and a plurality of sample images.
  • Each of the sample images illustrates the appearance of a fiducial at a specified roll value (e.g., similar to FIG. 16F).
  • the user is prompted to select which images most closely corresponds to the appearance of the fiducial in the medical image.
  • the accuracy of the roll estimate can be improved.
  • the sample images are presented to the user in a series of steps (e.g., a first set of images at roll increments of 30 degrees, a second set of images at roll increments of 5 degrees, and a third set of images at roll increments of 1 degree).
  • FIG. 2 IB provides an alternative embodiment, wherein a user is asked to adjust a position of a slider (or provide another input) that causes a sample image that includes a representation of the fiducial at a given roll angle to change. The position of the slide is adjusted until the sample images corresponds to the appearance of the fiducial in the image.
  • this information can be used in conjunction with the user inputs of heading, incline, and/or insertion to provide appropriate motor commands.
  • FIGs. 22A-22B illustrate an embodiment of a user input device for providing image space control of a medical instrument.
  • the user input device is similar to that which is shown in FIGs. 20A-20D.
  • the user input device is configured to provide absolute or target-based inputs for heading and incline. That is, using the wheel for heading and the slider for incline, the user can select desired angles. By selecting confirm move, the system can determine the appropriate motor commands and cause movement of the instrument until the specified angles are reached. In some embodiments, the system only moves while the user holds down the confirm move button, although this need not be the case in all embodiments.
  • the mechanisms for providing absolute or target-based inputs for heading and incline should not be limited to only the wheel and slider shown. Other mechanisms are possible.
  • insert and retract commands can be provided using the appropriate arrows.
  • insert commands can be provided in an absolute or target-based manner. For example, a user can specify a desired insertion or retraction (e.g., in terms of millimeters, pixels, etc.) and the system can generate motor commands configured to cause movement to the target.
  • FIG. 23 illustrates another embodiment of a user input device for providing image space control of a medical instrument.
  • arrows are provided for each of insert, heading, and incline in order to provide relative control. Pressing any of the arrows can cause motion in the indicated direction (e.g., insert or retract; alter heading in the plane of the image; or alter incline into or out of the plane of direction) relative to the current position and/or heading of the instrument,
  • FIG. 24 is a flowchart depicting an example user space control method 300.
  • the method 300 begins at block 302 at which the system displays a medical image to the user.
  • the medical image can be displayed, for example, on a graphical user interface.
  • the medical image can be a two-dimensional medical image, such as an X-ray.
  • the medical image can include a view of at least a distal end of a medical instrument as well as one or more fiducials positioned on the instrument.
  • a roll estimate for the instrument is determined based on the medical image.
  • the roll estimate is determined based on a two- dimensional appearance of the one or more fiducials in the medical image.
  • the roll estimate is determined based on a computer vision analysis of the one or more fiducials in the medical image.
  • the roll estimate is determined based on a user input, for example, as described with reference to FIGs. 21A and 21B. For example, the user may select which of a plurality of sample images, each corresponding to a different roll angle, most closely corresponds to the appearance of the fiducial in the medical image.
  • a user input can be received from a user input device.
  • the user input can comprise at least one of a heading command to change a heading of the medical instrument within a plane of the medical image, or an incline command to change an incline of the medical instrument into or out of the plane of the medical image.
  • the user input comprises an insert or retract command.
  • the user inputs can be provided relative to the current position of the instrument or as absolute or desired targets.
  • the method 300 determines one or more motor commands configured to cause a robotic system coupled to the medical instrument to move the robotic medical instrument according to the user input.
  • the motor commands comprise pullwire commands configured to actuate one or more pullwires of the medical instrument.
  • the systems, processes, and methods described herein are implemented using a computing system, such as the one illustrated in FIG. 10.
  • the example computer system 1002 is in communication with one or more computing systems 1020 and/or one or more data sources 1022 via one or more networks 1018. While FIG. 10 illustrates an embodiment of a computing system 1002, it is recognized that the functionality provided for in the components and modules of computer system 1002 can be combined into fewer components and modules, or further separated into additional components and modules.
  • the computer system 1002 can comprise a pose determination module 1014 that carries out the functions, methods, acts, and/or processes described herein.
  • the module 1014 is executed on the computer system 1002 by a central processing unit 1006 discussed further below.
  • module refers to logic embodied in hardware or firmware or to a collection of software instructions, having entry and exit points. Modules are written in a program language, such as JAVA, C, C++, and/or the like. Software modules can be compiled or linked into an executable program, installed in a dynamic link library, or can be written in an interpreted language such as BASIC, PERL, LAU, PHP, or Python and/or any such languages.
  • Modules can be called from other modules or from themselves, and/or can be invoked in response to detected events or interruptions.
  • Modules implemented in hardware include connected logic units such as gates and flip-flops, and/or can include programmable units, such as programmable gate arrays and/or processors.
  • the modules described herein refer to logical modules that can be combined with other modules or divided into sub-modules despite their physical organization or storage.
  • the modules are executed by one or more computing systems and can be stored on or within any suitable computer readable medium, or implemented in-whole or in-part within special designed hardware or firmware. Not all calculations, analysis, and/or optimization require the use of computer systems, though any of the above-described methods, calculations, processes, or analyses can be facilitated through the use of computers. Further, in some embodiments, process blocks described herein can be altered, rearranged, combined, and/or omitted.
  • the computer system 1002 includes one or more processing units (CPU) 1006, which can comprise a microprocessor.
  • the computer system 1002 further includes a physical memory 1010, such as random access memory (RAM) for temporary storage of information, a read only memory (ROM) for permanent storage of information, and a mass storage device 1004, such as a backing store, hard drive, rotating magnetic disks, solid state disks (SSD), flash memory, phase-change memory (PCM), 3D XPoint memory, diskette, or optical media storage device.
  • the mass storage device can be implemented in an array of servers.
  • the components of the computer system 1002 are connected to the computer using a standards-based bus system.
  • the bus system can be implemented using various protocols, such as Peripheral Component Interconnect (PCI), Micro Channel, SCSI, Industrial Standard Architecture (ISA) and Extended ISA (EISA) architectures.
  • PCI Peripheral Component Interconnect
  • ISA Industrial Standard Architecture
  • EISA Extended ISA
  • the computer system 1002 includes one or more input/output (I/O) devices and interfaces 1012, such as a keyboard, mouse, touch pad, and printer.
  • the I/O devices and interfaces 1012 can include one or more display devices, such as a monitor, which allows the visual presentation of data to a user. More particularly, a display device provides for the presentation of GUIs as application software data, and multi-media presentations, for example.
  • the I/O devices and interfaces 1012 can also provide a communications interface to various external devices.
  • the computer system 1002 can comprise one or more multi-media devices 1008, such as speakers, video cards, graphics accelerators, and microphones, for example.
  • the computer system 1002 can run on a variety of computing devices, such as a server, a Windows server, a Structure Query Language server, a Unix Server, a personal computer, a laptop computer, and so forth. In other embodiments, the computer system 1002 can run on a cluster computer system, a mainframe computer system and/or other computing system suitable for controlling and/or communicating with large databases, performing high volume transaction processing, and generating reports from large databases.
  • the computing system 1002 is generally controlled and coordinated by an operating system software, such as z/OS, Windows, Linux, UNIX, BSD, PHP, SunOS, Solaris, MacOS, ICloud services or other compatible operating systems, including proprietary operating systems. Operating systems control and schedule computer processes for execution, perform memory management, provide file system, networking, and I/O services, and provide a user interface, such as a graphical user interface (GUI), among other things.
  • GUI graphical user interface
  • the computer system 1002 illustrated in FIG. 10 is coupled to a network 1018, such as a LAN, WAN, or the Internet via a communication link 1016 (wired, wireless, or a combination thereof).
  • Network 1018 communicates with various computing devices and/or other electronic devices.
  • Network 1018 is communicating with one or more computing systems 1020 and one or more data sources 1022.
  • the pose determination module 1014 can access or can be accessed by computing systems 1020 and/or data sources 1022 through a web-enabled user access point. Connections can be a direct physical connection, a virtual connection, and other connection type.
  • the web-enabled user access point can comprise a browser module that uses text, graphics, audio, video, and other media to present data and to allow interaction with data via the network 1018.
  • the output module can be implemented as a combination of an all-points addressable display such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, or other types and/or combinations of displays.
  • the output module can be implemented to communicate with input devices 1012 and they also include software with the appropriate interfaces which allow a user to access data through the use of stylized screen elements, such as menus, windows, dialogue boxes, tool bars, and controls (for example, radio buttons, check boxes, sliding scales, and so forth).
  • the output module can communicate with a set of input and output devices to receive signals from the user.
  • the computing system 1002 can include one or more internal and/or external data sources (for example, data sources 1022).
  • data sources 1022 can be implemented using a relational database, such as DB2, Sybase, Oracle, CodeBase, and Microsoft® SQL Server as well as other types of databases such as a flat-file database, an entity relationship database, and object-oriented database, and/or a record-based database.
  • relational database such as DB2, Sybase, Oracle, CodeBase, and Microsoft® SQL Server
  • other types of databases such as a flat-file database, an entity relationship database, and object-oriented database, and/or a record-based database.
  • the computer system 1002 can also access one or more databases 1022.
  • the databases 1022 can be stored in a database or data repository.
  • the computer system 1002 can access the one or more databases 1022 through a network 1018 or can directly access the database or data repository through I/O devices and interfaces 1012.
  • the data repository storing the one or more databases 1022 can reside within the computer system 1002.
  • a Uniform Resource Locator can include a web address and/or a reference to a web resource that is stored on a database and/or a server.
  • the URL can specify the location of the resource on a computer and/or a computer network.
  • the URL can include a mechanism to retrieve the network resource.
  • the source of the network resource can receive a URL, identify the location of the web resource, and transmit the web resource back to the requestor.
  • a URL can be converted to an IP address, and a Doman Name System (DNS) can look up the URL and its corresponding IP address.
  • DNS Doman Name System
  • URLs can be references to web pages, file transfers, emails, database accesses, and other applications.
  • the URLs can include a sequence of characters that identify a path, domain name, a file extension, a host name, a query, a fragment, scheme, a protocol identifier, a port number, a username, a password, a flag, an object, a resource name and/or the like.
  • the systems disclosed herein can generate, receive, transmit, apply, parse, serialize, render, and/or perform an action on a URL.
  • a cookie also referred to as an HTTP cookie, a web cookie, an internet cookie, and a browser cookie, can include data sent from a website and/or stored on a user’s computer. This data can be stored by a user’s web browser while the user is browsing.
  • the cookies can include useful information for websites to remember prior browsing information, such as a shopping cart on an online store, clicking of buttons, login information, and/or records of web pages or network resources visited in the past. Cookies can also include information that the user enters, such as names, addresses, passwords, credit card information, etc. Cookies can also perform computer functions. For example, authentication cookies can be used by applications (for example, a web browser) to identify whether the user is already logged in (for example, to a web site).
  • the cookie data can be encrypted to provide security for the consumer.
  • Tracking cookies can be used to compile historical browsing histories of individuals.
  • Systems disclosed herein can generate and use cookies to access data of an individual.
  • Systems can also generate and use JSON web tokens to store authenticity information, HTTP authentication as authentication protocols, IP addresses to track session or identity information, URLs, and the like.
  • actions such as “advancing a catheter or microcatheter” or “advancing one portion of the device (e.g., linearly) relative to another portion of the device to rotate the distal end of the device” include instructing advancing a catheter” or “instructing advancing one portion of the device,” respectively.
  • the ranges disclosed herein also encompass any and all overlap, sub-ranges, and combinations thereof.
  • Language such as “up to,” “at least,” “greater than,” “less than,” “between,” and the like includes the number recited. Numbers preceded by a term such as “about” or “approximately” include the recited numbers. For example, “about 10 mm” includes “10 mm.”
  • Terms or phrases preceded by a term such as “substantially” include the recited term or phrase. For example, “substantially parallel” includes “parallel.”

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Robotics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

L'invention concerne des systèmes et des procédés de commande d'espace d'image d'un instrument médical. Selon un exemple, un système est configuré pour afficher une image médicale bidimensionnelle comprenant une vue d'au moins une extrémité distale d'un instrument. Le système peut déterminer, sur la base d'un ou de plusieurs repères sur l'instrument, une estimation de roulis de l'instrument. Le système peut en outre recevoir une entrée utilisateur comportant une instruction de cap pour modifier un cap de l'instrument dans un plan de l'image médicale, ou une instruction d'inclinaison pour modifier une inclinaison de l'instrument dans le plan ou hors du plan de l'image médicale. Sur la base de l'estimation de roulis et de l'entrée utilisateur, le système peut générer une ou plusieurs instructions de moteur configurées pour amener un système robotique accouplé à l'instrument médical à déplacer l'instrument médical robotique.
PCT/US2022/040118 2021-11-24 2022-08-11 Commande d'espace d'image pour outils endovasculaires WO2023096679A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CA3239010A CA3239010A1 (fr) 2021-11-24 2022-08-11 Commande d'espace d'image pour outils endovasculaires
EP22765314.4A EP4437557A1 (fr) 2021-11-24 2022-08-11 Commande d'espace d'image pour outils endovasculaires
AU2022395958A AU2022395958A1 (en) 2021-11-24 2022-08-11 Image space control for endovascular tools

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163264531P 2021-11-24 2021-11-24
US63/264,531 2021-11-24
US17/810,102 US11690683B2 (en) 2021-07-01 2022-06-30 Vision-based position and orientation determination for endovascular tools
US17/810,102 2022-06-30

Publications (1)

Publication Number Publication Date
WO2023096679A1 true WO2023096679A1 (fr) 2023-06-01

Family

ID=83193547

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/040118 WO2023096679A1 (fr) 2021-11-24 2022-08-11 Commande d'espace d'image pour outils endovasculaires

Country Status (4)

Country Link
EP (1) EP4437557A1 (fr)
AU (1) AU2022395958A1 (fr)
CA (1) CA3239010A1 (fr)
WO (1) WO2023096679A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160228032A1 (en) * 2013-03-15 2016-08-11 Hansen Medical, Inc. Flexible instrument localization from both remote and elongation sensors
US20170273665A1 (en) * 2016-03-28 2017-09-28 Siemens Medical Solutions Usa, Inc. Pose Recovery of an Ultrasound Transducer

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160228032A1 (en) * 2013-03-15 2016-08-11 Hansen Medical, Inc. Flexible instrument localization from both remote and elongation sensors
US20170273665A1 (en) * 2016-03-28 2017-09-28 Siemens Medical Solutions Usa, Inc. Pose Recovery of an Ultrasound Transducer

Also Published As

Publication number Publication date
CA3239010A1 (fr) 2023-06-01
EP4437557A1 (fr) 2024-10-02
AU2022395958A1 (en) 2024-06-06

Similar Documents

Publication Publication Date Title
US12193764B2 (en) Systems and methods for guidance of intraluminal devices within the vasculature
US20230116327A1 (en) Robot-assisted driving systems and methods
JP5285270B2 (ja) 自動ガイドワイヤ操作システム
US20080043902A1 (en) Method of Three-Dimensional Device Localization Using Single-Plane Imaging
US11707332B2 (en) Image space control for endovascular tools
US20080097200A1 (en) Location and Display of Occluded Portions of Vessels on 3-D Angiographic Images
US11690683B2 (en) Vision-based position and orientation determination for endovascular tools
Wu et al. Comparative analysis of interactive modalities for intuitive endovascular interventions
US12121307B2 (en) Vision-based position and orientation determination for endovascular tools
US20240000517A1 (en) Image space control for endovascular tools
WO2023096679A1 (fr) Commande d'espace d'image pour outils endovasculaires
US20250058087A1 (en) Image space controlled manual catheter
US20250058086A1 (en) Image space controlled manual catheter
EP3389539B1 (fr) Système d'aide à la navigation
WO2025038851A1 (fr) Cathéter manuel contrôlé par un espace image
EP4342410A1 (fr) Re-routage dans des interventions pulmonaires
Dankelman et al. Comparative Analysis of Interactive Modalities for Intuitive Endovascular Interventions
WO2024061859A1 (fr) Réacheminement dans le cadre d'interventions associées aux poumons
WO2024233520A1 (fr) Systèmes et procédés de mise à jour peropératoire d'une interface utilisateur graphique sur la base de données d'imagerie peropératoire

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22765314

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022395958

Country of ref document: AU

Ref document number: AU2022395958

Country of ref document: AU

ENP Entry into the national phase

Ref document number: 3239010

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 2022395958

Country of ref document: AU

Date of ref document: 20220811

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2022765314

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022765314

Country of ref document: EP

Effective date: 20240624