WO2024191723A1

WO2024191723A1 - Cannula assembly having sensors for identifying tools in a surgical site

Info

Publication number: WO2024191723A1
Application number: PCT/US2024/018788
Authority: WO
Inventors: JR. Bryce C. Klontz; Joseph Peter Corrigan; Joshua John GIBSON
Original assignee: New View Surgical
Priority date: 2023-03-10
Filing date: 2024-03-07
Publication date: 2024-09-19

Abstract

A robotic surgical system for performing a robotic surgical procedure on a patient, which includes a cannula assembly including a cannula tube having a distal end portion configured for insertion into a patient. The cannula tube may also have a housing coupled to the cannula tube so as to be positioned within the patient when the distal end of the cannula tube is inserted into the patient. The housing including an image sensor configured to provide image data of the surgical site when the housing is in an open position within the patient. The system is configured to determine, based at least in part upon the image data of the surgical site, that the surgical task is safe to be performed by a robotic surgical tool in an automated mode.

Description

CANNULA ASSEMBLY HAVING SENSORS FOR IDENTIFYING TOOLS IN A

SURGICAL SITE

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application Serial No. 63/489,484, filed March 10, 2023, the complete disclosure of which is incorporated herein by reference for all purposes.

BACKGROUND

[0002] Minimally invasive surgery involves inserting surgical tools, e.g., staplers, graspers, electrosurgical devices, endoscopes, etc., into a body of a patient through small openings or incisions. In order to enable such surgical tools to be inserted through small openings, cannula assemblies may be employed to create the openings and to maintain a seal around each surgical tool while the surgical tool is being used inside the body. Cannula assemblies typically include a cannula, which is a tubular shaped device having an internal lumen extending from the proximal end to the distal end thereof. The cannula may also have one or more seals in a housing located at its proximal end. The cannula assembly may also have an obturator that is selectively insertable into the internal lumen of the cannula such that, when the obturator is fully inserted into therein, the distal end of the obturator extends beyond the distalmost end of the cannula. The distal end of the obturator typically has either a sharp tip (e.g., a pointed cutting blade) or a blunt tip. When the obturator is fully inserted into the cannula, the two components are together pressed into the abdominal wall of the patient such that the distal end of the obturator creates an incision or opening in the abdominal wall of the patient. After the obturator and cannula are together fully inserted, the obturator is removed from the internal lumen of the cannula, leaving the cannula in place within the incision created by the distal tip of the obturator. The internal lumen of the cannula may then be used for inserting the surgical tools into the surgical space within a patient.

[0003] Typically, in addition to cannula assemblies forming individual incisions for each surgical tool, an individual incision may also be made through the patient by a cannula that is thereafter dedicated to holding an illumination and/or imaging device, e.g., a traditional endoscope or laparoscope. A surgical tool combining a cannula and an imaging device in a single unit is disclosed, for example, in U.S. Patent No. 8,834,358, the disclosure of which is herein incorporated by reference in its entirety.

SUMMARY

[0004] The following presents a simplified summary of the claimed subject matter in order to provide a basic understanding of some aspects of the claimed subject matter. This summary is not an extensive overview of the claimed subject matter. It is intended to neither identify key or critical elements of the claimed subject matter nor delineate the scope of the claimed subject matter. Its sole purpose is to present some concepts of the claimed subject matter in a simplified form as a prelude to the more detailed description that is presented later.

[0005] In accordance with various embodiments thereof, systems and methods are provided for use in a surgical procedure. In an embodiment, there is provided a cannula assembly including a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient. The cannula tube may have an internal lumen extending from the proximal end portion to the distal end portion. The cannula assembly may also include a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end of the cannula tube is inserted into the patient. The housing may be movable relative to the cannula tube between a closed position and an open position. The housing may include a light source and an image sensor configured to provide image data when the housing is in the open position within the patient. The system may also include a processor configured to receive the image data from the image sensor and to process the image data to identify at least one surgical tool within the patient. In some embodiments, the housing may be rotatable about an axis transverse to the longitudinal axis of the cannula tube such that, when in the open position, the housing may be positioned further laterally relative to the longitudinal axis of the cannula tube as compared to the closed position.

[0006] According to various embodiments thereof, it is also contemplated that the processor may be configured to identify the at least one surgical tool within the patient by processing an identifying indicia of the at least one surgical tool. The identifying indicia of the at least one surgical tool may include one of a QR code and a serial number displayed on the at least one surgical tool. Additionally or alternatively, the identifying indicia on the at least one surgical tool may include one of a shape and a color of the at least one surgical tool.

[0007] Still further, the cannula assembly, in accordance with various embodiments thereof, may include an identification data storage device having stored data that correlates the identifying indicia to particular aspects, types or features of surgical tools. As such, the processor may be configured to identify a particular type of surgical tool within the patient by comparing the received image data to the stored data of the identification data storage device. In still further embodiments, the processor may be configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool.

[0008] Additionally or alternatively, in accordance with various embodiments thereof, there may be included a second processor configured to further process additional data related to the identified at least one surgical tool. For example, in some embodiments, the second processor may be configured to process data related to at least one of the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and the duration of time for which the identified surgical tool was utilized during the surgical procedure. In the same or other embodiments, the additional data that the second processor processes may be employed to track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and/or identify design improvements that can be made to the at least one surgical tool.

[0009] In still further embodiments, a surgical system may be provided for use in a surgical procedure. The surgical system may include a cannula assembly including a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient. The cannula tube may have an internal lumen extending from the proximal end portion to the distal end portion. The cannula assembly may also include an obturator configured to be removably inserted into the internal lumen so as to assist in insertion of the cannula tube into the patient. The system may also include a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end of the cannula tube is inserted into the patient. In embodiments, the housing may be movable relative to the cannula tube between a closed position and an open position. The housing may include a light source and an image sensor configured to receive image data from at least one surgical tool within the surgical site when the housing is in the open position within the patient. The system may also include a processor configured to receive the image data from the image sensor and to process the image data to identify the at least one surgical tool within the patient. Still further, the system may also include, in various embodiments, a display device configured to display images related to the image data. [0010] In some embodiments, the processor may be configured to identify the at least one surgical tool within the patient by processing an identifying indicia of the at least one surgical tool. The identifying indicia of the at least one surgical tool may include one of a QR code and a serial number displayed on the at least one surgical tool. Additional or alternatively, the identifying indicia on the at least one surgical tool may include one of a shape and a color of the at least one surgical tool.

[0011] In various embodiments, the surgical system may also include an identification data storage device having stored data that correlates the identifying indicia to particular types of surgical tools. The processor may be configured to identify a particular type of surgical tool within the patient by comparing the received image data to the stored data of the identification data storage device. The processor may be configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool.

[0012] In still other embodiments, the surgical system may also include a second processor configured to further process additional data related to the identified surgical tool. The second processor may be configured to process data related to the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and/or the duration of time for which the identified surgical tool was utilized during the surgical procedure. Still further, the additional data that the second processor processes may be employed to track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and/or identify design improvements that can be made to the at least one surgical tool.

[0013] In still further embodiments, there is provided a cannula assembly for use in a surgical procedure. The cannula assembly may include a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient. The cannula assembly may also include a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end portion of the cannula tube is inserted into the patient. The housing may include a sensor configured to provide sensor data related to a structure affixed to or built into at least one surgical tool within the patient. The cannula assembly may also include a processor configured to receive the sensor data from the sensor and to process the sensor data to identify the at least one surgical tool.

[0014] In such embodiments, the structure may include one or more of an ultra-wideband transmitter, a holographic sticker, one or more braille-like dots, a near-field communication tag, metal contacts, a magnetic strip reader, a colored sticker and a Bluetooth or wifi component. The cannula assembly may also include an identification data storage device having stored data that correlates the identifying indicia to particular types of surgical tools, and the processor may be configured to identify a particular type of surgical tool within the patient by comparing the received sensor data to the stored data of the identification data storage device.

[0015] In still further embodiments, the processor may be configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool. The cannula assembly may also include a second processor configured to further process additional data related to the identified at least one surgical tool. The second processor may be configured to process data related to at least one of the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and the duration of time for which the identified surgical tool was utilized during the surgical procedure. Additionally or alternatively, the additional data that the second processor processes may be employed to at least one of track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and identify design improvements that can be made to the at least one surgical tool.

[0016] Among various other advantages provided by certain embodiments as will be evident from the Detailed Description below, there may also be the benefit that fewer punctures through a patient, e.g., through an abdominal wall or other bodily surface, are made during a surgical procedure. As set forth above, a surgeon typically performs a laparoscopic procedure using multiple cannulas inserted through individual incisions, wherein at least one such cannula and incision is occupied by an illumination/imaging device, such as a traditional endoscope and/or laparoscope. According to various embodiments thereof, there may be provided a cannula assembly and/or system therefor that eliminates the need for this separate puncture by a cannula assembly for a traditional endoscope/laparoscope, since it provides, in certain embodiments, a cannula assembly which provides both an illumination/imaging device (e.g., mounted or coupled to the cannula tube) and an internal lumen through which a separate surgical tool (e.g., a surgical stapler, etc.) may be inserted. The reduction of at least one puncture during a surgical procedure, as may be enabled in certain embodiments, may improve the safety of the surgical procedure by avoiding potential complications, reducing pain and/or speeding the patient’s recovery.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 shows a surgical system including a cannula assembly, in accordance with various embodiments.

[0018] FIG. 2 shows a block diagram illustrating an example of a device controller in accordance with various embodiments. [0019] FIG. 3 shows a block diagram illustrating an example of an imaging controller for a system in accordance with aspects.

DETAILED DESCRIPTION

[0020] Generally, there is provided imaging systems and, more particularly, endoscopic imaging systems. Systems and methods in accordance with various embodiments provide a cannula assembly that includes a cannula tube with a movable sensor housing. The sensor housing includes a light source and an imaging sensor configured to provide image data of a surgical site. The image data may relates to another surgical tool that is located within the surgical site, and the cannula assembly may have a processor that is configured to identify the other surgical tool using some identifying indicia of the other surgical tool. The identifying indicia can be employed for a variety of different purposes, as will be set forth in additional detail below.

[0021] Reference will now be made in detail to specific embodiments illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth to provide a thorough understanding. However, it will be apparent to one of ordinary skill in the art that embodiments may be practiced without these specific details. In other instances, known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0022] FIG. 1 illustrates an example embodiment. According to this example embodiment, there is provided a cannula assembly 111 A. The cannula assembly 111 A includes a cannula tube 209 having a longitudinal axis 209a, a proximal end portion 209b, and a distal end portion 209c configured for insertion into a patient. The cannula tube 209 has an internal lumen (not visible in this view) extending from the proximal end portion 209b to the distal end portion 209c. The cannula assembly 111 A also includes a sensor housing 217 coupled to the cannula tube 209 between the proximal and distal ends 209b, 209c of the cannula tube 209 so as to be positioned within the patient when the distal end portion 209c of the cannula tube 209 is inserted into the patient. In this embodiment, the sensor housing 217 is movable relative to the cannula tube 209 between a closed position and an open position. The sensor housing 217 includes a light source 235 and an image sensor 231 configured to provide image data when the sensor housing 217 is in the open position within the patient.

[0023] In various embodiments, the light source 235 can be a dimmable light-emitting device, such as a LED, a halogen bulb, an incandescent bulb, or other suitable light emitter. Generally, the image sensors 231 can be any device configured to detect light reflected from the light source 235 and to output an image signal. The image sensor 231 can be, for example, a charged coupled device (“CCD”) or other suitable imaging sensor. In some embodiments, the image sensor 231 includes at least two lenses providing stereo imaging. In some embodiments, the image sensor 231 can be an omnidirectional camera.

[0024] In still further embodiments, the cannula assembly 111 A may also include a processor or device controller 201 that is configured to receive the image data from the image sensor 231 and to process the image data for display on a separate display device, e.g., display device 107. Advantageously, the cannula assembly 111 A is configured such that the sensor housing 217 is rotatable about an axis (not shown in FIG. 1) that is transverse to the longitudinal axis 209a of the cannula tube 209. In this way, when the sensor housing 217 is in the above-mentioned open position, the sensor housing 217 is moved to a position that is more lateral relative to the longitudinal axis 209a of the cannula tube 209 as compared to the position of the sensor housing 217 when in the closed position.

[0025] FIG. 1 also illustrates a surgical tool 500. Surgical tool 500 is shown in FIG. 1 as being an endoscopic surgical stapler that is configured to be inserted through a cannula, e.g., a traditional cannula that is inserted through a separate puncture through the patient’s abdominal wall. However, it should be understood by persons skilled in the art that surgical tool 500 may be any conceivable type of surgical tool, without limitation to its type or use.

[0026] FIG. 1 also illustrates the surgical tool 500 as having an identifying indicia 501. In the embodiment shown, the identifying indicia 501 is any indicia that may be unique to a surgical tool 500, such that identifying indicia 501 may identify at least one aspect, feature or characteristic of the surgical tool 500. For example, the identifying indicia 501 may identify the type of surgical tool 500, e.g., a surgical stapler, a grasper, an electrocautery device, etc. Additionally or alternatively, the identifying indicia 501 may identify the specific manufacturer of the surgical tool 500, and/or the family of surgical tools, e.g., a manually-operated stapler, a powered stapler, etc. Still further, the identifying indicia 501 may identify a specific model, size or feature of the surgical tool 500, e.g., a surgical stapler having a 30mm length, a surgical stapler having a 60mm length, a surgical stapler having two rows or staples, a surgical stapler having three rows of staplers, a surgical stapler having a lockout mechanism, etc. Additionally or alternatively, the identifying indicia 501 may identify the specific tool itself as compared to other surgical tools of the same type, e.g., the specific serial number of the tool as differentiated from every other surgical tool of the same type.

[0027] In the embodiment shown, the identifying indicia 501 is a QR code that is unique to the surgical tool 500, the QR code being configured to identify at least one aspect, feature or characteristic of the surgical tool 500. In this embodiment, the QR code may be displayed, e.g., printed or otherwise attached or connected to, a portion of the surgical tool 500 that is visible within a patient by an imaging device such as, e.g., the imaging sensor 231. In this way, the imaging sensor 235 can generate imaging data related to the QR code and transmit same to a processor (described in additional detail below) for further processing.

[0028] In the embodiment shown, the identifying indicia 501, e.g., the QR code, is printed on a portion of the surgical stapler, specifically a movable jaw thereof. Additional or alternatively, the identifying indicia 501, e.g., the QR code, may be displayed on other portions of the surgical stapler, e.g., a shaft of the stapler. Advantageously, the identifying indicia 501 may be displayed on any portion of the surgical tool that is less likely to have its image be obscured or rendered difficult to be imaged while in use during surgery. For example, the movable jaw of a surgical stapler is known to be present in a surgical site by virtue of it being the portion of the stapler that is designed to engage with tissue within the surgical site; thus displaying the QR code thereon may help ensure that the QR code will be visible to, and imaged by, the imaging sensor 231 during at least some portion of the surgical procedure. Alternatively, the identifying indicia 501 may be displayed on the shaft of a surgical stapler at a longitudinal location thereof that is known to be present in a surgical site by virtue of it being positioned beyond the distalmost edge of a cannula through which it has been inserted. Displaying the identifying indicia 501 on the shaft of the surgical stapler may have the benefit of it not being obscured or covered by a portion of tissue, e.g. a portion of tissue that is grasped between the jaws of the stapler for subsequent cutting and stapling, when the image sensor 231 is imaging it. Of course, it should be recognized by persons skilled in the art that the identifying indicia 501 may be displayed at any suitable location on a surgical tool 500.

[0029] Although not shown herein, it will be understood by those skilled in the art that a QR code is only one of many different types of identifying indicia 501 that may be employed according to various embodiments. More specifically, the identifying indicia 501 may be any other type of, e.g., in this embodiment, visually distinguishable entity that can be imaged by image sensor 231. For example, in other embodiments, the identifying indicia 501 may be any type of code, e.g., a bar code or the like, that is capable of being imaged by image sensor 231. Furthermore, in other embodiments, the identifying indicia 501 may be any type of symbol, e.g., a mark, emblem, insignia, figure, logo or the like, that is capable of being imaged by the image sensor 231. Still further, in other embodiments, the identifying indicia 501 may be any type of character, e.g., a number or series of numbers such as a serial number, an alphabetical character and/or characters forming a word, etc. More specifically, the identifying indicia 501 could be any indicia that is capable of being imaged by the image sensor 231 and recognized by a processor for further processing.

[0030] It should also be recognized by those skilled in the art that, while described herein in connection with embodiments that employ an identifying indicia 501 that is displayed on a surgical tool, e.g., a symbol or character etc. that is printed, mounted, attached or otherwise coupled to the surgical tool 500, other embodiments are contemplated in which the identifying indicia 501 is a shape of the surgical tool itself. More specifically, in various embodiments, the shape of the surgical tool itself may provide an identifying indicia 501 of the surgical tool 500. In this case, the shape of the surgical tool 500 may be imaged by the image sensor 231, and the surgical tool 500 may be recognizable by the processor when the processor receives such image data of the shape of the surgical tool 500. For example, in embodiments, the processor may be configured to receive shape image data from the surgical stapler, and to process such shape image data so as to recognize the surgical tool as such a surgical stapler. And similar to that described hereinabove, employing the shape of the surgical tool as the identifying indicia 501 may enable the processor to identify and process any conceivable aspect, feature or characteristic of the surgical tool 500. For example, the identifying shape indicia 501 may identify the type of surgical tool 500, e.g., a surgical stapler, etc., and/or it may identify the specific manufacturer of the surgical tool 500 and/or the family of tools, e.g., a manually-operated stapler, etc., and/or it may identify a specific model, size or feature of the surgical tool 500, e.g., a surgical stapler having a 30mm length, etc.

[0031] In other embodiments, it is contemplated that the identifying indicia 501 may be a color, or colors, associated with the surgical tool. More specifically, in various embodiments, the color of the surgical tool itself, or the color of a portion of the surgical tool itself, may provide an identifying indicia 501 of the surgical tool 500. In this case, the color of the surgical tool 500 or portion thereof may be imaged by the image sensor 231, and the surgical tool 500 may be recognizable by the processor when the processor receives such color image data of the surgical tool 500. For example, in embodiments, the processor may be configured to receive color image data from a surgical stapler, and to process such color image data so as to recognize the surgical tool as such a surgical stapler. And similar to that described hereinabove, employing the color of the surgical tool as the identifying indicia 501 may enable the processor to identify and process any conceivable aspect, feature or characteristic of the surgical tool 500. For example, the identifying color indicia 501 may identify the type of surgical tool 500, e.g., a surgical stapler, etc., and/or it may identify the specific manufacturer of the surgical tool 500 and/or it may identify the family of tools, e.g., a manually-operated stapler, etc., and/or it may identify a specific model, size or feature of the surgical tool 500, e.g., a surgical stapler having a 30mm length, etc.

[0032] It should also be recognized by those skilled in the art that, in certain embodiments, the system may employ identifying indicia 501 that are sensed based on visual images, e.g., that are capable of being imaged by the image sensor 231, and/or may employ identifying indicia 501 based on other types of sensing. For example, in various embodiments, the system may employ identifying indicia 501 based on three-dimensional sensing technologies. Such three-dimensional sensing technologies may, in certain embodiments, identify a tool or type of tool based upon, e.g., a three-dimensional shape of the tool in addition to, or instead of, the two-dimensional profile of the tool. Examples of these types of three-dimensional sensing technologies may include, e.g., LIDAR scanners, structured illumination systems such as infrared time-of-flight or 3D scanner systems, and/or radar chips.

[0033] As set forth above, some of the types of identifying indicia 501 may include sensing technologies that are affixed to or built into the tool (as well as, in some embodiments, having sensing technology incorporated into the cannula assembly). For example, in an embodiment, the system may include ultra-wideband (UWB) receivers, e.g., incorporated into the cannula assembly 111A, along with an accompanying transmitter, e.g., integrated into the tool for broadcasting data related to the tool, e.g., a tool serial number or the like. In another embodiment, the system may include holographic stickers which may be sensed by the, e.g., the image sensor described above, and displayed via the display device. In still another embodiment, the surgical tool may include, e.g., braille-like dots and the cannula assembly may include, e.g., a capacitive or other pressure/contact sensor, configured to sense the dots.

[0034] In still another embodiment, the cannula assembly 111A may include a near-field communication (NFC) reader and accompanying NFC-enabled tag on the surgical tool. NFC is a set of communication protocols that enable communication between two electronic devices over a distance of about 4cm or less. Generally, NFC is based on inductive coupling between two antennas present on NFC-enabled devices using a frequency of 13.56 MHz in the globally available unlicensed radio frequency ISM band using the ISO/IEC 18000-3 air interface standard at data rates ranging from 106 to 424 kbits/second.

[0035] In still another embodiment, the surgical tool may include metal contacts, e.g., on the outside thereof, and these metal contacts may communicate with similar metal contacts on the cannula assembly 111A. Additionally or alternatively, the surgical tool may have integrated thereon a magnetic strip, similar to, e.g., credit card magnetic strips, that can be read in much the same way as by a compatible magnetic strip swipe reader. Still further, communication between the cannula assembly and the surgical tool may be accomplished via, e.g., WiFi, Bluetooth, or any other communication or sensing protocol that employs identifying indicia 501 that may be digital in nature.

[0036] Although not shown herein, it will be understood by those skilled in the art that the cannula assembly 111A (and other cannula assemblies shown and described herein) may include other components and features in addition to those described herein. For example, any of the herein-described cannula assembly 111A may include sealing components, such as an instrument seal for sealing around an instrument inserted therethrough, a zero seal for sealing the cannula assembly in the absence of any instrument inserted therethrough, and/or any number of different ports, e.g., insufflation or irrigation ports, for the introduction of various gases or liquids into the surgical site.

[0037] In the embodiment shown in FIG. 1, the cannula assembly 111 A also includes other features such as a housing 200 (which may house, e.g., the afore-mentioned instrument and zero seals among other components), a device controller 201, an actuator handle 205 and an obturator 211. As set forth above, the cannula tube 209, the obturator 211, and the sensor housing 217 of the cannula assembly 111A can be inserted into the body of a patient (e.g., patient 117) so as to provide an image of the patient’s surgical site, e.g., the interior abdominal region, from the sensor housing 217, and the device controller 201 can be one or more devices that process signals and data to generate image stream 127A.

[0038] The cannula tube 209 may be formed of a variety of cross-sectional shapes. For example, the cannula tube 209 can have a generally round or cylindrical, ellipsoidal, triangular, square, rectangular, and D-shaped (in which one side is flat). In some embodiments and as described hereinabove, the cannula tube 209 includes an internal lumen 202 into which the obturator 211 is inserted. The obturator 211 can be retractable and/or removable from the cannula tube 209. In some embodiments, the obturator 211 is made of solid, non-transparent material. In another embodiment, all or parts of the obturator 211are made of optically transparent or transmissive material such that the obturator 211 does not obstruct the view through the camera (discussed below). The obturator 211 may have a tip shape that is configured to penetrate, either via incision or via insertion between tissue planes, through the abdominal wall of the patient. [0039] The sensor housing 217 can be integral with the cannula tube 209 or it may be formed as a separate component that is coupled to the cannula tube 209. In either case, the sensor housing 217 can be disposed on or coupled to the cannula tube 209 at a position proximal to the distalmost end of the cannula tube 209 such that it is positioned within the patient’s body cavity when the distal end portion of the cannula tube 209 has been inserted into the patient. In some embodiments, the sensor housing 217 can be actuated by the actuator handle 205 to open, for example, after being inserted into the patient’s 117 body cavity. The sensor housing 217 may reside along cannula tube 209 in the distal direction such that it is positioned within the body cavity of a patient (e.g., patient 117) during a surgical procedure. At the same time, sensor housing 217 can be positioned proximal to the distal end 209c such that it does not interfere with the insertion of the distal end 209c of the cannula tube 209 as it is inserted into a patient (e.g., patient 117). In addition, the sensor housing 217 can positioned proximally from the distal end 209c so as to protect the electronic components therein as the distal end 209c is inserted into the patient.

[0040] In the embodiment shown in FIG. 1, the system 100 may also include an imaging controller 105 and a display device 107 having a display 145. The imaging controller 105 can be a computing device connected to the display device 107 and the cannula assembly 111 A through one or more wired or wireless communication channels 123 A. The communication channels 123A may use various serial, parallel, video transmission protocols suitable for their respective signals such as image stream 127A and processed image stream 133. The imaging controller 105 can include hardware, software, or a combination thereof for performing operations. The display device 107 can be a liquid crystal display (LCD) display, organic light emitting diode displays (OLED), cathode ray tube display, or other suitable display device. In some embodiments, the display device 107 can be a stereoscopic head-mounted display, such as a virtual reality headset. [0041] It should be noted that, while the description hereinbelow describes various components, operations and functions as potentially being present or performed by one or either of the device controller 201 and/or the image controller 105, it is contemplated that the below-described, components, operations and functions may be present or performed entirely in a single one of the device controller 201 or the image controller 105; that the components may be shared across the device controller 201 and the image controller 105 such that the device controller 201 and the image controller 105 may share responsibility for performing any one or more of the herein-described operations or functions; or that additional processors or controllers may be employed to perform the below-described, components, operations and functions.

[0042] As will be shown below, FIG. 2 illustrates a device controller 201 that has components for, and performs certain operations and functions, while FIG. 3 illustrates an image controller 105 that has components for, and performs certain operations and functions. It should be recognized by those skilled in the art that, in accordance with other embodiments thereof, there may be included processors either internal or external to the cannula assembly 111 A for performing these operations and functions, and that it is not intended to be limited to any particular structure or location of such components, operations or functions.

[0043] FIG. 2 shows a functional block diagram illustrating an example of a device controller 201, in accordance with various embodiments. In the embodiment shown, the device controller 201 can include a processor 305, a memory device 307, a storage device 309, a communication interface 311, a transmitter/receiver 313, an image processor 315, and a data bus 319. In some embodiments, the memory device 307 of device controller/processor 201 includes the stored data, e.g., as a look-up table or the like, that relates to the identifying indicia 501. Of course, as noted above, in other embodiments, the stored data relating to the identifying indicia 501, may additionally or alternatively be stored in memory device 407 of the image controller 105 (see FIG 3), and thus the operations described hereinabove related to data stored by the memory device 307 may also or alternatively be performed in connection with data stored by the memory device 407.

[0044] In some embodiments, the processor 305 can include one or more microprocessors, microchips, or application-specific integrated circuits. The memory device 307 can include one or more types of random-access memory (RAM), read-only memory (ROM) and cache memory employed during execution of program instructions. As set forth above, the memory device 307 can include stored data that relates to the identifying indicia 501, and more specifically, may include stored data to which the image stream 127 can be compared so as to enable the processor 201 to recognize and identify the surgical tool 500 based on the identifying indicia 501.

[0045] The processor 305 can use the data buses 319 to communicate with the memory device 307, the storage device 309, the communication interface 311, and the image processor 315. The storage device 309 can comprise a computer-readable, non-volatile hardware storage device that stores information and program instructions. For example, the storage device 309 can be one or more, flash drives and/or hard disk drives. The transmitter/receiver 313 can be one or more devices that encodes/decodes data into wireless signals.

[0046] The processor 305 executes program instructions (e.g., an operating system and/or application programs), which can be stored in, e.g., the storage device 309. The image processing module 359 can include program instructions that, using the image signal 365 from an imaging sensor (e.g., image sensor 231), generate the image stream 127. In some embodiments, the image processor 315 can be a device configured to receive an image signal

365 from an image sensor (e.g., image sensor 231) and to condition those images. In accordance with various embodiments, conditioning the image signal 365 can include normalizing the size, exposure, and brightness of the images. Also, conditioning the image signal 365 can include removing visual artifacts and/or stabilizing the images to reduce blurring due to motion.

[0047] In further embodiments thereof, it is contemplated that the image processing module 359 can add information to the image stream 127 relating to the surgical tool 500 based on it having been identified by its identifying indicia 501. For example, it is contemplated that the image processing module 359 can obtain, e.g., from the memory device 307, tool identification data 502 related to the surgical tool 500 that has been identified, and to use that tool identification data 502 to generate a tool identification label 504 that can be added to the image stream 127. This tool identification label 504 can be displayed as part of the image stream 127 in display 145 on the display device 107 (see, e.g., FIG. 1). In this way, a user that is viewing the display device 107 can see in real-time the tool identification label 504 related to the identified surgical tool 500.

[0048] This tool identification label 504 can be any type of information that relates to the identified surgical tool 500. More specifically, in various embodiments and as set forth above in connection with the identification indicia 501, the tool identification label 504 can be a label that identifies any conceivable aspect, feature or characteristic of the surgical tool 500. For example, in some embodiments, the tool identification label 504 can be a label that identifies the type of surgical tool 500, e.g., a surgical stapler, etc., and/or it may be a label that identifies the specific manufacturer of the surgical tool 500 and/or it may be a label that identifies the family of tools to which surgical tool 500 belongs, e.g., a manually-operated stapler, etc., and/or it may be a label that identifies a specific model, size or feature of the surgical tool 500, e.g., a surgical stapler having a 30mm length, etc., and/or it may be a label that identifies the specific surgical tool differentiated from every other tool of its type, e.g., by its serial number. [0049] Still further, the tool identification label 504 that is added to the image stream 127 by the image processing module 359 may include other type of data related to that surgical tool 500. For example, in accordance with various embodiments, the tool identification label 504 that is added to the image stream 127 by the image processing module 359 may include instructional data related to that surgical tool 500, e.g., a label that instructs the surgeon regarding some operation of the surgical tool 500. Such an operational instruction could be, e.g., an instruction to engage a tissue within the surgical site, an instruction to operate a feature of the surgical tool 500, an instruction to actuate a particular feature of the surgical tool 500, or any other type of operational instruction.

[0050] Still other types of data related to the surgical tool 500 may be included in the tool identification label 504 that is added to the image stream 127 by the image processing module 359. For example, in accordance with various embodiments, the tool identification label 504 that is added to the image stream 127 by the image processing module 359 may include safety data related to that surgical tool 500, e.g., a label that advises the surgeon regarding some aspect of safety related to the surgical tool 500. Such a safety instruction could be, e.g., an instruction that the surgical tool 500 is currently in an unsafe operating condition, an instruction to avoid engaging with the surgical tool 500 a particular section of type of tissue in the surgical site, a reminder to perform or avoid a particular operation related to the surgical tool 500, or any other type of safety instruction.

[0051] It is noted that the device controller 201 is merely representative of various possible equivalent-computing devices that can perform the processes and functions described herein. To this extent, in some embodiments, the functionality provided by the device controller 201 can be any combination of general and/or specific purpose hardware and/or program instructions. In each embodiment, the program instructions and hardware can be created using standard programming and engineering techniques. [0052] FIG. 3 shows a functional block diagram illustrating the imaging controller 105 in accordance with various embodiments. As noted above, the image controller 105 may take the place of (or be taken the place by), and/or may share components, operations and/or functions with the device controller 201. In this embodiment, the imaging controller 105 may include a processor 405, a memory device 407, a storage device 409, a network interface 413, an image processor 421, and a data bus 431. Also, the imaging controller 105 can include image input connections 461A and image output connection 463 that receive and transmit image signals from the image processor 421.

[0053] In embodiments, the imaging controller 105 can include one or more microprocessors, microchips, or application-specific integrated circuits. The memory device 407 can include one or more types of random-access memory (RAM), read-only memory (ROM) and cache memory employed during execution of program instructions. As set forth above, while an embodiment is described hereinabove in connection with FIG. 2 wherein the memory device 307 of device controller/processor 201 includes the stored data that relates to the identifying indicia 501, in other embodiments, stored data relating to the identifying indicia 501, may additionally or alternatively be stored in memory device 407 of the image controller 105. Thus, the operations described hereinabove related to data stored by the memory device 307 may also or alternatively be performed in connection with data stored by the memory device 407. For example, data relating to the identifying indicia 501 that is stored by the memory device 407 can be employed by a processor, e.g., processor 405, for comparison to the image stream 127 so as to enable the processor 201 to recognize and identify the surgical tool 500 based on the identifying indicia 501.

[0054] According to various embodiments thereof, it is also contemplated that one or more of the processors described hereinabove, e.g., the processor 305 of device controller 201 and/or the processor 405 of the imaging controller 105, or a separate processor (not shown) may be employed to generate data that provides real-time or subsequent insights that can be leveraged by users or manufacturers of the system 100. For example, in various embodiments, one or more of the processors may be configured to process data related to the manner in which the identified surgical tool 500 was utilized by the surgeon during the surgical procedure. For example, the one or more of the processors may be configured, after the surgical tool 500 has been identified via its identifying indicia 501, to further process image data 127 related to the operating steps that were performed by the surgeon while using the surgical tool 500. In this way, it can be determined by the processor (or by subsequent review of the imaging data 127) whether the surgeon performed the steps in a preferred or optimal way, and/or whether the surgeon performed the step in a suboptimal or unsafe manner. Additionally or alternatively, the one or more of the processors may be configured, after the surgical tool 500 has been identified, to further process image data 127 related to the time it took for the surgeon to perform the operating steps of the surgical tool 500, thereby enabling the processor (or a subsequent review of the imaging data 127) to determine whether the surgeon performed the steps, or any specific step or steps, in a preferred duration of time, and/or whether the surgeon performed the steps, or any specific step or steps, in a duration that was too fast or slow.

[0055] In such embodiments, wherein one or more of the processors may be configured to process data related to the manner in which the identified surgical tool 500 was utilized by the surgeon during the surgical procedure, the additional data that such second processor, e.g., processor 405, processes may be used for a variety of different purposes. For example, in some embodiments, the additional data that such second processor processes may be used to track the use of the at least one surgical tool. Thus, in the embodiment described hereinabove, the one or more of the processors may be configured, after the surgical tool 500 has been identified via its identifying indicia 501, to further process image data 127 or other data that provides visual evidence that the surgical tool 500 was employed during the surgical procedure. This visual evidence could be useful in a variety of different ways, e.g., in order to demonstrate surgeon liability in the event that the surgeon is accused of using a surgical tool 500 that was unsuitable and/or in order to demonstrate the lack of surgeon liability by showing that the surgical tool 500 was the correct surgical tool to use in a given surgical situation. Additionally or alternatively, this visual evidence could be useful, e.g., in order to determine whether a surgical tool 500 was used in a hospital or other surgical location for which it was not authorized to be used (e.g., if it was used in a hospital that it was not sold to, if it was used in a country for which it is has not received clearance for use, etc.), and/or whether a surgical tool that is intended for single use only is, in fact, being reused and thereby potentially used in an unauthorized or unsafe manner.

[0056] In further embodiments, the additional data that such second processor, e.g., processor 405, processes may be used to help train subsequent users of the surgical tool 500 on the proper (or improper) use of the surgical tool 500. Thus, in the embodiment described hereinabove, the one or more of the processors may be configured, after the surgical tool 500 has been identified via its identifying indicia 501, to further process image data 127 or other data that provides visual training or guidance to someone viewing the surgical procedure (or someone subsequently viewing the image data 127) about how best to use the surgical tool 500 during the surgical procedure. This visual guidance could be useful in a variety of different ways, e.g., it may be useful to show a trainee how to perform the steps in a preferred or optimal way, and/or how performing the steps in a different way could potentially be a suboptimal or unsafe manner. Additionally or alternatively, this visual guidance could be useful to show a trainee the preferred or optimal amount of time for a surgeon to perform the operating steps, or any specific step or steps, of the surgical tool 500, thereby enabling the surgeon to speed up or slow down his or her technique to be more consistent with the optimal duration(s). [0057] In still further embodiments, the additional data that such second processor, e.g., processor 405, processes may be used to identify design improvements that can be made to the surgical tool 500. Thus, in the embodiment described hereinabove, the one or more of the processors may be configured, after the surgical tool 500 has been identified via its identifying indicia 501, to further process image data 127 or other data that helps identify operational advantages or disadvantages that can be addressed or improved by altering or modifying the design of the surgical tool 500. This data relating to potential design improvements could be useful in a variety of different ways, e.g., it may be useful to show a designer or manufacturer of the surgical tool 500 that surgeons are performing a given step in a suboptimal or unsafe manner, and the designer or manufacturer may choose to alter the design of the surgical tool 500 so as to discourage or prevent the surgeon from using the surgical tool 50 in that suboptimal or unsafe way, e.g., adding a lockout mechanism if a surgeon is performing a step out of sequence; altering, moving or removing an actuation mechanism if the surgeon is incorrectly actuating the device; lengthening or shortening a component if the surgeon is not optimally engaging the tissue with the current dimensions of the surgical tool 500, etc. Additionally or alternatively, this data may be useful to show a designer or manufacturer of the surgical tool 500 that the current design of the surgical tool is encouraging or causing surgeons to perform a given step too slowly or too quickly, and may enable the designer or manufacturer to consider or implement alterations or modifications to the design of the surgical tool 500 so as to discourage or prevent the surgeon from performing the given step too slowly or too quickly, e.g., altering or modifying the length of an actuation mechanism, e.g. handle or trigger stroke, so that the device can be actuated more quickly or slowly; changing a gear ratio so that a motorized actuation of a driver takes longer or slower so as to have improved tissue effects; combining, eliminating or adding certain steps in the operation of the surgical tool 500 so as to enable a surgeon to slow down or speed up his or her surgical technique, etc. Of course, given that the surgical tool 500 may be any conceivable type of surgical tool, there is no limit to the number of ways that a surgical device may potentially have its design improved, and there is likewise no limit to the number of ways that the processors could identify or suggest such design improvements.

[0058] There are no limitations in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects only. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims. Only the terms of the appended claims are intended to be limiting, along with the full scope of equivalents to which such claims are entitled. It is also to be understood that the terminology used herein, e.g., “and”, “or”, “including”, “at least” as well as the use of plural or singular forms, etc., is for the purpose of describing examples of embodiments and is not intended to be limiting.

Claims

CLAIMS What is claimed is:

1. A cannula assembly for use in a surgical procedure, comprising: a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient; a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end portion of the cannula tube is inserted into the patient, the housing movable relative to the cannula tube between a closed position and an open position, the housing including a light source and an image sensor configured to provide image data when the housing is in the open position within the patient; and a processor configured to receive the image data from the image sensor and to process the image data to identify at least one surgical tool within the patient.

2. The cannula assembly of claim 1, wherein the processor is configured to identify the at least one surgical tool within the patient by processing an identifying indicia of the at least one surgical tool.

3. The cannula assembly of claim 2, wherein the identifying indicia of the at least one surgical tool includes one of a QR code and a serial number displayed on the at least one surgical tool.

4. The cannula assembly of claim 2, wherein the identifying indicia on the at least one surgical tool includes one of a shape and a color of the at least one surgical tool.

5. The cannula assembly of claim 2, wherein the identifying indicia includes a structure that is affixed to or built into the surgical tool.

6. The cannula assembly of claim 2, further comprising: an identification data storage device having stored data that correlates the identifying indicia to particular types of surgical tools, wherein the processor is configured to identify a particular type of surgical tool within the patient by comparing the received image data to the stored data of the identification data storage device.

7. The cannula assembly of claim 1, wherein the processor is configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool.

8. The cannula assembly of claim 2, further comprising: a second processor configured to further process additional data related to the identified at least one surgical tool.

9. The cannula assembly of claim 8, wherein the second processor is configured to process data related to at least one of the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and the duration of time for which the identified surgical tool was utilized during the surgical procedure.

10. The cannula assembly of claim 9, wherein the additional data that the second processor processes is employed to at least one of track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and identify design improvements that can be made to the at least one surgical tool.

11. The cannula assembly of claim 1, wherein the housing is rotatable about an axis transverse to a longitudinal axis of the cannula tube such that, when in the open position, the housing is positioned laterally relative to the longitudinal axis of the cannula tube.

12. A surgical system for use in a surgical procedure, comprising: a cannula assembly including a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient, the cannula tube having an internal lumen extending from the proximal end portion to the distal end portion, and an obturator configured to be removably inserted into the internal lumen so as to enable in insertion of the cannula tube into the patient; a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end portion of the cannula tube is inserted into the patient, the housing movable relative to the cannula tube between a closed position and an open position, the housing including a light source and an image sensor configured to receive image data from at least one surgical tool within the surgical site when the housing is in the open position within the patient; a processor configured to receive the image data from the image sensor and to process the image data to identify the at least one surgical tool within the patient; and a display device configured to display images related to the image data.

13. The surgical system of claim 12, wherein the processor is configured to identify the at least one surgical tool within the patient by processing an identifying indicia of the at least one surgical tool.

14. The surgical system of claim 13, wherein the identifying indicia of the at least one surgical tool includes one of a QR code and a serial number displayed on the at least one surgical tool.

15. The surgical system of claim 13, wherein the identifying indicia on the at least one surgical tool includes one of a shape and a color of the at least one surgical tool.

16. The cannula assembly of claim 13, wherein the identifying indicia includes a structure that is affixed to or built into the surgical tool.

17. The surgical system of claim 13, further comprising: an identification data storage device having stored data that correlates the identifying indicia to particular types of surgical tools, wherein the processor is configured to identify a particular type of surgical tool within the patient by comparing the received image data to the stored data of the identification data storage device.

18. The surgical system of claim 12, wherein the processor is configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool.

19. The surgical system of claim 12, further comprising: a second processor configured to further process additional data related to the identified at least one surgical tool.

20. The surgical system of claim 19, wherein the second processor is configured to process data related to at least one of the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and the duration of time for which the identified surgical tool was utilized during the surgical procedure.

21. The surgical system of claim 19, wherein the additional data that the second processor processes is employed to at least one of track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and identify design improvements that can be made to the at least one surgical tool.

22. The surgical system of claim 12, wherein the housing is rotatable about an axis transverse to the longitudinal axis of the cannula tube such that, when in the open position, the housing is positioned further laterally relative to the longitudinal axis of the cannula tube.

23. A cannula assembly for use in a surgical procedure, comprising: a cannula tube having a longitudinal axis, a proximal end portion and a distal end portion configured for insertion into a patient; a housing coupled to the cannula tube between the proximal and distal ends of the cannula tube so as to be positioned within the patient when the distal end portion of the cannula tube is inserted into the patient, the housing including a sensor configured to provide sensor data related to a structure affixed to or built into at least one surgical tool within the patient; and a processor configured to receive the sensor data from the sensor and to process the sensor data to identify the at least one surgical tool.

24. The cannula assembly of claim 23, wherein the structure includes one or more of an ultra-wideband transmitter, a holographic sticker, one or more braille-like dots, a near- field communication tag, metal contacts, a magnetic strip reader, a colored sticker and a Bluetooth or wifi component.

25. The cannula assembly of claim 24, further comprising: an identification data storage device having stored data that correlates the identifying indicia to particular types of surgical tools, wherein the processor is configured to identify a particular type of surgical tool within the patient by comparing the received sensor data to the stored data of the identification data storage device.

26. The cannula assembly of claim 23, wherein the processor is configured to provide identification data to a display device such that the display viewed by the surgeon via the display device includes an identification label of the at least one surgical tool.

27. The cannula assembly of claim 23, further comprising: a second processor configured to further process additional data related to the identified at least one surgical tool.

28. The cannula assembly of claim 27, wherein the second processor is configured to process data related to at least one of the manner in which the identified surgical tool was utilized by the surgeon during the surgical procedure, and the duration of time for which the identified surgical tool was utilized during the surgical procedure.

29. The cannula assembly of claim 27, wherein the additional data that the second processor processes is employed to at least one of track the use of the at least one surgical tool, help train the use of the at least one surgical tool, and identify design improvements that can be made to the at least one surgical tool.