EP4150608A1

EP4150608A1 - Simulation-based directed surgical development system

Info

Publication number: EP4150608A1
Application number: EP22727473.5A
Authority: EP
Inventors: Charles J. Scheib; Iv Frederick E. Shelton; Dwight Alan Meglan
Original assignee: Cilag GmbH International
Current assignee: Cilag GmbH International
Priority date: 2021-05-21
Filing date: 2022-05-20
Publication date: 2023-03-22
Also published as: WO2022243966A1; JP2024521110A

Abstract

An interactive and dynamic surgical simulation system may be used in the context of a computer-implemented interactive surgical system. The surgical simulation system may enable simulation-based directed surgical development. A processor may be configured to receive a plurality of training task-environment parameter sets from a surgical data system. The processor may generate, for each training task-environment parameter set, a plurality of candidate action sets. Each candidate action set, when executed within the simulated environment indicated by the corresponding training task-environment, set may cause performance, with a corresponding effectiveness, of the simulated surgical task. This information may be used to train a model to then determine the target action set. The target action set may represent a way of performing the simulated surgical task, optimized according to defined effectiveness.

Description

Simulation-Based Directed Surgical Development System Cross-Reference to Related Applications [001] This application claims the benefit of Provisional U.S. Patent Application No.63/191,681, May 21, 2021, the disclosure of which is incorporated herein by reference in its entirety. [002] This application is related to the following, filed contemporaneously, the contents of each of which are incorporated by reference herein: · U.S. Patent Application No.17/332,594, filed May 27, 2021, (Attorney Docket No. END9338USNP1), titled METHODS FOR SURGICAL SIMULATION · U.S. Patent Application No.17/332,524, filed May 27, 2021, (Attorney Docket No. END9338USNP2), titled SURGICAL SIMULATION OBJECT RECTIFICATION SYSTEM · U.S. Patent Application No.17/332,399, filed May 27, 2021, (Attorney Docket No. END9338USNP3), titled SURGICAL SIMULATION NAVIGATION SYSTEM · U.S. Patent Application No.17/332,441, filed May 27, 2021, (Attorney Docket No. END9338USNP4), titled SURGICAL SIMULATION SYSTEM WITH COORDINATED IMAGINING · U.S. Patent Application No.17/332,462, filed May 27, 2021, (Attorney Docket No. END9338USNP5), titled SURGICAL SIMULATION SYSTEM WITH SIMULATED SURGICAL EQUIPMENT COORDINATION · U.S. Patent Application No.17/332,197, filed May 27, 2021, (Attorney Docket No. END9338USNP6), titled SIMULATION-BASED SURGICAL PROCEDURE PLANNING SYSTEM · U.S. Patent Application No.17/332,449, filed May 27, 2021, (Attorney Docket No. END9338USNP8), titled SURGICAL ADVERSE EVENT SIMULATION SYSTEM · U.S. Patent Application No.17/332,496, filed May 27, 2021, (Attorney Docket No. END9338USNP9), titled SIMULATION-BASED SURGICAL ANALYSIS SYSTEM · U.S. Patent Application No.17/332,480, filed May 27, 2021, (Attorney Docket No. END9338USNP10), titled DYNAMIC ADAPTATION SYSTEM FOR SURGICAL SIMULATION Background ^[003] Surgical simulations, such as computer-based, three-dimensional simulations of a surgical environment and/or surgical procedure for example, present an opportunity to advance the surgical arts. Surgical simulations have potential to benefit surgical training, planning, development, and the like. For example, surgical simulations may be used to train surgeons in new procedures and/or to improve the performance of procedures they already know. Surgical simulations may be used as a virtual “dress rehearsal” to help a surgeon prepare for an upcoming procedure. And surgical simulations may be used to experiment with unproven procedures and techniques. [004] However, surgical simulation platforms are complex systems that face many limitations in capabilities, scope, and applicability. For example, many platforms are technology “silos,” specifically programmed and tailored to address a particular learning objective or to simulate the operation of a singular piece of equipment, such as simulating the operation of a surgical robot. Limitations, such as these, may dimmish a platform’s effectiveness as a tool to advance the surgical arts. And such limitations may represent significant technological roadblocks to the integration of simulation-based applications into other aspects of the surgical process, such a pre-operative planning, intra-operative support, post-operative analysis, and the like. [005] Accordingly, innovation in surgical simulation technology, such as technical advancements that address surgical simulation capabilities, scope, and applicability for example, may accelerate further progress in the surgical arts. Summary ^[006] An interactive and dynamic surgical simulation system is disclosed. The surgical simulation system may be used in the context of a computer-implemented interactive surgical system. The surgical simulation system may enable simulation-based directed surgical development. A device may be used to determine a target action set for performing a simulated surgical task in a simulated environment. For example, the target action set may represent a way of performing the simulated surgical task, optimized according to defined effectiveness metric. ^[007] A processor may be configured to receive a plurality of training task-environment parameter sets from a surgical data system. Each training task-environment parameter set may include anatomical information, physiological information, surgical setup information, and the like. Each training task-environment parameter set may correspond to a respective simulated environment within which a simulated surgical task is to be completed. ^[008] The processor may generate, for each training task-environment parameter set, a plurality of candidate action sets. Each candidate action set may include a parameterized surgical activity within the respective simulated environment. For example, a simulated transection activity may be parameterized by relative anatomical location, instrument selection, instrument application direction, and the like. Each candidate action set, when executed within a simulated environment indicated by the corresponding training task-environment set may cause performance of the simulated surgical task. That performance may result in a respective effectiveness that is indicated by a corresponding effectiveness metric. ^[009] This information may be used to train a model. The processor may train a model with the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. And the processor may use the model to then determine a target action set based on the target task-environment parameter set. ^[0010] There is provided, according to the present invention, a computer-implemented method for determining a target action set, the method comprising: receiving a plurality of environment parameter sets, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generating, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment, causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; training a machine- learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determining a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. ^[0011] It may be understood that an action set (e.g. a candidate action set, a target action set, or an input action set) can refer to a set of parameters which define one or more actions taken during a surgery. In some embodiments, an action set can refer to a number of surgical choices taken by an operator. In some embodiments, an action set can include specific layout decisions (both of the surgical staff and of the surgical instrumentation). It can also refer to the specific equipment used during a surgery. In other words, it may refer to portions of a surgical procedure which have consequences on the surgery, both in duration, surgical outcomes, effectiveness of the procedure, etc. It is to be understood that the candidate, target, and input action sets are all data sets having the same format. That is, a machine-learning model can be trained on the basis of a candidate action set to help define a relationship between any arbitrary “input” action set. This allows the machine- learning model to be used to find a target action set within a given simulation having a certain effectiveness. ^[0012] It may also be understood that an environment parameter set (e.g. an environment parameter set, a target environment parameter set, or an input environment parameter set) can refer to a set of parameters which define the overall environment of a surgical simulation. In other words, it refers to the parameters which initialize and define the constraints and physics of a surgical simulation. It is to be understood that the target and environment parameter sets are all data sets having the same format as that of the environment parameter set. That is, a machine-learning model can be trained on the basis of an environment parameter set to help define a relationship between any arbitrary “input” environment parameter set. This allows the machine-learning model to be used to find a target action set within a given simulation having a certain effectiveness. [0013] In some embodiments, “training a machine-learning model” may refer to using a set of labelled data to train the model in a supervised fashion. In this case, the combination of the environment parameter set and the action set can be considered to be the input data (as this refers to a specific set of actions run within a specific simulation environment). This data for training purposes can be “labelled” with its associated effectiveness metric (i.e. the output of the simulation, which relates to how effective the specific action was within that simulation). By feeding this labelled data into the machine-learning model, the model can generate a relationship between different actions/environments and the associated effectiveness. ^[0014] For the avoidance of doubt, an “environment parameter set” may be the same parameter set that the application refers to as a “training task-environment parameter set” (i.e. a parameter set that can be used to train the machine learning model). ^[0015] Both the initialization of the surgical simulation (which is defined by the environment parameter set) and the specific choices/actions taken within the simulation (which is defined by the candidate action set) have implications on the overall effectiveness of the surgery (defined by the effectiveness metric). ^[0016] In some embodiments, “determining a target action set” may refer to selecting an action set from a plurality of action sets within a predetermined simulation (as defined by an environment parameter set). The target action set may be selected based on which target action set (as an input to the machine learning model) produces the best output efficiency metric from the machine learning model. ^[0017] In other words, the present invention uses the output data of a simulation to improve the training of a machine learning model, and then uses the improved trained artificial intelligence to make an improved conclusion regarding surgical actions/choices that can be performed within a certain surgical set-up. ^[0018] One advantage of this is that a machine-learning model (such as a neural network) can be trained with additional data which may not arise commonly during a live surgical procedure. In other words, the simulation can provide data regarding “riskier” surgical choices and their consequences to a machine learning model. The informed intelligence coming from the machine learning model can inform future decisions on surgery, by highlighting which decisions actually have beneficial effects without needing to risk real patients to gather the appropriate data. [0019] The consequence of improving the training of a machine learning model in the way of the present embodiment is that improved surgical outcomes can be achieved via the target action set determined. ^[0020] In some embodiments, receiving the plurality of environment parameter sets comprises receiving the plurality of environment parameter sets from a surgical data system that records real-world environment data from real-world performances of the surgical task and that generates the plurality of environment parameter sets from the real-world environment data. ^[0021] In some embodiments, each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[0022] In some embodiments, each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[0023] In some embodiments, the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. ^[0024] In some embodiments, the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. [0025] In some embodiments, the parameterized activity comprises a surgical robotic guidance path. [0026] In some embodiments, the simulated surgical task comprises lower anterior resection anastomosis; and each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task. ^[0027] In some embodiments, the simulated surgical task comprises any of segmentectomy and lobectomy; and each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task. [0028] In some embodiments, the method further comprises generating an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. ^[0029] In some embodiments, training the machine-learning model comprises training the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. [0030] There is also provided, according to the present invention, a device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment corresponding to the corresponding environment set causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. [0031] In some embodiments, the surgical data system is configured to record real-world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data. ^[0032] In some embodiments, each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[0033] In some embodiments, each candidate action set comprises a parameterized surgical activity within the respective simulated environment. [0034] In some embodiments, the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. ^[0035] In some embodiments, the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. [0036] In some embodiments, the parameterized activity comprises a surgical robotic guidance path. [0037] In some embodiments, the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task. ^[0038] In some embodiments, the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task. ^[0039] In some embodiments, the processor is further configured to generate an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. [0040] In some embodiments, the processor is configured to train the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. ^[0041] There is also provided, according to the present invention, a device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment causes performance of the simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine- learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. ^[0042] In some embodiments, the surgical data system is configured to record real-world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data. ^[0043] In some embodiments, each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[0044] In some embodiments, each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[0045] There is also provided, according to the present invention, a device for determining a target action set, the device comprising: a processor configured to: receive a machine- learning model trained with the plurality of environment parameter sets, respective plurality of candidate action sets, and corresponding effectiveness metrics, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric, determine a target action set based on the environment parameter set and the machine- learning model; wherein each environment parameter set corresponds to the respective simulated environment within which the simulated surgical task is to be completed; wherein the plurality of environment parameter sets are from a surgical data system, wherein for each environment parameter set, a plurality of candidate action sets when executed within the simulated environment corresponding to the corresponding environment set causes performance of the simulated surgical task with an effectiveness that is indicated by the corresponding effectiveness metric. ^[0046] In some embodiments, each environment parameter set comprises anatomical information, physiological information, and surgical setup information, and wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[0047] There is also provided, according to the present invention, a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out a method according to the invention. Brief Description of the Drawings ^[0048] FIG.1 is a block diagram of a computer-implemented interactive surgical system. [0049] FIG.2 shows an example surgical system being used to perform a surgical procedure in an operating room. [0050] FIG.3 shows an example surgical hub paired with a visualization system, a robotic system, and an intelligent instrument, in accordance with at least one aspect of the present disclosure. ^[0051] FIG.4 illustrates a surgical data network having a communication hub configured to connect modular devices located in one or more operating theaters of a healthcare facility, or any room in a healthcare facility specially equipped for surgical operations, to the cloud, in accordance with at least one aspect of the present disclosure. ^[0052] FIG.5 illustrates an example computer-implemented interactive surgical system. [0053] FIG.6 illustrates an example surgical hub comprising a plurality of modules coupled to the modular control tower. [0054] FIG 7. is a block diagram of an example surgical simulator system. ^[0055] FIG.8 is a block diagram of an example surgical simulator system. ^[0056] FIG.9 is a block diagram depicting an example surgical simulator user interface device. [0057] FIG.10 is a flow chart of an example surgical simulator operation. ^[0058] FIGs.11A-B illustrate example surgical procedural plan data structures for use with a computer-implemented interactive surgical system and/or a surgical simulator. ^[0059] FIG.12 is a block diagram of an example machine learning framework. ^[0060] FIG.13 is a block diagram of an example surgeon agent. [0061] FIG.14 illustrates an example simulation activity parameterization data set. ^[0062] FIG.15 illustrates an example surgical simulation flow for machine learning training. ^[0063] FIG.16 is a flow chart of an example surgical simulation operation. Detailed Description [0064] Surgical simulation systems, devices, and methods may include aspects of integration with other medical equipment, data sources, processes, and institutions. Surgical simulation systems, devices, and methods may include aspects of integration with a computer- implemented interactive surgical system and/or with one or more elements of a computer- implemented interactive surgical system, for example. ^[0065] Referring to FIG.1, a computer-implemented interactive surgical system 100 may include one or more surgical systems 102 and a cloud-based system (e.g., the cloud 104 that may include a remote server 113 coupled to a storage device 105). Each surgical system 102 may include at least one surgical hub 106 in communication with the cloud 104 that may include a remote server 113. ^[0066] One or more simulation devices 103, 111 may be in communication with and/or integrated as part of the computer-implemented interactive surgical system 100. For example, the simulation device 103 may be an element of the one or more surgical systems 102. For example, the simulation device 103 may be in communication with one or more surgical hubs 106. For example, the simulation device 111 may be in communication with the computer-implemented interactive surgical system 100 via the cloud 104. ^[0067] In one example, as illustrated in FIG.1, the surgical system 102 includes a visualization system 108, a robotic system 110, and a handheld intelligent surgical instrument 112, which are configured to communicate with one another and/or the hub 106. In some aspects, a surgical system 102 may include an M number of hubs 106, an N number of visualization systems 108, an O number of robotic systems 110, and a P number of handheld intelligent surgical instruments 112, where M, N, O, and P may be integers greater than or equal to one. ^[0068] In various aspects, the visualization system 108 may include one or more imaging sensors, one or more image-processing units, one or more storage arrays, and one or more displays that are strategically arranged with respect to the sterile field, as illustrated in FIG.2. In one aspect, the visualization system 108 may include an interface for HL7, PACS, and EMR. Various components of the visualization system 108 are described under the heading “Advanced Imaging Acquisition Module” in U.S. Patent Application Publication No. US 2019-0200844 A1 (U.S. Patent Application No.16/209,385), titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety. ^[0069] As illustrated in FIG.2, a primary display 119 is positioned in the sterile field to be visible to an operator at the operating table 114. In addition, a visualization tower 111 is positioned outside the sterile field. The visualization tower 111 may include a first non- sterile display 107 and a second non-sterile display 109, which face away from each other. The visualization system 108, guided by the hub 106, is configured to utilize the displays 107, 109, and 119 to coordinate information flow to operators inside and outside the sterile field. For example, the hub 106 may cause the visualization system 108 to display a snapshot of a surgical site, as recorded by an imaging device 124, on a non-sterile display 107 or 109, while maintaining a live feed of the surgical site on the primary display 119. The snapshot on the non-sterile display 107 or 109 can permit a non-sterile operator to perform a diagnostic step relevant to the surgical procedure, for example. ^[0070] In one aspect, the hub 106 may also be configured to route a diagnostic input or feedback entered by a non-sterile operator at the visualization tower 111 to the primary display 119 within the sterile field, where it can be viewed by a sterile operator at the operating table. In one example, the input can be in the form of a modification to the snapshot displayed on the non-sterile display 107 or 109, which can be routed to the primary display 119 by the hub 106. ^[0071] Referring to FIG.2, a surgical instrument 112 is being used in the surgical procedure as part of the surgical system 102. The hub 106 may also be configured to coordinate information flow to a display of the surgical instrument 112. For example, in U.S. Patent Application Publication No. US 2019-0200844 A1 (U.S. Patent Application No. 16/209,385), titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety. A diagnostic input or feedback entered by a non- sterile operator at the visualization tower 111 can be routed by the hub 106 to the surgical instrument display 115 within the sterile field, where it can be viewed by the operator of the surgical instrument 112. Example surgical instruments that are suitable for use with the surgical system 102 are described under the heading “Surgical Instrument Hardware” and in U.S. Patent Application Publication No. US 2019-0200844 A1 (U.S. Patent Application No.16/209,385), titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety, for example. [0072] FIG.2 depicts an example of a surgical system 102 being used to perform a surgical procedure on a patient who is lying down on an operating table 114 in a surgical operating room 116. A robotic system 110 may be used in the surgical procedure as a part of the surgical system 102. The robotic system 110 may include a surgeon’s console 118, a patient side cart 120 (surgical robot), and a surgical robotic hub 122. The patient side cart 120 can manipulate at least one removably coupled surgical tool 117 through a minimally invasive incision in the body of the patient while the surgeon views the surgical site through the surgeon’s console 118. An image of the surgical site can be obtained by a medical imaging device 124, which can be manipulated by the patient side cart 120 to orient the imaging device 124. The robotic hub 122 can be used to process the images of the surgical site for subsequent display to the surgeon through the surgeon’s console 118. [0073] Other types of robotic systems can be readily adapted for use with the surgical system 102. Various examples of robotic systems and surgical tools that are suitable for use with the present disclosure are described in U.S. Patent Application Publication No. US 2019- 0201137 A1 (U.S. Patent Application No.16/209,407), titled METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety. ^[0074] Various examples of cloud-based analytics that are performed by the cloud 104, and are suitable for use with the present disclosure, are described in U.S. Patent Application Publication No. US 2019-0206569 A1 (U.S. Patent Application No.16/209,403), titled METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety. ^[0075] In various aspects, the imaging device 124 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, Charge-Coupled Device (CCD) sensors and Complementary Metal-Oxide Semiconductor (CMOS) sensors. ^[0076] The optical components of the imaging device 124 may include one or more illumination sources and/or one or more lenses. The one or more illumination sources may be directed to illuminate portions of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and/or surgical instruments. ^[0077] The one or more illumination sources may be configured to radiate electromagnetic energy in the visible spectrum as well as the invisible spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is that portion of the electromagnetic spectrum that is visible to (i.e., can be detected by) the human eye and may be referred to as visible light or simply light. A typical human eye will respond to wavelengths in air that are from about 380 nm to about 750 nm. ^[0078] The invisible spectrum (e.g., the non-luminous spectrum) is that portion of the electromagnetic spectrum that lies below and above the visible spectrum (i.e., wavelengths below about 380 nm and above about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, and they become invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, and they become invisible ultraviolet, x-ray, and gamma ray electromagnetic radiation. [0079] In various aspects, the imaging device 124 is configured for use in a minimally invasive procedure. Examples of imaging devices suitable for use with the present disclosure include, but not limited to, an arthroscope, angioscope, bronchoscope, choledochoscope, colonoscope, cytoscope, duodenoscope, enteroscope, esophagogastro-duodenoscope (gastroscope), endoscope, laryngoscope, nasopharyngo-neproscope, sigmoidoscope, thoracoscope, and ureteroscope. ^[0080] The imaging device may employ multi-spectrum monitoring to discriminate topography and underlying structures. A multi-spectral image is one that captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or by the use of instruments that are sensitive to particular wavelengths, including light from frequencies beyond the visible light range, e.g., IR and ultraviolet. Spectral imaging can allow extraction of additional information the human eye fails to capture with its receptors for red, green, and blue. The use of multi-spectral imaging is described in greater detail under the heading “Advanced Imaging Acquisition Module” in .S. Patent Application Publication No. US 2019-0200844 A1 (U.S. Patent Application No. 16/209,385), titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety. Multi-spectrum monitoring can be a useful tool in relocating a surgical field after a surgical task is completed to perform one or more of the previously described tests on the treated tissue. It is axiomatic that strict sterilization of the operating room and surgical equipment is required during any surgery. The strict hygiene and sterilization conditions required in a “surgical theater,” i.e., an operating or treatment room, necessitate the highest possible sterility of all medical devices and equipment. Part of that sterilization process is the need to sterilize anything that comes in contact with the patient or penetrates the sterile field, including the imaging device 124 and its attachments and components. It will be appreciated that the sterile field may be considered a specified area, such as within a tray or on a sterile towel, that is considered free of microorganisms, or the sterile field may be considered an area, immediately around a patient, who has been prepared for a surgical procedure. The sterile field may include the scrubbed team members, who are properly attired, and all furniture and fixtures in the area. ^[0081] Referring now to FIG.3, a hub 106 is depicted in communication with a visualization system 108, a robotic system 110, and a handheld intelligent surgical instrument 112. The hub 106 includes a hub display 135, an imaging module 138, a generator module 140, a communication module 130, a processor module 132, a storage array 134, and an operating-room mapping module 133. In certain aspects, as illustrated in FIG.3, the hub 106 further includes a smoke evacuation module 126 and/or a suction/irrigation module 128. During a surgical procedure, energy application to tissue, for sealing and/or cutting, is generally associated with smoke evacuation, suction of excess fluid, and/or irrigation of the tissue. Fluid, power, and/or data lines from different sources are often entangled during the surgical procedure. Valuable time can be lost addressing this issue during a surgical procedure. Detangling the lines may necessitate disconnecting the lines from their respective modules, which may require resetting the modules. The hub modular enclosure 136 offers a unified environment for managing the power, data, and fluid lines, which reduces the frequency of entanglement between such lines. Aspects of the present disclosure present a surgical hub for use in a surgical procedure that involves energy application to tissue at a surgical site. The surgical hub includes a hub enclosure and a combo generator module slidably receivable in a docking station of the hub enclosure. The docking station includes data and power contacts. The combo generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component that are housed in a single unit. In one aspect, the combo generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combo generator module to a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and/or particulates generated by the application of therapeutic energy to the tissue, and a fluid line extending from the remote surgical site to the smoke evacuation component. In one aspect, the fluid line is a first fluid line and a second fluid line extends from the remote surgical site to a suction and irrigation module slidably received in the hub enclosure. In one aspect, the hub enclosure comprises a fluid interface. Certain surgical procedures may require the application of more than one energy type to the tissue. One energy type may be more beneficial for cutting the tissue, while another different energy type may be more beneficial for sealing the tissue. For example, a bipolar generator can be used to seal the tissue while an ultrasonic generator can be used to cut the sealed tissue. Aspects of the present disclosure present a solution where a hub modular enclosure 136 is configured to accommodate different generators, and facilitate an interactive communication therebetween. One of the advantages of the hub modular enclosure 136 is enabling the quick removal and/or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in a surgical procedure that involves energy application to tissue. The modular surgical enclosure includes a first energy- generator module, configured to generate a first energy for application to the tissue, and a first docking station comprising a first docking port that includes first data and power contacts, wherein the first energy-generator module is slidably movable into an electrical engagement with the power and data contacts and wherein the first energy-generator module is slidably movable out of the electrical engagement with the first power and data contacts. Further to the above, the modular surgical enclosure also includes a second energy-generator module configured to generate a second energy, different than the first energy, for application to the tissue, and a second docking station comprising a second docking port that includes second data and power contacts, wherein the second energygenerator module is slidably movable into an electrical engagement with the power and data contacts, and wherein the second energy-generator module is slidably movable out of the electrical engagement with the second power and data contacts. In addition, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port, configured to facilitate communication between the first energy-generator module and the second energy-generator module. Referring to FIG. 3, aspects of the present disclosure are presented for a hub modular enclosure 136 that allows the modular integration of a generator module 140, a smoke evacuation module 126, and a suction/irrigation module 128. The hub modular enclosure 136 further facilitates interactive communication between the modules 140, 126, 128. The generator module 140 can be a generator module with integrated monopolar, bipolar, and ultrasonic components supported in a single housing unit slidably insertable into the hub modular enclosure 136. The generator module 140 can be configured to connect to a monopolar device 142, a bipolar device 144, and an ultrasonic device 146. Alternatively, the generator module 140 may comprise a series of monopolar, bipolar, and/or ultrasonic generator modules that interact through the hub modular enclosure 136. The hub modular enclosure 136 can be configured to facilitate the insertion of multiple generators and interactive communication between the generators docked into the hub modular enclosure 136 so that the generators would act as a single generator. [0082] FIG.4 illustrates a surgical data network 201 comprising a modular communication hub 203 configured to connect modular devices located in one or more operating theaters of a healthcare facility, or any room in a healthcare facility specially equipped for surgical operations, to a cloud-based system (e.g., the cloud 204 that may include a remote server 213 coupled to a storage device 205). In one aspect, the modular communication hub 203 comprises a network hub 207 and/or a network switch 209 in communication with a network router. The modular communication hub 203 also can be coupled to a local computer system 210 to provide local computer processing and data manipulation. The surgical data network 201 may be configured as passive, intelligent, or switching. A passive surgical data network serves as a conduit for the data, enabling it to go from one device (or segment) to another and to the cloud computing resources. An intelligent surgical data network includes additional features to enable the traffic passing through the surgical data network to be monitored and to configure each port in the network hub 207 or network switch 209. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port. ^[0083] Modular devices 1a-1n located in the operating theater may be coupled to the modular communication hub 203. The network hub 207 and/or the network switch 209 may be coupled to a network router 211 to connect the devices 1a-1n to the cloud 204 or the local computer system 210. Data associated with the devices 1a-1n may be transferred to cloud- based computers via the router for remote data processing and manipulation. Data associated with the devices 1a-1n may also be transferred to the local computer system 210 for local data processing and manipulation. Modular devices 2a-2m located in the same operating theater also may be coupled to a network switch 209. The network switch 209 may be coupled to the network hub 207 and/or the network router 211 to connect to the devices 2a-2m to the cloud 204. Data associated with the devices 2a-2n may be transferred to the cloud 204 via the network router 211 for data processing and manipulation. Data associated with the devices 2a-2m may also be transferred to the local computer system 210 for local data processing and manipulation. ^[0084] It will be appreciated that the surgical data network 201 may be expanded by interconnecting multiple network hubs 207 and/or multiple network switches 209 with multiple network routers 211. The modular communication hub 203 may be contained in a modular control tower configured to receive multiple devices 1a-1n/2a-2m. The local computer system 210 also may be contained in a modular control tower. The modular communication hub 203 is connected to a display 212 to display images obtained by some of the devices 1a-1n/2a-2m, for example during surgical procedures. In various aspects, the devices 1a-1n/2a-2m may include, for example, various modules such as an imaging module 138 coupled to an endoscope, a generator module 140 coupled to an energy-based surgical device, a smoke evacuation module 126, a suction/irrigation module 128, a communication module 130, a processor module 132, a storage array 134, a surgical device coupled to a display, and/or a non-contact sensor module, among other modular devices that may be connected to the modular communication hub 203 of the surgical data network 201. ^[0085] In one aspect, the surgical data network 201 may comprise a combination of network hub(s), network switch(es), and network router(s) connecting the devices 1a-1n/2a-2m to the cloud. Any one of or all of the devices 1a-1n/2a-2m coupled to the network hub or network switch may collect data in real time and transfer the data to cloud computers for data processing and manipulation. It will be appreciated that cloud computing relies on sharing computing resources rather than having local servers or personal devices to handle software applications. The word “cloud” may be used as a metaphor for “the Internet,” although the term is not limited as such. Accordingly, the term “cloud computing” may be used herein to refer to “a type of Internet-based computing,” where different services— such as servers, storage, and applications—are delivered to the modular communication hub 203 and/or computer system 210 located in the surgical theater (e.g., a fixed, mobile, temporary, or field operating room or space) and to devices connected to the modular communication hub 203 and/or computer system 210 through the Internet. The cloud infrastructure may be maintained by a cloud service provider. In this context, the cloud service provider may be the entity that coordinates the usage and control of the devices 1a- 1n/2a-2m located in one or more operating theaters. The cloud computing services can perform a large number of calculations based on the data gathered by smart surgical instruments, robots, and other computerized devices located in the operating theater. The hub hardware enables multiple devices or connections to be connected to a computer that communicates with the cloud computing resources and storage. [0086] Applying cloud computer data processing techniques on the data collected by the devices 1a-1n/2a-2m, the surgical data network can provide improved surgical outcomes, reduced costs, and improved patient satisfaction. At least some of the devices 1a-1n/2a-2m may be employed to view tissue states to assess leaks or perfusion of sealed tissue after a tissue sealing and cutting procedure. At least some of the devices 1a-1n/2a-2m may be employed to identify pathology, such as the effects of diseases, using the cloud-based computing to examine data including images of samples of body tissue for diagnostic purposes. This may include localization and margin confirmation of tissue and phenotypes. At least some of the devices 1a-1n/2a-2m may be employed to identify anatomical structures of the body using a variety of sensors integrated with imaging devices and techniques such as overlaying images captured by multiple imaging devices. The data gathered by the devices 1a-1n/2a-2m, including image data, may be transferred to the cloud 204 or the local computer system 210 or both for data processing and manipulation including image processing and manipulation. The data may be analyzed to improve surgical procedure outcomes by determining if further treatment, such as the application of endoscopic intervention, emerging technologies, a targeted radiation, targeted intervention, and precise robotics to tissue-specific sites and conditions, may be pursued. Such data analysis may further employ outcome analytics processing, and using standardized approaches may provide beneficial feedback to either confirm surgical treatments and the behavior of the surgeon or suggest modifications to surgical treatments and the behavior of the surgeon. ^[0087] The operating theater devices 1a-1n may be connected to the modular communication hub 203 over a wired channel or a wireless channel depending on the configuration of the devices 1a-1n to a network hub. The network hub 207 may be implemented, in one aspect, as a local network broadcast device that works on the physical layer of the Open System Interconnection (OSI) model. The network hub may provide connectivity to the devices 1a-1n located in the same operating theater network. The network hub 207 may collect data in the form of packets and sends them to the router in half duplex mode. The network hub 207 may not store any media access control/Internet Protocol (MAC/IP) to transfer the device data. Only one of the devices 1a-1n can send data at a time through the network hub 207. The network hub 207 may not have routing tables or intelligence regarding where to send information and broadcasts all network data across each connection and to a remote server 213 (FIG.4) over the cloud 204. The network hub 207 can detect basic network errors such as collisions, but having all information broadcast to multiple ports can be a security risk and cause bottlenecks. [0088] The operating theater devices 2a-2m may be connected to a network switch 209 over a wired channel or a wireless channel. The network switch 209 works in the data link layer of the OSI model. The network switch 209 may be a multicast device for connecting the devices 2a-2m located in the same operating theater to the network. The network switch 209 may send data in the form of frames to the network router 211 and works in full duplex mode. Multiple devices 2a-2m can send data at the same time through the network switch 209. The network switch 209 stores and uses MAC addresses of the devices 2a-2m to transfer data. ^[0089] The network hub 207 and/or the network switch 209 may be coupled to the network router 211 for connection to the cloud 204. The network router 211 works in the network layer of the OSI model. The network router 211 creates a route for transmitting data packets received from the network hub 207 and/or network switch 211 to cloud-based computer resources for further processing and manipulation of the data collected by any one of or all the devices 1a-1n/2a-2m. The network router 211 may be employed to connect two or more different networks located in different locations, such as, for example, different operating theaters of the same healthcare facility or different networks located in different operating theaters of different healthcare facilities. The network router 211 may send data in the form of packets to the cloud 204 and works in full duplex mode. Multiple devices can send data at the same time. The network router 211 uses IP addresses to transfer data. [0090] In an example, the network hub 207 may be implemented as a USB hub, which allows multiple USB devices to be connected to a host computer. The USB hub may expand a single USB port into several tiers so that there are more ports available to connect devices to the host system computer. The network hub 207 may include wired or wireless capabilities to receive information over a wired channel or a wireless channel. In one aspect, a wireless USB short-range, high-bandwidth wireless radio communication protocol may be employed for communication between the devices 1a-1n and devices 2a-2m located in the operating theater. [0091] In examples, the operating theater devices 1a-1n/2a-2m may communicate to the modular communication hub 203 via Bluetooth wireless technology standard for exchanging data over short distances (using short-wavelength UHF radio waves in the ISM band from 2.4 to 2.485 GHz) from fixed and mobile devices and building personal area networks (PANs). The operating theater devices 1a-1n/2a-2m may communicate to the modular communication hub 203 via a number of wireless or wired communication standards or protocols, including but not limited to Wi-Fi (IEEE 802.11 family), WiMAX (IEEE 802.16 family), IEEE 802.20, new radio (NR), long-term evolution (LTE), and Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, and Ethernet derivatives thereof, as well as any other wireless and wired protocols that are designated as 3G, 4G, 5G, and beyond. The computing module may include a plurality of communication modules. For instance, a first communication module may be dedicated to shorter-range wireless communications such as Wi-Fi and Bluetooth, and a second communication module may be dedicated to longer-range wireless communications such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, and others. ^[0092] The modular communication hub 203 may serve as a central connection for one or all of the operating theater devices 1a-1n/2a-2m and may handle a data type known as frames. Frames may carry the data generated by the devices 1a-1n/2a-2m. When a frame is received by the modular communication hub 203, it is amplified and transmitted to the network router 211, which transfers the data to the cloud computing resources by using a number of wireless or wired communication standards or protocols, as described herein. ^[0093] The modular communication hub 203 can be used as a standalone device or be connected to compatible network hubs and network switches to form a larger network. The modular communication hub 203 can be generally easy to install, configure, and maintain, making it a good option for networking the operating theater devices 1a-1n/2a-2m. ^[0094] FIG.5 illustrates a computer-implemented interactive surgical system 200. The computer- implemented interactive surgical system 200 is similar in many respects to the computer- implemented interactive surgical system 100. For example, the computer-implemented interactive surgical system 200 includes one or more surgical systems 202, which are similar in many respects to the surgical systems 102. Each surgical system 202 includes at least one surgical hub 206 in communication with a cloud 204 that may include a remote server 213. In one aspect, the computer-implemented interactive surgical system 200 comprises a modular control tower 236 connected to multiple operating theater devices such as, for example, intelligent surgical instruments, robots, and other computerized devices located in the operating theater. As shown in FIG.6, the modular control tower 236 comprises a modular communication hub 203 coupled to a computer system 210. [0095] As illustrated in the example of FIG.5, the modular control tower 236 may be coupled to an imaging module 238 that may be coupled to an endoscope 239, a generator module 240 that may be coupled to an energy device 241, a smoke evacuator module 226, a suction/irrigation module 228, a communication module 230, a processor module 232, a storage array 234, a smart device/instrument 235 optionally coupled to a display 237, and a non-contact sensor module 242. The operating theater devices may be coupled to cloud computing resources and data storage via the modular control tower 236. A robot hub 222 also may be connected to the modular control tower 236 and to the cloud computing resources. The devices/instruments 235, visualization systems 208, among others, may be coupled to the modular control tower 236 via wired or wireless communication standards or protocols, as described herein. The modular control tower 236 may be coupled to a hub display 215 (e.g., monitor, screen) to display and overlay images received from the imaging module, device/instrument display, and/or other visualization systems 208. The hub display also may display data received from devices connected to the modular control tower in conjunction with images and overlaid images. ^[0096] FIG.6 illustrates a surgical hub 206 comprising a plurality of modules coupled to the modular control tower 236. The modular control tower 236 may comprise a modular communication hub 203, e.g., a network connectivity device, and a computer system 210 to provide local processing, visualization, and imaging, for example. As shown in FIG.6, the modular communication hub 203 may be connected in a tiered configuration to expand the number of modules (e.g., devices) that may be connected to the modular communication hub 203 and transfer data associated with the modules to the computer system 210, cloud computing resources, or both. As shown in FIG.6, each of the network hubs/switches in the modular communication hub 203 may include three downstream ports and one upstream port. The upstream network hub/switch may be connected to a processor to provide a communication connection to the cloud computing resources and a local display 217. Communication to the cloud 204 may be made either through a wired or a wireless communication channel. ^[0097] The surgical hub 206 may employ a non-contact sensor module 242 to measure the dimensions of the operating theater and generate a map of the surgical theater using either ultrasonic or laser-type non-contact measurement devices. An ultrasound-based non- contact sensor module may scan the operating theater by transmitting a burst of ultrasound and receiving the echo when it bounces off the perimeter walls of an operating theater as described under the heading “Surgical Hub Spatial Awareness Within an Operating Room” in .S. Patent Application Publication No. US 2019-0200844 A1 (U.S. Patent Application No.16/209,385), titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY, filed December 4, 2018, which is herein incorporated by reference in its entirety, in which the sensor module is configured to determine the size of the operating theater and to adjust Bluetooth-pairing distance limits. A laser-based non- contact sensor module may scan the operating theater by transmitting laser light pulses, receiving laser light pulses that bounce off the perimeter walls of the operating theater, and comparing the phase of the transmitted pulse to the received pulse to determine the size of the operating theater and to adjust Bluetooth pairing distance limits, for example. ^[0098] The computer system 210 may comprise a processor 244 and a network interface 245. The processor 244 can be coupled to a communication module 247, storage 248, memory 249, non-volatile memory 250, and input/output interface 251 via a system bus. The system bus can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 9-bit bus, Industrial Standard Architecture (ISA), Micro-Charmel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), USB, Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Small Computer Systems Interface (SCSI), or any other proprietary bus. ^[0099] The processor 244 may be any single-core or multicore processor such as those known under the trade name ARM Cortex by Texas Instruments. In one aspect, the processor may be an LM4F230H5QR ARM Cortex-M4F Processor Core, available from Texas Instruments, for example, comprising an on-chip memory of 256 KB single-cycle flash memory, or other non-volatile memory, up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, a 32 KB single-cycle serial random access memory (SRAM), an internal read-only memory (ROM) loaded with StellarisWare® software, a 2 KB electrically erasable programmable read-only memory (EEPROM), and/or one or more pulse width modulation (PWM) modules, one or more quadrature encoder inputs (QEI) analogs, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which are available for the product datasheet. ^[00100] In one aspect, the processor 244 may comprise a safety controller comprising two controller-based families such as TMS570 and RM4x, known under the trade name Hercules ARM Cortex R4, also by Texas Instruments. The safety controller may be configured specifically for IEC 61508 and ISO 26262 safety critical applications, among others, to provide advanced integrated safety features while delivering scalable performance, connectivity, and memory options. ^[00101] The system memory may include volatile memory and non-volatile memory. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer system, such as during start-up, is stored in non-volatile memory. For example, the non-volatile memory can include ROM, programmable ROM (PROM), electrically programmable ROM (EPROM), EEPROM, or flash memory. Volatile memory includes random-access memory (RAM), which acts as external cache memory. Moreover, RAM is available in many forms such as SRAM, dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). ^[00102] The computer system 210 also may include removable/non-removable, volatile/non- volatile computer storage media, such as for example disk storage. The disk storage can include, but is not limited to, devices like a magnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zip drive, LS-60 drive, flash memory card, or memory stick. In addition, the disk storage can include storage media separately or in combination with other storage media including, but not limited to, an optical disc drive such as a compact disc ROM device (CD-ROM), compact disc recordable drive (CD-R Drive), compact disc rewritable drive (CD-RW Drive), or a digital versatile disc ROM drive (DVD-ROM). To facilitate the connection of the disk storage devices to the system bus, a removable or non-removable interface may be employed. ^[00103] It is to be appreciated that the computer system 210 may include software that acts as an intermediary between users and the basic computer resources described in a suitable operating environment. Such software may include an operating system. The operating system, which can be stored on the disk storage, may act to control and allocate resources of the computer system. System applications may take advantage of the management of resources by the operating system through program modules and program data stored either in the system memory or on the disk storage. It is to be appreciated that various components described herein can be implemented with various operating systems or combinations of operating systems. [00104] A user may enter commands or information into the computer system 210 through input device(s) coupled to the I/O interface 251. The input devices may include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processor through the system bus via interface port(s). The interface port(s) include, for example, a serial port, a parallel port, a game port, and a USB. The output device(s) use some of the same types of ports as input device(s). Thus, for example, a USB port may be used to provide input to the computer system and to output information from the computer system to an output device. An output adapter may be provided to illustrate that there can be some output devices like monitors, displays, speakers, and printers, among other output devices that may require special adapters. The output adapters may include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device and the system bus. It should be noted that other devices and/or systems of devices, such as remote computer(s), may provide both input and output capabilities. ^[00105] The computer system 210 can operate in a networked environment using logical connections to one or more remote computers, such as cloud computer(s), or local computers. The remote cloud computer(s) can be a personal computer, server, router, network PC, workstation, microprocessor-based appliance, peer device, or other common network node, and the like, and typically includes many or all of the elements described relative to the computer system. For purposes of brevity, only a memory storage device is illustrated with the remote computer(s). The remote computer(s) may be logically connected to the computer system through a network interface and then physically connected via a communication connection. The network interface may encompass communication networks such as local area networks (LANs) and wide area networks (WANs). LAN technologies may include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE 802.5 and the like. WAN technologies may include, but are not limited to, point-to-point links, circuit-switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet-switching networks, and Digital Subscriber Lines (DSL). ^[00106] In various aspects, the computer system 210 of FIG.6, the imaging module 238 and/or visualization system 208, and/or the processor module 232 of FIGS.5-6, may comprise an image processor, image-processing engine, media processor, or any specialized digital signal processor (DSP) used for the processing of digital images. The image processor may employ parallel computing with single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) technologies to increase speed and efficiency. The digital image-processing engine can perform a range of tasks. The image processor may be a system on a chip with multicore processor architecture. [00107] The communication connection(s) may refer to the hardware/software employed to connect the network interface to the bus. While the communication connection is shown for illustrative clarity inside the computer system, it can also be external to the computer system 210. The hardware/software necessary for connection to the network interface may include, for illustrative purposes only, internal and external technologies such as modems, including regular telephone-grade modems, cable modems, and DSL modems, ISDN adapters, and Ethernet cards. ^[00108] FIG 7. is a block diagram of an example surgical simulator system. The surgical simulator system may include a simulation device 30000. The surgical simulator system may include an application creation device 30002, a human interface device 30004, a surgeon agent device 30006, and/or a surgical data system 30008. ^[00109] The simulation device 30000 may provide core simulation functionality. For example, the loading/running of one or more simulations, the reception and processing of user control information input, the generation and transmission of visual, audible, and/or haptic information output, the collection of simulation operation and activity information, and the primary simulation cycle processing may be performed by the simulation device 30000. [00110] The application creation device 30002 may provide simulation authoring functionality. Individual simulation applications may be stored as application modules 30010 at the simulation device 30000. The application modules 30010 may be created, modified, and/or deleted by the application creation device 30002. The application modules 30010 may include computer readable and/or executable instructions to direct an operation of the simulation device 30000. For example, the application modules 30010 may include any filetype suitable for storing information to run a surgical simulation, for example, simulation scripts, programming code, structure data files such as Extensible Markup Language (XML) files, database files, and the like. [00111] The application creation device 30002 may include a graphical user interface with controls to author application modules 30010. The application creation device 3002 may communicate with the simulation device 30000 to retrieve, modify, and/or load application modules 30010 for simulation operation. For example, the graphical user interface may include interface structures to allow a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution. The application creation device 30002 may be provided as a stand-alone device and/or integrated with one or more other devices of the surgical simulation system, such as integrated with the simulation device 30000 for example. [00112] The human interface device 30004 may include any hardware, software, and/or combination thereof that enables a human user to interact with a simulation provided by the simulation device 30000. The human interface device 30004 may enable a user to provide control input to the simulation device 300000 and/or to receive output information (such as visual, audible, and/or haptic information) from the simulation device 30000. In one example, the human interface device 30004 may include a traditional desktop computer. ^[00113] The human interface device 30004 may include suitable physical equipment. For example, the human interface device 30004 may include physical equipment that mimic physically and/or virtually aspects of a surgical procedure. For example, such equipment may include bench-top units, part-task virtual reality units, high fidelity virtual reality units, high fidelity full-size patient units, suite units, high fidelity full operating room units, full physics virtual reality units, surgical robot console units, and the like. For example, the human interface device 30004 may include devices such as the computer-based simulator interfaces disclosed by Gallager et al, “Simulations for Procedural Training,” Fundamentals of Surgical Simulation, Principles and Practice, Springer (2012). [00114] The human interface device 30004 may include physical equipment that mimics, physically and/or virtually, surgical instruments. For example, the human interface device 30004 may include physical devices that mimic surgical instruments, appliances, and consumables, such as access equipment, such as trocars, hand-access ports, insufflation needles, and guiding sheaths; adjunctive hemostats, such as patches, gelatins, and powders; craniomaxillofacial appliances, like distractors and plates; balloons and inflators; catheters, like diagnostic catheters, access catheters, vascular catheters, and therapeutic catheters; energy sealing and dissecting devices, like tissue sealers, shears, blades, and forceps; orthopedic equipment, like reduction wires, compression screws, plates, implants, drills, burrs, rods, and connectors; ligation instruments, like open and endoscopic clip appliers; microwave ablation equipment; ancillary endoscopic instruments, like drains, sutures, ligature, needle holders, retrievers, and suture clips; surgical stapling equipment, like open staplers, endoscopic staplers, cutter staplers, powered staplers, circular staplers, vascular staplers, linear staplers, staple cartridges, and staple line reinforcement applicators; wound closure materials, like suture, adhesives, needles, and knotless tissue control devices; imaging devices, like minimally invasive imaging devices; and the like. For example, the human interface device 30004 may include virtual reality handheld controllers, that when operated with a virtual reality headset, mimics the surgical instruments, appliances, and consumables, such as those disclosed above. ^[00115] The human interface device 30004 may include a display that communicates visual representations of the simulation to the user. The human interface device 30004 may include a computer display. The human interface device 30004 may include a virtual reality headset display. For example, the virtual reality headset display may be used display the surgical environment, such as that disclosed in FIG.2, herein. A user with such a virtual reality headset display may view and/or interact with any of the elements in the surgical operating room 116, including, for example, the patient, the robotic system 110, the surgeon’s console 118, the surgical robotic hub 122, one or more surgical tools 117, the imaging device 124, the patient side cart 120, one or more displays 119, 107, 109, and the like. ^[00116] The human interface device 30006 may present visual information that represents the point of the view of the surgeon. The human interface device 30006 may present visual information from a simulated imaging device, such as an arthroscope, angioscope, bronchoscope, choledochoscope, colonoscope, cytoscope, duodenoscope, enteroscope, esophagogastro-duodenoscope (gastroscope), endoscope, laryngoscope, nasopharyngo- neproscope, sigmoidoscope, thoracoscope, ureteroscope, and their related instruments, controls, and the like. The human interface device 30006 may present visual information from a simulated supplemental intra-operative imaging equipment, like computed tomography (CT) units, magnetic resonance imaging (MRI) units, image-guided surgery units, intra-operative ultrasound units; fluoroscopy units, and the like. Such point-of-view visual information, surgical imaging information, and supplemental intra-operative imaging information may be displayed in any combination to the user suitable for the simulation’s operation. For example, such information may be presented to the user as a single full- screen view, a tiled window view, a picture-in-a-picture view, or registered to a simulated display unit in a virtual reality view. [00117] The human interface device 30004 may include a physical and/or virtual reality surgical robot surgeon console. For example, an example surgeon-console-like human interface device 30004 may include a display, such as a stereo vision display and control inputs, including hand-held manipulators, foot pedals, and the like. For example, the surgeon- console-like human interface device 30004 may include an interface of the surgeon’s console 118, disclosed herein. The human interface device 30004 may enable voice controls via, for example, a microphone and speech recognition functionality. The human interface device 30004 may provide audible feedback via, for example, a speaker. The human interface device 30004 may provide haptic feedback via, for example, vibration, force feedback, air vortex rings, and ultrasound techniques. ^[00118] As implemented, the human interface device 30004 may be provided as a stand-alone device and/or integrated with one or more other devices of the surgical simulation system, such as integrated with the simulation device 30000 for example. The simulation device 30000 may include an interface module 30012 to communicate with the human interface device 30004. In an example, human interface device 30004 may be integrated into one or more elements of the computer-implemented interactive surgical system 100. For example, the human interface device 30004 may be integrated into the computer system 210. For example, the human interface device 30004 may be integrated into the hub 106. For example, the human interface device 30004 may be integrated into the visualization system 108. The interface module 30012 may communicate with the one or more elements of the computer-implemented interactive surgical system 100 via the surgical data system interface module 30014 for example. ^[00119] In an embodiment, more than one human interface device 30004 may concurrently engage with the simulation device 30000. For example, a multi-person simulation application ^[00120] The surgeon agent device 30006 may include any hardware and/or software suitable for providing a computer-based control and response to the input and output of the simulation device 30000. The surgeon agent device 30006 may include a computer process that mimics human input to the simulation device 30000. For example, the surgeon agent device 30006 may be able to record and register control inputs, such as basic instrument manipulation. The surgeon agent device 30006 may include a computer process that can access a input/output application programming interface (API) of the simulation device 30000. For example, the API may reveal one or more input/output functions that may be directed according to the surgeon agent device 3006. The functions may include granular manipulation and physics-based input/output functions, such as functions that directly control the location and movement of instruments. The functions may include less granular surgical-activity-based input/output functions, such as a ligation activity, a suturing activity, a stapling activity, and the like. The functions may include less granular surgical task and/or stage-based input/output functions, such as surgical access function, organ mobilization function, and the like. Each function may include parameter consistent with its level of granularity. The parameters may provide specific details to direct the operation of the function within the simulation. The surgeon agent 30006 may include functionality for generating and operating multiple simulation runs. For example, a user may wish to estimate the duration of various suturing techniques. A surgeon agent device 30006 may be used to script the simulation of any number of different techniques, each of which can be run via the simulation device, and the metrics collected by the simulation device may be used to estimate the difference in durations. [00121] The surgeon agent device 30006 may be provided as a stand along device and/or integrated with one or more other devices of the surgical simulation system, such as integrated with the simulation device 30000 for example. The simulation device 30000 may include an interface module 30012 to communicate with the surgeon agent device 30006. For example, the surgeon agent device 30006 may be integrated as a module of the simulation device 30000. For example, the surgeon agent device 30006 may be integrated into an application module 30010 of the simulation device. [00122] The surgical data system 30008 may include any hardware and/or software suitable for providing external, structured surgical information and functionality to the simulation device 30000. The surgical data system 30008 may include the structure and/or functions described in connection with FIGs.1-6 herein. For example, the surgical data system 30008 may include one or more elements of a computer-implemented interactive surgical system 100. The surgical data system 30008 may include, for example, a surgical hub 106. For example, the simulation device 30000 include a surgical data system interface module 30014 that enables communication with the surgical hub 106 via the surgical hub’s communication module 130. The surgical data system 30008 may include, for example, on or more surgical data repositories. For example, the surgical data system 30008 may include the computer system 210 located in the surgical theater. For example, the surgical data system 30008 may include the remote server 213 in the cloud 204. ^[00123] A surgical data system 30008, such as the surgical hub 106 for example, may provide data to the simulation device 30000 and/or the application creation device 30002. For example, the data may include any surgical data collected and/or generated by the surgical hub 106. Also for example, the simulation device 30000 may receive similar data directly from any of the networked devices disclosed in FIGs 1-6. Such data may include information about a live surgical procedure, for example. Such data may include information about a past surgical procedure. Such data may include information about future, scheduled surgical procedures. [00124] Information about the surgical procedures may include information about the patient, the staff, the procedure as planned, the procedure as experienced, and post-operative activity including patient outcomes. For example, the information received and used by the simulation device may include patient records, patient imaging, models of patient anatomy, patient lab results, patient medical history, and the like. For example, the information received and used by the simulation device may include a staff manifest for a procedure, details about the past procedures of the specific staff members, staff metrics, experience, recent scheduling and workload, and historical surgical activity, such instrument use statistics, procedure duration, and the like. For example, the information received and used by the simulation device may include procedure plans, equipment and inventory information, pull-lists, checklists, procedure plan analysis and recommendations. For example, the information received and used by the simulation device may include any data collected or generated during a live procedure, such as procedure progress, milestones, patient information, vitals, operating theater setup, staff movement, imaging, instrument use, surgical technique, such as that captured by video, recorded manually, and/or inferred from smart-instrument reporting for example, duration, abnormal event reporting, and the like. Any data captured during a live procedure may also be stored and made available as a past procedure. For example, the information received and used by the simulation device may include post-operative records, patient recovery information, and patient outcome information, post-operative diagnostic information, such as labs, imaging, etc., ^[00125] The simulation device 30000 may include any computer or processing platform suitable for executing one or more simulations. The simulation may include a computer-modeled environment of a surgical procedure. For example, the simulation may include a model of a patient’s anatomy and/or physiology. For example, the simulation may include a model of the actions and/or instruments of one or more healthcare professionals, such as the actions of a surgeon, nurse, other doctor, technician, or the like. ^[00126] The simulation device 30000 may include one or more functional modules. Each module may include hardware, software, or a combination thereof that enable functionality of the module. One or more modules, operating in concert, may represent a computer framework on which a simulation of a medical procedure may be executed. The modules may include hardware elements, such as a computer processing unit, a graphics processing unit, a field- programmable gate array (FPGAs), communications hardware, memory, and the like. The modules may include software elements that when executed by a processor cause the module to perform certain functions. [00127] The simulation device may include a core simulation module 30016, a simulation applications module directory 30018, the interface module 30012, an object properties module 30020, a physics module 30022, a physiology model 30024, a texture model 30026, a 3D graphics pipeline 30028, the surgical data system interface module 30014, a metrics extraction module 30030, a session storage and management module 30032, for example. The simulation device may include an operating system module 30034. [00128] The core simulation model 30016 may provide primary simulation functionality of the simulation device 30000. For example, the core simulation module 30016 may include code for initializing a simulation, for communicating and interacting with other modules of the simulation device 30000, and/or for managing architectural level simulation parameters. For example, the core simulation module 30016 may include a master event clock to provide time alignment and/or coordination of the operation of the modules of the simulation device 30000. For example, the core simulation module 30016 may establish the overall simulation frame rate. ^[00129] The core simulation module 30016 may include core for providing a master simulation cycle. The core simulation module 30016 may run one or more iteration of the master simulation cycle. Each iteration of the master simulation cycle may represent an individual time slice for simulation. In an example, the core simulation module 30016 may run the master simulation cycle according to the flow disclosed in FIG.10. ^[00130] The simulation applications module directory 30018 may manage the storing, retrieving, and/or linking of the one or more application modules 30010. Each application module 30010 may include code that directs the application-level aspects of a simulation. For example, an application module 30010 may include the functionality to provide a simulation of specific anatomy, of specific teaching scope, of specific equipment, or the like. In an example simulation device 30000, an application-specific simulation device 30000 may operate with a single application module 30010 with or without a simulation application module directory 30010. The simulation application module directory 30018 may operate based on interaction with the core simulation module 30016 and/or the application creation device 30002. ^[00131] The interface module 30012 may provide functionality for interacting with the human interface device 30004 and/or the surgeon agent device 30006. For example, the interface module 30012 may include one or more drivers to translate information received from human interface device 30004 into software commands, interrupts, and the like. For example, the interface module 30012 may include a software application programming interface (API) for interacting with the surgeon agent 30006. The interface module 30012 may provide information received from the human interface module 30004 and/or the surgeon agent device 30006 to other modules of the simulation device 30000. For example, the interface module 30012 may receive a control input from the human interface module 30004 and/or the surgeon agent device 30006 that represents movement of a simulated instrument and provide that information to one or more other modules of the simulation device 30000 so the movement may be represented in the simulation. ^[00132] The interface module 30012 may provide the API to enable a more granular interaction with the surgeon agent device 30006. For example, the API may provide an interface to receive simulation parameters and simulation settings from the surgeon agent device 30006. Such simulation parameters and/or simulation settings may be like those input by the user via the application creation device 30002, for example. For example, the surgeon agent device 30006 may be enabled to run one or more computer-controlled simulation trials through the simulation device 30000. For example, the surgeon agent device 30006 may be enabled to run multiple simulations, each with alternative interactions. ^[00133] The interface module 30012 may send output from the simulation device 30000 to the human interface device 30004 and/or the surgeon agent device 30006. For example, the output may include visual output, haptic output, audio output, and/or structured data output, or the like. [00134] The object properties module 30020 may provide functionality for managing the simulated appearance and/or behavior of objects within in the simulation. Simulated objects may include objects such as anatomy, instrument, equipment, consumables, fluids, and the like. An object’s appearance may be managed by object properties, such as location, dimensions, scale, material, parent/child relationships, vertices, faces, interactivity, transparency, trajectory, rendering properties, textures, surface reflectivity, motion blur, layering, and the like. An object’s behavior may be managed by object properties, such as physics properties, mass, motion, collision behavior, elasticity, viscosity, surface tension, rigging constraints, hardness, shear strength, tearing behavior, grain, and the like. [00135] The physics module 30022 may provide functionality to calculate the physical responses and/or interaction of objects within the simulation. The physical module may determine such responses and/or interactions according to classical mechanics, fluid mechanics, soft body dynamics, Brownian motion, collision detection, cloth behavior, finite element analysis, and the like. The physics module 30022 may include commercial and/or open- source modules, such as PhysX (TM), Simulation Open Framework Architecture (SOFA) (TM), VisSim (TM), and the like. [00136] The physiology module 30024 may provide functionality to calculate physiological responses and/or interactions of the anatomy and/or patient as a whole in the simulation. The physiology module 30024 may provide physiological models for key organs and/or systems. The physiological models may include mathematical models, statistical models, or the like. For example, the physiology module 30024 may module the patient’s vitals to calculate their response and/or interaction to activities performed during the simulation. For example, a circulatory model may calculate blood pressure in response to a severed vessel in the simulation. The physiology module 30024 and the physics module 30022 may coordinate with each other during the calculation of each state of the simulation. For example, blood pressure calculated by the circulatory model may be used to determine fluid dynamics properties calculated by the physics module 30022 and managed by the object properties module 30020. ^[00137] The texture module 30026 may provide functionality to determine, retrieve, and/or generate the appropriate surfacing of objects within the simulation. The texture module 30026 may include one or more surfacing modalities that may be controlled according to parameters of the simulation. The surfacing modalities may include artificially generated surfaces, surfaces based on real-world imagery, and combinations thereof. The texture module 30026 may coordinate operation with the physics module 30022 to provide accurate haptic feedback to the user via the user interface module 30012. ^[00138] The 3D graphics pipeline 30028 may provide functionality for visual rendering of the simulation environment. The 3D graphics pipeline 30028 may receive object properties and a perspective. The 3D graphics pipeline 30028 may determine the visualization to be presented to the user that represents the objects in 3D space as viewed from the camera perspective. The 3D graphics pipeline 30028 may determine geometric aspects of the rendering, such as lighting, projection, clipping, view transformation, and the like. The 3D graphics pipeline 30028 may determine rasterization aspects of the rendering, such as fragmentation, pixel shading, vertex shading, geometry sharing, texture filtering, and the like. The 3D graphics pipeline 30028 may coordinate with the texture module 30026 to provide accurate visual feedback to the user via the interface module 30012. [00139] The surgical data system interface module 30014 may provide interactive connectivity to one or more elements of computer-implemented interactive surgical system 100. Information from the one or more elements of the computer-implemented interactive surgical system 100 may be communicated via the surgical data system interface module 30014 to one more modules of the simulation device 30000 to influence operation of a simulation. For example, the surgical data system interface module 30014 may receive information about a surgical procedure an communicate it to a corresponding application module 30010. For example, the surgical data system interface module 30014 may receive information about an instrument and communicate it to the object properties module 30020. For example, the surgical data system interface module 30014 may receive information about a patient and communicate to the physiology module. For example, the surgical data system interface module 30014 may receive information about tissue imaging and communicate it to the texture module 30026. ^[00140] Information from the modules of the simulation device 30000 may be provided, via the surgical data system interface 30014, to one or more elements of the computer- implemented interactive surgical system 100. For example, one or more elements of the computer-implemented interactive surgical system 100 may receive statistics related to a simulated procedure plan from the metrics extraction module 30030. For example, one or more elements of the computer-implemented interactive surgical system 100 may receive replayed simulation visualization procedure plan from the session storage and management module 30032. For example, the surgical data system interface module 30014 may provide a communications pathway between the interface module 30012 and one or more elements of the computer-implemented interactive surgical system 100. For example, a surgeon during a live surgical procedure may access simulation information and/or operate a simulation from the operating theater. For example, a surgeon may use the surgeon console 118 to access and/or interact with a simulation that corresponds to the live surgical procedure. [00141] The metrics extraction module 30014 may provide recording functionality of various parameters related to the operation of the simulation. For example, the metrics extraction module 30014 may record metrics related to the simulation as a whole, such as duration, number of activities, number of movements, complexity of movements, staff employed, staff movement, equipment and/or instrument changes, etc. For example, the metrics extraction module 30014 may record metrics related to a particular aspect of the simulation, such as simulated patient vitals, complications, collisions, bleeding, etc. The metrics extraction module 30014 may maintain a master log of metric-related events during a simulation. For metrics extraction module 30014 may record metric-related events according to a configuration from the application module 30010 employed for the simulation. ^[00142] The session storage and management module 30032 may provide management functionality of the main simulation run-record. For example, the session storage and management module 30032 may store the information to enable a simulation to be rerun, viewed, and/or analyzed in its entirety. The session storage and management module 30032 may store the information about each input, simulation state, and output, such as the input, simulation state, and output disclosed with regard to FIG.10. The session storage and management module 30032 may enable a previous simulation to be recalled, copied, and initialized with new user input. To illustrate, a surgeon in training may recall a simulation run by an experienced surgeon, pause the simulation at a critical step, and attempt that step on her own. The session storage and management module 30032 may provide overlay functionality between various runs of a particular simulation. Such overlays may highlight similarities and differences and may enhance training. ^[00143] The operating system module 30034 may manage the hardware and/or software resources for the simulation device 30000. The operating system module 30034 may provide common computing system-level services for the other modules of simulation device 30000. For example, the operating system module 30034 may provide hardware input and output handling, memory allocation, hardware interrupt handling, software interrupt handling, thread processing, single task handling, multi-task handling, and the like. The simulation device 30000 may be a real-time computing device. The operating system module 30034 may include a real-time operating system. For example, the operating system module 30034 may be driven by the events and frame rate established by the core simulation module 30016. ^[00144] FIG.8 is a block diagram of an example surgical simulator system. The simulation device 30000 is depicted with an example hardware architecture. For example, the simulation device 30000 may include a processor 30034, a memory 30036, a storage 30038, a display adapter 30040, a manipulation interface adapter 30042, a surgical data system adapter 30044, and/or a network adapter 30046. One or more of the processor 30034, a memory 30036, a storage 30038, a display adapter 30040, a manipulation interface adapter 30042, a surgical data system adapter 30044, and/or a network adapter 30046 may be used to enable operation of the modules of the simulation device 30000 disclosed herein. ^[00145] The processor 30046 may include computer processing unit, graphics processing unit, any suitable microcontroller, microprocessor, field programmable gate array (FPGA), application specific integrated circuit (ASIC), or the like, and/or any combination thereof that is suitable for processing and delivering a 3D simulated environment for interaction with a computer agent and/or human user. In one example, the processor 30046 may include one or more processing units. The processor 30046 may be a processor of any suitable depth to perform the digital processing requirements disclosed herein. For example, the processor 30046 a 32-bit processor, a 64-bit processor, a 128-bit processor, or the like. [00146] Such processors may comprise, or may be in communication with, media, for example computer-readable media, that may store instructions that, when executed by the processor, can cause the processor to perform the steps described herein as carried out, or assisted, by a processor. Embodiments of computer-readable media may comprise, but are not limited to, an electronic, optical, magnetic, or other storage device capable of providing a processor, such as the processor in a web server, with computer-readable instructions. Other examples of media comprise, but are not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configured processor, all optical media, all magnetic tape or other magnetic media, or any other medium from which a computer processor can read. The processor, and the processing, described may be in one or more structures, and may be dispersed through one or more structures. The processor may comprise code for carrying out one or more of the methods (or parts of methods) described herein. ^[00147] The memory 30036 may include any component or collection of components suitable for storing data. For example, the memory 30036 may include volatile memory and/or nonvolatile memory. The memory 30036 may include random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), (electrically erasable programmable read-only memory) EEPROM, flash memory, or the like. ^[00148] The storage 30038 may include any component or collection of components suitable for storing large quantities of data. For example, storage 30038 may include hard disk drives (HDD), solid state drives (SSD), network-attached storage (NAS), or the like. The storage 30038 may include a database structure and/or a database management system (DBMS). ^[00149] The display adapter 30040 may include any component or collection of components suitable for outputting the visual representation of a 3D simulation environment. For example, the display adapter 30040 may include a graphics card, a display card, a graphics adapter, or the like. The display adapter 30040 may be used to generates a feed of output images to a display device, such as a display of the human interface device 30004. The display adapter 30040 may include a graphics processing unit (GPU). The display adapter 30040 may include hardware to render a graphics pipeline, for example. The manipulation interface adapter 30042 may include any component or collection of components suitable for receiving manipulation information from the human interface device and/or outputting feedback information to the human interface device. For example, the manipulation interface adapter 30042 may receive motion tracking information from a virtual reality headset and in turn, manipulate the view being displayed to the user. For example, the manipulation interface adapter 30042 may receive control input indicative of a user manipulating a surgical instrument and, in turn, output haptic feedback to the user’s handheld device. For example, the manipulation interface adapter 30042 may receive control information from a traditional desktop keyboard and mouse. The manipulation interface adapter may include input/output hardware such as serial input/output ports, parallel input/output ports, universal asynchronous receiver transmitters (UARTs), discrete logic input/output pins, analog-to-digital converters, digital-to-analog converters, universal serial bus (USB) ports, USB-C ports, FireWire ports, High Performance Parallel Interface (HIPPI), Thunderbolt port, Yapbus, Ethernet, Gigabit Ethernet, and/or any other suitable peripheral interface technology. ^[00150] The surgical data system adapter 30044 may include any component or collection of components suitable for communicating with the surgical data system 30008. The surgical data system adapter 30044 may include communications hardware to establish a physical channel between the simulation device 30000 and the surgical data system 30008. For example, the surgical data system adapter 30044 may include a communication port such as, a USB port, USB-C ports, FireWire ports, HIPPI port, Thunderbolt port, Yapbus port, Ethernet port, Gigabit Ethernet port, and/or any other suitable peripheral interface. The surgical data system adapter 30044 may include hardware, software, and/or a combination thereof to establish a logical channel between the simulation device 30000 and the surgical data system 30008 over the network adapter 30046 and the network 30048. [00151] The network adapter 30046 may include any component or collection of components suitable for communication over a network, such as network 30048 for example. The network adapter 30046 may enable communication over networks such as local area networks (LANs), wide area networks (WANs), and/or mobile networks. LAN technologies may include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE 802.5, Wi-Fi/IEEE 802.11, and the like. WAN technologies may include, but are not limited to, point-to-point links, circuit-switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet-switching networks, and Digital Subscriber Lines (DSL). The mobile networks may include communication links based on one or more of the following mobile communication protocols: GSM/GPRS/EDGE (2G), UMTS/HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR) or 5G, etc. [00152] In an embodiment, the network adapter 30046 may include a wireless network adapter, such as a 5G network adapter. Such a 5G network adapter 30046 may use a 5G New Radio (NR) transceiver to provide enhanced mobile broadband (eMBB) with ultra-reliable and low latency communications (URLLC). Such a 5G network adapter 30046 may use wireless bands, such as higher wireless bands like the 3.5Ghz-7Ghz and/or the 24 GHz-48 GHz bands. The network 30048 servicing such a 5G network adapter 30046 may include a public wireless network, a semi-private (e.g., network slicing-based) network, and/or a fully private wireless network. ^[00153] FIG.9 is a block diagram depicting an example surgical simulator human user interface device 30004. The human user interface device 30004 is depicted with an example hardware architecture. For example, the human user interface device 30004 may include a processor 30050, a memory 30052, a display subsystem 30054, and/or a manipulation subsystem 30056. [00154] The processor 30050 may include computer processing unit, graphics processing unit, any suitable microcontroller, microprocessor, field programmable gate array (FPGA), application specific integrated circuit (ASIC), or the like, and/or any combination thereof that is suitable for handling the processing associated with displaying visual information received from the simulation device 30000, processing manipulation information for sending to the simulation device, processing feedback information received from the simulation device 30000, and the like. The processor 30050 may include a microcontroller to interface with one or more local sensors to sense control manipulation from the user and/or to interface with one or more local actuators to provide feedback from the user. [00155] Such processors may comprise, or may be in communication with, media, for example computer-readable media, that may store instructions that, when executed by the processor, can cause the processor to perform the steps described herein as carried out, or assisted, by a processor. Embodiments of computer-readable media may comprise, but are not limited to, an electronic, optical, magnetic, or other storage device capable of providing a processor, such as the processor in a web server, with computer-readable instructions. Other examples of media comprise, but are not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configured processor, all optical media, all magnetic tape or other magnetic media, or any other medium from which a computer processor can read. The processor, and the processing, described may be in one or more structures, and may be dispersed through one or more structures. The processor may comprise code for carrying out one or more of the methods (or parts of methods) described herein. ^[00156] The memory 30036 may include any component or collection of components suitable for storing data. For example, the memory 30036 may include volatile memory and/or nonvolatile memory. The memory 30036 may include random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), (electrically erasable programmable read-only memory) EEPROM, flash memory, or the like. ^[00157] The display subsystem 30054 may include any component or collection of components suitable for displaying visual representations of a 3D simulation from the simulation device 30000 to a user. The display subsystem may include display hardware such as a monitor, a digital projector, a smart phone, a digital headset, a virtual reality headset, a stereoscopic display, a robotic surgery surgeon’s console display, a surgical display unit, a surgical microscope, and the like. [00158] The manipulation subsystem 30056 may include any component or collection of components suitable for collecting manipulation controls from the user to send to the simulation device 30000 and/or providing feedback information, received from the simulation device 30000, to the user. Manipulation from the user may include any interface with sensors that engage with the user, for example, engaging to indicate a user’s intent in the simulation. For example, the interfaces may include keyboards, mice, joysticks, physical equipment that mimics the size, shape, and operation of actual surgical instruments, virtual reality hand-held controllers, smart gloves, motion sensing systems (such as hand tracking systems, for example), a robotic surgery surgeon’s console manipulators and/or controls, a physical unit that mimics the size, shape, and operation of an actual robotic surgery surgeon’s console manipulators and/or controls, and the like. For example, the interface may include a point of view sensor, such as an accelerometer, in a headset to indicate a user’s point of view within the simulation. [00159] Feedback from the simulation device 30000 may include any interface with an actuator that provides sensory input to the user. For example, the feedback may include haptic feedback, force feedback, temperature feedback, moisture feedback, audio feedback, olfactory feedback, and the like. For example, a force feedback and/or haptic actuator in the manipulator of a robotic surgery surgeon’s console may be used to simulate the feedback the user would feel if operating such a manipulator in a live procedure. For example, a force feedback and/or haptic actuator in a user device that mimics the size, shape, and operation of actual surgical stapler may be used to simulate the feedback the user would feel if operating such a device on live tissue, including force feedback when engaging the tissue and firing the stapler for example. ^[00160] FIG.10 is a flow chart of an example surgical simulator operation. At 30058, a simulation application may be loaded. For example, the core simulation module 30016 may cause data associated with a particular application module 30010 to be loaded into memory 30036. The loaded data may include instructions for the processor 30034 to operate a particular simulation. The loaded data may include a procedural plan for the simulation. For example, the procedural plan may be structured as disclosed herein, for example with regard to FIGs.11A-B. The loaded data may include an initial state for the simulation. ^[00161] At 30060, the simulation output may be determined and/or sent. For example, the simulation output may be determined and/or sent by the simulation device 30000. Here, the core simulation module 30016 may reference a current state of the simulation (e.g., an initial state and/or a subsequent state). The core simulation module 30016 may engage one or more other modules to process the current state for output. For example, the core simulation module may engage any of the object properties module 30020, the texture module 30026, the application module 30010, the 3D graphics pipeline 30028, the interface module 30012, and/or the surgical data system interface module 30014 to process the current simulation state into information for output. Information related to the output maybe processed and/or stored by the metrics extraction module 30030 and/or the session storage and management module 30032, for example. ^[00162] In a human-operated simulation session, for example, output information may be sent via the display adapter 30040 and/or the manipulation interface adapter 30042 to the display subsystem 30054 and/or the manipulation subsystem 30056 of the human interface device 30004. In a computer-controlled simulation session, for example, output information may be sent via the interface module 30012 to a surgeon agent 30006. Also for example, in a computer controlled simulation session, output information may be sent (e.g., processed locally) at an application module 30010. In a session accessed via the surgical data system 30008, for example, output information may be sent by the surgical data system interface module 30014 via the surgical data system adapter 30044 and/or the network adapter 30046. [00163] At 30062, simulation input may be received and/or processed. For example, simulation input may be received and/or processed by the simulation device 30000. Here, the core simulation module may engage with the interface device, the surgical data system interface module, and/or the application module 30010 to receive control input. Information related to the input maybe processed and/or stored by the metrics extraction module 30030 and/or the session storage and management module 30032, for example. ^[00164] In a human-operated simulation session, for example, input information may be sent from a manipulation subsystem 30056 of the human interface device 30004 and received via the manipulation interface adapter 30042. In a computer-controlled simulation session, for example, input information may be sent from a surgeon agent 30006 and received via the interface module 30012. Also for example, in a computer controlled simulation session, input information may be received (e.g., processed locally) at an application module 30010. In a session accessed via the surgical data system 30008, for example, input information may be received via the surgical data system adapter 30044 and/or the network adapter 30046 and initially handled by the surgical data system interface module 30014. [00165] At 30064, a subsequent simulation state may be determined. For example, a subsequent simulation state may be determined from the current simulation state and/or the any received input. The core simulation module 30016 may engage one or more of the other modules of the simulation device 30000 to determine the subsequent simulation state. For example, the code simulation module 30016 may engage the application module, the object properties module, the physics module, the physiology module, and the like. The subsequent simulation state may be determined by operation of the processor 30034. Information related to the input maybe processed and/or stored by the metrics extraction module 30030 and/or the session storage and management module 30032, for example. [00166] At this stage, the process may loop to receiving input at 30060. Each iteration of this flow may represent a corresponding time cycle in the simulation. The framerate of the simulation may be set to a level suitable for the goal of the simulation and the processing capabilities of the surgical simulation device 30000. Lower framerates may enable processing that achieves a live human interaction simulation. Higher framerates may enable greater simulation fidelity. For example, when operating computer-controlled simulations, with a surgeon agent 30006 for example, a higher framerate may be used, even if the higher framerate causes the processing time of the simulation to exceed the real- world time it is simulating. [00167] FIGs.11A-B illustrate example surgical procedural plan data structures for use with a computer-implemented interactive surgical system and/or a surgical simulator. A surgical procedure plan may include information that outlines the staff, equipment, technique, and steps that may be used to perform a surgical procedure. For example, the procedure plan may include a staff manifest indicating what roles and/or what specific health care professionals are to be involved in the procedure. The procedure plan may include a listing of equipment, such as durable surgical equipment, imaging equipment, instruments, consumables, etc. that may be used during the procedure. For example, the procedure plan may include a pick list for a surgical technician to use to assemble the appropriate tools and materials for the surgeon and the surgery when prepping the operating theater. The procedure plan may include information about the procedure’s expected technique. For example, the procedure plans for the same surgical goal may include different methods of access, mobilization, inspection, tissue joining, wound closure, and the like. ^[00168] The procedure plan may reflect a surgeon’s professional judgement with regard to an individual case. The procedure plan may reflect a surgeon’s preference for and/or experience with a particular technique. The procedure plan may map specific surgical tasks to roles and equipment. The procedure plan may provide an expected timeline for the procedure. [00169] The procedure plan may include one or more decision points and/or branches. Such decision points and/or branches may provide surgical alternatives that are available for particular aspects of the procedure, where selection of one of the alternatives may be based on information from the surgery itself. For example, the choice of one or more alternatives may be selected based on the particular planes of the particular patient’s anatomy, and the surgeon may select an alternative based on her assessment of the patient’s tissue during the live surgery. [00170] The procedural plan may include one or more contingencies. These may include information about unlikely but possible situations that may arise during the live surgery. The contingencies may include one or more surgical tasks that may be employed if the situation does occur. The contingencies may be used to ensure that adequate equipment, staff, and/or consumables are at the ready during the procedure. ^[00171] The procedure plan may be recorded in one or more data structures. A procedure plan data structure may be used to record data about a future live surgery, about a completed live surgery, about a future simulated surgery, about a completed simulated surgery, and the like. A procedure plan data structure for live surgeries may be used by the computer- implemented interactive surgical system 100. For example, the procedure plan data structure for live surgeries may be used by surgical hub 106 to enhance situational awareness and/or the operational aspects of the computer-implemented interactive surgical system 100. The procedure plan data structure for live surgeries may be used by the surgical hub 106 to record discrete elements of the live surgery for structured analysis. ^[00172] A procedure plan data structure may be used by a simulation device 30000. For example, the procedure plan data structure may be used by the simulation device 30000 to establish a setting and/or one or more objectives for a simulation session. For example, the procedure plan data structure may be used by the simulation device 30000 to record the discrete elements of the simulated surgery for structured analysis. ^[00173] The procedure plan data structure may include any structure suitable for capturing data elements related to the procedure. For example, the procedure plan may be recorded in a tree-like data structure, such as the one shown in FIG.11A, for example. Here, the root of the tree structure represents the core procedure data 30066. The core procedure data 30066 may include information about the procedure as a whole, such as procedure name, procedure code, patient name, date, time, and the like. For a simulation, the core procedure data 30066 may include information about simulation device, such as device ID, software version, user, the simulation run settings, such as frame rate, resolution, connected user interface devices, and the like. [00174] The procedure data may include leaves of the tree structure. The first level of leaves may include data regarding the main aspects of the procedure plan, such as the procedure setup 30068, one or more procedure stages 30070, one or more contingencies 30072, and the data regarding the result of the procedure 30074. [00175] The setup data 30068 may include information related to the preparations and/or initial state of the procedure. For example, the setup data 30068 may include elements such as staff manifest, staff roles and/or staff IDs, operating room ID, an equipment list, a room layout, an initial surgical table position, a listing of instruments and/or consumables on prepared in the surgical field, any initial settings associated with equipment, pre-surgical imaging, patient record, etc. For a simulation, the setup data 30068 may include information related the simulated environment, such as a record of the simulated anatomy, a record of the simulated physiology, pre-surgical imaging, and the like. ^[00176] The stage data 30070 may include data elements related to a major milestone of the procedure. For example, a stage of a procedure may include a milestone such as establishing access. The stage data 30070 may include information related to the staff, equipment, technique, and steps that may be used to perform the particular stage of the procedure. The stage data 30070 may include a stage ID. ^[00177] The stage may be further detailed by one or more sub-leaves, such as one or more surgical tasks 30076. The surgical task may represent a discrete surgical step within a given stage. For example, within the stage of access, placing a trocar may be a surgical task. The surgical task data 30076 may include a task ID. The surgical task data 30076 may include information related to the particular task, such as the staff and/or surgeon performing the task, the equipment to be used, the particular technique being applied, the patient vitals at the time the task is being performed, other environment information, and the list. Each task may be further detailed with goal data 30078, data related to an anatomy-instrument interaction 30080, and result data 30082. The goal data 30078 may include information indicative of the relative success of the task performance. The goal data 30078 may include information about expected task duration, acceptable performance specificity, efficiency modality, avoidance of complications, and the like. The result data 30082 may include information related to one or more goals. The result data 30082 may record the surgical performance (e.g., live and/or simulated) relative to the goals. ^[00178] The task data 30076 may include one or more elements of anatomy-instrument interaction data 30080. The anatomy-instrument interaction data 30080 may represent a granular indication of surgical performance. The anatomy-instrument interaction data 30080 may represent the one or more specific activities used to perform the surgical task. The anatomy-instrument interaction data 30080 may represent the observable behavior of the surgeon. ^[00179] In an example, the anatomy-instrument interaction data 30080 may include the specific positions, forces, angles, and the like being applied to the anatomy by the surgeon. For example in a live surgery, data recorded from smart instruments by the surgical hub 106 may be captured as anatomy-instrument interaction data 30080. For example, a smart surgical stapler in cooperation with other elements of the computer-implemented interactive surgical system 100 may record stapler position, angle, tip forces, jaw forces, staple cartridge type, closing pressure, firing rate, and the like. In a simulated surgery, similar data elements may be captured. [00180] The contingency data 30072 may indicate any complications that may be relevant to the procedure. Each contingency data 30072 may include one or more task data elements 30084 that address the appropriate response to the particular complication. The contingency data 30072 may indicate deviations from an original procedure plan. Also for example, contingency data may be cross-referenced to one or more tasks 30078 and/or anatomy-instrument interactions 30080. For example, if a certain performance in an anatomy-instrument interactions 30080 could lead to a complication, the nature of that performance and a cross-reference to the contingency may include in the result data 30082 associated with that anatomy-instrument interactions 30080. [00181] The result data 30074 may be indicative of the result of the procedure. Here overall metrics of the surgical performance may be stored, notes, actual and/or simulated patient recovery information, and/or patient outcomes. For example, the result data 30074 may include efficiency information, cost information, surgical duration, workload metrics, percentage of planned consumables used, and the like. ^[00182] FIG.11B illustrates a procedural plan data structure with the above disclosed elements, which further establishes structure of alternative steps for completing a particular procedure, task, or activity. As shown, the procedure represented by the procedure data 30086 may include two alternative setups, each indicated by respective setup data—a first setup data 30088, 30090 and a second setup data 30092. The first setup data 30088, 30090 may include two alternative tasks 30094, 30096. The second setup data 30092 may include one task 30098. In this illustration, the procedure represented by procedure data 30086 may be accomplished in three different ways. First via first setup 30088 and the first task 30094. Second via the first setup 30090 and the second task 30096. And third via the second setup 30092 and its corresponding task 30098. ^[00183] Each path of the tree structure may represent a particular set of alternative ways to perform the procedure. Such a structure may be useful to aid the creation of a particular procedure plan for a particular live and/or simulated surgery. Such a structure may be useful to simulate many possible alternatives of a procedure to assess the differences in results. ^[00184] FIG.12 is a block diagram of an example machine learning framework. Machine learning is a branch of artificial intelligence that seeks to build computer systems that may learn from data without human intervention. These techniques may rely on the creation of analytical models that may be trained to recognize patterns within a dataset, such as a data collection. These models may be deployed to apply these patterns to data, such as biomarkers, to improve performance without further guidance. For example, such a model may include the machine learning module 33500. [00185] Machine learning may be supervised (e.g., supervised learning). A supervised learning algorithm may create a mathematical model from training a dataset (e.g., training data). The training data may consist of a set of training examples. A training example may include one or more inputs and one or more labeled outputs. The labeled output(s) may serve as supervisory feedback. In a mathematical model, a training example may be represented by an array or vector, sometimes called a feature vector. The training data may be represented by row(s) of feature vectors, constituting a matrix. Through iterative optimization of an objective function (e.g., cost function), a supervised learning algorithm may learn a function (e.g., a prediction function) that may be used to predict the output associated with one or more new inputs. A suitably trained prediction function may determine the output for one or more inputs that may not have been a part of the training data. Example algorithms may include linear regression, logistic regression, and neutral network. Example problems solvable by supervised learning algorithms may include classification, regression problems, and the like. [00186] Machine learning may be unsupervised (e.g., unsupervised learning). An unsupervised learning algorithm may train on a dataset that may contain inputs and may find a structure in the data. The structure in the data may be similar to a grouping or clustering of data points. As such, the algorithm may learn from training data that may not have been labeled. Instead of responding to supervisory feedback, an unsupervised learning algorithm may identify commonalities in training data and may react based on the presence or absence of such commonalities in each train example. Example algorithms may include Apriori algorithm, K-Means, K-Nearest Neighbors (KNN), K-Medians, and the like. Example problems solvable by unsupervised learning algorithms may include clustering problems, anomaly/outlier detection problems, and the like ^[00187] Machine learning may include reinforcement learning, which may be an area of machine learning that may be concerned with how software agents may take actions in an environment to maximize a notion of cumulative reward. Reinforcement learning algorithms may not assume knowledge of an exact mathematical model of the environment (e.g., represented by Markov decision process (MDP)) and may be used when exact models may not be feasible. Reinforcement learning algorithms may be used in autonomous vehicles or in learning to play a game against a human opponent. ^[00188] Machine learning may be a part of a technology platform called cognitive computing (CC), which may constitute various disciplines such as computer science and cognitive science. CC systems may be capable of learning at scale, reasoning with purpose, and interacting with humans naturally. By means of self-teaching algorithms that may use data mining, visual recognition, and/or natural language processing, a CC system may be capable of solving problems and optimizing human processes. ^[00189] In an example framework, a simulation device 33502 may enable the training 33504 of the machine learning model 33500. The simulation device 33502 may represent a complex system that receives setup information, such as simulated procedure and/or surgical task, anatomy, physiology, equipment, instruments, and the like. The simulation device 33502 may represent a complex system that receives activity information, such as instrument manipulation, simulated tasks and/or simulated surgical activity primitives, and the like. The simulation device 33502 may represent a complex system that receives setup information and activity information, and in response, outputs one or more results, such as final procedure state, a performance score, an efficiency score, a simulated patient outcome, and the like. For example, the result may include a particular state, value, metric, or the like. For example, the result may include a function, such as a weighed function, of a particular state, value, metric, or the like. The datasets represented by the combinations of respective setup information, activity information, and results associated with the simulation device 33502 may be used to train 33504 the machine learning model 33500. As disclosed herein, the training 33504 may be supervised learning, unsupervised learning, reinforcement learning, cognitive computing, and the like. ^[00190] The output of machine learning’s training process may be a model for predicting outcome(s) on a new dataset. For example, a linear regression learning algorithm may be a cost function that may minimize the prediction errors of a linear prediction function during the training process by adjusting the coefficients and constants of the linear prediction function. When a minimal may be reached, the linear prediction function with adjusted coefficients may be deemed trained and constitute the model the training process has produced. For example, a neural network (NN) algorithm (e.g., multilayer perceptrons (MLP)) for classification may include a hypothesis function represented by a network of layers of nodes that are assigned with biases and interconnected with weight connections. The hypothesis function may be a non-linear function (e.g., a highly non-linear function) that may include linear functions and logistic functions nested together with the outermost layer consisting of one or more logistic functions. The NN algorithm may include a cost function to minimize classification errors by adjusting the biases and weights through a process of feedforward propagation and backward propagation. When a global minimum may be reached, the optimized hypothesis function with its layers of adjusted biases and weights may be deemed trained and constitute the model the training process has produced. [00191] Data collection may be performed for machine learning as a first stage of the machine learning lifecycle. Data collection may include steps such as identifying various data sources, collecting data from the data sources, integrating the data, and the like. For example, for training a machine learning model for predicting surgical complications and/or post-surgical recovery rates, data sources containing pre-surgical data, such as a patient’s medical conditions and biomarker measurement data, may be identified. Such data sources may be a patient’s electronical medical records (EMR), a computing system storing the patient’s pre-surgical biomarker measurement data, and/or other like datastores. The data from such data sources may be retrieved and stored in a central location for further processing in the machine learning lifecycle. The data from such data sources may be linked (e.g., logically linked) and may be accessed as if they were centrally stored. Surgical data and/or post-surgical data may be similarly identified, collected. Further, the collected data may be integrated. In examples, a patient’s pre-surgical medical record data, pre- surgical biomarker measurement data, pre-surgical data, surgical data, and/or post-surgical may be combined into a record for the patient. The record for the patient may be an EMR. ^[00192] Data preparation may be performed for machine learning as another stage of the machine learning lifecycle. Data preparation may include data preprocessing steps such as data formatting, data cleaning, and data sampling. For example, the collected data may not be in a data format suitable for training a model. In an example, a patient’s integrated data record of pre-surgical EMR record data and biomarker measurement data, surgical data, and post- surgical data may be in a rational database. Such data record may be converted to a flat file format for model training. In an example, the patient’s pre-surgical EMR data may include medical data in text format, such as the patient’s diagnoses of emphysema, pre-operative treatment (e.g., chemotherapy, radiation, blood thinner). Such data may be mapped to numeric values for model training. For example, the patient’s integrated data record may include personal identifier information or other information that may identifier a patient such as an age, an employer, a body mass index (BMI), demographic information, and the like. Such identifying data may be removed before model training. For example, identifying data may be removed for privacy reasons. As another example, data may be removed because there may be more data available than may be used for model training. In such case, a subset of the available data may be randomly sampled and selected for model training and the remainder may be discarded. ^[00193] Data preparation may include data transforming procedures (e.g., after preprocessing), such as scaling and aggregation. For example, the preprocessed data may include data values in a mixture of scales. These values may be scaled up or down, for example, to be between 0 and 1 for model training. For example, the preprocessed data may include data values that carry more meaning when aggregated. In an example, there may be multiple prior colorectal procedures a patient has had. The total count of prior colorectal procedures may be more meaningful for training a model to predict surgical complications due to adhesions. In such case, the records of prior colorectal procedures may be aggregated into a total count for model training purposes. ^[00194] Model training may be another aspect of the machine learning lifecycle. The model training process as described herein may be dependent on the machine learning algorithm used. A model may be deemed suitably trained after it has been trained, cross validated, and tested. Accordingly, the dataset from the data preparation stage (e.g., an input dataset) may be divided into a training dataset (e.g., 60% of the input dataset), a validation dataset (e.g., 20% of the input dataset), and a test dataset (e.g., 20% of the input dataset). After the model has been trained on the training dataset, the model may be run against the validation dataset to reduce overfitting. If accuracy of the model were to decrease when run against the validation dataset when accuracy of the model has been increasing, this may indicate a problem of overfitting. The test dataset may be used to test the accuracy of the final model to determine whether it is ready for deployment or more training may be required. ^[00195] Model deployment may be another aspect of the machine learning lifecycle. The model may be deployed as a part of a standalone computer program. The model may be deployed as a part of a larger computing system. A model may be deployed with model performance parameters(s). Such performance parameters may monitor the model accuracy as it is used for predicating on a dataset in production. For example, such parameters may keep track of false positives and false positives for a classification model. Such parameters may further store the false positives and false positives for further processing to improve the model’s accuracy. ^[00196] Post-deployment model updates may be another aspect of the machine learning cycle. For example, a deployed model may be updated as false positives and/or false positives are predicted on production data. In an example, for a deployed MLP model for classification, as false positives occur, the deployed MLP model may be updated to increase the probably cutoff for predicting a positive to reduce false positives. In an example, for a deployed MLP model for classification, as false negatives occur, the deployed MLP model may be updated to decrease the probably cutoff for predicting a positive to reduce false negatives. In an example, for a deployed MLP model for classification of surgical complications, as both false positives and false negatives occur, the deployed MLP model may be updated to decrease the probably cutoff for predicting a positive to reduce false negatives because it may be less critical to predict a false positive than a false negative. ^[00197] For example, a deployed model may be updated as more live production data become available as training data. In such case, the deployed model may be further trained, validated, and tested with such additional live production data. In an example, the updated biases and weights of a further-trained MLP model may update the deployed MLP model’s biases and weights. Those skilled in the art recognize that post-deployment model updates may not be a one-time occurrence and may occur as frequently as suitable for improving the deployed model’s accuracy. ^[00198] In the example framework, the machine learning model 33500 may be used to model solutions that drive toward a specific result. For example, the model 33500 may be structured and/or trained to receive as input an instance of setup information and to output corresponding activity information. For example, the model 33500 may be structured and/or trained to receive as input an instance of activity information and to output corresponding setup information. In an example, the machine learning model 33500 may be used to model solutions that drive toward a variable result. For example, the model 33500 may be structured and/or trained to receive as input an instance of setup information and a desired result and to output corresponding activity information. For example, the model 33500 may be structured and/or trained to receive as input an instance of activity information and a desired result and to output corresponding setup information. [00199] An example training set may include the results of a simulation of all setups and/or activities that have a result within a predefined range of acceptance. To illustrate, the model may address suture placement. For example, the setup, activity, and results may include elements such as suture pattern, tension, wound healing, and the like. In an example, the simulation may determine tension on colon when reconnected in a low anterior resection (LAR) anastomosis and may include predicted resultant profusion in the results. In an example, the simulation may determine lung volume post expansion following a segmentectomy and/or lobectomy. An example training set may include the results of multiple run automated simulations of a particular surgical procedure with an activity set, and the model may enable to determine outcomes of a preestablished starting set. ^[00200] In an example, the training data may be aggregated to provide a summary view of possible choices and/or variations, such as choices and/or variations within a predefined number of variable steps. The choices and/or variations may include those where the outcome is with a acceptable result-set. Such summary may include a proposed approach path for best outcomes. For example, such summary may propose an approach based on expert review, customer satisfaction surveys, outcomes feedback, and the like. For example, such summary may communicate an adverse event risk score. The adverse event risk score may update progressively up or down as choices evolve. The summary may include an interactive tool to comparing and/or contrasting different approaches. The interactive tool may enable comparing and/or contrasting multiple autorun simulations. [00201] In an example, the simulation device 33502 may be used to create ground truth data sets for machine learning and/or artificial intelligence training. For example, the simulation device 33502 may produce situations to establish bounding parameters and/or edge cases for machine learning and/or artificial intelligence training. For example, the simulation device 33502 may provide output of robotic path guidance. Such guidance may be direct guidance, geofenced, or the like. For example, the simulation device 33502 may provide feedback data. For example, data may be used from a simulation that is driven from database. The data may include information regarding similar cases. The database may include machine learning and/or artificial intelligence model information trained with similar cases. For example, the simulation device 33502 may use a machine learning model to predict medical risks. The prediction may include information of what devices, equipment, and/or consumables may be appropriate in view of the predicted risk, for example. ^[00202] In an example, a machine learning model 33500 may be trained to apply notations, comment, and/or annotations to surgical videos. For example, such notations, comment, and/or annotations may be incorporated into a training set, such that future videos and/or future simulations may be processed according to the model for review. The model may output notations, comment, and/or annotations at appropriate points within the processed video and/or simulation. For example, the machine learning model 33500 may be trained to present complex simulations in real time. ^[00203] In an example, the simulation device 33502 may be used to create supplemental training sets (e.g., training sets to supplement real-world training sets). For example, the the simulation device 33502 may be used to create supplemental training sets that include cases that are rare in the real-world, such as rare anatomic conditions, rare physiology, and/or rare adverse events. Such supplemental training sets may be used to for training 33504 a machine learning model 33500 for example. Similarly, the simulation device 33502 may be used to create patient specific hybrid data sets. For an example, the simulation device 33502 (e.g., via an application module) and/or a machine learning model 33500 may be used to check input parameters of patient specific topics. Such checking may result in changes to the simulated anatomy via modifications to and/or addition of certain objects and/or object properties. For example, the patient specific anatomy an include adding irregularities, such as multiple pulmonary artery branches and/or tumor ingrowths for example, to the simulation. For example, such patient specific modification may be input from remote healthcare inputs and/or pre-operative monitoring. Such patient specific data sets may be included for simulation to generate datasets for machine learning model training, for example. The simulation device 33502 may be used to create supplemental training sets that incorporate rare patient-specific irregularities. ^[00204] In an example, the simulation device 33502 may be used to create training data sets related to procedure platform, such as robotic, laproscopic, open, hybrid, and the like. Here, similar surgical goals may be simulated using various platforms and/or variation within a particular platform. Such setup data and activity data may be used with their corresponding simulated results to train a machine learning model 33500 to recommend approaches for future surgeries, for example. [00205] FIG.13 is a block diagram of an example surgeon agent 33506. The surgeon agent 33506 may include the surgeon agent 30006 disclosed herein with reference to FIG.7, for example. The surgeon agent 33506 may provide computer-based control of a simulated task. Such a surgeon agent 33506 may be used to simulate many variations of the surgical task. Such simulations may be used to generate large datasets for analysis and/or for training a machine learning model, such as machine learning model 33500, for example. Such datasets may be used to analyze and/or model aspects of the surgical task subject to variation. ^[00206] In an example the surgeon agent 33506 may receive procedure plan task data 33508. The procedure plan task data 33508 may include information about a particular surgical task. For example, the procedure plan task data 33508 may include one or more simulation parameters related to the surgical task. Such simulation parameter may provide information on how a particular surgical task is to be handled in simulation. Simulation parameters for the surgical task may include a parameterized representation of the surgical task, each of which include, for example, one or more specific manipulations and one or more reference locations. For example, for a suturing task, an example parameterized representation may include a manipulation, such as an initial suture insertion, with angle, for example, and a reference location, such as a point defined relative to a simulated wound. The procedure plan task data 333508 may be structured as disclosed herein with regard to FIGs.11A-B for example. [00207] At 33510, the surgeon agent 33506 may receive one or more simulation parameters from the procedure plan task data 33508. The surgeon agent 33506 may include logic to translate simulation parameters into one or more control inputs to a simulation. For example, the surgeon agent 33506 may mimic control inputs based on a determination made from the simulation parameters. For example, the surgeon agent 33506 may engage an application programming interface (API) of the simulation to determine control inputs and/or as a method to control the simulation. At 33512, the surgeon agent may translate the one or more simulation parameters into one or more control inputs. ^[00208] At 33514, the surgeon agent 33506 may output the control inputs to a simulation device 33516. The simulation device 33516 may perform a simulation cycle based on such control inputs and in turn send an output to the surgeon agent. The output may include a representation of the simulation state in view of the control inputs. The output may include one or results metrics for example. ^[00209] At 33518, the surgeon agent may receive the simulation output. The surgeon agent may loop to subsequent control inputs, if present. Subsequent control inputs may be translated from additional simulation parameters. Such additional simulation parameters may have been received from the procedure plan task data 33508 and/or a subsequent reception of additional procedure plan task data 33508. Such a surgeon agent may enable computer- based control of a surgical simulation according to a set of parameterized input data. As a building block, one or more instances of surgeon agents may be used with one or more simulations to test many variations of a particular task through simulation. Such building blocks may be combined to test variations of surgical activities of any length, for example. For example, a one or more surgeon agents may be used with one or more elements of corresponding procedure plan task data to enable computer-based control of many variations of entire surgical procedures in simulation. [00210] FIG.14 illustrates an example simulation activity parameterization data set. The parameterization of a surgical activity for simulation via computer-based control may include the operation of a surgical activity parameterization tool 33520. The surgical activity parameterization tool 33520 may be used to generation parameterized surgical activity data 33522. The surgical activity parameterization tool 33520 may be based on a simulation platform. For example, the surgical activity parameterization tool 33520 may be implemented in an application module 30010 of the simulation device 30000 disclosed herein with regard to FIG.7. The surgical activity parameterization tool 33520 may be a stand-alone tool. [00211] The surgical activity parameterization tool 33520 may include a processor configured to perform the step, functions, and/or capabilities disclosed here. The surgical activity parameterization tool 33520 may enable a user to define one or more parameters to represent a surgical activity. For example, the surgical activity parameterization tool 33520 may enable a user to define one or more parameters to represent a surgical activity, such that a computer-based control agent, like the surgeon agent device 30006 as disclosed herein with regard to FIG.7 for example, may perform the surgical activity. Such parameters may represent a parameterized surgical activity data 33520. ^[00212] In an example, parameterized surgical activity data 33520 may include a target object 33522, one or more activity parameters 33524, and/or one or more constraints 33526. The target object 33522 may include information that indicates a particular object, collection of objects, and/or portion of an object within the simulation environment with which the surgical activity is to be done. The activity parameters 33524 may include information that indicates the particular data elements and/or values that define the variations available in performing the activity. The constraints 33526 may include information that indicates any limitations associated with activity, such limitations may be used by the simulation to prevent certain interpolating activity from being performed. ^[00213] To illustrate, parameterized surgical activity data 33520 may be used to define a wound closure activity. In such an example, the target object 33522 may include information indicative of the simulated wound, including for example, the tissue object within which the wound is located, geometric information about the location, size, and shape of the wound, the wound depth, subsurface objects, and the like. For example, parameterized data such as coordinates of a center line 33528, an upper edge 33530, a lower edge, 33532, and the like, may be used to indicate location of the wound in simulation. The activity parameters 33524 may include information that indicates what manipulations are expected relative to the target object 33522. For example, parameterized data such a suture width 33534, suture spacing, suture pattern, suture tension, suture depth, suture type, and the like, may be used to indicate the nature and quality of the suturing activity. For example, other parameterized data such as suture technique, instrument, suture material, needle type, knot type, and the like may be used to define aspects of how the activity is to be performed. The constraints 33526 may include data that indicates any specific limitations to constrain the activity operation. For example, the constraints 33526 may include duration, dexterity, precision, accuracy, collision constraints, applied pressure, adjacent tissue contact, and the like. ^[00214] The parameterize surgical activity 33520 may provide a structure to represent many different ways to perform a particular activity. For example, a number of different sutures that may be performed by applying various combinations and/or permutations of parameter values. ^[00215] The surgical activity parameterization tool 33520 may include a user interface to enable the definition of the data elements and/or their values. In an embodiment, the surgical activity parameterization tool 33520 may provide a “macro” recording capability to record actual user input controls and to populate the parameterized surgical activity data 33520 with elements, values, and/or elements and values based on the recorded user input. The surgical activity parameterization tool 33520 may include a media player-like user interface 33538 to record, playback, edit, and the like, user input controls. [00216] FIG.15 illustrates an example surgical simulation flow for machine learning training. The parameterize surgical activity data structure may enable a computationally effective approach to simulation-based training of a machine learning models. In an example, at 33540, one or more simulation environments may be generated. For example, the one or more simulation environments may be generated by a combination and/or permutation of setup data elements, such as the setup data 30088, 30090, 30092 as disclosed herein with regard to FIG.11B for example. Such generation may result in one or more training task- environment parameter sets 33542. ^[00217] At 33544, for each training task-environment parameter set 33542, possible activity combinations may be generated. For example, such generation may include various combinations and/or permutations of procedural task and/or activity data, such as task data 30094, 30096, 30098 disclosed herein with regard to FIG.11B and/or parameterized surgical activity data 33520 disclosed herein with regard to FIG.14 for example. Such generation may output one or more candidate action sets 33546. In an example, each training task-environment parameter sets 33542 may have one or more candidate action sets 33546 associated with it. In this way, various surgical setups and/or various surgical approaches may be generated for simulation under computer-based control input. The combination of training task-environment parameter sets 33542 and candidate action sets 33546 may represent the input space being simulated. The combination of training task- environment parameter sets 33542 and candidate action sets 33546 may represent a portion of the training data that may be used for a machine learning model. ^[00218] In an example, the training task-environment parameter sets may be received from a surgical data system, such as the surgical data system 30008 disclosed herein with regard to FIG.7 for example. The surgical data system may records real-world environment data from real-world performances of a surgical task. The surgical data system may use such real-world environment data to generates the plurality of training task-environment parameter sets. With this approach, the training task-environment parameter sets may better represent a typical anatomy, physiology, and the like. Similarly, this approach may be used so the training task-environment parameter sets may better represent anatomy, physiology, and the like for a particular patient population, for example. [00219] At 33548, candidate action set 33546 with its corresponding training task-environment parameter set 33542 may be simulated and/or evaluated. For example, the candidate action set 33546 with its corresponding training task-environment parameter set 33542 may be provided as input to a simulation device, such as the simulation device 30000 disclosed herein with regard to FIG.7, for example. For example, the training task-environment parameter set 33542 may be provided as setup information to the simulation. For example, the candidate action set 33546 may be driven by a surgeon agent, such as surgeon agent 30006 for example, to operate the simulation. ^[00220] The evaluation of the performance of each respective candidate action set 33546 and training task-environment parameter set 33542 pair may result in one or more effectiveness metrics 33550. The evaluation may include any performance metric appropriate in view of the desired machine learning training. For example, the evaluation may include a comparison of the final simulation state to a preferred target simulation state. For example, the evaluation may include an assessment of certain performance characteristics, such as procedure duration, procedure movement, equipment changes, instrument usage, consumable usage, and the like. In an example, the effectiveness metric 33550 may include a function, such as a weighted function, of one or more performance results. In an example, the effectiveness metric 33550 may include a vector of one or more performance results. ^[00221] At 33552, each candidate action set 33546 and training task-environment parameter set 33542 pair and its respective effectiveness metric 33550 may serve as training data for training a machine learning model. The training may include any training algorithm appropriate in view of the model objective and the nature of the training data. For example, the training may include supervised learning, unsupervised learning, reinforcement learning, cognitive computing, and the like. [00222] The output of such training may include a model 33554. And at 33556, the model may be employed to determine new candidate action set 33546 and training task-environment parameter set 33542 pairs that are consistent with the training and the effectiveness metrics 33550. In an example, the model 33554 may receive a “new” task-environment parameter set (e.g., a target task-environment parameter set 33558). And the model may output a corresponding “new” action set (e.g., a target action set 33560). In an example, the model may be trained such that it may receive a “new” action set and output a corresponding “new” task-environment parameter set. In this sense, the simulation data may be used to model surgical environments, surgical activities, and their corresponding results to determine new environments (e.g., surgical setups) and/or new activities (e.g., procedures) that aim to achieve (e.g., maximize, minimize, or the like) a result (e.g., a particular performance objective). ^[00223] The determined target action set 33560 and training task-environment parameter set 33558 pair may be simulated at 33562. The simulation may include an evaluation and a corresponding effectiveness metric. Such metric may be used to verify the model 33554, for example. [00224] To illustrate the process, in an example, the procedure may include a suturing procedure. One or more environments may be defined. For example, a predefined surgical site may be warped to establish one or more training task-environment parameter sets. For example, a single predefined surgical site may be used to generate a single training task-environment parameter set for modeling. ^[00225] One or more suturing approached may be defined. For example, a simulation of a needle passing through tissue on either side of a suture site may be developed (e.g., developed using a tool, such as the surgical activity parameterization tool 33520). For example, a computer vision-based approach may be applied to extract relevant geometric parameters from the suture site. For example, the suture site and/or the suturing activity may be parametrically defined. ^[00226] The suture bites may be placed in simulation and evaluated according to effectiveness metric, such as wound interfacial pressure. For example, the effectiveness metric may be used to train a model to maximize the effectiveness metric across a number of variations, such as suture placement, depth, count, force, and the like. In an example, the model may continue learning/automation across the various options and in view of the global suture force to compute the best goal criteria over the entire suture site. The module may be used with a target environment to output a suture approach that maximizes the effectiveness metric for the suture site. [00227] FIG.16 is a flow chart of an example surgical simulation operation. For example, the example surgical simulation operation may include a process to determine a target action set to perform a simulated surgical task in a simulated environment. The simulated environment for the simulation surgical task may be indicated by a target task-environment parameter set. ^[00228] At 33564, a plurality of training task-environment parameter sets may be received. Each training task-environment parameter set may indicate a respective simulated environment within which a simulated surgical task is to be completed. For example, the plurality of training task-environment parameter sets may be received from a surgical data system. The surgical data system may record real-world environment data from real-world performances of the surgical task. The surgical data system may generate the plurality of training task-environment parameter sets from the real-world environment data. For example, each training task-environment parameter set may include anatomical information, physiological information, and/or surgical setup information. [00229] At 33566, for each training task-environment parameter set, a plurality of candidate action sets may be generated. Each candidate action set may include a parameterized surgical activity within the respective simulated environment. For example, the parameterized activity may include a simulated transection parameterized by any of relative anatomical location, instrument selection, and/or instrument application direction. For example, the parameterized activity may include suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. For example, the parameterized activity may include a surgical robotic guidance path. ^[00230] At 335658, Each candidate action set may be executed within a simulated environment indicated by the corresponding training task-environment set. And when the candidate action set is so executed, it may cause performance of the simulated surgical task. Such performance may have an effectiveness that is indicated by a corresponding effectiveness metric. For example, each performance may be evaluated, at 33570, to determine its effectiveness metric. The corresponding effectiveness metrics may be stored. In an example, an output may be generated. The output may be a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. ^[00231] At 33572, a model may be trained. For example, the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric may be used to train the model. [00232] At 33574, a target action set may be determined based on the target task-environment parameter set and the model. The training may be based on a machine learning algorithm, such as a supervised learning algorithm, an unsupervised learning algorithm, a reinforcement learning algorithm, and the like. ^[00233] In an example, an action set (e.g. a candidate action set, a target action set, or an input action set) may refer to a set of parameters which define one or more actions taken during a surgery. For example, an action set may refer to a number of surgical choices taken by an operator. For example, an action set may include specific layout decisions (both of the surgical staff and of the surgical instrumentation). An action set may also refer to the specific equipment used during a surgery. In other words, it may refer to portions of a surgical procedure which have consequences on the surgery, both in duration, surgical outcomes, effectiveness of the procedure, etc. For example, the candidate, target, and input action sets may each employ a compatible and/or same format. A machine-learning model can be trained, for example, on the basis of a candidate action set to help define a relationship between any arbitrary “input” action set. This allows the machine-learning model to be used to find a target action set within a given simulation having a certain effectiveness. ^[00234] Similarly, an environment parameter set (e.g. an environment parameter set, a target environment parameter set, or an input environment parameter set) may refer to a set of parameters which define the overall environment of a surgical simulation. It may refer to the parameters which initialize and define the constraints and physics of a surgical simulation. The target and environment parameter sets, for example, may employ a compatible and/or same format as that of the environment parameter set. A machine- learning model may be trained, for example, on the basis of an environment parameter set to help define a relationship between any arbitrary “input” environment parameter set. This allows the machine-learning model to be used to find a target action set within a given simulation having a certain effectiveness. [00235] In an example, the machine-learning model may be trained using a set of labelled data to train the model in a supervised fashion. For example, the combination of the environment parameter set and the action set may be considered to be the input data (as this refers to a specific set of actions run within a specific simulation environment). This data for training purposes may be “labelled” with its associated effectiveness metric (i.e. the output of the simulation, which relates to how effective the specific action was within that simulation). By feeding this labelled data into the machine-learning model, the model can generate a relationship between different actions/environments and the associated effectiveness. ^[00236] The simulated surgical task and the effectiveness metric may include any surgical activity subject to simulation with an objective goal subject to computer evaluation. For example, the simulated surgical task may include a lower anterior resection anastomosis. And the effectiveness metric may include perfusion resulting from the performance of the simulated surgical task. For example, the simulated surgical task may include any of segmentectomy and lobectomy. And the effectiveness metric may include an indication of output lung volume resulting from the performance of the simulated surgical task. ^[00237] The following non-exhaustive list of embodiments also forms part of the present disclosure. [00238] Embodiment 1. A computer-implemented method for determining a target action set, the method comprising: receiving a plurality of environment parameter sets, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generating, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment, causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; training a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determining a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. ^[00239] It may be understood that an action set (e.g. a candidate action set, a target action set, or an input action set) can refer to a set of parameters which define one or more actions taken during a surgery. In some embodiments, an action set can refer to a number of surgical choices taken by an operator. In some embodiments, an action set can include specific layout decisions (both of the surgical staff and of the surgical instrumentation). It can also refer to the specific equipment used during a surgery. In other words, it may refer to portions of a surgical procedure which have consequences on the surgery, both in duration, surgical outcomes, effectiveness of the procedure, etc. It is to be understood that the candidate, target, and input action sets are all data sets having the same format. That is, a machine-learning model can be trained on the basis of a candidate action set to help define a relationship between any arbitrary “input” action set. This allows the machine- learning model to be used to find a target action set within a given simulation having a certain effectiveness. ^[00240] It may also be understood that an environment parameter set (e.g. an environment parameter set, a target environment parameter set, or an input environment parameter set) can refer to a set of parameters which define the overall environment of a surgical simulation. In other words, it refers to the parameters which initialize and define the constraints and physics of a surgical simulation. It is to be understood that the target and environment parameter sets are all data sets having the same format as that of the environment parameter set. That is, a machine-learning model can be trained on the basis of an environment parameter set to help define a relationship between any arbitrary “input” environment parameter set. This allows the machine-learning model to be used to find a target action set within a given simulation having a certain effectiveness. ^[00241] In one embodiment “training a machine-learning model” may refer to using a set of labelled data to train the model in a supervised fashion. In this case, the combination of the environment parameter set and the action set can be considered to be the input data (as this refers to a specific set of actions run within a specific simulation environment). This data for training purposes can be “labelled” with its associated effectiveness metric (i.e. the output of the simulation, which relates to how effective the specific action was within that simulation). By feeding this labelled data into the machine-learning model, the model can generate a relationship between different actions/environments and the associated effectiveness. ^[00242] For the avoidance of doubt, where the present embodiment refers to an “environment parameter set”, this may be the same parameter set that the application refers to as a “training task-environment parameter set” (i.e. a parameter set that can be used to train the machine learning model). ^[00243] Both the initialization of the surgical simulation (which is defined by the environment parameter set) and the specific choices/actions taken within the simulation (which is defined by the candidate action set) have implications on the overall effectiveness of the surgery (defined by the effectiveness metric). [00244] In one embodiment, “determining a target action set” may refer to selecting an action set from a plurality of action sets within a predetermined simulation (as defined by an environment parameter set). The target action set may be selected based on which target action set (as an input to the machine learning model) produces the best output efficiency metric from the machine learning model. ^[00245] In other words, the present invention uses the output data of a simulation to improve the training of a machine learning model, and then uses the improved trained artificial intelligence to make an improved conclusion regarding surgical actions/choices that can be performed within a certain surgical set-up. ^[00246] One advantage of this is that a machine-learning model (such as a neural network) can be trained with additional data which may not arise commonly during a live surgical procedure. In other words, the simulation can provide data regarding “riskier” surgical choices and their consequences to a machine learning model. The informed intelligence coming from the machine learning model can inform future decisions on surgery, by highlighting which decisions actually have beneficial effects without needing to risk real patients to gather the appropriate data. [00247] The consequence of improving the training of a machine learning model in the way of the present embodiment is that improved surgical outcomes can be achieved via the target action set determined. ^[00248] Embodiment 2. The computer-implemented method of embodiment 1, wherein receiving the plurality of environment parameter sets comprises receiving the plurality of environment parameter sets from a surgical data system that records real-world environment data from real-world performances of the surgical task and that generates the plurality of environment parameter sets from the real-world environment data. [00249] Embodiment 3. The computer-implemented method of any one of embodiments 1 to 2, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[00250] Embodiment 4. The computer-implemented method of any one of embodiments 1 to 3, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[00251] Embodiment 5. The computer-implemented method of embodiment 4, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. ^[00252] Embodiment 6. The computer-implemented method of embodiment 4, wherein the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. ^[00253] Embodiment 7. The computer-implemented method of embodiment 4, wherein the parameterized activity comprises a surgical robotic guidance path. [00254] Embodiment 8. The computer-implemented method of any one of embodiments 1 to 7, wherein the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task. ^[00255] Embodiment 9. The computer-implemented method of any one of embodiments 1 to 8, wherein the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task. [00256] Embodiment 10. The computer-implemented method of any one of embodiments 1 to 9, further comprising generating an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. [00257] Embodiment 11. The computer-implemented method of any one of embodiments 1 to 10, wherein training the machine-learning model comprises training the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. [00258] Embodiment 12. A device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment corresponding to the corresponding environment set causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. ^[00259] Embodiment 13. The device of embodiment 12, wherein the surgical data system is configured to record real-world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data. ^[00260] Embodiment 14. The device of any one of embodiments 12 to 13, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[00261] Embodiment 15. The device of any one of embodiments 12 to 14, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[00262] Embodiment 16. The device of embodiment 15, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. [00263] Embodiment 17. The device of embodiment 15, wherein the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. ^[00264] Embodiment 18. The device of embodiment 15, wherein the parameterized activity comprises a surgical robotic guidance path. ^[00265] Embodiment 19. The device of any one embodiments 12 to 18, wherein the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task. [00266] Embodiment 20. The device of any one embodiments 12 to 18, wherein the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task. ^[00267] Embodiment 21. The device of any one of embodiments 12 to 20, wherein the processor is further configured to generate an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. ^[00268] Embodiment 22. The device of any one of embodiments 12 to 21, wherein the processor is configured to train the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. [00269] Embodiment 23. A device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment causes performance of the simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model. ^[00270] Embodiment 24. The device of embodiment 23, wherein the surgical data system is configured to record real-world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data. ^[00271] Embodiment 25. The device of any one of embodiments 23 to 24, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[00272] Embodiment 26. The device of any one of embodiments 23 to 25, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[00273] Embodiment 27. A device for determining a target action set, the device comprising: a processor configured to: receive a machine-learning model trained with the plurality of environment parameter sets, respective plurality of candidate action sets, and corresponding effectiveness metrics, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric, determine a target action set based on the environment parameter set and the machine-learning model; wherein each environment parameter set corresponds to the respective simulated environment within which the simulated surgical task is to be completed; wherein the plurality of environment parameter sets are from a surgical data system, wherein for each environment parameter set, a plurality of candidate action sets when executed within the simulated environment corresponding to the corresponding environment set causes performance of the simulated surgical task with an effectiveness that is indicated by the corresponding effectiveness metric. ^[00274] Embodiment 28. The device of embodiment 27, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information, and wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment. ^[00275] Embodiment 29. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of embodiments 1 to 11. [00276] The following non-exhaustive list of aspects also forms part of the present disclosure. ^[00277] Aspect 1. A device for determining a target action set for performing a simulated surgical task in a simulated environment indicated by a target task-environment parameter set, the device comprising: a processor configured to: receive a plurality of training task- environment parameter sets from a surgical data system, wherein each training task- environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each training task- environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within a simulated environment indicated by the corresponding training task-environment set causes performance of the simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a model with the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric; and determine a target action set based on the target task-environment parameter set and the model. [00278] Aspect 2. The device of Aspect 1, wherein the processor is further configured to receive the plurality of training task-environment parameter sets from a surgical data system that records real-world environment data from real-world performances of the surgical task and that generates the plurality of training task-environment parameter sets from the real-world environment data. [00279] Aspect 3. The device of Aspect 1, wherein each training task-environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[00280] Aspect 4. The device of Aspect 1, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated surgical environment. ^[00281] Aspect 5. The device of Aspect 4, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. ^[00282] Aspect 6. The device of Aspect 1, further comprising generating an output that is a human- readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. ^[00283] Aspect 7. The device of Aspect 1, wherein training the model comprises training the model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. ^[00284] Aspect 8. A method for determining a target action set to perform a simulated surgical task in a simulated environment indicated by a target task-environment parameter set, the method comprising: receiving a plurality of training task-environment parameter sets, wherein each training task-environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generating, for each training task-environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within a simulated environment indicated by the corresponding training task-environment set causes performance of the simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; training a model with the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric; and determining a target action set based on the target task- environment parameter set and the model. ^[00285] Aspect 9. The method of Aspect 8, wherein receiving the plurality of training task- environment parameter sets comprises receiving the plurality of training task-environment parameter sets from a surgical data system that records real-world environment data from real-world performances of the surgical task and that generates the plurality of training task-environment parameter sets from the real-world environment data. [00286] Aspect 10. The method of Aspect 8, wherein each training task-environment parameter set comprises anatomical information, physiological information, and surgical setup information. ^[00287] Aspect 11. The method of Aspect 8, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated surgical environment. ^[00288] Aspect 12. The method of Aspect 11, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction. [00289] Aspect 13. The method of Aspect 11, wherein the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension. ^[00290] Aspect 14. The method of Aspect 11, wherein the parameterized activity comprises a surgical robotic guidance path. ^[00291] Aspect 15. The method of Aspect 8, wherein the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding training task-environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task. ^[00292] Aspect 16. The method of Aspect 8, wherein the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding training task-environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task. ^[00293] Aspect 17. The method of Aspect 8, further comprising generating an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric. ^[00294] Aspect 18. The method of Aspect 8, wherein training the model comprises training the model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm. ^[00295] Aspect 19. A device for determining a target action set for performing a simulated surgical task in a simulated environment indicated by a target task-environment parameter set, the device comprising: a processor configured to: receive a model trained with the plurality of training task-environment parameter sets, respective plurality of candidate action sets, and corresponding effectiveness metrics, wherein each training task-environment parameter set corresponds to the respective simulated environment within which the simulated surgical task is to be completed; wherein the plurality of training task-environment parameter sets are from a surgical data system, wherein for each training task-environment parameter set, a plurality of candidate action sets when executed within the simulated environment indicated by the corresponding training task-environment set causes performance of the simulated surgical task with an effectiveness that is indicated by the corresponding effectiveness metric; and determine a target action set based on the target task- environment parameter set and the model. [00296] Aspect 20. The device of Aspect 19, wherein each training task-environment parameter set comprises anatomical information, physiological information, and surgical setup information, and wherein each candidate action set comprises a parameterized surgical activity within the respective simulated surgical environment.

Claims

Claims 1. A computer-implemented method for determining a target action set, the method comprising: receiving a plurality of environment parameter sets, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generating, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment, causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; training a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determining a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model.

2. The computer-implemented method of claim 1, wherein receiving the plurality of environment parameter sets comprises receiving the plurality of environment parameter sets from a surgical data system that records real-world environment data from real-world performances of the surgical task and that generates the plurality of environment parameter sets from the real-world environment data.

3. The computer-implemented method of any one of claims 1 to 2, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information.

4. The computer-implemented method of any one of claims 1 to 3, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment.

5. The computer-implemented method of claim 4, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction.

6. The computer-implemented method of claim 4, wherein the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension.

7. The computer-implemented method of claim 4, wherein the parameterized activity comprises a surgical robotic guidance path.

8. The computer-implemented method of any one of claims 1 to 7, wherein the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task.

9. The computer-implemented method of any one of claims 1 to 8, wherein the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task.

10. The computer-implemented method of any one of claims 1 to 9, further comprising generating an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric.

11. The computer-implemented method of any one of claims 1 to 10, wherein training the machine-learning model comprises training the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm.

12. A device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment corresponding to the corresponding environment set causes performance of the respective simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model may comprise a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model.

13. The device of claim 12, wherein the surgical data system is configured to record real- world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data.

14. The device of any one of claims 12 to 13, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information.

15. The device of any one of claims 12 to 14, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment.

16. The device of claim 15, wherein the parameterized activity comprises a simulated transection parameterized by any of relative anatomical location, instrument selection, and instrument application direction.

17. The device of claim 15, wherein the parameterized activity comprises a simulated suturing parameterized by any of suture location, suture pattern, suture number, suture material, and suture tension.

18. The device of claim 15, wherein the parameterized activity comprises a surgical robotic guidance path.

19. The device of any one claims 12 to 18, wherein the simulated surgical task comprises lower anterior resection anastomosis; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of perfusion resulting from the performance of the simulated surgical task.

20. The device of any one claims 12 to 18, wherein the simulated surgical task comprises any of segmentectomy and lobectomy; wherein each candidate action set, when executed within the simulated environment indicated by the corresponding environment set causes performance of the simulated surgical task with effectiveness that is indicated by the corresponding effectiveness metric, wherein the corresponding effectiveness metric is an indication of output lung volume resulting from the performance of the simulated surgical task.

21. The device of any one of claims 12 to 20, wherein the processor is further configured to generate an output that is a human-readable aggregation of the plurality of task environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric.

22. The device of any one of claims 12 to 21, wherein the processor is configured to train the machine-learning model with any of a supervised learning algorithm, an unsupervised learning algorithm, and a reinforcement learning algorithm.

23. A device for determining a target action set, the device comprising: a processor configured to: receive a plurality of environment parameter sets from a surgical data system, wherein each environment parameter set corresponds to a respective simulated environment within which a simulated surgical task is to be completed; generate, for each environment parameter set, a plurality of candidate action sets, wherein each candidate action set, when executed within the respective simulated environment causes performance of the simulated surgical task with an effectiveness that is indicated by a corresponding effectiveness metric; train a machine-learning model with the plurality of environment parameter sets, each respective plurality of candidate action sets, and each corresponding effectiveness metric, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric; and determine a target action set based on a target environment parameter set and a corresponding effectiveness metric which is output by the trained machine-learning model.

24. The device of claim 23, wherein the surgical data system is configured to record real- world environment data from real-world performances of the surgical task and is further configured to generate the plurality of environment parameter sets from the real-world environment data.

25. The device of any one of claims 23 to 24, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information.

26. The device of any one of claims 23 to 25, wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment.

27. A device for determining a target action set, the device comprising: a processor configured to: receive a machine-learning model trained with the plurality of environment parameter sets, respective plurality of candidate action sets, and corresponding effectiveness metrics, such that the trained machine-learning model comprises a functional relationship between a model input and a model output, wherein the model input comprises an input environment parameter set and an input action set, and wherein the output comprises an output effectiveness metric, determine a target action set based on the environment parameter set and the machine-learning model; wherein each environment parameter set corresponds to the respective simulated environment within which the simulated surgical task is to be completed; wherein the plurality of environment parameter sets are from a surgical data system, wherein for each environment parameter set, a plurality of candidate action sets when executed within the simulated environment corresponding to the corresponding environment set causes performance of the simulated surgical task with an effectiveness that is indicated by the corresponding effectiveness metric.

28. The device of claim 27, wherein each environment parameter set comprises anatomical information, physiological information, and surgical setup information, and wherein each candidate action set comprises a parameterized surgical activity within the respective simulated environment.

29. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 11.