[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2023004806A1 - Ai模型的设备部署方法、系统及存储介质 - Google Patents

Ai模型的设备部署方法、系统及存储介质 Download PDF

Info

Publication number
WO2023004806A1
WO2023004806A1 PCT/CN2021/109878 CN2021109878W WO2023004806A1 WO 2023004806 A1 WO2023004806 A1 WO 2023004806A1 CN 2021109878 W CN2021109878 W CN 2021109878W WO 2023004806 A1 WO2023004806 A1 WO 2023004806A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
model
behavior tree
nodes
current
Prior art date
Application number
PCT/CN2021/109878
Other languages
English (en)
French (fr)
Inventor
张洪洋
周冠兴
唐秉超
Original Assignee
西门子股份公司
西门子（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司, 西门子（中国）有限公司 filed Critical 西门子股份公司
Priority to PCT/CN2021/109878 priority Critical patent/WO2023004806A1/zh
Priority to EP21951396.7A priority patent/EP4365731A1/en
Priority to CN202180100662.4A priority patent/CN117677930A/zh
Priority to US18/292,570 priority patent/US20240265275A1/en
Publication of WO2023004806A1 publication Critical patent/WO2023004806A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/10—Requirements analysis; Specification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/34—Graphical or visual programming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/36—Software reuse
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines

Definitions

  • the present invention relates to the field of industrial automation, in particular to an artificial intelligence (AI) model device deployment method, system and computer-readable storage medium.
  • AI artificial intelligence
  • AI Artificial intelligence
  • IT companies and automation companies are trying to introduce AI into industrial automation systems.
  • the trained AI model needs to be deployed to a specific device.
  • Common deployment operations usually include model format conversion, inference application construction, device configuration, communication configuration, etc.
  • the following situations are often encountered: For different devices or different AI application types, there are often large differences in workflows, making it difficult to reuse.
  • each step of the workflow usually needs to be operated in different scenarios, which makes the execution and management process of the workflow more complicated. Therefore, in the field of industrial automation, the development and deployment process of AI applications is complex, diverse, and lacks interoperability, making it difficult for AI to integrate into industrial automation systems.
  • the embodiments of the present invention propose an AI model device deployment method, and on the other hand, an AI model device deployment system and a computer-readable storage medium are proposed to reduce AI model device deployment. the complexity.
  • An AI model device deployment method proposed in an embodiment of the present invention includes: receiving the current behavior tree constructed and configured by the user based on the preset nodes used to construct various behavior trees; the nodes include a start node, a control Nodes and action nodes; wherein, a behavior tree is used to represent a workflow, control nodes are used to implement logic control in the workflow, action nodes are used to implement business operations in the workflow, and the action nodes Including: a reusable independent action node that encapsulates relevant business operations in the AI model deployment process; executes the workflow corresponding to the current behavior tree, builds the predetermined current AI model into an AI application and deploys it to users specified target device.
  • the current behavior tree is: a behavior tree built by the user by adding, connecting and configuring start nodes, required control nodes and required action nodes; or: from the task template library, Select the corresponding fixed behavior tree template according to the required AI application type or the target device type to be deployed, and perform the first configuration based on the fixed behavior tree template and the behavior tree obtained after adjustment or no adjustment; the first configuration Including: determining the current AI model and the target device that needs to be deployed; or: selecting an intelligent behavior tree template based on automatic machine learning from the task template library, and the behavior obtained after the second configuration is performed based on the intelligent behavior tree template tree; the second configuration includes: determining the type of AI application and the target device to be deployed, and the intelligent behavior tree template automatically determines a suitable AI model from an AI model library according to the type of AI application.
  • it further includes: constructing the current behavior tree into a knowledge graph instance based on the grammar of the knowledge graph for storage; the knowledge graph instance includes: nodes respectively representing each node in the behavior tree, and representing Multiple edges for relationships between nodes or node attributes.
  • it further includes: when the user constructs the current behavior tree, recommending control nodes and/or action nodes, error checking, and intelligent mapping of node attributes for the user based on the stored historical knowledge graph instance.
  • each node is composed of an interface part and an implementation part; for the implementation part, each node is a containerized application program; for the interface part, each node is a Graphical elements for placement, connection, and attribute configuration; when each node in the current behavior tree is executed, the input in the attribute configuration of the interface part of the node will be read and transmitted to the implementation part of the node, and After the implementation part completes the corresponding operation, the operation result will be converted into the output in the attribute configuration of the node interface part.
  • it further includes: converting each node added in the current behavior tree into a node instance and adding it to a node instance list for storage or viewing, and the node instance list is in list form or tree form Renders all nodes in the current behavior tree.
  • the action nodes in the current behavior tree include: the following action nodes executed sequentially: model importer, model trainer, and application deployer; or include: the following action nodes executed sequentially: model importer , model trainer, model converter, and application deployer; or include: the following action nodes executed sequentially: model adapter, model trainer, and application deployer; among them, the model importer is used to import the trained AI model; the model trainer is used to retrain the imported training model with new data sources or training parameters; the model converter is used to convert the trained model from the source format to the target format; the model adapter is used based on automatic machine learning according to the current The determined AI application type automatically selects the appropriate AI model from the model library; the application deployer is used to package the AI model into an AI application and deploy it to the specified target device.
  • the device deployment system of the AI model proposed in the embodiment of the present invention includes: a node storage module, which is provided with nodes for constructing various behavior trees, and the nodes include a start node, a control node and an action node; wherein, a behavior tree It is used to represent a workflow, the control node is used to implement the logic control in the workflow, the action node is used to implement the business operation in the workflow, and the action node includes: the related AI model deployment process A reusable independent action node encapsulated by business operations; the engineering editing module is used to provide a human-computer interaction interface for the user to construct and configure a behavior tree based on the nodes in the node storage module, and according to the user's construction and configuration operations to generate the current behavior tree for deploying the current AI model; the behavior tree engine is used to execute the workflow corresponding to the current behavior tree, construct the current AI model as an AI application and deploy it to the target specified by the user in the device.
  • it further includes: a knowledge graph module, configured to construct and store the current behavior tree as a knowledge graph instance based on the syntax of the knowledge graph; the knowledge graph instance includes: respectively representing each of the behavior trees A node of nodes, and a number of edges representing relationships between nodes or attributes of nodes.
  • a knowledge graph module configured to construct and store the current behavior tree as a knowledge graph instance based on the syntax of the knowledge graph; the knowledge graph instance includes: respectively representing each of the behavior trees A node of nodes, and a number of edges representing relationships between nodes or attributes of nodes.
  • the knowledge graph module is further used to recommend, error check, and control node and/or action node for the user based on the stored historical knowledge graph instance when the user constructs the current behavior tree. Smart mapping of attributes.
  • it further includes: a task template library, which is used to store at least one behavior tree template based on the nodes in the node storage module;
  • the at least one behavior tree template includes: corresponding to different AI application types or different At least one fixed behavior tree template of the equipment type; and/or, a general-purpose intelligent behavior tree template based on automatic machine learning;
  • the engineering editing module is further used to respond to the user's request to the task template through the human-computer interaction interface library query request, provide the at least one behavior tree template to the user for selection; when it is determined that the user selects a fixed behavior tree template for an AI application type or a target device type, according to the user's
  • the first configuration includes: determining the current AI model and the target device that needs to be deployed; after determining that the user selects the intelligent behavior
  • the current behavior tree is generated according to the second configuration performed by the user on the fixed behavior tree template;
  • the second configuration includes: determining
  • each node is composed of an interface part and an implementation part; for the implementation part, each node is a containerized application program; for the interface part, each node is a module that can be edited in the project Drag-and-drop, connection and attribute configuration graphic elements on the human-computer interaction interface; when the behavior tree engine executes each node in the current behavior tree, the input in the attribute configuration of the node interface part will be Read and transmit to the implementation part of the node, and after the implementation part completes the corresponding operation, the operation result will be converted into the output in the attribute configuration of the interface part of the node; the node storage module includes: interface storage library and a node service module; wherein, the interface repository is used to store the interface part of each node; the node service module is used to store the implementation part of each node.
  • the engineering editing module includes: a behavior tree editor, which is used to provide human-computer interaction for users to construct and edit the composition and connection relationship of the current behavior tree; The human-computer interaction for configuring the attributes of each node; the variable configurator is used to provide human-computer interaction for users to configure global variables; the node instance list module is used to convert each node added in the current behavior tree into A node instance is added to a node instance list for storage or viewing, and the node instance list presents all nodes in the current behavior tree in a list form or a tree form.
  • Another AI model device deployment system proposed in the embodiment of the present invention includes: at least one memory and at least one processor, wherein: the at least one memory is used to store computer programs; the at least one processor is used to call the The computer program stored in the at least one memory executes the device deployment method of the AI model as described in any one of the above implementation manners.
  • a computer-readable storage medium proposed in an embodiment of the present invention has a computer program stored thereon; the computer program can be executed by a processor and implement the device deployment method of the AI model described in any one of the above implementation modes.
  • the deployment operation process of the AI model is represented by a behavior tree diagram, that is, the relevant operations in the deployment process of the AI model are packaged into independent nodes, so that the nodes are reusable. It realizes the decoupling of specific business and engineering platforms, and then organizes nodes in the form of behavior tree to generate an intuitive AI application operation process from development to actual deployment, thus reducing the complexity of AI model device deployment.
  • each behavior tree instance is represented and saved in the form of knowledge graph, and on this basis, the recommendation function of nodes, attributes or workflows can be realized, which can further reduce the complexity of device deployment of AI models.
  • behavior tree templates for various AI application types and behavior tree templates integrated with automatic machine learning in the template task library, users can choose the behavior tree corresponding to their tasks instead of creating from scratch , which further greatly reduces the complexity of device deployment for AI models.
  • FIG. 1A and FIG. 1B are respectively exemplary flow charts of a device deployment method for an AI application in an embodiment of the present invention.
  • Fig. 2 is a schematic diagram of a behavior tree in an example of the present invention.
  • FIG. 3A and FIG. 3B are respectively schematic diagrams of a node instance list in an example of the present invention.
  • Fig. 4 is a schematic diagram of a global variable in an example of the present invention.
  • FIG. 5 is a schematic diagram of node recommendation when a user builds a behavior tree in an example of the present invention.
  • Fig. 6 is a schematic diagram of the definition and hierarchical relationship of classes in the knowledge map in the embodiment of the present invention.
  • Fig. 7 is a schematic diagram of object attributes and data attributes in the knowledge graph in the embodiment of the present invention.
  • Fig. 8 is a schematic diagram of an example of a knowledge map in an example of the present invention.
  • Fig. 9 is an exemplary flow chart of a device deployment system for AI applications in an embodiment of the present invention.
  • Fig. 10 is an exemplary flow chart of another AI application device deployment system in an embodiment of the present invention.
  • the trained model can be saved as a frozen pb model (frozen pb), a saved model (Saved Model) or a meta graph model (Meta Graph).
  • an AI application program consisting of data acquisition, data preprocessing, inference, data postprocessing and result sending modules.
  • Kubeflow is a machine learning toolbox for Kubernetes, which is suitable for deploying machine learning systems in various environments for development, testing, and production-level services.
  • Kubeflow in industrial scenarios has some limitations.
  • Kubeflow is based on Kubernetes, which means that the target device must support Kubernetes, and users must be familiar with Kubernetes. Because of the complexity of how Kubernetes works, it requires more expertise in other concepts. This will place a lot of constraints on the deployment of AI applications to industrial fields, especially for automation engineers.
  • the automation company provides dedicated inference equipment and corresponding deployment software. Due to different types of AI applications, there are different templates in the software. Users use these templates to configure and deploy applications. This has many limitations for users: insufficient flexibility, lack of interoperability, and inconsistent experience with different templates. There are also many limitations for software suppliers: the development cycle is long and it is difficult to reuse.
  • an embodiment of the present invention considers providing a behavior tree-based device deployment solution for the AI model.
  • the basic idea is to use a behavior tree diagram to represent the deployment operation process of the AI model.
  • the relevant operations in the deployment process of the AI model such as data preprocessing, data postprocessing, AI model retraining, AI model format conversion, device configuration, communication, etc., are packaged into independent nodes to make the nodes reusable and realize specific business and Decoupling of engineering platforms. Nodes are then organized in the form of a behavior tree to generate an intuitive AI application operation process from development to deployment.
  • each behavior tree instance is represented and saved in the form of a knowledge graph, and on this basis, the recommendation function of nodes, attributes or workflows is realized.
  • automatic machine learning can also be integrated into the template task library, so that users can select a workflow corresponding to their tasks, instead of creating a workflow from scratch.
  • FIG. 1A and FIG. 1B are respectively exemplary flow charts of a device deployment method for an AI application in an embodiment of the present invention. As shown in Figure 1A, the method may include the following steps:
  • Step 101 setting nodes for constructing various behavior trees.
  • the nodes may include: a start node, a control node and an action node. Among them, a behavior tree is used to represent a workflow.
  • the start node is the root node of the behavior tree, and there is only one start node in a behavior tree.
  • the control node defines how to execute branching in the tree, or whether to execute branching, and is used to implement logic control in the workflow, such as sequence, parallelism, selector, loop, counting, etc.
  • the control node is independent of the specific business of the AI application. By controlling nodes, users can create various workflows according to their needs.
  • the action node is a leaf node of the tree, and is used to realize the business operation in the workflow, and the action node has no output connection.
  • These action nodes include: reusable independent action nodes that encapsulate related business operations in the AI model deployment process. Through action nodes, encapsulated work can be reused in different workflows.
  • Data collection nodes which may include:
  • File Reader Used to read data from files in CSV, TDMS, etc. formats.
  • Database connector supports connection with common databases such as MS SQL and MongoDB.
  • OPC-UA client used to obtain data from OPC-UA servers commonly used in the field of automation.
  • Data processing nodes which may include:
  • Data Cleaner Used to format source data, delete abnormal data and duplicate data, etc.
  • Data enhancer used to perform spatial transformation, color transformation and other processing on the data to expand the data set.
  • Model management nodes which may include:
  • Model Importer This node is used to import a trained model from a model storage area such as a model library or a local PC.
  • Model Trainer Used to retrain the imported training model with new data sources or training parameters.
  • Model Converter Used to convert a trained model from a source format to a target format. For example, freeze pb (.pb) to ONNX (.ONNX) from TensorFlow.
  • the model adapter is used to realize automatic feature extraction, model selection, parameter optimization and other model adaptation processes based on the currently determined AI application type based on automatic machine learning, that is, to automatically select the appropriate AI model from the model library.
  • Application Deployer Used to package the AI model and its peripheral programs into AI applications, and then deploy them to specified devices.
  • each node may consist of an interface part and an implementation part.
  • each node will be a containerized application containing functional code and runtime dependencies.
  • Containerized applications can run independently and are exposed through specific network communication interfaces, such as RESTful API interfaces and remote procedure call (RPC) interfaces.
  • RPC remote procedure call
  • each node is a graphical element that can be dragged and dropped in the workflow editor, and each node has a property panel for configuring the node, such as input and output.
  • Each node can be configured and executed individually.
  • the input configured by the user in the node interface section will be read and translated to the corresponding containerized application.
  • the operation result such as the converted model will be converted back to the output of the node interface part.
  • the nodes can be stored in a node storage module, and the interface part and implementation part of each node can be stored separately.
  • the interface part can be stored in the interface storage module
  • the implementation part can be stored in the node service module.
  • Step 102 receiving the current behavior tree for deploying the current AI model constructed and configured by the user based on the nodes.
  • a graphical human-computer interaction interface may be provided, including: a behavior tree editing interface, a node configuration interface, a variable configuration interface, a node instance list interface, and an operation result feedback interface.
  • the behavior tree editing interface is used to provide human-computer interaction for the user to construct and edit the composition and connection relationship of the current behavior tree.
  • the node configuration interface is used to provide human-computer interaction for users to configure the attributes of each node.
  • variable configuration interface is used to provide human-computer interaction for users to configure global variables.
  • the node instance list interface is used to convert each node added in the current behavior tree into a node instance and add it to a node instance list for storage, viewing or modification.
  • the node instance list is presented in list form or tree form All nodes in the current behavior tree.
  • the running result feedback interface is used to present the intermediate and final execution results of the current behavior tree.
  • the current behavior tree can be: the user adds and connects the required nodes on the behavior tree editing interface, including the start node, the required control node and the required action node, and configures the node properties through the node configuration interface The resulting behavior tree.
  • the node configuration interface can be popped up by clicking the corresponding node on the behavior tree editing interface or sliding and hovering over the corresponding node.
  • the global variable can be further configured.
  • Fig. 2 is a schematic diagram of a behavior tree in an example of the present invention.
  • the behavior tree is built by adding and connecting a series of nodes with specific functions, as shown in Figure 2, the behavior tree includes: the start node (StartNode) on the far left; on the right side of the start node and connected to it
  • output value 1 input value 1.
  • the general execution logic in the embodiment of the present invention is from left to right and from top to bottom, and the specific execution logic is controlled by the control node.
  • the control node there are some numbers indicating the execution order in the upper right corner of the action node, that is, the execution order is to execute actions 1 and 2 sequentially, then execute actions 3 and 4 in parallel, then execute action 5 in a loop, and then execute actions sequentially 6 and action 7.
  • FIG. 3A is a list display
  • FIG. 3B is a tree display
  • global variables as shown in FIG. 4 may also be configured.
  • a task template library can be set. At least one fixed behavior tree template.
  • the device type may include an edge (Edge) device, an industrial personal computer (IPC, Industrial Personal Computer) device, an AI computing stick, or an AI computing module.
  • AI application types can include object detection, object classification, time series, etc.
  • the user can determine the fixed behavior tree template to be selected according to the type of the target device or the type of AI application. After selecting a fixed behavior tree template, it can be used directly or use the node storage module, especially the interface storage library. The nodes in it are modified, and then the corresponding configuration is completed and the current behavior tree is generated.
  • the specific configuration may include: configuring the data set for retraining the AI model, and selecting the desired current AI model.
  • it can also include configurations of some node attributes, global variables or local variables, etc. The details may be determined according to actual needs, and are not limited here.
  • a general intelligent behavior tree template based on automatic machine learning may also be stored in the task template library.
  • an AutotoML-based smart behavior tree template After the user selects the smart behavior tree template, the current behavior tree can be generated only by simple configuration.
  • the specific configuration can include: configuring the data set for retraining the AI model, configuring the required AI application type, and the intelligent behavior tree template can determine the appropriate current AI model by itself.
  • it can also include configurations of some node attributes, global variables or local variables, etc. The details may be determined according to actual needs, and are not limited here.
  • the behavior tree template in the task template library can be pre-configured, or the current behavior tree can be stored as a corresponding behavior tree template according to the user's instruction.
  • the behavior tree template only includes the current behavior tree All nodes and the association relationship between nodes in , do not need to store their configuration data.
  • the above-mentioned action node named application deployer should be added to the current behavior tree, and the target device should be configured in the configuration panel of the application deployer node.
  • a model importer and a model trainer can be added before the application deployer. If format conversion of the AI model is required, a model converter can be added between the model trainer and the application deployer. Or, in another embodiment, a model adapter and a model trainer may also be added before the application deployer, and the model adapter automatically selects a suitable AI model for training.
  • Step 103 Execute the workflow corresponding to the current behavior tree according to the execution logic of the current behavior tree, construct the predetermined current AI model as an AI application, and deploy it to the target device specified by the user.
  • the deployed AI application can be used to monitor the operating status of the device.
  • the input configured by the user in the interface part of the node is read and transformed into the implementation part of the node.
  • the result of the operation is converted back to the output of the interface part of the node.
  • the intermediate results and final results during the execution process can be displayed.
  • the debugging process is similar to the formal execution process, and the logic is executed according to the behavior tree to execute the current behavior.
  • the tree corresponds to the workflow, and the intermediate results and final results in the execution process are fed back to the operation result feedback interface of the human-machine interface for display.
  • the status of each node is monitored, and the exceptions and errors encountered during the execution are prompted or processed accordingly.
  • the user can execute debugging for each node separately, and check whether its input and output are as expected according to the feedback execution result, and if not, the node needs to be checked and reconfigured.
  • the user can debug the entire behavior tree workflow, which means that the workflow will clear the runtime data and execute from the root node. If the workflow is executed with expected results, the above step 103 can be executed to deploy the application program of the AI model to the target device.
  • the current behavior tree that has been constructed by the user can be saved as a behavior tree instance.
  • the current behavior tree is constructed as a knowledge graph instance based on the syntax of the knowledge graph for storage.
  • the knowledge graph example includes: nodes respectively representing each node in the behavior tree, and multiple edges representing relationships between nodes or node attributes.
  • Figure 5 shows a schematic diagram of node recommendation when a user builds a behavior tree. As shown in the gray dashed box in Figure 5, when creating an action node for a sequence node, the system recommends Action 5, Action 6, and Action 7, etc., and gives matching degrees of 75%, 50%, and 20%, respectively. To facilitate the user to choose.
  • the syntax of the knowledge graph may be as shown in FIG. 6 and FIG. 7 .
  • Fig. 6 shows the definition and hierarchical relationship of classes in the knowledge map in this embodiment. As shown in Figure 6, there are three subclasses under the class "owl:Thing": workflow, node and data object.
  • the node class corresponds to the node of the behavior tree, and there are three subclasses: start node (StartNode), control node (ControlNode) and action node (ActionNode). Specific logical node types such as sequence, parallel, selector, loop, count, etc. are subclasses of control nodes, and specific action node types such as model trainer, model converter, etc. are subclasses of action nodes.
  • the workflow class corresponds to the workflow of the behavior tree.
  • Data Object is the base class for node attributes, global variable lists, and blackboards.
  • the node attribute class has subclasses such as input and output. Among them, the blackboard is used to record the list of node instances.
  • Fig. 7 shows a schematic diagram of object attributes and data attributes in the knowledge graph in this embodiment. As shown in Figure 7, where:
  • HasGVL hasGVL
  • the knowledge graph example shown in Figure 8 can be obtained.
  • log service error handling service
  • user management service user management service
  • project management service is further provided.
  • the device deployment method of the AI model in the embodiment of the present invention has been described in detail above, and the device deployment system of the AI model in the embodiment of the present invention will be described in detail below.
  • the AI model device deployment system in the embodiment of the present invention can be used to implement the AI model device deployment method in the embodiment of the present invention.
  • the details not disclosed in the system embodiment of the present invention please refer to the corresponding method in the method embodiment of the present invention description, and will not be repeated here.
  • FIG. 9 is an exemplary structural diagram of an AI model device deployment system in an embodiment of the present invention.
  • the system can interact with multiple external databases.
  • an AI model library 10 for storing and managing pre-trained AI models
  • a data set library 20 for storing data sets for retraining AI models
  • a knowledge graph library for storing knowledge graph instances of historical behavior trees 30.
  • the system may include: a node storage module 910 , a project editing module 920 , a behavior tree engine 930 , a knowledge graph module 940 and a task template library 950 .
  • the node storage module 910 is provided with nodes for constructing various behavior trees, and the nodes include a start node, a control node and an action node; wherein, a behavior tree is used to represent a workflow, and a control node is used to realize the described Logical control in the workflow, action nodes are used to implement business operations in the workflow, and the action nodes include: reusable independent action nodes that encapsulate related business operations in the AI model deployment process, Such as model importer, model trainer, model converter, model adapter, and application deployer.
  • each action node can be composed of an interface part and an implementation part; for the implementation part, each action node is a containerized application program; for the interface part, each action node is a person who can edit the project Graphical elements for dragging and dropping, connection and attribute configuration on the computer interaction interface; when each action node is executed, the input in the attribute configuration of the interface part of the action node will be read and transmitted to the implementation part of the action node , and after the implementation part completes the corresponding operation, the operation result will be transformed into the output in the attribute configuration of the action node interface part.
  • the node storage module 910 in this embodiment may include: an interface storage library 911 and a node service module 912 .
  • the interface repository 911 is used to store the interface part of each action node, and stores the start node and the control node
  • the node service module 912 is used to store the implementation part of each action node.
  • a public service module 960 may be further included for storing application programs for log service, error handling service, user management service, and project management service.
  • the engineering editing module 920 is used to provide a human-computer interaction interface for the user to construct and configure a behavior tree based on the nodes in the node storage module, and generate the current behavior for deploying the current AI model according to the user's construction and configuration operations Tree.
  • the current behavior tree can be stored in XML format, or can also be stored in script S format such as Python. The details may be determined according to actual needs, and are not limited here.
  • the project editing module 920 may include a behavior tree editor 921, a node configurator 922, a variable configurator 923, a data blackboard 924, a running/debugging monitor 925, and the like.
  • the behavior tree editor 921 is used to provide human-computer interaction for the user to construct and edit the composition and connection relationship of the current behavior tree.
  • the node configurator 922 is used to provide human-computer interaction for the user to configure the attributes of each node.
  • variable configurator 923 is used to provide human-computer interaction for users to configure global variables.
  • the data blackboard 924 is used to convert each node added in the current behavior tree into a node instance and add it to a node instance list for storage or viewing.
  • the node instance list presents the current behavior tree in list form or tree form. All nodes in the behavior tree.
  • the running/debugging monitor 925 is used to send debugging or running instructions to the behavior tree engine 930 , receive and present the intermediate execution results and final execution results fed back by the behavior tree engine 930 .
  • the behavior tree engine 930 is used to execute the workflow corresponding to the current behavior tree according to the execution logic of the current behavior tree when receiving the debugging or running instruction, and after the user specifies the target device, the current AI model Build as an AI application and deploy to the target device 40 specified by the user.
  • the intermediate results and final results in the execution process are fed back to the running/debugging monitor 925 of the project editing module 920 for display.
  • the status of each node is monitored during execution, and abnormalities and errors encountered during execution are dealt with accordingly.
  • the deployed AI model can be used to monitor the operating status of the device.
  • the behavior tree engine 930 may specifically include: a behavior tree executor 931 , a state monitor 932 and an error handler 933 .
  • the behavior tree executor 931 executes the workflow corresponding to the current behavior tree according to the execution logic of the current behavior tree, and after the user specifies the target device, builds the predetermined current AI model into an AI application and deploys it to the in the target device specified by the user. Specifically, when each node in the current behavior tree is executed, the input in the attribute configuration of the interface part of the node is read and converted to the implementation part of the node; after the implementation part completes specific operations, the The operation result is converted into the output of the interface part of the node and provided to the project editing module 920 for display.
  • the state monitor 932 is used to monitor the state of each node during the execution of the current behavior tree by the behavior tree executor 931 . For example, when running/debugging the current behavior tree, users can add some nodes or custom data to monitor.
  • the error handler 933 is used to deal with exceptions and errors encountered during execution.
  • an error (Error) attribute can be set for each node, such as an error ID and an error message, etc., so that detailed error information can be obtained through the error attribute.
  • the knowledge graph module 940 is used to construct the current behavior tree into a knowledge graph instance based on the grammar of the knowledge graph for storage; the knowledge graph instance includes: nodes respectively representing each node in the behavior tree, and nodes representing nodes Multiple edges of a relationship or node attribute.
  • the knowledge graph module 940 is also used for recommending control nodes and/or action nodes, error checking, and intelligent mapping of node attributes based on stored historical knowledge graph instances when the user constructs the current behavior tree.
  • the task template library 950 is used to store at least one behavior tree template based on nodes in the node storage module; the at least one behavior tree template includes: at least one fixed behavior tree template corresponding to different AI application types or different device types and/or, a generic automated machine learning-based intelligent behavior tree template.
  • the engineering editing module 920 may be further configured to provide the at least one behavior tree template to the user for selection in response to the user's query request for the task template library through the human-computer interaction interface.
  • the first configuration includes: determining the current AI model and the target device to be deployed;
  • the second configuration includes: determine the current AI application type and the target device to be deployed;
  • the intelligent behavior tree template can be automatically selected from an AI model library according to the AI application type configured by the user Select the appropriate current AI model.
  • the device deployment system of the AI model may only include some of the above modules.
  • the AI model device deployment system may only include the above-mentioned node storage module 910, engineering editing module 920 and behavior tree engine 930; or, only include the above-mentioned node storage module 910, engineering editing module 920, Behavior tree engine 930 and knowledge map module 940; or, only include the above-mentioned node storage module 910, project editing module 920, behavior tree engine 930 and task template library 950. It is not limited here.
  • FIG. 10 is a schematic structural diagram of another AI model device deployment system according to an embodiment of the present application.
  • the system can be used to implement the method shown in FIG. 1 or implement the system shown in FIG. 9 .
  • the system may include: at least one memory 1001 , at least one processor 1002 and at least one display 1003 .
  • some other components may also be included, such as communication ports and the like. These components communicate over bus 1004 .
  • At least one memory 1001 is used to store computer programs.
  • the computer program can be understood as each module of the device deployment system including the AI model shown in FIG. 9 .
  • at least one memory 1001 can also store an operating system and the like.
  • the operating system includes but is not limited to: Android operating system, Symbian operating system, Windows operating system, Linux operating system and so on.
  • At least one processor 1002 is used to call at least one computer program stored in the memory 1001 to execute the device deployment method of the AI model described in the embodiment of the present application.
  • the processor 1002 may be a CPU, a processing unit/module, an ASIC, a logic module or a programmable gate array, and the like. It can receive and send data through the communication port.
  • At least one display 1003 is used to display a human-computer interaction interface.
  • At least one processor 1002 is configured to invoke a computer program stored in at least one memory 1001 to enable the system to execute the operations in the device deployment method of the AI model in any of the above implementation manners.
  • a hardware module may include specially designed permanent circuits or logic devices (such as special-purpose processors, such as FPGAs or ASICs) to perform specific operations.
  • Hardware modules may also include programmable logic devices or circuits (eg, including general-purpose processors or other programmable processors) temporarily configured by software to perform particular operations.
  • programmable logic devices or circuits eg, including general-purpose processors or other programmable processors
  • an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, the computer program can be executed by a processor and implement the device deployment method of the AI model described in the embodiment of the present application .
  • a system or device equipped with a storage medium may be provided, on which the software program code for realizing the functions of any implementation manner in the above-mentioned embodiments is stored, and the computer (or CPU or MPU of the system or device) ) to read and execute the program code stored in the storage medium.
  • an operating system or the like operated on a computer may also complete part or all of the actual operations through instructions based on program codes.
  • Embodiments of storage media for providing program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), Tape, non-volatile memory card, and ROM.
  • the program code can be downloaded from a server computer via a communication network.
  • the deployment operation process of the AI model is represented by a behavior tree diagram, that is, the relevant operations in the deployment process of the AI model are packaged into independent nodes, so that the nodes are reusable. It realizes the decoupling of specific business and engineering platforms, and then organizes nodes in the form of behavior tree to generate an intuitive AI application operation process from development to actual deployment, thus reducing the complexity of AI model device deployment.
  • each behavior tree instance is represented and saved in the form of knowledge graph, and on this basis, the recommendation function of nodes, attributes or workflows can be realized, which can further reduce the complexity of device deployment of AI models.
  • behavior tree templates for various AI application types and behavior tree templates integrated with automatic machine learning in the template task library, users can choose the behavior tree corresponding to their tasks instead of creating from scratch , which further greatly reduces the complexity of device deployment for AI models.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Stored Programmes (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本发明实施例中公开了一种AI模型的设备部署方法、系统及存储介质。其中，方法包括：接收用户基于预先设置的用于构建各种行为树的节点构建并配置完成的当前行为树；所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点；执行所述当前行为树对应的工作流，将预先确定的当前AI模型构建为AI应用并部署到用户指定的目标设备中。本发明实施例中的技术方案能够降低AI模型部署的复杂度。

Description

AI模型的设备部署方法、系统及存储介质 技术领域
本发明涉及工业自动化领域，特别是一种人工智能(AI)模型的设备部署方法、系统及计算机可读存储介质。
背景技术
人工智能(AI)在计算机视觉、机器人技术、预测性维护等工业领域有着广泛的应用。IT公司和自动化公司都在尝试将AI引入工业自动化系统。
在这个过程中，训练好的AI模型需要被部署到特定的设备中。常见的部署操作通常包括模型格式转换、推理应用构建、设备配置、通信配置等。在AI模型的部署过程中，经常会遇到以下情况：对于不同的设备或不同的AI应用类型，工作流往往存在较大的差异，难以重用。此外，工作流的每一步通常需要在不同的场景下进行操作，使得工作流的执行和管理过程较为复杂。因此，在工业自动化领域，AI应用的开发和部署过程复杂、形式多样、且缺乏互操作性，导致AI难以融入工业自动化系统。
发明内容
有鉴于此，本发明实施例中一方面提出了一种AI模型的设备部署方法，另一方面提出了一种AI模型的设备部署系统和计算机可读存储介质，用以降低AI模型的设备部署复杂度。
本发明实施例中提出的一种AI模型的设备部署方法，包括：接收用户基于预先设置的用于构建各种行为树的节点构建并配置完成的当前行为树；所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点；执行所述当前行为树对应的工作流，将预先确定的当前AI模型构建为AI应用并部署到用户指定的目标设备中。
在一个实施方式中，所述当前行为树为：用户通过添加、连接和配置开始节点、所需的控制节点和所需的动作节点构建而成的行为树；或者为：从任务模板库中，根据所需的AI应用类型或需要部署的目标设备类型选取对应的固定行为树模板，并基于所述固 定行为树模板进行第一配置以及调整或不调整后得到的行为树；所述第一配置包括：确定当前AI模型及其需要部署的目标设备；或者为：从任务模板库中选取一基于自动机器学习的智能行为树模板，并基于所述智能行为树模板进行第二配置后得到的行为树；所述第二配置包括：确定AI应用类型和需要部署的目标设备，由所述智能行为树模板根据所述AI应用类型从一AI模型库中自动确定合适的AI模型。
在一个实施方式中，进一步包括：将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储；所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系或节点属性的多条边。
在一个实施方式中，进一步包括：在用户构建所述当前行为树时，基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。
在一个实施方式中，每个节点均由接口部分和实现部分组成；对于实现部分，每个节点是一个容器化应用程序；对于接口部分，每个节点是一个可在人机交互界面上进行拖放、连接及属性配置的图形元素；所述当前行为树中的每个节点被执行时，所述节点接口部分的属性配置中的输入将被读取并传输给所述节点的实现部分，并在所述实现部分完成对应操作后，操作结果将转换为所述节点接口部分的属性配置中的输出。
在一个实施方式中，进一步包括：将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储或查看，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
在一个实施方式中，所述当前行为树中的动作节点包括：顺序执行的如下动作节点：模型导入器、模型训练器和应用程序部署器；或者包括：顺序执行的如下动作节点：模型导入器、模型训练器、模型转换器和应用程序部署器；或者包括：顺序执行的如下动作节点：模型适配器、模型训练器和应用程序部署器；其中，模型导入器用于从模型存储区导入经过训练的AI模型；模型训练器用于对导入的训练模型进行新数据源或训练参数的再训练；模型转换器用于将训练好的模型从源格式转换为目标格式；模型适配器用于基于自动机器学习根据当前确定的AI应用类型从模型库中自动选取合适的AI模型；应用程序部署器用于将AI模型打包为AI应用，并部署到指定的目标设备。
本发明实施例中提出的AI模型的设备部署系统，包括：节点存储模块，其中设置有用于构建各种行为树的节点，所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现 所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点；工程编辑模块，用于提供用户基于所述节点存储模块中的节点进行行为树构建并配置的人机交互界面，并根据所述用户的构建和配置操作，生成用于部署当前AI模型的当前行为树；行为树引擎，用于执行所述当前行为树对应的工作流，将所述当前AI模型构建为AI应用并部署到用户指定的目标设备中。
在一个实施方式中，进一步包括：知识图谱模块，用于将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储；所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系或节点属性的多条边。
在一个实施方式中，所述知识图谱模块进一步用于在用户构建所述当前行为树时，基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。
在一个实施方式中，进一步包括：任务模板库，用于存储至少一个基于所述节点存储模块中的节点构建的行为树模板；所述至少一个行为树模板包括：对应于不同AI应用类型或不同设备类型的至少一个固定行为树模板；和/或，一通用的基于自动机器学习的智能行为树模板；所述工程编辑模块进一步用于响应于用户通过所述人机交互界面对所述任务模板库的查询请求，将所述至少一个行为树模板提供给所述用户进行选择；在确定所述用户选择针对一AI应用类型或一目标设备类型的固定行为树模板时，根据所述用户对所述固定行为树模板进行的第一配置以及调整或不调整，生成所述当前行为树；所述第一配置包括：确定当前AI模型及其需要部署的目标设备；在确定用户选择所述智能行为树模板时，根据所述用户对所述固定行为树模板进行的第二配置，生成所述当前行为树；所述第二配置包括：确定当前的AI应用类型和需要部署的目标设备；所述智能行为树模板能够根据所述用户配置的AI应用类型从一AI模型库中自动选取合适的当前AI模型。
在一个实施方式中，所述每个节点均由接口部分和实现部分组成；对于实现部分，每个节点是一个容器化应用程序；对于接口部分，每个节点是一个可在所述工程编辑模块的人机交互界面上进行拖放、连接及属性配置的图形元素；所述行为树引擎在执行所述当前行为树中的每个节点时，所述节点接口部分的属性配置中的输入将被读取并传输给所述节点的实现部分，并在所述实现部分完成对应操作后，操作结果将转换为所述节点接口部分的属性配置中的输出；所述节点存储模块包括：接口存储库和节点服务模块；其中，所述接口存储库用于存储每个节点的接口部分；所述节点服务模块用于存储每个节点的实现部分。
在一个实施方式中，所述工程编辑模块包括：行为树编辑器，用于提供用户对当前 行为树的组成和连接关系进行构建和编辑的人机交互；节点配置器，用于提供用户对每个节点的属性进行配置的人机交互；变量配置器，用于提供用户对全局变量进行配置的人机交互；节点实例列表模块，用于将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储或查看，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
本发明实施例中提出的又一种AI模型的设备部署系统，包括：至少一个存储器和至少一个处理器，其中：所述至少一个存储器用于存储计算机程序；所述至少一个处理器用于调用所述至少一个存储器中存储的计算机程序执行如上任一实施方式所述的AI模型的设备部署方法。
本发明实施例中提出的一种计算机可读存储介质，其上存储有计算机程序；所述计算机程序能够被一处理器执行并实现如上任一实施方式所述的AI模型的设备部署方法。
从上述方案中可以看出，由于本发明实施例中，用行为树图来表示AI模型的部署操作流程，即将AI模型部署过程中的相关操作封装成独立的节点，使节点具有可重用性，实现了特定业务与工程平台的解耦，然后以行为树的形式组织节点，生成从开发到实际部署的直观的AI应用操作流程，从而降低了AI模型的设备部署复杂度。
此外，通过采用知识图谱的形式来表示和保存各个行为树实例，并在此基础上实现节点、属性或工作流的推荐功能，从而可进一步降低AI模型的设备部署复杂度。
进一步地，通过将针对各种AI应用类型的行为树模板以及集成了自动机器学习的行为树模板设置在模板任务库中，使得用户可以从中选择与其任务相对应的行为树，而不是从头开始创建，进一步大大降低了AI模型的设备部署复杂度。
附图说明
下面将通过参照附图详细描述本发明的优选实施例，使本领域的普通技术人员更清楚本发明的上述及其它特征和优点，附图中：
图1A和图1B分别为本发明实施例中一种AI应用的设备部署方法的示例性流程图。
图2为本发明一个例子中的行为树的示意图。
图3A和图3B分别为本发明一个例子中的节点实例列表的示意图。
图4为本发明一个例子中一种全局变量的示意图。
图5为本发明一个例子中用户构建行为树时的节点推荐示意图。
图6为本发明实施例中知识图谱中类的定义和层次关系示意图。
图7为本发明实施例中知识图谱中对象属性和数据属性的示意图。
图8为本发明一个例子中的知识图谱实例的示意图。
图9为本发明实施例中一种AI应用的设备部署系统的示例性流程图。
图10为本发明实施例中又一种AI应用的设备部署系统的示例性流程图。
其中，附图标记如下：
标号 含义
101～104 步骤
10 AI模型库
20 数据集库
30 知识图谱库
910 节点存储模块
911 接口存储库
912 节点服务模块
920 工程编辑模块
921 行为树编辑器
922 节点配置器
923 变量配置器
924 数据黑板
925 运行/调试监控器
930 行为树引擎
931 行为树执行器
932 状态监控器
933 错误处理器
940 知识图谱模块
950 任务模板库
1001 存储器
1002 处理器
1003 显示器
1004 总线
具体实施方式
本发明实施例中，考虑到(1)目前流行的AI框架如TensorFlow、PyTorch或scikit-learn，主要集中在AI模型的算法、训练和推理上，并将训练好的模型保存为一些文件格式。但没有直观易用的工作流编排器来解决部署的问题。以TensorFlow为例，训练好的模型可以保存为冻结的pb模型(frozen pb)、已保存的模型(Saved Model)或元图模型(Meta Graph)。当用户需要将模型部署到特定的设备上时，需要开发一个由数据采集、数据预处理、推理、数据后处理和结果发送模块组成的AI应用程序。通常，应用程序都是用Python、C++等编程语言编写的。用户还需要手动将所有依赖关系安装到设备中，例如安装TensorFlow作为推理引擎。为了与其他工业设备进行通信，用户需要将通信模块集成到应用中。由于整个过程包括基于代码的开发和基于命令的操作，如果目标设备发生变化或应用类型发生变化，整个过程需要从头开始重新启动，很难重用和管理。(2)Kubeflow是Kubernetes的机器学习工具箱，适用于将机器学习系统部署到各种环境中进行开发、测试和生产级服务。然而，在工业场景中使用Kubeflow却有一些局限性。首先，Kubeflow是基于Kubernetes的，这意味着目标设备必须支持Kubernetes，用户必须熟悉Kubernetes。由于Kubernetes的工作原理比较复杂，因此需要更多其他概念方面的专业知识。这将给AI应用程序部署到工业领域带来很多限制，尤其是对于自动化工程师而言。其次，当用户使用Kubeflow流水线创建工作流时，首先需要使用领域特定的语言来创建所有的组件，然后通过配置组件的输入和输出来创建工作流，这对不熟悉它的终端用户来说是不友好的。(3)自动化公司提供专用的推理设备和相应的部署软件。由于AI应用类型不同，软件中有不同的模板。用户使用这些模板来配置和部署应用程序。这对用户来说有很多的限制：灵活性不够，且缺乏互操作性，不同模板的使用体验不一致。对软件供应商也有很多局限性：开发周期长，难以重用。
因此，为了提供一种简单易用的AI模型的设备部署方案，本发明实施例中考虑提供一种基于行为树的AI模型的设备部署方案。其基本思想是用行为树图来表示AI模型的部署操作流程。即将AI模型部署过程中的相关操作如数据预处理、数据后处理、AI模型再训练、AI模型格式转换、设备配置、通信等封装成独立的节点，使节点具有可重用性，实现特定业务与工程平台的解耦。然后以行为树的形式组织节点，生成从开发到部署的直观的AI应用操作 流程。此外，采用知识图谱的形式来表示和保存各个行为树实例，并在此基础上实现节点、属性或工作流的推荐功能。进一步地，还可以将自动机器学习集成到模板任务库中，使得用户可以从中选择与其任务相对应的工作流，而不是从头开始创建工作流。
为使本发明的目的、技术方案和优点更加清楚，以下举实施例对本发明进一步详细说明。
图1A和图1B分别为本发明实施例中一种AI应用的设备部署方法的示例性流程图。如图1A所示，该方法可包括如下步骤：
步骤101，设置用于构建各种行为树的节点。所述节点可包括：开始节点、控制节点和动作节点。其中，一个行为树用于表征一个工作流。
开始节点为行为树的根节点，一个行为树中只有一个开始节点。
控制节点定义如何在树中执行分支，或者是否执行分支，用于实现所述工作流中的逻辑控制，如顺序、并行、选择器、循环、计数等。控制节点独立于AI应用的特定业务。通过控制节点，用户可以根据自己的需求创建各种工作流。
动作节点是树的叶节点，用于实现所述工作流中的业务操作，动作节点没有输出连接。这些动作节点中包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点。通过动作节点，封装后的工作可以在不同的工作流中重用。
下面列出了一些示例性的动作节点：
(1)数据收集类节点，其可包括：
文件读取器：用于从CSV、TDMS等格式的文件中读取数据。
数据库连接器：支持与MS SQL、MongoDB等常用数据库的连接。
OPC-UA客户端：用于从自动化领域常用的OPC-UA服务器获取数据。
(2)数据处理类节点，其可包括：
数据清理器：用于对源数据进行格式化，删除异常数据以及重复数据等。
数据转换器：用于将源数据从源格式转换为目标格式。
数据增强器：用于对数据进行空间变换，颜色变换等处理以扩充数据集。
(3)模型管理类节点，其可包括：
模型导入器：该节点用于从模型存储区如模型库或本地PC机导入经过训练的模型。
模型训练器：用于对导入的训练模型进行新数据源或训练参数的再训练。
模型转换器：用于将训练好的模型从源格式转换为目标格式。例如，从TensorFlow冻结pb(.pb)到ONNX(.ONNX)。
模型适配器，用于基于自动机器学习根据当前确定的AI应用类型实现自动化的特征提取、模型选择和参数优化等模型适配过程，即实现从模型库中自动选取合适的AI模型。
(4)部署工具包
应用程序部署器：用于将AI模型及其周边程序打包为AI应用，然后将其部署到指定的设备。
本实施例中，每个节点可由接口部分和实现部分组成。对于实现部分，每个节点将是一个容器化的应用程序，其中包含功能代码和运行依赖项。容器化的应用程序可独立运行，并通过特定的网络通信接口，如RESTful API接口和远程过程调用(RPC)接口对外公开。对于接口部分，每个节点都是一个图形元素，可以在工作流编辑器中进行拖放，且每个节点都有一个属性面板，用于配置节点，例如输入和输出。
每个节点都可以单独配置和执行。在执行节点时，用户在节点接口部分配置的输入将被读取并转换到相应的容器化应用程序。在完成特定的AI操作如模型转换后，操作结果如转换后的模型将转换回节点接口部分的输出。
具体实现时，所述节点可存储在节点存储模块中，并且每个节点的接口部分和实现部分可单独存储。例如，接口部分可存储在接口存储模块中，实现部分可存储在节点服务模块中。
步骤102，接收用户基于所述节点构建并配置完成的用于部署当前AI模型的当前行为树。
本实施例中，可提供一图形化的人机交互界面，并包括：行为树编辑界面、节点配置界面、变量配置界面、节点实例列表界面、以及运行结果反馈界面等。
其中，行为树编辑界面用于提供用户对当前行为树的组成和连接关系进行构建和编辑的人机交互。
节点配置界面用于提供用户对每个节点的属性进行配置的人机交互。
变量配置界面用于提供用户对全局变量进行配置的人机交互。
节点实例列表界面用于将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储、查看或修改，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
运行结果反馈界面用于呈现当前行为树的中间执行结果和最终执行结果。
相应地，当前行为树可以是：用户通过在行为树编辑界面上添加、连接所需的节点，包括开始节点、所需的控制节点和所需的动作节点，并通过节点配置界面进行节点属性配置后得到的行为树。实际应用中，节点配置界面可通过在行为树编辑界面点击相应的节点或滑动悬浮在相应的节点上而弹出。此外，当需要全局变量时，可进一步对全局变量进行配置。
图2为本发明一个例子中的行为树的示意图。该行为树是通过添加和连接一系列具有特定功能的节点来构建的，如图2所示，该行为树中包括：位于最左侧的开始节点(StartNode)；位于开始节点右侧并与其连接的第一个顺序节点(Sequence)；位于第一个顺序节点右侧并与其连接的第二个顺序节点、并行节点(Parallel)、循环节点(Loop)、第六动作节点(ActionNode)(简称动作6)和第七动作节点(简称动作7)；位于第二个顺序节点右侧并与其连接的第一动作节点(简称动作1)和第二动作节点(简称动作2)；位于并行节点右侧并与其连接的第三动作节点(简称动作3)和第四动作节点(简称动作4)；位于循环节点右侧并与其连接的第五动作节点(简称动作5)。其中，第一动作节点的属性配置中配置有两个输入值：输入1＝值1，输入2＝值2，和一个输出值：输出1＝输出值1，且该输出值1与第二动作节点的一个输入值建立有映射关系，即第二动作节点的输入1＝动作1.输出值1。此外，第二动作节点还具有另一个输入值：输入2＝值2，和一个输出值：输出1＝输出值1，且该输出值1与第三动作节点的一个输入值建立有映射关系，即第三动作节点的输入1＝动作2.输出值1。此外，第三动作节点还具有另一个输入值：输入2＝值1，和一个输出值：输出1＝输出值1。
本发明实施例中的一般执行逻辑是从左到右、从上到下的，并且具体的执行逻辑是由控制节点控制的。图2中，在动作节点的右上角具有一些表示执行顺序的数字，即执行顺序为顺序执行动作1和动作2，然后并行执行动作3和动作4，接着循环执行动作5，之后再顺序执行动作6和动作7。
针对图2所示的行为树，可得到如图3A和图3B所示的节点实例列表。其中，图3A为列表显示，图3B为树状显示。相应地，还可以配置如图4所示的全局变量。
此外，为方便用户构建行为树，节省时间，并最大限度地降低操作难度，本实施例中，可设置一任务模板库，该任务模板库中可存储有针对不同设备类型或不同AI应用类型的至少一个固定行为树模板。例如，设备类型可包括边缘(Edge)设备、工业计算机(IPC，Industrial Personal Computer)设备、AI计算棒或AI计算模块。AI应用类型可包括对象检测、对象分类、时间序列等。具体应用中，用户可根据目标设备的类型或者是 AI应用的类型确定所要选取的固定行为树模板，在选定一固定行为树模板后，可以直接使用也可利用节点存储模块特别是接口存储库中的节点对其进行修改，之后完成相应的配置并生成当前行为树。具体配置可包括：配置对AI模型进行再训练的数据集、选定所需的当前AI模型。此外，还可包括一些节点属性配置，全局变量或局部变量等的配置。具体可根据实际需要确定，此处不对其进行限定。
或者，该任务模板库中也可存储一通用的基于自动机器学习的智能行为树模板。例如，基于AotoML的智能行为树模板。当用户选取该智能行为树模板后，只需进行简单的配置便可生成当前行为树。具体配置可包括：配置对AI模型进行再训练的数据集、配置所需的AI应用类型，智能行为树模板便可自行确定合适的当前AI模型。此外，还可包括一些节点属性配置，全局变量或局部变量等的配置。具体可根据实际需要确定，此处不对其进行限定。
本实施例中，任务模板库中的行为树模板可以是预先配置的，也可以是根据用户的指令，将当前行为树存储为对应的行为树模板，此时行为树模板中仅包括当前行为树中的所有节点及节点间的关联关系，可不存储其配置数据。
本实施例中，为了实现AI模型的设备部署，当前行为树中应添加有上述名为应用程序部署器的动作节点，并在应用程序部署器节点的配置面板中配置有目标设备。此外，还可在应用程序部署器之前添加一个模型导入器和一个模型训练器，若是需要进行AI模型的格式转换，则可在模型训练器和应用程序部署器之间再添加一个模型转换器。或者，在另一个实施方式中，也可在应用程序部署器之前添加一个模型适配器和一个模型训练器，由该模型适配器自动选取合适的AI模型进行训练。
步骤103，按照所述当前行为树的执行逻辑，执行所述当前行为树对应的工作流，将预先确定的当前AI模型构建为AI应用并部署到用户指定的目标设备中。
之后，便可利用部署后的AI应用监控设备的运行状态。
如前所示，在执行节点时，用户在节点接口部分配置的输入将被读取并转换到节点的实现部分。在实现部分完成特定的操作后，操作结果将转换回节点接口部分的输出。
具体实现时，可将执行过程中的中间结果及最终结果显示出来。此外，若不确定当前行为树是否存在语法错误等问题，步骤103之前，可先对其进行调试，其中调试过程与正式执行过程类似，都是按照所述行为树执行逻辑，执行所述当前行为树对应的工作流，并将执行过程中的中间结果及最终结果反馈给人机交互界面的运行结果反馈界面进行显示。执行期间对各节点的状态进行监控，并对于执行过程中遇到的异常及错误进行 相应的提示或处理。
本实施例中，用户可以针对每个节点分别执行调试，并根据反馈的执行结果检查其输入和输出是否跟预期一样，如果不一样，则需要检查并重新配置节点。单节点调试通过后，用户可以对整个行为树工作流进行调试，这意味着工作流将清除运行时数据并从根节点执行。如果工作流以预期的结果执行，则可以执行上述的步骤103，将AI模型的应用程序部署到目标设备中。此外，在有些应用中，也可先对添加应用程序部署器这一动作节点之前的行为树进行调试，在调试通过后，再添加应用程序部署器并配置对应的目标设备。
在一些实施方式中，对于相同的应用场景，例如将相同类型的AI模型部署到相同类型的目标设备中并实现相同的AI应用类型的情况，为了避免重复构建行为树的人力、时间等的浪费，对于用户已构建完成的当前行为树可作为一个行为树实例进行保存。当需要再次构建相同的行为树时，直接调用该行为树实例即可。
此外，为了进一步增加行为树构建的便利性，在用户创建行为树时，为用户进行节点的推荐、错误检查以及节点属性的智能映射，可进一步将行为树实例转化为知识图谱实例进行保存。相应地，可如图1B所示，进一步包括如下步骤：
步骤104，将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储。所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系或节点属性的多条边。
相应地，在用户创建行为树时，便可基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。本实施例中可通过相似性计算与比较、深度学习等方法来实现上述功能。图5中显示了用户构建行为树时的节点推荐示意图。如图5的灰色虚线框内所示，在为顺序节点创建动作节点时，系统推荐了动作5、动作6和动作7等，并分别给出了匹配度75％、50％、20％等，以方便用户进行选择。
本实施例中，知识图谱的语法可如图6和图7所示。
图6示出了本实施例中知识图谱中类的定义和层次关系。如图6所示，类“owl:Thing”下有三个子类：工作流、节点和数据对象。
其中，节点类对应行为树的节点，共有三个子类：开始节点(StartNode)、控制节点(ControlNode)和动作节点(ActionNode)。具体的逻辑节点类型如顺序、并行、选择器、循环、计数等是控制节点的子类，特定的动作节点类型如模型训练器、模型转换 器等是动作节点的子类。
工作流类对应行为树的工作流。
数据对象是节点属性、全局变量列表和黑板的基类。节点属性类具有输入和输出等子类。其中，黑板用于记录节点实例列表。
图7示出了本实施例中知识图谱中对象属性和数据属性的示意图。如图7所示，其中：
-“具有全局变量(hasGVL)”创建从工作流到全局变量列表的关系，用于工作流实例查找属于它的全局变量列表。
-“具有黑板(hasBlackBoard)”创建从工作流到黑板的关系，用于工作流实例查找属于它的黑板，从而查找到属于它的节点实例列表。
-“具有开始节点(hasStartNode)”创建工作流到开始节点(StartNode)的关系，用于工作流实例查找开始节点实例，使工作流实例可以查找工作流中的所有节点实例。
-“具有控制节点(hasControlNode)”创建从节点到控制节点(ControlNode)的关系。
-“具有动作节点(hasActionNode)”创建从节点到动作节点(ActionNode)的关系。
-“具有较高优先级(hasHigherPriority)”创建同一父节点下的节点之间的顺序关系，用于表示优先级顺序。例如，在图2中，A1和A2在顺序节点下，A1将在A2之前执行，因此在知识图谱中应该有一个关系：A1具有较高优先级A2(A1 hasHigherPriority A2)。
-“具有相同优先级(hasSamePriority)”创建同一父节点下的节点之间的并列关系。例如，在图2中，A3和A4在并行节点下，它们将同时执行，因此在知识图谱中应该有一个关系：A3具有相同优先级A4(A3 hasSamePriority A4)。
-“具有属性(hasProperties)”用于动作节点(ActionNode)和控制节点(ControlNode)来表示属性，范围应该是属性(properties)类。
-“具有输入(hasInputs)”用于动作节点(ActionNode)和控制节点(ControlNode)来表示输入，范围应该是输入(inputs)类。
-“具有输出(hasOutputs)”用于动作节点(ActionNode)和控制节点(ControlNode)来表示输出，范围应该是输出(outputs)类。
例如，针对图2所示的行为树实例，可得到如图8所示的知识图谱实例。
此外，本发明实施例中，还进一步提供日志服务、错误处理服务、用户管理服务和 项目管理服务等。
以上对本发明实施例中AI模型的设备部署方法进行了详细描述，下面再对本发明实施例中AI模型的设备部署系统进行详细描述。本发明实施例中的AI模型的设备部署系统可用于实施本发明实施例中的AI模型的设备部署方法，对于本发明系统实施例中未详细披露的细节可参见本发明方法实施例中的相应描述，此处不再一一赘述。
图9为本发明实施例中一种AI模型的设备部署系统的示例性结构图。如图9所示，该系统可与外部的多个数据库进行交互。例如，用于存储和管理预先训练的AI模型的AI模型库10、用于存储用于再训练AI模型的数据集的数据集库20和用于存储历史行为树的知识图谱实例的知识图谱库30。如图9所示，该系统可包括：节点存储模块910、工程编辑模块920、行为树引擎930、知识图谱模块940和任务模板库950。
其中，节点存储模块910中设置有用于构建各种行为树的节点，所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点，如模型导入器、模型训练器、模型转换器、模型适配器、以及应用程序部署器等。
具体实现时，每个动作节点可由接口部分和实现部分组成；对于实现部分，每个动作节点是一个容器化应用程序；对于接口部分，每个动作节点是一个可在所述工程编辑模块的人机交互界面上进行拖放、连接及属性配置的图形元素；每个动作节点被执行时，所述动作节点接口部分的属性配置中的输入将被读取并传输给所述动作节点的实现部分，并在所述实现部分完成对应操作后，操作结果将转换为所述动作节点接口部分的属性配置中的输出。相应地，本实施例中的节点存储模块910可包括：接口存储库911和节点服务模块912。其中，接口存储库911用于存储每个动作节点的接口部分，并存储开始节点和控制节点；节点服务模块912用于存储每个动作节点的实现部分。
此外，如图9所示，本实施例中，还可以进一步包括公共服务模块960，用于存储进行日志服务、错误处理服务、用户管理服务和项目管理服务等的应用程序。
工程编辑模块920用于提供用户基于所述节点存储模块中的节点进行行为树构建并配置的人机交互界面，并根据所述用户的构建和配置操作，生成用于部署当前AI模型的当前行为树。具体实现时，当前行为树可以XML格式存储，或者也可以以Python等脚本S格式存储。具体可根据实际需要确定，此处不对其进行限定。
在一个实施方式中，工程编辑模块920可包括行为树编辑器921、节点配置器922、 变量配置器923、数据黑板924和运行/调试监控器925等。
其中，行为树编辑器921用于提供用户对当前行为树的组成和连接关系进行构建和编辑的人机交互。
节点配置器922用于提供用户对每个节点的属性进行配置的人机交互。
变量配置器923用于提供用户对全局变量进行配置的人机交互。
数据黑板924用于将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储或查看，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
运行/调试监控器925用于向行为树引擎930发送调试或运行指令，接收并呈现行为树引擎930反馈的中间执行结果和最终执行结果。
行为树引擎930用于在接收到所述调试或运行指令时，按照当前行为树的执行逻辑，执行所述当前行为树对应的工作流，并在用户指定目标设备后，将所述当前AI模型构建为AI应用并部署到用户指定的目标设备40中。同时，将执行过程中的中间结果及最终结果反馈给工程编辑模块920的运行/调试监控器925显示。此外，执行期间还对各节点状态进行监控，并对于执行过程中遇到的异常及错误进行相应的处理。之后，便可利用部署后的AI模型监控设备的运行状态。
在一个实施方式中，行为树引擎930可具体包括：行为树执行器931、状态监控器932和错误处理器933。
其中，行为树执行器931按照所述当前行为树的执行逻辑，执行所述当前行为树对应的工作流，并在用户指定目标设备后，将预先确定的当前AI模型构建为AI应用并部署到用户指定的目标设备中。具体地，在执行当前行为树中的每个节点时，读取所述节点的接口部分的属性配置中的输入，并将其转换到节点的实现部分；在实现部分完成特定的操作后，将操作结果转换为所述节点的接口部分的输出提供给所述工程编辑模块920进行显示。
状态监控器932用于在行为树执行器931执行所述当前行为树期间对各节点状态进行监控。例如，在运行/调试当前行为树时，用户可以添加一些节点或自定义数据进行监控。
错误处理器933用于对执行过程中遇到的异常及错误进行相应的处理。例如，可为每个节点设置错误(Error)属性，如可包括错误ID和错误消息等，这样便可以通过错误属性获取详细的错误信息。
知识图谱模块940用于将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储；所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系或节点属性的多条边。知识图谱模块940还用于在用户构建所述当前行为树时，基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。
任务模板库950用于存储至少一个基于所述节点存储模块中的节点构建的行为树模板；所述至少一个行为树模板包括：对应于不同AI应用类型或不同设备类型的至少一个固定行为树模板；和/或，一通用的基于自动机器学习的智能行为树模板。
相应地，工程编辑模块920可进一步用于响应于用户通过所述人机交互界面对所述任务模板库的查询请求，将所述至少一个行为树模板提供给所述用户进行选择。在确定所述用户选择针对一AI应用类型或一目标设备类型的固定行为树模板时，根据所述用户对所述固定行为树模板进行的第一配置以及调整或不调整，生成所述当前行为树；所述第一配置包括：确定当前AI模型及其需要部署的目标设备；在确定用户选择所述智能行为树模板时，根据所述用户对所述固定行为树模板进行的第二配置，生成所述当前行为树；所述第二配置包括：确定当前的AI应用类型和需要部署的目标设备；所述智能行为树模板能够根据所述用户配置的AI应用类型从一AI模型库中自动选取合适的当前AI模型。
实际应用中，AI模型的设备部署系统也可仅包括上述模块中的部分模块。例如，在一个实施方式中，AI模型的设备部署系统可仅包括上述的节点存储模块910、工程编辑模块920和行为树引擎930；或者，仅包括上述的节点存储模块910、工程编辑模块920、行为树引擎930和知识图谱模块940；又或者，仅包括上述的节点存储模块910、工程编辑模块920、行为树引擎930和任务模板库950。此处不对其进行限定。
图10为本申请实施例中又一种AI模型的设备部署系统的结构示意图，该系统可用于实施图1中所示的方法，或实现图9中所示的系统。如图10所示，该系统可包括：至少一个存储器1001、至少一个处理器1002和至少一个显示器1003。此外，还可以包括一些其它组件，例如通信端口等。这些组件通过总线1004进行通信。
其中，至少一个存储器1001用于存储计算机程序。在一个实施方式中，该计算机程序可以理解为包括图9所示的AI模型的设备部署系统的各个模块。此外，至少一个存储器1001还可存储操作系统等。操作系统包括但不限于：Android操作系统、Symbian操作系统、Windows操作系统、Linux操作系统等等。
至少一个处理器1002用于调用至少一个存储器1001中存储的计算机程序，执行本申请 实施例中所述的AI模型的设备部署方法。处理器1002可以为CPU，处理单元/模块，ASIC，逻辑模块或可编程门阵列等。其可通过所述通信端口进行数据的接收和发送。
至少一个显示器1003用于显示人机交互界面。
具体地，至少一个处理器1002用于调用至少一个存储器1001中存储的计算机程序使所述系统执行上述任一实施方式中的AI模型的设备部署方法中的操作。
需要说明的是，上述各流程和各结构图中不是所有的步骤和模块都是必须的，可以根据实际的需要忽略某些步骤或模块。各步骤的执行顺序不是固定的，可以根据需要进行调整。各模块的划分仅仅是为了便于描述采用的功能上的划分，实际实现时，一个模块可以分由多个模块实现，多个模块的功能也可以由同一个模块实现，这些模块可以位于同一个设备中，也可以位于不同的设备中。
可以理解，上述各实施方式中的硬件模块可以以机械方式或电子方式实现。例如，一个硬件模块可以包括专门设计的永久性电路或逻辑器件(如专用处理器，如FPGA或ASIC)用于完成特定的操作。硬件模块也可以包括由软件临时配置的可编程逻辑器件或电路(如包括通用处理器或其它可编程处理器)用于执行特定操作。至于具体采用机械方式，或是采用专用的永久性电路，或是采用临时配置的电路(如由软件进行配置)来实现硬件模块，可以根据成本和时间上的考虑来决定。
此外，本申请实施例中还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序能够被一处理器执行并实现本申请实施例中所述的AI模型的设备部署方法。具体地，可以提供配有存储介质的系统或者装置，在该存储介质上存储着实现上述实施例中任一实施方式的功能的软件程序代码，且使该系统或者装置的计算机(或CPU或MPU)读出并执行存储在存储介质中的程序代码。此外，还可以通过基于程序代码的指令使计算机上操作的操作系统等来完成部分或者全部的实际操作。还可以将从存储介质读出的程序代码写到插入计算机内的扩展板中所设置的存储器中或者写到与计算机相连接的扩展单元中设置的存储器中，随后基于程序代码的指令使安装在扩展板或者扩展单元上的CPU等来执行部分和全部实际操作，从而实现上述实施方式中任一实施方式的功能。用于提供程序代码的存储介质实施方式包括软盘、硬盘、磁光盘、光盘(如CD-ROM、CD-R、CD-RW、DVD-ROM、DVD-RAM、DVD-RW、DVD+RW)、磁带、非易失性存储卡和ROM。可选择地，可以由通信网络从服务器计算机上下载程序代码。
从上述方案中可以看出，由于本发明实施例中，用行为树图来表示AI模型的部署操作流程，即将AI模型部署过程中的相关操作封装成独立的节点，使节点具有可重用性，实现 了特定业务与工程平台的解耦，然后以行为树的形式组织节点，生成从开发到实际部署的直观的AI应用操作流程，从而降低了AI模型的设备部署复杂度。
此外，通过采用知识图谱的形式来表示和保存各个行为树实例，并在此基础上实现节点、属性或工作流的推荐功能，从而可进一步降低AI模型的设备部署复杂度。
进一步地，通过将针对各种AI应用类型的行为树模板以及集成了自动机器学习的行为树模板设置在模板任务库中，使得用户可以从中选择与其任务相对应的行为树，而不是从头开始创建，进一步大大降低了AI模型的设备部署复杂度。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (15)

  1. AI模型的设备部署方法，其特征在于，包括：
    接收用户基于预先设置的用于构建各种行为树的节点构建并配置完成的当前行为树；所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点；
    执行所述当前行为树对应的工作流，将预先确定的当前AI模型构建为AI应用并部署到用户指定的目标设备中。
  2. 根据权利要求1所述的AI模型的设备部署方法，其特征在于，所述当前行为树为：用户通过添加、连接和配置开始节点、所需的控制节点和所需的动作节点构建而成的行为树；或者为：
    从任务模板库中，根据所需的AI应用类型或需要部署的目标设备类型选取对应的固定行为树模板，并基于所述固定行为树模板进行第一配置以及调整或不调整后得到的行为树；所述第一配置包括：确定当前AI模型及其需要部署的目标设备；或者为：
    从任务模板库中选取一基于自动机器学习的智能行为树模板，并基于所述智能行为树模板进行第二配置后得到的行为树；所述第二配置包括：确定AI应用类型和需要部署的目标设备，由所述智能行为树模板根据所述AI应用类型从一AI模型库中自动确定合适的AI模型。
  3. 根据权利要求2所述的AI模型的设备部署方法，其特征在于，进一步包括：将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储；所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系或节点属性的多条边。
  4. 根据权利要求3所述的AI模型的设备部署方法，其特征在于，进一步包括：在用户构建所述当前行为树时，基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。
  5. 根据权利要求1至4中任一项所述的AI模型的设备部署方法，其特征在于，每个节点均由接口部分和实现部分组成；对于实现部分，每个节点是一个容器化应用程序；对于接口部分，每个节点是一个可在人机交互界面上进行拖放、连接及属性配置的图形元素；
    所述当前行为树中的每个节点被执行时，所述节点接口部分的属性配置中的输入将被读取并传输给所述节点的实现部分，并在所述实现部分完成对应操作后，操作结果将转换为所述节点接口部分的属性配置中的输出。
  6. 根据权利要求1至4中任一项所述的AI模型的设备部署方法，其特征在于，进一步包括：将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储或查看，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
  7. 根据权利要求1至4中任一项所述的AI模型的设备部署方法，其特征在于，所述当前行为树中的动作节点包括：
    顺序执行的如下动作节点：模型导入器、模型训练器和应用程序部署器；或者包括：
    顺序执行的如下动作节点：模型导入器、模型训练器、模型转换器和应用程序部署器；或者包括：
    顺序执行的如下动作节点：模型适配器、模型训练器和应用程序部署器；
    其中，模型导入器用于从模型存储区导入经过训练的AI模型；
    模型训练器用于对导入的训练模型进行新数据源或训练参数的再训练；
    模型转换器用于将训练好的模型从源格式转换为目标格式；
    模型适配器用于基于自动机器学习根据当前确定的AI应用类型从模型库中自动选取合适的AI模型；
    应用程序部署器用于将AI模型打包为AI应用，并部署到指定的目标设备。
  8. AI模型的设备部署系统，其特征在于，包括：
    节点存储模块，其中设置有用于构建各种行为树的节点，所述节点包括开始节点、控制节点和动作节点；其中，一个行为树用于表征一个工作流，控制节点用于实现所述工作流中的逻辑控制，动作节点用于实现所述工作流中的业务操作，且所述动作节点包括：将AI模型部署过程中的相关业务操作封装而成的可重用的独立的动作节点；
    工程编辑模块，用于提供用户基于所述节点存储模块中的节点进行行为树构建并配置的人机交互界面，并根据所述用户的构建和配置操作，生成用于部署当前AI模型的当前行为树；
    行为树引擎，用于执行所述当前行为树对应的工作流，将所述当前AI模型构建为AI应用并部署到用户指定的目标设备中。
  9. 根据权利要求8所述的AI模型的设备部署系统，其特征在于，进一步包括：知识图谱模块，用于将所述当前行为树基于知识图谱的语法构建成一知识图谱实例进行存储；所述知识图谱实例包括：分别表示所述行为树中的各个节点的节点，以及表示节点之间关系 或节点属性的多条边。
  10. 根据权利要求9所述的AI模型的设备部署系统，其特征在于，所述知识图谱模块进一步用于在用户构建所述当前行为树时，基于存储的历史知识图谱实例为所述用户进行控制节点和/或动作节点的推荐、错误检查以及节点属性的智能映射。
  11. 根据权利要求8所述的AI模型的设备部署系统，其特征在于，进一步包括：
    任务模板库，用于存储至少一个基于所述节点存储模块中的节点构建的行为树模板；所述至少一个行为树模板包括：对应于不同AI应用类型或不同设备类型的至少一个固定行为树模板；和/或，一通用的基于自动机器学习的智能行为树模板；
    所述工程编辑模块进一步用于响应于用户通过所述人机交互界面对所述任务模板库的查询请求，将所述至少一个行为树模板提供给所述用户进行选择；在确定所述用户选择针对一AI应用类型或一目标设备类型的固定行为树模板时，根据所述用户对所述固定行为树模板进行的第一配置以及调整或不调整，生成所述当前行为树；所述第一配置包括：确定当前AI模型及其需要部署的目标设备；在确定用户选择所述智能行为树模板时，根据所述用户对所述固定行为树模板进行的第二配置，生成所述当前行为树；所述第二配置包括：确定当前的AI应用类型和需要部署的目标设备；所述智能行为树模板能够根据所述用户配置的AI应用类型从一AI模型库中自动选取合适的当前AI模型。
  12. 根据权利要求8至11中任一项所述的AI模型的设备部署系统，其特征在于，所述每个节点均由接口部分和实现部分组成；对于实现部分，每个节点是一个容器化应用程序；对于接口部分，每个节点是一个可在所述工程编辑模块的人机交互界面上进行拖放、连接及属性配置的图形元素；
    所述行为树引擎在执行所述当前行为树中的每个动作节点时，所述节点接口部分的属性配置中的输入将被读取并传输给所述节点的实现部分，并在所述实现部分完成对应操作后，操作结果将转换为所述节点接口部分的属性配置中的输出；
    所述节点存储模块包括：接口存储库和节点服务模块；其中，所述接口存储库用于存储每个节点的接口部分；所述节点服务模块用于存储每个节点的实现部分。
  13. 根据权利要求8至11中任一项所述的AI模型的设备部署系统，其特征在于，所述工程编辑模块包括：
    行为树编辑器，用于提供用户对当前行为树的组成和连接关系进行构建和编辑的人机交互；
    节点配置器，用于提供用户对每个节点的属性进行配置的人机交互；
    变量配置器，用于提供用户对全局变量进行配置的人机交互；
    节点实例列表模块，用于将所述当前行为树中添加的每个节点转换为一个节点实例添加到一节点实例列表中进行存储或查看，所述节点实例列表以列表形式或树状形式呈现所述当前行为树中的所有节点。
  14. AI模型的设备部署系统，其特征在于，包括：至少一个存储器和至少一个处理器，其中：
    所述至少一个存储器用于存储计算机程序；
    所述至少一个处理器用于调用所述至少一个存储器中存储的计算机程序执行如权利要求1至7中任一项所述的AI模型的设备部署方法。
  15. 计算机可读存储介质，其上存储有计算机程序；其特征在于，所述计算机程序能够被一处理器执行并实现如权利要求1至7中任一项所述的AI模型的设备部署方法。
PCT/CN2021/109878 2021-07-30 2021-07-30 Ai模型的设备部署方法、系统及存储介质 WO2023004806A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2021/109878 WO2023004806A1 (zh) 2021-07-30 2021-07-30 Ai模型的设备部署方法、系统及存储介质
EP21951396.7A EP4365731A1 (en) 2021-07-30 2021-07-30 Device deployment method for ai model, system, and storage medium
CN202180100662.4A CN117677930A (zh) 2021-07-30 2021-07-30 Ai模型的设备部署方法、系统及存储介质
US18/292,570 US20240265275A1 (en) 2021-07-30 2021-07-30 Device Deployment Method for AI Model, System, and Storage Medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/109878 WO2023004806A1 (zh) 2021-07-30 2021-07-30 Ai模型的设备部署方法、系统及存储介质

Publications (1)

Publication Number Publication Date
WO2023004806A1 true WO2023004806A1 (zh) 2023-02-02

Family

ID=85087403

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/109878 WO2023004806A1 (zh) 2021-07-30 2021-07-30 Ai模型的设备部署方法、系统及存储介质

Country Status (4)

Country Link
US (1) US20240265275A1 (zh)
EP (1) EP4365731A1 (zh)
CN (1) CN117677930A (zh)
WO (1) WO2023004806A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116048542A (zh) * 2023-02-11 2023-05-02 之江实验室 一种计算机视觉深度学习模型的优化部署方法与装置
CN117131001A (zh) * 2023-10-25 2023-11-28 北京优特捷信息技术有限公司 一种分布式大数据有序智能分管方法、系统及存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107899244A (zh) * 2017-11-29 2018-04-13 武汉秀宝软件有限公司 一种ai模型的构建方法及系统
CN110244941A (zh) * 2019-06-17 2019-09-17 三角兽（北京）科技有限公司 任务开发方法、装置、电子设备及计算机可读存储介质
CN111079244A (zh) * 2019-06-20 2020-04-28 中国航天系统科学与工程研究院 一种基于行为树的分布式仿真平台
US10803392B1 (en) * 2017-03-10 2020-10-13 Amazon Technologies, Inc Deploying machine learning-based models
CN110297640B (zh) * 2019-06-12 2020-10-16 北京三快在线科技有限公司 模型部署的方法、装置、存储介质及电子设备
CN111861020A (zh) * 2020-07-27 2020-10-30 深圳壹账通智能科技有限公司 模型部署方法、装置、设备及存储介质
CN112181367A (zh) * 2019-07-04 2021-01-05 北京航天长峰科技工业集团有限公司 一种可配置化业务流程模型
CN112395072A (zh) * 2019-08-14 2021-02-23 北京三快在线科技有限公司 模型部署的方法、装置、存储介质及电子设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7643892B2 (en) * 2007-09-28 2010-01-05 Rockwell Automation Technologies, Inc. Historian integrated with MES appliance

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10803392B1 (en) * 2017-03-10 2020-10-13 Amazon Technologies, Inc Deploying machine learning-based models
CN107899244A (zh) * 2017-11-29 2018-04-13 武汉秀宝软件有限公司 一种ai模型的构建方法及系统
CN110297640B (zh) * 2019-06-12 2020-10-16 北京三快在线科技有限公司 模型部署的方法、装置、存储介质及电子设备
CN110244941A (zh) * 2019-06-17 2019-09-17 三角兽（北京）科技有限公司 任务开发方法、装置、电子设备及计算机可读存储介质
CN111079244A (zh) * 2019-06-20 2020-04-28 中国航天系统科学与工程研究院 一种基于行为树的分布式仿真平台
CN112181367A (zh) * 2019-07-04 2021-01-05 北京航天长峰科技工业集团有限公司 一种可配置化业务流程模型
CN112395072A (zh) * 2019-08-14 2021-02-23 北京三快在线科技有限公司 模型部署的方法、装置、存储介质及电子设备
CN111861020A (zh) * 2020-07-27 2020-10-30 深圳壹账通智能科技有限公司 模型部署方法、装置、设备及存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116048542A (zh) * 2023-02-11 2023-05-02 之江实验室 一种计算机视觉深度学习模型的优化部署方法与装置
CN116048542B (zh) * 2023-02-11 2023-10-31 之江实验室 一种计算机视觉深度学习模型的优化部署方法与装置
CN117131001A (zh) * 2023-10-25 2023-11-28 北京优特捷信息技术有限公司 一种分布式大数据有序智能分管方法、系统及存储介质
CN117131001B (zh) * 2023-10-25 2024-01-16 北京优特捷信息技术有限公司 一种分布式大数据有序智能分管方法、系统及存储介质

Also Published As

Publication number Publication date
US20240265275A1 (en) 2024-08-08
EP4365731A1 (en) 2024-05-08
CN117677930A (zh) 2024-03-08

Similar Documents

Publication Publication Date Title
US11681502B2 (en) Industrial automation domain-specific language programming paradigm
US11822906B2 (en) Industrial programming development with a converted industrial control program
US20220138004A1 (en) System and method for automated production and deployment of packaged ai solutions
US7117449B1 (en) Method and apparatus to present an integrated process modeler
US11455149B2 (en) Object configuration utilities
Hallerbach et al. Context-based configuration of process variants
US20040128001A1 (en) Method and apparatus for an integrated process modeller
US11256755B2 (en) Tag mapping process and pluggable framework for generating algorithm ensemble
CN111797969A (zh) 神经网络模型的转换方法及相关装置
US20180224821A1 (en) Rich contextualization of automation data
WO2023004806A1 (zh) Ai模型的设备部署方法、系统及存储介质
WO2023004805A1 (zh) 工作流建模实施系统、方法及存储介质
EP4459525A1 (en) Workflow generation method and system, and medium and program product
US10747941B2 (en) Tag mapping process and pluggable framework for generating algorithm ensemble
CN115495069B (zh) 基于模型驱动的煤炭工业软件流程实现方法、装置及设备
EP1588349A4 (en) METHOD AND APPARATUS FOR INTEGRATED PROCESS MODELER
Lochmann et al. An integrated view on modeling with multiple domain-specific languages
CN117806654B (zh) 一种基于Tekton的自定义云原生DevOps流水线系统及方法
Zhao et al. An AutomationML extension towards interoperability of 3D virtual commissioning software applications
Giachetti et al. Integration of domain-specific modelling languages and UML through UML profile extension mechanism.
US11835941B2 (en) Industrial automation smart object parent/child data collection propagation
Marín et al. A general framework for the development of MDD projects
Ahrendt et al. Integrated Formal Methods: 15th International Conference, IFM 2019, Bergen, Norway, December 2–6, 2019, Proceedings
Heinrich et al. MDA applied: a task-model driven tool chain for multimodal applications
CN117940891A (zh) 管理应用、尤其是开发包括事件工件的应用的方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21951396

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180100662.4

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2021951396

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021951396

Country of ref document: EP

Effective date: 20240129

NENP Non-entry into the national phase

Ref country code: DE