CN114168114A

CN114168114A - Operator registration method, device and equipment

Info

Publication number: CN114168114A
Application number: CN202210124808.5A
Authority: CN
Inventors: 李常宝; 薛铮; 贾贺; 袁媛; 肖广华
Original assignee: CETC 15 Research Institute
Current assignee: CETC 15 Research Institute
Priority date: 2022-02-10
Filing date: 2022-02-10
Publication date: 2022-03-11

Abstract

The embodiment of the specification discloses an operator registration method, device and equipment. The method comprises the following steps: acquiring data to be processed; determining the port type of the data to be processed based on the data type to which the data to be processed belongs; and based on the port type, carrying out operator registration through operator basic information configuration and operator port information configuration. By adopting the method provided by the embodiment of the specification, data conversion operators among different types of ports can be realized, data conversion among different development languages of the same port type can be realized, data transmission among the same development languages of the same port type can be realized, execution of development operators based on multiple development languages in the same flow can be realized, and the expansibility of the operators can be improved. When the operator is registered, code development is not needed, the operator development process is simplified, and automatic adaptation of the operator registration stage is realized.

Description

Operator registration method, device and equipment

Technical Field

The present disclosure relates to the field of computer technology and big data, and in particular, to an operator registration method, apparatus, and device.

Background

With the continuous development of computers, data processing methods based on algorithms such as machine learning and deep learning are gradually popularized. However, the implementation of algorithms, especially new algorithms, requires professional programming knowledge. Therefore, the algorithm is packaged into operators, and the operators are integrated into a platform capable of executing the process, the platform aims to reduce the professional requirements on operators, and the realization of the algorithm process can be completed only by building the process. However, this type of platform uses separate tasks as the operator agent, each task having embedded therein fixed logic and input-output configuration items. The logic sequence among operators is fixed, the front-back relation of the operators is fixed, the flow solidification is serious, and the operators are difficult to develop; the method comprises the steps that the rear end form of an operator is fixed in a mode of a base class, execution entities of all operators inherit the base class, and processable data types, processing logic methods and data insights of the operator are defined in the base class. The operator rear-end entity realizes the logic function, the insight function and the data input and output function of the operator. However, since all functions in all operator entities need to be implemented by developers, that is, each operator is a project, the operator development workload is large, and operators developed by other development languages cannot be added. Operator development is complex, and operator library expansion capability is poor.

Therefore, a new method is needed, which can simplify operator development, reduce operator development workload, and make the process more flexible.

Disclosure of Invention

The embodiment of the specification provides an operator registration method, an operator registration device and operator registration equipment, which are used for solving the following technical problems: the existing operator platform takes a single task as an operator main body, and each task is embedded with a fixed logic and input/output configuration item. The logic sequence among operators is fixed, the front-back relation of the operators is fixed, the flow solidification is serious, and the operators are difficult to develop; the method comprises the steps that the rear end form of an operator is fixed in a mode of a base class, execution entities of all operators inherit the base class, and processable data types, processing logic methods and data insights of the operator are defined in the base class. The operator rear-end entity realizes the logic function, the insight function and the data input and output function of the operator. However, since all functions in all operator entities need to be implemented by developers, that is, each operator is a project, the operator development workload is large, and operators developed by other development languages cannot be added. Operator development is complex, and operator library expansion capability is poor.

In order to solve the above technical problem, the embodiments of the present specification are implemented as follows:

an embodiment of the present specification provides an operator registration method, including:

acquiring data to be processed;

determining the port type of the data to be processed based on the data type to which the data to be processed belongs;

and based on the port type, carrying out operator registration through operator basic information configuration and operator port information configuration.

An embodiment of the present specification further provides an operator registration apparatus, including:

the acquisition module acquires data to be processed;

the port determining module is used for determining the port type of the data to be processed based on the data type of the data to be processed;

and the registration module is used for registering the operator through operator basic information configuration and operator port information configuration based on the port type.

An embodiment of the present specification further provides an electronic device, including:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to:

acquiring data to be processed;

In the operator registration method provided in the embodiment of the present specification, the type of data to be processed is taken as a fulcrum, and the type of the data to be processed is defined as a preset port type. The port type is used as a reference, data conversion operators among different types of ports are provided, data conversion among different development languages of the same port type is achieved, data transmission among the same development languages of the same port type can be achieved, execution of development operators based on multiple development languages in the same flow is achieved, and expansibility of the operators is improved. When the operator is registered, the operator configuration interface can be completed through simple dragging only by filling in the basic information and the port information of the operator, an operator entity is generated, code development is not needed, the operator development process is simplified, and automatic adaptation in the operator registration stage is realized.

Drawings

In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.

Fig. 1 is a schematic diagram of an operator registration method provided in an embodiment of the present specification;

fig. 2 is a schematic diagram of an operator structure provided in an embodiment of the present specification;

fig. 3 is a schematic diagram of an operator registration process provided in an embodiment of the present specification;

FIG. 4 is a schematic diagram illustrating generation of an operator configuration interface according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an experiment execution logic provided in an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of script execution logic provided by an embodiment of the present specification;

FIG. 7 is a schematic diagram illustrating an operator implementation provided in an embodiment of the present disclosure;

fig. 8 is a schematic diagram of an operator registration apparatus according to an embodiment of the present disclosure.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any inventive step based on the embodiments of the present disclosure, shall fall within the scope of protection of the present application.

Fig. 1 is a schematic diagram of an operator registration method provided in an embodiment of the present specification. As shown in fig. 1, an embodiment of the present specification provides an operator registration method, including:

step S101: and acquiring data to be processed.

In the embodiment of the present specification, the data to be processed is data subjected to operator registration.

Step S103: and determining the port type of the data to be processed based on the data type of the data to be processed.

In an embodiment of the present specification, the data types include: the method comprises the steps of developing a data type based on a development framework, a data type based on a native database, data based on a document, data based on a graph and a visual data type based on an operator model;

the determining the data port data to be processed based on the data type to which the data to be processed belongs specifically includes:

if the data to be processed belongs to a data type based on a development framework, the port type of the data to be processed is a first port;

if the data to be processed belongs to the data type based on the native database, the port type of the data to be processed is a second port;

if the data to be processed belongs to the data type based on the document, the port type of the data to be processed is a third port;

if the data to be processed belongs to the data type based on the graph, the port type of the data to be processed is a fourth port;

and if the data to be processed belongs to the visualization data type based on the operator model, the port type of the data to be processed is a fifth port.

In one embodiment of the present description, the development framework may be spark, dataset, pandas: daaframe et al, the specific type of development framework does not constitute a limitation of the present application. The method provided by the embodiment of the specification can be applied to various development frameworks.

In this specification embodiment, the first port is used to carry data in a development framework. In an embodiment of the present specification, the development framework is a spark framework, and the data type port based on the spark framework is a dataset port, and the type of the data type port is used for carrying dataset type data in spark.

In the embodiment of the specification, the native database is data derived from a development framework and is used for bearing connection information of various data sources in different development frameworks. In one embodiment of the present description, the second port is a dataSource port.

In one embodiment of the present description, the third port is used to carry data of a document in a preset format. In one embodiment of the present specification, a document is a json-formatted file based on the data type of the document. In one embodiment of the present disclosure, the third port is a json port, and the json port is used for carrying various data in json format.

In one embodiment of the present description, the fourth port is used to carry data for drawing various graphics. In a specific embodiment of the present specification, the fourth port is echarts-based graphics data, and the fourth port is a graph port for carrying data for drawing various graphics.

In one embodiment of the present description, the fifth port is used to carry visualization data for various models as well as the algorithmic model itself. In one particular embodiment of the present description, the fifth port is a model port.

It should be noted that the operator registration method provided in this specification may support the foregoing five port types. The first port, the second port, the third port, the fourth port and the fifth port are only schematic illustrations of the ports, and do not represent the sequence of the ports.

Step S105: and based on the port type, carrying out operator registration through operator basic information configuration and operator port information configuration.

In this embodiment of the present specification, the performing operator registration through operator basic information configuration and operator port information configuration based on the port type specifically includes:

and configuring operator basic information and operator port information based on an operator execution body base class and the port type to complete operator registration, wherein the operator execution body base class is integrated by an operator execution entity.

In this embodiment of the present specification, the performing, based on the operator execution base class and the port type, configuration of operator basic information and operator port information to complete operator registration specifically includes:

based on the operator execution body base class and the port type, carrying out operator basic information and operator port information configuration;

and performing interface configuration based on the configured operator basic information and the configured operator port information to complete operator registration.

In an embodiment of the present specification, the operator execution base class includes an operator execution base class and a port execution base class, where the execution base class includes basic information of a port, an abstract execution function of an operator, and a generation function of operator insight data.

In an embodiment of the present specification, the development language of the operator execution entity includes: one or more of JAVA/SCALA development language, PYTHON development language and C/C + + development language;

wherein,

an operator execution entity based on JAVA/SCALA development language directly inherits an operator base class to realize the function of an operator;

the function of an operator execution entity based on the PYTHON development language is divided into two execution modes of whether an input port and an output port exist or not, and the operator function is realized;

the operator execution entity function based on the C/C + + development language realizes the operator function by calling the dynamic library.

In an embodiment of this specification, the operator execution entity based on the JAVA/SCALA development language directly inherits the operator base class to implement an operator function, which specifically includes:

an operator execution entity based on JAVA/SCALA development language directly inherits an operator base class, receiving and processing of operator input data and operator configuration parameters and transmission of output port data are completed through an operator execution function, and visual data are generated through an operator insight function, so that an operator function is realized.

In this embodiment of the present specification, the functions of the operator execution entity based on the PYTHON development language are divided into two execution modes, i.e., whether there are input/output ports, to implement the operator functions, and specifically include:

if the operator execution entity based on the PYTHON development language has an input/output port, converting input data into a file in a preset format;

based on the file with the preset format, realizing an operator function according to an execution script path and an execution environment of the execution script;

if the operator execution entity based on the PYTHON development language does not have an input/output port, calling a script, executing console information through the script, fixing the console information to a preset type of output port, and realizing an operator function.

In an embodiment of the present specification, the operator execution entity is an operator execution entity based on a JAVA/SCALA development language, and the JAVA/SCALA operator execution entity directly inherits the operator base class to implement an operator execution function and an operator insight function. And the operator execution function completes three steps of receiving operator input data and parameters, processing the input data and transmitting output port data. The operator insight function generates visual data according to the specific data processing process, processing mode, applied algorithm/model and the like of the operator.

In an embodiment of the present specification, the operator execution entity is an operator execution entity based on a PYTHON development language, a logic implementation main body of the PYTHON operator is a PYTHON script, that is, functions of the PYTHON operator execution entity are divided into two execution modes according to whether an operator has an input/output port, and if the operator has the input/output port, the execution process includes receiving input data and parameters, converting input data parameters, executing the script, and receiving script output. The method comprises the following specific steps: 1) and processing input data, and writing port data information into a file in a json file format. Each port in the file is divided into three fields of a port name (portName), a port type (portType) and port data (portValue), wherein the port name type corresponds to port information in an operator entity, and the port data are processed aiming at different types: performing file disk-dropping operation on the dataset port data, persisting the dataset port data, generating a reading path of corresponding data, and setting the data reading path as a port data field of the type of port; the data source port data is database connection information corresponding to the port; json port data is port content transmitted in a port; graph port data is processed in the same way as json port data. 2) And starting a new process to directly execute the script according to the path of the executed script and the script execution environment. The executed script needs to create a description according to the provided operator, before the operator executes the main body, the class method provided in the provided transfer _ input _ class. 3) The method for providing the PYTHON script is the same as the input port processing mode, the transfer _ input _ class is used for writing port data into an intermediate file, and corresponding output port contents are obtained in a PYTHON operator execution entity according to file contents. If the execution script has no input/output port, the PYTHON operator execution entity is only responsible for calling the script, acquiring the script execution console information, and fixedly arranging the information in the json type output port so as to know the execution condition and the error information of the operator.

In one embodiment of the present specification, the operator execution entity is an operator execution entity based on a C/C + + development language, and the function of the C/C + + operator execution main body is to call a compiled C/C + + executable file through the provided execution entity. And in the execution class, JNI is used for calling a compiled DLL dynamic library, and the main function of an operator is realized by the dynamic library.

In an embodiment of this specification, the file in the preset format is a json file, and a name of the file in the preset format includes: port name, port type and port data;

wherein,

the port name and the port type and the port information in the operator entity;

the port data is a set of execution modes of the first port, the second port, the third port, the fourth port and the fifth port.

In the embodiment of the present specification, the basic information of the operator includes an operator ID and an operator name, and the port information of the operator is a basis for connecting lines between operators.

The operator ID and the operator name are unique identifiers of operators, wherein the operator ID is generated dynamically, the operator ID aims at different processes, and the same operator name can have different operator IDs.

In order to further understand the operator registration method provided in the embodiments of the present specification, the following description will be given in conjunction with the logical structure of an operator. Fig. 2 is a schematic diagram of an operator structure provided in an embodiment of the present specification. In this embodiment, the logical structure of the operator is an independent entity that performs data processing work by using input data as a driver and output data as a target, and in a specific implementation, the logical structure of the operator includes: operator back end, operator front end and operator database. The operator rear end is an execution entity, and the operator front end can realize a visual operator component which can be dragged, edited, configured and executed.

In the logical structure of the operator, the operator front end comprises basic information of the operator, port information of the operator, a configuration interface of the operator and an insight interface of the operator. The basic information of the operator at least comprises an operator ID and an operator name, port information of the operator is the basis for connecting the operators, a configuration interface of the operator is used for configuring variable parameters in operator processing logic and/or parameters required by the operator, and an insight interface of the operator is used for visually displaying the operator processing result and part of operator displayable content.

In the logical structure of the operator, the rear end of the operator is the core of the operator and is used for completing the main function of the operator; the operator database is used as a data base to ensure the uniqueness of the operator, and the connection between the operator rear end and the operator front end is carried out; the visualization component in the operator front end is a key, is used for completing the creation, configuration and display of operators in the process, and is a precondition for the execution of an operator rear end entity.

The operator back end provides a data conversion operator among different types of ports, so that data conversion among different development languages of the same type of ports is realized, and data transmission among the same development voice of the same type of ports is realized.

In this embodiment, the operator backend is mainly used for information executed by an operator, and includes: the execution logic of the operator, the parameters required by the operator and the information of the input and output ports of the operator. The operator database mainly stores basic information of operators and basic information of operator execution. The operator front end is mainly used for basic information of an operator, parameters required by the operator and information of an operator input/output port.

In the embodiment of the present specification, the operator creating process mainly determines an operator function, and implements a function at the back end of an operator, that is, implements development of an operator execution entity. And (4) registering an operator according to the operator function, further testing the operator function, and finally completing the operator establishing process.

The operator rear end is equivalent to a server, when operator registration is carried out, data only needs to be called from the operator rear end or the server, configuration is carried out through operator IDs and operator names, registration is carried out, and an operator entity is generated through dragging.

In the embodiment of the present specification, the operator database includes basic information of an operator, port information of the operator, development language of the operator, and backend configuration information of the operator. The basic information of the operator at least comprises an operator ID and an operator name, and the basic information of the operator also comprises an operator alias and an operator classification. The operator ID is the English name of the operator, the operator alias is the Chinese name of the operator, and the operator ID is used for selecting the operator name when the operator front end registers the operator. The operator back end configuration information comprises information of positioning execution main bodies such as a class path of an operator execution class, a jar packet path, a script/DLL file path and the like.

The front end interface of the operator is positioned by the basic information of the operator, the back end of the operator is positioned by the execution information of the operator, and the one-to-one correspondence of the front end information and the back end information of the operator is completed. Basic information such as names, ids and the like of the specified operators cannot be repeated during database design, so that the uniqueness of the operators is ensured.

In the embodiment of the specification, the operator registration is to generate information of an operator database, and determine a configuration interface according to the input/output port type, so as to automatically generate an operator front-end component. In a specific embodiment, operator registration generates information of an operator database through operation of a registration interface, and an operator front-end component is automatically generated by using an operator ID and an operator name during registration as unique identifiers according to an input/output port type selected by the registration interface and a configured configuration interface.

In the embodiments of the present specification, the operator functions are based on business scenarios. For example, in one embodiment of the present specification, if the svm algorithm is implemented to process data, then a svm operator is required. According to the execution flow of the algorithm, the algorithm needs to perform relevant processing on input data, generate a corresponding model, and then transmit the model. Therefore, the svm operator requires a data input port, a data processing process, and a model output port. The svm operator is only an exemplary illustration of the application, and the function determination of other operators also adopts the method, generates corresponding models according to the execution flow of the algorithm, and then transfers the models.

The operator execution entity needs to integrate an operator execution body base class, and the operator execution body base class defines a processing mode of an operator input/output port, namely, the execution port data is acquired and transmitted according to a port name and a port type.

In a specific embodiment, the operator execution body base class may include an operator base class and a port base class, where the operator base class is used to record an abstract execution function of an operator and/or a generation function of insight data; the port base class is used to record port data.

In one embodiment of the present specification, a system-defined portpad base class is used as a bearer entity for port data, and the bearer entity records the name, type, data structure and additional information of a port. The name of the port can be used as the port identification.

The additional information is the additional information aiming at the data to be processed, the specific content of the additional information is determined by the data type of the data to be processed, and the additional information can simplify the calculation process. In a specific embodiment, the dataset port (first port) carries statistical information of current data, and the total number, average number, maximum and minimum values of the statistical data are stored in the port data when the data are generated for the first time. Therefore, the calculation of statistical information of the same data for multiple times is reduced, and the time loss in the calculation process is reduced.

In the operator executive body base class, an abstract executive function and/or a generating function of the insight data of the operator are/is further defined. In a specific embodiment, the abstract execution function of the operator is used for completing the implementation of the execution function of the operator, and the return values are all map structure data with the name of the output port being key, so as to transmit the output port data backwards. And the generating function of the operator insight data is used for generating the unique insight data of the operator and providing data support for the display of the front-end insight page.

Uploading an operator jar packet and formulating a class path of an operator execution class if the registered operator development language is JAVA/SCALA in the registration process; and if the uploaded operator development language is PYTHON, displaying a third-party library supported in the current PYTHON executable environment on a registration interface, and providing a button for installing other libraries. Supporting the uploaded file to be converted into a py file (a single script) or a zip package (a plurality of py files and operators use data files), selecting a unique execution file as an operator execution main body entry file after the uploading is successful, and completing the specific logic function of the operator by calling other scripts or independently executing the file; if the uploading operator development language is C/C + +, the uploading file is supported and then converted into a C/cpp file, and the gcc command is executed to realize automatic compiling by providing a compiling environment in the uploading process. And after uploading, selecting the only dynamic library file as an operator execution main body entry file, wherein the dynamic library completes the specific logic function of the operator by calling other dynamic libraries or independently executing. And (4) carrying out operator configuration interface self-defining generation after the operator basic information is filled, and finally finishing operator registration.

And in the operator registration stage, a configuration interface self-generation step can be carried out after the configuration of the basic information of the operator is finished. The selection of various parameter configuration schemes such as a text box, a radio box, a multi-selection box and the like can be carried out according to the parameter requirements. And after the configuration scheme is selected, filling information such as corresponding parameter names, default parameters, parameter check rules and the like. And clicking to determine after all the parameter configuration schemes are configured, generating a configuration interface corresponding to the operator, and completing the self-generating step of the front-end configuration interface.

And after the operator is created and registered, an operator registration frame can be obtained. On the operator registry framework, existing operator lists can be seen. In practical application, operators can be selected according to requirements, connection among the operators is carried out according to operator execution logic by taking ports as a reference, an experiment flow is formed, and parameter configuration is carried out on configuration interfaces of all the operators. And after clicking an execution button, determining an operator execution sequence according to an operator connection relation in the experiment flow file by a program, generating an operator execution entity in a reflection mode according to operator execution information when starting to execute an operator, executing an operator execution function, completing the realization of an operator function, executing an operator insight data generation function, assigning the result to an insight interface of a corresponding operator through a related interface provided by the program, and completing the generation of the operator insight interface. And sequentially executing each operator to finish the execution of the experimental flow. And in the operator executing process, the execution log of each operator can be obtained on the operator front-end interface.

In order to further understand the operator registration method provided in the embodiments of the present specification, the following description will be made in conjunction with the operator registration process. Fig. 3 is a schematic diagram of an operator registration process provided in an embodiment of the present specification, and as shown in fig. 3, based on an operator registration framework, to-be-processed data completes operator registration at an operator front end. The operator registration mainly comprises the steps of operator basic information configuration and operator port configuration. After the configuration of the basic information of the operator and the configuration of the operator port are completed, the generation of an operator configuration interface is further performed based on the configuration of operator execution information, so that the generation of an operator front-end component is completed, and meanwhile, the information of an operator database is generated.

To further understand the foregoing description about the generation of the operator configuration interface, the following description will be made with reference to specific schematic diagrams. Fig. 4 is a schematic diagram of generation of an operator configuration interface provided in an embodiment of the present specification, and as shown in fig. 4, the generation of the operator configuration interface includes:

based on an operator registration framework, carrying out operator basic information configuration and operator port configuration;

determining whether additional configuration items are needed for operator execution; in the embodiment of the present specification, the additional configuration item refers to whether an interface needs to be called, whether configuration information other than an operator itself is needed, and whether configuration content needs an additional presentation manner. Generally, in specific implementation, the judgment of the additional configuration item is often performed based on the function of an operator, the function of the general operator is complex, the configuration information is changed according to a superior operator, the configuration information needs to call an interface to obtain, and cannot be simply input, or the configuration interface has a complex expression form.

If the operator needs to execute the additional configuration item, judging whether a configuration interface template provided by the configuration interface meets the requirement of the additional configuration item, if so, selecting a corresponding template, or generating the configuration interface by dragging by using a provided related assembly; and if the configuration interface does not meet the requirements, customizing the configuration interface and generating the configuration interface. It should be noted that the custom configuration interface can be saved as a template, and used as a configuration interface template for subsequent use.

And if the operator does not need additional configuration items, the related parameter configuration is carried out through a simple component, and a configuration interface is automatically generated.

And after the configuration interface is obtained based on the method, operator registration is completed, and an operator component is generated.

For further understanding of the operator registration method provided in the embodiments of the present specification, the following description will be made in conjunction with a logic diagram of operator execution. Fig. 5 is a schematic diagram of an experiment execution logic provided in an embodiment of the present specification, and as shown in fig. 5, in an operator execution process, input port data input, input data processing, output port data generation, and insight data generation are mainly performed, and after an operator is registered by using the registration method provided in the embodiment of the present specification, execution of different execution bodies can be realized, and data conversion operators between different types of ports can be realized, data conversion between different development languages of the same port type can be realized, and data transfer between the same development languages of the same port type can be realized.

It is further understood that the aforementioned experiment execution logic, as embodied in the script, is further described. Fig. 6 is a schematic diagram of a script execution logic provided in an embodiment of this specification, and as shown in fig. 6, in an operator execution process, based on port data conversion and a script execution position, a script is executed through a script execution tool class, so as to implement a back-end logic of an operator.

In order to further understand the operator registration method provided in the embodiment of the present specification, the following description will be given in conjunction with the execution process of an operator, taking JAVA development language as an example, and fig. 7 is a schematic diagram of the execution of the operator provided in the embodiment of the present specification. As shown in fig. 7, when the process starts to execute the current operator, it is first determined whether the current operator belongs to the system operator, and if the current operator belongs to the system operator, the process acquires an operator execution class file, so as to generate an execution class entity; if the current operator does not belong to the system operator, the process acquires an operator execution software package (jar file) path and an execution class file, and then an execution class entity is generated.

Further, after the execution type entity is obtained, whether the execution type entity is a JAVA operator needs to be further judged, if the execution type entity is the JAVA operator, operator execution is completed, output port data is generated, and insight data is generated; if the operator is not JAVA, the conversion of input data is needed, an execution script or other executable files are generated, and then the converted data is output, so that the execution of the operator is completed, output port data is generated, and insight data is generated.

By adopting the method provided by the embodiment of the specification, data conversion between different development languages with the same port type can be realized, data transmission between the same development languages with the same port type can be realized, execution of development operators based on multiple development languages in the same flow is realized, and the expansibility of the operators is improved. When the operator is registered, the operator configuration interface can be completed through simple dragging only by filling in the basic information and the port information of the operator, an operator entity is generated, code development is not needed, the operator development process is simplified, and automatic adaptation in the operator registration stage is realized.

The above details describe an operator registration method, and accordingly, the present specification further provides an operator registration apparatus, as shown in fig. 8. Fig. 8 is a schematic diagram of an operator registration apparatus provided in an embodiment of the present specification, where the apparatus includes:

an obtaining module 801, which obtains data to be processed;

a port determining module 803, which determines the port type of the data to be processed based on the data type to which the data to be processed belongs;

the registration module 805 performs operator registration through operator basic information configuration and operator port information configuration based on the port type.

Further, the data types include: the method comprises the steps of developing a data type based on a development framework, a data type based on a native database, data based on a document, data based on a graph and a visual data type based on an operator model;

Further, the registering an operator through operator basic information configuration and operator port information configuration based on the port type specifically includes:

Further, the performing, based on the operator execution base class and the port type, configuration of operator basic information and operator port information to complete operator registration specifically includes:

Further, the operator execution body base class includes an operator execution body base class and a port execution body base class, and the execution body base class includes basic information of a port, an abstract execution function of an operator, and a generation function of operator insight data.

Further, the development language of the operator execution entity includes: one or more of JAVA/SCALA development language, PYTHON development language and C/C + + development language;

wherein,

Further, the operator execution entity based on the JAVA/SCALA development language directly inherits the operator base class to realize the operator function, which specifically includes:

Further, the functions of the operator execution entity based on the PYTHON development language are divided into two execution modes, namely whether the execution mode has an input/output port, so that the operator functions are realized, and the method specifically comprises the following steps:

Further, the file in the preset format is a json file, and the name of the file in the preset format includes: port name, port type and port data;

wherein,

Further, the basic information of the operators comprises operator IDs and operator names, and the port information of the operators is the basis for connecting the operators.

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

acquiring data to be processed;

wherein,

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, and the nonvolatile computer storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and the relevant points can be referred to the partial description of the embodiments of the method.

The apparatus, the electronic device, the nonvolatile computer storage medium and the method provided in the embodiments of the present description correspond to each other, and therefore, the apparatus, the electronic device, and the nonvolatile computer storage medium also have similar advantageous technical effects to the corresponding method.

In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the various elements may be implemented in the same one or more software and/or hardware implementations in implementing one or more embodiments of the present description.

As will be appreciated by one skilled in the art, the present specification embodiments may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The description has been presented with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the description. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only an example of the present specification, and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An operator registration method, the method comprising:

acquiring data to be processed;

2. The method of claim 1, wherein the data types comprise: the method comprises the steps of developing a data type based on a development framework, a data type based on a native database, data based on a document, data based on a graph and a visual data type based on an operator model;

3. The method according to claim 1, wherein the operator registration is performed by operator basic information configuration and operator port information configuration based on the port type, specifically including:

4. The method of claim 3, wherein the configuring of the operator basic information and the operator port information based on the operator execution base class and the port type to complete operator registration comprises:

5. The method of claim 3, wherein the operator execution body base class comprises an operator execution body base class and a port execution body base class, the execution body base class comprising basic information of a port, an abstract execution function of an operator, and a generation function of operator insight data.

6. The method of claim 3, wherein the operator execution entity development language comprises: one or more of JAVA/SCALA development language, PYTHON development language and C/C + + development language;

wherein,

7. The method of claim 6, wherein the operator execution entity based on the JAVA/SCALA development language directly inherits the operator base class to implement an operator function, and specifically comprises:

8. The method as claimed in claim 6, wherein the PYTHON development language based operator execution entity has two execution modes of input and output ports, and the method for implementing the operator function comprises:

9. The method of claim 8, wherein the file of the preset format is a json file, and the name of the file of the preset format comprises: port name, port type and port data;

wherein,

10. The method of claim 1, wherein the basic information of the operators comprises operator IDs and operator names, and the port information of the operators is the basis for connecting lines between the operators.

11. An operator registration apparatus, the apparatus comprising:

the acquisition module acquires data to be processed;

12. An electronic device, comprising:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

acquiring data to be processed;