CN116974638A

CN116974638A - Data processing method, apparatus, device, computer program, and storage medium

Info

Publication number: CN116974638A
Application number: CN202310661658.6A
Authority: CN
Inventors: 熊彬
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-06-05
Filing date: 2023-06-05
Publication date: 2023-10-31

Abstract

The application provides a data processing method, a device, an electronic device, a software program and a storage medium, wherein the related embodiments can be applied to various scenes such as cloud technology, cloud security, intelligent traffic and the like, and the method comprises the following steps: acquiring a data processing request; acquiring each sub data and an identifier of each sub data in response to the data processing request; performing de-duplication on the plurality of sub-data according to the identifier of each sub-data to obtain target data; according to the visual configuration information, performing visual data pruning on the target data to obtain target data to be rendered; the method and the device can automatically perform visual rendering on the acquired data to adapt to different visual use requirements of the target object.

Description

Data processing method, apparatus, device, computer program, and storage medium

Technical Field

The present application relates to data processing technology of computer hardware devices, and in particular, to a data processing method, an apparatus, an electronic device, a computer program product, and a storage medium.

Background

After the state and information of the networking equipment are collected through a preset network protocol in a private cloud scene, the state and information of the networking equipment can be timely known by the target object through visual display, but in the related technology, the problems of repeated data and incapability of slicing are caused after the state and information of the networking equipment are collected, and the target object needs to be manually subjected to visual processing, so that the processing difficulty of the target object is increased.

Disclosure of Invention

In view of this, the embodiments of the present application provide a data processing method, apparatus, electronic device, software program, and storage medium, which can perform data processing on the state and information of the networking device collected through the preset network protocol, and also automatically perform visual rendering on the obtained data, so as to adapt to different visual use requirements of the target object.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a data processing method, which comprises the following steps:

acquiring a data processing request; the data processing request comprises visual configuration information of data to be processed; the data to be processed comprises a plurality of sub-data;

Acquiring each sub data and an identifier of each sub data in response to the data processing request;

performing de-duplication on the plurality of sub-data according to the identifier of each sub-data to obtain target data;

according to the visual configuration information, performing visual data pruning on the target data to obtain target data to be rendered;

and carrying out data rendering on the target data to be rendered to obtain a visual processing result.

The embodiment of the application also provides a data processing device, which comprises:

the information transmission module is used for acquiring a data processing request; the data processing request comprises visual configuration information of data to be processed; the data to be processed comprises a plurality of sub-data;

the information processing module is used for responding to the data processing request and acquiring each sub data and an identifier of each sub data;

the information processing module is used for carrying out de-duplication on the plurality of sub-data according to the identifier of each sub-data to obtain target data;

the information processing module is used for pruning the visual data according to the visual configuration information to obtain the target data to be rendered;

The information processing module is used for carrying out data rendering on the target data to be rendered to obtain a visual processing result.

In some embodiments, the information processing module is further configured to send, by using a network management system, a data acquisition request to a proxy server in response to the data processing request, where the data acquisition request is configured to request the proxy server to perform data interaction with a network device according to a preset network protocol, so as to acquire the plurality of sub-data; and receiving each piece of sub data sent by the proxy server after responding to the data acquisition request through the network management system.

In some embodiments, the information processing module is further configured to obtain, by using the network management system, a globally unique identifier corresponding to each sub-data and a cluster name of a cluster corresponding to the corresponding sub-data; and combining the globally unique identifier and the cluster name for each sub-data to obtain the identifier of the corresponding sub-data.

In some embodiments, the information processing module is further configured to traverse identifiers of the plurality of sub-data and determine duplicate identifiers in the plurality of sub-data; and deleting the sub data corresponding to the repeated identifier from the data to be processed to obtain the target data.

In some embodiments, the information processing module is further configured to determine, according to the visual configuration information, a set of keys excluding types and a set of keys including types; determining a unique key set and a repeated key set in the target data; pruning the keys of the target data according to the key set of the exclusion type, the unique key set, the key set of the inclusion type and the repeated key set to obtain the keys of the target data to be rendered; and taking the target data corresponding to the key of the target data to be rendered as the target data to be rendered.

In some embodiments, the information processing module is further configured to determine, according to the visual configuration information, data of a hidden type in the target data; adding keys corresponding to the hidden type data into an initial exclusion type key set to obtain an exclusion type key set; determining reserved type data in the target data according to the visual configuration information; and adding the key corresponding to the data with the reserved type into the key set with the initial type to obtain the key set with the type.

In some embodiments, the information processing module is further configured to traverse the target data to obtain each key corresponding to the target data; determining the occurrence times of each key according to the traversing result of the target data; and determining a unique key set and a repeated key set corresponding to the target data according to the occurrence times of each key.

In some embodiments, the information processing module is further configured to cache the target data into a network data server or a cache interface; responding to the received visualized data pruning request through a network management system, and detecting cache aging in a cache interface to obtain a cache aging detection result; when the cache aging detection result is greater than or equal to the cache expiration date, the network management system acquires the target data from a network data server; and when the cache aging detection result is smaller than the cache validity period, the network management system acquires the target data from the cache interface.

In some embodiments, the information processing module is further configured to prune the key of the target data according to the repeated key set and the unique key set to obtain a first pruning result; adding keys in the key set comprising the type into the first pruning result to obtain a second pruning result; and deleting keys in the key set of the exclusion type in the second pruning result to obtain the keys of the target data to be rendered.

In some embodiments, the information processing module is further configured to determine a visualization gallery corresponding to the target data to be rendered; and carrying out visualization processing on the target data to be rendered through the visualization gallery to obtain a data visualization processing result, wherein the data visualization processing result is matched with the visualization configuration information.

In some embodiments, the information processing module is further configured to obtain identification information of a key of the target data to be rendered; determining a first visual gallery corresponding to identification information of keys of the target data to be rendered; or analyzing the visual configuration information to obtain a second visual gallery matched with the target object of the network management system.

The embodiment of the application also provides electronic equipment, which comprises:

a memory for storing executable instructions;

and the processor is used for realizing the preamble data processing method when the executable instructions stored in the memory are run.

The embodiment of the application also provides a computer readable storage medium which stores executable instructions which when executed by a processor realize the data processing method.

The embodiment of the application also provides a computer program product, which comprises a computer program or instructions, wherein the computer program or instructions realize the data processing method when being executed by a processor.

The embodiment of the application has the following beneficial effects:

1) The embodiment of the application obtains each sub data and the identifier of each sub data by responding to the data processing request; performing de-duplication on the plurality of sub-data according to the identifier of each sub-data to obtain target data; thus, the duplication of the sub data can be realized, thereby reducing the sub data redundancy caused by the duplication of the sub data and reducing the storage cost of the sub data.

2) According to the visual configuration information, performing visual data pruning on the target data to obtain target data to be rendered; and carrying out data rendering on the target data to be rendered to obtain a visual processing result. Therefore, the acquired data can be automatically visually rendered so as to adapt to different visual use requirements of the target object.

Drawings

Fig. 1 is a schematic diagram of an application scenario of a data processing method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of an implementation of a data processing method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a process for pruning visual data of target data according to an embodiment of the present application;

FIG. 4 is a diagram illustrating pruning of keys of target data according to an embodiment of the present application

FIG. 5 is a schematic diagram illustrating a data processing method according to an embodiment of the present application;

fig. 6 is a schematic diagram of an acquiring process of sub data of a plurality of private cloud servers in an embodiment of the present application;

FIG. 7 is a schematic diagram of a pruning layer data deduplication process according to an embodiment of the present application;

FIG. 8 is a schematic diagram of visual data pruning logic of a pruning layer according to an embodiment of the present application;

FIG. 9 is a schematic diagram of an algorithm logic of visual data pruning of a pruning layer according to an embodiment of the present application;

FIG. 10 is a schematic diagram of a visual gallery in a data processing method according to the present application;

fig. 11 is a schematic structural diagram of an electronic device for executing the data processing method provided by the present application.

Detailed Description

The present application will be further described in detail with reference to the accompanying drawings, for the purpose of making the objects, technical solutions and advantages of the present application more apparent, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

Before describing embodiments of the present application in further detail, the terms and terminology involved in the embodiments of the present application will be described, and the terms and terminology involved in the embodiments of the present application will be used in the following explanation.

1) In response to a condition or state that is used to represent the condition or state upon which the performed operation depends, the performed operation or operations may be in real-time or with a set delay when the condition or state upon which it depends is satisfied; without being specifically described, there is no limitation in the execution sequence of the plurality of operations performed.

2) Based on the conditions or states that are used to represent the operations that are being performed, one or more of the operations that are being performed may be in real-time or with a set delay when the conditions or states that are being relied upon are satisfied; without being specifically described, there is no limitation in the execution sequence of the plurality of operations performed.

3) Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud computing business model application-based network technology, information technology, integration technology, management platform technology, application technology and the like can be collectively called to form a resource pool, and the resource pool is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

4) Cloud Security (Cloud Security) refers to a generic term for Security software, hardware, target objects, institutions, and secure Cloud platforms based on Cloud computing business model applications. Cloud security fuses emerging technologies and concepts such as parallel processing, grid computing, unknown virus behavior judgment and the like, acquires the latest information of Trojan horse and malicious programs in the Internet through abnormal detection of software behaviors in a network by a large number of netlike clients, sends the latest information to a fictive engine server for automatic analysis and processing, and distributes solutions of viruses and Trojan horse to each client.

5) A Server cluster (Server cluster) refers to a cluster of at least two servers that together perform the same service, and appears to a client as if there is only one Server. The server cluster can use a plurality of computers to perform parallel computation so as to obtain high computation speed, and can also use a plurality of computers to perform backup, so that any machine breaks the whole system or can normally operate. The data processing method provided by the application can be applied to cloud server use scenes and distributed server use scenes, and can be used for realizing state detection and fault repair of server hard disks in different use scenes. For example, cloud servers (CVM, cloud Virtual Machine) are a simple, efficient, secure, reliable computing service with flexible processing capabilities. The management mode is simpler and more efficient than the traditional single physical server. The target object can quickly create or release any plurality of cloud servers for the business process of the target object without purchasing hardware in advance, and store the data of the cloud server target object. The data and the program of the target object in the use environment of the distributed server can be distributed in a plurality of servers instead of being located on one server, and similarly, the use environment of the distributed server also needs to be configured with a large number of hard disks, and the data processing method provided by the application also needs to realize the visual display of the operation data of the network equipment in the cloud server or the computing resource data of the cloud server.

6) A network management system (NMS Network Management System) for managing devices, subsystems, routers, switches, hubs, and some auxiliary devices in the network; a system-wide network view is provided to a network administrator.

Before explaining the data processing method provided by the application, firstly explaining the defects in the related art, the inventor finds out in the research that aiming at the use scene of private cloud, in the related art, after collecting the state and information of networking equipment through a simple network management protocol (SNMP Simple Network Management Protocol) in the private cloud scene, the state and information of the networking equipment can be timely known by the target object, but in the related art, the problems of repeated data and incapability of slicing are caused after collecting the state and information of the networking equipment, and an automatic visualization solution is lacked, the target object needs to be manually subjected to the visualization processing, and the manual visualization processing is slow and has poor accuracy.

In order to solve the above-mentioned drawbacks, the present application provides a data processing method, which not only can perform data processing on the state and information of the network equipment collected by the preset network protocol, but also can automatically perform visualization processing on the obtained data so as to adapt to different use requirements of the target object.

Fig. 1 is a schematic view of an application scenario of a data processing method according to an embodiment of the present application, referring to fig. 1, with the continuous development of computer technology, a cloud server (CVM Cloud Virtual Machine) may provide a secure and reliable elastic computing service, and may also provide different instance types to satisfy a specific usage scenario of a target object. The electronic device 100 may include two different types of terminals or electronic devices, namely, the terminal 10-1 and the terminal 10-2, and the target object may be selected according to the use requirement of the private cloud use scenario. The terminals (including the terminal 10-1 and the terminal 10-2) are provided with corresponding clients capable of executing the data processing function, wherein the clients acquire different information from the corresponding cloud server 200 through the network 300 for the terminals (including the terminal 10-1 and the terminal 10-2) and can deploy different services in the cloud server. The terminal is connected to the server 200 through the network 300, and the network 300 may be a wide area network or a local area network, or a combination of the two, and uses a wireless link to implement data transmission. When the cloud server 200 corresponds to a private cloud product, the data processing method provided by the application can be applied to the following three use scenes, so that the management of equipment or cloud server computing resources in the private cloud scene is realized, and the visual presentation is carried out to a target object.

1) Enterprise network management based on private cloud scenarios: the network management system may be used for device management in an enterprise network, such as enterprise servers, and routers, switches, etc. that interact with the enterprise servers. The network management system polls proxy servers in the private cloud clusters, operation data of each accessed network device can be obtained, and then the obtained operation data of each accessed network device is stored in the data storage server for subsequent data processing and visual display.

2) Cloud computing resource management based on private cloud scenarios: the data processing method provided by the application can be applied to management of the cloud computing resources of multiple clusters, and the data conditions of the cloud computing resources are obtained through a network management system, and then data processing and visual display are carried out. The visualized display data can help a cloud computing resource manager to better manage cloud computing resources and optimize the utilization rate of the cloud computing resources.

In some embodiments, the cloud server or the cloud server cluster provides examples with different types composed of a CPU, a memory, a storage and a network, and stores the service data of the target object in a hard disk of the cloud server, but in the operation of the cloud server, a large amount of resource fragments are generated in the task processing process, so that the redundancy of resources is caused, the processing speed of the cloud server network is reduced, the task processing speed is influenced, and the use effect of the cloud server network is influenced. As shown in connection with fig. 1, in an embodiment provided by the present application, cloud server 200 may be written in software code environments of different programming languages, and code objects may be different types of code entities. For example, in software code in the C language, a code object may be a function. In software code in the JAVA language, a code object may be a class.

3) Internet of things equipment management based on private cloud scene: the data processing method provided by the application can be applied to management of the Internet of things equipment. And polling proxy servers in the private cloud clusters through the network management system, acquiring the operation data of each Internet of things device, and then storing the acquired operation data of each Internet of things device in a corresponding data storage server so as to facilitate subsequent data processing and visual display. Therefore, an Internet of things device manager can be helped to better manage the Internet of things device through the visual display effect, and the running efficiency and stability of the device are optimized.

Referring to fig. 2, fig. 2 is a schematic flow chart of an implementation of a data processing method according to an embodiment of the present application, where the embodiment of the present application may be implemented in combination with Cloud technology (Cloud technology) refers to a hosting technology that integrates serial resources such as hardware, software, and networks in a wide area network or a local area network to implement calculation, storage, processing, and sharing of data, and may also be understood as a generic term of network technology, information technology, integration technology, management platform technology, application technology, and the like applied based on a Cloud computing business model. Background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites, so cloud technologies need to be supported by cloud computing.

It should be noted that cloud computing is a computing mode, which distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can acquire computing power, storage space and information service as required. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed. As a basic capability provider of cloud computing, a cloud computing resource pool platform, referred to as a cloud platform for short, is established, and is generally called an infrastructure as a service (IaaS Infrastructure as a Service), and multiple types of virtual resources are deployed in the resource pool for external clients to select for use. The cloud computing resource pool mainly comprises: computing devices (which may be virtualized machines, including operating systems), storage devices, and network devices. And giving different operation rights to the target object subjected to data processing through the cloud server.

Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside. In the related art, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data identifier (ID IDentity), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object. The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID Redundant Array of Independent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.

It will be appreciated that the execution body of the steps shown in fig. 2 may be implemented by a terminal running the data processing apparatus alone or by a server running the data processing apparatus. Of the processing steps shown in fig. 2, a server is taken as an implementation example, and the steps shown in fig. 2 are described below.

Step 201: and acquiring a data processing request, wherein the data processing request comprises visual configuration information of data to be processed, and the data to be processed comprises a plurality of sub-data.

In some embodiments, since the application scenarios of the data processing method provided by the present application are different, the data to be processed may also be of different types, for example: when the data processing method is applied to enterprise network management in a private cloud-based scenario, the data to be processed comprises a plurality of sub-data which can be operation data (including but not limited to CPU power, memory residual quantity change and pressure data of a server) of an enterprise server and data transmission quantity change data of a router which performs information interaction with the enterprise server.

In some embodiments, when the data processing method is applied to a cloud computing resource management scenario based on a private cloud scenario, computing resources of each cloud server in the cloud server cluster may be detected, and the obtained sub-data may be node requirement information of each node in the cloud server cluster, where the node requirement information includes at least one of the following: the demand information for the selector component of any target cluster node, the affinity demand information for any target cluster node, and the stain and tolerance demand information for any target cluster node.

In some embodiments, when the data processing method is applied to a cloud computing resource management scenario based on a private cloud scenario, the source of the sub data may be: analyzing the content of the maximum available copy number calculation request of any cloud server to obtain resource demand information of each copy in the copy group, wherein the resource demand information comprises at least one of the following: CPU core number requirement information, memory requirement information and extensible resource requirement information. Wherein the affinity requirement information refers to affinity rules between the application instance and other application instances; for application instances with affinity, they may be deployed on the same compute node; for application instances that do not have affinity, they cannot be deployed on the same compute node. The labels are used to indicate the affinity between the application instance and other application instances.

The tolerance is the attribute data of the key value type on the application instance (such as pod) for configuring the stain of the tolerable computing node, and the scheduler can only schedule the application instance to the computing node where the application instance can tolerate the node stain.

In some embodiments, the replica group information of the target cluster in the child data may further include: label information, stain information, workload information, persistent volume declaration (PVC), persistent Volume (PV) information, and the like.

In some embodiments, label (Label) information is used to identify whether the cloud server 200 has affinity for an application instance (pod). The tagged computing cloud server 200 has affinity for the application instance, i.e., the application instance can be deployed on the tagged cloud server 200. The blobs are key property data defined on top of the compute nodes for letting the compute nodes refuse to schedule the application instance to run in the corresponding cloud server 200.

In some embodiments, the workload information may include: the resources such as a workload API object (StatefulSet), thread (replyment), replica controller (ReplicaSet), and controller (Daemoset) that ensures that all (some) nodes run one replica are used to manage stateful applications. The resource information includes the number of application instances and affinity rules for the application instances. Only application instances that fit the affinity rules of the workload information may be deployed on the cloud server 200.

In some embodiments, persistent volume information, including node Affinity (node Affinity information), directly affects the scheduling result, that is, if an application instance to be deployed needs to use persistent volume resources, cloud server 200 acts as a node whose node Affinity needs to be adapted to the node Affinity contained in the persistent volume information, and the application instance may only be deployed on the present computing node. The persistent volume declaration refers to declaration information for a PV, and may include: selected node (selected-node) information directly affects the scheduling result.

In some embodiments, the information of the workload may include: and resources such as Stateful Set, replyment, replicaset, daemon Set and the like. The resource information includes the number of application instances, affinity rules for the application instances, and the like. Only application instances that fit the affinity rules of the workload may be deployed on the cloud server 200 as a computing node. For different types of the sub data, the application is not particularly limited, and the types of the sub data can be flexibly adjusted according to different application scenes of the data processing method provided by the application.

Step 202: in response to the data processing request, each sub-data and an identifier of each sub-data are obtained.

In some embodiments, the acquiring each sub-data may be implemented by a preset network protocol (e.g., a simple network protocol), and the network management system sends a data acquisition request to the proxy server in response to the data processing request, where the data acquisition request is used to request the proxy server to perform data interaction with the network device according to the preset network protocol, so as to acquire multiple sub-data. For example: the SNMP protocol may be a data acquisition message (SNMP Get Request message) that contains an identification of the network device to be acquired (OID Object Identifier). The network management system receives each sub-data transmitted by the proxy server after responding to the data acquisition request, for example: the proxy server can send the plurality of sub-data obtained after analysis to the network management system through a data reply message (SNMP Get Response message) of the SNMP protocol, and finally the network management system sends all the obtained sub-data to the network data server for storage.

In some embodiments, the identifier of each sub-data may be combined by a globally unique identifier (Globally Unique Identifier, GUID) and the cluster name (clusterName) of the cluster to which the corresponding sub-data corresponds, where, because the identifiers of the sub-data in different clusters may overlap, it is necessary to generate a globally unique identifier and combine with a prefix cluster name to { clusterName } { GUID } as the unique identifier of the sub-data.

In some embodiments, the globally unique identifier may be generated according to a pseudo-random algorithm, and a 128-bit random number is calculated with reference to equation (1) using the real-time (up to milliseconds) of the network management system as a seed value. Combining the physical address of the network management system with the global unique identifier name space of the hardware equipment of the network management system to generate a unique 6-byte identifier, and finally combining the 128-bit random numbers generated by the first two parts according to a certain rule to generate a 128-bit global unique identifier (formula 1):

wherein T is _low T is the real-time division _mid T is seconds of real time _high For real-time milliseconds, version is the MAC address (physical address) of the network management system, variant is the GUID namespace of the hardware devices of the network management system, nodeid is the combination rule of 128-bit random numbers, Is an exclusive or operation.

Step 203: and de-duplicating the plurality of sub-data according to the identifier of each sub-data to obtain target data.

In some embodiments, all sub-data that may be acquired first, the pending data list and the deduplicated data list (the initial, duplicate data list is empty) may be configured. And traversing the data in the data list to be processed, and for each piece of sub-data, inquiring the combination set of the GUID and the OID which have appeared by taking the combination of the GUID and the OID of the sub-data as a key word. If not, the data is added to the de-duplicated data list and the combination of its GUID and OID is added to the combined set of GUID and OID that has already occurred. If so, this data is indicated as having occurred and needs to be removed. By de-duplicating all the sub-data, the data redundancy of the sub-data can be effectively reduced, and the data storage cost of the sub-data is reduced.

In some embodiments, the de-duplicated sub-data may be stored in a network data database, or in order to facilitate use of a visualization layer, all sub-data after pruning may be cached in a cache interface (Redis), where the storage manner of the sub-data is not particularly limited in the present application.

Step 204: and pruning the visual data according to the visual configuration information to obtain the target data to be rendered.

As the target data can be cached in the network data server or the cache interface; therefore, the network management system can respond to the received visualized data pruning request to detect the cache aging in the cache interface, and a cache aging detection result is obtained; when the cache aging detection result is greater than or equal to the cache validity period, the network management system acquires target data from the network data server; and when the cache aging detection result is smaller than the cache validity period, the network management system acquires target data from the cache interface. Thereby ensuring the accuracy of the target data.

Referring to fig. 3, fig. 3 is a schematic diagram illustrating a process of pruning visual data of target data in an embodiment of the present application, and it is to be understood that an execution body of the steps shown in fig. 3 may be implemented by a terminal running a data processing apparatus alone or may be implemented by a server running the data processing apparatus. Of the processing steps shown in fig. 3, a server is taken as an implementation example, and the steps shown in fig. 3 are described below.

Step 301: according to the visual configuration information, a key set excluding types and a key set including types are determined.

In some embodiments, the key set used in the data processing method provided by the present application has four kinds:

1) exclude type key sets (excludedKeys), 2) unique key sets (uniqueKeys), 3) include type key sets (includeedkeys) and 4) duplicate key sets (duplicate keys), each of which is described below.

In some embodiments, determining the excluded type of key set and the included type of key set from the visual configuration information may be accomplished by:

determining hidden type data in the target data according to the visual configuration information; adding the key corresponding to the hidden type data into the key set of the initial exclusion type to obtain the key set of the exclusion type; determining reserved type data in the target data according to the visual configuration information; and adding the key corresponding to the reserved type data into the key set which initially comprises the type to obtain the key set which comprises the type. Taking the sub data as node requirement information of the cloud server 200 as an example, the requirement information includes: 1) demand information for selector components of target cluster nodes, 2) affinity demand information for target cluster nodes, 3) stain and tolerance demand information for target cluster nodes, the target object 1 hopes to only show the demand information for selector components of target cluster nodes in the data visualization effect, thus, the key set of the type is [ "demand information for selector components of target cluster nodes" ], the key set of the type is excluded [ "affinity demand information for target cluster nodes", "stain and tolerance demand information for target cluster nodes" ]; for the same requirement information target object 2, only 'stain and tolerance requirement information for the target cluster node' is expected to be displayed in the data visualization effect, therefore, the included type key set is [ 'stain and tolerance requirement information for the target cluster node' ], the excluded type key set is [ 'affinity requirement information for the target cluster node', 'requirement information for a selector component of the target cluster node' ], so that keys in the included type key set and keys in the deleted excluded type key set can be flexibly adjusted according to the visualization requirements of different target objects, and the accuracy of keys of target data to be rendered is ensured.

In some embodiments, the exclusion type key set (excluddys) and the inclusion type key set (includdys) can be obtained through analysis of the visual configuration information, and as different target objects have different use habits and different visual requirements, the exclusion type key set (excluddys) and the inclusion type key set (includdys) can meet the use requirements of different target objects, and customized data visual services can be provided.

Step 302: a unique key set and a duplicate key set in the target data are determined.

In some embodiments, the unique key set and the duplicate key set in the target data may be obtained by:

traversing the target data to obtain each key corresponding to the target data; determining the occurrence times of each key according to the traversing result of the target data; and determining a unique key set and a repeated key set corresponding to the target data according to the occurrence times of each key. Taking the sub data as node requirement information of the cloud server 200 as an example, the requirement information includes: 1) demand information for selector components of the target cluster node, 2) affinity demand information for the target cluster node, 3) stain and tolerance demand information for the target cluster node; wherein 1) the number of occurrences of the demand information for the selector component of the target cluster node is 1, 2) the number of occurrences of the affinity demand information for the target cluster node is 2, 3) the number of occurrences of the stain and tolerance demand information for the target cluster node is 5; thus, the unique key set is [ "dirty and tolerance requirement information for target cluster node" ], and the duplicate key set is [ "affinity requirement information for target cluster node", "requirement information for selector component of target cluster node" ].

Step 303: pruning the keys of the target data according to the key set excluding the type, the unique key set, the key set including the type and the repeated key set to obtain the keys of the target data to be rendered.

Referring to fig. 4, fig. 4 is a schematic diagram illustrating a process of pruning a key of target data in an embodiment of the present application, and it is to be understood that an execution body of the steps shown in fig. 4 may be implemented by a terminal running a data processing apparatus alone or may be implemented by a server running the data processing apparatus. Of the processing steps shown in fig. 4, a server is taken as an implementation example, and the steps shown in the drawings are described below.

Step 3031: pruning is carried out on the keys of the target data according to the repeated key set and the unique key set, and a first pruning result is obtained.

In some embodiments, taking the sub-data as node requirement information of the cloud server 200 as an example, the requirement information includes: 1) demand information for selector components of the target cluster node, 2) affinity demand information for the target cluster node, 3) stain and tolerance demand information for the target cluster node; wherein 1) the number of occurrences of the demand information for the selector component of the target cluster node is 1, 2) the number of occurrences of the affinity demand information for the target cluster node is 2, 3) the number of occurrences of the stain and tolerance demand information for the target cluster node is 5; thus, the unique key set is [ "dirty and tolerance requirement information for target cluster node" ], and the duplicate key set is [ "affinity requirement information for target cluster node", "requirement information for selector component of target cluster node" ]. Pruning the keys of the target data to obtain a first pruning result of [ "stain and tolerance demand information for target cluster nodes" ]

Step 3032: and adding keys in the key set comprising the type into the first pruning result to obtain a second pruning result.

In some embodiments, the target object 2 is expected to only show the "stain and tolerance requirement information for the target cluster node" in the data visualization effect, so that the set of keys including the type is [ "stain and tolerance requirement information for the target cluster node" ], the set of keys excluding the type is [ "affinity requirement information for the target cluster node", "requirement information for the selector component of the target cluster node" ], and step 3032 is executed to add keys in the set of keys including the type, resulting in the second pruning result of [ "stain and tolerance requirement information for the target cluster node" ].

Step 3033: and deleting keys in the key set of the exclusion type in the second pruning result to obtain keys of the target data to be rendered.

In some embodiments, the target object 2 is expected to only show "stain and tolerance requirement information for the target cluster node" in the data visualization effect, so that the key set including the type is [ "stain and tolerance requirement information for the target cluster node" ], the key set excluding the type is [ "affinity requirement information for the target cluster node", "requirement information for the selector component of the target cluster node" ], and the embodiment shown in the integrating step 3032 is further executed to obtain that the key of the target data to be rendered is [ "stain and tolerance requirement information for the target cluster node" ].

Step 304: and taking the target data corresponding to the key of the target data to be rendered as the target data to be rendered.

When the target data corresponding to the key of the target data to be rendered is used as the target data to be rendered in step 304, step 205 may be continuously executed to render the target data to be rendered, so as to obtain a rendering result meeting the use requirement of the target object.

Step 205: and performing data rendering on the target data to be rendered to obtain a visual processing result.

In some embodiments, a visualization gallery corresponding to target data to be rendered may be first determined; and then, carrying out visualization processing on the target data to be rendered through a visualization gallery to obtain a data visualization processing result, wherein the data visualization processing result is matched with the visualization configuration information. The visual gallery corresponding to the target data to be rendered has two sources:

1) Acquiring identification information of a key of target data to be rendered; determining a first visual gallery corresponding to identification information of keys of target data to be rendered; for example, the CPU, the memory, and the server may be configured to correspond to three visual libraries of graphs, bars, and pie charts, respectively.

2) The visual configuration information is analyzed to obtain a second visual gallery matched with the target object of the network management system, and the target object of the network management system can be a payment target object of the data processing method provided by the application, so that the visual gallery of the payment target object can be selected to present more various visual effects, and the application is not particularly limited.

In order to better illustrate the working process of the data processing method provided by the application, a preset network protocol is taken as a simple network protocol as an example, and the process of obtaining the visualized processing result of the sub-data by the data processing method provided by the application in the data processing scene of the private cloud is described below.

Referring to fig. 5, fig. 5 is a schematic process diagram of a data processing method in an embodiment of the present application, and it is to be understood that the execution body of the steps shown in fig. 5 may be implemented by a terminal running a data processing apparatus alone or may be implemented by a server running the data processing apparatus. Of the processing steps shown in fig. 5, a server is taken as an implementation of an execution body, and the following description will be given with respect to the steps shown in fig. 5, specifically including the following steps:

Step 501: acquiring a data processing request; the data processing request comprises visual configuration information of the data to be processed; the data to be processed includes sub-data of a plurality of private cloud servers.

Referring to fig. 6, fig. 6 is a schematic diagram illustrating a process of acquiring sub-data of a plurality of private cloud servers according to an embodiment of the present application, in some embodiments, a network management system sends a data acquisition request to a proxy server in response to a data processing request, where the data processing request of the network management system may be implemented based on a simple gateway detection protocol, and the simple network management protocol is implemented by adding a new management information structure and a management information base on the basis of an SGMP protocol. Simplicity and extensibility are embodied in SNMP, which contains Database type (Database Schema), an application layer protocol (Application Layer Protocol) and some profile files. The SNMP management protocol not only can enhance the efficiency of the network management system, but also can be used to manage and detect resources in the network in real time. As shown in fig. 6, in a polling manner by a network management system (network management system), in a plurality of private cloud clusters, a message based on SNMP protocol may be sent to a proxy server periodically (the period time limit is flexibly adjusted according to different usage scenarios, and the period time limit is stored in the network management system), and sub-data (including operation data of a device and status data of a computing resource) of a private cloud server of a network device and corresponding operation information are acquired by using SNMP GET through a Management Information Base (MIB), a version number and a security parameter, and detection data is acquired at fixed time intervals. The network management system receives the sub-data of each private cloud server sent by the proxy server after responding to the data acquisition request.

As shown in fig. 6, taking the transmission of sub data of the proxy server 1 (Agent 1), the device 1, the management information base 1 (MIB 1) and the private cloud servers of the network management system as an example, the process of acquiring sub data of a plurality of private cloud servers includes the following steps:

step 601: the network management system sends an SNMP GetRequest message to the Agent1.

The SNMP GetRequest message contains the OID of the device 1 to be acquired.

Step 602: agent1 sends an SNMP GetRequest message to device 1.

After receiving the SNMP GetRequest message, the Agent1 sends an SNMP GetRequest message to the device 1 to be detected, and requests the device 1 to return the variable value corresponding to the OID.

Step 603: the device 1 returns an SNMP GetResponse message to the Agent1.

After receiving the SNMP GetRequest message, the device 1 searches MIB1, obtains a variable value corresponding to the OID, packages the variable value in the SNMP GetResponse message, and returns the variable value to Agent1.

Step 604: agent1 obtains the variable value corresponding to the OID.

After the Agent1 receives the SNMP GetResponse message, the data of the SNMP GetResponse message is parsed and returned to the MIB1.

Step 605: the information management library 1 returns the acquired variable values.

The information management base MIB1 may return a corresponding data variable value to the Agent1 according to the identification information.

Step 606: agent1 returns an SNMP GetResponse message.

And the Agent1 packages the acquired child data of all the private cloud servers in an SNMP GetResponse message and sends the SNMP GetResponse message to a network management system.

Step 607: and the network management system sends the acquired sub data of the private cloud servers of the multiple clusters to the DataServer server.

Through the processing of step 601-step 607, the obtained sub data of the private cloud server may be saved by the DataServer server.

Step 502: in response to the data processing request, sub-data of each private cloud server and an identifier of the sub-data of each private cloud server are obtained.

In some embodiments, identifiers of sub-data of the private cloud server may be used for data deduplication, reducing data redundancy, and reducing hardware device costs for data storage.

Referring to fig. 6, the identifier of the sub data of the private cloud server is composed of two parts, namely: the method comprises the steps that a network management system obtains a global unique identifier corresponding to sub-data of each private cloud server and a cluster name of a cluster corresponding to the sub-data of the corresponding private cloud server; and combining the global unique identifier and the cluster name aiming at the sub-data of each private cloud server to obtain the identifier of the sub-data of the corresponding private cloud server.

Here, because the OID of the sub-data of the private cloud server in different clusters may be duplicated, it is necessary to generate a globally unique identifier GUID and combine the prefix cluster name with the cluster name { cluster name } { GUID } as a unique identifier of the sub-data of the private cloud server.

In some embodiments, the GUID may be generated according to a pseudo-random algorithm, and a 28-bit random number is calculated using the real-time (accurate to millisecond) of the network management system as a seed value. And combining the MAC address (physical address) of the network management system with the GUID naming space of the hardware equipment of the network management system to generate a unique 6-byte identifier, and finally combining the 128-bit random numbers generated by the first two parts together according to a certain rule to generate a 128-bit GUID serving as an identifier of the sub-data of the private cloud server.

Step 503: and de-duplicating the sub-data of the plurality of private cloud servers according to the identifier of the sub-data of each private cloud server to obtain target data.

Referring to fig. 7, fig. 7 is a schematic diagram of a data deduplication process of a pruning layer in an embodiment of the present application, as shown in fig. 7, specifically including the following steps:

Step 701: the network management system acquires sub-data of a plurality of private cloud servers.

The network management system may obtain sub-data of a plurality of private cloud servers from a network data server (DataServer).

Step 702: the network management system traverses identifiers of sub-data of the plurality of private cloud servers.

Wherein the identifier of the sub data of the private cloud server is { clusterName } { GUID }.

Step 703: the network management system deletes the sub data of the private cloud server corresponding to the duplicated identifier.

Traversing identifiers of sub-data of the plurality of private cloud servers, determining duplicate identifiers in the sub-data of the plurality of private cloud servers; and deleting the sub-data of the private cloud server corresponding to the repeated identifier from the data to be processed to obtain target data. In some embodiments, a data list to be processed and a data list after deduplication may be formed first according to the acquired sub-data of all private cloud servers. And traversing the data in the data list to be processed, and inquiring the combination set of the GUID and the OID which have appeared by taking the combination of the GUID and the OID as a key word for each data. If not, the data is added to the de-duplicated data list and the combination of its GUID and OID is added to the combined set of GUID and OID that has already occurred. If so, this data is indicated as having occurred and needs to be removed. Thus, the sub-data of the private cloud server after pruning of all the devices in different clusters is obtained.

Step 704: the network management system stores the target data in the server.

Step 705: the network management system stores the target data in the cache interface.

In some embodiments, the sub-data of all private cloud servers after pruning may be stored in a server, for example, may be a database of network data deployed in the server, or in order to facilitate use of a visualization layer, the sub-data of all private cloud servers after pruning is also cached in a cache interface, and the storage mode of the sub-data of the private cloud servers is not particularly limited in the present application.

Step 504: and pruning the visual data according to the visual configuration information to obtain the target data to be rendered.

Referring to fig. 8, fig. 8 is a schematic view of visual data pruning logic of a pruning layer in the embodiment of the present application, as shown in fig. 8, a network management system may include a visual layer and a pruning layer, where target data may be located in a cache interface or in a database of a network data server, and a pruning algorithm module is configured to prune keys of the target data by using a pruning method in the foregoing embodiment, and specifically includes the following steps:

step 801: the pruning layer responds to the pruning request of the visualization layer to acquire target data.

Step 802: the visualization layer sends the data requesting the presentation to the pruning layer.

The visualization layer may define, through the visualization configuration information, which data needs to be displayed during the visualization processing, for example, the target data includes three indexes of CPU, memory, and server pressure, but because the server pressure is fixed at each time, the visualization only wants to display the CPU and the memory, and needs to prune using the visualization data at this time.

Step 803: the visualization layer sends a data pruning request to the pruning layer.

Step 804: the pruning layer performs data pruning by utilizing a pruning algorithm module in response to the data pruning request.

In some embodiments, according to the visual configuration information, the object data is pruned to obtain the object data to be rendered by the following manner:

determining hidden type data in the target data according to the visual configuration information; adding the key corresponding to the hidden type data into the key set of the initial exclusion type to obtain the key set of the exclusion type; determining reserved type data in the target data according to the visual configuration information; adding keys corresponding to the reserved type data into a key set which initially comprises types to obtain the key set comprising the types; determining a unique key set and a repeated key set in the target data; pruning the keys of the target data according to the key set excluding the type, the unique key set, the key set including the type and the repeated key set to obtain the keys of the target data to be rendered; and taking the target data corresponding to the key of the target data to be rendered as the target data to be rendered.

The key set used in the sub-data processing of the private cloud server is four in number:

traversing the target data to obtain each key corresponding to the target data; determining the occurrence times of each key according to the traversing result of the target data; and determining a unique key set and a repeated key set corresponding to the target data according to the occurrence times of each key.

In some embodiments, pruning the keys of the target data to obtain the keys of the target data to be rendered may be accomplished by:

1) Pruning is carried out on keys of the target data according to the repeated key set and the unique key set, and a first pruning result is obtained; for example: the keys of the target data include three key types, CPU, memory, server pressure, but since server pressure is fixed at each time, the first pruning result of the visualization process may be only CPU and memory. 2) Adding keys in a key set comprising types into the first pruning result to obtain a second pruning result; 3) And deleting keys in the key set of the exclusion type in the second pruning result to obtain keys of the target data to be rendered.

In order to more clearly describe the pruning process of the data processing method provided by the present application, referring to table 1, the pruning process of the data processing method provided by the present application is described by taking pruning the target data in table 1 as an example.

TABLE 1

The visualization configuration information requires the use of the meta field in the target data, and the value field is not used. And even if data a is duplicated, it remains as an uniqueKeys. Even if c is not repeated, it will still be regarded as a duplicate keys. By executing the pruning method of the above embodiment, the end result is that the duplicate keys are [ "b", "c" ], the uniqueKeys are [ "a", "d" ], the extraddkeys are [ "c" ], and the includeddys are [ "a" ].

By adding keys in the key set including the type and deleting keys in the key set excluding the type in the repeated key and unique key first pruning result, the key of the target data to be finally rendered is obtained as [ "a", "d" ].

Referring to fig. 9, fig. 9 is an algorithm logic diagram of visual data pruning of a pruning layer in an embodiment of the present application, in order to perform data processing on sub-data of a private cloud server in any private cloud network scene according to different visual configuration requirements, the data processing method provided by the present application may implement the algorithm of visual data pruning shown in fig. 9, where the pruning algorithm module in fig. 8 embodiment may execute the pruning algorithm logic shown in fig. 9, as shown in fig. 9, it may be understood that an execution body of the steps shown in fig. 9 may be implemented by a terminal running a data processing apparatus alone, or may be implemented by a server running the data processing apparatus. Of the processing steps shown in fig. 9, a server is taken as an implementation example, and the following description will be given with respect to the steps shown in fig. 9:

Step 901: an object is received.

Wherein the received one object is to contain as parameters the object of the following properties:

1) array: an array of objects is necessary, and objects may be nested.

2) extraded keys: keys to be excluded from the result by default.

3) Includedkeys: keys to be included in the result by default. Containing type keys

4) specificKey: if the object is nested, extraction begins with the specified key.

Step 902: the input parameters are checked.

Wherein, input parameters are invalid, throw out errors, input parameters are valid, and trigger a cyclic array.

Step 903: checking whether a specified key exists, if so, triggering the use of the nested object, otherwise, using the object.

In some embodiments, by performing steps 902-903, it may be first verified whether the excludedKeys and the includeedkeys contain duplicate keys, and if so, throw an error. It is then verified whether the array is empty or is an object array, and if not, an error is thrown.

Step 904: the loop object key is executed.

Step 905: checking whether a key exists, if so, adding a value count, and if not, adding the key to the key list.

In some embodiments, by performing steps 904-905, the array of rays may be traversed first, and keys and values extracted from each object, combined into a string as keys, and the keys added to the allKeys object. If the key is already present, the counter is incremented by 1. The allKeys objects are then traversed and separated into duplicate keys and unique keys based on the number of times the keys appear. Finally, adding keys in the includekeys and deleting keys in the exincluding keys from the processing results of the repeated keys and the unique keys, and returning a final result object. The return value of the function is an object containing two arrays of duplicate keys and uniqueKeys.

Step 805: the pruning layer acquires pruning results and sends the pruning results to the visualization layer.

Step 806: and the visualization layer performs data rendering according to the pruning result.

After the execution of steps 801-806 shown in fig. 8 is completed, the execution of step 505 may be continued to perform data rendering on the target data to be rendered.

Step 505: and performing data rendering on the target data to be rendered to obtain a visual processing result.

In some embodiments, referring to fig. 10, fig. 10 is a schematic diagram of a visual gallery in the data processing method provided by the present application, a visual gallery corresponding to target data to be rendered may be first determined; and then, carrying out visualization processing on the target data to be rendered through a visualization gallery to obtain a data visualization processing result, wherein the data visualization processing result is matched with the visualization configuration information. The visual gallery corresponding to the target data to be rendered has two sources:

1) Acquiring identification information of a key of target data to be rendered; determining a first visual gallery corresponding to identification information of keys of target data to be rendered; for example, the CPU, the memory, and the server may be configured to correspond to three visual libraries of graphs, bars, and pie charts, respectively. As shown in fig. 10, the name of the visual gallery in the selected state in 1003 may be displayed in 1001, three visual galleries of visual gallery 1 (pie chart), visual gallery 2 (bar chart) and visual gallery 3 (graph) may be displayed in 1003, and the visual processing result may be previewed in 1004.

2) The visual configuration information is analyzed to obtain a second visual gallery matched with the target object of the network management system, as shown in fig. 10, the name of the target object can be input in 1002, wherein the name of the target object can be either a Chinese name or an English name, a state value is used for indicating whether the target object is a paying user, a state value is 1, a representation target object is a paying user, a state value is 0, a representation target object is a non-paying user, and the target object of the network management system can be the paying target object of the data processing method provided by the application, so that the visual gallery of the paying target object can be selected to present more various visual effects.

Through the data processing of steps 501 to 505 shown in fig. 5, the following effects can be achieved:

1) The detection efficiency of the network equipment is improved: the network management system periodically acquires the data states of the network devices in the private cloud clusters in a polling mode, so that the detection data of the devices can be acquired rapidly, and the detection efficiency is improved.

2) Reducing data redundancy: in the pruning layer, by generating the GUID and matching with the prefix clusteriname to use the GUID as a unique identifier and then combining and de-duplication processing is carried out on the OIDs in different clusters, redundant data caused by the repetition of the OIDs in different clusters can be avoided, and therefore the data quantity and the storage cost are reduced.

3) The data processing efficiency is improved: repeated data can be reduced through pruning, so that the processing efficiency of the data is greatly improved, and meanwhile, the pruned data can be cached in Redis, so that the visualization layer can be used conveniently.

4) And improving the data visualization effect: through the API interface of the visualization layer, the data returned by the pruning layer can be conveniently requested, and the data is displayed in a graphical mode, so that the visualization effect of the data is improved, and the detection data is more visual and easier to understand.

Referring to fig. 11, fig. 11 is a schematic structural diagram of an electronic device 100 for executing a data processing method provided by the present application, where the electronic device 100 shown in fig. 11 includes: each processor 410, memory 450, each network interface 420, and target object interface 430. The various components in the electronic device 100 are coupled together by a bus system 440. It is understood that the bus system 440 is used to enable connected communication between these components. The bus system 440 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled in fig. 11 as bus system 440.

The processor 410 may be an integrated circuit chip having signal processing capabilities such as a general purpose processor, such as a microprocessor or any conventional processor, a digital signal processor (Digital Signal Processor, DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

The target object interface 430 includes one or more output devices 431, including one or more speakers and/or one or more visual displays, that enable presentation of media content. The target object interface 430 also includes one or more input devices 432, including target object interface components that facilitate target object input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

Memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard drives, optical drives, and the like. Memory 450 optionally includes one or more storage devices physically remote from processor 410.

Memory 450 includes volatile memory or nonvolatile memory, and may also include both volatile and nonvolatile memory. The non-volatile Memory may be a Read Only Memory (ROM) and the volatile Memory may be a random access Memory (Random Access Memory, RAM). The memory 450 described in embodiments of the present application is intended to comprise any suitable type of memory.

In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules and data structures, or subsets or supersets thereof, as exemplified below.

An operating system 451 including system programs, e.g., framework layer, core library layer, driver layer, etc., for handling various basic system services and performing hardware-related tasks, for implementing various basic services and handling hardware-based tasks;

a network communication module 452 for accessing other electronic devices via one or more (wired or wireless) network interfaces 420, the exemplary network interface 420 comprising: bluetooth, wireless compatibility authentication (WiFi), and universal serial bus (Universal Serial Bus, USB), etc.;

A presentation module 453 for enabling presentation of information (e.g., for operating peripheral devices and displaying target object interfaces of content and information) via one or more output devices 431 (e.g., a display screen, speakers, etc.) associated with target object interface 430;

an input processing module 454 for detecting one or more target object inputs or interactions from one of the one or more input devices 432 and translating the detected inputs or interactions.

In some embodiments, the apparatus provided by the embodiments of the present application may be implemented in software, and fig. 11 shows a data processing apparatus 455 stored in a memory 450, which may be in the form of a program, a plug-in, or the like, including the following software modules: the information transmission module 4081 and the information processing module 4082 are logical, and thus may be arbitrarily combined or further split according to the implemented functions. The functions of the respective modules will be described hereinafter.

The function of the various software modules in the data processing device 455 will be described further below, wherein,

an information transmission module 4081 for acquiring a data processing request; the data processing request comprises visual configuration information of the data to be processed; the data to be processed comprises a plurality of sub-data;

An information processing module 4082 for acquiring each sub data and an identifier of each sub data in response to a data processing request;

the information processing module 4082 is further configured to perform deduplication on the plurality of sub-data according to the identifier of each sub-data, to obtain target data;

the information processing module 4082 is further configured to prune the target data according to the visual configuration information to obtain target data to be rendered;

the information processing module 4082 is further configured to perform data rendering on the target data to be rendered, to obtain a visualization processing result.

In some embodiments, the information processing module 4082 is further configured to send, by the network management system, a data acquisition request to the proxy server in response to the data processing request, where the data acquisition request is configured to request the proxy server to perform data interaction with the network device according to a preset network protocol, so as to acquire a plurality of sub-data; each sub-data transmitted by the proxy server after responding to the data acquisition request is received through the network management system.

In some embodiments, the information processing module 4082 is further configured to obtain, through the network management system, the globally unique identifier corresponding to each sub-data and the cluster name of the cluster corresponding to the corresponding sub-data; for each sub-data, the globally unique identifier and the cluster name are combined to obtain the identifier of the corresponding sub-data.

In some embodiments, the information processing module 4082 is further configured to traverse identifiers of the plurality of sub-data to determine duplicate identifiers in the plurality of sub-data; and deleting the sub data corresponding to the repeated identifier from the data to be processed to obtain target data.

In some embodiments, the information processing module 4082 is further configured to determine, based on the visual configuration information, a set of keys that excludes the type and a set of keys that includes the type; determining a unique key set and a repeated key set in the target data; pruning the keys of the target data according to the key set excluding the type, the unique key set, the key set including the type and the repeated key set to obtain the keys of the target data to be rendered; and taking the target data corresponding to the key of the target data to be rendered as the target data to be rendered.

In some embodiments, the information processing module 4082 is further configured to determine, according to the visualization configuration information, the hidden type data in the target data; adding the key corresponding to the hidden type data into the key set of the initial exclusion type to obtain the key set of the exclusion type; determining reserved type data in the target data according to the visual configuration information; and adding the key corresponding to the reserved type data into the key set which initially comprises the type to obtain the key set which comprises the type.

In some embodiments, the information processing module 4082 is further configured to traverse the target data to obtain each key corresponding to the target data; determining the occurrence times of each key according to the traversing result of the target data; and determining a unique key set and a repeated key set corresponding to the target data according to the occurrence times of each key.

In some embodiments, the information processing module 4082 is further configured to cache the target data into a network data server or a cache interface; responding to the received visualized data pruning request through a network management system, and detecting cache aging in a cache interface to obtain a cache aging detection result; when the cache aging detection result is greater than or equal to the cache validity period, the network management system acquires target data from the network data server; and when the cache aging detection result is smaller than the cache validity period, the network management system acquires target data from the cache interface.

In some embodiments, the information processing module 4082 is further configured to prune the keys of the target data according to the repeated key set and the unique key set to obtain a first pruning result; adding keys in a key set comprising types into the first pruning result to obtain a second pruning result; and deleting keys in the key set of the exclusion type in the second pruning result to obtain keys of the target data to be rendered.

In some embodiments, the information processing module 4082 is further configured to determine a visualization gallery corresponding to the target data to be rendered; and carrying out visualization processing on the target data to be rendered through a visualization gallery to obtain a data visualization processing result, wherein the data visualization processing result is matched with the visualization configuration information.

In some embodiments, the information processing module 4082 is further configured to obtain identification information of a key of the target data to be rendered; determining a first visual gallery corresponding to identification information of keys of target data to be rendered; or analyzing the visual configuration information to obtain a second visual gallery matched with the target object of the network management system.

a memory for storing executable instructions;

and the processor is used for realizing the preamble data processing method when executing the executable instructions stored in the memory.

The embodiment of the application also provides a computer readable storage medium which stores executable instructions, and the executable instructions realize the data processing method when being executed by a processor.

According to the electronic device shown in fig. 11, in one aspect of the application, the application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer readable storage medium and executes the computer instructions to cause the computer device to perform the different embodiments and combinations of embodiments provided in various alternative implementations of the data processing method provided by the embodiments of the application

The above embodiments are merely examples of the present application and are not intended to limit the scope of the present application, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A method of data processing, the method comprising:

2. The method of claim 1, wherein said obtaining each of said sub-data and an identifier of each of said sub-data in response to said data processing request comprises:

The network management system responds to the data processing request and sends a data acquisition request to a proxy server, wherein the data acquisition request is used for requesting the proxy server to perform data interaction with network equipment according to a preset network protocol so as to acquire the plurality of sub-data;

the network management system receives each of the sub-data transmitted by the proxy server after responding to the data acquisition request.

3. The method of claim 2, wherein obtaining an identifier for each of the sub-data comprises:

the network management system acquires a global unique identifier corresponding to each piece of sub data and a cluster name of a cluster corresponding to the corresponding piece of sub data;

and combining the globally unique identifier and the cluster name for each sub-data to obtain the identifier of the corresponding sub-data.

4. The method of claim 1, wherein de-duplicating the plurality of sub-data according to the identifier of each sub-data to obtain target data, comprises:

traversing identifiers of the plurality of sub-data, determining duplicate identifiers in the plurality of sub-data;

And deleting the sub data corresponding to the repeated identifier from the data to be processed to obtain the target data.

5. The method according to claim 1, wherein the pruning the target data according to the visual configuration information to obtain target data to be rendered includes:

determining a key set excluding types and a key set comprising types according to the visual configuration information;

determining a unique key set and a repeated key set in the target data;

pruning the keys of the target data according to the key set of the exclusion type, the unique key set, the key set of the inclusion type and the repeated key set to obtain the keys of the target data to be rendered;

and taking the target data corresponding to the key of the target data to be rendered as the target data to be rendered.

6. The method of claim 5, wherein determining the excluded type of key set and the included type of key set based on the visual configuration information comprises:

determining hidden type data in the target data according to the visual configuration information;

Adding keys corresponding to the hidden type data into an initial exclusion type key set to obtain an exclusion type key set;

determining reserved type data in the target data according to the visual configuration information;

and adding the key corresponding to the data with the reserved type into the key set with the initial type to obtain the key set with the type.

7. The method of claim 5, wherein determining the unique key set and the duplicate key set in the target data comprises:

traversing the target data to obtain each key corresponding to the target data;

determining the occurrence times of each key according to the traversing result of the target data;

and determining a unique key set and a repeated key set corresponding to the target data according to the occurrence times of each key.

8. The method according to any one of claims 1 to 7, wherein after obtaining the target data, the method further comprises:

caching the target data into a network data server or a cache interface;

the network management system responds to the received visual data pruning request and detects cache aging in the cache interface to obtain a cache aging detection result;

When the cache aging detection result is greater than or equal to the cache expiration date, the network management system acquires the target data from a network data server;

and when the cache aging detection result is smaller than the cache validity period, the network management system acquires the target data from the cache interface.

9. The method of claim 5, wherein pruning the keys of the target data to obtain the keys of the target data to be rendered based on the excluded type of key set, the unique key set, the included type of key set, and the repeated key set, comprises:

pruning is carried out on the keys of the target data according to the repeated key set and the unique key set, and a first pruning result is obtained;

adding keys in the key set comprising the type into the first pruning result to obtain a second pruning result;

and deleting keys in the key set of the exclusion type in the second pruning result to obtain the keys of the target data to be rendered.

10. The method according to any one of claims 1 to 7, wherein performing data rendering on the target data to be rendered to obtain a visualization processing result, includes:

Determining a visual gallery corresponding to the target data to be rendered;

and carrying out visualization processing on the target data to be rendered through the visualization gallery to obtain a data visualization processing result, wherein the data visualization processing result is matched with the visualization configuration information.

11. The method of claim 10, wherein determining a visualization gallery corresponding to the target data to be rendered comprises:

acquiring identification information of a key of the target data to be rendered;

determining a first visual gallery corresponding to identification information of keys of the target data to be rendered; or,

and analyzing the visual configuration information to obtain a second visual gallery matched with the target object of the network management system.

12. A data processing apparatus, the apparatus comprising:

The information processing module is further used for carrying out de-duplication on the plurality of sub-data according to the identifier of each sub-data to obtain target data;

the information processing module is further used for pruning the visual data according to the visual configuration information to obtain the target data to be rendered;

the information processing module is also used for carrying out data rendering on the target data to be rendered to obtain a visual processing result.

13. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the data processing method of any one of claims 1 to 11.

14. An electronic device, the electronic device comprising:

a memory for storing executable instructions;

a processor for implementing the data processing method of any one of claims 1 to 11 when executing executable instructions stored in said memory.

15. A computer readable storage medium storing executable instructions which when executed by a processor implement the data processing method of any one of claims 1 to 11.