CN111901410A - Server grouping method, device, equipment and computer readable storage medium - Google Patents
Server grouping method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111901410A CN111901410A CN202010725778.4A CN202010725778A CN111901410A CN 111901410 A CN111901410 A CN 111901410A CN 202010725778 A CN202010725778 A CN 202010725778A CN 111901410 A CN111901410 A CN 111901410A
- Authority
- CN
- China
- Prior art keywords
- server
- grouping
- target
- servers
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 36
- 230000006870 function Effects 0.000 claims description 60
- 238000013139 quantization Methods 0.000 claims description 39
- 238000012423 maintenance Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009394 selective breeding Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1042—Peer-to-peer [P2P] networks using topology management mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a server grouping method, which comprises the following steps: analyzing the received server grouping request to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group. By applying the technical scheme provided by the embodiment of the invention, the influence of human subjective factors is greatly reduced, the grouping accuracy is improved, and the grouping efficiency is greatly improved. The invention also discloses a server grouping device, equipment and a storage medium, and has corresponding technical effects.
Description
Technical Field
The present invention relates to the field of computer application technologies, and in particular, to a server grouping method, apparatus, device, and computer-readable storage medium.
Background
The scale of the data center is getting bigger and bigger nowadays, the number of machines in a machine room is thousands, and the management of operation and maintenance personnel becomes extremely complicated. There are two scenarios, where a large number of servers need to be grouped, and the first scenario is to group all servers in a network according to configuration information to facilitate maintenance, for example, the servers with the same configuration and the same computing power belong to the same group as much as possible, so that an administrator can quickly find similar servers when performing system maintenance. The other situation is that the running servers are grouped, machines with similar resources at the current moment are found out to form a cluster, distribution can be performed on the basis of the cluster when complex tasks are convenient, generally, the distributed tasks can be better executed by the servers with similar configuration and calculation capacity due to resource balance, and the distribution strategy is superior to the distribution of the tasks to the machines with different configurations.
The two scenes involve grouping the servers in the server cluster, the existing server grouping mode is to group the servers in the server cluster through an artificial selection mode, but in reality, the number of the servers in one data center is often thousands, the server grouping task amount is huge, errors are prone to occurring, the influence of artificial subjective factors is large, the grouping accuracy is low, and the grouping efficiency is low.
In summary, how to effectively solve the problems of low grouping accuracy, low grouping efficiency and the like of the server grouping mode selected manually is a problem which needs to be solved urgently by a person skilled in the art at present.
Disclosure of Invention
The invention aims to provide a server grouping method, which greatly reduces the influence of human subjective factors, improves the grouping accuracy and greatly improves the grouping efficiency; another object of the present invention is to provide a server grouping device, an apparatus, and a computer-readable storage medium.
In order to solve the technical problems, the invention provides the following technical scheme:
a server grouping method, comprising:
analyzing the received server grouping request to obtain a target grouping task;
respectively acquiring parameters of each target server corresponding to the target grouping task in each server;
and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group.
In a specific embodiment of the present invention, when the target grouping task is a server maintenance task, respectively obtaining target server parameters corresponding to the target grouping task in each server, includes:
respectively acquiring each original configuration parameter corresponding to the server maintenance task in each server;
carrying out quantization processing on each original configuration parameter by using a configuration quantization function to obtain each target configuration parameter;
clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm, wherein the clustering and grouping method comprises the following steps:
and clustering and grouping the servers based on the target configuration parameters by using the fuzzy C-means clustering algorithm.
In an embodiment of the present invention, the quantizing each of the original configuration parameters by using a configuration quantization function includes:
acquiring a configuration quantization function preset through a function preset interface;
and quantizing each original configuration parameter by using the configuration quantization function.
In an embodiment of the present invention, after clustering and grouping the servers based on the target configuration parameters by using the fuzzy C-means clustering algorithm to obtain the server groups, the method further includes:
and performing parallel maintenance operation on the servers by taking the server groups as units.
In a specific embodiment of the present invention, when the target grouping task is a service distribution task, respectively obtaining parameters of each target server corresponding to the target grouping task in each server, includes:
respectively acquiring each real-time performance parameter corresponding to the service distribution task in each server;
clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm, wherein the clustering and grouping method comprises the following steps:
and clustering and grouping the servers based on the real-time performance parameters by using the fuzzy C-means clustering algorithm.
In an embodiment of the present invention, after clustering and grouping the servers based on the real-time performance parameters by using the fuzzy C-means clustering algorithm to obtain the server groups, the method further includes:
selecting a target server group from each server group;
and distributing the service to be processed to each target server in the target server group so as to utilize each target server to process the service.
In a specific embodiment of the present invention, clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm includes:
acquiring a server subset corresponding to each preset server class;
obtaining centroids preset for each server subset respectively;
respectively calculating each distance value between each server and each centroid according to each target server parameter;
respectively calculating probability values of the servers belonging to the preset server classes according to the current distance values;
recalculating each centroid according to each current probability value;
calculating each distance value between each server and each recalculated centroid according to each target server parameter;
calculating a value function by combining the current probability values and the current distance values;
judging whether the value function meets a preset condition or not;
if not, the step of respectively calculating the probability value of each server belonging to each preset server class according to each distance value is repeatedly executed;
and if so, acquiring each current centroid, and clustering and grouping the servers according to each current centroid.
A server grouping apparatus comprising:
the task obtaining module is used for analyzing the received server grouping request to obtain a target grouping task;
the parameter acquisition module is used for respectively acquiring parameters of each target server corresponding to the target grouping task in each server;
and the server group obtaining module is used for clustering and grouping the servers based on the target server parameters by using a fuzzy C-means clustering algorithm to obtain the server groups.
A server grouping apparatus comprising:
a memory for storing a computer program;
a processor for implementing the steps of the server grouping method as described above when executing the computer program.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the server grouping method as described above.
By applying the method provided by the embodiment of the invention, the received server grouping request is analyzed to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group. By acquiring target server parameters of each server according to the target grouping task and automatically clustering and grouping each server based on each target server parameter by using the fuzzy C-means clustering algorithm, compared with the existing manual server grouping mode, the method greatly reduces the influence of human subjective factors, improves the grouping accuracy and greatly improves the grouping efficiency.
Correspondingly, embodiments of the present invention further provide a server grouping device, an apparatus, and a computer-readable storage medium corresponding to the server grouping method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of an implementation of a server grouping method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another implementation of a server grouping method according to an embodiment of the present invention;
FIG. 3 is a network topology diagram of a server according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating grouped servers according to an embodiment of the present invention;
FIG. 5 is a flowchart of another implementation of a server grouping method according to an embodiment of the present invention;
FIG. 6 is a block diagram of a server grouping apparatus according to an embodiment of the present invention;
fig. 7 is a block diagram of a server grouping device according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a flowchart of an implementation of a server grouping method in an embodiment of the present invention, where the method may include the following steps:
s101: and analyzing the received server grouping request to obtain a target grouping task.
When the servers in the server cluster need to be clustered and grouped, a server grouping request is sent to a server grouping center, and the server grouping request comprises a target grouping task. And the server grouping center receives the server grouping request and analyzes the server grouping request to obtain a target grouping task.
When each server in the server cluster needs to be maintained, a server grouping request is generated, and a corresponding target grouping task is a server maintenance task; and when the temporary service needs to be processed, a server grouping request is generated, and the corresponding target grouping task is a service distribution task.
S102: and respectively acquiring parameters of each target server corresponding to the target grouping task in each server.
And after the target grouping task is obtained through analysis, acquiring parameters of each target server corresponding to the target grouping task in each server respectively. If the target grouping task is a server maintenance task, each target server parameter may include CPU configuration information, memory configuration information, hard disk configuration information, network card configuration information, redundant array of disks configuration information, operating system configuration information, and the like; when the target grouping task is a service distribution task, the target server parameters may include a current CPU utilization rate, a current memory utilization rate, a current hard disk capacity utilization rate, a current network bandwidth utilization rate, a database connection number ratio, a hard disk IO bandwidth ratio, and the like.
S103: and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group.
After acquiring parameters of each target server corresponding to the target grouping task in each server, clustering and grouping the servers based on the parameters of each target server by using a fuzzy C-means clustering algorithm to obtain each server group. By acquiring target server parameters of each server according to the target grouping task and automatically clustering and grouping each server based on each target server parameter by using the fuzzy C-means clustering algorithm, compared with the existing manual server grouping mode, the method greatly reduces the influence of human subjective factors, improves the grouping accuracy and greatly improves the grouping efficiency.
The fuzzy C-means clustering algorithm obtains the membership degree of each sample point to all class centers by optimizing an objective function, thereby determining the class of the sample points to achieve the purpose of automatically classifying the sample data.
By applying the method provided by the embodiment of the invention, the received server grouping request is analyzed to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group. By acquiring target server parameters of each server according to the target grouping task and automatically clustering and grouping each server based on each target server parameter by using the fuzzy C-means clustering algorithm, compared with the existing manual server grouping mode, the method greatly reduces the influence of human subjective factors, improves the grouping accuracy and greatly improves the grouping efficiency.
It should be noted that, based on the first embodiment, the embodiment of the present invention further provides a corresponding improvement scheme. In the following embodiments, steps that are the same as or correspond to those in the first embodiment may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the following modified embodiments.
Example two:
referring to fig. 2, fig. 2 is a flowchart of another implementation of a server grouping method in an embodiment of the present invention, where the method may include the following steps:
s201: and analyzing the received server grouping request to obtain a target grouping task.
S202: and when the target grouping task is a server maintenance task, respectively acquiring each original configuration parameter corresponding to the server maintenance task in each server.
And when the target grouping task is a server maintenance task, respectively acquiring each original configuration parameter corresponding to the server maintenance task in each server. Each original configuration parameter may include CPU configuration information, memory configuration information, hard disk configuration information, network card configuration information, redundant array of disks configuration information, operating system configuration information, and the like. Referring to fig. 3, fig. 3 is a server network topology diagram according to an embodiment of the present invention. The management host needs to be in a network segment with each managed server, the user side can be connected with the management host through the Ethernet, and a user side Web page can show all the servers in the current server cluster and configuration parameters of the servers, and is mainly used for interacting with users, so that the main function of the management host is data display. The management host may establish a communication connection with each server being managed through the gigabit switch management network.
S203: and acquiring a configuration quantization function preset through a function preset interface.
Because the collected original configuration parameters are not necessarily suitable for clustering calculation, for example, the hard disk interface is an enumeration type and is not a continuous number, and direct calculation has no meaning. Therefore, each original configuration parameter needs to be preprocessed. The function presetting interface for setting the configuration quantization function required by quantizing each original configuration parameter can be preset, and a user can preset the configuration quantization function through the function presetting interface according to actual needs, so that the flexibility of parameter quantization is improved, and the user experience is improved.
And after acquiring each original configuration parameter corresponding to the server maintenance task in each server, acquiring a configuration quantization function preset through a function preset interface.
S204: and carrying out quantization processing on each original configuration parameter by using a configuration quantization function to obtain each target configuration parameter.
After the configuration quantization function preset through the function preset interface is obtained, the configuration quantization function is used for carrying out quantization processing on each original configuration parameter to obtain each target configuration parameter.
The process of quantizing each original configuration parameter may include: firstly, quantizing all the original configuration parameters into a digital class, secondly, considering the difference between different enumerations and taking the difference into account in the quantization process. After quantization, each target configuration parameter of each server becomes a digital type which can be calculated, and each target configuration parameter constitutes multidimensional data. If D ═ D can be used1，d2…dj…dnDenotes a server cluster, djRepresenting the jth server of the server cluster. p is a radical ofjTarget configuration parameter set, p, quantized for jth serverj＝{cpu1j,mem1j,hd1j,net1j,raidj,osjWherein, cpu1jFor CPU configuration information of jth server, mem1jFor memory configuration information of the jth server, hd1jNet configuration information for the hard disk of the jth server1jNetwork card configuration information, raid, for the jth serverjConfiguring information, os, for RAID of jth serverjConfiguring information for the operating system of the jth server.
Taking an example of a quantization process of main frequency in a CPU, as shown in a quantization formula below, when the main frequency is greater than 3Ghz, a quantization value of an attribute of the CPU is 100, a quantization value of 60 between 2.0 and 3.0Ghz and a quantization value of 30 less than 2.0Ghz, the quantization formula provides a default quantization process, but provides a channel for user rewriting, because various performance parameters in a real scene are different due to historical reasons, a fixed quantization function cannot accurately describe the difference, and along with the iterative development of the technology, a new performance index is added, so that only a solution of a frame layer is provided, a specific measurement standard is handed to a user, and thus, the actual situation can be met to the maximum extent.
Wherein, rate1jIs CPU main frequency, cache1jFor CPU cache size, rate2jIs the main frequency value of the memory, size1jIs the memory capacity value, size2jInterface as a value of hard disk capacityjOf the hard disk interface type, speed1jFor hard disk speed, cache2jFor hard disk data buffer capacity, speed2jFor network card bandwidth, type1jIs a network card type, type2jAs operating system type, versionjIs the operating system version.
S205: and obtaining the server subsets corresponding to the preset server classes respectively.
Presetting a total number of server classes, supposing that each server in the server cluster is divided into m classes, obtaining the server subsets corresponding to each preset server class, for example, using ciRepresenting a subset of class i servers.
S206: and acquiring centroids preset for the server subsets respectively.
Presetting a mass center for each server subset, and after obtaining the server subsets corresponding to the preset server classes, obtaining the mass centers preset for the server subsets, such as viDenotes the ciA centroid of the subset of class servers.
S207: and respectively calculating each distance value between each server and each centroid according to each target configuration parameter.
After the centroids preset for the server subsets are obtained, the distance values between the servers and the centroids are calculated according to the parameters of the target servers. The respective distance values between the respective servers to the respective centroids may be calculated, for example, by the following formula:
wherein, dist (d)j,vi) Denotes djServer distance ciCentroid viQ is a distance index, and is default to 2, α, β, γ, λ, μ, η are weighted values of the target configuration parameters, α + β + γ + λ + μ + η ═ 1, as generally set the weighted values of the target configuration parameters as follows:
a standard api interface may be provided to allow the user to make weight parameter modifications according to the respective service.
S208: and respectively calculating the probability value of each server belonging to each preset server class according to the current distance values.
After the distance values between each server and each centroid are respectively calculated, the probability value of each server belonging to each preset server class is respectively calculated according to the current distance values, for example, the probability value of each server belonging to each preset server class can be calculated by the following formula:
wherein u isijDenotes djServers under ciProbability of this class.
Obtaining a membership matrix U:
the above matrix satisfies the following formula:
i.e. the sum of the values of each row equals 1, certain server djAlthough belonging to preset server classes ciAre different, but their sum of probabilities must be equal to 1.
S209: and recalculating each centroid according to each current probability value.
After the probability values of the servers belonging to the preset server classes are respectively calculated according to the current distance values, recalculating the centroids according to the current probability values, for example, recalculating the centroids according to the current probability values by using the following formula:
s210: and calculating each distance value between each server and each recalculated centroid according to each target server parameter.
And after recalculating each centroid according to each current probability value, calculating each distance value between each server and each recalculated centroid according to each target server parameter. The calculation process may refer to step S207, which is not described herein.
S211: and calculating a cost function by combining the current probability values and the current distance values.
After calculating the distance values between each server and each recalculated centroid according to the parameters of each target server, calculating a cost function by combining the current probability values and the current distance values, and calculating a cost function J by the following formula:
s212: and judging whether the cost function meets a preset condition, if so, executing step S213, and if not, executing step S208.
After the cost function is calculated by combining the current probability values and the current distance values, whether the cost function meets preset conditions is judged, if yes, the centroid obtained by recalculation is the optimal centroid position of the server subset represented by the centroid, step S213 is executed, if not, the centroid obtained by recalculation is not the optimal centroid position of the server subset represented by the centroid, and step S208 is executed again.
When the cost function is first calculated, the predetermined condition may be that the cost function is less than a predetermined value, e.g.1Judging whether the value function J is smaller than a preset value1(ii) a When the cost function is not obtained by the first calculation, the preset condition may be that the current calculated cost function and the last calculated cost function J are combinedpreThe difference between them is less than a fixed valueConstant value2Judging the value function J and the value function J obtained by the last calculationpreWhether the difference up to now is less than a fixed value2。
S213: and acquiring each current centroid, and clustering and grouping the servers according to each current centroid to obtain each server group.
And after the fact that the calculated value function meets the preset conditions is determined, obtaining each current centroid, namely the latest calculated centroid, and clustering and grouping the servers according to each current centroid to obtain each server group. The process of clustering the servers may include: processing the membership matrix U, circularly traversing the membership matrix U, wherein each row represents a server djBelonging to a subset of servers ciIf u isijIf the server is more than 0.5, the server is considered to belong to the server subset, otherwise, the server belongs to other server subsets. Referring to fig. 4, fig. 4 is a schematic diagram of grouped servers according to an embodiment of the present invention. It can be seen from fig. 4 that the servers are grouped into 4 server groups.
S214: and performing parallel maintenance operation on each server by taking each server group as a unit.
And after clustering and grouping the servers according to the current centroids to obtain server groups, performing parallel maintenance operation on the servers by taking the server groups as units. By performing parallel maintenance operation on each server by taking each server group as a unit, the maintenance efficiency of each server in the server cluster is greatly improved.
It should be noted that, in the embodiments of the present invention, the "current" mentioned in each step refers to the corresponding parameter value newly obtained in the previous step.
Example three:
referring to fig. 5, fig. 5 is a flowchart of another implementation of a server grouping method in an embodiment of the present invention, where the method may include the following steps:
s501: and analyzing the received server grouping request to obtain a target grouping task.
S502: and when the target grouping task is a service distribution task, respectively acquiring each real-time performance parameter corresponding to the service distribution task in each server.
And when the target grouping task is a service distribution task, respectively acquiring each real-time performance parameter corresponding to the service distribution task in each server. Each real-time performance parameter may include a current CPU utilization, a current memory utilization, a current hard disk capacity utilization, a current network bandwidth utilization, a database connection number ratio, a hard disk IO bandwidth ratio, and the like. E.g. can use sjRepresenting the quantized real-time performance parameter set, s, of the jth serverj＝{cpu2j,mem2j,hd2j,net2j,dbj,IOjWherein, cpu2jIs the current CPU utilization, mem, of the jth server2jIs the current memory utilization of the jth server, hd2jIs the current hard disk capacity utilization, net, of the jth server2jIs the current network bandwidth utilization, db, of the jth serverjRatio of database connections, IO, for the jth serverjAnd the hard disk IO bandwidth of the jth server is in proportion.
S503: and obtaining the server subsets corresponding to the preset server classes respectively.
S504: and acquiring centroids preset for the server subsets respectively.
S505: and respectively calculating each distance value between each server and each centroid according to each real-time performance parameter.
S506: and respectively calculating the probability value of each server belonging to each preset server class according to the current distance values.
S507: and recalculating each centroid according to each current probability value.
S508: and calculating each distance value between each server and each recalculated centroid according to each target server parameter.
S509: and calculating a cost function by combining the current probability values and the current distance values.
S510: and judging whether the cost function meets a preset condition, if so, executing step S511, and if not, executing step S506.
S511: and acquiring each current centroid, and clustering and grouping the servers according to each current centroid to obtain each server group.
The way of clustering and grouping the servers in the server cluster according to the real-time performance parameters can be similar to the way of clustering and grouping the servers in the server cluster according to the target configuration parameters in the second embodiment, which only needs to set the target configuration parameter pjTransformation into a set of real-time performance parameters sj. Two scenarios to solve dist (d)j,vi) The distance functions are different, different distance functions need to be switched in different scenes, the parameters of the distance functions externally provide standard restful interfaces, and when different temporary tasks come, each parameter of the distance functions can be customized by a user so as to better meet actual requirements.
S512: and selecting a target server group from the server groups.
And after clustering and grouping the servers according to the current centroids to obtain the server groups, selecting a target server group from the server groups.
The target server group is generally a relatively idle server with relatively abundant resources, low pressure, and less occupied resources.
S513: and distributing the service to be processed to each target server in the target server group so as to utilize each target server to process the service.
After the target server group is selected and obtained, the service to be processed is distributed to each target server in the target server group, and the service processing is carried out by using each target server, so that the high-efficiency service processing is realized.
It should be noted that, in the embodiments of the present invention, the "current" mentioned in each step refers to the corresponding parameter value newly obtained in the previous step.
Corresponding to the above method embodiment, the embodiment of the present invention further provides a server grouping apparatus, and the server grouping apparatus described below and the server grouping method described above may be referred to in correspondence with each other.
Referring to fig. 6, fig. 6 is a block diagram of a server grouping apparatus according to an embodiment of the present invention, where the apparatus may include:
the task obtaining module 61 is used for analyzing the received server grouping request to obtain a target grouping task;
a parameter obtaining module 62, configured to obtain parameters of each target server corresponding to the target grouping task in each server respectively;
and a server group obtaining module 63, configured to perform clustering grouping on the servers based on the target server parameters by using a fuzzy C-means clustering algorithm to obtain each server group.
The device provided by the embodiment of the invention is applied to analyze the received server grouping request to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group. By acquiring target server parameters of each server according to the target grouping task and automatically clustering and grouping each server based on each target server parameter by using the fuzzy C-means clustering algorithm, compared with the existing manual server grouping mode, the method greatly reduces the influence of human subjective factors, improves the grouping accuracy and greatly improves the grouping efficiency.
In one embodiment of the present invention, the parameter obtaining module 62 includes:
the original configuration parameter acquisition submodule is used for respectively acquiring each original configuration parameter corresponding to the server maintenance task in each server;
the parameter quantization submodule is used for performing quantization processing on each original configuration parameter by using a configuration quantization function to obtain each target configuration parameter;
the server group obtaining module is specifically a module for clustering and grouping the servers based on the target configuration parameters by using a fuzzy C-means clustering algorithm.
In one embodiment of the present invention, the parameter quantization sub-module comprises:
a quantization function obtaining unit, configured to obtain a configuration quantization function preset through a function preset interface;
and the parameter quantization unit is used for performing quantization processing on each original configuration parameter by using the configuration quantization function.
In one embodiment of the present invention, the apparatus may further include:
and the server maintenance module is used for clustering and grouping the servers based on the target configuration parameters by using a fuzzy C-means clustering algorithm to obtain each server group, and then performing parallel maintenance operation on the servers by taking each server group as a unit.
In one embodiment of the present invention, the parameter obtaining module 62 includes:
the implementation performance parameter acquisition submodule is used for respectively acquiring each real-time performance parameter corresponding to the service distribution task in each server;
the server group obtaining module is a module for clustering and grouping the servers based on the real-time performance parameters by using a fuzzy C-means clustering algorithm.
In one embodiment of the present invention, the apparatus may further include:
the server group selection module is used for clustering and grouping the servers based on the real-time performance parameters by using a fuzzy C-means clustering algorithm to obtain each server group, and then selecting a target server group from each server group;
and the service processing module is used for distributing the service to be processed to each target server in the target server group so as to process the service by using each target server.
In one embodiment of the present invention, the server group obtaining module 63 includes:
the server subset acquisition submodule is used for acquiring server subsets corresponding to the preset server classes respectively;
the mass center obtaining submodule is used for obtaining mass centers preset for the server subsets respectively;
the first distance calculation submodule is used for calculating each distance value between each server and each centroid according to each target server parameter;
the probability calculation submodule is used for calculating probability values of the servers belonging to the preset server classes according to the current distance values;
the mass center calculating submodule is used for recalculating each mass center according to each current probability value;
the second distance calculation submodule is used for calculating each distance value between each server and each recalculated centroid according to each target server parameter;
the value function calculation submodule is used for calculating a value function by combining the current probability values and the current distance values;
the judgment submodule is used for judging whether the value function meets the preset condition or not;
the repeated execution sub-module is used for repeatedly executing the step of respectively calculating the probability values of the servers belonging to the preset server classes according to the distance values when the fact that the value function does not meet the preset conditions is determined;
and the server group obtaining submodule is used for obtaining each current centroid when the fact that the value function meets the preset condition is determined, and clustering and grouping the servers according to each current centroid.
Corresponding to the above method embodiment, referring to fig. 7, fig. 7 is a schematic diagram of a server grouping device provided by the present invention, where the device may include:
a memory 71 for storing a computer program;
the processor 72, when executing the computer program stored in the memory 71, may implement the following steps:
analyzing the received server grouping request to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group.
For the introduction of the device provided by the present invention, please refer to the above method embodiment, which is not described herein again.
Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of:
analyzing the received server grouping request to obtain a target grouping task; respectively acquiring target server parameters corresponding to the target grouping tasks in the servers; and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For the introduction of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method.
The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A server grouping method, comprising:
analyzing the received server grouping request to obtain a target grouping task;
respectively acquiring parameters of each target server corresponding to the target grouping task in each server;
and clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm to obtain each server group.
2. The server grouping method according to claim 1, wherein when the target grouping task is a server maintenance task, respectively obtaining each target server parameter corresponding to the target grouping task in each server comprises:
respectively acquiring each original configuration parameter corresponding to the server maintenance task in each server;
carrying out quantization processing on each original configuration parameter by using a configuration quantization function to obtain each target configuration parameter;
clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm, wherein the clustering and grouping method comprises the following steps:
and clustering and grouping the servers based on the target configuration parameters by using the fuzzy C-means clustering algorithm.
3. The server grouping method according to claim 2, wherein quantizing each of the original configuration parameters using a configuration quantization function includes:
acquiring a configuration quantization function preset through a function preset interface;
and quantizing each original configuration parameter by using the configuration quantization function.
4. The server grouping method according to claim 3, wherein after clustering and grouping the servers based on the target configuration parameters by using the fuzzy C-means clustering algorithm to obtain the server groups, further comprising:
and performing parallel maintenance operation on the servers by taking the server groups as units.
5. The server grouping method according to claim 1, wherein when the target grouping task is a service distribution task, respectively obtaining each target server parameter corresponding to the target grouping task in each server comprises:
respectively acquiring each real-time performance parameter corresponding to the service distribution task in each server;
clustering and grouping the servers based on the parameters of the target servers by using a fuzzy C-means clustering algorithm, wherein the clustering and grouping method comprises the following steps:
and clustering and grouping the servers based on the real-time performance parameters by using the fuzzy C-means clustering algorithm.
6. The method of claim 5, wherein after clustering and grouping the servers based on the real-time performance parameters by using the fuzzy C-means clustering algorithm to obtain the server groups, the method further comprises:
selecting a target server group from each server group;
and distributing the service to be processed to each target server in the target server group so as to utilize each target server to process the service.
7. The server grouping method according to any one of claims 1 to 6, wherein clustering groups the servers based on the target server parameters using a fuzzy C-means clustering algorithm, comprises:
acquiring a server subset corresponding to each preset server class;
obtaining centroids preset for each server subset respectively;
respectively calculating each distance value between each server and each centroid according to each target server parameter;
respectively calculating probability values of the servers belonging to the preset server classes according to the current distance values;
recalculating each centroid according to each current probability value;
calculating each distance value between each server and each recalculated centroid according to each target server parameter;
calculating a value function by combining the current probability values and the current distance values;
judging whether the value function meets a preset condition or not;
if not, the step of respectively calculating the probability value of each server belonging to each preset server class according to each distance value is repeatedly executed;
and if so, acquiring each current centroid, and clustering and grouping the servers according to each current centroid.
8. A server grouping apparatus, comprising:
the task obtaining module is used for analyzing the received server grouping request to obtain a target grouping task;
the parameter acquisition module is used for respectively acquiring parameters of each target server corresponding to the target grouping task in each server;
and the server group obtaining module is used for clustering and grouping the servers based on the target server parameters by using a fuzzy C-means clustering algorithm to obtain the server groups.
9. A server grouping apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the server grouping method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the server grouping method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010725778.4A CN111901410A (en) | 2020-07-24 | 2020-07-24 | Server grouping method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010725778.4A CN111901410A (en) | 2020-07-24 | 2020-07-24 | Server grouping method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111901410A true CN111901410A (en) | 2020-11-06 |
Family
ID=73189915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010725778.4A Pending CN111901410A (en) | 2020-07-24 | 2020-07-24 | Server grouping method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111901410A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463381A (en) * | 2020-12-01 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Server automatic grouping method and device based on clustering validity index |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457575A (en) * | 2010-10-22 | 2012-05-16 | 国际商业机器公司 | Server consolidation system |
| US20120303618A1 (en) * | 2011-05-23 | 2012-11-29 | Cisco Technology, Inc. | Clustering-Based Resource Aggregation within a Data Center |
| CN106487591A (en) * | 2016-10-20 | 2017-03-08 | 郑州云海信息技术有限公司 | A kind of server automatic get-on carriage statistic device and its system |
| CN109729155A (en) * | 2018-12-13 | 2019-05-07 | 平安医疗健康管理股份有限公司 | A kind of distribution method and relevant apparatus of service request |
| CN109976901A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of resource regulating method, device, server and readable storage medium storing program for executing |
| CN110162388A (en) * | 2019-04-26 | 2019-08-23 | 深圳智链物联科技有限公司 | A kind of method for scheduling task, system and terminal device |
-
2020
- 2020-07-24 CN CN202010725778.4A patent/CN111901410A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457575A (en) * | 2010-10-22 | 2012-05-16 | 国际商业机器公司 | Server consolidation system |
| US20120303618A1 (en) * | 2011-05-23 | 2012-11-29 | Cisco Technology, Inc. | Clustering-Based Resource Aggregation within a Data Center |
| CN106487591A (en) * | 2016-10-20 | 2017-03-08 | 郑州云海信息技术有限公司 | A kind of server automatic get-on carriage statistic device and its system |
| CN109976901A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of resource regulating method, device, server and readable storage medium storing program for executing |
| CN109729155A (en) * | 2018-12-13 | 2019-05-07 | 平安医疗健康管理股份有限公司 | A kind of distribution method and relevant apparatus of service request |
| CN110162388A (en) * | 2019-04-26 | 2019-08-23 | 深圳智链物联科技有限公司 | A kind of method for scheduling task, system and terminal device |
Non-Patent Citations (1)
| Title |
|---|
| 肖宇鹏，何云斌，万静，李松: "《基于模糊Ｃ－均值的空间不确定数据聚类》", 《计算机工程》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463381A (en) * | 2020-12-01 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Server automatic grouping method and device based on clustering validity index |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| US10354201B1 (en) | Scalable clustering for mixed machine learning data | |
| CN111506390B (en) | Video transcoding scheduling method and system based on containerized deployment | |
| CN109167835A (en) | A kind of physics resource scheduling method and system based on kubernetes | |
| CN109740755B (en) | Data processing method and related device based on gradient descent method | |
| CN111913649B (en) | Data processing method and device for solid state disk | |
| CN112764920B (en) | Edge application deployment method, device, equipment and storage medium | |
| CN113296929A (en) | Resource matching method, device and system based on cloud computing | |
| US20220253222A1 (en) | Data reduction method, apparatus, computing device, and storage medium | |
| CN112527448A (en) | Openstack-based dynamic load adjustment method and system | |
| CN114780244A (en) | Container cloud resource elastic allocation method and device, computer equipment and medium | |
| CN114385463A (en) | Data acquisition method and device and electronic equipment | |
| CN111901410A (en) | Server grouping method, device, equipment and computer readable storage medium | |
| CN111314234B (en) | Flow distribution method and device, storage medium and electronic equipment | |
| CN117407178A (en) | Acceleration sub-card management method and system for self-adaptive load distribution | |
| CN116737387A (en) | Cloud computing resource prediction system and method based on power grid digital service | |
| CN116521335A (en) | Distributed task scheduling method and system for inclined image model production | |
| CN113419863B (en) | Data distribution processing method and device based on node capacity | |
| CN116363416A (en) | Image de-duplication method and device, electronic equipment and storage medium | |
| CN113760484B (en) | Data processing method and device | |
| CN116126415A (en) | Scene-based cloud component configuration recommendation method and system | |
| CN111598390B (en) | Method, device, equipment and readable storage medium for evaluating high availability of server | |
| CN111628901B (en) | Index anomaly detection method and related device | |
| CN113177613A (en) | System resource data distribution method and device | |
| CN114610234A (en) | Storage system parameter recommendation method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201106 |