WO2015196692A1 - Cloud computing system and processing method and apparatus for cloud computing system - Google Patents
Cloud computing system and processing method and apparatus for cloud computing system Download PDFInfo
- Publication number
- WO2015196692A1 WO2015196692A1 PCT/CN2014/090398 CN2014090398W WO2015196692A1 WO 2015196692 A1 WO2015196692 A1 WO 2015196692A1 CN 2014090398 W CN2014090398 W CN 2014090398W WO 2015196692 A1 WO2015196692 A1 WO 2015196692A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- cloud computing
- computing system
- node
- disk
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title abstract description 8
- 238000000034 method Methods 0.000 claims description 19
- 230000004044 response Effects 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 description 7
- 238000011084 recovery Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 230000002688 persistence Effects 0.000 description 2
- 238000013523 data management Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/2885—Hierarchically arranged intermediate devices, e.g. for hierarchical caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/40—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
Definitions
- the present invention relates to the field of cloud computing technologies, and in particular to a cloud computing system and a processing method and apparatus for a cloud computing system.
- Cloud Computing is Grid Computing, Distributed Computing, Parallel Computing, Utility Computing, Network Storage Technologies, and Virtualization. ), load balancing (Load Balance) and other traditional computer technology and network technology development and fusion products. It aims to integrate multiple relatively low-cost computing entities into a system with powerful computing capabilities through the network.
- Distributed caching is an area in the field of cloud computing. Its role is to provide distributed storage services for massive data and high-speed read and write access.
- the distributed cache system is composed of a plurality of server nodes and clients interconnected; the server node is responsible for data storage, and the client can perform operations such as writing, reading, updating, and deleting data to the server.
- data cannot be stored only on a single server node (hereinafter referred to as "node"), but a copy of the same data is stored on multiple nodes, which are backups of each other.
- the most common storage mode is the master-slave mode, in which one node acts as the master node and the other nodes act as slaves.
- the identity of the master node is obtained through election or other algorithms.
- data updates generally occur on the primary node.
- the standby node obtains data from the primary node for synchronization, and the data access can obtain data from the primary node or obtain data from the standby node, depending on the consistency of the access. Strategy.
- the data storage mode is generally classified according to NRW, where N represents the number of copies of the data, R represents the number of copies of the data obtained in one data access request, and W represents once.
- N represents the number of copies of the data
- R represents the number of copies of the data obtained in one data access request
- W represents once.
- the minimum number of participating nodes for the data update request ie, how many nodes are updated).
- the distributed caching system implements the persistence function
- the data distributed on the server is saved on the disk.
- the server cannot provide read and write services. Since the distributed cache system data has multiple replicas, the system can still provide read and write services through the replicas of other nodes as long as the other servers are in a normal state.
- a distributed cache system node has multiple disks attached, only one or a few of them are damaged for some reason, causing the server to fail to provide services. According to the foregoing, the entire cluster is still available because other servers are normally available. . Assume that during this time, another server has a similar situation, and that node is not able to provide services normally. It is likely that the number of copies cannot satisfy the NRW policy, and the distributed cache cluster is completely unable to provide services. Typically, when the more commonly used NRW is 3/2/2, the two nodes are dropped, only one node is normal, and the read and write operations cannot meet the minimum operation requirements on the two copies.
- the technical problem to be solved by the present invention is to provide a cloud computing system and a processing method and apparatus for the cloud computing system, which can improve the tolerance of the system to disk failure.
- an embodiment of the present invention provides a processing method of a cloud computing system, including:
- the node disk status report includes: a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
- the steps of performing corresponding operations include:
- the operation request is an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a minimum number of participating nodes of a predetermined data update request of the cloud computing system, Responding to the update request; otherwise, rejecting the update request; or
- the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by a predetermined data access request of the cloud computing system Responding to the data access request; otherwise, rejecting the data access request.
- the steps include:
- the primary node of the cloud computing system When the operation request is an update request, and the state of the disk of the primary node storing the data is normal, the primary node of the cloud computing system performs data update to the disk of the data of the primary node; the cloud computing The slave node of the system acquires data to be synchronized from the master node, and the slave node performs data update on the disk where the data of the slave node is located;
- the first slave node of the cloud computing system When the operation request is an update request, and the state of the disk of the primary node storing the data is a failure, the first slave node of the cloud computing system performs data to the disk where the data of the first slave node is located. Updating; the second slave node of the cloud computing system acquires data to be synchronized from the first slave node; and the second node performs data update to the disk where the data of the second slave node is located; The state of the disk storing the data by a slave node and the second slave node is normal.
- the requested steps include:
- the operation request is a data access request
- the state of the disk of the primary node storing the data is normal
- Obtaining a second copy of the data from a disk of the at least one slave node of the cloud computing system selecting, from the first copy and the second copy, a copy of the latest version; Sending a copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal;
- the operation request is a data access request
- the state of the disk of the primary node storing the data is a failure
- the method further includes:
- the node disk status report of the cloud computing system is obtained from the node.
- the embodiment of the invention further provides a processing device of a cloud computing system, including:
- a first receiving unit receiving an operation request of the client for the cloud computing system
- a search unit configured to search, according to a node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks;
- the node disk status report includes : a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
- the operation unit performs a corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- the operating unit includes:
- the operation request being an update request
- the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data update request of the cloud computing system Responding to the update request when at least the number of participating nodes;
- a first rejecting subunit rejecting the update when the number of the disks in the cloud computing system storing the data and in a normal state is less than a minimum number of participating nodes of a predetermined data update request of the cloud computing system request;
- the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data access request of the cloud computing system Responding to the data access request when the number of data copies obtained is obtained;
- a second rejecting subunit rejecting the data when the number of the disks in the cloud computing system storing the data and in a normal state is less than the number of data copies acquired by the cloud computing system for a predetermined data access request Access request.
- the device further includes:
- the second receiving unit receives the node disk status report of the cloud computing system from the node.
- An embodiment of the present invention further provides a cloud computing system, including: a client, a processing device, a node, and a disk corresponding to the node;
- Each of the nodes of the cloud computing system stores a respective disk of the data identifier corresponding data and a status of each of the disks;
- the node disk status report includes: the nodes in the cloud computing system The state of the disk, the data identifier corresponding to the data stored in the disk; and the corresponding operation according to the state of each of the disks storing the data identifier corresponding data in each node in the cloud computing system.
- the node sends a node disk status report to the processing device.
- FIG. 1 is a schematic flowchart of a processing method of a cloud computing system according to an embodiment of the present invention
- FIG. 2 is a schematic structural diagram of a processing apparatus of a cloud computing system according to an embodiment of the present invention
- FIG. 3 is a schematic structural diagram of a cloud computing system according to an embodiment of the present invention.
- FIG. 4 and FIG. 5 are schematic structural diagrams of an application scenario of a cloud computing system according to an embodiment of the present invention.
- a processing method of a cloud computing system includes:
- Step 11 Receive an operation request of the client for the cloud computing system; the operation request may be a data update request or a data access request.
- Step 12 Acquire, according to the operation request, a data identifier to be operated in the cloud computing system; for example, the operation request is to update the copy 1 in FIG. 4, and the copy 1 is a data identifier.
- Step 13 Search, according to the node disk status report of the cloud computing system, the status of each disk in which the data identifier corresponding data is stored in each node of the cloud computing system and the status of each of the disks; the node disk status report includes The state of the disk in each node of the cloud computing system, the data identifier corresponding to the data stored in the disk; the state of the disk is normal or faulty; in FIG. 4, the disk state report of node A is: (node A : Disk I, Replica 1, Failure; Disk II, Replica 2, Normal; Disk III, Replica 3, Normal).
- Step 14 Perform corresponding operations according to states of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- the method further includes:
- Step 10 Obtain a node disk status report of the cloud computing system from a node.
- the node sends a report if it detects that the disk storing the data is corrupted or fails, or sends a report based on the request.
- step 14 includes:
- the operation request is an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a minimum number of participating nodes of a predetermined data update request of the cloud computing system, Responding to the update request; otherwise, rejecting the update request; or
- the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by a predetermined data access request of the cloud computing system Responding to the data access request; otherwise, rejecting the data access request.
- the primary node of the cloud computing system When the operation request is an update request, and the state of the disk of the primary node storing the data is normal, the primary node of the cloud computing system performs data update to the disk of the data of the primary node; the cloud computing The slave node of the system acquires data to be synchronized from the master node, and the slave node performs data update on the disk where the data of the slave node is located;
- the first slave node of the cloud computing system When the operation request is an update request, and the state of the disk of the primary node storing the data is a failure, the first slave node of the cloud computing system performs data to the disk where the data of the first slave node is located. Updating; the second slave node of the cloud computing system acquires data to be synchronized from the first slave node; and the second node performs data update to the disk where the data of the second slave node is located; The state of the disk storing the data by a slave node and the second slave node is normal.
- the operation request is a data access request
- the state of the disk of the primary node storing the data is normal
- obtaining the first copy of the data from the disk where the data of the primary node of the cloud computing system is located Obtaining, from at least one (or two or three, according to actual conditions) of the cloud computing system, a second copy of the data from a disk on which the data of the node is located; from the first copy And selecting, in the second copy, a copy of the latest version; and sending the copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal;
- the operation request is a data access request
- the state of the disk of the primary node storing the data is a failure
- FIG. 5 is a distributed cache storage system composed of three nodes.
- the storage system has three copies of each data, and updates and accesses data in a manner of 322.
- the number of read request access copies specified by the cloud computing system is 2. When one disk is broken, it can still respond to the update or data access operation request. When two disks are broken, the operation request cannot be responded to.
- the system when a node disk failure occurs, and even multiple nodes simultaneously fail the disk, as long as the number of available disks on the cluster can satisfy the NRW policy, the system can ensure consistency and availability, and may even affect all data.
- the service, and the fact that the system is completely unable to provide services, will provide services as much as possible.
- a processing device of a cloud computing system includes:
- the first receiving unit 21 receives an operation request of the client for the cloud computing system
- the obtaining unit 22 is configured to acquire, according to the operation request, a data identifier to be operated in the cloud computing system;
- the searching unit 23 is configured to search, according to the node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report
- the method includes: a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
- the operation unit 24 performs a corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- the operating unit 24 includes:
- the operation request being an update request
- the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data update request of the cloud computing system Responding to the update request when at least the number of participating nodes;
- a first rejecting subunit rejecting the update when the number of the disks in the cloud computing system storing the data and in a normal state is less than a minimum number of participating nodes of a predetermined data update request of the cloud computing system request;
- the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data access request of the cloud computing system Responding to the data access request when the number of data copies obtained is obtained;
- a second rejecting subunit rejecting the data when the number of the disks in the cloud computing system storing the data and in a normal state is less than the number of data copies acquired by the cloud computing system for a predetermined data access request Access request.
- the device further includes:
- the second receiving unit 25 receives the node disk status report of the cloud computing system from the node.
- a cloud computing system includes: a client 31, a processing device 32, a node 33, and a disk 34 corresponding to the node 33;
- the processing device 32 receives an operation request of the client 31 for the cloud computing system; and according to the operation request, acquires a data identifier to be operated in the cloud computing system; and reports a node disk status according to the cloud computing system. And searching, in each of the nodes 33 of the cloud computing system, a disk that stores the data identifier corresponding data and a status of each of the disks 34; the node disk status report includes: each node 33 of the cloud computing system a state of the disk, a data identifier corresponding to the data stored in the disk; performing a corresponding operation according to a state of each of the disks 34 storing the data identifier corresponding data in each node in the cloud computing system .
- the node 33 sends a node disk status report to the processing device 32.
- the first application scenario describes a method for implementing availability when a disk is damaged in a multi-disk path in a cloud computing distributed cache system.
- Pre-step The client establishes a connection with multiple server nodes in the distributed cache system.
- the server nodes establish connections and run normally.
- Each server has several disks for data persistence. Different data points. Slices are persisted on different disks.
- the number of copies of the data is N
- the number of copies of the read request access is R
- write The minimum number of successful requests for copying is W
- Step A Under normal circumstances, all the disks on each node work normally, and the data has N copies in the system.
- the master performs data update processing on the disk where the data is located, the slave synchronizes the data from the master, and updates the data to the disk where the data is on the slave.
- the return is performed.
- Client data update success message
- the master/Slave processes the request, and after obtaining the accessed data copy from the disk where the R node data is located, the latest copy is selected from the R data copies and returned to the client.
- Step B When the node A starts, it finds that a certain disk failure is inaccessible, but the other disks are still normal. Or, during the running of the node A, it is found that the disk fails to access multiple times and it is determined that the disk is faulty. Node A does not switch to a node failure, but continues to provide read and write services, while recording the identity of the failed disk and the corresponding data copy on that disk.
- Step C When the client initiates a data update request, and the data is exactly distributed on the fault disk of node A described in step B, when updating data to the node, node A directly returns failure; when the data is updated in W After the node is successfully completed (the node A is not included in the W nodes), the data update success message is returned to the client;
- node A When the client initiates a data access request, node A directly returns a failure, and the master/Slave processes the request, and obtains the accessed data copy from the disk where the data of the R nodes (the R nodes do not include node A) is from the R. The latest copy is selected from the data copies and returned to the client.
- Step D When the client initiates a data update and an access request, and the data is not distributed on the fault disk of node A described in step B, the processing manner is the same as step A.
- Step E When Node B is running, multiple access failures to a disk determine that the disk is faulty. Node B does not switch to a node failure, but continues to provide read and write services, while recording the identity of the failed disk and the corresponding data copy on the disk.
- Step F When the client initiates a data update and access request, and the data is exactly distributed on the fault disk of the node B described in step E, based on the above assumption, not on the fault disk of the node A in step B, then When the node updates the data, the Node B directly returns the failure; when the data update is successfully completed on the W nodes (the W nodes do not include the Node B), the data update success message is returned to the client;
- the Node B When the client initiates a data access request, the Node B directly returns a failure, and the Master/Slave processes the request, and obtains the accessed data copy from the disk where the R nodes (the R nodes do not include the Node B) data, from the R The latest copy is selected from the data copy and returned to the client.
- Step G When the client initiates a data update request, and the data is exactly distributed on the fault disk of the node A described in step B, based on the above assumption, if it is not on the fault disk of the node B described in step E, then When the node updates and accesses the data, the process is the same as step C, and the result is that it can be updated and accessed normally.
- the system when a node disk failure occurs, and even multiple nodes simultaneously fail the disk, as long as the number of available copies of the remaining disk on the cluster can satisfy the NRW policy, the system can ensure consistency and availability, and may even have no effect. The service of all data, and the fact that the system is completely unable to provide services, will provide services as much as possible.
- the embodiment of the invention provides an implementation method for improving the availability of a distributed cache system in the case of multiple disk corruption.
- the consistency of the system is enhanced, and the availability of the system is enhanced, thereby optimizing the application experience.
- the server node and the client form a distributed cache system.
- a master node responsible for processing client updates and access requests.
- a distributed cache storage system consisting of three nodes.
- the storage system has three copies of each data and updates and accesses data in 322 mode.
- Step 1 in the initial normal phase, the system receives the client request, assuming that the data is located on the disk I of the node A, copy 1 (corresponding to the above data identification), the copy of the disk I of the node B, and the copy of the disk III of the node C. 1 on.
- replica 1 on node B is master, on the other two nodes.
- the copy is a slave.
- the copy 2 on node A is the master, and the copy on the other two nodes is the slave.
- the copy 3 on node A is the master, and the copy on the other two nodes is the slave.
- Step 2 When the client initiates a data update request, the Node B master updates the data to the copy 1 on the disk 1.
- the slave synchronizes the data from the master and updates the data to the disk where the data is on the slave.
- Step 3 As shown in FIG. 4, it is assumed that the disk I on the node A is damaged, resulting in the copy 1 being unavailable.
- the data of the update request initiated by the client is located on the replica 1 of the node A
- the data of the copy 1 of the disk I is updated by the master of the node B
- the slave of the node C synchronizes the data from the master
- the data is copied to the copy of the disk C of the node C.
- Data update is performed.
- the data update success message is returned to the client;
- Step 4 In the case of step 3, when the update and access request initiated by the client is located on the copy 2 or the copy 3 of the node A, since the copies of the three nodes are available, the processing flow is the same as the step 2.
- Step 5 as shown in FIG. 5, when the disk II on the node B is damaged, the copy 3 of the node B is unavailable.
- the data of the update and access request initiated by the client is located on the replica 1 of the node A, the replicas on the node B and the node C are available, and the NRW policy is satisfied, and the processing flow is the same as step 3.
- Step 6 In the case of step 5, when the update and access request initiated by the client is located on the copy 2 of the node A, since the copy 2 of the three nodes is available, the processing flow is the same as the step 2.
- Step 7 In the case of step 5, when the data of the update request initiated by the client is located on the copy 3 of the node A, the copy 3 of the node B is damaged, and the copy 3 of the node C is available.
- the data is updated by the A-Node master to the replica 3 on the disk III, the slave of the node C synchronizes the data from the master, and updates the data on the replica 3 of the disk C on the node C.
- the distributed cache cluster can provide read and write services for all data as long as the damaged disk copy is not duplicated.
- each node is actually partially damaged.
- the damaged disk does not store a copy of the same data, the actual available disk of the entire system. At the top, it still holds at least two copies of all the data, and is fully qualified to provide all services normally. Even if a copy of the same data is stored on the damaged disk, the available data on other disks can still meet the consistency and availability, and can provide read and write services, which cannot be provided only for the data that is damaged at the same time. Read and write access.
- the present invention is directed to a distributed cache system.
- the available resources can be fully utilized, and the copy resources conforming to the requirements of consistency and availability can be integrated to improve the availability of the system and improve the tolerance of the system to faults. That is to say, in the distributed cache system of the cloud computing field, a disk and data management mechanism is provided, and even in the case of a failure of a node part disk, the data on the available disk can be utilized as much as possible, and the ability to provide the service is maintained. Enables the server to provide consistent and available storage services with fewer disks or data resources.
- the above technical solution adopted by the embodiment of the present invention is directed to a distributed cache system, in the case of disk corruption, the available resources can be fully utilized, and the copy resources conforming to the consistency and availability requirements are integrated to improve the system availability as much as possible. Improve the system's tolerance to failure.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Hardware Redundancy (AREA)
Abstract
The present invention provides a cloud computing system and a processing method and apparatus for a cloud computing system. The processing method for a cloud computing system comprises: receiving an operation request of a client for a cloud computing system; acquiring a data identifier, on which an operation is to be performed, in the cloud computing system according to the operation request; searching for each disk, which stores data corresponding to the data identifier, in each node of the cloud computing system and the state of each disk according to a node disk state report of the cloud computing system, the node disk state report comprising states of disks in each node of the cloud computing system and data identifiers corresponding to data stored in the disks; and performing a corresponding operation according to the state of each disk, which stores the data corresponding to the data identifier, in each node of the cloud computing system. The present invention can improve the system tolerance of disk failure.
Description
本发明涉及云计算技术领域,特别是指一种云计算系统以及云计算系统的处理方法和装置。The present invention relates to the field of cloud computing technologies, and in particular to a cloud computing system and a processing method and apparatus for a cloud computing system.
目前,云计算(Cloud Computing)是网格计算(Grid Computing)、分布式计算(Distributed Computing)、并行计算(Parallel Computing)、效用计算(Utility Computing)、网络存储(Network Storage Technologies)虚拟化(Virtualization)、负载均衡(Load Balance)等传统计算机技术和网络技术发展融合的产物。它旨在通过网络把多个成本相对较低的计算实体,整合成一个具有强大计算能力的系统。分布式缓存是云计算范畴中的一个领域,其作用是提供海量数据的分布式存储服务以及高速读写访问的能力。Currently, Cloud Computing is Grid Computing, Distributed Computing, Parallel Computing, Utility Computing, Network Storage Technologies, and Virtualization. ), load balancing (Load Balance) and other traditional computer technology and network technology development and fusion products. It aims to integrate multiple relatively low-cost computing entities into a system with powerful computing capabilities through the network. Distributed caching is an area in the field of cloud computing. Its role is to provide distributed storage services for massive data and high-speed read and write access.
分布式缓存系统是由若干服务器节点和客户端互相连接构成的;服务器节点负责数据的存储,客户端可以对服务器做数据的写入、读取、更新、删除等操作。一般来说,数据不可能只保存在单个服务器节点(以下简称“节点”)上,而是在多台节点上保存同一个数据的副本,互为备份。最常见的存储模式为主备模式,其中一个节点作为主节点(master),其他节点作为备节点(slave),主节点的身份通过选举或其他算法获取。为简化流程,数据更新一般发生在主节点上,备节点从主节点获取数据进行同步,而数据访问可以从主节点中获取数据,也可以从备节点中获取数据,具体看该访问的一致性策略。The distributed cache system is composed of a plurality of server nodes and clients interconnected; the server node is responsible for data storage, and the client can perform operations such as writing, reading, updating, and deleting data to the server. In general, data cannot be stored only on a single server node (hereinafter referred to as "node"), but a copy of the same data is stored on multiple nodes, which are backups of each other. The most common storage mode is the master-slave mode, in which one node acts as the master node and the other nodes act as slaves. The identity of the master node is obtained through election or other algorithms. To simplify the process, data updates generally occur on the primary node. The standby node obtains data from the primary node for synchronization, and the data access can obtain data from the primary node or obtain data from the standby node, depending on the consistency of the access. Strategy.
在分布式缓存系统中,根据一致性及可用性的要求,一般将该数据存储方式按NRW进行分类,其中N表示数据的副本数、R表示一次数据访问请求中获取的数据副本数,W表示一次数据更新请求的最少参与节点数(即多少个节点上的数据更新完成)。In the distributed cache system, according to the requirements of consistency and availability, the data storage mode is generally classified according to NRW, where N represents the number of copies of the data, R represents the number of copies of the data obtained in one data access request, and W represents once. The minimum number of participating nodes for the data update request (ie, how many nodes are updated).
当分布式缓存系统实现持久化功能时,分布在该服务器上的数据保存在磁盘上。在实际情况下,如果磁盘发生故障,该服务器就无法提供读写服务了。由于分布式缓存系统数据保存有多个副本的特性,这时,只要其他服务器处于正常状态,系统依然可以通过其他节点的副本正常提供读写服务。
When the distributed caching system implements the persistence function, the data distributed on the server is saved on the disk. In the real world, if the disk fails, the server cannot provide read and write services. Since the distributed cache system data has multiple replicas, the system can still provide read and write services through the replicas of other nodes as long as the other servers are in a normal state.
如果分布式缓存系统节点挂接了多块磁盘,其中只有一个或者少数几个磁盘由于某种原因损坏,导致该服务器不能正常提供服务,根据前述,由于其他服务器为正常可用,整个集群还是可用的。假定在这段时间内,另一个服务器也发生了类似情况,那个节点也不能正常提供服务,很可能使得副本数无法满足NRW策略,那么分布式缓存集群就彻底无法提供服务了。典型的情况是在比较常用的NRW为3/2/2的条件下,两个节点宕掉,只有一个节点正常,读写操作都无法满足最小在两个副本上操作的要求。If a distributed cache system node has multiple disks attached, only one or a few of them are damaged for some reason, causing the server to fail to provide services. According to the foregoing, the entire cluster is still available because other servers are normally available. . Assume that during this time, another server has a similar situation, and that node is not able to provide services normally. It is likely that the number of copies cannot satisfy the NRW policy, and the distributed cache cluster is completely unable to provide services. Typically, when the more commonly used NRW is 3/2/2, the two nodes are dropped, only one node is normal, and the read and write operations cannot meet the minimum operation requirements on the two copies.
发明内容Summary of the invention
本发明要解决的技术问题是,提供一种云计算系统以及云计算系统的处理方法和装置,能够提高系统对磁盘故障的容忍性。The technical problem to be solved by the present invention is to provide a cloud computing system and a processing method and apparatus for the cloud computing system, which can improve the tolerance of the system to disk failure.
为解决上述技术问题,本发明的实施例提供一种云计算系统的处理方法,包括:To solve the above technical problem, an embodiment of the present invention provides a processing method of a cloud computing system, including:
接收客户端对云计算系统的操作请求;Receiving an operation request of the client to the cloud computing system;
根据所述操作请求,获取所述云计算系统中待操作的数据标识;Obtaining, according to the operation request, a data identifier to be operated in the cloud computing system;
根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;And searching, according to the node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report includes: a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。And performing corresponding operations according to states of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
所述根据各个所述磁盘的状态,进行相应的操作的步骤包括:According to the state of each of the disks, the steps of performing corresponding operations include:
所述操作请求为更新请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;否则,拒绝所述更新请求;或者The operation request is an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a minimum number of participating nodes of a predetermined data update request of the cloud computing system, Responding to the update request; otherwise, rejecting the update request; or
所述操作请求为数据访问请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;否则,拒绝所述数据访问请求。
The operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by a predetermined data access request of the cloud computing system Responding to the data access request; otherwise, rejecting the data access request.
所述当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求的步骤包括:Responding to the update request when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the minimum number of participating nodes of the data update request scheduled by the cloud computing system The steps include:
当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为正常时,所述云计算系统的主节点向主节点的所述数据所在磁盘进行数据更新;所述云计算系统的从节点从所述主节点获取待同步的数据,所述从节点向所述从节点的所述数据所在磁盘进行数据更新;When the operation request is an update request, and the state of the disk of the primary node storing the data is normal, the primary node of the cloud computing system performs data update to the disk of the data of the primary node; the cloud computing The slave node of the system acquires data to be synchronized from the master node, and the slave node performs data update on the disk where the data of the slave node is located;
当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为故障时,所述云计算系统的第一从节点向所述第一从节点的所述数据所在磁盘进行数据更新;所述云计算系统的第二从节点从所述第一从节点获取待同步的数据;所述第二节点向所述第二从节点的所述数据所在磁盘进行数据更新;所述第一从节点和所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is an update request, and the state of the disk of the primary node storing the data is a failure, the first slave node of the cloud computing system performs data to the disk where the data of the first slave node is located. Updating; the second slave node of the cloud computing system acquires data to be synchronized from the first slave node; and the second node performs data update to the disk where the data of the second slave node is located; The state of the disk storing the data by a slave node and the second slave node is normal.
所述当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求的步骤包括:Responding to the data access when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by the cloud computing system for a predetermined data access request The requested steps include:
当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为正常时,从所述云计算系统的主节点的所述数据所在磁盘中获取所述数据的第一副本,从所述云计算系统的至少一个从节点的所述数据所在磁盘中获取所述数据的第二副本;从所述第一副本和所述第二副本中,选取最新版本的副本;并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常;When the operation request is a data access request, and the state of the disk of the primary node storing the data is normal, obtaining the first copy of the data from the disk where the data of the primary node of the cloud computing system is located Obtaining a second copy of the data from a disk of the at least one slave node of the cloud computing system; selecting, from the first copy and the second copy, a copy of the latest version; Sending a copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal;
当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为故障时,从所述云计算系统的至少一个从节点的所述数据所在磁盘中获取所述数据的第三副本;从至少一个所述第三副本中,选取最新版本的副本,并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is a data access request, and the state of the disk of the primary node storing the data is a failure, obtaining the data from the disk where the data of the at least one slave node of the cloud computing system is located Three copies; from at least one of the third copies, selecting a copy of the latest version and transmitting the copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal.
所述接收客户端的操作请求的步骤之前,所述方法还包括:Before the step of receiving an operation request of the client, the method further includes:
从节点获取所述云计算系统的节点磁盘状态报告。The node disk status report of the cloud computing system is obtained from the node.
本发明实施例还提供一种云计算系统的处理装置,包括:The embodiment of the invention further provides a processing device of a cloud computing system, including:
第一接收单元,接收客户端对云计算系统的操作请求;
a first receiving unit, receiving an operation request of the client for the cloud computing system;
获取单元,根据所述操作请求,获取所述云计算系统中待操作的数据标识;Obtaining, according to the operation request, acquiring a data identifier to be operated in the cloud computing system;
查找单元,根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;a search unit, configured to search, according to a node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report includes : a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
操作单元,根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。The operation unit performs a corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
所述操作单元包括:The operating unit includes:
第一响应子单元,所述操作请求为更新请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;a first response subunit, the operation request being an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data update request of the cloud computing system Responding to the update request when at least the number of participating nodes;
第一拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,拒绝所述更新请求;a first rejecting subunit, rejecting the update when the number of the disks in the cloud computing system storing the data and in a normal state is less than a minimum number of participating nodes of a predetermined data update request of the cloud computing system request;
第二响应子单元,所述操作请求为数据访问请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;a second response subunit, the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data access request of the cloud computing system Responding to the data access request when the number of data copies obtained is obtained;
第二拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,拒绝所述数据访问请求。a second rejecting subunit, rejecting the data when the number of the disks in the cloud computing system storing the data and in a normal state is less than the number of data copies acquired by the cloud computing system for a predetermined data access request Access request.
所述的装置,还包括:The device further includes:
第二接收单元,从节点接收所述云计算系统的节点磁盘状态报告。The second receiving unit receives the node disk status report of the cloud computing system from the node.
本发明实施例还提供一种云计算系统,包括:客户端、处理装置、节点、所述节点对应的磁盘;An embodiment of the present invention further provides a cloud computing system, including: a client, a processing device, a node, and a disk corresponding to the node;
所述处理装置,接收所述客户端对云计算系统的操作请求;根据所述操作请求,获取所述云计算系统中待操作的数据标识;根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个所述节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中所述
磁盘的状态、所述磁盘中存储的数据所对应的数据标识;根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。Receiving, by the processing device, an operation request of the client to the cloud computing system; acquiring, according to the operation request, a data identifier to be operated in the cloud computing system; searching according to a node disk status report of the cloud computing system Each of the nodes of the cloud computing system stores a respective disk of the data identifier corresponding data and a status of each of the disks; the node disk status report includes: the nodes in the cloud computing system
The state of the disk, the data identifier corresponding to the data stored in the disk; and the corresponding operation according to the state of each of the disks storing the data identifier corresponding data in each node in the cloud computing system.
所述节点,给所述处理装置发送节点磁盘状态报告。The node sends a node disk status report to the processing device.
本发明实施例提供的上述技术方案的有益效果如下:The beneficial effects of the above technical solutions provided by the embodiments of the present invention are as follows:
针对分布式缓存系统,在有磁盘损坏的情况下,可以充分利用可用的资源,整合出符合一致性和可用性要求的副本资源,尽可能提高系统的可用性,提高系统对故障的容忍性。For distributed cache systems, in the case of disk corruption, you can make full use of available resources, consolidate replica resources that meet the requirements of consistency and availability, maximize system availability, and improve system tolerance to failure.
图1为本发明实施例所述的一种云计算系统的处理方法的流程示意图;1 is a schematic flowchart of a processing method of a cloud computing system according to an embodiment of the present invention;
图2为本发明实施例所述的一种云计算系统的处理装置的结构示意图;2 is a schematic structural diagram of a processing apparatus of a cloud computing system according to an embodiment of the present invention;
图3为本发明实施例所述的一种云计算系统的结构示意图;3 is a schematic structural diagram of a cloud computing system according to an embodiment of the present invention;
图4和图5均为本发明实施例所述的一种云计算系统的应用场景的结构示意图。4 and FIG. 5 are schematic structural diagrams of an application scenario of a cloud computing system according to an embodiment of the present invention.
为使本发明要解决的技术问题、技术方案和优点更加清楚,下面将结合附图及具体实施例进行详细描述。The technical problems, the technical solutions, and the advantages of the present invention will be more clearly described in the following description.
如图1所示,为本发明实施例所述的一种云计算系统的处理方法,包括:As shown in FIG. 1 , a processing method of a cloud computing system according to an embodiment of the present invention includes:
步骤11,接收客户端对云计算系统的操作请求;操作请求可以为数据更新请求或者数据访问请求等。Step 11: Receive an operation request of the client for the cloud computing system; the operation request may be a data update request or a data access request.
步骤12,根据所述操作请求,获取所述云计算系统中待操作的数据标识;例如,操作请求为更新图4中的副本1,副本1为数据标识。Step 12: Acquire, according to the operation request, a data identifier to be operated in the cloud computing system; for example, the operation request is to update the copy 1 in FIG. 4, and the copy 1 is a data identifier.
步骤13,根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;磁盘的状态为正常或者故障,图4中,节点A的磁盘状态报告为:(节点A:磁盘I,副本1,故障;磁盘II,副本2,正常;磁盘III,副本3,正常)。
Step 13: Search, according to the node disk status report of the cloud computing system, the status of each disk in which the data identifier corresponding data is stored in each node of the cloud computing system and the status of each of the disks; the node disk status report includes The state of the disk in each node of the cloud computing system, the data identifier corresponding to the data stored in the disk; the state of the disk is normal or faulty; in FIG. 4, the disk state report of node A is: (node A : Disk I, Replica 1, Failure; Disk II, Replica 2, Normal; Disk III, Replica 3, Normal).
步骤14,根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。Step 14: Perform corresponding operations according to states of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
步骤14之前,所述方法还包括:Before step 14, the method further includes:
步骤10,从节点获取所述云计算系统的节点磁盘状态报告。节点检测到存储一数据的磁盘损坏或者发生故障,则发送报告;或者基于请求来发送报告。Step 10: Obtain a node disk status report of the cloud computing system from a node. The node sends a report if it detects that the disk storing the data is corrupted or fails, or sends a report based on the request.
其中,步骤14包括:Wherein step 14 includes:
所述操作请求为更新请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;否则,拒绝所述更新请求;或者The operation request is an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a minimum number of participating nodes of a predetermined data update request of the cloud computing system, Responding to the update request; otherwise, rejecting the update request; or
所述操作请求为数据访问请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;否则,拒绝所述数据访问请求。The operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by a predetermined data access request of the cloud computing system Responding to the data access request; otherwise, rejecting the data access request.
具体为:Specifically:
当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为正常时,所述云计算系统的主节点向主节点的所述数据所在磁盘进行数据更新;所述云计算系统的从节点从所述主节点获取待同步的数据,所述从节点向所述从节点的所述数据所在磁盘进行数据更新;When the operation request is an update request, and the state of the disk of the primary node storing the data is normal, the primary node of the cloud computing system performs data update to the disk of the data of the primary node; the cloud computing The slave node of the system acquires data to be synchronized from the master node, and the slave node performs data update on the disk where the data of the slave node is located;
当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为故障时,所述云计算系统的第一从节点向所述第一从节点的所述数据所在磁盘进行数据更新;所述云计算系统的第二从节点从所述第一从节点获取待同步的数据;所述第二节点向所述第二从节点的所述数据所在磁盘进行数据更新;所述第一从节点和所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is an update request, and the state of the disk of the primary node storing the data is a failure, the first slave node of the cloud computing system performs data to the disk where the data of the first slave node is located. Updating; the second slave node of the cloud computing system acquires data to be synchronized from the first slave node; and the second node performs data update to the disk where the data of the second slave node is located; The state of the disk storing the data by a slave node and the second slave node is normal.
当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为正常时,从所述云计算系统的主节点的所述数据所在磁盘中获取所述数据的第一副本,从所述云计算系统的至少一个(也可以为两个或者3个,根据实际情况设定)从节点的所述数据所在磁盘中获取所述数据的第二副本;从所述第一副本和所述第二副本中,选取最新版本的副本;并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常;
When the operation request is a data access request, and the state of the disk of the primary node storing the data is normal, obtaining the first copy of the data from the disk where the data of the primary node of the cloud computing system is located Obtaining, from at least one (or two or three, according to actual conditions) of the cloud computing system, a second copy of the data from a disk on which the data of the node is located; from the first copy And selecting, in the second copy, a copy of the latest version; and sending the copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal;
当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为故障时,从所述云计算系统的至少一个从节点的所述数据所在磁盘中获取所述数据的第三副本;从至少一个所述第三副本中,选取最新版本的副本,并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is a data access request, and the state of the disk of the primary node storing the data is a failure, obtaining the data from the disk where the data of the at least one slave node of the cloud computing system is located Three copies; from at least one of the third copies, selecting a copy of the latest version and transmitting the copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal.
例如,图5为一个由3个节点组成的分布式缓存存储系统,该存储系统每个数据有三个副本,采用322的方式更新及访问数据。云计算系统规定的读请求访问副本数量为2,当有一个磁盘坏掉时,仍然能够响应更新或数据访问操作请求,当有两个磁盘坏掉时,则不能响应操作请求。For example, FIG. 5 is a distributed cache storage system composed of three nodes. The storage system has three copies of each data, and updates and accesses data in a manner of 322. The number of read request access copies specified by the cloud computing system is 2. When one disk is broken, it can still respond to the update or data access operation request. When two disks are broken, the operation request cannot be responded to.
本发明中,当发生节点磁盘故障,甚至多个节点同时发生故障磁盘,只要集群上剩余的可用磁盘上副本数能满足NRW策略,系统就可以保证一致性和可用性,甚至可能毫不影响所有数据的服务,更不会发生系统彻底无法提供服务的情况,也就尽可能提供了服务。In the present invention, when a node disk failure occurs, and even multiple nodes simultaneously fail the disk, as long as the number of available disks on the cluster can satisfy the NRW policy, the system can ensure consistency and availability, and may even affect all data. The service, and the fact that the system is completely unable to provide services, will provide services as much as possible.
当然,在部分磁盘损坏继续提供服务的情况下,随之带来磁盘恢复后数据的恢复问题,这可以通过分布式缓存数据恢复功能完成,也就是从其他节点上获取副本数据来修复。Of course, in the case that part of the disk damage continues to provide services, the problem of data recovery after disk recovery is brought about by the distributed cache data recovery function, that is, the copy data is obtained from other nodes to be repaired.
如图2所示,为本发明实施例所述的一种云计算系统的处理装置,包括:As shown in FIG. 2, a processing device of a cloud computing system according to an embodiment of the present invention includes:
第一接收单元21,接收客户端对云计算系统的操作请求;The first receiving unit 21 receives an operation request of the client for the cloud computing system;
获取单元22,根据所述操作请求,获取所述云计算系统中待操作的数据标识;The obtaining unit 22 is configured to acquire, according to the operation request, a data identifier to be operated in the cloud computing system;
查找单元23,根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;The searching unit 23 is configured to search, according to the node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report The method includes: a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;
操作单元24,根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。The operation unit 24 performs a corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
所述操作单元24包括:The operating unit 24 includes:
第一响应子单元,所述操作请求为更新请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;
a first response subunit, the operation request being an update request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data update request of the cloud computing system Responding to the update request when at least the number of participating nodes;
第一拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,拒绝所述更新请求;a first rejecting subunit, rejecting the update when the number of the disks in the cloud computing system storing the data and in a normal state is less than a minimum number of participating nodes of a predetermined data update request of the cloud computing system request;
第二响应子单元,所述操作请求为数据访问请求;当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;a second response subunit, the operation request is a data access request; when the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data access request of the cloud computing system Responding to the data access request when the number of data copies obtained is obtained;
第二拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,拒绝所述数据访问请求。a second rejecting subunit, rejecting the data when the number of the disks in the cloud computing system storing the data and in a normal state is less than the number of data copies acquired by the cloud computing system for a predetermined data access request Access request.
所述的装置,还包括:The device further includes:
第二接收单元25,从节点接收所述云计算系统的节点磁盘状态报告。The second receiving unit 25 receives the node disk status report of the cloud computing system from the node.
如图3所示,为本发明所述的一种云计算系统,包括:客户端31、处理装置32、节点33、所述节点33对应的磁盘34;As shown in FIG. 3, a cloud computing system according to the present invention includes: a client 31, a processing device 32, a node 33, and a disk 34 corresponding to the node 33;
所述处理装置32,接收所述客户端31对云计算系统的操作请求;根据所述操作请求,获取所述云计算系统中待操作的数据标识;根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个所述节点33中存储所述数据标识对应数据的磁盘以及各个所述磁盘34的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点33中所述磁盘的状态、所述磁盘中存储的数据所对应的数据标识;根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘34的状态,进行相应的操作。The processing device 32 receives an operation request of the client 31 for the cloud computing system; and according to the operation request, acquires a data identifier to be operated in the cloud computing system; and reports a node disk status according to the cloud computing system. And searching, in each of the nodes 33 of the cloud computing system, a disk that stores the data identifier corresponding data and a status of each of the disks 34; the node disk status report includes: each node 33 of the cloud computing system a state of the disk, a data identifier corresponding to the data stored in the disk; performing a corresponding operation according to a state of each of the disks 34 storing the data identifier corresponding data in each node in the cloud computing system .
所述节点33,给所述处理装置32发送节点磁盘状态报告。The node 33 sends a node disk status report to the processing device 32.
以下描述本发明实施例的两个应用场景。Two application scenarios of the embodiment of the present invention are described below.
第一应用场景为描述一种云计算分布式缓存系统中、多磁盘路径下当磁盘损坏情况时可用性的实现方法。The first application scenario describes a method for implementing availability when a disk is damaged in a multi-disk path in a cloud computing distributed cache system.
前置步骤:客户端与分布式缓存系统中多个服务器节点建立连接,服务器节点间之间互相建立连接并且运行正常,每个服务器都有若干块磁盘用于数据的持久化,不同的数据分片持久化在不同的磁盘上。数据副本数为N,读请求访问副本数为R,写
请求最少更新副本成功数为W,系统的单次最大故障容忍度为O(表示容许O个节点上的请求发生故障,如单点故障则O=1,O<W),一致性要求W+R>N。Pre-step: The client establishes a connection with multiple server nodes in the distributed cache system. The server nodes establish connections and run normally. Each server has several disks for data persistence. Different data points. Slices are persisted on different disks. The number of copies of the data is N, and the number of copies of the read request access is R, write
The minimum number of successful requests for copying is W, and the system's single maximum fault tolerance is O (indicating that the request on O nodes is allowed to fail, such as single point failure, O=1, O<W), consistency requirement W+ R>N.
步骤A:系统正常情况下,每个节点上的所有磁盘正常工作,数据在系统中有N个副本。当客户端发起数据更新请求时,由Master向数据所在磁盘进行数据更新处理,slave从master同步数据,并向slave上数据所在磁盘进行数据更新,当数据更新在W个节点上成功完成后,返回客户端数据更新成功消息;Step A: Under normal circumstances, all the disks on each node work normally, and the data has N copies in the system. When the client initiates a data update request, the master performs data update processing on the disk where the data is located, the slave synchronizes the data from the master, and updates the data to the disk where the data is on the slave. When the data update is successfully completed on the W nodes, the return is performed. Client data update success message;
当客户端发起数据访问请求时,由Master/Slave处理请求,从R个节点数据所在磁盘获取访问的数据副本后,从该R个数据副本中选取最新的副本返回给客户端。When the client initiates a data access request, the master/Slave processes the request, and after obtaining the accessed data copy from the disk where the R node data is located, the latest copy is selected from the R data copies and returned to the client.
步骤B:节点A启动时,发现某个磁盘故障无法访问,但其他磁盘仍然正常;或者,节点A运行过程中,发现某个磁盘多次访问失败,判定为该磁盘故障。节点A不切换成节点故障,而是继续提供读写服务,同时记录下故障磁盘和该磁盘上对应的数据副本的标识。Step B: When the node A starts, it finds that a certain disk failure is inaccessible, but the other disks are still normal. Or, during the running of the node A, it is found that the disk fails to access multiple times and it is determined that the disk is faulty. Node A does not switch to a node failure, but continues to provide read and write services, while recording the identity of the failed disk and the corresponding data copy on that disk.
步骤C:当客户端发起数据更新请求时,并且该数据恰好分布在步骤B所述节点A的故障磁盘上,则当向该节点更新数据时,节点A直接返回失败;当数据更新在W个节点上(这W个节点中不包含节点A)成功完成后,返回给客户端数据更新成功消息;Step C: When the client initiates a data update request, and the data is exactly distributed on the fault disk of node A described in step B, when updating data to the node, node A directly returns failure; when the data is updated in W After the node is successfully completed (the node A is not included in the W nodes), the data update success message is returned to the client;
当客户端发起数据访问请求时,节点A直接返回失败,由Master/Slave处理请求,从R个节点(这R个节点中不包含节点A)数据所在磁盘获取访问的数据副本后,从该R个数据副本中选取最新的副本返回给客户端。When the client initiates a data access request, node A directly returns a failure, and the master/Slave processes the request, and obtains the accessed data copy from the disk where the data of the R nodes (the R nodes do not include node A) is from the R. The latest copy is selected from the data copies and returned to the client.
步骤D:当客户端发起数据更新和访问请求,并且该数据不分布在步骤B所述节点A的故障磁盘上,则处理方式同步骤A。Step D: When the client initiates a data update and an access request, and the data is not distributed on the fault disk of node A described in step B, the processing manner is the same as step A.
步骤E:当节点B在运行过程中,对某个磁盘多次访问失败判定该磁盘为故障。节点B不切换成节点故障,而是继续提供读写服务,同时记录下故障磁盘和该磁盘上对应的数据副本的标识。Step E: When Node B is running, multiple access failures to a disk determine that the disk is faulty. Node B does not switch to a node failure, but continues to provide read and write services, while recording the identity of the failed disk and the corresponding data copy on the disk.
假定节点B的故障磁盘和节点A的故障磁盘上保存的副本无重合。继续下一步骤。Assume that the failed disk of Node B and the copy saved on the failed disk of Node A do not coincide. Continue to the next step.
步骤F:当客户端发起数据更新和访问请求,并且该数据恰好分布在步骤E所述节点B的故障磁盘上,基于上述假定,则不在步骤B所述节点A的故障磁盘上,则当向该节点更新数据时,节点B直接返回失败;当数据更新在W个节点上(这W个节点中不包含节点B)成功完成后,返回给客户端数据更新成功消息;
Step F: When the client initiates a data update and access request, and the data is exactly distributed on the fault disk of the node B described in step E, based on the above assumption, not on the fault disk of the node A in step B, then When the node updates the data, the Node B directly returns the failure; when the data update is successfully completed on the W nodes (the W nodes do not include the Node B), the data update success message is returned to the client;
当客户端发起数据访问请求时,节点B直接返回失败,由Master/Slave处理请求,从R个节点(这R个节点中不包含节点B)数据所在磁盘获取访问的数据副本后,从该R个数据副本中选取最新的副本,返回给客户端。When the client initiates a data access request, the Node B directly returns a failure, and the Master/Slave processes the request, and obtains the accessed data copy from the disk where the R nodes (the R nodes do not include the Node B) data, from the R The latest copy is selected from the data copy and returned to the client.
步骤G:当客户端发起数据更新请求时,并且该数据恰好分布在步骤B所述节点A的故障磁盘上,基于上述假定,则不在步骤E所述节点B的故障磁盘上,则当向该节点更新和访问数据时,处理过程同步骤C,结果是可以正常更新和访问到。Step G: When the client initiates a data update request, and the data is exactly distributed on the fault disk of the node A described in step B, based on the above assumption, if it is not on the fault disk of the node B described in step E, then When the node updates and accesses the data, the process is the same as step C, and the result is that it can be updated and accessed normally.
本发明实施例中,当发生节点磁盘故障,甚至多个节点同时发生故障磁盘,只要集群上剩余的可用磁盘上副本数能满足NRW策略,系统就可以保证一致性和可用性,甚至可能毫不影响所有数据的服务,更不会发生系统彻底无法提供服务的情况,也就尽可能提供了服务。In the embodiment of the present invention, when a node disk failure occurs, and even multiple nodes simultaneously fail the disk, as long as the number of available copies of the remaining disk on the cluster can satisfy the NRW policy, the system can ensure consistency and availability, and may even have no effect. The service of all data, and the fact that the system is completely unable to provide services, will provide services as much as possible.
当然,在部分磁盘损坏继续提供服务的情况下,随之带来磁盘恢复后数据的恢复问题,这可以通过分布式缓存数据恢复功能完成,也就是从其他节点上获取副本数据来修复。Of course, in the case that part of the disk damage continues to provide services, the problem of data recovery after disk recovery is brought about by the distributed cache data recovery function, that is, the copy data is obtained from other nodes to be repaired.
本发明实施例提供了一种在分布式缓存系统在多磁盘损坏情况下提高可用性的实现方法,在一致性不变的情况下,增强了系统的可用性,从而优化了应用体验。The embodiment of the invention provides an implementation method for improving the availability of a distributed cache system in the case of multiple disk corruption. The consistency of the system is enhanced, and the availability of the system is enhanced, thereby optimizing the application experience.
以下结合图4和图5,描述第二应用场景。The second application scenario will be described below with reference to FIGS. 4 and 5.
具体为:针对322模式的主备存储系统详细描述单节点出现磁盘损坏和多节点同时出现磁盘损坏下,可用性实现方案。Specifically: for the 322 mode primary and secondary storage systems, a detailed description of the availability of a single node disk damage and multi-node disk corruption, availability implementation.
由服务器节点和客户端构成分布式缓存系统,对一个特定的数据,有一个主节点(master)负责处理客户端的更新及访问请求,有若干个备节点用于同步master的数据并接收客户端的数据访问请求(slave不处理数据更新请求)。The server node and the client form a distributed cache system. For a specific data, there is a master node responsible for processing client updates and access requests. There are several standby nodes for synchronizing the master data and receiving client data. Access request (slave does not process data update request).
环境:一个由3个节点组成的分布式缓存存储系统,该存储系统每个数据有三个副本,采用322的方式更新及访问数据。Environment: A distributed cache storage system consisting of three nodes. The storage system has three copies of each data and updates and accesses data in 322 mode.
本发明实施例包括如下步骤:Embodiments of the present invention include the following steps:
步骤1,初始正常阶段,系统接收客户端请求,假定数据位于节点A的磁盘I上副本1(相当于上述的数据标识)、节点B的磁盘I上副本1、以及节点C的磁盘III上副本1上。为描述简化起见,假定节点B上的副本1是master,其他两个节点上的
副本是slave。节点A上的副本2是master,其他两个节点上的副本是slave。节点A上的副本3是master,其他两个节点上的副本是slave。Step 1, in the initial normal phase, the system receives the client request, assuming that the data is located on the disk I of the node A, copy 1 (corresponding to the above data identification), the copy of the disk I of the node B, and the copy of the disk III of the node C. 1 on. For the sake of simplicity of description, assume that replica 1 on node B is master, on the other two nodes.
The copy is a slave. The copy 2 on node A is the master, and the copy on the other two nodes is the slave. The copy 3 on node A is the master, and the copy on the other two nodes is the slave.
步骤2,当客户端发起数据更新请求时,由B节点Master向磁盘I上副本1进行数据更新,slave从master同步数据,并向slave上数据所在磁盘进行数据更新,当数据更新在W=2个节点上成功完成后,返回给客户端数据更新成功消息。由于所有磁盘都正常,实际所有副本都更新成功了;当客户端发起数据访问请求时,三个节点都处理请求,从R=2个节点数据所在磁盘获取访问的数据副本后,返回客户端,实际所有节点副本都读取成功了。Step 2: When the client initiates a data update request, the Node B master updates the data to the copy 1 on the disk 1. The slave synchronizes the data from the master and updates the data to the disk where the data is on the slave. When the data is updated at W=2 After successful completion on each node, the data update success message is returned to the client. Since all the disks are normal, all the actual copies are successfully updated; when the client initiates a data access request, all three nodes process the request, and obtain the accessed data copy from the disk where the R=2 node data is located, and then return to the client. All actual node copies are read successfully.
步骤3,如图4所示,假定节点A上磁盘I损坏,导致副本1不可用。当客户端发起的更新请求的数据位于节点A副本1上时,由B节点Master向磁盘I上副本1进行数据更新,节点C的slave从master同步数据,并向节点C磁盘III上副本上数据进行数据更新,这时,数据更新在W=2个节点上成功完成后,返回给客户端数据更新成功消息;Step 3. As shown in FIG. 4, it is assumed that the disk I on the node A is damaged, resulting in the copy 1 being unavailable. When the data of the update request initiated by the client is located on the replica 1 of the node A, the data of the copy 1 of the disk I is updated by the master of the node B, the slave of the node C synchronizes the data from the master, and the data is copied to the copy of the disk C of the node C. Data update is performed. At this time, after the data update is successfully completed on W=2 nodes, the data update success message is returned to the client;
当客户端发起数据访问请求的数据位于节点A副本1上时,节点A直接返回失败,从节点B和节点C的副本1上获得数据后,(满足R=2)返回给客户端。When the data of the client initiating the data access request is located on the replica 1 of the node A, the node A directly returns the failure, and after obtaining the data from the replica 1 of the node B and the node C, (satisfying R=2) is returned to the client.
步骤4,在步骤3情况下,当客户端发起的更新和访问请求位于节点A副本2或者副本3上时,由于三个节点的副本均可用,则处理流程同步骤2。Step 4: In the case of step 3, when the update and access request initiated by the client is located on the copy 2 or the copy 3 of the node A, since the copies of the three nodes are available, the processing flow is the same as the step 2.
步骤5,如图5所示,当节点B上磁盘II损坏,导致节点B的副本3不可用。当客户端发起的更新和访问请求的数据位于节点A副本1上时,节点B和节点C上的副本均可用,满足NRW策略,则处理流程同步骤3。Step 5, as shown in FIG. 5, when the disk II on the node B is damaged, the copy 3 of the node B is unavailable. When the data of the update and access request initiated by the client is located on the replica 1 of the node A, the replicas on the node B and the node C are available, and the NRW policy is satisfied, and the processing flow is the same as step 3.
步骤6,在步骤5情况下,当客户端发起的更新和访问请求位于节点A副本2上时,由于三个节点的副本2均可用,则处理流程同步骤2。Step 6. In the case of step 5, when the update and access request initiated by the client is located on the copy 2 of the node A, since the copy 2 of the three nodes is available, the processing flow is the same as the step 2.
步骤7,在步骤5情况下,当客户端发起的更新请求的数据位于节点A副本3上时,B节点的副本3损坏,C节点的副本3可用。由A节点Master向磁盘III上副本3进行数据更新,节点C的slave从master同步数据,并向节点C磁盘II上副本3上数据进行数据更新,这时数据更新在W=2个节点上成功完成后,返回客户端数据更新成功消息;Step 7. In the case of step 5, when the data of the update request initiated by the client is located on the copy 3 of the node A, the copy 3 of the node B is damaged, and the copy 3 of the node C is available. The data is updated by the A-Node master to the replica 3 on the disk III, the slave of the node C synchronizes the data from the master, and updates the data on the replica 3 of the disk C on the node C. At this time, the data update succeeds on W=2 nodes. After completion, return the client data update success message;
当客户端发起数据访问请求的数据位于节点A副本3上时,节点B直接返回失败,从节点A和节点C的副本3上获得数据后,(满足R=2)返回客户端。
When the data of the client initiating the data access request is located on the replica 3 of the node A, the node B directly returns the failure, and after obtaining the data from the replica 3 of the node A and the node C, the data is returned to the client (satisfying R=2).
从上面可以看到,即使节点A和节点B都存在磁盘损坏的情况下,只要损坏磁盘的副本不重复,分布式缓存集群还是可以提供全部数据的读写服务。As can be seen from the above, even if both node A and node B have disk corruption, the distributed cache cluster can provide read and write services for all data as long as the damaged disk copy is not duplicated.
上述应用场景中,如果有两个故障节点,每个节点实际都是部分磁盘损坏,在较乐观的情况下,如果损坏的磁盘上存放的不是同一个数据的副本,则实际整个系统的可用磁盘上,还是保存着所有数据的至少两个副本,完全具备正常提供所有服务的条件。即使在损坏的磁盘上恰好存放着同一个数据的副本,那么其他磁盘上的可用数据,依然可以满足一致性和可用性,可以提供读写服务,仅对同时损坏的这部分数据而言,无法提供读写访问。In the above application scenario, if there are two faulty nodes, each node is actually partially damaged. In a more optimistic situation, if the damaged disk does not store a copy of the same data, the actual available disk of the entire system. At the top, it still holds at least two copies of all the data, and is fully qualified to provide all services normally. Even if a copy of the same data is stored on the damaged disk, the available data on other disks can still meet the consistency and availability, and can provide read and write services, which cannot be provided only for the data that is damaged at the same time. Read and write access.
综上所述,本发明的有益效果如下:In summary, the beneficial effects of the present invention are as follows:
本发明针对分布式缓存系统,在有磁盘损坏的情况下,可以充分利用可用的资源,整合出符合一致性和可用性要求的副本资源,尽可能提高系统的可用性,提高系统对故障的容忍性。也就是说,在云计算领域分布式缓存系统中,提供一种磁盘和数据管理机制,即使在节点部分磁盘发生故障情况下,依然能够尽可能利用可用磁盘上的数据,保持提供服务的能力,使得服务端在较少的磁盘或数据资源的情况下,提供一致性和可用性的存储服务。The present invention is directed to a distributed cache system. In the case of disk corruption, the available resources can be fully utilized, and the copy resources conforming to the requirements of consistency and availability can be integrated to improve the availability of the system and improve the tolerance of the system to faults. That is to say, in the distributed cache system of the cloud computing field, a disk and data management mechanism is provided, and even in the case of a failure of a node part disk, the data on the available disk can be utilized as much as possible, and the ability to provide the service is maintained. Enables the server to provide consistent and available storage services with fewer disks or data resources.
以上所述是本发明的优选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本发明所述原理的前提下,还可以作出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.
本发明实施例采用的上述技术方案,针对分布式缓存系统,在有磁盘损坏的情况下,可以充分利用可用的资源,整合出符合一致性和可用性要求的副本资源,尽可能提高系统的可用性,提高系统对故障的容忍性。
The above technical solution adopted by the embodiment of the present invention is directed to a distributed cache system, in the case of disk corruption, the available resources can be fully utilized, and the copy resources conforming to the consistency and availability requirements are integrated to improve the system availability as much as possible. Improve the system's tolerance to failure.
Claims (10)
- 一种云计算系统的处理方法,包括:A method for processing a cloud computing system, comprising:接收客户端对云计算系统的操作请求;Receiving an operation request of the client to the cloud computing system;根据所述操作请求,获取所述云计算系统中待操作的数据标识;Obtaining, according to the operation request, a data identifier to be operated in the cloud computing system;根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;And searching, according to the node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report includes: a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。And performing corresponding operations according to states of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- 根据权利要求1所述的方法,其中,所述根据各个所述磁盘的状态,进行相应的操作的步骤包括:The method of claim 1, wherein the step of performing a corresponding operation according to the state of each of the disks comprises:在所述操作请求为更新请求,并且所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;否则,拒绝所述更新请求;或者And when the operation request is an update request, and the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a minimum number of participating nodes of a data update request scheduled by the cloud computing system Responding to the update request; otherwise, rejecting the update request; or在所述操作请求为数据访问请求,并且所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;否则,拒绝所述数据访问请求。The operation request is a data access request, and the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to the number of data copies acquired by the cloud computing system for a predetermined data access request. Responding to the data access request; otherwise, rejecting the data access request.
- 根据权利要求2所述的方法,其中,所述当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求的步骤包括:The method of claim 2, wherein said number of said disks in said cloud computing system storing said data and in a normal state is greater than or equal to a minimum of a predetermined data update request of said cloud computing system When participating in the number of nodes, the steps of responding to the update request include:当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为正常时,所述云计算系统的主节点向主节点的所述数据所在磁盘进行数据更新;所述云计算系统的从节点从所述主节点获取待同步的数据,所述从节点向所述从节点的所述数据所在磁盘进行数据更新;When the operation request is an update request, and the state of the disk of the primary node storing the data is normal, the primary node of the cloud computing system performs data update to the disk of the data of the primary node; the cloud computing The slave node of the system acquires data to be synchronized from the master node, and the slave node performs data update on the disk where the data of the slave node is located;当所述操作请求为更新请求,并且存储所述数据的主节点的磁盘的状态为故障时,所述云计算系统的第一从节点向所述第一从节点的所述数据所在磁盘 进行数据更新;所述云计算系统的第二从节点从所述第一从节点获取待同步的数据;所述第二节点向所述第二从节点的所述数据所在磁盘进行数据更新;所述第一从节点和所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is an update request, and the state of the disk of the primary node storing the data is a fault, the data of the first slave node of the cloud computing system to the data of the first slave node is located Performing data update; the second slave node of the cloud computing system acquires data to be synchronized from the first slave node; and the second node performs data update to the disk where the data of the second slave node is located; The state of the disk storing the data of the first slave node and the second slave node is normal.
- 根据权利要求2所述的方法,其中,所述当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求的步骤包括:The method according to claim 2, wherein said number of said disks in said cloud computing system storing said data and in a normal state is greater than or equal to a predetermined data access request acquisition by said cloud computing system The number of copies of the data, the steps of responding to the data access request include:当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为正常时,从所述云计算系统的主节点的所述数据所在磁盘中获取所述数据的第一副本,从所述云计算系统的至少一个从节点的所述数据所在磁盘中获取所述数据的第二副本;从所述第一副本和所述第二副本中,选取最新版本的副本;并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常;When the operation request is a data access request, and the state of the disk of the primary node storing the data is normal, obtaining the first copy of the data from the disk where the data of the primary node of the cloud computing system is located Obtaining a second copy of the data from a disk of the at least one slave node of the cloud computing system; selecting, from the first copy and the second copy, a copy of the latest version; Sending a copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal;当所述操作请求为数据访问请求,并且存储所述数据的主节点的磁盘的状态为故障时,从所述云计算系统的至少一个从节点的所述数据所在磁盘中获取所述数据的第三副本;从至少一个所述第三副本中,选取最新版本的副本,并将所述最新版本的副本发送给所述客户端;所述第二从节点的存储所述数据的磁盘的状态为正常。When the operation request is a data access request, and the state of the disk of the primary node storing the data is a failure, obtaining the data from the disk where the data of the at least one slave node of the cloud computing system is located Three copies; from at least one of the third copies, selecting a copy of the latest version and transmitting the copy of the latest version to the client; the state of the disk storing the data of the second slave node is normal.
- 根据权利要求1所述的方法,其中,所述接收客户端的操作请求的步骤之前,所述方法还包括:The method of claim 1, wherein before the step of receiving an operation request from a client, the method further comprises:从节点获取所述云计算系统的节点磁盘状态报告。The node disk status report of the cloud computing system is obtained from the node.
- 一种云计算系统的处理装置,包括:A processing device for a cloud computing system, comprising:第一接收单元,接收客户端对云计算系统的操作请求;a first receiving unit, receiving an operation request of the client for the cloud computing system;获取单元,根据所述操作请求,获取所述云计算系统中待操作的数据标识;Obtaining, according to the operation request, acquiring a data identifier to be operated in the cloud computing system;查找单元,根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个节点中存储所述数据标识对应数据的各个磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中磁盘的状态、所述磁盘中存储的数据所对应的数据标识;a search unit, configured to search, according to a node disk status report of the cloud computing system, each disk in the node of the cloud computing system, where the data corresponding to the data identifier is stored, and a status of each of the disks; the node disk status report includes : a state of a disk in each node of the cloud computing system, and a data identifier corresponding to data stored in the disk;操作单元,根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。The operation unit performs a corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- 根据权利要求6所述的装置,其中,所述操作单元包括: The apparatus of claim 6, wherein the operating unit comprises:第一响应子单元,在所述操作请求为更新请求,并且当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,则响应所述更新请求;a first response subunit, wherein the operation request is an update request, and the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data update of the cloud computing system Responding to the update request when the minimum number of participating nodes is requested;第一拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据更新请求的最少参与节点数量时,拒绝所述更新请求;a first rejecting subunit, rejecting the update when the number of the disks in the cloud computing system storing the data and in a normal state is less than a minimum number of participating nodes of a predetermined data update request of the cloud computing system request;第二响应子单元,在所述操作请求为数据访问请求,并且当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量大于或等于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,则响应所述数据访问请求;a second response subunit, wherein the operation request is a data access request, and the number of the disks in the cloud computing system storing the data and in a normal state is greater than or equal to a predetermined data of the cloud computing system Responding to the data access request when accessing the number of data copies requested to be obtained;第二拒绝子单元,当所述云计算系统中存储所述数据且处于正常状态的所述磁盘的数量小于所述云计算系统预定的一次数据访问请求获取的数据副本数量时,拒绝所述数据访问请求。a second rejecting subunit, rejecting the data when the number of the disks in the cloud computing system storing the data and in a normal state is less than the number of data copies acquired by the cloud computing system for a predetermined data access request Access request.
- 根据权利要求6所述的装置,其中,还包括:The apparatus of claim 6 further comprising:第二接收单元,从节点接收所述云计算系统的节点磁盘状态报告。The second receiving unit receives the node disk status report of the cloud computing system from the node.
- 一种云计算系统,包括:客户端、处理装置、节点、所述节点对应的磁盘;A cloud computing system includes: a client, a processing device, a node, and a disk corresponding to the node;所述处理装置,接收所述客户端对云计算系统的操作请求;根据所述操作请求,获取所述云计算系统中待操作的数据标识;根据所述云计算系统的节点磁盘状态报告,查找所述云计算系统的各个所述节点中存储所述数据标识对应数据的磁盘以及各个所述磁盘的状态;所述节点磁盘状态报告包括:所述云计算系统的各个节点中所述磁盘的状态、所述磁盘中存储的数据所对应的数据标识;根据所述云计算系统中的各个节点中存储所述数据标识对应数据的各个所述磁盘的状态,进行相应的操作。Receiving, by the processing device, an operation request of the client to the cloud computing system; acquiring, according to the operation request, a data identifier to be operated in the cloud computing system; searching according to a node disk status report of the cloud computing system And storing, in each of the nodes of the cloud computing system, a disk of the data identifier corresponding data and a status of each of the disks; the node disk status report includes: a status of the disk in each node of the cloud computing system And the data identifier corresponding to the data stored in the disk; performing the corresponding operation according to the state of each of the disks in which the data identifier corresponding data is stored in each node in the cloud computing system.
- 根据权利要求9所述的系统,其中,所述节点,给所述处理装置发送节点磁盘状态报告。 The system of claim 9 wherein said node sends a node disk status report to said processing device.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410289531.7A CN105323271B (en) | 2014-06-24 | 2014-06-24 | Cloud computing system and processing method and device thereof |
CN201410289531.7 | 2014-06-24 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015196692A1 true WO2015196692A1 (en) | 2015-12-30 |
Family
ID=54936632
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2014/090398 WO2015196692A1 (en) | 2014-06-24 | 2014-11-05 | Cloud computing system and processing method and apparatus for cloud computing system |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN105323271B (en) |
WO (1) | WO2015196692A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108206768A (en) * | 2016-12-20 | 2018-06-26 | 阿里巴巴集团控股有限公司 | Cluster monitoring and switching method and device |
CN108173672B (en) * | 2017-12-04 | 2021-06-08 | 华为技术有限公司 | Method and device for detecting fault |
CN110321225B (en) * | 2019-07-08 | 2021-04-30 | 腾讯科技(深圳)有限公司 | Load balancing method, metadata server and computer readable storage medium |
CN113485648B (en) * | 2021-07-14 | 2022-04-26 | 华能吉林发电有限公司 | Storage resource control system based on cloud platform |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102202087A (en) * | 2011-04-25 | 2011-09-28 | 中兴通讯股份有限公司 | Method for identifying storage equipment and system thereof |
CN103257977A (en) * | 2012-02-21 | 2013-08-21 | 阿里巴巴集团控股有限公司 | Method and device for obtaining identification numbers |
CN103747072A (en) * | 2013-12-30 | 2014-04-23 | 乐视网信息技术(北京)股份有限公司 | Data reading and writing method and application server |
CN103763155A (en) * | 2014-01-24 | 2014-04-30 | 国家电网公司 | Multi-service heartbeat monitoring method for distributed type cloud storage system |
-
2014
- 2014-06-24 CN CN201410289531.7A patent/CN105323271B/en active Active
- 2014-11-05 WO PCT/CN2014/090398 patent/WO2015196692A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102202087A (en) * | 2011-04-25 | 2011-09-28 | 中兴通讯股份有限公司 | Method for identifying storage equipment and system thereof |
CN103257977A (en) * | 2012-02-21 | 2013-08-21 | 阿里巴巴集团控股有限公司 | Method and device for obtaining identification numbers |
CN103747072A (en) * | 2013-12-30 | 2014-04-23 | 乐视网信息技术(北京)股份有限公司 | Data reading and writing method and application server |
CN103763155A (en) * | 2014-01-24 | 2014-04-30 | 国家电网公司 | Multi-service heartbeat monitoring method for distributed type cloud storage system |
Non-Patent Citations (1)
Title |
---|
GAO, HONG ET AL.: "Distributed Cache of Cloud Computing Technology and Its Application in the Internet of Things", ZTE TECHNOLOGY JOURNAL, vol. 17, no. 4, August 2011 (2011-08-01), pages 37 - 42, XP055246677 * |
Also Published As
Publication number | Publication date |
---|---|
CN105323271A (en) | 2016-02-10 |
CN105323271B (en) | 2020-04-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
US20240168673A1 (en) | Methods and systems to interface between a multi-site distributed storage system and an external mediator to efficiently process events related to continuity | |
US11409622B1 (en) | Methods and systems for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system without using an external mediator | |
US12063270B2 (en) | Commissioning and decommissioning metadata nodes in a running distributed data storage system | |
US11550679B2 (en) | Methods and systems for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system | |
WO2016070375A1 (en) | Distributed storage replication system and method | |
US11360867B1 (en) | Re-aligning data replication configuration of primary and secondary data serving entities of a cross-site storage solution after a failover event | |
US11709743B2 (en) | Methods and systems for a non-disruptive automatic unplanned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system | |
EP3039549B1 (en) | Distributed file system using consensus nodes | |
US9367261B2 (en) | Computer system, data management method and data management program | |
US11537314B1 (en) | Resynchronization of individual volumes of a consistency group (CG) within a cross-site storage solution while maintaining synchronization of other volumes of the CG | |
US12050558B2 (en) | Facilitating immediate performance of volume resynchronization with the use of passive cache entries | |
GB2484086A (en) | Reliability and performance modes in a distributed storage system | |
US20170293540A1 (en) | Failover of application services | |
US20220317897A1 (en) | Performing various operations at the granularity of a consistency group within a cross-site storage solution | |
CN105493474A (en) | System and method for supporting partition level journaling for synchronizing data in a distributed data grid | |
WO2015196692A1 (en) | Cloud computing system and processing method and apparatus for cloud computing system | |
JP2013206072A (en) | Data matching system, data matching method, and data matching program | |
CN116389233B (en) | Container cloud management platform active-standby switching system, method and device and computer equipment | |
EP2980707B1 (en) | Method for creating a database clone of a distributed database, system for creating a database clone of a distributed database, program and computer program product | |
Edward et al. | Mongodb architecture | |
WO2023125412A9 (en) | Method and system for synchronous data replication | |
Li et al. | High-Available Cloud Platform Based on OpenStack | |
Hu et al. | Research on reliability of Hadoop distributed file system | |
JP5845298B2 (en) | Nodes and programs |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14896116 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14896116 Country of ref document: EP Kind code of ref document: A1 |