WO2021072881A1

WO2021072881A1 - Object storage-based request processing method, apparatus and device, and storage medium

Info

Publication number: WO2021072881A1
Application number: PCT/CN2019/118550
Authority: WO
Inventors: 周波
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-10-16
Filing date: 2019-11-14
Publication date: 2021-04-22
Also published as: CN110888838B; CN110888838A

Abstract

An object storage-based request processing method, apparatus and device, and a storage medium. The method comprises: parsing a received access request and reading a request parameter of a preset dimension from the parsing result (S10); calling, on the basis of the request parameter, a preset object storage gateway function to perform user authentication on the access request (S20); if the user authentication passes, searching for a local cache field carried by a request header of the access request, and reading a parameter value corresponding to the local cache field (S30); testing whether the parameter value is a preset value (S40); if the parameter value is the preset value, obtaining, from a local storage space, target data requested by the access request (S50); calculating a current cyclic redundancy check value of the target data by using a cyclic redundancy check algorithm, and reading a historical cyclic redundancy check value corresponding to the target data from the local storage space (S60); performing, according to the current cyclic redundancy check value and the historical cyclic redundancy check value, data integrity verification on the target data (S70); and if the verification passes, returning the target data to an initiating end of the access request (S80). User authentication is first performed on the access request. If the authentication passes, it is determined, according to the parameter value of the local cache field in the request header, whether target data to be accessed is stored locally; and if yes, said target data is returned to the initiating end of the access request, so that a system does not need to obtain data from a distal end and return same upon receipt of any access request, which simplifies the data transmission in a network and the query search process at a server, and reduces the network cost of data acquisition.

Description

Object storage-based request processing method, device, equipment and storage medium To

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 16, 2019, the application number is 201910985628.4, and the invention title is "object storage-based request processing methods, devices, equipment, and storage media", and its entire contents Incorporated in the application by reference.

Technical field

This application relates to the field of computer technology, and in particular to a request processing method, device, device and storage medium based on object storage.

Background technique

Data is the foundation of computing and even business, and reliable storage of data is a service provided by cloud storage. With the rapid development of cloud computing technology, which has promoted the networking of computing and storage, the current major Internet technology companies have launched cloud services. Current cloud storage providers have invested a lot of resources in reliability and achieved good results.

The mainstream cloud service providers all provide image archive storage, and the archive storage is charged according to the storage access frequency and capacity. It is found in the application that the access of image data has certain time characteristics. In most of the systems connected, the possibility of data uploaded to the back-end storage is more than 90% accessed within 2 days, and the access after more than 2 days is less than 5%, and the cycle of cloud vendors converting ordinary storage to archive storage Statistics are done on a monthly basis, and the data needs to be charged according to the size of the data when the data is retrieved. Therefore, how to simplify the transmission of file data on the network and the query and retrieval on the server to the greatest extent, and reduce the network when the enterprise is acquiring data Cost has become an urgent problem to be solved.

Summary of the invention

The main purpose of this application is to provide a request processing method, device, equipment and storage medium based on object storage, which aims to solve the inability of the prior art to simplify the transmission of file data in the network and the query and retrieval process on the server side, and reduce the data The technical problem of the network cost at the time of acquisition.

In order to achieve the above objective, this application provides a request processing method based on object storage, and the method includes the following steps:

Analyze the received access request, and read the request parameters of the preset dimensions from the analysis result;

Calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

When the user is authenticated, search for the local cache field carried in the request header of the access request, and read the parameter value corresponding to the local cache field;

Detecting whether the parameter value is a preset value;

If yes, obtain the target data requested by the access request from the local storage space;

Calculating the current cyclic redundancy check value of the target data by using a cyclic redundancy check algorithm, and reading the historical cyclic redundancy check value corresponding to the target data from the local storage space;

Performing a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

When the verification is passed, the target data is returned to the initiator of the access request.

In addition, in order to achieve the above objective, this application also proposes a request processing device based on object storage, the device including:

The request parsing module is used to parse the received access request, and read the request parameters of the preset dimensions from the parsing result;

The request authentication module is configured to call a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

The parameter acquisition module is configured to search for the local cache field carried in the request header of the access request when the user is authenticated, and read the parameter value corresponding to the local cache field;

Numerical value detection module for detecting whether the parameter value is a preset value;

A data acquisition module, configured to acquire the target data requested by the access request from the local storage space when the parameter value is the preset value;

The data acquisition module is further configured to calculate the current cyclic redundancy check value of the target data using a cyclic redundancy check algorithm, and read the historical cyclic redundancy corresponding to the target data from the local storage space Check value

The data acquisition module is further configured to perform a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

The data acquisition module is further configured to return the target data to the initiator of the access request when the verification is passed.

In addition, in order to achieve the above object, this application also proposes a request processing device based on object storage, the device including: a memory, a processor, and a computer readable that is stored on the memory and can run on the processor. Instructions, the computer-readable instructions are configured to implement the steps of the object storage-based request processing method as described above.

In addition, in order to achieve the above-mentioned object, this application also proposes a storage medium with computer-readable instructions stored on the storage medium, and when the computer-readable instructions are executed by a processor, the object storage-based request as described above is realized. Processing method steps.

Since this application first authenticates the request when it receives an access request, when the user is authenticated, it directly determines whether the target data accessed by the request is stored locally according to the parameter value of the local cache field in the request header when the user authentication is passed. The target data is transmitted back to the initiator of the access request, so that the system does not have to obtain and return data from the remote when receiving each access request, thus simplifying the data transmission in the network and the query and retrieval process on the server. Reduce the network cost of data acquisition.

Description of the drawings

FIG. 1 is a schematic structural diagram of a request processing device based on object storage in a hardware operating environment related to a solution of an embodiment of the present application;

2 is a schematic flowchart of a first embodiment of a request processing method based on object storage according to this application;

3 is a schematic flowchart of a second embodiment of a request processing method based on object storage according to this application;

4 is a schematic flowchart of a third embodiment of a request processing method based on object storage according to this application;

Fig. 5 is a structural block diagram of a first embodiment of a request processing apparatus based on object storage in this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

Referring to FIG. 1, FIG. 1 is a schematic structural diagram of a request processing device based on object storage in a hardware operating environment involved in a solution of an embodiment of the application.

As shown in FIG. 1, the request processing device based on object storage may include a processor 1001, such as a central processing unit (Central Processing Unit). Processing Unit, CPU), communication bus 1002, user interface 1003, network interface 1004, memory 1005. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, can also be a stable non-volatile memory (Non-Volatile Memory, NVM), such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the request processing device based on object storage, and may include more or less components than those shown in the figure, or a combination of certain components, or different components. Component arrangement.

As shown in FIG. 1, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and computer readable instructions.

In the request processing device based on object storage shown in FIG. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with users; the request processing device based on object storage of this application The processor 1001 and the memory 1005 may be set in a request processing device based on object storage. The request processing device based on object storage uses the processor 1001 to call computer-readable instructions stored in the memory 1005 and execute the The request processing method based on object storage.

The embodiment of the present application provides a request processing method based on object storage. Refer to FIG. 2, which is a schematic flowchart of the first embodiment of the request processing method based on object storage in this application.

In this embodiment, the request processing method based on object storage includes the following steps:

Step S10: Parse the received access request, and read the request parameters of the preset dimensions from the parsing result;

It should be noted that the execution subject of the method in this embodiment may be a distributed file system or a cluster (Ceph) that can provide object storage, block storage, and file storage. In recent years, Ceph has been widely used because of its ability to provide three types of storage: object storage, block storage, and file storage, as well as open source features. There are more and more cloud storage clusters built using Ceph, and the storage capacity of a single cluster is also increasing. getting bigger. For example, in a medical image storage system, Ceph object storage is usually used to store massive medical pictures. The object storage-based request processing method proposed in this embodiment is mainly used to optimize user access conditions involved in the medical image storage system, and improve user access efficiency and the security of the medical image storage system.

In this step, the request parameters of the preset dimensions may include: Uniform Resource Locator (Uniform Resource Locator) carried in the access request. Locator (URL) in the object name, operation action, bucket (Bucket) name, authentication type field, message header declaration (SignedHeaders) field and signature value (Signature) field and other parameters.

In addition, the distributed file system Ceph (hereinafter referred to as the Ceph system) in this embodiment may roughly include: a hypertext transfer protocol (HTTP) front-end module, a presentation layer state transfer application program interface (REST API) Five modules: general processing layer, application program interface operation execution layer, interface adaptation layer and interface layer. For each access request received, the above modules in the Ceph system can work together to respond to the access request.

In the specific implementation, the HTTP front-end module in the Ceph system first parses the access request when it receives the access request sent by the application client, and then reads the request parameters of the aforementioned preset dimensions from the analysis result, and then transfers these requests Parameters sent to REST API general processing layer.

Of course, in practical applications, the operation of reading the request parameters of the preset dimensions from the analysis result can also be performed by REST The API is executed by the general processing layer, which is not limited in this embodiment.

Step S20: Call a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

It should be noted that the user authentication in this embodiment, also called Auth authentication, is to verify whether the access user corresponding to the access request is legal, whether the operation action (read/write/modify of data, etc.) is allowed, and the request URL Whether the name of the access object exists, whether the access user has the access authority to the access object, etc. The preset object storage gateway function may be the rgw_process_authenticated function in the pre-written process_request method.

Further, the REST of this embodiment When the API general processing layer performs user authentication operations, it can be implemented based on the information contained in the authentication type field, the message header declaration (SignedHeaders) field, and the signature value (Signature) field. Normally, the authentication type field defines the user authentication method or type, such as Basic-Basic authentication method or AWS4 (AWS Signature Version4) Server authentication method, etc., and the authentication type field also specifies the target signature algorithm used for request signature value calculation. The message header declaration field specifies which message headers are used to calculate the signature value of the access request. The signature value field gives the exact signature value that should be obtained after calculating the requested signature value.

In the specific implementation, REST in the Ceph system The API general processing layer can call the rgw_process_authenticated function in the process_request method based on the read request parameters to perform user authentication on the access request.

Step S30: when the user is authenticated, search for the local cache field carried in the request header of the access request, and read the parameter value corresponding to the local cache field;

It should be noted that this embodiment extends the application program interface of the Ceph system, that is, in the request header (HTTP header, An important part of the Hypertext Transfer Protocol, used for parameter transfer) adds the local cache "Local-cached" field, so that it can be judged whether the local is locally based on the parameter value corresponding to the Local-cached field in the request header of each access request The data required for the access request exists, and follow-up operations are performed according to the judgment result.

In specific implementation, the application program interface operation execution layer in the Ceph system can query the local cache field carried in the request header of the access request when the user is authenticated, and read the parameter value corresponding to the local cache field.

Step S40: Detect whether the parameter value is a preset value;

It should be understood that the parameter value in this embodiment is True or False, and the preset value is True. If the parameter value corresponding to the local cache field is False, it indicates that the data or access object required by the access request does not exist in the database corresponding to the Ceph system. At this time, the Ceph system needs to obtain the data first, and then send it back to the client. ; If the parameter value corresponding to the local cache field is True, it indicates that the data or access object required by the access request exists in the database corresponding to the Ceph system. At this time, it is only necessary to verify the integrity of the data or access object required by the access request. Then return the access result to the client.

In specific implementation, the application program interface operation execution layer in the Ceph system can detect whether the parameter value is a preset value when it reads the parameter value corresponding to the local cache field, and then execute the corresponding request response operation according to the detection result.

Step S50: If yes, obtain the target data requested by the access request from the local storage space;

It should be understood that the target data may be the resource to be accessed by the access request, or the access result returned by the Ceph system to the client after the resource is accessed. Further, it is considered that errors may occur during data transmission or storage, and such errors will cause the original structure of the data to be destroyed, so that the data receiver or the data saver may receive or save the wrong data. Therefore, after reading the target data from the local storage space, the Ceph system of this embodiment will also perform a cyclic redundancy check on the read target data to ensure the integrity of the target data.

Step S60: Calculate the current cyclic redundancy check value of the target data using a cyclic redundancy check algorithm, and read the historical cyclic redundancy check value corresponding to the target data from the local storage space;

It should be understood that the cyclic redundancy check (Cyclic Redundancy Check) Check, CRC) is a hash function that generates a short fixed-digit check code based on data such as network data packets or computer files. It is mainly used to detect or verify possible errors after data transmission or storage. Correspondingly, in this embodiment, the historical cyclic redundancy check value is the cyclic redundancy check value calculated by the cyclic redundancy check algorithm before the target data is stored in the local storage space. In practical applications, the check value can be associated with the target data and then saved for subsequent reading and verification.

The cyclic redundancy check algorithm of this embodiment roughly includes the following steps:

(1) Set up a CRC register in the Ceph system and assign it to the "remainder initial value";

(2) XOR the first 8-bit character of the above target data with the CRC register, and store the result of the XOR value in the CRC register;

(3) Shift the CRC register to the right by one bit, the most significant bit of the CRC register (Most Significant Bit, MSB) zero padding, move out and check the Least Significant Bit (LSB);

(4) If the LSB is 0, repeat step (3); if the LSB is 1, it means that the CRC register is XORed with 0x31;

(5) Repeat steps (3) and (4) until all 8 shifts are completed, at which time an 8-bit data has been processed;

(6) Repeat steps (2) to (5) until all target data are processed;

(7) Finally, the CRC value is the CRC value after the NOR operation is performed on the data of the CRC register and the "exclusive OR value result".

Step S70: Perform a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

It should be understood that in the data integrity check process, if the current cyclic redundancy check value is consistent with the historical cyclic redundancy check value, it indicates that the data integrity check has passed; if the current cyclic redundancy check value is Inconsistent historical cyclic redundancy check values indicate that the data integrity check failed.

Step S80: When the verification is passed, the target data is returned to the initiator of the access request.

In specific implementation, when the application program interface operation execution layer in the Ceph system detects that the parameter value corresponding to the local cache field is True, it obtains the target data requested by the access request from the local storage space, and then uses the CRC algorithm to After the integrity check of the target data is performed and the verification is passed, the target data is returned to the initiator of the access request to realize a quick response to the access request.

Because this embodiment first authenticates the request when the access request is received, and when the user is authenticated, it directly determines whether the target data accessed by the request is stored locally according to the parameter value of the local cache field in the request header when the user authentication is passed. Directly return the target data to the initiator of the access request, so that the system does not have to obtain and return the data from the remote when receiving each access request, thus simplifying the data transmission in the network and the query and retrieval process on the server , Which reduces the network cost of data acquisition.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of a second embodiment of a request processing method based on object storage in this application.

Based on the above-mentioned first embodiment, in this embodiment, the step S20 includes:

Step S201: Read the authentication type field, the message header declaration field, and the signature value field included in the request parameter;

It should be understood that, as described in the first embodiment, the authentication type field defines the method or type of user authentication, such as Basic-Basic authentication method or AWS4 (AWS Signature Version4) Server authentication method, etc., and the authentication type field also specifies the target signature algorithm used for request signature value calculation. The message header declaration field specifies which message headers are used to calculate the signature value of the access request. The signature value field gives the exact signature value that should be obtained after the request signature value calculation is performed. If the calculated signature value is consistent with the exact signature value given in the signature value field, it indicates that the access request is authentic, and vice versa. It is not credible.

In the specific implementation, REST in the Ceph system The API general processing layer can perform user authentication on the access request based on the read request parameters.

Step S202: Determine a user authentication method corresponding to the access request according to the authentication type field, where the user authentication method includes a target signature algorithm;

In concrete implementation, REST After reading the authentication type field in the request parameter, the API general processing layer can determine the user authentication method corresponding to the access request and the target signature algorithm used for request signature value calculation according to the authentication type field.

For example, the authentication field read by the REST API general processing layer is Authorization: AWS4-HMAC-SHA256 indicates that the authentication method corresponding to the access request is the server identity verification method based on the AWS4 algorithm. The target signature algorithm used in this authentication is the AWS4 algorithm; Hash-based message authentication code (Hash-based Message Authentication Code, HMAC), it stipulates that HMAC operation needs to be performed on the data in the declaration field of the request message header; SHA256 means that the hash value length used by the signature algorithm is 256 bits.

Step S203: Calculate the target signature value corresponding to the access request according to the message header declaration field and the target signature algorithm through a preset object storage gateway function;

It is understandable that the message header declaration field specifies which message headers are used to calculate the signature value of the access request. At the same time, it also specifies the order of these message headers, so that the subsequent signature calculation will be Canonical. The sequence of message headers spliced by the Request function is consistent with the sequence specified in the message header declaration field. And for access requests, in order to prevent tampering with the request address, the SHA256 value of the requested content, the request timestamp and other information, the host; x-amz-content-sha256; x-amz-date and other parameters in the message header declaration field are required Carry.

In concrete implementation, REST After the API general processing layer obtains the message header declaration field, it first calls the preset object storage gateway function (rgw_process_authenticated function) to extract valid signature data from the request parameters according to the message header declaration field (that is, the message participating in the calculation of the signature value) Header), and then calculate the target signature value corresponding to the access request through the target signature algorithm based on these valid signature data. Specifically, REST The API general processing layer can call a preset object storage gateway function to extract valid signature data from the request parameters according to the message header declaration field; and then calculate the access request corresponding to the access request according to the target signature algorithm and the valid signature data. The target signature value.

For example, call the rgw_process_authenticated function to extract the valid signature data from the request parameters according to the message header declaration field as "host:10.47.193.31", "x-amz-content-sha256: e3b0c44298fc1c149" and "X-Amz-Date: 20190426T061340Z", the target signature algorithm is a hash (SHA256) algorithm, and the target signature value "Hash (10.47.193.31e3b0c44298fc1c14920190426T061340Z)=6ab57bc9beb4e6558dc4c9824aa156bdc9a357260150dbab624

Step S204: Perform user authentication on the access request based on the signature value field and the target signature value.

It should be understood that the signature value field gives the exact signature value that should be obtained after the request signature value calculation is performed. For example, the signature value field Signature=6ab57bc9beb4e6558dc4c9824aa156bdc9a357260150dbabd0a589c74910b624, where the signature value "6ab57bc9beb4e6558dc4c9824aac156bdc9a357589 is the exact value of the signature.

In concrete implementation, REST After the API general processing layer calculates the target signature value, it can compare the target signature value with the exact signature value contained in the signature value field. If the two are exactly the same, it indicates that the access request is credible and the user is authenticated. .

In this embodiment, the authentication type field, the message header declaration field, and the signature value field contained in the request parameters are read; and then the user authentication method corresponding to the access request is determined according to the authentication type field. The user authentication method includes the target signature algorithm; Suppose the object storage gateway function calculates the target signature value corresponding to the access request according to the message header declaration field and the target signature algorithm; and then authenticates the access request based on the signature value field and the target signature value, thus realizing effective authentication of the access request and avoiding The occurrence of malicious access has ensured the information security of the Ceph system.

Referring to FIG. 4, FIG. 4 is a schematic flowchart of a third embodiment of a request processing method based on object storage in this application.

Based on the foregoing embodiments, in this embodiment, after the step S40, the method further includes:

Step S401: If the parameter value is not the preset value, perform data preprocessing on the request parameter to obtain the target request parameter;

It should be noted that the data preprocessing in this step may be permission processing on request parameters. In this embodiment, the client (the initiator of the access request) needs to follow corresponding access rules, such as bucket access rules or object access rules, when accessing resources/data in the Ceph system. Among them, bucket access rules specify the users who have access rights to the objects in the bucket and the types of access rights that these users have. Object access rules specify the users who have object access rights and the types of access rights that these users have. For example, one user may only have read permissions, while another user may have read and write permissions.

Therefore, in this embodiment, if the application program interface operation execution layer in the Ceph system detects that the parameter value corresponding to the local cache field is False, it will perform data preprocessing on the request parameter carried in the access request to obtain the target request parameter. Specifically, the identification information corresponding to the initiator of the access request may be extracted from the request parameters; then the corresponding target access rule is searched in the preset access rule table according to the identification information, and the preset access The corresponding relationship between the identification information and the access rule is stored in the rule table; the request parameter is then assigned according to the target access rule to obtain the target request parameter.

Wherein, the identification information may be information that can distinguish the initiators of different access requests, such as Internet Protocol addresses, device serial numbers, and so on. The target access rule may be a pre-created bucket access rule or an object access rule, and these access rules may be associated with the identification information of the initiator and then stored in a preset access rule table.

In specific implementation, if the application program interface operation execution layer detects that the parameter value is not the preset value, it indicates that the data or access object required by the access request does not exist in the database corresponding to the Ceph system. To be able to provide access services to clients smoothly, the Ceph system needs to perform the above-mentioned data preprocessing on the request parameters to obtain the target request parameters, and then perform subsequent data pull operations based on the target request parameters.

Step S402: Perform authority authentication on the access request based on the target request parameter;

It should be understood that the authorization authentication is to verify whether the authorization type (such as read, write, change, check, etc.) possessed by the client to access the access object is the same or partially the same as the authorization requested by the access request. If it is, it is determined that the authority authentication is passed.

In specific implementation, after the application program interface operation execution layer assigns the request parameters according to the target access rules to obtain the target request parameters, it can authenticate the access request according to the target request parameters. Specifically, you can read the attribute value corresponding to the host field in the target request parameter (usually Internet Protocol address, IP address), and then query the corresponding permission type in the user permission list according to the attribute value, and then query the corresponding permission type. The permission type is matched with the permission type requested by the access request. If the match is successful, the permission authentication is passed, otherwise, the permission authentication is not passed.

Step S403: When the authority authentication is passed, encapsulate the access request to obtain a data acquisition request;

It should be understood that when the permission authentication is passed, it indicates that the access request can access the target data owned by the Ceph system, but since the data or access object required by the access request does not exist in the database corresponding to the Ceph system at this time, it is necessary to check The access request is repackaged and sent to the rados interface adaptation layer. The rados interface adaptation layer reads the data stored in the underlying rados and obtains the original data of the accessed object, such as target data such as read_version, write_version, status, and size.

Specifically, the application program interface operation execution layer may obtain the access permission corresponding to the access request when the permission authentication is passed; and then add the access permission as a parameter to be added to the request parameter to obtain a new Request parameters; the original request parameters of the access request are replaced with new request parameters to obtain the data acquisition request.

Step S404: Send the data acquisition request to the interface adaptation layer, so that the interface adaptation layer returns corresponding target data according to the data acquisition request;

In a specific implementation, the application program interface operation execution layer can send the repackaged data acquisition request to the rados interface adaptation layer, and the rados interface adaptation layer reads the target data stored in the underlying rados according to the data acquisition request.

Step S405: Return the target data to the initiator of the access request.

In a specific implementation, after the application program interface operation execution layer obtains the target data stored in the bottom layer of rados, it can transmit the target data back to the initiator of the access request to complete the response to the access request.

In this embodiment, when it is detected that the parameter value is not a preset value, data preprocessing is performed on the request parameter to obtain the target request parameter; the access request is authenticated based on the target request parameter; when the authorization is passed, the access request is performed Encapsulate to obtain the data acquisition request; send the data acquisition request to the interface adaptation layer so that the interface adaptation layer returns the corresponding target data according to the data acquisition request; the target data is returned to the initiator of the access request, so that it can be locally When the target data requested by the access request does not exist in the storage space, the target data can be obtained safely and conveniently, ensuring a smooth response to the access request.

In addition, the embodiment of the present application also proposes a storage medium, and the storage medium may be a non-volatile readable storage medium or a volatile readable storage medium.

The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the request processing method based on object storage as described above are realized.

For the method implemented when the computer-readable instruction is executed, refer to the various embodiments of the object storage-based request processing method of this application, which will not be repeated here.

Referring to FIG. 5, FIG. 5 is a structural block diagram of a first embodiment of a request processing apparatus based on object storage in this application.

As shown in Figure 5, the object storage-based request processing apparatus proposed in the embodiment of the present application includes:

The request parsing module 501 is used for parsing the received access request, and reading request parameters of preset dimensions from the parsing result;

The request authentication module 502 is configured to call a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

The parameter acquisition module 503 is configured to search for the local cache field carried in the request header of the access request when the user is authenticated, and read the parameter value corresponding to the local cache field;

The value detection module 504 is configured to detect whether the parameter value is a preset value;

The data acquisition module 505 is configured to acquire the target data requested by the access request from the local storage space when the parameter value is the preset value;

The data acquisition module 505 is further configured to use a cyclic redundancy check algorithm to calculate the current cyclic redundancy check value of the target data, and to read the historical cyclic redundancy check value corresponding to the target data from the local storage space. Residual check value;

The data acquisition module 505 is further configured to perform a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

The data acquisition module 505 is further configured to return the target data to the initiator of the access request when the verification is passed.

Based on the first embodiment of the request processing apparatus based on object storage in this application, a second embodiment of the request processing apparatus based on object storage in this application is proposed.

In this embodiment, the request authentication module 502 is also used to read the authentication type field, the message header declaration field, and the signature value field contained in the request parameter; determine the corresponding access request according to the authentication type field The user authentication method includes a target signature algorithm; the target signature value corresponding to the access request is calculated according to the message header declaration field and the target signature algorithm through a preset object storage gateway function; based on the signature The value field and the target signature value perform user authentication on the access request.

Further, the request authentication module 502 is also used to call a preset object storage gateway function to extract valid signature data from the request parameters according to the message header declaration field; according to the target signature algorithm and the valid signature data Calculate the target signature value corresponding to the access request.

Further, the data acquisition module 505 is further configured to perform data preprocessing on the request parameter when the parameter value is not the preset value to obtain the target request parameter; based on the target request parameter pair Perform permission authentication on the access request; when the permission authentication is passed, encapsulate the access request to obtain a data acquisition request; send the data acquisition request to the interface adaptation layer, so that the interface adaptation layer Return corresponding target data according to the data acquisition request; and return the target data to the initiator of the access request.

Further, the data acquisition module 505 is further configured to extract the identification information corresponding to the initiator of the access request from the request parameters;

Find the corresponding target access rule in the preset access rule table according to the identification information, and the corresponding relationship between the identification information and the access rule is stored in the preset access rule table; and the request is made according to the target access rule The parameters are assigned to obtain the target request parameters.

Further, the data acquisition module 505 is further configured to obtain the access permission corresponding to the access request when the permission authentication is passed; add the access permission as a parameter to be added to the request parameter to obtain A new request parameter; the access request is encapsulated according to the new request parameter to obtain a data acquisition request.

Further, the data acquisition module 505 is also configured to acquire the target data requested by the access request from a local storage space; use a cyclic redundancy check algorithm to calculate the current cyclic redundancy check value of the target data, And read the historical cyclic redundancy check value corresponding to the target data from the local storage space; perform the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value Data integrity check; when the check passes, the target data is returned to the initiator of the access request.

For other embodiments or specific implementations of the object storage-based request processing apparatus of the present application, reference may be made to the foregoing method embodiments, which will not be repeated here.

Claims

A request processing method based on object storage, characterized in that the method includes:

Analyze the received access request, and read the request parameters of the preset dimensions from the analysis result;

Calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

When the user is authenticated, search for the local cache field carried in the request header of the access request, and read the parameter value corresponding to the local cache field;

Detecting whether the parameter value is a preset value;

If yes, obtain the target data requested by the access request from the local storage space;

Calculating the current cyclic redundancy check value of the target data by using a cyclic redundancy check algorithm, and reading the historical cyclic redundancy check value corresponding to the target data from the local storage space;

Performing a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

When the verification is passed, the target data is returned to the initiator of the access request.
The method according to claim 1, wherein the step of invoking a preset object storage gateway function based on the request parameters to perform user authentication on the access request comprises:

Reading the authentication type field, the message header declaration field, and the signature value field contained in the request parameter;

Determining a user authentication method corresponding to the access request according to the authentication type field, where the user authentication method includes a target signature algorithm;

Calculating the target signature value corresponding to the access request according to the declaration field of the message header and the target signature algorithm through a preset object storage gateway function;

Perform user authentication on the access request based on the signature value field and the target signature value.
3. The method of claim 2, wherein the step of calculating the target signature value corresponding to the access request according to the message header declaration field and the target signature algorithm through a preset object storage gateway function comprises:

Calling a preset object storage gateway function to extract valid signature data from the request parameters according to the message header declaration field;

Calculate the target signature value corresponding to the access request according to the target signature algorithm and the valid signature data.
5. The method of claim 1, wherein after the step of detecting whether the parameter value is a preset value, the method further comprises:

If the parameter value is not the preset value, perform data preprocessing on the request parameter to obtain the target request parameter;

Performing authority authentication on the access request based on the target request parameter;

When the authority authentication is passed, encapsulate the access request to obtain a data acquisition request;

Sending the data acquisition request to the interface adaptation layer, so that the interface adaptation layer returns corresponding target data according to the data acquisition request;

The target data is sent back to the initiator of the access request.
The method according to claim 4, wherein the step of performing data preprocessing on the request parameters to obtain target request parameters comprises:

Extracting the identification information corresponding to the initiator of the access request from the request parameters;

Searching for a corresponding target access rule in a preset access rule table according to the identification information, and the preset access rule table stores a correspondence between the identification information and the access rule;

The request parameter is assigned according to the target access rule to obtain the target request parameter.
The method according to claim 4, wherein the step of encapsulating the access request to obtain the data acquisition request when the authority authentication is passed includes:

When the authority authentication is passed, obtain the access authority corresponding to the access request;

Adding the access authority as a parameter to be added to the request parameter to obtain a new request parameter;

The access request is encapsulated according to the new request parameter to obtain a data acquisition request.
A request processing device based on object storage, characterized in that the device comprises:

The request parsing module is used to parse the received access request, and read the request parameters of the preset dimensions from the parsing result;

The request authentication module is configured to call a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

The parameter acquisition module is configured to search for the local cache field carried in the request header of the access request when the user is authenticated, and read the parameter value corresponding to the local cache field;

Numerical value detection module for detecting whether the parameter value is a preset value;

A data acquisition module, configured to acquire the target data requested by the access request from the local storage space when the parameter value is the preset value;

The data acquisition module is further configured to calculate the current cyclic redundancy check value of the target data using a cyclic redundancy check algorithm, and read the historical cyclic redundancy corresponding to the target data from the local storage space Check value

The data acquisition module is further configured to perform a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

The data acquisition module is further configured to return the target data to the initiator of the access request when the verification is passed.
The device according to claim 7, wherein the request authentication module is further configured to read the authentication type field, the message header declaration field, and the signature value field contained in the request parameter; according to the authentication type field Determine the user authentication method corresponding to the access request, where the user authentication method includes a target signature algorithm; calculate the target signature corresponding to the access request according to the message header declaration field and the target signature algorithm through a preset object storage gateway function Value; user authentication is performed on the access request based on the signature value field and the target signature value.
The device according to claim 8, wherein the request authentication module is further configured to call a preset object storage gateway function to extract valid signature data from the request parameters according to the message header declaration field; The target signature algorithm and the valid signature data calculate the target signature value corresponding to the access request.
The device according to claim 7, wherein the data acquisition module is further configured to perform data preprocessing on the request parameter when the parameter value is not the preset value, so as to obtain the target request Parameters; perform permission authentication on the access request based on the target request parameters; when the permission authentication is passed, encapsulate the access request to obtain a data acquisition request; send the data acquisition request to the interface adaptation layer , So that the interface adaptation layer returns corresponding target data according to the data acquisition request; and transmits the target data back to the initiator of the access request.
The device according to claim 10, wherein the data acquisition module is further configured to extract the identification information corresponding to the initiator of the access request from the request parameter; and preset the identification information according to the identification information. Find the corresponding target access rule in the access rule table, the preset access rule table stores the correspondence between identification information and the access rule; assign values to the request parameters according to the target access rule to obtain the target request parameter.
The device according to claim 10, wherein the data acquisition module is further configured to acquire the access permission corresponding to the access request when the permission authentication is passed; add the access permission as a parameter to be added To the request parameter to obtain a new request parameter; encapsulate the access request according to the new request parameter to obtain a data acquisition request.
A request processing device based on object storage, characterized in that, the device includes: a memory, a processor, and computer-readable instructions stored on the memory and running on the processor, and the computer readable When the instruction is executed by the processor, the processor is caused to execute the following steps:

Analyze the received access request, and read the request parameters of the preset dimensions from the analysis result;

Calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

When the user is authenticated, search for the local cache field carried in the request header of the access request, and read the parameter value corresponding to the local cache field;

Detecting whether the parameter value is a preset value;

If yes, obtain the target data requested by the access request from the local storage space;

Calculating the current cyclic redundancy check value of the target data by using a cyclic redundancy check algorithm, and reading the historical cyclic redundancy check value corresponding to the target data from the local storage space;

Performing a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

When the verification is passed, the target data is returned to the initiator of the access request.
The device according to claim 13, wherein the step of calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request comprises:

Reading the authentication type field, the message header declaration field, and the signature value field contained in the request parameter;

Determining a user authentication method corresponding to the access request according to the authentication type field, where the user authentication method includes a target signature algorithm;

Calculating the target signature value corresponding to the access request according to the declaration field of the message header and the target signature algorithm through a preset object storage gateway function;

Perform user authentication on the access request based on the signature value field and the target signature value.
The device according to claim 14, wherein the step of calculating the target signature value corresponding to the access request according to the message header declaration field and the target signature algorithm through a preset object storage gateway function comprises:

Calling a preset object storage gateway function to extract valid signature data from the request parameters according to the message header declaration field;

Calculate the target signature value corresponding to the access request according to the target signature algorithm and the valid signature data.
The device according to claim 13, wherein after the step of detecting whether the parameter value is a preset value, the method further comprises:

If the parameter value is not the preset value, perform data preprocessing on the request parameter to obtain the target request parameter;

Performing authority authentication on the access request based on the target request parameter;

When the authority authentication is passed, encapsulate the access request to obtain a data acquisition request;

Sending the data acquisition request to the interface adaptation layer, so that the interface adaptation layer returns corresponding target data according to the data acquisition request;

The target data is sent back to the initiator of the access request.
The device according to claim 16, wherein the step of performing data preprocessing on the request parameters to obtain target request parameters comprises:

Extracting the identification information corresponding to the initiator of the access request from the request parameters;

Searching for a corresponding target access rule in a preset access rule table according to the identification information, and the preset access rule table stores a correspondence between the identification information and the access rule;

The request parameter is assigned according to the target access rule to obtain the target request parameter.
A storage medium, characterized in that computer-readable instructions are stored on the storage medium, and when the computer-readable instructions are executed by a processor, the processor is caused to perform the following steps:

Analyze the received access request, and read the request parameters of the preset dimensions from the analysis result;

Calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request;

When the user is authenticated, search for the local cache field carried in the request header of the access request, and read the parameter value corresponding to the local cache field;

Detecting whether the parameter value is a preset value;

If yes, obtain the target data requested by the access request from the local storage space;

Calculating the current cyclic redundancy check value of the target data by using a cyclic redundancy check algorithm, and reading the historical cyclic redundancy check value corresponding to the target data from the local storage space;

Performing a data integrity check on the target data according to the current cyclic redundancy check value and the historical cyclic redundancy check value;

When the verification is passed, the target data is returned to the initiator of the access request.
18. The storage medium of claim 18, wherein the step of calling a preset object storage gateway function based on the request parameters to perform user authentication on the access request comprises:

Reading the authentication type field, the message header declaration field, and the signature value field contained in the request parameter;

Determining a user authentication method corresponding to the access request according to the authentication type field, where the user authentication method includes a target signature algorithm;

Calculating the target signature value corresponding to the access request according to the declaration field of the message header and the target signature algorithm through a preset object storage gateway function;

Perform user authentication on the access request based on the signature value field and the target signature value.
18. The storage medium of claim 18, wherein after the step of detecting whether the parameter value is a preset value, the method further comprises:

If the parameter value is not the preset value, perform data preprocessing on the request parameter to obtain the target request parameter;

Performing authority authentication on the access request based on the target request parameter;

When the authority authentication is passed, encapsulate the access request to obtain a data acquisition request;

Sending the data acquisition request to the interface adaptation layer, so that the interface adaptation layer returns corresponding target data according to the data acquisition request;

The target data is sent back to the initiator of the access request.