CN118642883B

CN118642883B - Data processing method, device and equipment based on network card and readable storage medium

Info

Publication number: CN118642883B
Application number: CN202411097861.6A
Authority: CN
Inventors: 施路; 周肇勋; 刘杰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2024-08-12
Filing date: 2024-08-12
Publication date: 2024-11-05
Anticipated expiration: 2044-08-12
Also published as: CN118642883A

Abstract

The application discloses a data processing method, a device, equipment and a readable storage medium based on a network card, wherein the method comprises the following steps: dividing an initial block storage message into N unit storage messages with target storage lengths based on the unit storage lengths, generating redundancy check values of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and converting the message format of the unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and the N redundancy check values; the storage length of the N unit network messages is equal to the target storage length; and sending the N transmission network messages to the second equipment so that the second equipment writes the N transmission network messages into the node storage component. By adopting the application, the processing efficiency of the network card storage service can be improved, and the consumption of computing resources can be reduced.

Description

Data processing method, device and equipment based on network card and readable storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device and readable storage medium based on a network card.

Background

At present, in the storage service of the network card, a plurality of components are respectively guaranteed to be correct, the network card needs to unload a block storage (block) message of a front-end device in a block storage device into a logic circuit component of the network card, the logic circuit component generates a correctness check value of the block storage message after obtaining the block storage message, forwards the block storage message and the check value thereof to a storage development component (Storage Performance Development Kit, SPDK) in a System On Chip (SOC), and the correctness of the block storage message and the correctness check value thereof is checked by the storage development component, and if the correctness is correct, the block storage message and the correctness check value thereof are continuously transmitted to a kernel processing component (Kernel Transmission Control Protocol, KERNEL TCP).

In the kernel processing component, the block storage message and the correctness check value thereof are also checked, when the block storage message is correctly converted into a network (net) message through the TCP protocol, and because the maximum transmission unit (Maximum Transmission Unit, MTU) of the network message supported by the TCP protocol is far smaller than the block storage message, the block storage message is required to be divided into a plurality of message fragments, and an integrity check value (checksum in the TCP protocol) and the correctness check value are generated for each message fragment, and all the message fragments and the corresponding integrity check value and correctness check value are sent to the message sending component. The message sending component checks the integrity check value, determines the integrity of a plurality of message fragments, checks the correctness check value, determines the correctness of the plurality of message fragments, and sends the plurality of message fragments and the integrity check value and the correctness check value respectively corresponding to the plurality of message fragments to the back-end equipment when all message fragments pass the verification, wherein each component needs to consume resources to carry out data consistency check, and each component needs to realize exception handling after errors, so that the overall link processing time delay of the storage service of the network card is high, the processing efficiency is low, and the computing resource consumption is large.

Disclosure of Invention

The embodiment of the application provides a data processing method, device and equipment based on a network card and a readable storage medium, which can improve the processing efficiency of the network card storage service and reduce the consumption of computing resources.

The embodiment of the application provides a data processing method based on a network card, which is applied to the network card, wherein the network card comprises a logic circuit component and a system on chip component; the method comprises the following steps:

the method comprises the steps of obtaining an initial block storage message transmitted by a block storage component in first equipment through a logic circuit component, dividing the initial block storage message into N unit storage messages with target storage lengths based on unit storage lengths of a node storage component of second equipment, generating redundancy check values of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and transmitting the N access block storage messages to a system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

In the system-on-chip assembly, carrying out message format conversion on unit storage messages in N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

And sending the N transmission network messages to the second equipment so that the second equipment checks each transmission network message through the node storage component, and writing the N transmission network messages into the node storage component when all the N transmission network messages pass the check.

Wherein, still include:

acquiring unit storage length of a node storage component of the second equipment, determining storage length of message header information associated with a remote direct access protocol, and acquiring preset storage length aiming at a redundancy check value;

and determining the target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

Wherein generating a redundancy check value for each unit storage message includes:

Performing post-bit filling processing on the unit storage message based on the bit length P of the service operation value to obtain a bit filling storage message, and performing exclusive OR division operation on the bit filling storage message based on the service operation value to obtain a service redundancy value; p is a positive integer; the service operation value is determined based on a preset storage length for the redundancy check value;

if the bit growth of the service redundancy value meets the pre-bit filling condition, performing pre-bit filling processing on the service redundancy value, and determining the service redundancy value after the pre-bit filling processing as a redundancy check value;

And if the bit length of the service redundancy value does not meet the pre-bit supplementing condition, determining the service redundancy value as a redundancy check value.

The bit compensation storage message comprises M unit bit values, wherein M is a positive integer; based on the service operation value, performing exclusive OR division operation on the bit-filling storage message to obtain a service redundancy value, including:

based on the bit length P of the service operation value, determining continuous P unit bit values taking a target bit as a starting position in M unit bit values of the bit-supplementing storage message as unit operation data, and performing exclusive OR operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the complementary bit storage message;

Determining the unit bit values except for the P unit bit values in the M unit bit values as Q bit values to be complemented, performing post-complementation processing on the unit redundancy value based on the bit length P of the service operation value and the Q bit values to obtain new unit operation data with the same bit length as the service operation value, and continuously performing exclusive OR operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for finishing post-complementation processing, and determining the finally obtained unit redundancy value as the service redundancy value; q is the difference between M and P.

In the system-on-chip component, converting a message format of a unit storage message in the N access block storage messages by a remote direct access protocol to obtain N unit network messages, and generating N transport network messages according to the N unit network messages and the N redundancy check values, including:

Obtaining unit storage messages in N access block storage messages through a system-on-chip component, converting the message format of the N unit storage messages through a remote direct access protocol to obtain N unit network messages, and generating message header information corresponding to each unit network message respectively;

Respectively encapsulating N unit network messages, message header information and N redundancy check values corresponding to the N unit network messages into N transmission network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

The system-on-chip component comprises a protocol stack component; transmitting the N transport network messages to the second device, including:

N transmission network messages are sent to a protocol stack assembly;

in a protocol stack assembly, obtaining a unit network message in each transmission network message and a redundancy check value associated with the unit network message;

generating N target check values based on the N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, transmitting the N transmission network messages to the second equipment through the protocol stack assembly.

Wherein, still include:

Acquiring N transmission network messages sent by the second equipment through the node storage component through the system-on-chip component, checking each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check;

in the logic circuit assembly, carrying out message format conversion on unit network messages in N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages;

and transmitting the initial block storage message to the first device.

The embodiment of the application provides another data processing method based on a network card, which is applied to the network card, wherein the network card comprises a logic circuit component and a system on chip component; the method comprises the following steps:

Acquiring N transmission network messages sent by the second equipment through the node storage component through the system-on-chip component, checking each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage assembly in the second equipment; a transmission network message comprises a unit network message and a corresponding redundancy check value;

In the logic circuit assembly, carrying out message format conversion on unit network messages in N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

and transmitting the initial block storage message to the first device.

In one aspect, the embodiment of the application provides another data processing method based on a network card, which is applied to second equipment, wherein the second equipment performs data interaction with first equipment through the network card; the method comprises the following steps:

acquiring N transmission network messages sent by first equipment through a network card; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

And acquiring unit network messages in each transmission network message and redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to the unit network messages are the same as the associated redundancy check values, respectively storing each transmission network message into a node storage component.

Wherein, store each transmission network message to the node storage assembly separately, include:

And acquiring message header information in the N transmission network messages, determining storage addresses corresponding to the N transmission network messages respectively based on the division ordering values in the N message header information, and storing each transmission network message to a storage unit corresponding to the associated storage address in the node storage component based on the N storage addresses.

Wherein, still include:

When a storage service reading request sent by a first device is obtained, N stored transmission network messages are obtained from a node storage component, each transmission network message is verified, when all the N transmission network messages pass verification, the N transmission network messages are sent to a network card, so that the network card obtains N unit storage messages through the N transmission network messages, and an initial block storage message is generated based on the N unit storage messages; the initial block store message is for transmission to the first device.

In one aspect, an embodiment of the present application provides a network card, including:

the logic circuit component is used for acquiring an initial block storage message transmitted by the block storage component in the first equipment, dividing the initial block storage message into N unit storage messages with target storage lengths based on the unit storage length of the node storage component of the second equipment, generating a redundancy check value of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and transmitting the N access block storage messages to the system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

The system-on-chip component is used for converting the message format of unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

And the system-on-chip assembly is further used for sending the N transmission network messages to the second equipment, so that the second equipment checks each transmission network message through the node storage assembly, and when all the N transmission network messages pass the check, the N transmission network messages are written into the node storage assembly.

The logic circuit component is further configured to obtain a unit storage length of the node storage component of the second device, determine a storage length of header information associated with the remote direct access protocol, and obtain a preset storage length for the redundancy check value;

the logic circuit component is further used for determining a target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

The logic circuit component is specifically configured to perform post-bit filling processing on the unit storage message based on a bit length P of the service operation value to obtain a bit filling storage message, and perform exclusive or division operation on the bit filling storage message based on the service operation value to obtain a service redundancy value; p is a positive integer; the service operation value is determined based on a preset storage length for the redundancy check value;

The logic circuit component is specifically configured to perform pre-bit-filling processing on the service redundancy value if the bit growth of the service redundancy value meets the pre-bit-filling condition, and determine the service redundancy value after the pre-bit-filling processing as a redundancy check value;

And the logic circuit component is specifically used for determining the service redundancy value as a redundancy check value if the bit length of the service redundancy value does not meet the pre-bit-filling condition.

The bit compensation storage message comprises M unit bit values, wherein M is a positive integer;

The logic circuit component is specifically configured to determine, from M unit bit values of the bit-filling storage message, consecutive P unit bit values with a target bit as a starting position as unit operation data, and perform an exclusive-or operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the complementary bit storage message;

The logic circuit component is specifically configured to determine the unit bit values except for the P unit bit values in the M unit bit values as Q bit values to be complemented, perform post-complementation processing on the unit redundancy value based on the bit length P of the service operation value and the Q bit values to be complemented to obtain new unit operation data with the same bit length as the service operation value, and continuously performing exclusive or operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for finishing post-complementation processing, and determine the finally obtained unit redundancy value as the service redundancy value; q is the difference between M and P.

The system-on-chip component is specifically configured to obtain unit storage messages in N access block storage messages, perform message format conversion on the N unit storage messages through a remote direct access protocol, obtain N unit network messages, and generate message header information corresponding to each unit network message respectively;

The system-on-chip component is specifically configured to encapsulate N unit network messages, header information corresponding to the N unit network messages, and N redundancy check values, respectively, into N transmission network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

The system-on-chip component comprises a storage development component and a protocol stack component;

the storage development component is used for sending the N transmission network messages to the protocol stack component;

The protocol stack component is used for acquiring the unit network message in each transmission network message and the redundancy check value associated with the unit network message;

And the protocol stack assembly is also used for generating N target check values based on the N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, the N transmission network messages are sent to the second equipment through the protocol stack assembly.

Wherein,

The system-on-chip assembly is further used for acquiring N transmission network messages sent by the second equipment through the node storage assembly, checking each transmission network message, and sending the N transmission network messages to the logic circuit assembly when all the N transmission network messages pass the check;

the logic circuit component is further used for converting the message format of unit network messages in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages;

And the logic circuit component is also used for transmitting the initial block storage message to the first device.

In one aspect, another network card is provided in an embodiment of the present application, including:

The system-on-chip component is used for acquiring N transmission network messages sent by the second equipment through the node storage component, checking the redundancy check value in each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage assembly; a transmission network message comprises a unit network message and a corresponding redundancy check value;

The logic circuit component is used for converting the message format of unit network messages in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

In one aspect, an embodiment of the present application provides a data processing apparatus, including:

The message acquisition module is used for acquiring N transmission network messages sent by the first equipment through the network card; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

The message checking module is used for acquiring the unit network messages in each transmission network message and the redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to each unit network message are the same as the associated redundancy check values, respectively storing each transmission network message into the node storage component.

In one possible implementation manner, the message checking module is configured to, when each transport network message is stored in the node storage component, perform the following operations:

In one possible implementation, the packet verification module is further configured to perform the following operations:

In one aspect, an embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;

The processor is connected with the memory and the network interface, wherein the network interface is used for providing a data communication function, the memory is used for storing a computer program, and the computer program when executed by the processor causes the computer device to execute the method provided by the embodiment of the application.

In one aspect, the present application provides a computer readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided by the embodiment of the present application.

In one aspect, embodiments of the present application provide a computer program product comprising a computer program stored on a computer readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the method provided by the embodiment of the present application.

According to the embodiment of the application, through combining the characteristics of the unit storage length of the node storage assembly, the maximum transmission unit of the block storage message and the maximum transmission unit of the network message, the initial block storage message transmitted by the block storage assembly is divided in the logic circuit assembly through the target storage length smaller than or equal to the unit storage length, N unit storage messages with the target storage length are obtained, the redundancy check value of each unit storage message is generated, N access block storage messages are generated according to the N unit storage messages and the N redundancy check values, the N access block storage messages are transmitted to the system-on-chip assembly, the access block storage messages can be network messages obtained by adding message information such as check values, and the unit storage messages can be message fragments in a block storage message format. The target storage length may be less than or equal to the maximum transmission unit of the unit network packet in the remote direct access protocol, so that after the unit storage packet is converted into the packet format in the system-on-chip component by the remote direct access protocol, the storage length of the obtained unit network packet is the same as the storage length of the unit storage packet. When the unit storage message is transmitted between the logic circuit component and the system-on-chip component, message slicing is not required to be carried out because of the difference of the maximum transmission unit of the block storage message and the maximum transmission unit of the network message, and multiple times of correctness checking is not required to be carried out because of the difference of redundancy check values of the block storage message and the network message, and the integrity checking is not required to be carried out on the message slicing. Generating N transmission network messages through N unit network messages and N redundancy check values, wherein one transmission network message comprises one unit network message and an associated redundancy check value, the storage length of the N unit network messages is equal to the target storage length, the storage length of the N transmission network messages is smaller than or equal to the unit storage length, and the transmission network messages can be network messages with message formats conforming to a remote direct access protocol. And the second equipment checks each transmission network message through the node storage component, and writes the N transmission network messages into the node storage component when all the N transmission network messages pass the check. Because the storage length of the unit network message is the same as that of the unit storage message, the redundancy check value generated by the unit network message is the same as that generated by the unit storage message, and the redundancy check value generated by the logic circuit component can be carried to the tail end of the network card, so that the computing resource consumption of the system-on-chip component is reduced, the time delay of the whole link is reduced, and the processing efficiency of the network card storage service is improved.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a data processing scenario I according to an embodiment of the present application;

fig. 3 is a schematic flow chart of a data processing method based on a network card according to an embodiment of the present application;

fig. 4 is a second flow chart of a data processing method based on a network card according to an embodiment of the present application;

FIG. 5 is a second schematic diagram of a data processing scenario provided in an embodiment of the present application;

Fig. 6 is a flowchart of a data processing method based on a network card according to an embodiment of the present application;

Fig. 7 is a flow chart diagram of a data processing method based on a network card according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a network card according to an embodiment of the present application;

fig. 9 is a schematic diagram of a second structure of a network card according to an embodiment of the present application;

FIG. 10 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present application. As shown in fig. 1, the network architecture may include a first device 100, a network card 200, and a second device 300, where a communication connection exists between the first device 100 and the network card 200, and a communication connection exists between the network card 200 and the second device 300. The communication connection is not limited to a connection manner, and may be directly or indirectly connected through a wired communication manner, may be directly or indirectly connected through a wireless communication manner, or may be other manners, and the present application is not limited herein.

It should be understood that the first device 100 may interact with the second device 300 through the network card 200, the network card 200 may be integrated with the first device 100, or the network card 200 may be a separate device, which is not limited herein by the embodiment of the present application. When the network card 200 is a stand-alone device, the network card 200 may be connected to the first device 100 through a peripheral component interconnect express (PERIPHERAL COMPONENT INTERCONNECT EXPRESS, PCIE) bus. The second device 300 may be configured with a network access device corresponding to the network card 200, and the second device 300 may perform data interaction with the first device 100 through the network access device. Optionally, the network architecture may include one or more first devices, where each of the one or more first devices is connected to the network card 200, and performs data interaction with the second device 300 through the network card 200.

Wherein the first device 100 may include: smart phones, tablet computers, notebook computers, desktop computers, intelligent voice interaction devices, intelligent home appliances (e.g., smart televisions), wearable devices, vehicle terminals, aircraft and other intelligent terminals with data processing functions. The vehicle-mounted terminal can be terminal equipment in an intelligent traffic scene and an auxiliary driving scene. It should be understood that each terminal device in the terminal device cluster shown in fig. 1 may be provided with an application client having a data processing function, and when the application client runs in each terminal device, data interaction may be performed between the application client and the second device 300 shown in fig. 1.

The application client may specifically include: vehicle clients, smart home clients, entertainment clients (e.g., game clients), multimedia clients (e.g., video clients), social clients, and information-based clients (e.g., news clients), etc. The application client in the embodiment of the present application may be integrated in a certain client (for example, a social client), or may be an independent client (for example, a news client), and the embodiment of the present application does not limit the type of the application client.

The second device 300 may be a server corresponding to the application client, and the second device 300 may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content distribution networks), and basic cloud computing services such as big data and an artificial intelligent platform.

The cloud technology is a hosting technology for unifying serial resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business mode, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data need strong system rear shield support so as to be realized through cloud computing.

As shown in fig. 1, the first device 100 may include a block storage component, the network card 200 may include a logic circuit component and a system on chip component (SOC), and the second device 300 may include a node storage component (Cell). The first device 100 may be a front-end device, or may be referred to as a host (host), the Network card 200 may be a Smart Network card (Smart Network INTERFACE CARD, SNIC), and a high-performance Network access card with a Network processor as a core may be used to process various Network protocols or storage protocols in the host. The Block storage component may be a device component such as a hard disk drive (HDD, hard Disk Drive), a Solid state drive (SSD, solid STATE DRIVE), or the like, which has a fixed-size Block of data (Block) as a basic storage unit. The logic Circuit component may be implemented by an ASIC (Application-SPECIFIC INTEGRATED Circuit) or an FPGA (Field-Programmable GATE ARRAY, programmable logic array). The second device 300 may be a back-end device that provides storage business services.

It will be appreciated that the first device 100 may be provided with a plurality of Virtual Machines (VMs), each of which may have one or more Virtual Functions (VFs) running therein, each of the Virtual functions may correspond to one or more protocol queues (queues), the plurality of protocol queues including a storage service read queue and a storage service write queue, the storage service read queue may be used to perform a read task of a storage service, the storage service write queue may be used to perform a write task of a storage service, and the plurality of protocol queues may be stored into the block storage component.

For easy understanding, taking the writing task of the storage service as an example, the first device 100 may send the initial block storage packet Wen Xiezai (offload) in the block storage component to the network card 200, and the network card 200 encapsulates the initial block storage packet into a transport network packet based on a remote direct access protocol (Remote Direct Memory Access, RDMA) and sends the transport network packet to the second device 300, so as to complete writing of the storage service. The initial block storage message may be an IO (Input/Output ACCESS MESSAGE) access message for a block storage (block) component in the operating system, i.e., a BLK message. The transport network message may be a Network (NET) message in a message format conforming to the RDMA protocol, i.e., a NET-RDMA message. Offloading may include offloading for RDMA protocols, user datagram protocol (User Datagram Protocol, UDP) offloading, erasure Coding (EC) offloading, etc., embodiments of the application are not limited in this respect. Offloading may refer to separating various network protocols and storage protocols in the host for execution by the network card.

The embodiment of the application processes various network protocols and storage protocols separated by the first equipment through the network card, wherein the network card is provided with an independent computing unit, such as a logic circuit component, a system on chip component and the like, can finish the execution of a storage service reading queue, a storage service writing queue and the like, can greatly reduce the consumption of computing resources of the first equipment, can execute intensive storage service through a special processor, improves the speed of message transmission and improves the network performance and efficiency of the storage service.

Referring to fig. 2, fig. 2 is a schematic diagram of a scenario of data processing according to an embodiment of the present application. As shown in fig. 2, the first device may include a block storage component, the first device may be the first device 100 as shown in fig. 1, the network card may include a logic circuit component and a system-on-chip component, the network card may be the network card 200 as shown in fig. 1, wherein the system-on-chip component may include a storage development component (SPDK) and a Protocol Stack component (Protocol Stack), and the second device 300 may include a node storage component (Cell).

For easy understanding, taking a writing task of a storage service as an example, the first device may offload an initial block storage packet in the block storage component to the network card, where the network card obtains the initial block storage packet transmitted by the first device through the logic circuit component. The initial block storage message may be an IO access message for a block storage component in the operating system, i.e., a BLK message.

The logic circuit component of the network card may obtain a unit storage length of the node storage component of the second device, where the unit storage length may be a maximum storage length of a storage unit in the node storage component. The logic circuit component may divide the initial block storage message into N unit storage messages having a target storage length based on the unit storage length, where the unit storage messages may be message fragments of BLK format, and the target storage length may be smaller than the unit storage length. The logic circuit component may generate a redundancy check value (Cyclic Redundancy Check, CRC, also referred to as cyclic redundancy check) for each unit memory packet. The logic circuit component can generate N access block storage messages through N unit storage messages and N redundancy check values, and the N access block storage messages are transmitted to the system-on-chip component of the network card. The access block storage message may be a BLK message obtained by adding a message information such as a check value, and one access block storage message may include a unit storage message and a corresponding redundancy check value. The target storage length may be less than or equal to a maximum transmission unit of a unit network packet in remote direct access protocol (RDMA).

In the system-on-chip component, the storage development component can convert the message format of unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, wherein the remote direct access protocol is a data transmission protocol based on messages and is used for a high-performance network communication technology. The unit network message may be a message of a network device in the operating system, i.e. a NET message. The storage development component can generate N transmission network messages through N unit network messages and N redundancy check values, and the storage development component can transmit the N transmission network messages to a protocol stack component in the system-on-chip component. The transport network message may be a NET message with a message format conforming to RDMA protocol, i.e., a NET-RDMA message. A transport network packet may include a unit network packet and associated redundancy check values, where the storage lengths of the N unit network packets are equal to the target storage length and the storage lengths of the N transport network packets are less than or equal to the unit storage length.

The protocol stack component may use an RDMA API (Application Programming Interface ), for example, an RDMA connection with the second device may be created through a libibverbs library, where libibverbs library is a canonical library that provides a range of RDMA interfaces for performing RDMA operations, including but not limited to memory registration, sending and receiving data, performing RDMA reads and writes, and the like. The protocol stack assembly can register a storage area in a node storage assembly of the second device, N transmission network messages are sent to the second device, the second device checks redundancy check values in each transmission network message through the node storage assembly, and when all the N transmission network messages pass the check, the N transmission network messages are written into the node storage assembly.

The reading task of the storage service and the writing task of the storage service may be a reverse process, in the reading task of the storage service, the second device may obtain the stored N transport network messages from the node storage component, check the redundancy check value in each transport network message, when all the N transport network messages pass the check, send the N transport network messages to the network card, the network card may receive the N transport network messages through the protocol stack component, transfer the N transport network messages to the storage development component, perform message format conversion on the unit network messages in the N transport network messages by the storage development component to obtain N unit storage messages, transfer the N unit storage messages to the logic circuit component, and recombine the N unit storage messages into an initial block storage message by the logic circuit component, and send the initial block storage message to the block storage component in the first device for storage.

It can be appreciated that the network card proposed by the embodiment of the present application can be applied to Yun Yingpan (Cloud Block Store, CBS), allowing a user to create and manage a persistent block storage volume in a cloud environment, and providing a data block-level storage service with high durability and high reliability for a cloud server (Cloud Virtual Machine, CVM). The network card can be used for providing correctness storage check and RDMA network function unloading, so that the storage performance of the second equipment is improved, and safe and reliable storage service can be provided. The network card may also be used to implement firewall rules, security detection and prevention systems (Intrusion Detection and Prevention System, IDPS), and other security protocols to improve the security of the storage services. Support Software-defined networking (Software-Defined Networking, SDN) and network function virtualization (Network Functions Virtualization, NFV), improving data transfer rates.

According to the embodiment of the application, the characteristics of the unit storage length of the node storage assembly, the maximum transmission unit of the block storage message and the maximum transmission unit of the network message are combined, and the initial block storage message transmitted by the block storage assembly is divided in the logic circuit assembly through the target storage length smaller than or equal to the unit storage length, so that N unit storage messages with the target storage length are obtained, wherein the unit storage messages can be message fragments in a block storage message format. The target storage length may be less than or equal to the maximum transmission unit of the unit network packet in the remote direct access protocol, so that after the unit storage packet is converted into the packet format in the system-on-chip component by the remote direct access protocol, the storage length of the obtained unit network packet is the same as the storage length of the unit storage packet. Because the storage length of the unit network message is the same as that of the unit storage message, the redundancy check value generated by the unit network message is the same as that generated by the unit storage message, and the redundancy check value generated by the logic circuit component can be carried to the tail end of the network card, so that the computing resource consumption of the system-on-chip component is reduced, the time delay of the whole link is reduced, and the processing efficiency of the network card storage service is improved.

Referring to fig. 3, fig. 3 is a flowchart illustrating a data processing method based on a network card according to an embodiment of the application, where the data processing method may be performed by the network card, and the network card may be the network card 200 shown in fig. 1. The present data processing method will be described below by way of example as being executed by a network card. The data processing method at least comprises the following steps of S101-S103:

Step S101, acquiring an initial block storage message transmitted by a block storage component in first equipment through a logic circuit component, dividing the initial block storage message into N unit storage messages with target storage lengths based on unit storage lengths of a node storage component of second equipment, generating a redundancy check value of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and transmitting the N access block storage messages to a system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

Specifically, the network card may include a logic circuit component and a system on chip component, and the first device may establish a communication connection with the second device through the network card to perform data interaction. The first device may be a front-end device or a host. The first device may comprise a block storage device. The network card may be an intelligent network card, a high performance network access card with a network processor as a core, which may be used to process various network protocols or storage protocols from a host. The block storage component may be a device component such as a hard disk drive, solid state drive, etc. that has fixed-size blocks of data as the primary storage units. The logic circuit components may be implemented by an ASIC or FPGA. The second device may be a backend device that provides storage business services.

The first device may operate with a plurality of protocol queues, the plurality of protocol queues including a storage service read queue and a storage service write queue, the storage service read queue may be used to perform a read task of the storage service, the storage service write queue may be used to perform a write task of the storage service, and the plurality of protocol queues may be stored into the block storage component.

In the writing task of the storage service, the first device can unload the initial block storage message in the block storage component to the network card, and the network card acquires the initial block storage message transmitted by the first device through the logic circuit component. The initial block storage message may be an IO access message for a block storage component in the operating system, i.e., a BLK message. The logic circuit component of the network card may obtain a unit storage length of the node storage component of the second device, where the unit storage length may be a maximum storage length of a storage unit in the node storage component. Offloading may include offloading for RDMA protocols, offloading for UDP protocols, offloading for erasure codes, etc., and embodiments of the application are not limited in this regard. Offloading may refer to separating various network protocols and storage protocols in the host for execution by the network card.

The logic circuit component may divide the initial block memory message into N unit memory messages having a target memory length based on a unit memory length, wherein the target memory length may be less than the unit memory length and the target memory length may be less than or equal to a maximum transmission unit of a unit network message in a remote direct access protocol (RDMA). The unit storage message may be a message fragment in BLK format. The logic circuit component may generate a redundancy check (CRC) value for each unit memory packet. The logic circuit component can generate N access block storage messages through N unit storage messages and N redundancy check values, and the N access block storage messages are transmitted to the system-on-chip component of the network card. The access block storage message may be a BLK message obtained by adding a message information such as a check value, and one access block storage message may include a unit storage message and a corresponding redundancy check value. The redundancy check value may also be called cyclic redundancy check, and is a check code commonly used in the communication field, and includes a series of data coding rules such as shift and division. Common coding methods are CRC-32, CRC-16, etc.

Step S102, in a system-on-chip component, converting a message format of a unit storage message in N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

Specifically, the system-on-chip component may perform message format conversion on unit storage messages in the N access block storage messages through a remote direct access protocol (RDMA) to obtain N unit network messages, where the remote direct access protocol is a data transmission protocol based on a message, and is used in a high-performance network communication technology. The unit network message may be a message of a network device in the operating system, i.e. a NET message. The system-on-chip component can generate N transport network messages through N unit network messages and N redundancy check values, and the system-on-chip component can transmit the N transport network messages to a protocol stack component in the system-on-chip component. The transport network message may be a NET message with a message format conforming to RDMA protocol, i.e., a NET-RDMA message. A transport network packet may include a unit network packet and associated redundancy check values, where the storage lengths of the N unit network packets are equal to the target storage length and the storage lengths of the N transport network packets are less than or equal to the unit storage length.

Step S103, N transmission network messages are sent to the second equipment, so that the second equipment checks each transmission network message through the node storage component, and when all the N transmission network messages pass the check, the N transmission network messages are written into the node storage component.

Specifically, the system-on-chip component may send N transport network packets to the second device, where the second device checks, by using the node storage component, the redundancy check value in each transport network packet, and writes the N transport network packets into the node storage component when all the N transport network packets pass the check.

Referring to fig. 4, fig. 4 is a second flowchart of a data processing method based on a network card according to an embodiment of the present application, where the data processing method may be performed by the network card, and the network card may be the network card 200 shown in fig. 1. The present data processing method will be described below by way of example as being executed by a network card. The data processing method at least includes the following steps S201 to S207:

step S201, obtaining an initial block storage message transmitted by a block storage component in first equipment through a logic circuit component;

Specifically, reference may be made to the specific description of step S101 in the corresponding embodiment of fig. 3, and the embodiment of the present application is not repeated here.

Step S202, obtaining a unit storage length of a node storage component of the second device, determining a storage length of message header information associated with a remote direct access protocol, and obtaining a preset storage length for a redundancy check value; and determining the target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

Specifically, the network card may acquire a unit storage length of the node storage component of the second device, where the unit storage length may be a maximum storage length of a storage unit in the node storage component. The network card may determine a storage length of header information associated with a remote direct access protocol (RDMA) and obtain a preset storage length for the redundancy check value. The storage length of the redundancy check value is determined based on the service operation value generating the redundancy check value, and the storage length of the redundancy check value may be a bit length P-1 of the service operation value.

It can be understood that the unit storage length, the storage length of the header information, and the preset storage length are all preset numerical information, and the network card can determine the difference between the unit storage length and the storage length of the header information and the preset storage length as a storable length, and determine the target storage length based on the storable length. Wherein the target storage length may be less than or equal to the target storage length.

Step S203, dividing the initial block storage message into N unit storage messages with target storage length based on the unit storage length of the node storage component of the second device;

Specifically, referring to fig. 5 again, fig. 5 is a schematic diagram of a second data processing scenario according to the embodiment of the present application, as shown in fig. 5, the logic circuit component may divide the initial block storage packet into N unit storage packets with the target storage length based on the unit storage length. The target storage length may be less than or equal to a maximum transmission unit of a unit network packet in remote direct access protocol (RDMA). The unit storage messages can be message fragments in BLK format, and the N unit storage messages can comprise unit storage message 1, unit storage messages 2, … … and unit storage message N. Wherein the target storage length may be smaller than the unit storage length. For example, the initial block memory message may have a memory length of 12KB (Kilo Byte) and the target memory length may be 4KB, and then the logic circuit component may divide the initial block memory message into 3 unit memory messages having a memory length of 4 KB.

When the storage length of the initial block storage message is not aligned with the target storage length, for example, the storage length of the initial block storage message is 15KB, the target storage length is 4KB, the logic circuit component may divide the initial block storage message into a target storage message and a message to be divided aligned with the target storage length, where the storage length of the message to be divided may be 12KB, and the target storage message may be 3KB, and then the logic circuit component may divide the message to be divided into 3 unit storage messages with the storage length of 4KB, determine the target storage message as a message not aligned with the target storage length, or patch the target storage length to the target storage length, and in a subsequent process, perform the same processing as the unit storage message on the target storage message.

Step S204, based on the bit length P of the service operation value, performing post-bit compensation processing on the unit storage message to obtain a bit compensation storage message, and based on the service operation value, performing exclusive OR division operation on the bit compensation storage message to obtain a service redundancy value; p is a positive integer; the service operation value is determined based on a preset storage length for the redundancy check value; if the bit growth of the service redundancy value meets the pre-bit filling condition, performing pre-bit filling processing on the service redundancy value, and determining the service redundancy value after the pre-bit filling processing as a redundancy check value; and if the bit length of the service redundancy value does not meet the pre-bit supplementing condition, determining the service redundancy value as a redundancy check value.

Specifically, referring to fig. 5 again, as shown in fig. 5, the logic circuit component may generate a redundancy check value (CRC) of each unit storage packet, for example, may generate a redundancy check value crc_1 corresponding to the unit storage packet 1, redundancy check values crc_2 and … … corresponding to the unit storage packet 2, and redundancy check value crc_n corresponding to the unit storage packet N, where the redundancy check values may be generated by including CRC-8, CRC-16, CRC-32, and other ways in which the CRC-8, CRC-16, and CRC-32 have different check value lengths and complexities, for convenience of understanding, taking CRC-8 as an example for illustration, the service operation value of CRC-8 may be a fixed value determined based on a preset storage length for the redundancy check value, the service operation value may be a binary number 11001, and the bit length P of the service operation value is 5.

Taking a binary number 11010011 as an example of a unit storage message in the N unit storage messages, the logic circuit component can supplement the bit P-1 bit value 0 after the unit storage message based on the bit length P of the service operation value to obtain a supplement storage message, the supplement storage message is a binary number 110100110000, and the logic circuit component can perform exclusive or division operation on the supplement storage message based on the service operation value to obtain the service redundancy value.

The exclusive or division operation of the bit-filling storage message can be as follows: based on the bit length P of the service operation value, determining continuous P unit bit values taking a target bit as a starting position in M unit bit values of the bit-supplementing storage message as unit operation data, and performing exclusive OR operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the complementary bit storage message; determining the unit bit values except for the P unit bit values in the M unit bit values as Q bit values to be complemented, performing post-complementation processing on the unit redundancy value based on the bit length P of the service operation value and the Q bit values to obtain new unit operation data with the same bit length as the service operation value, and continuously performing exclusive OR operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for finishing post-complementation processing, and determining the finally obtained unit redundancy value as the service redundancy value; q is the difference between M and P.

Specifically, the logic circuit component may determine, based on the bit length P of the service operation value, continuous P unit bit values with a target bit as a starting position, among M unit bit values of the complementary bit storage packet, as unit operation data, where the target bit is a highest bit of bits with a unit bit value of 1 in the complementary bit storage packet, the bit length of the unit operation data is the same as the bit length of the service operation value, and the unit operation data may be a binary number 10110.

The logic circuit component can exclusive-or (XOR) the unit operation data 10110 with the service operation value 11001 to obtain a unit redundancy value, the unit redundancy value is a binary number 1111 (i.e., an operation result 01111 of the exclusive-or operation), the logic circuit component can determine the unit bit values except for the P unit bit values in the M unit bit values as Q bit values to be complemented, Q is a difference value between the M and the P, the Q bit values to be complemented can be binary numbers 010000, the logic circuit component can perform post-complementation processing on the unit redundancy value based on the bit length P of the service operation value and the Q bit values to obtain new unit operation data with the same bit length as the service operation value, the new unit operation data can be a binary number 11110, the logic circuit component can continuously exclusive-or operate the unit operation data 11110 based on the service operation value 11001, obtaining a unit redundancy value 111, performing post-interpolation processing on the unit redundancy value 111 based on the bit value which is not used for interpolation processing in the Q bit values to be compensated, obtaining unit operation data 11111 with the same bit length as the service operation value, performing exclusive OR operation on the unit operation data 11111 and the service operation value 11001, obtaining a unit redundancy value 110, performing post-interpolation processing on the unit redundancy value 110 based on the bit value which is not used for interpolation processing in the Q bit values to be compensated, obtaining unit operation data 11000 with the same bit length as the service operation value, performing exclusive OR operation on the unit operation data 11000 and the service operation value 11001, obtaining a unit redundancy value 1, performing post-interpolation processing on the unit redundancy value 1 based on the bit value which is not used for interpolation processing in the Q bit values to be compensated, obtaining a unit redundancy value 100, the Q bit values to be complemented are all used to complete post-bit-complementing processing, and the logic circuit component may determine the final unit redundancy value 100 as the service redundancy value 100.

If the bit length of the service redundancy value meets the pre-bit-filling condition, the logic circuit component can perform pre-bit-filling processing on the service redundancy value, and determine the service redundancy value 0100 after the pre-bit-filling processing as the redundancy check value 0100, wherein the pre-bit-filling condition means that the bit length of the service redundancy value is smaller than P-1. If the bit length of the service redundancy value does not meet the pre-fill bit condition, the logic circuit component may directly determine the service redundancy value as a redundancy check value. The service redundancy value is judged based on the pre-bit compensation condition, so that the bit length of the unified redundancy check value is P-1.

Step S205, generating N access block storage messages according to N unit storage messages and N redundancy check values, and transmitting the N access block storage messages to a system-on-chip component;

Specifically, referring to fig. 5 again, as shown in fig. 5, the logic circuit component may add header information of the BLK packet to each unit storage packet, generate an access block storage packet together with the associated redundancy check value, and transmit N access block storage packets to the system-on-chip component of the network card. The access block storage message may be a BLK message obtained by adding message information such as a check value, and the access block storage message may include header information of the BLK message, a unit storage message, and a corresponding redundancy check value. Specific content of header information of the BLK message the embodiment of the present application is not limited herein.

Step S206, obtaining unit storage messages in N access block storage messages through a system-on-chip component, converting the message format of the N unit storage messages through a remote direct access protocol to obtain N unit network messages, and generating message header information corresponding to each unit network message respectively; respectively encapsulating N unit network messages, message header information and N redundancy check values corresponding to the N unit network messages into N transmission network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

Specifically, the system-on-chip component may include a storage development component and a protocol stack component, where the storage development component may acquire unit storage messages in N access block storage messages, perform message format conversion on the unit storage messages in the N access block storage messages by using a remote direct access protocol (RDMA), obtain N unit network messages, and generate Header information of an RDMA protocol corresponding to each unit network message, where the Header information of the RDMA protocol may include Route Header (RH), basic transmission information (Base Transport Header, BTH) and RDMA protocol information, the Route information may include a source address and a destination address of the message in each system, the basic transmission information includes a type and a sequence number of the message, and the RDMA protocol information may include detailed information of RDMA operation, such as an RDMA read or write operation, a back end storage address, a front end storage address, and the like. Remote direct access protocol is a message-based data transmission protocol that is a technique for high performance network communications. The unit network message may be a message of a network device in the operating system, i.e. a NET message.

The storage development component can package the N unit network messages, the message header information corresponding to the N unit network messages and the N redundancy check values into N transmission network messages respectively. The transport network message may be a NET message with a message format conforming to RDMA protocol, i.e., a NET-RDMA message. A transport network message may include header information of an RDMA protocol, a unit network message, and an associated redundancy check value, where the storage lengths of the N unit network messages are equal to the target storage length, and the storage lengths of the N transport network messages are less than or equal to the unit storage length.

Step S207, N transmission network messages are sent to a protocol stack assembly; in a protocol stack assembly, obtaining a unit network message in each transmission network message and a redundancy check value associated with the unit network message; generating N target check values based on the N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, transmitting the N transmission network messages to the second equipment through the protocol stack assembly.

Specifically, the storage development component may transmit N transport network messages to a protocol stack component in the system-on-chip component, where the protocol stack component may verify the correctness of a unit network message in the transport network message obtained by converting the message format.

The protocol stack component can acquire the unit network message in each transmission network message and the redundancy check value associated with the unit network message, and perform exclusive OR division operation on the service operation value and the N unit network messages to obtain N target check values. If the target check value corresponding to each unit network message is the same as the associated redundancy check value, performing Kernel Bypass processing (Bypass Kernel) on each transport network message by an integrated user mode communication (RDMA) module in the protocol stack module, wherein the Kernel Bypass processing allows the message to Bypass the Kernel processing module (Kernel), and directly and remotely accessing the node memory of the second device through the RDMA protocol.

It can be understood that through the kernel bypass processing, the kernel processing component is not needed to be carried out on the transmission network message, so that the cost of copying and checking the message among different system components can be reduced, and the processing time delay is reduced.

It may be understood that the reading task of the storage service and the writing task of the storage service may be a reverse process, and the reading task of the storage service may be: the method comprises the steps that through a system-on-chip component, N transmission network messages sent by second equipment through a node storage component are obtained, redundancy check values in each transmission network message are checked, and when all the N transmission network messages pass the check, the N transmission network messages are sent to a logic circuit component; in the logic circuit assembly, carrying out message format conversion on unit network messages in N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages; and transmitting the initial block storage message to the first device.

Specifically, the second device may obtain the stored N transport network messages from the node storage component, check the redundancy check value in each transport network message, when all the N transport network messages pass the check, send the N transport network messages to the network card, the network card may receive the N transport network messages through the protocol stack component, transmit the N transport network messages to the storage development component, and the storage development component converts the message format of the unit network messages in the N transport network messages to obtain N unit storage messages, transmit the N unit storage messages to the logic circuit component, and the logic circuit component recombines the N unit storage messages into an initial block storage message, and sends the initial block storage message to the block storage component in the first device for storage.

According to the embodiment of the application, by combining the characteristics of the unit storage length of the node storage assembly, the maximum transmission unit of the BLK message and the maximum transmission unit of the NET message, the initial block storage message transmitted by the block storage assembly is divided at the source of the network card by the target storage length smaller than or equal to the unit storage length, so that N unit storage messages with the target storage length are obtained, the storage length of the unit storage message can be smaller than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, the storage length of the unit storage message is the same as the storage length of the unit storage message after the message format conversion of the unit storage message in the on-chip system assembly by the remote direct access protocol, and therefore, the correctness of the unit storage message can be checked by the redundancy check value of the unit storage message, and when the unit storage message is transmitted between the logic circuit assembly and the on-chip system assembly, the message is not required to be sliced due to the difference of the maximum transmission unit of the BLK message and the maximum transmission unit of the NET message, and the redundancy check value of the NET message is not required to be checked for multiple times. Generating N transmission network messages through N unit network messages and N redundancy check values, wherein one transmission network message comprises one unit network message and an associated redundancy check value, the storage length of the N unit network messages is equal to the target storage length, the storage length of the N transmission network messages is smaller than or equal to the unit storage length, and the transmission network messages can be NET messages with message formats conforming to RDMA protocols, namely NET-RDMA messages. And the second equipment checks each transmission network message through the node storage component, and writes the N transmission network messages into the node storage component when all the N transmission network messages pass the check. Because the storage length of the unit network message is the same as that of the unit storage message, the redundancy check value generated by the unit network message is the same as that generated by the unit storage message, and the redundancy check value generated by the logic circuit component can be carried to the tail end of the network card, so that the computing resource consumption of the system-on-chip component is reduced, the time delay of the whole link is reduced, and the processing efficiency of the network card storage service is improved.

On the other hand, the verification of multiple redundancy check values is not needed among different components of the network card, so that each component is not needed to realize exception handling after errors, the processing cost of the exception handling is reduced, and meanwhile, the verification algorithm between the first equipment and the second equipment can be consistent due to the fact that the redundancy check values are unified, and code multiplexing, realization and verification are facilitated. The core bypass processing can be carried out on the message by the protocol stack assembly capable of realizing the RDMA protocol, so that the problem of data errors caused by the factors such as failure of the core processing assembly, link errors and the like can be avoided, and the core processing assembly is not needed for transmitting the network message by the core bypass processing, so that the cost of copying and checking the message among different system assemblies can be reduced, and the processing time delay is reduced.

Referring to fig. 6, fig. 6 is a flowchart illustrating a data processing method based on a network card according to an embodiment of the present application, where the data processing method may be performed by the network card, and the network card may be the network card 200 shown in fig. 1. The present data processing method will be described below by way of example as being executed by a network card. The data processing method at least includes the following steps S301 to S303:

Step S301, obtaining N transmission network messages sent by a second device through a node storage component through a system-on-chip component, checking each transmission network message, and sending the N transmission network messages to a logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage assembly in the second equipment; a transmission network message comprises a unit network message and a corresponding redundancy check value;

Specifically, the network card may include a logic circuit component and a system on chip component, and the first device may establish a communication connection with the second device through the network card to perform data interaction. The first device may be a front-end device, or may be referred to as a host, the network card may be an intelligent network card, and a high-performance network access card with a network processor as a core may be used to process various network protocols or storage protocols in the host. The logic circuit components may be implemented by an ASIC or FPGA. The second device may be a backend device that provides storage business services.

In a reading task of the storage service, the network card may acquire N transport network packets sent by the second device through the node storage component through the system on chip component, where storage lengths of the N transport network packets are smaller than or equal to unit storage lengths of the node storage component in the second device, the unit storage lengths may be a maximum storage length of a storage unit in the node storage component, and the transport network packets may be NET packets with a packet format conforming to an RDMA protocol, that is, NET-RDMA packets. A transport network message may include a unit network message and associated redundancy check value. The unit network message may be a message of a network device in the operating system, i.e. a NET message. The specific generation process of the N transport network packets may be referred to the description of step S201 to step S207 in the corresponding embodiment of fig. 4, and the embodiments of the present application are not repeated here.

The system-on-chip component can acquire the unit network message in each transmission network message and the redundancy check value associated with the unit network message, and perform exclusive OR division operation on the service operation value and the N unit network messages to obtain N target check values. And if the target check value corresponding to each unit network message is the same as the associated redundancy check value, transmitting the N transmission network messages to the logic circuit component.

Step S302, in the logic circuit assembly, converting the message format of unit network messages in N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

Specifically, the logic circuit component may perform message format conversion on unit network messages in the N transport network messages through a remote direct access protocol, to obtain N unit storage messages. The remote direct access protocol is a message-based data transmission protocol, and is used for high-performance network communication. The unit storage message may be a message fragment in BLK format, and the logic circuit component may reorganize the N unit storage messages into an initial block storage message based on header information associated with the N unit storage messages, respectively. The initial block storage message may be an IO access message for a block storage component in an operating system, i.e., a BLK message.

Step S303, transmitting the initial block storage message to the first device.

Specifically, the logic circuit component may transmit the initial block storage packet to the first device, and the first device may store the initial block storage packet through the block storage component. The block storage component may be a device component, such as a hard disk drive, a solid state drive, etc., that uses a fixed-size data block as a base storage unit.

According to the embodiment of the application, the system-on-chip component is used for acquiring N transmission network messages sent by the second equipment through the node storage component, checking each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the checking. The storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component in the second equipment; a transport network message includes a unit network message and a corresponding redundancy check value. In the logic circuit assembly, the unit network messages in the N transmission network messages are subjected to message format conversion through a remote direct access protocol to obtain N unit storage messages, and the N unit storage messages are recombined into an initial block storage message based on message header information respectively associated with the N unit storage messages. The storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, the target storage length is smaller than the unit storage length, and the initial block storage message is transmitted to the first device. By combining the characteristics of the unit storage length of the node storage assembly, the maximum transmission unit of the BLK message and the maximum transmission unit of the NET message, and dividing the initial block storage message by the target storage length smaller than or equal to the unit storage length, N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage messages can be smaller than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, the storage length of the unit storage messages is the same as the storage length of the unit storage messages after the unit storage messages are subjected to message format conversion in the system-on-chip assembly through the remote direct access protocol, the redundancy check value generated by the unit network messages is the same as the redundancy check value generated by the unit storage messages, the unified check algorithm of the logic circuit assembly is realized, the end-to-end data protection from the front end of the first equipment to the rear end of the second equipment is realized, and the safety of storage service is improved.

Referring to fig. 7, fig. 7 is a flowchart illustrating a data processing method based on a network card according to an embodiment of the present application, where the data processing method may be performed by a second device, and the second device may be the second device 300 shown in fig. 1. The present data processing method will be described below by way of example as being executed by a network card. The data processing method at least includes the following steps S401 to S403:

Step S401, N transmission network messages sent by a first device through a network card are obtained; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

Specifically, the first device may establish a communication connection with the second device through the network card to perform data interaction. The first device may be a front-end device or a host. The network card may be an intelligent network card, a high performance network access card with a network processor as a core, which may be used to process various network protocols or storage protocols from a host. The second device may be a backend device that provides storage business services.

The second device may obtain, through the node storage component, N transport network packets sent by the first device through the network card. The transport network message may be a NET message with a message format conforming to RDMA protocol, i.e., a NET-RDMA message. The storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component in the second device. A transport network packet may include a unit network packet and associated redundancy check values, N unit network packets may be obtained by converting, by the network card, a packet format of a unit storage packet in N access block storage packets via a remote direct access protocol (RDMA), and an access block storage packet includes a unit storage packet and a redundancy check value (CRC) generated for the unit storage packet, where the unit storage packet may be a packet fragment in BLK format. The redundancy check value may also be called cyclic redundancy check, and is a check code commonly used in the communication field, and includes a series of data coding rules such as shift and division. Common coding methods are CRC-32, CRC-16, etc.

The N unit storage messages are obtained by dividing an initial block storage message based on unit storage length, and the initial block storage message can be an IO access message aiming at a block storage component in an operating system, namely a BLK message. The storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length. The specific generation process of the N transport network packets may be referred to the description of step S201 to step S207 in the corresponding embodiment of fig. 4, and the embodiments of the present application are not repeated here.

Step S402, obtaining unit network messages in each transmission network message and redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to the unit network messages are the same as the associated redundancy check values, respectively storing each transmission network message into a node storage component.

Specifically, the node storage component may obtain a unit network packet in each transport network packet and redundancy check values associated with the unit network packet, and perform exclusive or division operation on the service operation value and the N unit network packets to obtain N target check values. And if the target check value corresponding to each unit network message is the same as the associated redundancy check value, storing each transmission network message into the node storage component respectively.

It can be understood that the N unit network messages may be disordered during the transmission process of the network card and the second device, and the node storage component may sequence each transmission network message and then store the transmission network message, where the process may be: and acquiring message header information in the N transmission network messages, determining storage addresses corresponding to the N transmission network messages respectively based on the division ordering values in the N message header information, and storing each transmission network message to a storage unit corresponding to the associated storage address in the node storage component based on the N storage addresses.

Specifically, the node storage component may obtain Header information in N transport network packets, where the Header information of the RDMA protocol may include Route Header (RH), basic transport information (Base Transport Header, BTH), RDMA protocol information, and a partition ordering value, the Route information may include a source address and a destination address of the packet in each system, the basic transport information includes a type, a sequence number, and the like of the packet, and the RDMA protocol information may include detailed information of RDMA operation, such as an RDMA read or write operation, a back end storage address, a front end storage address, and the like. The sorting value can determine the sorting sequence among the N unit storage messages when the initial block storage message is divided to obtain the N unit storage messages.

The node storage component may determine storage addresses corresponding to the N transport network messages respectively based on the partition ordering values in the N message header information, and store each transport network message to a storage unit corresponding to the associated storage address in the node storage component based on the N storage addresses.

Step S403, when a storage service reading request sent by a first device is obtained, N stored transmission network messages are obtained from a node storage component, each transmission network message is checked, when all the N transmission network messages pass the check, the N transmission network messages are sent to a network card, so that the network card obtains N unit storage messages through the N transmission network messages, and an initial block storage message is generated based on the N unit storage messages; the initial block store message is for transmission to the first device.

Specifically, when the network card obtains a storage service reading request sent by the first device, the second device may obtain N stored transport network messages from the node storage component, check a redundancy check value in each transport network message, send the N transport network messages to the network card when all the N transport network messages pass the check, the network card may receive the N transport network messages through the protocol stack component, transmit the N transport network messages to the storage development component, perform message format conversion on unit network messages in the N transport network messages by the storage development component, obtain N unit storage messages, transmit the N unit storage messages to the logic circuit component, and reconstruct the N unit storage messages into an initial block storage message by the logic circuit component, and send the initial block storage message to the block storage component in the first device for storage.

The embodiment of the application acquires N transmission network messages sent by first equipment through a network card; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length; and acquiring unit network messages in each transmission network message and redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to the unit network messages are the same as the associated redundancy check values, respectively storing each transmission network message into a node storage component. By combining the characteristics of the unit storage length of the node storage assembly, the maximum transmission unit of the BLK message and the maximum transmission unit of the NET message, and dividing the initial block storage message by the target storage length smaller than or equal to the unit storage length, N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage messages can be smaller than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, the storage length of the unit storage messages is the same as the storage length of the unit storage messages after the unit storage messages are subjected to message format conversion in the system-on-chip assembly through the remote direct access protocol, the redundancy check value generated by the unit network messages is the same as the redundancy check value generated by the unit storage messages, the unified check algorithm of the logic circuit assembly is realized, the end-to-end data protection from the front end of the first equipment to the rear end of the second equipment is realized, and the safety of storage service is improved. And the ordering of N unit network messages in the transmission process of the network card and the second equipment is realized through the message header information, so that the feasibility of the storage service is improved.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a network card according to an embodiment of the application. As shown in fig. 8, the network card 1 includes a logic circuit component 510 and a system on chip component 520, wherein the system on chip component 520 includes a storage development component 521 and a protocol stack component 522.

The logic circuit component 510 is configured to obtain an initial block storage message transmitted by the block storage component in the first device, divide the initial block storage message into N unit storage messages with a target storage length based on a unit storage length of the node storage component of the second device, generate a redundancy check value of each unit storage message, generate N access block storage messages according to the N unit storage messages and the N redundancy check values, and transmit the N access block storage messages to the system-on-chip component 520; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

The system-on-chip component 520 is configured to perform message format conversion on unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generate N transport network messages according to the N unit network messages and the N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

The system-on-chip component 520 is further configured to send N transport network packets to the second device, so that the second device checks each transport network packet through the node storage component, and writes the N transport network packets into the node storage component when all the N transport network packets pass the check.

Wherein,

The logic circuit component 510 is further configured to obtain a unit storage length of the node storage component of the second device, determine a storage length of header information associated with the remote direct access protocol, and obtain a preset storage length for the redundancy check value;

The logic circuit component 510 is further configured to determine a target storage length based on the unit storage length, the storage length of the header information, and the preset storage length.

Wherein,

The logic circuit component 510 is specifically configured to perform post-bit-filling processing on the unit storage message based on the bit length P of the service operation value to obtain a bit-filling storage message, and perform exclusive-or division on the bit-filling storage message based on the service operation value to obtain a service redundancy value; p is a positive integer; the service operation value is determined based on a preset storage length for the redundancy check value;

The logic circuit component 510 is specifically configured to perform pre-bit-filling processing on the service redundancy value if the bit growth of the service redundancy value meets the pre-bit-filling condition, and determine the service redundancy value after the pre-bit-filling processing as a redundancy check value;

The logic circuit component 510 is specifically configured to determine the service redundancy value as the redundancy check value if the bit length of the service redundancy value does not meet the pre-bit-filling condition.

The logic circuit component 510 is specifically configured to determine, from M unit bit values of the bit-complement storage packet, consecutive P unit bit values with the target bit as a starting position as unit operation data, and perform an exclusive-or operation on the unit operation data based on the service operation value, to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the complementary bit storage message;

The logic circuit component 510 is specifically configured to determine a unit bit value except for P unit bit values in the M unit bit values as Q bit values to be complemented, perform post-complementation processing on the unit redundancy value based on the bit length P of the service operation value and the Q bit values to be complemented, obtain new unit operation data with the same bit length as the service operation value, and continuously perform exclusive or operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for completing post-complementation processing, and determine the finally obtained unit redundancy value as the service redundancy value; q is the difference between M and P.

Wherein,

The system-on-chip component 520 is specifically configured to obtain unit storage messages in the N access block storage messages, perform message format conversion on the N unit storage messages through a remote direct access protocol, obtain N unit network messages, and generate header information corresponding to each unit network message respectively;

The system-on-chip component 520 is specifically configured to encapsulate N unit network messages, header information corresponding to the N unit network messages, and N redundancy check values, respectively, into N transport network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

Wherein the system on chip component 520 includes a storage development component 521 and a protocol stack component 522;

A storage development component 521, configured to send N transport network packets to a protocol stack component 522;

a protocol stack component 522, configured to obtain a unit network packet in each transport network packet and a redundancy check value associated with the unit network packet;

The protocol stack component 522 is further configured to generate N target check values based on N unit network packets, and if the target check value corresponding to each unit network packet is the same as the associated redundancy check value, send N transport network packets to the second device through the protocol stack component 522.

Wherein,

The system-on-chip component 520 is further configured to obtain N transport network messages sent by the second device through the node storage component, check each transport network message, and send the N transport network messages to the logic circuit component 510 when all the N transport network messages pass the check;

The logic circuit component 510 is further configured to perform message format conversion on unit network messages in the N transport network messages through a remote direct access protocol, obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on header information associated with the N unit storage messages respectively;

The logic circuit component 510 is further configured to transmit an initial block storage message to the first device.

Fig. 9 is a schematic diagram of a second embodiment of a network card according to the present application. As shown in fig. 9, the network card 2 includes a system-on-chip component 610 and a logic circuit component 620.

The system-on-chip component 610 is configured to obtain N transport network messages sent by the second device through the node storage component, check a redundancy check value in each transport network message, and send the N transport network messages to the logic circuit component 620 when all the N transport network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage assembly; a transmission network message comprises a unit network message and a corresponding redundancy check value;

The logic circuit component 620 is configured to perform message format conversion on unit network messages in the N transport network messages through a remote direct access protocol, obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on header information associated with the N unit storage messages respectively; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

The logic circuit component 620 is further configured to transmit the initial block storage message to the first device.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 10, the data processing apparatus 1 includes a message acquisition module 710 and a message verification module 720.

The message obtaining module 710 is configured to obtain N transport network messages sent by the first device through the network card; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

the message check module 720 is configured to obtain a unit network message in each transport network message and redundancy check values associated with the unit network message, generate N target check values based on the N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, store each transport network message to the node storage component respectively.

In one possible implementation, the packet checking module 720 is configured to, when storing each transport network packet to the node storage component, specifically perform the following operations:

In one possible implementation, the packet verification module 720 is further configured to perform the following operations:

In the present embodiment, the term "module" or "unit" refers to a computer program or a part of a computer program having a predetermined function and working together with other relevant parts to achieve a predetermined object, and may be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Also, a processor (or multiple processors or memories) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of an overall module or unit that incorporates the functionality of the module or unit.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the application. As shown in fig. 11, the computer device 1000 may include: processor 1001, network interface 1004, and memory 1005, and in addition, the above-described computer device 1000 may further include: a user interface 1003, and at least one communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display (Display), a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface, among others. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may also optionally be at least one storage device located remotely from the processor 1001. As shown in fig. 11, an operating system, a network communication module, a user interface module, and a device control application may be included in the memory 1005, which is one type of computer-readable storage medium.

In the computer device 1000 shown in fig. 11, the network interface 1004 may provide a network communication network element; while user interface 1003 is primarily used as an interface for providing input to a user; and the processor 1001 may be used to invoke a device control application stored in the memory 1005 to implement:

Acquiring N transmission network messages sent by first equipment through a network card, and transmitting the N transmission network messages to a node storage component; n is a positive integer, and the storage length of N transmission network messages is smaller than or equal to the unit storage length of a node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by a network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to the target storage length, and the target storage length is smaller than the unit storage length;

It should be understood that the computer device 1000 described in the embodiments of the present application may perform the description of the data processing method in any of the embodiments corresponding to fig. 7, which is not repeated herein. In addition, the description of the beneficial effects of the same method is omitted.

Furthermore, it should be noted here that: the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, when the processor executes the computer program, the description of the data processing method in any one of the corresponding embodiments of fig. 7 is executed, so that a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present application, please refer to the description of the method embodiments of the present application.

The computer readable storage medium may be the data processing apparatus provided in any one of the foregoing embodiments or an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. that are provided on the computer device. Further, the computer-readable storage medium may also include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium may also be used to temporarily store data that has been or is to be displayed.

Furthermore, it should be noted here that: embodiments of the present application also provide a computer program product comprising a computer program stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer readable storage medium and executes the computer program to cause the computer device to perform the method provided by any of the corresponding embodiments of fig. 7 above.

The terms first, second and the like in the description and in the claims and drawings of embodiments of the application are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or modules but may, in the alternative, include other steps or modules not listed or inherent to such process, method, apparatus, article, or device.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied as electronic hardware, as a computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of network elements in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the solution. The skilled person may use different methods for implementing the described network elements for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

The method and related apparatus provided in the embodiments of the present application are described with reference to the flowchart and/or schematic structural diagrams of the method provided in the embodiments of the present application, and each flow and/or block of the flowchart and/or schematic structural diagrams of the method may be implemented by computer program instructions, and combinations of flows and/or blocks in the flowchart and/or block diagrams. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or structural diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or structures.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.

The modules in the device of the embodiment of the application can be combined, divided and deleted according to actual needs.

The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.

Claims

1. The data processing method based on the network card is characterized in that the method is applied to the network card, and the network card comprises a logic circuit component and a system-on-chip component; the method comprises the following steps:

The method comprises the steps that an initial block storage message transmitted by a block storage component in first equipment is obtained through a logic circuit component, the initial block storage message is divided into N unit storage messages with target storage lengths based on unit storage lengths of a node storage component of second equipment, redundancy check values of each unit storage message are generated, N access block storage messages are generated according to the N unit storage messages and the N redundancy check values, and the N access block storage messages are transmitted to a system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

in the system-on-chip component, converting a message format of a unit storage message in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and the N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

2. The method as recited in claim 1, further comprising:

and determining a target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

3. The method of claim 1, wherein generating the redundancy check value for each unit storage message comprises:

If the bit growth of the service redundancy value meets the pre-bit-supplementing condition, performing pre-bit-supplementing processing on the service redundancy value, and determining the service redundancy value after the pre-bit-supplementing processing as the redundancy check value;

And if the bit length of the service redundancy value does not meet the pre-bit supplementing condition, determining the service redundancy value as the redundancy check value.

4. The method of claim 3, wherein the bit-fill memory message comprises M unit bit values, M being a positive integer; the exclusive or division operation is performed on the bit-filling storage message based on the service operation value to obtain a service redundancy value, which comprises the following steps:

Based on the bit length P of the service operation value, determining continuous P unit bit values taking a target bit as a starting position in M unit bit values of the bit-supplementing storage message as unit operation data, and performing exclusive OR operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the bit-filling storage message;

Determining the unit bit values except the P unit bit values in the M unit bit values as Q bit values to be complemented, performing post-complementation processing on the unit redundancy values based on the bit length P of a service operation value and the Q bit values to be complemented to obtain new unit operation data with the same bit length as the service operation value, and continuously performing exclusive OR operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for finishing the post-complementation processing, and determining the finally obtained unit redundancy value as a service redundancy value; q is the difference between M and P.

5. The method of claim 1, wherein in the system-on-chip component, performing message format conversion on unit storage messages in the N access block storage messages by using a remote direct access protocol to obtain N unit network messages, and generating N transport network messages according to the N unit network messages and the N redundancy check values includes:

Obtaining unit storage messages in the N access block storage messages through the system-on-chip component, converting the message format of the N unit storage messages through a remote direct access protocol to obtain N unit network messages, and generating message header information corresponding to each unit network message respectively;

respectively encapsulating the N unit network messages, the message header information corresponding to the N unit network messages and the N redundancy check values into N transmission network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

6. The method of claim 1, wherein the system on chip component comprises a protocol stack component; the sending the N transport network packets to the second device includes:

transmitting N transmission network messages to the protocol stack assembly;

In the protocol stack assembly, obtaining a unit network message in each transmission network message and a redundancy check value associated with the unit network message;

Generating N target check values based on N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, transmitting the N transmission network messages to the second equipment through the protocol stack assembly.

7. The method as recited in claim 1, further comprising:

In the logic circuit assembly, converting a message format of a unit network message in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into the initial block storage message based on message header information respectively associated with the N unit storage messages;

and transmitting the initial block storage message to the first device.

8. The data processing method based on the network card is characterized in that the method is applied to the network card, and the network card comprises a logic circuit component and a system-on-chip component; the method comprises the following steps:

Acquiring N transmission network messages sent by a second device through a node storage component through the system-on-chip component, checking redundancy check values in each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component; a transmission network message comprises a unit network message and a corresponding redundancy check value;

In the logic circuit assembly, converting a message format of a unit network message in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

and transmitting the initial block storage message to first equipment.

9. The data processing method based on the network card is characterized in that the method is applied to second equipment, and the second equipment performs data interaction with first equipment through the network card; the method comprises the following steps:

Acquiring N transmission network messages sent by the first equipment through the network card; n is a positive integer, and the storage lengths of the N transmission network messages are smaller than or equal to the unit storage length of the node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by the network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

And acquiring unit network messages in each transmission network message and redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to each unit network message are the same as the associated redundancy check values, respectively storing each transmission network message into the node storage component.

10. The method of claim 9, wherein storing each transport network message to the node storage component comprises:

And acquiring message header information in N transmission network messages, determining storage addresses respectively corresponding to the N transmission network messages based on the dividing and sorting values in the N message header information, and storing each transmission network message to a storage unit corresponding to the associated storage address in the node storage component based on the N storage addresses.

11. The method as recited in claim 9, further comprising:

When a storage service reading request sent by the first device is obtained, obtaining N stored transmission network messages from the node storage component, checking each transmission network message, and when all the N transmission network messages pass the check, sending the N transmission network messages to the network card so that the network card obtains the N unit storage messages through the N transmission network messages, and generating the initial block storage message based on the N unit storage messages; the initial block storage message is used for being transmitted to the first device.

12. A network card, comprising:

The system-on-chip component is configured to perform message format conversion on unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generate N transport network messages according to the N unit network messages and the N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

The system-on-chip component is further configured to send the N transport network messages to the second device, so that the second device checks each transport network message through the node storage component, and when all of the N transport network messages pass the check, writes the N transport network messages into the node storage component.

13. A network card, comprising:

The system-on-chip component is used for acquiring N transmission network messages sent by the second equipment through the node storage component, checking the redundancy check value in each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component; a transmission network message comprises a unit network message and a corresponding redundancy check value;

the logic circuit component is configured to perform message format conversion on unit network messages in the N transport network messages through a remote direct access protocol to obtain N unit storage messages, and recombine the N unit storage messages into an initial block storage message based on header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

The logic circuit component is further configured to transmit the initial block storage packet to a first device.

14. A data processing apparatus, the data processing apparatus being applied to a second device, the data processing apparatus comprising:

The message acquisition module is used for acquiring N transmission network messages sent by the first equipment through the network card; n is a positive integer, and the storage lengths of the N transmission network messages are smaller than or equal to the unit storage length of the node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by the network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

and the message check module is used for acquiring the unit network messages in each transmission network message and the redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and respectively storing each transmission network message into the node storage component if the target check value corresponding to each unit network message is the same as the associated redundancy check value.

15. A computer device, comprising: a processor, a memory, and a network interface;

the processor is connected to the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store a computer program, and the processor is configured to invoke the computer program to cause the computer device to perform the method of any of claims 9-11.

16. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 9-11.

17. A computer program product, characterized in that the computer program product comprises a computer program stored in a computer readable storage medium and adapted to be read and executed by a processor to cause a computer device with the processor to perform the method of any of claims 9-11.