CN114296638A - Storage and computing integrated solid-state hard disk controller, solid-state hard disk, data storage system and method - Google Patents

Abstract

The embodiments of the present application relate to the application field of solid-state hard disks, and disclose a storage-computing integrated solid-state hard disk controller, a solid-state hard disk, a data storage system, and a data processing method between the storage-computing systems. The storage-computing integrated solid-state hard disk controller includes: The storage system is used for data storage; the computing system is used for data calculation; the storage and calculation path module is used to connect the storage system and the computing system, and is used to realize the data transmission processing between the storage system and the computing system. By designing a storage and calculation path module to realize data transmission between the storage system and the computing system, the present application can optimize the hardware path design between the storage and calculation systems inside the SSD controller, so that the storage system and the computing system can realize internal transmission standardization, thereby Reduce the amount of software development and improve the software development efficiency of solid-state drives.

Description

Storage-computing integrated solid-state hard disk controller, solid-state hard disk, data storage system and method

technical field

The present application relates to the application field of solid-state hard disks, and in particular, to a storage-computing integrated solid-state hard disk controller, a solid-state hard disk, a data storage system, and a data processing method between the storage and computing systems.

Background technique

A solid state drive (Solid State Drives, SSD) is a hard drive made of a solid state electronic storage chip array, and the solid state drive includes a control unit and a storage unit (FLASH memory chip or DRAM memory chip). At present, a considerable part of the solid-state hard disk system has a dynamic random access memory (Dynamic Random Access Memory, DRAM), so the SSD has a large data cache space for caching data. Flash memory (NAND Flash) is the main storage medium of solid-state drives.

Solid-state drives have become the main device for data storage. In the context of big data applications, massive data transmission occupies various buses and network bandwidth. At the same time, the computing power of the CPU itself contained in the solid-state drive is also getting stronger and stronger. In order to reduce the transmission of large amounts of data, various requirements for the integration of storage and computing are derived, such as edge computing (Edge Computing), machine learning (Machine Learning) application requirements in storage, that is, computing processing in the data-side SSD , only the result and part of the data are transmitted, thereby reducing mass data transmission and lightening the bus and network load.

At present, a dual-system architecture has been introduced into the storage-computing-integrated SSD: one system is used for data storage and processing, which is called a storage system, and the other system is used for data computing and processing, which is called a computing system, and the two systems are The communication between them is through the inter-core IPC (Inter-Process Communication) mechanism to complete the interaction of data and information. The IPC communication mechanism was originally used for inter-core communication. It is mainly based on the message mechanism. When the storage and computing integrated system is used, a custom protocol stack is required, that is, a lot of software development is required for both the storage system and the computing system. For example, under the computing system Drive development as a specific device, and continue data path development under the storage system. In the case of a custom protocol stack, the application and promotion in a general system will be limited to a certain extent.

In the process of realizing the present application, the applicant found that the current technical solution has at least the following technical problems: the communication mechanism between the internal storage and computing systems of the SSD controller integrated with storage and computing is not friendly to software development, resulting in the software development of the SSD. Development efficiency is insufficient.

SUMMARY OF THE INVENTION

The embodiments of the present application aim to provide a storage-computing integrated solid-state hard disk controller, a solid-state hard disk, a data storage system, and a data processing method between the storage-computing systems, which solve the problem of the internal storage-computing integrated solid-state hard disk controller. The communication mechanism between the storage and computing systems is not friendly to software development, which leads to the technical problem of insufficient software development efficiency of solid-state drives, thereby reducing the amount of software development and improving the software development efficiency of solid-state drives.

In order to solve the above-mentioned technical problems, the embodiments of the present application provide the following technical solutions:

In a first aspect, an embodiment of the present application provides an integrated storage-computing solid-state hard disk controller, which is used for storage-computing integrated solid-state storage, including:

storage system for data storage;

The computing system is used to perform data calculation; the storage and computing path module is used to connect the storage system and the computing system, and is used to realize data transmission between the storage system and the computing system.

In some embodiments, the storage path module includes:

The host module processing unit is used to simulate the function of the host side;

The device module processing unit is used to simulate the function of the device side;

The access processing unit is connected to the host module processing unit and the device module processing unit, and is used for processing the data requirements of the host module processing unit and the device module processing unit;

The data path interface is connected to the path processing unit and is used to process the interface requirements of the internal data bus.

In some embodiments, the storage and computing path module further includes:

The first system interface is connected to the host module processing unit, and is used for interacting with the host module processing unit;

The second system interface is connected to the device module processing unit, and is used for interacting with the device module processing unit.

In some embodiments, the storage-computing integrated solid-state disk controller further includes:

a first host interface, used for docking with the first host;

The second host interface is used to connect to the second host.

In some embodiments,

a storage system, including a first processor cluster, where the first processor cluster is used to process data storage operations;

A computing system includes a second processor cluster for processing data computing operations.

In some embodiments, the storage system further includes:

a front-end module, used for processing the communication protocol with the first host and distributing the data storage operation sent by the first host;

The data processing module is connected to the front-end module for the processing of the data path;

The mapping table management module is connected to the data processing module, which is used for the management of the mapping table and the management of the granularity of the data written to the flash memory;

The back-end module is connected to the mapping table management module and the flash media, and is used for flash data read and write and flash command management.

In some embodiments,

The computing system and the storage system share a dynamic random access memory and a dynamic random access memory controller, and the dynamic random access memory is connected to the dynamic random access memory controller;

The first processor cluster is connected to the storage and calculation path module, the second processor cluster is connected to the storage and calculation path module, and the front-end module is connected to the first processor cluster and the storage and calculation path module.

processing unit processing unit processing unit processing unit processing unit processing unit processing unit processing unit processing unit processing unit processing unit In some embodiments,

the first processor cluster includes application processors and/or real-time processors;

The second processor cluster includes application processors and/or real-time processors.

In a second aspect, an embodiment of the present application provides a solid-state hard disk, including:

Such as the storage-computing integrated solid-state hard disk controller of the first aspect;

At least one flash medium, connected to the SSD controller.

In a third aspect, an embodiment of the present application provides a data storage system, including:

Such as the solid-state hard disk of the second aspect;

The host system, which is connected to the solid-state hard disk in communication, includes:

the first host, used for docking with the storage system;

The second host is used for docking with the computing system.

In a fourth aspect, an embodiment of the present application provides a data processing method between storage and computing systems, which is applied to the storage system of the third aspect, and the method includes:

receiving an operation task command sent by the first host or the second host, wherein the operation task command includes information of at least one operation task;

At least one operation task is converted into an IO operation, and the IO operation is forwarded to the storage system or directly received by the storage system by the storage-computing channel module, so that the storage system processes the IO operation.

In a fifth aspect, the embodiments of the present application further provide a non-volatile computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the solid-state hard disk to execute the fourth aspect. The data processing method between the storage and computing systems.

The beneficial effects of the embodiments of the present application are: different from the prior art, an integrated storage-computing solid-state hard disk controller provided by the embodiments of the present application is applied to solid-state hard disks, and the storage-computation integrated solid-state hard disk controller includes: The storage system is used for data storage; the computing system is used for data calculation; the storage and calculation path module is used to connect the storage system and the computing system, and is used to realize the data transmission between the storage system and the computing system. By designing a storage and calculation path module to realize data transmission between the storage system and the computing system, the present application can optimize the internal hardware path design of the SSD controller, so that the storage system and the computing system can standardize the internal transmission, thereby reducing the amount of software development. Improve the software development efficiency of solid-state drives.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplifications do not constitute limitations of the embodiments, and elements with the same reference numerals in the drawings are denoted as similar elements, Unless otherwise stated, the figures in the accompanying drawings do not constitute a scale limitation.

FIG. 1 is a schematic structural diagram of a solid-state hard disk hardware framework provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a solid-state hard disk provided by an embodiment of the present application;

3 is a schematic structural diagram of a solid-state hard disk hardware framework provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a solid-state hard disk controller provided by an embodiment of the present application;

5 is a schematic diagram of a hardware system of a solid-state hard disk controller provided by an embodiment of the present application;

6 is a schematic diagram of a host and a storage device provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a standard PCIe hierarchical structure provided by an embodiment of the present application and the internal implementation of the PCIe hierarchical structure in the controller of the present application;

8 is a schematic diagram of an internal system of a storage-computing integrated solid-state hard disk controller provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a storage-computing integrated solid-state hard disk controller provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of another storage-computing integrated solid-state hard disk controller provided by an embodiment of the present application;

Fig. 11 is the structural representation of the storage calculation path module in Fig. 9;

12 is a schematic diagram of a connection between a storage and computing path module, a computing system, and a storage system provided by an embodiment of the present application;

13 is a schematic diagram of interaction between a storage and computing path module, a computing system, and a storage system provided by an embodiment of the present application;

14 is a schematic diagram of a hardware system framework of an integrated storage-computing solid-state hard disk provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of a first processor cluster provided by an embodiment of the present application;

FIG. 16 is a schematic structural diagram of a second processor cluster provided by an embodiment of the present application;

17 is a schematic diagram of an IO operation between an SSD and a host provided by an embodiment of the present application;

18 is a schematic structural diagram of a data storage system provided by an embodiment of the present application;

FIG. 19 is a schematic flowchart of a data processing method between storage and computing systems provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as there is no conflict with each other.

Please refer to FIG. 1, which is a schematic structural diagram of a solid-state hard disk hardware framework in the prior art;

As shown in FIG. 1, a solid state drive (Solid State Drives, SSD) generally includes a solid state drive controller, that is, a main controller (SSD Controller), a connector (Connector), a flash memory array, a cache unit, and other peripheral units.

Among them, the solid-state disk controller is used as a control arithmetic unit to manage the internal system of the SSD; the flash memory array (NANDFlash), as a storage unit, is used to store data, including user data and system data, and the flash memory array generally presents multiple channels (Channel, abbreviation). CH), a channel is independently connected to a group of NAND Flash, such as CH0/CH1...CHx. Among them, the flash memory (NAND Flash), whose characteristic is that it must be erased before writing, and the number of times of erasing each flash memory is limited; the cache unit is used to cache the mapping table, and the cache unit is generally a dynamic random access memory (Dynamic Random Access Memory). Memory, DRAM). A connector is used to connect a host, such as a PC or a server, and other peripheral units may include serial ports, sensors, registers, power chips, and other components.

Please refer to FIG. 2 again. FIG. 2 is a schematic structural diagram of a solid-state hard disk provided by an embodiment of the present application;

As shown in FIG. 2 , the solid state disk includes a flash memory medium and a solid state disk controller connected to the flash memory medium. Among them, the solid state disk is connected to the host through wired or wireless communication to realize data interaction.

Flash media, as the storage medium of solid-state drives, also known as flash memory, Flash, Flash memory or Flash particles, is a type of storage device and is a non-volatile memory that can last a long time without current supply. To save data, its storage characteristics are equivalent to hard disks, making flash memory media the basis of storage media for various types of portable digital devices.

Among them, the flash memory medium can be Nand FLASH. Nand FLASH uses a single transistor as a storage unit for binary signals. Its structure is very similar to that of ordinary semiconductor transistors. The difference is that the single transistor of Nand FLASH adds a floating gate and a control gate. The floating gate is used for To store electrons, the surface is covered by a layer of silicon oxide insulator, and is coupled with the control gate through a capacitor. When negative electrons are injected into the floating gate under the action of the control gate, the storage state of the single crystal of Nand FLASH is changed by " 1" becomes "0", and when the negative electrons are removed from the floating gate, the storage state changes from "0" to "1". Negative electrons are trapped, enabling data storage. That is, the storage unit of Nand FLASH is a floating gate transistor, and the floating gate transistor is used to store data in the form of charges. The amount of stored charge is related to the magnitude of the voltage applied to the floating gate transistor.

A Nand FLASH includes at least one Chip chip, each Chip chip is composed of several Block physical blocks, and each Block physical block includes several Page pages. Among them, Block physical block is the smallest unit of Nand FLASH to perform erase operation, Page page is the smallest unit of Nand FLASH to perform read and write operations, and the capacity of a Nand FLASH is equal to the number of its Block physical blocks * Page pages contained in a Block physical block The number * the capacity of a Page page. Specifically, the flash memory medium 200 can be classified into SLC, MLC, TLC, and QLC according to different voltage levels of the storage cells.

SSD controllers, including data converters, processors, caches, flash controllers, and interfaces.

The data converter is connected with the processor and the flash memory controller respectively, and the data converter is used for converting binary data to hexadecimal data and converting hexadecimal data to binary data. Specifically, when the flash memory controller writes data to the flash memory medium, the binary data to be written is converted into hexadecimal data by a data converter, and then written to the flash memory medium. When the flash memory controller reads data from the flash memory medium, it converts the hexadecimal data stored in the flash memory medium into binary data through a data converter, and then reads the converted data from the binary data page register. Wherein, the data converter may include binary data registers and hexadecimal data registers. The binary data register can be used to save the data converted from hexadecimal to binary, and the hexadecimal data register can be used to save the data converted from binary to hexadecimal.

The processor is connected to the data converter, the buffer, the flash memory controller and the interface respectively, wherein the processor and the data converter, the buffer, the flash memory controller and the interface can be connected by a bus or other means, and the processor is used for running the storage in the The non-volatile software programs, instructions and modules in the buffer can implement any method embodiment of the present application.

The buffer is mainly used to cache the read/write command sent by the host and the read data or write data obtained from the flash memory medium according to the read/write command sent by the host. As a non-volatile computer-readable storage medium, the buffer can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The buffer may include a program storage area, and the program storage area may store an operating system and an application program required by at least one function. Additionally, the cache may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the cache may optionally include memory located remotely from the processor. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The register may be Static Random Access Memory (SRAM), Tightly Coupled Memory (TCM), or Double DataRate Synchronous Dynamic Random Access Memory (DDR SRAM).

Flash controller, connected with flash media, data converters, processors and buffers, used to access back-end flash media, manage various parameters of flash media and data I/O; or, used to provide access interfaces and protocols , realize the corresponding SAS/SATA target protocol end or NVMe protocol end, obtain the I/O command issued by the host and decode and generate internal private data results to wait for execution; or, be responsible for the FTL (Flash translation layer, flash memory translation layer) core processing.

The interface is used to connect the host and the data converter, the processor and the buffer, and is used to receive the data sent by the host, or, to receive the data sent by the processor to realize the data transmission between the host and the processor. The interface can be a SATA-2 interface , SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface and M.2NVME/SATA protocol.

Please refer to FIG. 3 again. FIG. 3 is a schematic structural diagram of a solid-state hard disk hardware framework provided by an embodiment of the present application;

Compared with the SSD hardware framework in Figure 1, it has two interfaces, Connector0 and Connector1, which are used for data storage and data computing respectively. The separation of the host interface is used to solve the competition between computing application paths and data storage paths for bus bandwidth. The problem.

Please refer to FIG. 4 again. FIG. 4 is a schematic structural diagram of a solid-state hard disk controller provided by an embodiment of the present application;

As shown in FIG. 4 , the SSD controller includes two interfaces, namely a first interface and a second interface, which are respectively used for data storage and data calculation.

At present, due to the needs of computing applications, a dual-system architecture has been introduced in SSDs: one system is used for data storage processing, called a storage system, and the other system is used for data computing processing or application processing, called computing system. The two systems communicate through the IPC (Inter-Process Communication) mechanism to complete the exchange of data and information. The realization of its IPC includes software FIFO, hardware FIFO, shared memory, hardware IPC logic circuit and so on.

Please refer to FIG. 5 again. FIG. 5 is a schematic diagram of a hardware system of a solid-state hard disk controller according to an embodiment of the present application;

As shown in Figure 5, there is a hardware IPC design inside the SSD controller, that is, it includes an IPC system, and the data processing between the first processor cluster and the second processor cluster is realized through the IPC system. The design has an IPC software system. The data transmission path is realized through the hardware system and the software system, for example, the data computing processing or the data transmission path required for application processing is realized through the application IO path.

In terms of hardware, two hardware systems, namely the first processor cluster and the second processor cluster, exchange information through the IPC hardware interface, and a large amount of data is shared through DDR to realize data exchange.

From a software perspective, both software systems have a layered structure, so the IO operations passed between the two software systems are passed through the IPC system.

This data path exists as a specific device for the software of the computing system, and the underlying driver (running in SDL, Specific Drive Layer) is developed to dispatch or read IO operations, that is, write or read the information of IO operations. Interface to "FIFO/shared memory/hardware IPC logic circuit"; develop intermediate programs (running in MDL, MiddleLayer) to meet OS (Linux) docking requirements.

However, since the software design of the IPC system requires a custom protocol stack, whether it is a storage system or a computing system, a lot of software development is required, such as the development of specific device drivers under the computing system and the development of data paths under the storage system. This results in insufficient software development efficiency for solid-state drives.

In view of this, the present application provides a storage-computing integrated solid-state hard disk controller, a solid-state hard disk, a storage system, and a data processing method between the storage-computing system, so as to improve the software development efficiency of the solid-state hard disk.

The technical solutions of the present application are described in detail below with reference to the accompanying drawings.

Please refer to FIG. 6 again, FIG. 6 is a schematic diagram of a host and a storage device provided by an embodiment of the present application;

As shown in FIG. 6 , the host and the storage device communicate through a standard PCIe bus (through the PCIe protocol), where the PCIe bus is a PCB transmission line in the system, including slots, patch cords, and the like. Among them, in the traditional storage-computing integrated solid-state drive, two systems exist in the same solid-state drive controller, and there is no such standard path between the host and the device between the two systems. The standardized data path is realized in the controller, so that the storage system and the computing system can realize the internal transmission standardization.

Please refer to FIG. 7 again, FIG. 7 is a schematic diagram of a standard PCIe hierarchical structure provided by an embodiment of the present application and the internal implementation of the PCIe hierarchical structure in the controller of the present application;

As shown in Figure 7, the standard PCIe hierarchy includes two PCIe devices, namely a first PCIe device (PCIeDeviceA) and a second PCIe device (PCIe DeviceB), and each PCIe device includes a core layer (DeviceCore), a PCIe core Hardware/software interface (PCIe CoreHardware/SoftwareInterface), transaction layer (TransactionLayer), data link layer (Data Link Layer) and physical layer (Physical Layer). It can be understood that in the PCIe architecture, data packets are first generated in the device core layer (Device Core), and then pass through the device's transaction layer (Transaction Layer), data link layer (Data Link Layer) and Physical layer (Physical Layer), and finally sent out. The data at the receiving end also needs to pass through the physical layer, data link layer and transaction layer, and finally reach the core layer.

It can be understood that if the complete host and device modules are implemented in the SSD controller, the main problem is the high cost, and the internal transmission is obviously not suitable for this mode of transmission, and the analog circuit part of the interface at both ends (generally called for the PHY part), and even the control circuit does not need to be fully functional. Also take the PCIe hierarchy as an example, as shown in Figure 7, the left side is the standard hierarchy, what the software sees is the application layer, and does not care about other specific layers. Therefore, this application implements two layers inside the solid-state controller. The interaction of the system (A and B) replaces other layers through the data transmission layer to realize the data transmission required by the application layer.

In short, through the above implementation, the two systems in the SSD controller are regarded as a standard host system and a standard storage device system, thereby simplifying software design.

Please refer to FIG. 8. FIG. 8 is a schematic diagram of an internal system of an integrated storage-computing solid-state hard disk controller according to an embodiment of the present application;

As shown in Figure 8, the computing system and the storage system communicate through a storage and computing path, wherein the computing system is used as a standard host system. As seen from the computing system, the storage system is used as a standard storage system, and the computing system uses a standard protocol access to the storage system for read and write operations.

Please refer to FIG. 9. FIG. 9 is a schematic structural diagram of an integrated storage-computing solid-state hard disk controller according to an embodiment of the present application;

As shown in FIG. 9 , the storage-computing integrated solid-state hard disk controller 100 includes: a storage system 10 , a computing system 20 and a storage-computing path module 30 , wherein the storage system 10 and the computing system 20 pass through the storage-computing path module 30 . to connect.

Please refer to FIG. 10 again. FIG. 10 is a schematic structural diagram of another solid state disk controller with integrated storage and computing provided by an embodiment of the present application;

As shown in FIG. 10 , the storage-computing integrated solid-state disk controller 100 includes: a first host interface 41 , a second host interface 42 , a storage-computing path module 30 , a cache control module 21 , a processing module 11 , and a data path processing module 22 and the flash memory control module 23.

The first host interface 41 is used to connect to the first host, for example, used to receive data sent by the host, or receive data sent by the processor to realize data transmission between the host and the processor, and the interface may be a SATA- 2 interface, SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface and M.2NVME/SATA protocol.

Wherein, the second host interface 42 is used to connect to the second host, for example: used to receive data sent by the host, or receive data sent by the processor to realize data transmission between the host and the processor, and the interface may be a SATA- 2 interface, SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface and M.2NVME/SATA protocol.

The storage and computing path module 30 is used to realize the connection between the storage system 10 and the computing system 20 , so as to realize the data transmission between the storage system 10 and the computing system 20 .

The cache control module 21 includes a cache, which is mainly used to cache the read/write instructions sent by the host and the read data or write data obtained from the flash memory medium according to the read/write instructions sent by the host. As a non-volatile computer-readable storage medium, the buffer can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The buffer may include a program storage area, and the program storage area may store an operating system and an application program required by at least one function. Additionally, the cache may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the cache may optionally include memory located remotely from the processor. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The register may be Static Random Access Memory (SRAM), Tightly Coupled Memory (TCM), or Double DataRate Synchronous Dynamic Random Access Memory (DDRSRAM).

Wherein, the processing module 11 includes a processor, which is used for connecting with the data converter, the buffer, the flash memory controller and the interface, wherein the processor and the data converter, the buffer, the flash memory controller and the interface can be connected through a bus or other means connected, the processor is configured to run the non-volatile software programs, instructions and modules stored in the buffer, so as to implement any method embodiment of the present application. Or, it is used to provide access interfaces and protocols, implement the corresponding SAS/SATA target protocol end or NVMe protocol end, obtain I/O commands issued by the host, decode and generate internal private data results for execution; or, be responsible for FTL (Flash translation layer, flash translation layer) core processing.

The data path processing module 22 includes a data path hardware logic module, such as a Data PathProcessor, which is used to accelerate the processing of stored data.

The flash control module 23 includes a flash controller and is connected to a flash medium, a data converter, a processor and a buffer, and is used to access the back-end flash medium and manage various parameters and data I/O of the flash medium.

Please refer to FIG. 11 again, FIG. 11 is a schematic structural diagram of the storage and calculation path module in FIG. 9;

As shown in FIG. 11 , the storage and calculation path module 30 includes: a host module processing unit 31 , a device module processing unit 32 , a path processing unit 33 , a data path interface 34 , a second system interface 35 and a first system interface 36 , wherein , the host module processing unit 31 is connected to the channel processing unit 33, the channel processing unit 33 is connected to the device module processing unit 32, the data channel interface 34 is connected to the channel processing unit 33, the second system interface 35 is connected to the host module processing unit 31, and the first system interface 36 The device module processing unit 32 is connected.

It can be understood that, inside the SSD controller, for the computing system, there is actually no Host hardware module that manages standard storage devices, such as PCIe Host device-PCIe RC (Root Complex); for the storage system In fact, there is actually no Device hardware module for the host of the internal computing system, such as PCIe EP device, so the storage and computing channel module needs to implement the simulation function of the two devices and the processing function of the data transmission channel.

Specifically, the host module processing unit 31 is used to simulate the function of the host, such as the communication function of the host, such as the PCIe RC function, that is, the root complex (Root Complex, RC), that is, the root node ( Root) function, wherein the PCIe RC function refers to the function of communicating with other parts of the computer system, for example: the CPU accesses the memory through the root complex (Root Complex), or accesses the PCIe system through the root complex (Root Complex) PCIe device. Among them, the function of the host side belongs to a part of the standard protocol, and is implemented according to the standard protocol definition. In this embodiment of the present application, the host module processing unit 31 includes an emulator.

It can be understood that peripheral component interconnect express (PCIe) is a high-speed short-distance communication interface, which is widely used in computers, test instruments and other devices. The main components of a PCIe system are a root node (Root), a switch node (Switch) and an end node (Endpoint).

Among them, Root is responsible for managing all buses and nodes in the PCIe system, and is the bridge between the central processing unit (CPU) and the Endpoint in the PCIe system; Switch acts as a data forwarding node, connecting Switch and Endpoint; Endpoint is an end device, such as external Set (Peripheral). In the PCIe system, the Endpoint and the Endpoint cannot communicate directly, and must go through the Root.

Specifically, the device module processing unit 32 is used to simulate the function of the device side, for example, the PCIe EP function (PCIe Endpoint), that is, the function of the end node (Endpoint) of the PCIe system. Among them, the function of the device end belongs to a part of the standard protocol, and is implemented according to the standard protocol definition. In this embodiment of the present application, the device module processing unit 32 includes a simulator.

It should be noted that the host module processing unit 31 and the device module processing unit 32 only need to simulate part of the functions, because the data is transmitted on the internal data bus, and there is no need for analog circuits/interfaces. For example, the host module processing unit 31 and the device module processing unit 32 are used to simulate the data transmission function. As shown in FIG. 7 , the first PCIe device (PCIe Device A) and the second PCIe device (PCIe Device B) are implemented inside the SSD controller. ) interaction, the data transmission layer replaces other layers to realize the data transmission required by the application layer, which is equivalent to the function of the host module processing unit and the device module processing unit used to realize the function of the data transmission layer.

Specifically, the path processing unit 33 is connected to the host module processing unit 31 and the device module processing unit 32, and is used to process the data requirements of the host module processing unit 31 and the device module processing unit 32, for example: for processing data forwarding, format changes, etc.

Specifically, the data path interface 34 is connected to the path processing unit 33 for processing the interface requirements of the internal data bus.

Specifically, the first system interface 36 is connected to the device module processing unit 32, and is used for interactive operation with the device module processing unit 32, and is used for the communication between the storage system and the storage and computing path module.

Specifically, the second system interface 35 is connected to the host module processing unit 31, and is used for interactive operation with the host module processing unit 31, and is used for communication between the computing system and the storage-computation path module.

Processing unit Please refer to FIG. 12 again for the processing unit. FIG. 12 is a schematic diagram of a connection between a storage and computing path module, a computing system, and a storage system provided by an embodiment of the present application;

As shown in FIG. 12 , the storage system includes a first processor cluster, a DDR channel and an NVMe channel, and the computing system includes a second processor cluster and a DDR channel, wherein the first system interface is connected to the first processor cluster of the storage system, and the first system interface is connected to the first processor cluster of the storage system. A processor cluster is connected to the DDR channel, the data channel interface is connected to the DDR channel and the NVMe channel, the second system interface is connected to the second processor cluster of the storage system, and the second processor cluster is connected to the DDR channel. Among them, the DDR channel is used to connect to the DDR memory, that is, double-rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR); the NVMe channel is used to connect to the NVMe controller.

Please refer to FIG. 13 again. FIG. 13 is a schematic diagram of interaction between a data path module and a computing system and a storage system provided by an embodiment of the present application;

As shown in Figure 13, the second processor cluster of the processing unit processing unit processing unit computing system passes through the second system interface of the storage and computing channel module, through the host module processing unit, the channel processing unit, and the data channel interface, from the NVMe of the storage system. The channel is used to send an operation command to the first processor cluster of the storage system.

The first processor cluster of the storage system sends a response from the second system interface to the second processor cluster of the computing system through the first system interface of the storage and computing channel module, through the device module processing unit, the channel processing unit, and the host module processing unit. .

The data block is obtained from the flash memory medium by the back-end module, and the data block obtained by the back-end module reaches the second processor cluster through the NVMe channel and the DDR channel, and is calculated and processed by the second processor cluster.

Please refer to FIG. 14 again. FIG. 14 is a schematic diagram of a hardware system framework of an integrated storage-computing solid-state hard disk provided by an embodiment of the present application;

As shown in Figure 14, where Host1 is the first host, Host2 is the second host, Host1IF is the first host interface, Host2 IF is the second host interface, CPU Cluster1 is the first processor cluster, and CPU Cluster2 is the second processor iHost2Device Channel is the storage and computing channel module, FEModule is the front-end module, BE Module is the back-end module, NANDFlashArray is the flash memory medium, DDR is the DDR memory, DDRController/PHY is the DDR controller, which is used to control and process the interface requirements of DRAM , wherein the DDR controller includes a DDR interface.

The first processor cluster (CPU Cluster 1) includes a data storage and processing system, the second processor cluster (CPU Cluster 2) includes a data computing application system, and both the data storage processing system and the data computing application system have a hierarchical structure. The IO operations transmitted between the data storage processing system and the data computing application system are transmitted through the storage and computing channel module (iHost2Device Channel). Specifically, the transmission is completed through a data processing unit under a standard protocol, and the data processing unit includes data, operation commands, and the like. It can be understood that, for a driver, dispatching or reading an IO operation is to write or read the information of the IO operation to the first system interface or the second system interface of the storage path module.

Specifically, the first host (Host1) interacts with the flash medium (NAND Flash Array) via the first host interface (Host1 IF), the front-end module (FE Module), and the back-end module (BE Module);

The second host (Host2) via the second host interface (Host2 IF), the second processor cluster (CPUCluster2), the DDR memory (DDR), the DDR controller (DDR Controller/PHY), the memory computing channel module (iHost2Device Channel), The front-end module (FE Module) and the back-end module (BE Module) interact with the flash medium (NANDFlash Array).

Please refer to FIG. 15 again. FIG. 15 is a schematic structural diagram of a first processor cluster provided by an embodiment of the present application;

As shown in FIG. 15 , the first processor cluster 11 includes a data storage and processing system 110 for performing data storage operations. The data storage and processing system 110 includes:

The front-end module 111, namely (Front End, FE), is connected to the data storage interface, and is used for processing the communication protocol with the host system and distributing the data storage operation sent by the host system;

The data processing module 112, namely (Data Process, DP), is connected to the front-end module 111 and is used for data path processing, specifically, is used for command-level data processing, such as buffering data, etc.;

The mapping table management module 113, namely (Flash Translation Layer, FTL), is connected to the data processing module 112, and is used for the management of the mapping table and the management of the granularity of the data written to the flash memory;

The back-end module 114, namely (Back End, BE), is connected to the mapping table management module 113 and the flash memory medium, and is used for reading and writing flash data and managing flash commands.

In this embodiment of the present application, the first processor cluster includes an application processor and/or a real-time processor.

Please refer to FIG. 16 again. FIG. 16 is a schematic structural diagram of a second processor cluster provided by an embodiment of the present application;

As shown in FIG. 16 , the second processor cluster 21 includes a data computing application system 210 for performing data computing operations, wherein the data computing application system 210 includes: an operating system module 211 , a data computing application module 212 , and NVMe Device module 213 and PCIe device module 214 .

Specifically, the operating system module 211 is used to perform the bottom-level operation of the operating system. The operating system module 211 is the operating system layer (OS Kernel), that is, the operating system kernel, and is used to perform the bottom-level operations of the operating system, including command parsing and code compilation. and other underlying operations;

Specifically, the data computing application module 212 is connected to the operating system module 211, and is used to perform application processing related to the data computing task sent by the host. Perform application processing related to host tasks. Based on the operating system (Operation System, OS), the development of the application layer has a very high versatility and can be separated from the relevant hardware characteristics. The application includes the search engine's List Intersection (List Intersection), MySQL's retrieval, etc. The computing application module includes an application layer (ApplicationLayer, APL).

Specifically, the NVMe device module 213 is connected to the operating system module 211, the data computing application module 212 and the PCIe device module 214, and is used for converting data computing tasks into IO operations, and sending corresponding IO operations to the NVMe memory. In this embodiment of the present application, the NVMe device module 213 includes an NVMe device layer (NVMeDeviceLayer).

Specifically, the PCIe device module 214 is connected to the NVMe device module 213, the operating system module 211 and the front-end module of the first processor cluster, and is used for sending IO operations to the front-end module of the first processor cluster. In this embodiment of the present application, the PCIe device module 214 includes a PCIe device layer (PCIe Device Layer).

In this embodiment of the present application, the data computing application system 210 further includes an interface module 215. The interface module 215 is connected to the data computing application module 212 and the PCIe device module 214. The interface module 215 is used for sending IO operations, or the interface module 215 is communicatively connected to the host. Interface for receiving host commands sent by the host.

In this embodiment of the present application, the second processor cluster 21 includes an application processor and/or a real-time processor.

In this embodiment of the present application, the data storage processing system and the data computing application system run independently and in parallel. By running the data storage and processing system on the first processor cluster for special processing of storage transactions, and running the data computing application system on the second processor cluster for special processing of computing application transactions, the present application can improve the first processor Processing efficiency of the cluster and the second processor cluster.

Please refer to FIG. 17 again. FIG. 17 is a schematic diagram of an IO operation between an SSD and a host provided by an embodiment of the present application;

As shown in FIG. 17 , the second processor cluster (CPUCluster 2) functions as an internal host (InternalHost) in the function of the SSD controller, and the first processor cluster (CPU Cluster 1) functions as the SSD controller. The internal device (InternalDevice) implements the standard storage device path through the internal virtual host and device connection (VirtualHost2DeviceConnect).

It can be seen that the computing system does not need specific device paths, and can use standard storage device paths, typically, such as PCIe Device Layer and NVMe Device Layer paths, to complete the information with the storage system. Or data interaction, so that the computing system can read and write operations to the storage system through standard channels. For example: the back-end module (BE) of the storage system passes through the mapping table management module (FTL), and then passes through the data path module (DP) to the front-end module (FE), and the front-end module (FE) communicates with the PCIe device layer of the computing system. (PCIe Device Layer), NVMe Device Layer (NVMe Device Layer), and Application Layer (Application Layer, APL), so as to realize the interaction between the computing system and the storage system.

Please refer to FIG. 18 again. FIG. 18 is a schematic structural diagram of a data storage system provided by an embodiment of the present application;

As shown in FIG. 18 , the data storage system 400 includes: a solid-state drive 200 and a host system 300 , and the solid-state drive 200 is communicatively connected to the host system 300 , wherein the solid-state drive 200 includes a storage-computing integrated solid-state drive controller 100 and a flash memory medium 220 , the storage-computing integrated solid-state hard disk controller 100 is connected to the flash memory medium 220 .

Specifically, the storage-computing integrated solid-state disk controller 100 includes: a storage system 10, a computing system 20, a storage-computing path module 30, a first host interface 41 and a second host interface 42, wherein the first host interface 41 is connected to the storage system 10. The second host interface 42 is connected to the computing system 20 , and the storage and computing path module 30 is connected to the storage system 10 and the computing system 20 respectively, so as to realize data and information interaction between the storage system 10 and the computing system 20 .

In this embodiment of the present application, the storage system 10 is connected to the flash memory medium 220 to perform IO operations on the flash memory medium.

Specifically, the host system 300 includes a first host 310 and a second host 320 . The first host 310 communicates with the storage system 10 through the first host interface 41 , and the first host 310 sends data storage to the storage system 10 through the first host interface 41 . operation so that the storage system 10 processes the data storage operation, the second host 320 is communicatively connected to the computing system 20 through the second host interface 42, and the second host 320 sends the data computing operation to the computing system 20 through the second host interface 42, so that the computing System 20 handles data computing operations.

It can be understood that the first host and the second host may be the same host or different hosts, for example, the first host is a local host, and the second host is a cloud server. At the same time, the first and the second have no mandatory order relationship, they are only descriptive terms.

In the embodiments of the present application, a storage-computing integrated solid-state hard disk controller is provided, which is applied to a storage-computing integrated solid-state hard disk, and the storage-computing integrated solid-state hard disk controller includes a storage system for data storage; computing The system is used to perform data calculation; the storage and calculation path module is used to connect the storage system and the computing system, and is used to realize data transmission between the storage system and the computing system. By designing a storage and computing path module to realize data transmission between the storage system and the computing system, the present application can optimize the internal hardware path design of the SSD controller, so that the storage system and the computing system can standardize the internal IO transmission, thereby reducing the amount of software development , to improve the software development efficiency of solid-state drives.

Please refer to FIG. 19 again. FIG. 19 is a schematic flowchart of a data processing method between storage and computing systems provided by an embodiment of the present application;

Wherein, the data processing method between the storage and computing systems is applied to the data storage system mentioned in the above-mentioned embodiment, and the data storage system includes a solid-state hard disk and a host system. Specifically, the data processing method between the storage and computing systems The storage-computation-integrated solid-state hard disk controller applied to the solid-state hard disk of the above embodiment, wherein the storage-computation integrated solid-state hard disk controller includes a storage system, a computing system, and a storage-computation path module.

As shown in Figure 19, the data processing method includes:

Step S10: Receive an operation task command sent by the first host or the second host, wherein the operation task command includes information of at least one operation task;

Specifically, the storage-computing integrated solid-state hard disk controller receives the operation task command sent by the host system, specifically, receives the operation task command sent by the first host through the first host interface, or receives the second host through the second host interface. The operation task command, wherein the operation task command includes the information of at least one operation task.

Step S20: Convert at least one operation task into an IO operation, and the storage and computation path module forwards the IO operation to the storage system or directly receives the IO operation, so that the storage system processes the IO operation.

The operation task includes a computing task, and if the first host interface receives an operation task command sent by the first host, the storage system converts at least one operation task corresponding to the operation task command into an IO operation, and sends the operation task to the storage computing path module. The IO operation is sent, and the storage and computation path module forwards the IO operation to the computing system, so that the computing system processes the IO operation.

Specifically, after receiving the operation task command sent by the first host, the first processor cluster of the storage system converts the operation task corresponding to the operation task command into an IO operation, and passes through the first system interface of the storage and calculation path module, the device module The processing unit, the channel processing unit, the host module processing unit, and then to the second system interface, the second system interface sends the IO operation to the second processor cluster of the computing system, so that the second processor cluster of the computing system processes The IO operation is to obtain the result of the IO operation, such as the calculation result of the data.

The operation task includes a read and write task. If the second host interface receives an operation task command sent by the second host, the computing system converts at least one operation task corresponding to the operation task command into an IO operation, and sends the operation task to the storage computing path. The module sends the IO operation, and the storage and computation path module forwards the IO operation to the storage system, so that the storage system processes the IO operation.

Specifically, after receiving the operation task command sent by the second host, the second processor cluster of the computing system converts the operation task corresponding to the operation task command into an IO operation, and passes through the second system interface of the storage computing channel module and the host module. The processing unit, the channel processing unit, and the data channel interface send the IO operation to the NVMe channel of the storage system, and the NVMe channel sends the IO operation to the first processor cluster of the storage system, so that the first processor of the storage system The cluster processes the IO operation to obtain the result of the IO operation, such as obtaining the queried data.

In the embodiment of the present application, a data processing method between storage and computing systems is provided, which is applied to a data storage system. The method includes: receiving an operation task command sent by a first host or a second host, wherein the operation task command includes Information of at least one operation task; convert the at least one operation task into an IO operation, and the storage and computation path module forwards the IO operation to the storage system or directly receives the IO operation from the storage system, so that the storage system processes the IO operation. By receiving the operation task command sent by the first host or the second host, the storage and computing channel module forwards the IO operation corresponding to the operation task command to the storage system or directly receives it by the storage system, so that the storage system can process the IO operation, the present application can Realize the standardization of internal transmission, thereby reducing the amount of software development and improving the software development efficiency of solid-state drives.

Embodiments of the present application also provide a non-volatile computer storage medium, where the computer storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, so that the one or more processors described above can be executed. The data processing method between storage and computing systems in any of the above method embodiments can be executed, for example, the data processing method between storage and computing systems in any of the above method embodiments can be executed, for example, the above-described method shown in FIG. 19 can be executed. each step.

The apparatus or device embodiments described above are only illustrative, wherein the unit modules described as separate components may or may not be physically separated, and the components shown as module units may or may not be physical units, that is, It can be located in one place, or it can be distributed over multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence, or the parts that make contributions to related technologies, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic disks , CD-ROM, etc., including several instructions until a computer device (which may be a personal computer, server, or network device, etc.) executes the method of various embodiments or portions of embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; under the thinking of the present application, the technical features in the above embodiments or different embodiments can also be combined, The steps may be carried out in any order, and there are many other variations of the various aspects of the present application as described above, which are not provided in detail for the sake of brevity; although the present application has been It should be understood that: it is still possible to modify the technical solutions recorded in the foregoing embodiments, or to perform equivalent replacements to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technology of each embodiment of the application scope of the programme.

Claims (10)

1. A storage-computing integrated solid-state hard disk controller for storage-computing integrated solid-state storage is characterized in that, comprising: storage system for data storage; a computing system for performing data calculations; A storage and computing path module is connected to the storage system and the computing system, and is used for realizing data transmission between the storage system and the computing system. 2 . The storage-computing integrated solid-state hard disk controller according to claim 1 , wherein the storage-computing path module comprises: 2 . The host module processing unit is used to simulate the function of the host side; The device module processing unit is used to simulate the function of the device side; a path processing unit, connected to the host module processing unit and the device module processing unit, for processing data requirements of the host module processing unit and the device module processing unit; The data path interface is connected to the path processing unit and used for processing the interface requirements of the internal data bus. 3. The integrated storage-computing solid-state disk controller according to claim 2, wherein the storage-computing path module further comprises: a first system interface, connected to the host module processing unit, for performing interactive operations with the host module processing unit; The second system interface is connected to the device module processing unit, and is used for interacting with the device module processing unit. 4 . The storage-computing integrated solid-state hard disk controller according to claim 1 , wherein the storage-computing integrated solid-state hard disk controller further comprises: 4 . a first host interface, used for docking with the first host; The second host interface is used to connect to the second host. 5 . The storage-computing integrated solid-state hard disk controller according to claim 1 , wherein the storage system comprises a first processor cluster, and the first processor cluster is used for processing data storage operations; 6 . The computing system includes a second processor cluster for processing data computing operations. 6. The storage-computing integrated solid-state hard disk controller according to claim 5, wherein the storage system further comprises: a front-end module, used for processing the communication protocol with the first host and distributing the data storage operation sent by the first host; a data processing module, connected to the front-end module, for processing the data path; a mapping table management module, connected to the data processing module, for the management of the mapping table and the management of the granularity of data written to the flash memory; The back-end module is connected to the mapping table management module and the flash memory medium, and is used for flash data read and write and flash command management. 7. The storage-computing integrated solid-state hard disk controller according to claim 6, wherein, The computing system and the storage system share a dynamic random access memory and a dynamic random access memory controller, and the dynamic random access memory is connected to the dynamic random access memory controller; The first processor cluster is connected to the storage and calculation path module, the second processor cluster is connected to the storage and calculation path module, and the front-end module is connected to the first processor cluster and the storage and calculation path module. 8. A solid-state hard disk, comprising: The storage-computing integrated solid-state hard disk controller according to any one of claims 1-7; At least one flash memory medium is connected to the storage-computing integrated solid-state hard disk controller. 9. A data storage system, comprising: The solid state drive of claim 8; A host system, communicatively connected to the solid-state hard disk, the host system includes: a first host, configured to interface with the storage system; The second host is used for docking with the computing system. 10. A data processing method between storage and computing systems, characterized in that, applied to the data storage system as claimed in claim 9, the method comprising: receiving an operation task command sent by the first host or the second host, wherein the operation task command includes information of at least one operation task; Converting the at least one operation task into an IO operation, and the storage computing path module forwards the IO operation to the storage system or directly receives the IO operation by the storage system, so that the storage system processes the IO operation .

Images