[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN117009258A - File system disk access method, device and equipment - Google Patents

File system disk access method, device and equipment Download PDF

Info

Publication number
CN117009258A
CN117009258A CN202310987553.XA CN202310987553A CN117009258A CN 117009258 A CN117009258 A CN 117009258A CN 202310987553 A CN202310987553 A CN 202310987553A CN 117009258 A CN117009258 A CN 117009258A
Authority
CN
China
Prior art keywords
module
buffer
page
page buffer
vfs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310987553.XA
Other languages
Chinese (zh)
Inventor
谢永吉
张佳辰
柴稳
张宇
王剑
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Youzhuju Network Technology Co Ltd
Original Assignee
Beijing Youzhuju Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Youzhuju Network Technology Co Ltd filed Critical Beijing Youzhuju Network Technology Co Ltd
Priority to CN202310987553.XA priority Critical patent/CN117009258A/en
Publication of CN117009258A publication Critical patent/CN117009258A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0877Cache access modes
    • G06F12/0882Page mode
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The disclosure provides a method, a device and equipment for accessing a file system disk, which are applied to a kernel layer, wherein the method comprises the following steps: when the VFS module executes disk data writing operation through a first interface, the page buffer management module acquires a first buffer block in a storage medium of a hardware layer disk, wherein the first buffer block is an idle block, and the page buffer management module detects whether a physical address on the first buffer block is mapped with a space address; if not, the page buffer expansion module is utilized to allocate a virtual address for the page buffer expansion module, and a space mapping relation between the virtual address and a physical address is established; converting the first buffer block into a first page buffer, transmitting the spatial mapping relation and the virtual address to a VFS module through a page buffer expanding module, adding the first page buffer into an internal data structure by the VFS module, and copying data to the first page buffer. The method reduces one-time data copying, thereby improving the efficiency of the whole transmission process and reducing the consumption of CPU and RAM at the host side.

Description

File system disk access method, device and equipment
Technical Field
The present disclosure relates to the field of storage, and in particular, to a method, an apparatus, and a device for accessing a file system disk.
Background
With the rapid development of storage technology, SSD (solid state drive) based on NVMe (Nonvolatile Memory Express, nonvolatile memory host controller interface specification) technology is increasingly widely used in the field of enterprise data center storage. Most application programs access data to the NVMe SSD disk through a file system (e.g., ext4, xfs), which makes the efficiency of data access to the NVMe SSD disk through the file system critical.
At present, in the whole process of performing data writing operation on an NVMe SSD disk by an application program through a file system, two times of data copying are involved, namely, the data is copied from the user state application program to the kernel state, and the data is copied from the kernel state to a data storage medium of an NVMe controller, wherein the two times of data copying processes can cause low whole transmission efficiency, and the consumption of a host side CPU and a random access memory (Random Access Memory, RAM) is increased.
Disclosure of Invention
In view of this, the present disclosure provides a method for accessing a disk of a file system, so as to improve the data copy transmission efficiency and reduce the consumption of CPU and RAM.
In a first aspect, the present disclosure provides a method for accessing a file system disk, where the method is applied to a kernel layer, the kernel layer includes a page buffer expansion module and a page buffer management module, the page buffer expansion module is connected to a virtual file system VFS module and a driving module through at least one interface, the driving module is connected to a hardware layer, and the page buffer management module is located in the driving module, and the method includes:
the page buffer management module obtains a first buffer block in a storage medium of a hardware layer disk when the VFS module executes disk data writing operation through a first interface, wherein the first buffer block is an idle block, and the first interface is one of the interfaces;
the page buffer management module detects whether the physical address on the first buffer block is mapped with a space address;
if not, the page buffer expansion module is utilized to allocate a virtual address for the page buffer expansion module, and a space mapping relation between the virtual address and the physical address is established;
the page buffer memory management module converts the first buffer block into a first page buffer memory, and sends the space mapping relation and the virtual address to the VFS module through the page buffer memory expansion module, wherein the first page buffer memory is used for enabling the VFS module to add and copy data.
In a second aspect, an embodiment of the present disclosure further provides a method for accessing a file system disk, where the method is applied to a VFS module of a virtual file system, where the VFS module is located in a kernel layer, and the method includes:
when receiving a disk data writing operation from a user layer, the VFS module inquires whether a first page buffer exists in a file data block corresponding to data to be written, wherein the first page buffer is generated by converting a first buffer block in a storage medium of a hardware layer disk, and the first buffer block is an idle block;
if so, the VFS module adds the first page cache to an internal data structure and copies data to the first page cache.
In a third aspect, an embodiment of the present disclosure provides an access apparatus for a file system disk, the apparatus including:
the system comprises an acquisition module, a page buffer expansion module and a page buffer expansion module, wherein the acquisition module is used for acquiring a first buffer block in a storage medium of a hardware layer disk when a Virtual File System (VFS) module executes disk data writing operation through a first interface, the first buffer block is an idle block, and the first interface is one of at least one interface of the page buffer expansion module;
the detection module is used for detecting whether the physical address on the first buffer block is subjected to space address mapping;
the processing module is used for establishing a space mapping relation between the virtual address and the physical address by utilizing the virtual address distributed by the page buffer expansion module when the incompletion is detected, and converting the first buffer block into a first page buffer;
and the sending module is used for sending the space mapping relation and the virtual address to the VFS module through the page buffer expansion module, and the first page buffer is used for enabling the VFS module to add and copy data.
In a fourth aspect, an embodiment of the present disclosure further provides an access apparatus for a file system disk, where the apparatus includes:
the system comprises a query module, a first page buffer and a second page buffer, wherein the query module is used for querying whether a first page buffer exists in a file data block corresponding to data to be written or not when receiving a disk data writing operation from a user layer, the first page buffer is generated by converting a first buffer block in a storage medium of a hardware layer disk, and the first buffer block is an idle block;
and the copying module is used for adding the first page cache into an internal data structure and copying data to the first page cache.
In a fifth aspect, the present disclosure provides an electronic device comprising: the memory and the processor are in communication connection, computer instructions are stored in the memory, and the processor executes the computer instructions, so that the method for accessing the file system disk according to the first aspect or any implementation manner corresponding to the first aspect is executed.
Furthermore, the present disclosure provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the method for accessing a file system disk of the first aspect or any of its corresponding embodiments.
According to the access method, the device and the equipment for the file system disk, the Page Cache expansion module and the Page Cache management module are additionally arranged in the kernel layer, the existing Page Cache framework of the kernel layer is expanded, and therefore a Cache management mechanism of most kernels is multiplexed, and no perception is achieved for users. In addition, the Page buffer management module establishes the mapping between the physical address and the virtual address of the buffer block in the hardware layer, so that the storage medium of the hardware layer is registered as the Page Cache, the direct data copy on the storage medium is realized, the data is prevented from being copied twice, the method reduces the primary data copy, improves the data transmission efficiency, and reduces the consumption of the CPU and the RAM.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the prior art, the drawings that are required in the detailed description or the prior art will be briefly described, it will be apparent that the drawings in the following description are some embodiments of the present disclosure, and other drawings may be obtained according to the drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a block diagram of a Linux file I/O system provided by an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of writing data to an NVMe SSD disk according to an embodiment of the disclosure;
FIG. 3 is a schematic flow chart of writing data to an NVMe SSD disk according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a page buffer expansion module according to an embodiment of the present disclosure;
FIG. 5 is a flowchart of a method for accessing a file system disk according to an embodiment of the present disclosure;
FIG. 6 is a flow chart of a CMB address space release provided by an embodiment of the present disclosure;
FIG. 7 is a flowchart of another method for accessing a filesystem disk provided by an embodiment of the present disclosure;
FIG. 8 is a block diagram of an access device for a file system disk according to an embodiment of the present disclosure;
FIG. 9 is a block diagram of a VFS module provided by an embodiment of the present disclosure;
fig. 10 is a schematic hardware structure of an electronic device according to an embodiment of the disclosure;
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments.
First, technical fields and terms related to the present disclosure will be described.
The method is applied to the technical field of storage, and particularly relates to data storage based on an NVMe SSD solid state disk. Where NVMe (Nonvolatile Memory Express) is a communication protocol running on some interface for normalizing data transfer between a computer and a storage device. NVMe is used to connect the host and the memory subsystem. NVMe is specifically designed for non-volatile storage such as NAND, flash memory, etc., and the NVMe protocol is built on PCIe lanes.
Referring to fig. 1, a schematic diagram of a Linux file I/O system provided in this embodiment includes an application layer. A kernel layer (kernel) and a hardware layer, including at least one application in a user layer. The kernel layer includes a VFS (Virtual File System) module, a Page Cache (Page Cache), a block I/O module, and a Device Driver (Device Driver).
The Page Cache (abbreviated as ppcache) is understood to be a memory area managed by a Linux kernel, and a file can be read into a memory space through mmap and buffered I/O, generally into the Page Cache. In addition, page Cache is used to Cache Page data of a file, and buffer Cache is used to Cache block data of a block device (such as a disk). Pages are a logical concept, so Page Cache is peer to the file system; a block is a physical concept and therefore buffer cache is peer to the block device driver. Page Cache is one Page in size, typically 4K. When Linux reads and writes files, it is used to cache the logical content of the files, thereby speeding up the access to images and data on disk.
The mmap is a method for mapping files in a memory, namely mapping a file or other objects to an address space of a process, and realizing one-to-one mapping relation between file disk addresses and a section of virtual address in a process virtual address space.
In the Linux kernel, it can be further divided into three layers. The uppermost layer is a system call interface for implementing some basic functions, such as read and write. Below the system call interface is kernel code, which can be more precisely defined as architecture independent kernel code. These codes are common to all processor architectures supported by Linux. Below these codes are architecture-dependent codes that form part of what is commonly referred to as a BSP (Board Support Package, board level support package). These codes serve as processors and platform specific codes for a given architecture.
In the process of writing NVMe SSD disk data in the user layer, as shown in FIG. 2, two times of data copying are involved, namely, data is copied from a user-state application program to a kernel-state Page Cache (provided by a host RAM) at one time, and data is copied from the kernel-state Page Cache to a data solid state disk SSD of an NVMe controller in a Direct Memory Access (DMA) mode at one time. For the whole data transmission flow described above, since data is copied twice, the efficiency of the whole transmission is affected, and the consumption of the host-side CPU and RAM is increased.
In order to solve the above-mentioned problems, the present disclosure proposes an optimization scheme based on NVMe CMB (Controller Memory Buffer) characteristics. As shown in fig. 3, by expanding the management framework of the Linux kernel to the Page Cache, the NVMe driver can register the device memory such as CMB as the Page Cache, and can perform self-defined management, so that the data is prevented from being copied again, the efficiency of the whole transmission process is improved, and the consumption of the CPU and RAM on the host side is reduced.
Referring to fig. 4, a schematic diagram of a file system according to an embodiment of the present application is provided. The system comprises: user layer, kernel layer, and hardware layer. Wherein, the user layer comprises at least one user App; the kernel layer (kernel code of Linux kernel) includes: virtual File System (VFS) module, page cache expansion module, device Driver. The hardware layer comprises SSD and NVMe storage media.
In the embodiment of the application, a new page cache expansion module (Page Cache Extension module) and a page cache management module (Page Cache Management module) are added in the original kernel layer.
The Page Cache expansion module is configured to provide a Page Cache expansion interface, as shown in fig. 4, and provide a first interface and a second interface of a Page Cache. And the first interface and the second interface of the page buffer are used for communicating the VFS module and a Device Driver. It should be understood that the page buffer expansion module may further include a third interface, a fourth interface, and other interfaces, which is not limited in this embodiment.
The new Page adding Cache management module in the Device Driver is used for realizing the life cycle management of the Page Cache based on the NVMe CMB by using the expansion interfaces (the first interface and the second interface) provided by the Page Cache expansion module.
The embodiment provides a method for accessing a file system disk based on the structure frame shown in fig. 4, as shown in fig. 5, the method includes:
step S101: when the VFS module executes disk data writing operation through the first interface, the page buffer management module acquires a first buffer block in a storage medium of a hardware layer disk, wherein the first buffer block is an idle block.
Specifically, the application app sends a data write request to the kernel layer, and after the VFS module receives the write request, the VFS module invokes the first interface of the page buffer to perform a data write request operation to the lower layer.
In addition, the first interface is one of a plurality of interfaces established by the page buffer expansion module, such as a page buffer allocation interface (Page Cache Allocation).
The storage medium is a solid state disk SSD, in which a plurality of buffer resources may be divided in advance, and the first buffer block is a block of storage resources in the SSD, for example, as shown in fig. 4, and the first buffer block is a CMB buffer. It should be appreciated that a second buffer block, a third buffer block, etc. may also be included in the SSD. The page buffer and the page buffer management module are kernel layers (kernel) located above the SSD.
Step S102: the page buffer management module detects whether the physical address on the first buffer block completes the space address mapping.
Each buffer block corresponds to a physical address, and each physical address is required to establish a space mapping relation with a virtual address, and the steps are as follows: and judging whether the physical address on the current buffer block and a virtual address establish a space mapping relation.
Specifically, the first judgment method is as follows: judging whether a page table entry from the virtual address to the physical address exists, if so, indicating that space address mapping is completed; if not, the mapping is not complete.
And a judging method II: the kernel has a data structure for each NVMe device. A flag bit is added to the data structure to indicate whether the space address mapping is completed. For example, in the initialization case, the flag bit is set to 0, and when the page table mapping is established, the corresponding flag bit is set to 1. Further, whether the space address mapping is completed is determined by judging whether the flag bit is 0 or 1.
Step S103: if not, the page buffer expansion module is utilized to allocate a virtual address for the page buffer expansion module, and a space mapping relation between the virtual address and the physical address is established.
Specifically, the mapping relationship between the virtual address and the physical address is established, which can be referred to as a method step for establishing the mapping between the existing virtual address and the physical address, which is not limited in this embodiment.
Step S104: the page buffer management module converts the first buffer block into a first page buffer.
After the mapping relation is established, the Page buffer management module converts the first buffer block (such as CMB buffer) into Page buffer (Page buffer 1), which is equivalent to registering one buffer block in the hardware layer into Page buffer capable of writing data in the application layer.
Step S105: and the page buffer management module sends the space mapping relation and the virtual address to the VFS module through the page buffer expansion module.
After the VFS module receives the space mapping relation and the virtual address sent by the page buffer expansion module, the VFS module adds the first page buffer into an internal data structure, and directly copies the data to the first page buffer.
The step S102 further includes, if so, returning the virtual address to the VFS module, where the VFS module receives and copies the execution data to the first cache page.
Optionally, the Page buffer management module may be an NVMe CMB Cache module, and implement Page buffer life cycle management of the NVMe CMB based on a Page buffer expansion interface, that is, the Page buffer management module is used to manage driving of the NVMe. For the Page Cache expansion module, the following steps are mainly provided: page Cache Allocation (distribution interface) and Page Cache Deallocation (release interface), i.e. the aforementioned first and second interfaces.
Optionally, in a specific implementation manner of this embodiment, in the step S101, the method specifically includes: the page buffer memory management module obtains the first buffer memory block in an SSD in the controller based on a lock-free mechanism.
The lock-free mechanism is a lock-free Bitmap algorithm, the Bitmap algorithm refers to that a value corresponding to an element is marked by a bit, and a key is the element. Because bit is used as a unit to store data, the storage space can be greatly saved.
It should be noted that, in this embodiment, the buffer block is different from the buffer, and the buffer block is one resource block in the storage medium SSD, such as the first buffer block CMB buffer, and the storage medium SSD may include one or more buffer blocks, and the "buffer" mainly refers to a page buffer (page Cache), such as the first page buffer. The relation between the two is that the first buffer block CMB buffer is registered as the first page buffer after being converted (step S104 described above), thereby realizing direct writing of data on the first page buffer. In addition, in some embodiments, an area made up of multiple buffers of the storage medium SSD may also be referred to as a buffer.
Further, after the step S102, the method further includes: and if the space address mapping is completed, the space address mapping relation and the virtual address are sent to the VFS module through the page buffer expansion module. For example, the virtual address is returned to the VFS module, and after the VFS module receives the virtual address, the data writing operation is performed.
Optionally, in a specific implementation manner of this embodiment, the page buffer expansion module further includes a second interface of the page buffer, for example, the second interface is a release interface, and the second interface is connected to the VFS module and the driving module.
As shown in fig. 6, the method further includes: the page buffer memory management module obtains a first physical address, and a first mapping relation exists between the first physical address and the first virtual address. The page buffer management module detects whether the first physical address is the address of a buffer block in the storage medium; if yes, the buffer block is indicated to have completed space address mapping and registered as a page buffer, at this time, the buffer block corresponding to the first physical address is released based on a lock-free bitmap mechanism, the first mapping relation is unbinding, and a first message is sent to the VFS module.
The first message is used for indicating the VFS module to write back and release the page cache corresponding to the first virtual address. For example, the page buffer management module determines whether the address belongs to the CMB address space, and if so, releases the CMB buffer (i.e., the first buffer block) based on the lock-free bitmap mechanism. If not, i.e. not belonging to the CMB address space, it is stated that the address is not a buffer block in the SSD hardware disk, so the memory resource is freed in the usual way, i.e. the memory is freed by the buddy system.
According to the method provided by the embodiment, after data writing is completed, the CMB buffer block is reused by releasing the CMB address space, namely the first buffer block corresponding to the first page buffer, so that the beneficial effects of recovering the virtual address and the first buffer block CMB buffer are achieved.
In another embodiment, the present disclosure further provides a method for accessing a file system disk, where the method is applied to the VFS module in the kernel layer, and the method includes:
the first step: when receiving a disk data writing operation from a user layer, the VFS module inquires whether a first page buffer exists in a file data block corresponding to data to be written, wherein the first page buffer is generated by converting a first buffer block in a hardware layer disk SSD, and the first buffer block is an idle resource block in the SSD.
And a second step of: if so, the VFS module adds the first page cache to an internal data structure and copies the data onto the first page cache.
Specifically, the copying the data onto the first page cache includes: the VFS acquires a first virtual address, and a space address mapping relation exists between the first virtual address and the first physical address; and the VFS module copies the data to the first page cache according to the first virtual address, wherein the first virtual address is a virtual address distributed by a first interface in the page cache expansion module.
If the first page buffer memory does not exist, the VFS module copies the data to a second page buffer memory of the kernel layer, and then copies the data to a buffer block in the SSD for the second time through the second page buffer memory. Wherein the second page buffer is a page buffer in RAM.
Further, the method further comprises releasing the port resource and the memory resource.
In one embodiment, a method comprises: the VFS module receives a first message sent by the page buffer management module; the first message is used for indicating to release interface resources, the VFS module writes back the data, the first page buffer is released by calling a second interface, and the second interface is one of at least one interface in the page buffer expansion module.
In a specific embodiment, as shown in fig. 7, the method for accessing a file system disk includes:
step 1: the user App layer invokes a write file through a write system;
step 2: the VFS module inquires corresponding file data blocks to release the Page Cache;
step 3: if not, the page buffer expanding module judges whether registered memory allocation and release interfaces exist;
step 4: if yes, the VFS module calls a memory allocation interface provided by the Page Cache expansion module to allocate the Page Cache;
step 5: the VFS module converts the applied memory into a corresponding page and a virtual address, and returns to the VFS module.
Step 6: the VFS module adds the Page Cache into a self-maintained data structure (such as a linked list and a radix tree) for recording.
Step 7: and the VFS module copies the data to the Page Cache, so that the data is copied to the NVMe SSD once.
In addition, in the step 2, the method further includes: if the data does not exist, the VFS module copies the data to the page buffer memory in the kernel mode, and then copies the data to the SSD in the controller for the second time through the page buffer memory in the kernel mode.
Finally, when the Page Cache allocated based on the CMB buffer needs to be flushed down to the disk, the NVMe driver fills the physical address corresponding to the memory space into the SQE, the NVMe controller can identify whether the physical address belongs to the CMB area, if so, DMA copying is not performed, and at the moment, the NVMe controller directly writes the data in the CMB buffer into the SSD disk.
According to the embodiment, the management framework of the Linux kernel to the Page Cache is expanded, so that the NVMe driver can register the memory of the CMB device as the Page Cache and can perform self-defined management, direct data copy on the Page Cache is further realized, the secondary data copy is not needed, the efficiency of the whole transmission process is improved, and consumption of a CPU and a RAM on a host side is reduced.
The embodiment also provides an access device for a file system disk, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides an access device for a file system disk, as shown in fig. 8, including:
the obtaining module 801 is configured to obtain, when the VFS module performs a disk data write operation through a first interface, a first buffer block in a storage medium of a hardware layer disk, where the first buffer block is an idle block, and the first interface is one of at least one interface of the page buffer expansion module.
A detection module 802, configured to detect whether the physical address on the first buffer block completes space address mapping.
The processing module 803 is configured to, when detecting that the mapping of the spatial address is not complete, establish a spatial mapping relationship between the virtual address and the physical address by using the virtual address allocated to the page buffer expansion module, and convert the first buffer block into a first page buffer.
The sending module 804 is configured to send the spatial mapping relationship and the virtual address to the VFS module through the page buffer expansion module, where the first page buffer is used to enable the VFS module to add and copy data.
The VFS module adds the first page buffer to an internal data structure and copies data to the first page buffer.
In addition, each module is further used for realizing other method steps in the page buffer expansion module.
In a specific implementation manner of this embodiment, the obtaining module 801 is specifically configured to obtain the first cache block in a storage medium in the controller based on a lock-free mechanism.
In a specific implementation manner of this embodiment, the sending module 804 is further configured to send, in a case that space address mapping is completed, the space address mapping relationship and the virtual address to the VFS module through the page buffer expansion module.
In yet another implementation of this embodiment, the at least one interface includes a second interface, where the second interface is connected to the VFS module and the driving module.
The obtaining module 801 is further configured to obtain a first physical address, where a first mapping relationship exists between the first physical address and the first virtual address; a detection module 802, configured to detect whether the first physical address is a physical address of a buffer block in the storage medium; if so, the processing module 803 is used for releasing the buffer block corresponding to the first physical address based on a lock-free mechanism, unbinding the first mapping relation, and sending a first message to the VFS module by using the sending module 804, where the first message is used to instruct the VFS module to write back and release the page buffer corresponding to the first virtual address.
In another embodiment, the present disclosure further provides a VFS module, as shown in fig. 9, located in a kernel layer, the VFS module comprising:
the query module 901 is configured to query, when receiving a disk data writing operation from a user layer, whether a file data block corresponding to data to be written has a first page buffer, where the first page buffer is generated by converting a first buffer block in a storage medium of a disk of a hardware layer, and the first buffer block is an idle block.
A copy module 902, configured to add a first page buffer to an internal data structure and copy data onto the first page buffer.
In a specific implementation manner of this embodiment, the kernel layer further includes a page cache expansion module, where the page cache expansion module is connected to the VFS module through at least one interface.
The apparatus further comprises: the virtual address acquisition module is used for acquiring a first virtual address, and a space address mapping relation exists between the first virtual address and a first physical address; the copying module 902 is specifically configured to copy the data onto a first page cache according to the first virtual address.
In another specific implementation of this embodiment, the apparatus further includes: a receiving module and a releasing module.
The receiving module is used for receiving the first message sent by the page buffer management module; the release module is used for writing back the data and releasing the first page cache through the second interface. The second interface is one of at least one interface in the page buffer expansion module.
In another specific implementation manner of this embodiment, the copy module 902 is further configured to copy the data to a second page buffer of the kernel layer first and then copy the data to a buffer block in the storage medium through the second page buffer when the query module 901 detects that the first page buffer does not exist.
In addition, the query module 901 and the copy module 902 are further configured to implement other functions of the VFS module, which is not described in detail herein.
According to the device provided by the embodiment of the disclosure, the Page Cache expansion module and the Page Cache management module are additionally arranged in the kernel layer, and the existing Page Cache frame of the kernel layer is expanded, so that the Cache management mechanisms of most kernels are multiplexed, and no perception is achieved for users. In addition, the Page buffer management module establishes the mapping between the physical address and the virtual address of the buffer block in the hardware layer, so that the storage medium of the hardware layer is registered as the Page Cache, the direct data copy on the storage medium is realized, the data is prevented from being copied twice, the method reduces the primary data copy, improves the data transmission efficiency, and reduces the consumption of the CPU and the RAM.
It should be noted that, the access device and the VFS module of the file system disk in this embodiment are presented as functional units, where a unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and/or other devices that may provide the above functions.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The embodiment of the disclosure also provides an electronic device, which is provided with the access device of the file system disk shown in fig. 10.
Referring to fig. 10, a schematic structural diagram of an electronic device according to an alternative embodiment of the disclosure, as shown in fig. 10, the electronic device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 10.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created from the use of the electronic device of the presentation of one applet landing page, and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories. The electronic device also includes a communication interface 30 for the electronic device to communicate with other devices or communication networks.
The electronic device further comprises input means and output means (not shown in fig. 10). The processor 10, memory 20, input/output devices may be connected by a bus or other means. Alternatively, the input device may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointer stick, one or more mouse buttons, a trackball, a joystick, and the like. The output means may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. Such display devices include, but are not limited to, liquid crystal displays, light emitting diodes, displays and plasma displays. In some alternative implementations, the display device may be a touch screen.
The presently disclosed embodiments also provide a computer readable storage medium, and the methods described above according to the presently disclosed embodiments may be implemented in hardware, firmware, or as recordable storage medium, or as computer code downloaded over a network that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and is to be stored in a local storage medium, such that the methods described herein may be stored on such software processes on a storage medium using a general purpose computer, special purpose processor, or programmable or dedicated hardware.
The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present disclosure have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the disclosure, and such modifications and variations are within the scope defined by the appended claims.

Claims (12)

1. The method is characterized in that the method is applied to a kernel layer, the kernel layer comprises a page cache expansion module and a page cache management module, the page cache expansion module is connected with a Virtual File System (VFS) module and a driving module through at least one interface, the driving module is connected with a hardware layer, and the page cache management module is positioned in the driving module, and the method comprises the following steps:
the page buffer management module obtains a first buffer block in a storage medium of a hardware layer disk when the VFS module executes disk data writing operation through a first interface, wherein the first buffer block is an idle block, and the first interface is one of the interfaces;
the page buffer management module detects whether the physical address on the first buffer block is mapped with a space address;
if not, the page buffer expansion module is utilized to allocate a virtual address for the page buffer expansion module, and a space mapping relation between the virtual address and the physical address is established;
the page buffer memory management module converts the first buffer block into a first page buffer memory, and sends the space mapping relation and the virtual address to the VFS module through the page buffer memory expansion module, wherein the first page buffer memory is used for enabling the VFS module to add and copy data.
2. The method of claim 1, wherein the retrieving the first buffer block in the storage medium of the hardware layer disk comprises:
the page buffer management module obtains the first buffer block from a storage medium in the controller based on a lock-free mechanism.
3. The method according to claim 1, wherein the method further comprises:
and if the space address mapping is completed, the space address mapping relation and the virtual address are sent to the VFS module through the page buffer expansion module.
4. A method according to any one of claims 1-3, wherein the at least one interface includes a second interface, the second interface being coupled to the VFS module and the drive module, the method further comprising:
the page cache management module acquires a first physical address, and a first mapping relation exists between the first physical address and a first virtual address;
the page buffer management module detects whether the first physical address is a physical address of a buffer block in the storage medium;
and if so, releasing the buffer block corresponding to the first physical address based on a lock-free mechanism, unbinding the first mapping relation, and sending a first message to the VFS module, wherein the first message is used for indicating the VFS module to write back and release the page buffer corresponding to the first virtual address.
5. A method for accessing a file system disk, wherein the method is applied to a VFS module of a virtual file system, the VFS module being located in a kernel layer, the method comprising:
when receiving a disk data writing operation from a user layer, the VFS module inquires whether a first page buffer exists in a file data block corresponding to data to be written, wherein the first page buffer is generated by converting a first buffer block in a storage medium of a hardware layer disk, and the first buffer block is an idle block;
if so, the VFS module adds the first page cache to an internal data structure and copies data to the first page cache.
6. The method of claim 5, further comprising a page cache expansion module in the kernel layer, the page cache expansion module being coupled to the VFS module via at least one interface, the copying data onto the first page cache comprising:
the VFS module acquires a first virtual address, and a space address mapping relation exists between the first virtual address and a first physical address;
and the VFS module copies the data to the first page cache according to the first virtual address.
7. The method of claim 6, wherein the method further comprises:
the VFS module receives a first message sent by the page buffer management module;
and the VFS module writes back the data and releases the first page buffer through a second interface, wherein the second interface is one of at least one interface in the page buffer expansion module.
8. The method according to any one of claims 5-7, further comprising:
and if the first page cache does not exist, the VFS module copies the data to a second page cache of the kernel layer, and then copies the data to a buffer block in the storage medium through the second page cache for the second time.
9. An apparatus for accessing a file system disk, the apparatus comprising:
the system comprises an acquisition module, a page buffer expansion module and a page buffer expansion module, wherein the acquisition module is used for acquiring a first buffer block in a storage medium of a hardware layer disk when a Virtual File System (VFS) module executes disk data writing operation through a first interface, the first buffer block is an idle block, and the first interface is one of at least one interface of the page buffer expansion module;
the detection module is used for detecting whether the physical address on the first buffer block is subjected to space address mapping;
the processing module is used for establishing a space mapping relation between the virtual address and the physical address by utilizing the virtual address distributed by the page buffer expansion module when the incompletion is detected, and converting the first buffer block into a first page buffer;
and the sending module is used for sending the space mapping relation and the virtual address to the VFS module through the page buffer expansion module, and the first page buffer is used for enabling the VFS module to add and copy data.
10. An apparatus for accessing a file system disk, the apparatus comprising:
the system comprises a query module, a first page buffer and a second page buffer, wherein the query module is used for querying whether a first page buffer exists in a file data block corresponding to data to be written or not when receiving a disk data writing operation from a user layer, the first page buffer is generated by converting a first buffer block in a storage medium of a hardware layer disk, and the first buffer block is an idle block;
and the copying module is used for adding the first page cache into an internal data structure and copying data to the first page cache.
11. An electronic device, the electronic device comprising: the memory is connected with the processor;
the memory has stored therein computer instructions which, upon execution, cause the processor to perform the method of accessing a file system disk as claimed in any one of claims 1 to 4, or 5 to 8.
12. A computer-readable storage medium, wherein the computer-readable storage medium has stored thereon computer instructions,
the computer instructions for causing a computer to perform the method of accessing a file system disk according to any one of claims 1 to 4, or 5 to 8.
CN202310987553.XA 2023-08-07 2023-08-07 File system disk access method, device and equipment Pending CN117009258A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310987553.XA CN117009258A (en) 2023-08-07 2023-08-07 File system disk access method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310987553.XA CN117009258A (en) 2023-08-07 2023-08-07 File system disk access method, device and equipment

Publications (1)

Publication Number Publication Date
CN117009258A true CN117009258A (en) 2023-11-07

Family

ID=88572318

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310987553.XA Pending CN117009258A (en) 2023-08-07 2023-08-07 File system disk access method, device and equipment

Country Status (1)

Country Link
CN (1) CN117009258A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118886043A (en) * 2024-09-27 2024-11-01 苏州元脑智能科技有限公司 Data transmission method, device, equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118886043A (en) * 2024-09-27 2024-11-01 苏州元脑智能科技有限公司 Data transmission method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
US9213623B2 (en) Memory allocation with identification of requesting loadable kernel module
US7055015B2 (en) Information processing apparatus in which processes can reduce overhead of memory access and efficiently share memory
US11256445B2 (en) Virtual disk file format conversion method and apparatus
JP5347061B2 (en) Method and apparatus for storing data in a flash memory data storage device
JP5636034B2 (en) Mediation of mount times for data usage
KR101517258B1 (en) Apparatus, system, and method for cross-system proxy-based task offloading
CN107305534B (en) Method for simultaneously carrying out kernel mode access and user mode access
WO2015180598A1 (en) Method, apparatus and system for processing access information of storage device
CN102053869B (en) Resource allocation method and device for multi-operation system
US7996648B2 (en) Coupled symbiotic operating systems
JP2008541214A (en) Managing computer memory in a computing environment with dynamic logical partitioning
US11354233B2 (en) Method and system for facilitating fast crash recovery in a storage device
US20140040577A1 (en) Automatic Use of Large Pages
US11132291B2 (en) System and method of FPGA-executed flash translation layer in multiple solid state drives
CN111290827A (en) Data processing method and device and server
WO2018103022A1 (en) Method and apparatus for achieving frame buffering, electronic device and computer program product
CN111209080A (en) Transparent transmission method for graphic processor
KR102326280B1 (en) Method, apparatus, device and medium for processing data
CN117009258A (en) File system disk access method, device and equipment
KR20180041037A (en) Method for shared distributed memory management in multi-core solid state driver
US10831684B1 (en) Kernal driver extension system and method
US8593472B1 (en) System and method for accessing a frame buffer via a storage driver
US11422963B2 (en) System and method to handle uncompressible data with a compression accelerator
US20250028460A1 (en) Computational storage with configurable drives
CN118113455A (en) Memory access method and related device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination