CN118467214A - System and method for fault page handling - Google Patents

Abstract

A system and method for fault page handling. In some embodiments, the method comprises: querying a memory device for a fault page in the memory device; and receiving a response from the memory device identifying the failed page. Querying the memory device may include querying the memory device by a device driver; and querying the memory device may include querying the memory device using a complementary command of a cache coherency protocol.

Description

System and method for fault page handling

Technical Field

One or more aspects in accordance with embodiments of the present disclosure relate to computing systems, and more particularly to systems and methods for fault (fault) page processing.

Background

Memory devices connected to a host through an interface such as a computing high speed interconnect interface may from time to time experience failure of a memory cell. Thus, if a fault results in an uncorrectable error, the data stored in such a cell may be unreliable.

Aspects of the present disclosure are related to this general technical environment.

Disclosure of Invention

According to an embodiment of the present disclosure, there is provided a method comprising: querying a memory device for a fault page in the memory device; and receiving a response from the memory device identifying a failed page, wherein: querying the memory device includes querying the memory device by a device driver; and querying the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

In some embodiments, the cache coherency protocol includes an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

In some embodiments, the querying of the memory device includes: the memory device is queried in connection with device driver probing.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes performing a query of the memory device associated with a boot of the host.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes performing a query for a memory device associated with the connection of the memory device to a host.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

In some embodiments, a response from the memory device identifies a failing cache line; the method further comprises: and converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

In some embodiments, the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; the method further comprises: the first list and the second list are combined.

In some embodiments, a response from the memory device identifies a first failed cache line and a second failed cache line; the method further comprises: converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and converting the device physical address of the second failed cache line to a host physical address of the second failed cache line.

In some embodiments, a response from the memory device identifies a first failed cache line and a second failed cache line; the method further comprises: determining that the second failed cache line is in the same page as the first failed cache line; replacing the first failed cache line and the second failed cache line with the first failed cache line.

According to an embodiment of the present disclosure, there is provided a system including: a host, comprising: a processing circuit; and a memory connected to the processing circuit, the memory storing instructions that when executed by the processing circuit cause a method to be performed, the method comprising: querying a memory device for a fault page in the memory device; and receiving a response from the memory device identifying a failed page, wherein: querying the memory device includes querying the memory device by a device driver; and querying the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

In some embodiments, the cache coherency protocol includes an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

In some embodiments, the querying of the memory device includes: the memory device is queried in connection with device driver probing.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes performing a query of the memory device associated with a boot of the host.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes performing a query for a memory device associated with the connection of the memory device to a host.

In some embodiments, the device driver is configured to run in a host; the host is connected to the memory device; and querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

In some embodiments, a response from the memory device identifies a failing cache line; the method further comprises: and converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

In some embodiments, the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; the method further comprises: the first list and the second list are combined.

In some embodiments, a response from the memory device identifies a first failed cache line and a second failed cache line; the method further comprises: converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and converting the device physical address of the second failed cache line to a host physical address of the second failed cache line.

According to an embodiment of the present disclosure, there is provided a system including: a host, comprising: means for processing; and a memory connected to the means for processing, the memory storing instructions that when executed by the means for processing cause a method to be performed, the method comprising: querying a memory device for a fault page in the memory device; and receiving a response from the memory device identifying a failed page, wherein: querying the memory device includes querying the memory device by a device driver; and querying the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

Drawings

These and other features and advantages of the present disclosure will be appreciated and understood with reference to the specification, claims and appended drawings, wherein:

FIG. 1 is a system diagram according to an embodiment of the present disclosure;

FIG. 2A is a hybrid flowchart and block diagram illustrating soft disable (Soft disable) according to an embodiment of the present disclosure;

FIG. 2B is a hybrid flowchart and block diagram illustrating hard disable (hard disable) according to an embodiment of the present disclosure;

FIG. 2C is a diagram of a system reboot or insertion of a memory device, where the memory device is moved to a different slot, according to an embodiment of the disclosure;

FIG. 2D is a diagram of a system reboot or insertion of a memory device, where the memory device is not moved to a different slot, according to an embodiment of the disclosure;

FIG. 3 is a hybrid flowchart and block diagram illustrating at start-up page disabling in accordance with an embodiment of the present disclosure;

FIG. 4A is a flow chart illustrating the disabling of a page at start-up according to an embodiment of the present disclosure;

FIG. 4B is a diagram illustrating page disabling when inserting a memory device according to an embodiment of the present disclosure;

FIG. 5 is a hybrid flowchart and block diagram illustrating online updating of page disabling in accordance with an embodiment of the present disclosure;

FIG. 6 is a flow chart illustrating page disabling according to an embodiment of the present disclosure; and

Fig. 7 is a flow chart illustrating querying and page disabling of a device according to an embodiment of the present disclosure.

Detailed Description

The detailed description set forth below in connection with the appended drawings is intended as a description of exemplary embodiments of the systems and methods for fault page processing provided in accordance with the present disclosure and is not intended to represent the only forms in which the present disclosure may be constructed or utilized. The description sets forth the features of the present disclosure in connection with the illustrated embodiments. It is to be understood, however, that the same or equivalent functions and structures may be accomplished by different embodiments that are also intended to be encompassed within the scope of the disclosure. Like element numbers are intended to indicate like elements or features as shown elsewhere herein.

Expansion of memory capacity may be achieved by a memory device connected to the host system through a cache-coherence interface (e.g., through any suitable version of a computational high-speed interconnect (Compute Express Link ^TM) interface, which may include input/output protocols, protocols for accessing system memory, and protocols for accessing device memory, or any similar cache-coherence interface). Such memory devices may store primarily non-critical data, such as anonymous data, files, and removable data. With such memory devices, uncorrectable (poison) failed pages (e.g., memory pages having a sufficiently large number of bits such that the error correction code employed cannot correct them) may not be easily handled in a manner that reliably avoids applications using the memory device to access such pages. For example, an application may use a machine check exception log (MCElog), user level reliability, availability, serviceability (RAS) utility (availability) to detect a failed page. However, at system start-up, some time may elapse before MCELog starts up; during this delay interval, applications may access the failed page before it has been "offline (offlined)" or disabled (disabled) so that they are not available for access by kernel-level processes or user-level processes. More generally, user-level processes may not be ideally suited to address hardware failures. Further, at restart or when the memory device is disconnected and then reconnected without system power being turned off, the host may lack a mechanism for determining whether the memory device connected to the host is the same memory device as the previously connected memory device; this uncertainty may be an obstacle to reuse of the list information from poisoning of devices previously connected to the host.

Thus, in some embodiments, the device driver of the memory device may use the supplemental command to retrieve (retrieve) the list of poisoning from the memory device in several cases. As used herein, a "supplemental command" is a command that includes an argument (and may include additional arguments) that identifies a feature. The feature may be, for example, a report of a fault page. These conditions may include (i) startup of the host, (ii) connection of the memory device at system power-up, and (iii) the memory device reporting an error (which may or may not be triggered by detection of a previously undetected page of failure). In such embodiments, the device driver may be able to disable any failed page before any page is available to the application, and thus, there may not be a time interval for the application to be able to access the failed page.

FIG. 1 illustrates a portion of a computing system in some embodiments. The computing system includes a host 105, the host 105 including a Central Processing Unit (CPU) 110, a system memory 115 (which may be or include Dynamic Random Access Memory (DRAM)), and a persistent storage 120 (which may be a Solid State Drive (SSD)). The host 105 may also be connected to (or in some embodiments include) one or more secondary memories 125, which secondary memories 125 may be or include DRAMs, connected to the central processing unit 110 through an interface 130. For example, the interface may be a host peripheral component interconnect express (PCIe) root complex (root complex), and the secondary memory 125 may be configured as a peripheral component interconnect express (PCIe) endpoint. In such an embodiment, the communication protocol between the host 105 and the secondary storage 125 may be a cache coherence protocol, such as a computational high-speed interconnect (CXL) (in which case the supplemental command may be a mailbox command). The term used herein refers to secondary storage 125 as a component separate from host 105 and connected to host 105 (although it may share a housing with host 105).

System memory 115 may experience hardware failures of one or more memory cells from time to time. These faults may be detected as "errors" by circuitry of the system memory 115. In a Linux system, a user-level utility (e.g., machine check exception log (MCElog)) may use a data structure defined in a system management basic input/output system (BIOS) (SMBIOS), such as type 17 (defined in the system management basic input/output system), which defines various attributes of system memory, to provide reliability, availability, serviceability (RAS) support, and disable pages with errors. Such a system may prevent an application running on a host from accessing a memory page having data that is invalid due to an error by disabling the page with the error (which may be referred to as a failed page).

Disabling of pages may be performed as shown in fig. 2A and 2B. Fig. 2A illustrates soft disable, which may be performed when a correctable error is detected in a failed page 208 of system memory 115. To reduce the risk that such errors may become uncorrectable errors (the number of bit errors exceeds the number that can be corrected using error correction codes due to additional faults in the memory cells), after a correctable error is detected at 201, the error may be recorded in an error log at 202; at 203, data may be migrated to a different page 209 in system memory 115; and at 204, pages containing correctable errors may be disabled (made unavailable).

FIG. 2B illustrates a hard disable that may be performed when an uncorrectable error is detected in the fault page 208 of the system memory 115. After an uncorrectable error is detected at 211, the error may be recorded in an error log at 212. Then, at 213, the hardware may raise a bus error (SIGBUS) informing the Operating System (OS) that a process is attempting to access memory that is not physically addressable by the central processing unit 110, because the executing instruction includes an invalid address of the address bus. This may result in the termination (kill) of a process (owner) that owns the thread of which the instruction in question is a part. Pages containing uncorrectable errors may then be disabled (made unavailable) at 214.

Secondary memory 125 may also experience hardware failures of one or more memory units from time to time. These faults may be detected by circuitry of secondary storage 125. The above-described mechanism for handling a failed page in system memory 115 may not be readily adapted to handle a failed page in secondary memory 125 because, for example, data that allows the host to uniquely identify secondary memory 125 may not be available. For example, type 17 may not be suitable for secondary memory 125 (e.g., because type 17 is not available for PCIe devices).

As described above, when contact with secondary storage 125 is lost and reestablished, the host's inability to uniquely identify secondary storage 125 may be a barrier to the host's proper disabling of pages, such as in the event of a host reboot or secondary storage 125 disconnection and reconnection, while the system is powered on. For example, referring to FIG. 2C, secondary storage 125 may be a computing high speed interconnect device (CXL dev) that may be initially inserted into slot 0 of a host. In a Host Defined Memory (HDM), secondary memory 125 may be initially mapped to a first range of host memory addresses (ranging from 100 to 200), and the page at address 110 may be a failed page. The secondary memory 125 may then be pulled from slot 0 of the host and plugged back into slot 3 of the host. This may occur if the host is first shut down (before unplugging) and then started again (after plugging back), or it may occur if the host is not shut down. In either case (whether the host is off or not), the movement of secondary memory 125 from slot 0 to slot 3 may cause secondary memory 125 to become mapped to a different address range in host-defined memory (e.g., to addresses 500-600, as shown in FIG. 2C). This may have the following effect: the page at host-defined memory address 510 is a failed page, but a host lacking the way to determine that secondary memory 125 inserted into slot 3 was the same device previously inserted into slot 0 may lack sufficient information to infer whether the page at host-defined memory address 510 was a failed page, e.g., when secondary memory 125 is unplugged and reinserted, the correspondence (correspondence) between the device inserted into slot 3 and the device previously inserted into slot 0 may have been lost. Thus, when the secondary storage 125 is later inserted into a different slot, the host may not be able to properly disable the failed page after a restart or after unplugging and plugging back the secondary storage 125.

Referring to fig. 2D, a similar correspondence loss may occur if the host is restarted without unplugging the secondary storage 125, or if the secondary storage 125 is unplugged and then plugged back into the same slot (e.g., slot 0) without shutting down system power. Although in this case, the address in the host-defined memory (e.g., address 110) may be the same after the restart or after the unplug and plug-back, in each case the host may not be able to determine whether the auxiliary memory 125 inserted into slot 0 is the same device as the auxiliary memory 125 inserted into slot 0 after the restart or after the unplug and plug-back. Thus, even when the secondary storage 125 is later inserted into the same slot, the host may not be able to properly disable the failed page after a restart or after unplugging and plugging back the secondary storage 125.

Thus, in some embodiments, the host 105 may query the secondary storage 125 for a failed page and disable the page accordingly. Where the secondary storage 125 is a computing high-speed interconnect device, the query may be performed using supplemental commands, such as using a get_event_records mailbox command, using a get_poison_list mailbox command, or using a scan_media mailbox command (in a computing high-speed interconnect system). In some embodiments, these operations are performed by a device driver for secondary storage 125. Each such command may cause secondary storage 125 to return a list of failure addresses to the host, which may in response allow the host to disable the corresponding failure page.

Fig. 3 shows how this may be achieved during a device driver probing phase, for example, at host startup, or when secondary storage 125 is first connected to host 105 (e.g., at power-up of host 105). For example, as shown in fig. 3, at 301, device driver 320 may obtain device Hardware (HW) information from auxiliary memory 125 using specified vendor-specific extension capabilities (DESIGNATED VENDOR-specific extended capabilities, DVSEC), and at 302, obtain a host-defined memory address range (e.g., host physical address (host PHYSICAL ADDRESS, HPA) range) corresponding to the memory address range of auxiliary memory 125. The device driver 320 may generate a supplemental (e.g., mailbox (MB)) command at 303 and send the command to the firmware 340 of the secondary memory 125 using an appropriate function call to the operating system kernel 325 at 311. The kernel 325 may use data stored in a peripheral component interconnect express (PCIe) block 335 of a basic input/output system (BIOS) 330 of the host 105 to arrange for directing supplemental commands to the secondary memory 125.

In response to receiving the supplemental command (e.g., the get_event_records mailbox command, the get_potential_list mailbox command, or the scan_media mailbox command), the firmware 340 of the secondary memory 125 may read a Physical Address (PA) list from its internal metadata store at 312. This list may be referred to as a Fault Page List (FPL), or as an error page list, or as a poisoned list. The firmware 340 of the secondary storage 125 may then generate and return a supplemental command (e.g., mailbox (MB)) output (e.g., fault Page List (FPL)) to the device driver 320. The device driver 320 may perform address translation and various other processing operations (discussed in further detail below) on the list of fault pages stored in the output buffer of the device driver 320 and generate a fault page list structure (structure FPL) that is translated to host physical addresses and is suitable for disabling the fault pages.

Hereinafter, with reference to fig. 1 and 3, a detailed operation method of the host 105 and the auxiliary memory 125 will be described through fig. 4A to 7.

FIG. 4A illustrates a method for disabling a failed page at startup in some embodiments. At 402, the host 105 is started. At 404, the operating system of host 105 performs enumeration (enumeration) of peripheral component interconnect express devices, including computing high speed interconnect devices, and performs device setup. At 406, the host 105 performs Physical Address (PA) setup and sets a host-defined memory range (which may be stored in a table E820 of the host 105, i.e., a physical address mapping table). At 408, the device driver 320 may perform device driver probing and may retrieve device Hardware (HW) information from the secondary storage 125 using the specified vendor-specific extension capabilities (DVSEC). In this operation, the device driver 320 may also obtain the list of failed pages from the secondary memory 125 using the supplemental command and disable the failed pages, as described above. At 410, the operating system (410) may complete the boot process and the load and store operations to access secondary storage 125 may be performed by an application running on host 105. These applications may not be able to Inadvertently (INADVERTENTLY) access a poisoned page (e.g., a failed page) (as shown by the "X" in fig. 4A) because the failed page has been disabled. At 412, a daemon (daemons) such as MCELog may be initialized (e.g., an initialization (init) daemon may be running).

FIG. 4B illustrates a method for disabling a failed page when auxiliary memory 125 is inserted in some embodiments. At 420, the operating system is running (has previously been started), and at 422, secondary storage 125 may be connected to host 105 without shutting down system power. At 424, the host 105 performs Physical Address (PA) setting and sets the host-defined memory range. At 426, the device driver 320 may perform device driver probing and may retrieve device Hardware (HW) information from the secondary storage 125 using the specified vendor-specific extension capabilities (DVSEC). In this operation, the device driver 320 may also obtain the list of failed pages from the secondary memory 125 using the supplemental command and disable the failed pages, as described above. At 428, the secondary storage 125 may be ready for use, and the load and store operations to access the secondary storage 125 may be performed by an application running on the host 105. These applications may not be able to inadvertently access a poisoned page (e.g., a failed page) (as indicated by an "X" in fig. 4B) because the failed page has been disabled.

Fig. 5 shows a process of online updating of a list of poisons (or a list of failure pages) by the device driver 320. As used herein, an "online" update refers to an update that is not performed in response to a startup of the host 105 or an insertion of the secondary storage 125 without powering down the system. In the process of FIG. 5, load and store operations may occur at 501 when an application reads data from secondary storage 125 and writes data to secondary storage 125. These load and store operations may be performed by Root Complex (RC) 520 of host 105. In performing such load and store operations, secondary storage 125 may detect a poisoned error (e.g., an uncorrectable error in the memory) at 502. At 503, secondary storage 125 may, in response to detecting an uncorrectable error in the storage, establish a (make) entry in an event record of secondary storage 125, the entry including one or more addresses in the storage where one or more uncorrectable errors were detected. At 504, secondary storage 125 may send a Transaction layer packet (Transaction LAYER PACKET, TLP) to Root Complex (RC) 520 of host 105 informing the host that a failed page has been detected and (e.g., by an application running on the host) that a memory access involving a memory location with an uncorrectable error has been made. The transaction layer packet may be received by the root complex 520 of the host 105 and the root complex 520 of the host 105 may generate an exception at 505.

A machine check exception (MACHINE CHECK exception, MCE) process 525 (which may be an operating system kernel process) receives the exception and, at 506, the machine check exception process 525 may terminate the process involved in the access of the failed page. The process may be a user-level process (e.g., in some embodiments, secondary storage 125 may be allocated by the operating system only to user-level processes, and not to kernel-level processes). The termination of a process may help reduce the risk that any erroneous data received by the process due to an error in secondary storage 125 may cause additional errors (e.g., invalid data is written to persistent storage by a process involved in the access of the failed page).

The process for disabling the fault page may proceed as follows. The device driver 320 may periodically check the state of the machine check exception process 525 and thus, when an exception is generated by the root complex 520 of the host 105, it may determine that a new (previously undetected) uncorrectable memory error may have been detected in the secondary memory 125 (other conditions may cause the root complex 520 of the host 105 to generate an exception; thus, the generation of an exception may implicitly indicate that an uncorrectable memory error has been detected in the secondary memory 125). In response to making such a determination, the device driver 320 may send a supplemental command (e.g., a get_event_records mailbox command, a scan_media mailbox command, or (as shown) a get_poison_list mailbox command) at 508 to (i) request a timestamp of the last uncorrectable error detected and the total number of entries in the list of poisons of the secondary storage 125, and (ii) if the timestamp is more recent than the timestamp received at the last occasion that such a request was made by the device driver 320, the device driver 320 may request the entire list of poisons from the secondary storage 125 at 509. Upon receipt of the request, the secondary storage 125 may return (i) the entire poisoned list, or (ii) the first portion of the poisoned list if the poisoned list is too long to return (at once) (e.g., too long to fit into the output buffer of the secondary storage 125) at 510. The return of the poisoned list may be performed synchronously (e.g., without queuing the response in secondary storage 125). At 511, the device driver 320 may check whether the number of received poisoned list entries is equal to the total number of entries in the poisoned list of the secondary memory 125; if not, the device driver 320 may acquire another portion of the poisoned list, repeatedly until the number of received poisoned list entries equals the total number of entries in the poisoned list of the secondary memory 125. The device driver 320 may then perform disabling of the failed page at 512.

FIG. 6 illustrates a method for disabling pages in some embodiments. At 602, a supplemental command (e.g., mailbox command) credential check is performed. The credential check may involve determining whether secondary storage 125 supports a scan media mailbox command or a get poisson list command. Then, at 604, the device driver 320 may retrieve the list of poisons or a first portion of the list of poisons (e.g., if the list of poisons is too long to fit in the output buffer of the secondary memory 125). The list of poisoning may be a list of cache line aligned Device Physical Addresses (DPAs). At 606, the device driver 320 may translate the Device Physical Address (DPA) (which may each be the address of the failed cache line) of each poisoned list entry to a Host Physical Address (HPA). To perform this address translation, the device driver 320 may use memory mapped information obtained by the device driver 320 using the specified vendor-specific extension capabilities (DVSEC). The driver may populate a Fault Page List (FPL) (or error page list) data structure with host physical addresses obtained as a result of the translation. At 608, entries from adjacent poisoned lists (e.g., successively fetched (fetched) portions of the poisoned list from secondary storage 125, each portion may be a list of failed cache lines) may be combined (e.g., consolidated) into a single list, and each entry may be aligned with a host physical address page boundary. At 610, copies within any page (which may be the same after the page alignment performed at 608) may be removed (e.g., when two failed cache lines are in the same page, the copy set may be replaced with one of the copy sets), and at 612, the failed page list structure may be updated. The structure may include a count of the number of failed pages, an address of each failed page, and metadata (e.g., a timestamp of each failed page).

At 614, a list flag of poisoning (which indicates whether the list of poisoning maintained by secondary storage 125 is complete) may be checked. The list of poisoning may be incomplete, for example, if it is too long to fit into the buffer of secondary storage 125 for storing the list of poisoning. If this is the case, the list of poisoning may be unreliable; the device driver 320 may then obtain a timestamp of the buffer overflow (overflow) event and report to the operating system that the secondary storage 125 is unreliable. If the poisoned list flag is set (e.g., if no overflow has occurred), then (i) if there are no more poisoned list entries to be retrieved from secondary storage 125, then the device driver may perform disabling of all fault pages at 616, and (ii) if there are more poisoned list entries to be retrieved from secondary storage 125, then the device driver may begin the process of retrieving and processing poisoned list entries repeatedly at 604.

Fig. 7 is a flow chart in some embodiments. The method includes querying the memory device for a fault page in the memory device at 705; at 710, receiving a response from the memory device identifying a failed page; and disabling the fault page at 715. Querying the memory device may include querying the memory device by a device driver; and querying the memory device may include querying the memory device using a complementary command of the cache coherency protocol.

As used herein, "a portion" of something means "at least some" of the thing, and thus may mean less than all or all of the thing. Thus, the "part of the thing" includes the whole thing as a special case, that is, an example in which the whole thing is a part of the thing. As used herein, when the second number is "within Y" of the first number X, it means that the second number is at least X-Y and the second number is at most x+y. As used herein, when the second amount is "within Y% of the first amount, this means that the second amount is at least (1-Y/100) times the first amount, and the second amount is at most (1+Y/100) times the first amount. As used herein, the term "or" should be interpreted as "and/or" such that, for example, "a or B" means "a" or "B" or any one of "a and B".

The background art provided in the background section of this disclosure is only included to set context and the contents of this section are not admitted to be prior art. Any component or any combination of components described (e.g., in any system diagram included herein) may be used to perform one or more operations of any flowchart included herein. Further, (i) the operations are example operations and may involve various additional steps not explicitly covered, and (ii) the temporal order of the operations may be changed.

Each of the terms "processing circuitry" and "means for processing" is used herein to represent any combination of hardware, firmware, and software for processing data or digital signals. The processing circuit hardware may include, for example, application Specific Integrated Circuits (ASICs), general purpose or special purpose Central Processing Units (CPUs), digital Signal Processors (DSPs), graphics Processing Units (GPUs), and programmable logic devices such as Field Programmable Gate Arrays (FPGAs). In processing circuitry, as used herein, each function is performed by hardware configured to perform the function (i.e., hardwired), or by more general purpose hardware (such as a CPU) configured to execute instructions stored in a non-transitory storage medium. The processing circuitry may be fabricated on a single Printed Circuit Board (PCB) or distributed over several interconnected PCBs. The processing circuitry may comprise other processing circuitry; for example, the processing circuitry may include two processing circuits interconnected on a PCB: FPGA and CPU.

As used herein, when a method (e.g., adjustment) or a first quantity (e.g., a first variable) is referred to as being "based on" a second quantity (e.g., a second variable), it means that the second quantity is an input to the method or affects the first quantity, e.g., the second quantity may be an input (e.g., a unique input or one of several inputs) that is a function of calculating the first quantity, or the first quantity may be equal to the second quantity, or the first quantity may be the same as the second quantity (e.g., stored in memory at the same location or locations as the second quantity).

It will be understood that, although the terms "first," "second," "third," etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed herein could be termed a second element, component, region, layer or section without departing from the spirit and scope of the present inventive concept.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the inventive concepts. As used herein, the terms "substantially," "about," and the like are used as approximate terms, rather than degree terms, and are intended to account for inherent deviations in measured or calculated values that would be recognized by one of ordinary skill in the art.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. An expression such as "at least one of … …" modifies the entire list of elements before it, rather than modifying individual elements in the list. Furthermore, the use of "may" when describing embodiments of the inventive concepts refers to "one or more embodiments of the present disclosure. Furthermore, the term "exemplary" is intended to refer to an example or illustration. As used herein, the terms "use", "using" and "used" may be considered synonymous with the terms "utilization (utilize)", "utilizing" and "utilizing (utilized)", respectively.

It will be understood that when an element or layer is referred to as being "on," "connected to," "coupled to" or "adjacent to" another element or layer, it can be directly on, connected to, coupled to or adjacent to the other element or layer, or one or more intervening elements or layers may be present. In contrast, when an element or layer is referred to as being "directly on," "directly connected to," "directly coupled to," or "directly adjacent to" another element or layer, there are no intervening elements or layers present.

Some embodiments may include the features set forth in the following numbers.

1. A method, comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

querying the memory device includes querying the memory device by a device driver;

And

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

2. The method of statement 1, wherein the cache coherence protocol includes an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

3. The method of statement 1 or statement 2, wherein querying the memory device comprises: the memory device is queried in connection with device driver probing.

4. The method of any of the preceding statements, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes executing a query for a memory device associated with a boot of the host.

5. The method of any of the preceding statements, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

The querying of the memory device includes performing a query of the memory device related to the connection of the memory device to the host.

6. The method of any of the preceding statements, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

7. The method of any of the preceding statements, wherein:

a response from the memory device identifies a failed cache line; and

The method further comprises the steps of:

And converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

8. The method of any of the preceding statements, wherein:

the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; and

The method further comprises the steps of:

the first list and the second list are combined.

9. The method of any of the preceding statements, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and

The device physical address of the second failed cache line is translated to a host physical address of the second failed cache line.

10. The method of any of the preceding statements, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

Determining that the second failed cache line is in the same page as the first failed cache line; and

The first and second failed cache lines are replaced with the first failed cache line.

11. A system, comprising:

a host, comprising:

A processing circuit; and

A memory connected to the processing circuit, the memory storing instructions that when executed by the processing circuit cause a method to be performed, the method comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

Querying the memory device includes querying the memory device by a device driver; and

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

12. The system of statement 11, wherein the cache coherence protocol includes an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

13. The system of statement 11 or statement 12, wherein the query to the memory device comprises: the memory device is queried in connection with device driver probing.

14. The system of any one of statements 11-13, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes executing a query for a memory device associated with a boot of the host.

15. The system of any one of statement 11-14, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

The querying of the memory device includes performing a query of the memory device related to the connection of the memory device to the host.

16. The system of any one of statement 11-15, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

17. The system of any one of statements 11-16, wherein:

a response from the memory device identifies a failed cache line; and

The method further comprises the steps of:

And converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

18. The system of any one of statement 11-17, wherein:

the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; and

The method further comprises the steps of:

The first list and the second list are combined.

19. The system of any one of statements 11-18, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and

The device physical address of the second failed cache line is translated to a host physical address of the second failed cache line.

20. A system, comprising:

a host, comprising:

Means for processing; and

A memory connected to the means for processing, the memory storing instructions that when executed by the means for processing cause a method to be performed, the method comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

Querying the memory device includes querying the memory device by a device driver; and

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

Although exemplary embodiments of systems and methods for fault page handling have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Thus, it should be understood that systems and methods for fault page handling constructed in accordance with the principles of the present disclosure may be embodied in other ways than specifically described herein. The invention is also defined in the appended claims and equivalents thereof.

Claims (20)

1. A method for fault page handling, comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

Querying the memory device includes querying the memory device by a device driver; and

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

2. The method of claim 1, wherein the cache coherence protocol comprises an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

3. The method of claim 1, wherein querying the memory device comprises: the memory device is queried in connection with device driver probing.

4. The method according to claim 1, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes executing a query for a memory device associated with a boot of the host.

5. The method according to claim 1, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

The querying of the memory device includes performing a query of the memory device related to the connection of the memory device to the host.

6. The method according to claim 1, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

7. The method according to claim 1, wherein:

a response from the memory device identifies a failed cache line; and

The method further comprises the steps of:

And converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

8. The method according to claim 1, wherein:

the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; and

The method further comprises the steps of:

The first list and the second list are combined.

9. The method according to claim 1, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and

The device physical address of the second failed cache line is translated to a host physical address of the second failed cache line.

10. The method according to claim 1, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

Determining that the second failed cache line is in the same page as the first failed cache line; and

The first and second failed cache lines are replaced with the first failed cache line.

11. A system for fault page handling, comprising:

a host, comprising:

A processing circuit; and

A memory connected to the processing circuit, the memory storing instructions that when executed by the processing circuit cause a method to be performed, the method comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

Querying the memory device includes querying the memory device by a device driver; and

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.

12. The system of claim 11, wherein the cache coherence protocol comprises an input/output protocol, a protocol for accessing system memory, and a protocol for accessing device memory.

13. The system of claim 11, wherein the query to the memory device comprises: the memory device is queried in connection with device driver probing.

14. The system of claim 11, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes executing a query for a memory device associated with a boot of the host.

15. The system of claim 11, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

The querying of the memory device includes performing a query of the memory device related to the connection of the memory device to the host.

16. The system of claim 11, wherein:

the device driver is configured to run in a host;

the host is connected to the memory device; and

Querying the memory device includes querying the memory device based on a message sent to the host, the message informing the host to detect a failed page.

17. The system of claim 11, wherein:

a response from the memory device identifies a failed cache line; and

The method further comprises the steps of:

And converting the equipment physical address of the fault cache line into the host physical address of the fault cache line.

18. The system of claim 11, wherein:

the response from the memory device includes a first list of failed cache lines and a second list of failed cache lines; and

The method further comprises the steps of:

The first list and the second list are combined.

19. The system of claim 11, wherein:

a response from the memory device identifies a first failed cache line and a second failed cache line; and

The method further comprises the steps of:

converting the device physical address of the first failed cache line to a host physical address of the first failed cache line; and

The device physical address of the second failed cache line is translated to a host physical address of the second failed cache line.

20. A system for fault page handling, comprising:

a host, comprising:

Means for processing; and

A memory connected to the means for processing, the memory storing instructions that when executed by the means for processing cause a method to be performed, the method comprising:

Querying a memory device for a fault page in the memory device; and

A response is received from the memory device identifying a failed page,

Wherein:

Querying the memory device includes querying the memory device by a device driver; and

The querying of the memory device includes querying the memory device using a complementary command of a cache coherency protocol.