KR101689420B1 - Apparatus, system, and method for power reduction management in a storage device - Google Patents

Abstract

An apparatus, system, and method for power loss management of a non-volatile data storage device (102) are disclosed. The monitor module 510 initiates a power loss mode in the non-volatile data storage device 102 in response to the first power supply 130 failing to supply power to the non-volatile data storage device 102 higher than a predetermined threshold. The second power supply 124 provides power to the non-volatile data storage device 102 during at least the power hold time during the power loss mode. The power loss module 520 regulates the execution of operations within the process for the non-volatile data storage device 102 during the power loss mode so that operations within the essential processes are performed within the power maintenance time.

Description

[0001] APPARATUS, SYSTEM, AND METHOD FOR POWER REDUCTION MANAGEMENT IN A STORAGE DEVICE [0002]

The present invention relates to efficient power usage during power outages, and more particularly to preventing data loss of storage devices during power outages or dramatic power reduction.

Power usage is very important for computing devices in many different situations. In one situation, power is important when the external power supply is interrupted. For example, solid state storage devices (SSD) such as flash memory are non-volatile storage devices. Thus, the devices recorded in the SSD assume that the data, once written, is stored forever. However, in certain SSD devices, after receiving data, the SSD maintains data in volatile memory (such as DRAM, SRAM, registers, buffers, or the like) to perform multiple operations on the data . In the case of a power failure, the data held in the volatile memory may not be preserved.

U.S. Provisional Application No. 61 / 240,991, U.S. Provisional Application No. 61 / 245,622, U.S. Provisional Application No. 61 / 368,564

From the foregoing discussion it should be clear that there is a need for an apparatus, system and method for managing power reduction in a data storage device. Advantageously, such an apparatus, system, and method will store the verified data in a non-volatile data storage device, even at power interruption.

The present invention has been developed in response to the present state of the art, in particular in response to the problems and needs of the art which have not yet been completely solved by currently available storage devices. Accordingly, the present invention has been developed to provide a device, system, and method for power reduction management that overcomes many or all of the deficiencies of the art discussed above.

The method of the present invention is provided for power loss management. The method in the disclosed embodiments substantially comprises the steps necessary to carry out the functions provided below with respect to the operation of the apparatus and system described. In one embodiment, the method includes initiating a power loss mode to a non-volatile data storage device in response to a first power source that fails to provide power to the non-volatile data storage device beyond a predetermined threshold. The method includes, in another embodiment, receiving power from the second power source to the non-volatile data storage device during at least the power hold time during the power loss mode. In yet another embodiment, the method includes adjusting the execution of operations in the process to a non-volatile data storage device during a power loss mode such that operations within an essential process run within the power maintenance time.

In one embodiment, the method adjusts execution of operations in the process by removing one or more non-essential operations from operations within the process for the non-volatile data storage device. One or more non-essential operations, in one embodiment, include write operations, erase operations, and read operations that are not known to one or more clients of the non-volatile data storage device. Operations within an essential process, in one embodiment, include write operations known to one or more clients of the non-volatile data storage device. The method includes, in one embodiment, powering off the non-volatile data storage device without performing one or more non-essential operations.

The method, in another embodiment, controls the execution of operations within the process by omitting one or more non-essential operations in the one or more instruction queues such that one or more non-essential operations are not performed on the non-volatile data storage device. In another embodiment, the method adjusts the execution of operations within the process by reordering the selection of operations within the process to perform operations within the process that are essential within the power maintenance time.

In one embodiment, the method adjusts the performance of operations within the process by resetting one or more non-volatile memory elements of the non-volatile data storage device. The method includes, in another embodiment, performing operations within a process that are essential for one or more non-volatile memory elements in response to resetting one or more non-volatile memory regions. In one embodiment, the one or more non-volatile memory elements comprise substantially all non-volatile memory elements of the non-volatile data storage device. In another embodiment, the one or more non-volatile memory elements comprise one or more banks of non-volatile memory elements that handle pending non-essential operations from operations within the process. Each bank, in one embodiment, comprises a plurality of nonvolatile memory dies. Non-essential operations that are currently being processed are, in one embodiment, erase operations.

In one embodiment, the method further comprises deactivating operation of one or more sub-elements of the non-volatile data storage device in response to initiating the power loss mode. Deactivating the operation of one or more child elements, in one embodiment, conserves the energy of the second power supply to provide a power hold time. The method, in another embodiment, controls the execution of operations in the process by flushing buffered incomplete data associated with operations within a required process for a non-volatile memory of the non-volatile data storage device, Lt; / RTI > stores the buffered incomplete data within the power hold time.

The method includes, in one embodiment, storing, for a non-volatile data storage device, an identifier from which the non-volatile data storage device initiates a power loss mode. In another embodiment, the method includes storing, for a non-volatile data storage device, an identifier to identify that operations in an essential process complete execution within a power hold time. The identifier, in one embodiment, is stored for retrieval from a non-volatile data storage device in response to restoring power to the non-volatile data storage device. The second power supply comprises, in one embodiment, one or more capacitors capable of storing sufficient energy to provide a power hold time to perform operations within an essential process.

An apparatus for power loss management is provided in a plurality of modules configured to functionally perform the necessary steps to manage power loss in a storage device. Such modules of the described embodiments include a monitor module, a second power supply, and a completion module.

The monitor module, in one embodiment, initiates a power loss mode in the non-volatile data storage device in response to a first power source that fails to provide power to the non-volatile data storage device beyond a predetermined threshold. The second power supply, in one embodiment, powers the non-volatile data storage device during at least the power hold time during the power loss mode. The completion module, in one embodiment, controls the execution of operations within the process for the non-volatile data storage device during the power loss mode, such that operations within the essential processes are performed within the power maintenance time. In another embodiment, completion models are configured to perform operations in a process by resetting one or more non-volatile memory elements of a non-volatile data storage device that now processes non-essential operations from operations within the process to remove non- .

The system of the present invention is also provided for managing power loss in a storage device. The system may be implemented with an adapter, a first power source, at least one monitor module, at least one second power source, and one or more completion modules. In particular, the system comprises, in one embodiment, two or more non-volatile data storage devices.

The adapter, in one embodiment, is configured to accommodate two or more non-volatile data storage devices. The first power source, in one embodiment, is electrically coupled to the adapter to power two or more non-volatile data storage devices.

At least one monitor module, in one embodiment, initiates a power loss mode in non-volatile data storage devices in response to a first power source that fails to provide power above a predetermined threshold. At least one second power source, in one embodiment, is electrically coupled to the adapter. Each second power supply, in another embodiment, is configured to power at least one of the non-volatile data storage devices during at least the power hold time during the power loss mode.

The one or more completion modules, in one embodiment, regulate the execution of operations within the process for non-volatile data storage devices during the power loss mode, such that operations within the essential processes are performed within the power maintenance time. The one or more completion modules, in one embodiment, include a separate completion module for each non-volatile storage device, and each separate completion module controls the execution of operations within the process for the associated non-volatile data storage device. In another embodiment, the one or more completion modules are a single system completion module that coordinates the execution of operations within the process for two or more non-volatile data storage devices.

Reference to features, advantages, or similar language throughout this specification does not imply that all features and advantages that may be realized with the present invention should be in any single embodiment of the present invention. Rather, the language (s) referring to features and advantages are understood to mean that a particular feature, advantage, or characteristic described in connection with one embodiment is included in at least one embodiment of the invention. Accordingly, the discussion of features and advantages, and similar language, throughout this specification, while not necessarily required, refers to the same embodiment.

In addition, the described features, advantages, and characteristics of the present invention may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the present invention may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages that can not be provided in all embodiments of the invention may be recognized in specific embodiments.

These features and advantages of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as described hereinafter.

According to the present invention, an apparatus, system and method for managing the reduction of power in a data storage device are provided.

For a better understanding of the advantages of the present invention, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, here:
1 is a schematic block diagram illustrating one embodiment of a system including a storage device having a power management device that enables improved data handling in case of unexpected power failure;
2 is a schematic block diagram illustrating one embodiment of a solid state storage controller for a data storage device;
3 is a schematic block diagram illustrating one embodiment of a solid state storage controller having a write data pipeline and a read data pipeline in a data storage device;
4 is a schematic block diagram illustrating one embodiment of a bank interleave controller in a solid state storage controller;
5A is a schematic block diagram illustrating an embodiment of a power management apparatus;
Figure 5b is an example of a timeline associated with data corruption;
Figure 6 is a schematic block diagram illustrating one embodiment of a non-volatile memory in which the power management device and the power management device interact;
Figure 7 is a schematic block diagram illustrating one embodiment of a write data pipeline;
8 is a schematic flow diagram illustrating one embodiment of a method for improved data handling in the event of an unexpected outage; And
Figure 9 is a schematic flow diagram illustrating another embodiment of a method for improved data handling in the event of an unexpected outage.

Reference to features, advantages, or similar language throughout this specification does not imply that all features and advantages that may be realized with the present invention should be in any single embodiment of the present invention. Rather, the language (s) referring to features and advantages are understood to mean that a particular feature, advantage, or characteristic described in connection with one embodiment is included in at least one embodiment of the invention. Accordingly, the discussion of features and advantages, and similar language, throughout this specification, while not necessarily required, refers to the same embodiment.

In addition, the described features, advantages, and characteristics of the present invention may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the present invention may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages that can not be provided in all embodiments of the invention may be recognized in specific embodiments. These features and advantages of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as described hereinafter.

Many of the functional units described herein have been classified as modules in order to more specifically emphasize their performance independence. For example, the module may be implemented as a hardware circuit comprising custom VLSI circuits or ready-made semiconductors, such as gate arrays, logic chips, transistors, or other discrete components. The module may also be implemented as programmable hardware devices, such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like.

The modules may also be implemented in software for execution by various types of processors. For example, the executable code of the identified module may include one or more physical or logical blocks of computer instructions that may be organized, for example, as an object, procedure, or function. Nonetheless, the executables of the identified modules need not be physically co-located, but may include different instructions that, when logically linked together, contain the module and are stored in different locations to achieve the intended purpose for the module have.

In fact, the executable code of a module may be a single instruction, or multiple instructions, distributed evenly over several different code segments, between different programs, and across several memory devices. Likewise, operational data can be identified and shown therein within modules, and can be implemented in any suitable form and organized in any suitable type of data structure. The operational data may be collected into a single data set, distributed over different locations including over different storage devices, and may exist, at least in part, merely as electronic signals to the system or network. Where a module or portion of a module is executed in software, the software portion is stored in one or more computer readable media.

Reference throughout this specification to "one embodiment, " an embodiment, or similar language, means that a particular feature, structure, or characteristic described in connection with the embodiment is at least one ≪ / RTI > Accordingly, the appearances of the phrase "one embodiment, " an embodiment, or similar language throughout this specification are not necessarily required, but all refer to the same embodiment.

Reference to a computer-readable medium may take any form capable of executing stored machine-readable instructions for a digital processing device. The computer-readable medium may be embodied by a compact disk, a digital-video disk, a magnetic disk, a Bernoulli drive, a magnetic disk, a punch card, a flash memory, an integrated circuit, or other digital processing device memory device.

Moreover, the described features, structures, or characteristics of the invention may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are set forth such as examples of programming, software modules, user selection, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc. to provide a thorough understanding of embodiments of the invention. Details are provided. Those skilled in the relevant art will recognize, however, that the present invention may be practiced without one or more of the specific details, or with other methods, elements, materials, and so forth. In other instances, well-known structures, materials, or operations are not shown or described in detail in order to avoid obscuring aspects of the present invention.

The schematic flow charts contained herein are generally described in logical flow diagrams. As such, the order shown and the steps categorized refer to one embodiment of the method. Other steps and methods equivalent to function, logic, or effect of one or more steps of the described method, or portions thereof, may be envisioned. Moreover, the formats and symbols used are provided to illustrate the logical steps of the method and are understood to not limit the scope of the method. While various arrow types and line types may be used in the flowcharts, it is understood that they do not limit the scope of the corresponding method. In fact, some arrows or other connectors may be used to indicate only the logic flow of the method. For example, the arrows may indicate an idle or monitoring period of indefinite duration between enumerated steps of the depicted method. Moreover, the order in which a particular method occurs may strictly follow or not strictly follow the order of the corresponding steps shown.

Power reduction management

Figure 1 illustrates one embodiment of a system 100 for improved data management in the event of power failure, power reduction, or other power loss. In the illustrated embodiment, the system 100 includes a client 114 and a storage device 102. The client 114 may be a computer, such as a server, laptop, desktop, or other client device known in the art. The client 114 generally includes elements such as memory, processors, busses, and other elements as known to those skilled in the art.

The client 114 stores data in the storage device 102 and communicates data with the storage device 102 via a communication connection (not shown). The storage device 102 may be internal to the client 114 or external to the client 114. The communication connection may be a bus, a network, or other type of connection that allows the transfer of data between the client 114 and the storage device 102. In one embodiment, the storage device 102 is connected to the client 114 by a PCI connection, such as a PCI Express ("PCI-e"). The storage device 102 may be a card connected to the PCI-e connection to the client 114.

The storage device 102 also includes a first power connection 130 that connects the storage device 102 to a first power source that provides the storage device 102 with the power necessary to perform data storage operations such as reading, ). The storage device 102 receives the required power from the first power source through the first power connection 130 under normal operating conditions. In certain embodiments, such as the embodiment shown in FIG. 1, the first power connection 130 connects the storage device 102 to the client 114 and the client 114 provides power to the storage device 102 And operates with the first power supply. In certain embodiments, the first power connection 130 and the communication connection discussed above are part of the same physical connection between the client 114 and the storage device 102. For example, the storage device 102 may receive power through a PCI connection.

In other embodiments, the storage device 102 may be coupled to an external power supply via the first power connection 130. [ For example, the first power connection 130 may connect the storage device 102 to a first power source that is a power converter (often referred to as a power brick). Those skilled in the art will appreciate that there are a variety of ways that the storage device 102 can accept power, and various devices that can operate with the first power source for the storage device 102.

The storage device 102 provides a non-volatile store for the client 114. 1 is a block diagram of a system 100 that includes a write data pipeline 106, a read data pipeline 108, a non-volatile memory 110, a storage controller 104, a power management device 122, and a second power supply 124 Storage device 102 is shown. The storage device 102 may include additional elements not shown to provide a more simplified illustration of the storage device 102.

The non-volatile memory 110 stores data such that even when the storage device 102 is powered off, the data is retained. Examples of non-volatile memory 110 include solid-state memory (such as flash), hard disk, tape, or others. The storage device 102 also includes a storage controller 104 that organizes the storage and retrieval of data in non-volatile memory 110. The storage controller 104 may use one or more indexes to locate or retrieve data and may perform other operations on data stored in the storage device 102. For example, the storage controller 104 may include a groomer for performing data grooming operations, such as garbage collection.

As shown, the storage device 102, in certain embodiments, executes a write data pipeline 106 and a read data pipeline 108, examples of which are described in more detail below with respect to FIG. 3 do. The write data pipeline 106 may perform certain operations on the data as it is transferred from the client 114 to the non-volatile memory 110. [ These operations may include, for example, Error Acquisition Code (ECC) generation, encryption, compression, and others. Read data pipeline 108 may perform similar and potentially opposite operations on data that is read from non-volatile memory 110 and sent to client 114. [

The storage device 102 also includes a second power supply 124 that provides power in the event of a complete or partial power outage that results in the storage device 102 not fully receiving power through the first power connection 130 do. A power outage may unexpectedly cause the storage device 102 to cease receiving power through the first power connection 130 or may cause unexpected power outage of the power received by the storage device 102 via the first power connection 130 Which in any case leads to a significant reduction. A significant reduction in power, in one embodiment, includes a power fall below a predetermined limit. The predetermined threshold is selected to allow, in other embodiments, normal variations in power level from the first power connection 130. For example, the power for the building where the client 114 and the storage device 102 may not function. (Such as stopping the client 114 providing power to the storage device 102), a failure of the first power connection 130, or a failure of the first power supply may cause the storage device 102 to power up It can lead to stopping acceptance. A large number of various outages can cause unexpected power loss to the storage device 102.

The second power supply 124 may include one or more batteries, one or more capacitors, a bank of capacitors, a separate connection to a power supply, or the like. In one embodiment, the second power supply 124 provides power to the storage device 102 for at least a power hold time during a power down or other power reduction from the first power contact 130. [ The second power supply 124 may provide a power hold time that is long enough to enable the storage device 102 to flush data that is not in the non-volatile memory 110 to the non-volatile memory 110, . As a result, the storage device 102 may preserve data that is not permanently stored in the storage device 102 before a power shortage causes the storage device 102 to cease functioning. In a particular implementation, the second power supply 124 may include the smallest possible capacitors that can preserve space, reduce cost, and provide a predetermined power retention time to simplify the storage device 102 . In one embodiment, when the capacitors are generally more reliable, require less maintenance, and have a longer lifetime than the other options for providing the second power, one or more banks of capacitors are connected to the second power supply 124).

In one embodiment, the second power supply 124 is part of an electrical circuit that automatically provides power to the storage device 102 during partial or complete loss of power from the first power contact 130. Likewise, the system 100 may be configured to automatically accept or accept power from the second power supply 124 during partial or complete power loss. For example, in one embodiment, the second power supply 124 may be electrically coupled to the storage device 102 in parallel with the first power connection 130 such that the first power connection 130 During normal operation, the second power supply 124 is energized and the second power supply 124 automatically provides power to the storage device 102 in response to a power loss. In one embodiment, the system 100 may include a second power supply 124 and a first power connection 130 (not shown) to prevent current from the second power supply 124 from reaching the first power connection 130 Lt; RTI ID = 0.0 > and / or < / RTI > In another embodiment, power management device 122 may enable second power supply 124 to storage device 102 using a switch or the like in response to reduced power from first power connection 130 .

Examples of data that are not yet in non-volatile memory 110 may include data that may be held in non-volatile memory as the data moves through the write data pipeline 106. [ If data in the write data pipeline 106 is lost during a power failure (i.e., not written to or otherwise permanently stored in non-volatile memory 110), corruption and data loss may result.

In certain embodiments, the storage device 102 sends an acknowledgment to the client 114 at some point after the storage device 102 receives the data stored in the non-volatile memory 110. [ The write data pipeline 106, or a sub-element thereof, can generate an acknowledgment. It is advantageous to send the acknowledgment as soon as possible after the storage device 102 has received the data.

In certain embodiments, the write data pipeline 106 sends an acknowledgment before the data is actually stored in non-volatile memory 110. [ For example, the write data pipeline 106 may send acknowledgments while data is still being transferred to the non-volatile memory 110 via the write data pipeline 106. In such embodiments, it is highly desirable for the storage device 102 to flush all data while the storage controller 104 is in the process of preventing data corruption and for maintaining the integrity of the transmitted acknowledgment, Volatile memory 110 before power is sufficiently lost.

Moreover, in certain embodiments, some data in the write data pipeline 106 may be corrupted as a result of power failure. Power outages may include power interruption as well as unexpected changes in the power level being supplied. An unexpected change in the power level may be placed in the data in the storage device 102 in a critical state, but not in the non-volatile memory 110 yet. Data corruption may begin to occur before the power management device 122 immediately detects (recognizes) that there has been a power outage.

For example, the PCI-e specification indicates that, in the event of a power failure signal, data is impersonated, corrupted, and not stored in a particular environment. Similar potential damage may be caused by PCI, Serial Advanced Technology Attachment ("Serial ATA" or "SATA", Parallel ATA ("PATA"), Small Computer System Interface ("SCSI"), IEE 1394 ("FireWire" May occur in storage devices 102 that are connected to clients 114 using other connection types such as a universal serial bus ("USB"), PCIe-AS, or the like. (Which means that the received data until time is estimated to be corrupted), and after a period of time, a power failure is detected and a signal is received, the power management device 122 receives the signal and recognizes the power failure A delay between the correction that occurred and the power management device 122 that found the power outage may allow the corrupted data to enter the write data pipeline 106. In certain embodiments, And should not be stored in non-volatile memory 110. Alternatively, such corrupted data may be stored in non-volatile memory 110 and marked as corrupted as described below. Identifying the corrupted data and not storing such data in the non-volatile memory 110 will be used primarily to describe the functions and features hereinbefore. Additionally, the client 114 may determine that such data is not stored It should be noted that data that requires or is questionable about its integrity will not be accepted until the data integrity can be verified. As a result, the corrupted data should not be acknowledged.

The storage device 102 also includes a power management device 122. In certain embodiments, the power management device 122 is implemented as part of the storage controller 104. The power management device 122 may be, for example, a software driver or may be executed in firmware for the storage device 102. In other embodiments, the power management device 122 may be implemented in part in the software driver and partially in the storage controller 104, or the like. In one embodiment, at least a portion of the power management device 122 is implemented as part of the storage controller 104 at the storage device 102, or the like, so that even though the client 114 is no longer functioning The power management device 122 continues to function during partial or complete power loss using power from the second power supply 124.

In one embodiment, the power management device 122 initiates a power loss mode at the storage device 102 in response to a decrease in power from the first power contact 130. During a power loss mode, the power management device 122 flushes the data in the storage device 102 to the non-volatile memory 110, rather than being stored in the non-volatile memory 110, in one embodiment. In certain embodiments, the power management device 122 flushes the data in the storage device 102 to the non-volatile memory 110 without being stored in the non-volatile memory 110 yet. In certain embodiments described below, the power management device 122 may be configured to determine whether the second power supply 124 is operating before the second power supply 124 sufficiently loses power to complete the necessary operations, The execution of the data operations on the storage device 102 may be adjusted to ensure that the necessary operations are completed.

In certain embodiments, the essential operations include operations on data identified as being stored, such as the identified write operations. In other embodiments, the essential operations include operations on data that has been stored and identified as erased. In other embodiments, the essential operations include operations on data that has been stored, read, and has been confirmed to be erased. The power management device 122 may also remove nonessential operations to ensure that nonessential operations do not unnecessarily consume power and / or do not prevent essential operations from being performed; For example, the power management device 122 may remove erase operations, read operations, verified write operations, and the like.

In one embodiment, eliminating nonessential operations preserves power from the second power supply 124 and allows the second power supply 124 to provide power retention time. In another embodiment, the power management device 122 deactivates or otherwise disables operation of one or more of the sub-elements of the storage device 102 during the power loss mode to maintain power from the second power supply. For example, in various embodiments, the power management device 122 may include a read data pipeline 108, a read direct memory access ("DMA") engine, and / or storage 102 associated with non- ≪ / RTI > can be deactivated.

The power management device 122 also determines if any data is corrupted by a power outage and prevents corrupted data from being stored in the non-volatile memory 110 and ensures that the corrupted data is never destined to the storage device 102 ) That the user has not actually been stored. This prevents data corruption in the storage device 102 resulting from a power failure.

In one embodiment, the system 100 includes a plurality of storage devices 102. Power management device 122 manages power loss modes for each storage device 102 in a plurality of storage devices 102 in one embodiment and controls power loss modes for a plurality of storage devices 102 Of power loss mode. In other embodiments, each storage device 102 in a plurality of storage devices 102 includes a respective power management device 122 that manages a separate power loss mode for each storage device 102 . The power management device 122 may include a plurality of storage devices 102 to maintain power from a second power supply 124 for performing the necessary operations on one or more other storage devices 102, Or disable one or more of the storage devices 102 of the storage device 102. [

In one embodiment, system 100 includes one or more adapters for providing an electrical connection between client 114 and a plurality of storage devices 102. The adapter may, in various embodiments, include a slot or port that accepts a single storage device 102, an expansion card or daughter card that accepts two or more storage devices 102, or the like. have. For example, in one embodiment, a plurality of storage devices 102 may be coupled to separate ports or slots, respectively, In other exemplary embodiments, one or more adapters, such as daughter cards or the like, may be electrically coupled to the client 114 (i.e., coupled to one or more slots or ports of the client 114) Adapters can each provide a connection to more than one storage device 102.

In one embodiment, the system 100 includes a circuit board, such as a motherboard or the like, that accepts two or more adapters such as daughter cards or the like, each adapter having two or more storage devices 102 I accept it. In another embodiment, the adapters are coupled to the circuit board using PCI-e slots of the circuit board and the storage devices 102 are coupled to the adapters using PCI-e slots of the adapters. In another embodiment, each of the storage devices 102 includes a dual in-line memory module ("DIMM"), or the like, which is a non-volatile solid state storage device such as flash memory. In one embodiment, the circuit board, adapters, and storage devices 102 may be external to the client 114 and may include a separate first power connection 130. For example, circuit boards, adapters, and storage devices 102 may be housed in an external enclosure with a power supply unit ("PSU") and may be an eSATA, eSATAp, SCSI, FireWire, Fiber Channel, AS, or the like. ≪ / RTI > In other embodiments, the circuit board may be the motherboard of the client 114 and the adapters and storage devices 102 may be internal storage devices of the client 114. [

In view of the present invention, those skilled in the art will recognize many types of adapters and storage devices 102 for use in the system 100. For example, each adapter may accept two storage devices 102, four storage devices 102, or any number of storage devices 102. Likewise, the system 100 may include one adapter, two adapters, three adapters, four adapters, or any number of adapters. In one exemplary embodiment, the system 100 includes two adapters, each of which accepts four storage devices 102 for a total of eight storage devices 102.

In one embodiment, the second power supply 124 provides power to each of the plurality of storage devices 102. For example, the second power supply 124 may be located in a circuit on the main circuit board or motherboard and may provide power to some adapters. In another embodiment, the system 100 includes a plurality of second power supplies each providing power to a subset of the plurality of storage devices 102. For example, in one embodiment, each adapter may include a second power supply 124 for the storage devices 102 of the adapter. In another embodiment, each storage device 102 may include a second power supply 124 for the storage device 102. In view of the present invention, those skilled in the art will appreciate the different arrangements of the second power supplies 124 for providing power to a plurality of storage devices 102.

Solid state storage

2 illustrates an embodiment 200 of a solid state storage controller 202 that includes a write data pipeline 106 and a read data pipeline 108 in a solid state storage device 102 in accordance with the present invention. Fig. The solid state storage controller 202 may include a plurality of solid state storage controllers (0-N 104a-n), each of which controls the solid state storage device 110. [ In the illustrated embodiment, two solid state controllers are shown: a solid state controller 104a and a solid state controller N 104n, each of which controls solid state storage devices 110a-n. In the illustrated embodiment, the solid state storage controller 104a controls the data channel and the resulting solid state storage device 110a stores the data. The solid state storage controller N 104n controls the index metadata channel associated with the stored data and the associated solid state storage device 110n stores the index metadata. In another embodiment, the solid state storage controller 202 includes a single solid state controller 104a having a single solid state storage device 110a. In another embodiment, there are a plurality of solid state storage controllers 104a-n and associated solid state storage devices 110a-n. In one embodiment, one or more solid state controllers 104a-104n-1 coupled to their associated solid state storage devices 110a-110n-1 control data while their associated solid state storage devices 110a-110n- Is controlled by the at least one solid state storage controller 104n.

In one embodiment, the at least one solid state controller 104 is a field programmable gate array ("FPGA") and controller functionality is programmed into the FPGA. In a particular embodiment, the FPGA is a Xilinx (R) FPGA. In another embodiment, the solid state storage controller 104 includes components specifically designed with a solid state storage controller 104, such as an application specific integrated circuit ("ASIC") or a custom logic solution. Each solid state storage controller 104 generally includes a write data pipeline 106 and a read data pipeline 108, which are further described with respect to FIG. In another embodiment, the at least one solid state storage controller 104 is comprised of a combination of FPGA, ASIC, and custom logic elements.

Solid state storage

The solid state storage device 110 includes an array of nonvolatile solid state storage devices (not shown) disposed in banks 214 and connected in parallel via a bi-directional storage input / output ("I / O & 216, 218, 220). The storage I / O bus 210, in one embodiment, is capable of one-way communication at one time. For example, when data is written to the solid state storage device 110, the data can not be read from the solid state storage device 110. In another embodiment, the data can flow simultaneously in both directions. However, bi-directional, as used herein corresponding to a data bus, refers to a data path that can have data flowing in only one direction at any one time, but when data flowing in one direction relative to the bi-directional data bus is delayed, Data may flow in the opposite direction to the bi-directional data bus.

Solid state storage devices (eg SSS 0.0 216a) typically consist of a die (a package of one or more dies) or a die on a circuit board. As shown, even though some of these devices are packaged together as a chip package, a stack of chip packages, or some other package element, a solid state storage device (e.g., 216a) RTI ID = 0.0 > 218a). ≪ / RTI > As shown, a row of solid state storage devices 216a, 216b, 216m is indicated by a bank 214. As shown, there are many "n" banks 214a-n in the nxm solid state storage devices 216, 218, 220 in one arrangement of solid state storage devices 110 and a solid state There may be storage devices 216a-m, 218a-m, 220a-m. Of course, different embodiments may include different values for n and m. In one embodiment, solid state storage device 110a includes 20 solid state storage devices 216, 218, and 220 per bank 214 with eight banks 214. In one embodiment, In one embodiment, solid state storage medium 110a includes twenty-four solid state storage devices 216, 218, 220 per bank 214 with eight banks 214. In addition to the nxm storage devices 216, 218 and 220, one or more additional columns P may also be arranged in parallel with the other solid state storage devices 216a, 216b, 216m for one or more columns, . The added P columns store parity data for portions of an ECC chunk (i.e., ECC codeword) spanning m storage devices for a particular bank in one embodiment. In one embodiment, each of the solid state storage devices 216, 218, 220 is comprised of single level cell ("SLC") devices. In another embodiment, each solid state storage device 216, 218, 220 comprises multilevel cell ("MLC") devices.

In one embodiment, solid state storage devices (e.g., 216b, 218b, 220b) that share a common storage I / O bus 210a are packaged together. In one embodiment, solid state storage devices 216, 218, 220 may have more than one die per chip with one or more vertically stacked chips, and each die may be independently connected. In another embodiment, a solid state storage device (e.g., SSS 0.0 216a) may have one or more virtual dies per die and one or more dies per chip, and one or more vertically stacked chips and each virtual die may be independently connected . In another embodiment, the solid state device (SSS 0.0 216a) may have more than one virtual die per die and more than one die per chip, and some or all vertically stacked dies and each virtual die may be independently Can be connected.

In one embodiment, the two dies may be divided into four dice per group to form eight storage devices (e.g., SSS 0.0-SSS 8.0) 216a-220a in each separate bank 214a-n. Vertically stacked. In another embodiment, the 24 storage devices (e.g., SSS 0.0-SSS 0.24) 216 form logical bank 214a, resulting in each of the eight logical banks having 24 storage devices (e.g., SSS0 .0-SSS 8.24) 216, 218, and 220, respectively. Data is transferred to the solid state storage device 110 via the storage I / O bus 210 to all the storage devices of a particular group of storage devices (SSS 0.0-SSS 8.0) 216a, 218a, 220a. The storage control bus 212a is used to select a particular bank (e.g., bank 0 214a) so that the data received via the storage I / O bus 210, which is connected to all the banks 214, And is immediately recorded in the bank 214a.

In one embodiment, the storage I / O bus 210 is comprised of one or more independent I / O busses 210a.am, 210n.am, including "IIOBa-m", wherein the solid state The storage devices share one of the independent I / O buses connected to each of the solid state storage devices 216, 218, 220 in parallel so that all of the banks 214 are connected at the same time. For example, one channel of the storage I / O bus 210 may be simultaneously connected to the first solid state storage devices 216a, 218a, 220a of the respective banks 214a-n. The second channel of the storage I / O bus 210 may be simultaneously connected to the second solid state storage devices 216b, 218b, 220b of the respective banks 214a-n. The solid state storage devices 216a, 216b, and 216m in each row are connected at the same time. In one embodiment, when the solid state storage devices 216, 218, 220 are multi-level (physically stacked), all physical levels of the solid state storage devices 216, 218, do. As used herein, "simultaneously" also includes near simultaneous connections in which devices are connected at slightly different intervals to avoid switching noise. "Concurrent" is used in this context in which commands and / or data are distinguished from sequential or serial connections, which are transmitted individually in turn.

In general, the banks 214a-n are independently selected using the storage control bus 212. In one embodiment, the bank 214 is selected using chip enable or chip selection. If chip selection and chip enable are available, the storage control bus 212 may select the level of one of the multilevel solid state storage devices 216, 218, 220. In other embodiments, other instructions are used by the storage control bus 212 to individually select the level of one of the multi-level solid state storage devices 216, 218, 220. The solid state storage devices 216, 218, 220 may also be selected through a combination of control and address information sent to storage I / O bus 210 and storage control bus 212.

In one embodiment, each solid state storage device 216, 218, 220 is divided into erase blocks and each erase block is divided into pages. The erase block for solid state storage devices 216, 218 220 may be referred to as a physical erase block or "PEB ". A typical page is 2000 bytes ("2kB"). In one example, a solid state storage device (e.g., SSS 0.0) includes two registers and can program two pages, resulting in two register solid state storage devices 216, 218, . The bank 214 of the 20 solid state storage devices 216a, 216b and 216m may then have pages of 80 kB capacity connected to the same address exiting the channels of the storage I / O bus 210.

These groups of pages in the bank 214 of the 80 kB solid state storage devices 216a, 216b, 216m may be referred to as logical pages or virtual pages. Likewise, the erase blocks of each storage device 216a-m of the bank 214a may form a group to form a logical erase block or a virtual erase block. In one embodiment, erase blocks of pages in solid state storage devices 216, 218, 220 are erased when erase commands are received in solid state devices 216, 218, 220. While the size and number of erase blocks, pages, planes, or other logical and physical partitions in solid state storage devices 216, 218, 220 are expected to change over time with advances in technology , It is anticipated that many embodiments consistent with the novel features are possible and consistent with the general description herein.

In general, when a packet is written to a particular location in the solid state storage device 216, 218, 220, the packet is written to a location within a particular page that is specific to a particular physical erase block of a particular storage device of a particular bank And the physical address is sent to the storage I / O bus 210 and then the packet is transmitted. The physical address has sufficient information for the solid state storage device 216, 218, 220 to direct the packet to the indicated location within the page. All of the storage devices (e.g., SSS 0.0-SSS N.0.1 216a, 218a, 220a) in the storage devices of one column are simultaneously connected by a suitable bus in the storage I / O bus 210a.a Therefore, in order to reach the appropriate page and to avoid writing the data packet to the similarly addressed pages in the storage devices of this column (SSS 0.0-SSS N.0 216a, 218a, 220a), the data packet is written A bank 214a containing solid state storage (SSS 0.0 216a) with the correct page is simultaneously selected by the storage control bus 212.

Likewise, meeting the read command on the storage I / O bus 210 requires a simultaneous signal on the storage control bus 212 to select the appropriate page within the signal bank 214a and the bank 214a do. In one embodiment, the read command reads the entire page, and since there are a number of solid state storage devices 216a, 216b, 216m provided in parallel to the bank 214, the entire logical page is read as a read command . However, the read command may be subdivided into subcommands as described below for bank interleaving. The logical page may also be connected to a write operation.

The erase block erase command may be sent out to erase the erase block via the storage I / O bus 210 having a specific erase block address to erase the specific erase block. Generally, an erase block erase command may be sent over the parallel paths of the storage I / O bus 210 to erase a logical erase block each having a specific erase block address to erase a specific erase block. At the same time, a particular bank (e.g., bank 214a) is coupled to the storage control bus 212 to prevent erasure of similarly addressed erase blocks in all banks (banks 1-N414b-n) . Alternatively, a particular bank (e. G., Bank 214a) may be coupled to the storage control bus 212 (e. G., Bank 214a-n) to enable erasure of erase blocks similarly similarly addressed in all banks ). ≪ / RTI > Other instructions may also be transmitted to a particular location using a combination of storage I / O bus 210 and storage control bus 212. [ Those skilled in the art will recognize other ways to select a particular storage location using the bi-directional storage I / O bus 210 and the storage control bus 212.

In one embodiment, packets are continuously written to the solid state storage device 110. For example, the packets are streamed to the storage write buffers of the bank 214a of the storage devices 216 and when the buffers are filled, the packets are programmed into the indicated logical page. The packets then refill the storage write buffers, and when they are filled, the packets are written to the next logical page. The next logical page may be the same bank 214a or another bank (e.g., 214b). This process generally continues for each logical page until the logical erase block is filled. In another embodiment, streaming may continue beyond the logical erase block boundary and this process continues for each logical erase block.

In a read, modify, write operation, data packets associated with the requested data are located and read in a read operation. The data segments of the modified modification request data are not recorded at the location where they are read. Alternatively, the modified data segments are converted back into data packets and subsequently written to the next available location in the logical page being currently written. The index entries for the individual data packets are modified to indicate packets having the modified data segments. Entries or entries in the index for data packets associated with the same request data that have not been modified will contain pointers to the original location of the unmodified data packets. Thus, if the original request data is maintained, for example, to retain the previous version of the request data, the originally requested data will have pointers to all data packets as originally recorded. The newly requested data will have pointers to the indexes for some original data packets and pointers to the modified data packets in the logical page being currently written.

In a copy operation, the index includes an entry for the originally requested data mapped to a plurality of packets stored in the solid state storage device 110. [ When a copy is made, a new copy of the requested data is created and a new entry is created in the original packet with an index mapping a new copy of the requested data. A new copy of the requested data is also written to the solid state storage device 110 and its location is mapped to a new entry in the index. A new copy of the requested data packets may be used to identify the packets in the originally requested data that is referenced when the originally requested data that has not been propagated to the copy of the requested data is modified and the index is lost or corrupted.

Advantageously, consecutively written packets facilitate more uniform use of the solid state storage device 110 and allow the solid state storage controller 202 to store the stored hot spots and the usage levels of the various logical pages in the solid state storage device 110 To be monitored. Consecutively written packets also facilitate a robust and effective unwanted data collection system, which is described in detail below. The skilled artisan will recognize other advantages of continuous storage of data packets.

Solid State Storage Controller

In various embodiments, the solid state storage controller 202 also includes a data bus 204, a local bus 206, a buffer controller 208, buffers (0-N 222a-n), a master ("DMA") controller 226, a memory controller 228, a dynamic memory array 230, a static random memory array 232, a management controller 234, a management bus 236, A bridge 238 for the system bus 240, and miscellaneous logic 242. In other embodiments, the system bus 240 is coupled to one or more network interface cards ("NICs") 244, some of which may be remote DMA ("RDMA") controllers 246, One or more central processing units ("CPUs") 248, one or more external memory controllers 250 associated with external memory arrays 252, one or more storage controllers 254, peer controllers 256, And may include application specific processors 258. The elements 244-258 connected to the system bus 240 may be located in the client 114 or may be in other devices.

In general, the solid state storage controller (s) 104 communicate data to the solid state storage device 110 via the storage I / O bus 210. In a typical embodiment, when a solid state storage device is disposed in banks 214 and each bank 214 includes a plurality of storage devices 216a, 216b, 216m connected in parallel, the storage I / The bus 210 is an array of busses, one for each of the storage devices 216, 218, 220 of the columns, extending into the banks 214. As used herein, the term "storage I / O bus" may refer to one storage I / O bus 210 or one array of data independent buses 204. In one embodiment, each storage I / O bus 210 that connects to one column of storage devices (e.g., 216a, 218a, 220a) includes one column of storage devices 216a, 218a, 220a, Physical mapping to storage partitions (e. G., Erase blocks) that are connected to the storage partitions. This mapping (or bad block remapping) may be performed if the logical address mapped to the physical address of the storage partition is different from the storage partition < RTI ID = 0.0 >Lt; / RTI >

Data is also transferred from the requesting device 155 via the system bus 240, the bridge 238, the local bus 206, the buffer (s) 222, and finally the data bus 204 to the solid state storage controller ) ≪ / RTI > The data bus 204 is coupled to one or more buffers 222a-n that are typically controlled by a buffer controller 208. [ The buffer controller 208 generally controls the transfer of data from the local bus 206 to the buffers 222 and from the data bus 204 to the pipeline input buffer 306 and the output buffer 330. Buffer controller 208 may generally temporarily store data arriving from the requesting device in buffer 222 to evaluate different clock domains, prevent data conflicts, and the like, , Or vice versa. ≪ / RTI > Buffer controller 208 generally operates in conjunction with master controller 224 to coordinate data flow. When data arrives, the data will arrive on the system bus 240 and be transferred to the local bus 206 via bridge 238. [

Data is typically transferred from the local bus 206 to one or more data buffers 222 when derived by the master controller 224 and the buffer controller 208. The data then flows from the buffer (s) 222 to the data bus 204, through the solid state controller 104, and to the solid state storage 110, such as NAND flash or other wetting media. In one embodiment, the associated out-of-band metadata ("metadata") arriving with data and data may be stored in solid state storage devices 110a-110n-1 associated with one or more solid state storage controllers 104a-104n- ), But at least one channel (solid state storage controller 104n, solid state storage device 110n) is communicated using index information and one or more data channels that are internally generated in solid state storage device 102 It is dedicated to in-band metadata such as other metadata.

Local bus 206 is generally connected between devices embedded in solid state storage controller 202 and to devices embedded in solid state storage device 102 and devices 244-258 Bus or set of buses that allow communication of data and instructions between the buses. The bridge 238 facilitates communication between the local bus 206 and the system bus 240. Those skilled in the art will recognize other embodiments such as busses 240, 206, 204, 210 and bridges 238 of ring structures or switched star shapes and functions.

System bus 240 is typically a bus of client 114 or other device upon which solid state storage 102 is installed or connected. In one embodiment, the system bus 240 may be a PCI-e bus, a Serial Advanced Technology Attachment ("Serial ATA") bus, Parallel ATA, or the like. In another embodiment, the system bus 240 is an external bus such as a small computer system interface ("SCSI"), FireWire, Fiber Channel, USB, PCIe-AS, or the like. The solid state storage device 102 may be packaged into the device either internally or externally.

The solid state storage controller 202 includes a master controller 224 that controls higher level functions within the solid state storage device 102. The master controller 224 controls the data flow by interpreting object requests and other requests in various embodiments, and is used to map the object identifiers associated with the data for the physical location of the associated data, . Many of the functions described herein are wholly or partially controlled by the master controller 224. [

In one embodiment, the master controller 224 uses the embedded controller (s). In another embodiment, the master controller 224 uses local memory, such as a dynamic memory array 230 (dynamic random access memory "DRAM"), a static memory array 232 (static random access memory "SRAM & In one embodiment, the local memory is controlled using a master controller 224. In another embodiment, the master controller 224 connects to the local memory via the memory controller 228. In another embodiment, the master controller 224 operates a Linux server and can provide a variety of public server interfaces, such as the World Wide Web, hypertext creation language ("HTML"), and the like. In another embodiment, the master controller 224 uses a nanoprocessor. The master controller 224 may be configured using programmable or standard logic, or any combination of the controller types listed above. Those skilled in the art will recognize many embodiments of the master controller 224.

In one embodiment, when the storage device / solid state storage controller 202 manages a plurality of data storage devices / solid state storage devices 110a-n, the master controller 224 is coupled to the solid state storage controllers < RTI ID = (104a-n). For example, the master controller 224 may divide objects to be written to data storage devices (e.g., solid state storage devices 110a-n) so that a portion of the objects may be stored in each attached data storage device Lt; / RTI > This feature is a performance enhancement that allows faster storage and access to objects. In one embodiment, the master controller 224 is implemented using an FPGA. The firmware within the master controller 224 may be coupled to the management bus 236 via a system bus 240 passing through the network connected to the NIC 244 or other device connected to the system bus 240. [ Can be updated.

In one embodiment, the master controller 224, which manages objects, may be used by the client 114 or other device connected to the storage / solid state storage device 102 as a block storage device to store / And implements block storage to transfer data to a specific physical address in storage / solid state storage device 102. The master controller 224 then divides the blocks and stores the data blocks as they did to the objects. The master controller 224 then maps the physical addresses sent with the blocks to the actual locations determined by the blocks and the martor controller 224. The mapping is stored in the object index. In general, for block mimicry, a block device application program interface ("APT") may be used in the driver of a computer such as client 114, or as a block storage device And is provided to another apparatus.

In another embodiment, master controller 224 interfaces with NIC controllers 244 and embedded RDMA controllers 246 to signal just-in-time RDMA movement of data and instruction sets. NIC controller 244 may be hidden behind opaque ports to enable use of custom drivers. In addition, the driver for the client 114 may use the standard stack API and connect to the computer network 116 via an I / O memory driver operating in conjunction with the NICs 244.

In one embodiment, the master controller 224 is also an independent drive ("RAID") controller in a redundant array. When the data storage / solid state storage device 102 is networked with one or more other data storage devices / solid state storage devices 102, the master controller 224 may be a single layer RAID, a multi-layer RAID, RAID < / RTI > The master controller 224 also allows some objects to be stored in the RAID array and other objects to be stored without the RAID. In another embodiment, the master controller 224 may be a distributed RAID controller device. In another embodiment, the master controller 224 may include many RAID, distributed RAID, and other functions as described elsewhere. In one embodiment, the master controller 224 includes one or more storage elements 216, 218, 220 of a logical page protecting paradigm information stored in other storage elements 216, 218, 220 of the same logical page. And stores the data in a structure similar to RAID in which parity information is stored.

In one embodiment, the master controller 224 interfaces with single or redundant network managers (e.g., switches) that establish routing to balance bandwidth utilization, failover, and so on. In another embodiment, master controller 224 (via local bus 206) interfaces with integrated application specific logic and associated driver software. In another embodiment, the master controller 224 interfaces with the application specific processors 258 or the logic and associated driver software (via the external system bus 240). In another embodiment, master controller 224 (via computer network 116) interfaces with remote application specific logic and associated driver software. In another embodiment, the master controller 224 interfaces with a local bus 206 or an external bus attached hard disk drive ("HDD ") storage controller.

In one embodiment, the master controller 224 is operatively coupled to one or more storage device / solid state storage devices 102 when storage device / solid state storage device 102 may appear as a storage device connected via a SCSI bus, Internet SCSI ("iSCSI"), Fiber Channel, Storage controllers 254. The storage controller On the other hand, the storage / solid state storage device 102 can independently manage the objects and can appear as an object file system or a distributed object file system. Master controller 224 may also be connected by peer controllers 256 and / or application specific processors 258.

In another embodiment, the master controller 224 periodically verifies the FPGA code and / or controller software, identifies and / or verifies the FPGA code during operation (reset), checks the controller software during power on It provides a reset request, provides a reset request due to watchdog timeout, and works with an independently integrated management controller to provide other environmental measurements and settings of voltage, current, power, temperature, and threshold interrupts. In another embodiment, the master controller 224 manages the collection of unnecessary data to free up reuse of erase blocks. In another embodiment, the master controller 224 manages wear leveling. In another embodiment, the master controller 224 allows the data storage / solid state storage device 102 to be partitioned into multiple logical devices and allows partition-based media encryption. In yet another embodiment, the master controller 224 provides a solid state storage controller 104 with advanced, multi-bit ECC correction. Those skilled in the art will recognize other features and functions of the master controller 224 in the storage controller 202, or more specifically, in the solid state storage device 102.

In one embodiment, the solid state storage controller 202 includes a memory controller 228 that controls a dynamic random memory array 230 and / or a static random memory array 232. As noted above, the memory controller 228 may be independent of, or integrated with, the master controller 224. Memory controller 228 generally controls some type of volatile memory, such as DRAM (dynamic random memory array 230) and SRAM (static random memory array 232). In another example, the memory controller 228 also controls other memory types such as electrically erasable programmable read only memory ("EEPROM") and the like. In other embodiments, the memory controller 228 may control more than one memory type and the memory controller 228 may include more than one controller. In general, the memory controller 228 is executable to complement the SRAM 232 and controls as many SRAMs 232 as the DRAM 230.

In one embodiment, the object index is stored in memory 230, 232 and then periodically offloaded to the channel of solid state storage device 110n or other non-volatile memory. Those skilled in the art will recognize other uses and forms of the memory controller 228, the dynamic memory array 230, and the static memory array 232.

In one embodiment, solid state storage controller 202 is coupled between storage / solid state storage device 102 and one or more external memory controllers 250 and associated external memory arrays 252 and CPUs 248 And a DMA controller 226 for controlling the DMA operation of the DMA. Note that external memory controllers 250 and external memory arrays 252 are the requested external devices because they are external to storage / solid state storage device 102. Furthermore, the DMA controller 226 may also control RDMA operation with the requesting devices via the RDMA controller 246 associated with the NIC 244.

In one embodiment, the solid state storage controller 202 includes a management controller 234 coupled to the management bus 236. In general, the management controller 234 manages environmental measurements and status of the storage / solid state storage device 102. The management controller 234 may monitor the device temperature, fan speed, power supply settings, and the like via the management bus 236. The management controller 234 can provide readout and programming of an erasable programmable read-only memory ("EEPROM") for storage of FPGA code and controller software. In general, the management bus 236 is coupled to various devices within the storage / solid state storage device 102. The management controller 234 may communicate alerts, interrupts, etc. via the local bus 206 and may include separate connections to the system bus 240 or other buses. In one embodiment, the management bus 236 is an Inter-Integrated Circuit ("I2C") bus. Those skilled in the art will recognize other related functions and uses of the management controller 234 connected to the elements of the storage / solid state storage device 102 by the management bus 236. [

In one embodiment, the solid state storage controller 202 includes miscellaneous logic 242 that can be customized for a particular application. In general, in the case where the solid state device controller 202 or the master controller 224 is configured using an FPGA or other configurable controller, the on-demand logic may be included based on specific applications, ordering requirements, storage requirements, and the like.

Data pipeline

3 is a schematic diagram illustrating one embodiment 300 of a solid state storage controller 104 having a write data pipeline 106 and a read data pipeline 108 in a solid state storage device 102 in accordance with the present invention. Respectively. The embodiment 300 includes a data bus 204, a local bus 206, and a buffer control device 208, which are substantially similar to those described in connection with the solid state storage controller 202 of FIG. The write data pipeline 106 includes a packetizer 302 and an error correction code ("ECC") generator 304. In other embodiments, the write data pipeline 106 includes an input buffer 306, a write synchronization buffer 308, a write program module 310, a compression module 312, an encryption module 314, Unneeded data collector bypass 316, which has a portion within line 108, a media encryption module 318, and a write buffer 320. The read data pipeline 108 includes a read synchronization buffer 328, an ECC correction module 322, a de-packetizer 324, an alignment module 326, and an output buffer 330. In other embodiments, the read data pipeline 108 includes a media decryption module 332, a portion of the unwanted data collector bypass 316, a decryption module 334, an decompression module 336, 338 < / RTI > The solid state storage controller 104 also includes control and status registers 340 and control cues 342, a bank interleave controller 344, a synchronization buffer 346, a storage bus controller 348, and a multiplexer ) ≪ / RTI > The elements of the write data pipeline 106 and the read data pipeline 108 associated with the solid state controller 104 are described below. In other embodiments, synchronous solid state media 110 may be used and synchronization buffers 308 and 328 may be eliminated.

Historical data pipeline

The write data pipeline 106 receives data or metadata segments that are written to the solid state storage device either directly or indirectly through another write data pipeline 160 stage and is sized for the solid state storage medium 110 And a packetizer 302 for generating one or more packets that are made. The data or metadata segments are typically part of a data structure, such as an object, but may also include the entire data structure. In another embodiment, the data segment is a portion of a block of data, but may also include an entire block of data. In general, one set of data, such as a data structure, is received from a computer, such as client 114, or other computer or device, and communicated to solid state storage device 102 with data segments streamed to solid state storage device 102. [ Lt; / RTI > A data segment may also be known by another name, such as a data parcel, but includes all or part of a data structure or block of data as mentioned herein.

Each data structure is stored in one or more packets. Each data structure may have one or more container packets. Each packet has a header. The header may include a header type field. The type fields may include data, attributes, metadata, data segment delimiters (multi-packet), data structures, data connections, and the like. The header may also include information about the size of the packet, such as the number of bytes of data contained in the packet. The length of the packet can be set by the packet type. The header may contain information defining the relationship of the packet to the data structure. An example may be the use of an offset in the data packet header to identify the location of a data segment within the data structure. Those skilled in the art will recognize other information that may be included in the header added to the data by the packetizer 302 and other information that may be added to the data packet.

Each packet contains a header or possibly data from the data or metadata segments. The header of each packet contains appropriate information relating the packet to the data structure to which the packet belongs. For example, the header may include an object identifier or other data structure identifier and an offset from the data segment, object, data structure or data block from which the data packet is formed. The header may also include a logical address used by the storage bus controller 348 to store the packet. The header may also include information about the size of the packet, such as the number of bytes contained in the packet. The header may also include the number of sequences identified when the data segment belongs to other packets within the data structure when reconstructing the data segment or data structure. The header may include a header type field. The type fields may include data, data structure attributes, metadata, data segment delimiters (multi-packet), data structure types, data structure links, and so on. Those skilled in the art will recognize other information that may be included in the header added to the data or metadata by the packetizer 302 and other information that may be added to the packet.

The write data pipeline 106 includes an ECC generator 304 that generates one or more error correction codes ("ECC") for one or more packets received from the packetizer 302. ECC generator 304 typically uses an error correction algorithm to generate ECC check bits that are stored in one or more data packets. The ECC codes generated by ECC generator 304 with one or more data packets associated with ECC codes include an ECC chunk. ECC data stored in one or more data packets is used to detect and correct errors inserted into the data through transmission and storage. In one embodiment, the packets are streamed to the ECC generator 304 with unencoded blocks of length N. [ The length S of a certain behavioral pattern is calculated, added and output as an encoded block of length N + S. The values of N and S depend on the characteristics of the ECC algorithm selected to achieve the specific performance, efficiency, and robustness metrics. In one embodiment, there is no fixed relationship between ECC blocks and packets; A packet may contain more than one ECC block; An ECC block may include more than one packet; The first packet may end anywhere within the ECC block and the second packet may start after the end of the first packet in the same ECC block. In one embodiment, the ECC algorithm is not dynamically changed. In one embodiment, the ECC data stored as data packets is robust enough to correct errors in more than two bits.

Advantageously, using a robust ECC algorithm that allows for more than one bit correction or even double bit correction allows the life of the solid state storage media 110 to be extended. For example, if the flash memory is used as a storage medium in the solid state storage media 110, the flash memory can be written about 100,000 times without errors per erase cycle. This use limitation can be extended using a robust ECC algorithm. Having ECC generator 304 and corresponding ECC correction module 322 mounted on solid state storage device 102 allows solid state storage device 102 to internally correct the error and if it is robust If an unused ECC algorithm is used, it has a longer service life. However, in other embodiments, the ECC generator 304 may use a non-robust algorithm and may be single bit or double corrected. In another embodiment, solid state storage device 110 may include uncertain storage such as multi-level cell ("MLC") flash to increase capacity, and such storage is not reliable enough without a more robust ECC algorithm.

In one embodiment, the recording pipeline 106 receives a data segment that is written to the solid state storage medium 110 and provides the next stage (or more complex) of the recording data pipeline 106, (Another stage for the write data pipeline 106) includes an input buffer 306 that stores incoming data segments until it is ready to process the next data segment. The input buffer 306 generally allows for discrepancies between speed data segments that are received and processed by the write data pipeline 106 using an appropriately sized data buffer. The input buffer 306 is also coupled to the write data pipeline 106 at a greater rate than the data bus 204 may sustain by the write data pipeline 106 to improve the efficiency of operation of the data bus 204. [ To transmit the data. In general, when the write data pipeline 106 does not include the input buffer 306, the buffering function may be used to provide a network interface card ("NIC") or other device In solid state storage device 102 at client 114, such as within a storage device 102, but elsewhere, such as outside the record data pipeline 106. [

In another embodiment, the write data pipeline 106 also includes a write synchronization buffer 308 that buffers the packets received from the ECC generator 304 before writing the packets to the solid state storage medium 110. The write synchronization buffer 308 is located at the boundary between the local clock domain and the solid state storage clock domain and provides buffering to evaluate the clock domain difference. In other embodiments, synchronous solid state storage medium 110 may be used and synchronization buffers 308 and 328 may be eliminated.

In one embodiment, the write data pipeline 106 also receives one or more packets from the packetizer 302, either directly or indirectly, and sends the packets to the solid state storage 102 before sending them to the ECC generator 304. [ And a media encryption module 318 that encrypts one or more packets using a unique encryption key. In general, the entire packet including the headers is encrypted. In another embodiment, the headers are not encrypted. Herein, the encryption key is understood to mean a secret encryption key that is externally managed from the solid state storage controller 104.

The media encryption module 318 and the corresponding media decryption module 332 provide a level of security for the data stored in the solid state storage medium 110. If the solid state media 110 is connected to a different solid state storage controller 104, solid state storage device 102, or server, for example, if the data is encrypted with the media encryption module 318, The contents of the state storage medium 110 may not be read without using the same encryption key used during writing of data to the solid state storage medium 110, generally without significant effort.

In a typical embodiment, the solid state storage device 102 does not store the encryption key in the non-volatile storage and does not allow external access to the encryption key. The encryption key is provided to the solid state storage controller 104 during initialization. The solid state storage device 102 may use and store non-secret cryptographic temporary values used with the encryption key. Different temporary values may be stored for each packet. The data segments may be partitioned between multiple packets with specific temporal values to improve protection by the encryption algorithm.

The encryption key may be received from a client 114, a server, a key manager, or other device that manages the encryption key used by the solid state storage controller 104. In another embodiment, the solid state storage medium 110 may have more than two partitions and the solid state storage controller 104 may be a solid state storage medium, such as a solid state storage medium, Behave as if they were state storage controllers 104. In this embodiment, a particular media encryption key may be used for each partition.

In other embodiments, the write data pipeline 106 may also be configured to encrypt data or metadata segments received from the input buffer 306, either directly or indirectly, prior to transferring the data segments to the packetizer 302. [ Module 314, and the data segment is encrypted using the received encryption key along with the data segment. The encryption keys used by the encryption module 314 to encrypt the data may not be common to all data stored in the solid state storage device 102, but may be different based on their respective data structures, Together with received data segments. For example, an encryption key for a data segment that is encrypted by the encryption module 314 may be received in the data segment, or may be received as part of an instruction to record the data structure to which the data segment belongs. The solid state storage device 102 may use and store non-secret cryptographic temporary values in each data structure packet used with the cryptographic key. Different temporary values may be stored for each packet. The data segments may be partitioned between multiple packets with specific temporal values to improve protection by the encryption algorithm.

The encryption key may be received from the client 114, another computer, a key manager, or other device that maintains an encryption key used to encrypt the data segment. In one embodiment, the encryption keys are stored in the secure storage 102, the client 114, the computer, or one of the other external agents having the capability to execute industry standard methods to securely transmit and protect private and public keys And is transferred to the solid state storage controller 104.

In one embodiment, the encryption module 314 encrypts the first packet with the first encryption key received with the packet and encrypts the second packet with the second encryption key received with the second packet. In another embodiment, the encryption module 314 encrypts the first packet with the first encryption key received with the packet and passes the second data packet to the next stage without encryption. Advantageously, the encryption module 314, included in the write data pipeline 106 of the solid state storage device 102, is capable of encrypting data structures, either per data structure or per segment, without a single file system or other external system, Allowing to keep track of different encryption keys used to store data segments. Each requesting device 155 or an associated key manager independently manages the encryption keys used to encrypt only the data structures or data segments sent by the requesting device 155. [

In one embodiment, the encryption module 314 may encrypt one or more packets using a unique encryption key in the solid state storage device 102. The encryption module 314 may perform this media encryption independently or in addition to the encryption described above. Generally, the entire packet including the headers is encrypted. In another embodiment, the headers are not encrypted. Media encryption by the encryption module 314 provides a level of security for the data stored in the solid state storage medium 110. For example, when the data is encrypted with a unique encryption in the media specific solid state storage device 102, the local solid state storage medium 110 may be stored in different solid state storage controllers 104, solid state storage devices 102, The contents of the solid state storage medium 110 can not be read without using the same encryption key used during writing of data to the solid state storage medium 110, generally without significant effort.

In another embodiment, the write data pipeline 106 includes a compression module 312 that compresses data for a metadata segment before transferring the data segment to the packetizer device 302. The compression module 312 generally compresses data or metadata segments using compression routines known to those skilled in the art to reduce the storage size of the segments. For example, if the data segment includes a series of 512 zeros, the compression module 312 may replace the 512 zeros with a code or token indicating 512 zeros where the code is much larger than the space occupied by 512 zeros It is more compact.

In one embodiment, the compression module 312 compresses the first segment into a first compression routine and passes along the second segment without compression. In another embodiment, the compression module 312 compresses the first segment with a first compression routine and compresses the second segment with a second compression routine. It is advantageous to have such flexibility within solid state storage device 102 so that other devices that write data to client 114 or solid state storage device 102 may each specify a compression routine, While the other does not specify compression. The selection of compression routines may also be selected according to default settings based on each data structure type or data structure class. For example, a first data structure of a particular data structure may make it possible to override default compression routine settings, and a second data structure of the same data structure class and data structure type may use a default compression routine, The third data structure of the class and data structure type can not use compression.

In one embodiment, the write data pipeline 106 includes an unneeded data collector bypass 316 that receives data segments from the read data pipeline 108 as part of the data bypass in the unwanted data collection system. Unnecessary data collection systems generally mark packets that are no longer valid because the packets are typically marked or changed for deletion and the changed data is stored in a different location. In some respects, the unwanted data collection system determines if storage of a particular section can be restored. Such a determination may include a lack of available storage capacity, a percentage of data marked as invalid that has reached the limit, a consideration of valid data, an error detection rate for storage of such a section that has reached a limit, And the like. A number of factors can be considered as unnecessary collection algorithms to determine when storage of a certain section is restored.

Once the schedule section has been marked for recovery, valid packets in these sections should generally be relocated. Unnecessary data collector bypass 316 allows packets to be read into the read data pipeline 108 and then sent directly to the write data pipeline 106 without being sent from the solid state storage controller 104. In one embodiment, the unwanted data collector bypass 316 is part of an independent, unwanted data collector system that operates within the solid state storage device 102. This allows the solid state storage device 102 to manage data and the resulting data can improve performance, data reliability and avoid abuse and lack of use of any one location or area of the solid state storage medium 110 State storage medium 110 in order to increase the service life of the solid-state storage medium 110. [0050]

Unnecessary data collector bypass 316 coordinates insertion of segments into a write data pipeline 106 having clients 114 or other segments written by other devices. In the illustrated embodiment, the unneeded data collector bypass 316 is located in front of the packetizer 302 in the write data pipeline 106 and in the read data pipeline 108, But may also be located elsewhere in the read data pipeline 108 and the write data pipeline 106. Unnecessary data collector bypass 316 is used to improve the efficiency of storage within solid state storage medium 110 and thereby reduce the frequency of unneeded data collection by flushing the write data pipeline 106 to fill the remainder of the virtual page. ≪ / RTI >

In one embodiment, the write data pipeline 106 includes a write buffer 320 that buffers data for efficient write operations. In general, the write buffer 320 includes sufficient capacity for packets to fill at least one virtual page in the solid state storage medium 110. [ This allows the recording operation to transfer the entire page of data to the solid state storage medium 110 without interruption. By making the buffers in the write buffer 320 and the read data pipeline 108 of the write data pipeline 106 arbitrary in size to be the same capacity or larger than the storage write buffer in the solid state storage medium 110, It is more efficient to write and read data because a single write command can be made to transfer data of the entire virtual page to the solid state storage medium 110 instead of multiple instructions.

While the write buffer 320 is being filled, the solid state storage medium 110 may be used for other read operations. This is advantageous because the data may be written to the storage write buffer and other solid state devices having a smaller write buffer or no write buffer may be coupled to the solid state storage when the data flowing to the storage write buffer is stopped. The read operation can be blocked until the entire storage write buffer is filled and programmed. Another method for systems with or without a write buffer is to flush a write buffer that has not been filled to enable reading. In addition, this is inefficient because multiple write / program cycles are required to fill the page.

For the illustrated embodiment having a write buffer 320 made larger in size than the virtual page, a single write command containing a number of sub-instructions is then stored in each solid state storage element 216, 218, 220 A single program command may be followed to transfer the page's data from the write buffer to the indicated page in each solid state storage element 216,218,220. This technique eliminates partial page programming, which is known to reduce data reliability and durability, and has the advantage of releasing a target bank for reads and other instructions while the buffer is being filled.

In one embodiment, the write buffer 320 is a ping-pong buffer, where one side of this buffer is filled and then indicated for transmission at the appropriate time while the other side of the ping-pong buffer is filled. In another embodiment, write buffer 320 includes a first-in-first-out ("FIFO") register having a larger capacity than a virtual page of data segments. Those of skill in the art will recognize other forms of write buffers 320 that allow virtual pages of data to be stored prior to writing data to solid state storage medium 110. [

In another embodiment, the write buffer 320 is made smaller in size than the virtual page, and as a result, a page of smaller information may be written to the storage write buffer in the solid state storage medium 110. [ In this embodiment, in order to prevent the delay of the write data pipeline 106 from stopping the read operation, the data may need to be moved from one location to another location as part of the unwanted data collection process To form a queue. In the case of a data delay of the write data pipeline 106, the data may be supplied to the write buffer 320 via the unwanted data collector bypass 316 and then filled in pages of the virtual page before programming the data May be supplied to the storage recording buffer of the solid state storage medium (110). In this way, the data delay of the write data pipeline 106 does not delay reading from the solid state storage device 102.

In another embodiment, the write data pipeline 106 includes a write program module 310 having one or more user definable functions within the write data pipeline 106. The recording program module 310 allows the user to customize the recording data pipeline 106. [ The user can customize the recording data pipeline 106 based on specific data requirements or applications. In the case where the solid state storage controller 104 is an FPGA, the user can program the write data pipeline 106 relatively easily with on-demand commands and functions. The user may also use a recording program module 310 that includes on-demand functions with an ASIC, but customizing the ASIC may be more difficult than with an FPGA. Recording program module 310 may include buffers and a bypass mechanism to allow the first data segment to be executed in the recording program module 310 while the second data segment may continue through the recording data pipeline 106. [ Lt; / RTI > In other embodiments, the recording program module 310 may include a processor core that may be programmed via software.

Although the recording program module 310 is shown between the input buffer 306 and the compression module 312 but the recording program module 310 can be anywhere in the recording data pipeline 106 and the various stages 302- 0.0 > 320). ≪ / RTI > Furthermore, there may be a plurality of recording program modules 310 distributed among the various stages 302-320 that are programmed and operating independently. In addition, the order of the stages 302-320 may be changed. The skilled artisan will appreciate the available changes to the order of the stages 302-320 based on the particular user requirements.

Read data pipeline

The read data pipeline 108 includes an ECC correction module 322 that determines whether there is a data error in the ECC blocks of the request packet received from the solid state storage medium 110 by using the ECC stored in each ECC block of the request packet ). The ECC correction module 322 then corrects any errors in the request packet if there are any errors and the errors are correctable using ECC. For example, if the ECC can detect 6 bits of error but can correct only 3 bit errors, the ECC correction module 322 corrects the ECC blocks of the request packet with up to 3 bits of error. The ECC correction module 322 corrects errors in the bits by changing the error of the bits to the correct one or zero state so that the requested data packet is written to the solid state storage medium 110 when the ECC is generated for the packet Lt; / RTI >

If the ECC correction module 322 determines that the request packets have more bits than the ECC can correct, the ECC correction module 322 can not correct errors in the corrupted ECC blocks of the request packet, do. In one embodiment, the ECC correction module 322 sends an interrupt with a message indicating that the request packet is in an error state. The message may contain information that the ECC correction module 322 can not correct errors or that the ECC correction module 322 does not correct the error. In another embodiment, the ECC correction module 322 sends corrupted ECC blocks of the request packet along with the interrupt and / or message.

In one embodiment, a corrupted ECC block or a portion of a corrupted ECC block of a request packet that can not be corrected by the ECC correction module 322 is read and corrected by the master controller 224, To the ECC correction module 322 for further processing by the ECC correction module 322. In one embodiment, a corrupted ECC block of the request packet or a portion of the corrupted ECC block is sent to the device requesting the data. The requesting device 155 may replace the data using an alternate copy, such as a backup or mirror copy, and then use the alternate data of the requested data packet and send it to the read data pipeline 108, As shown in FIG. The requesting device 155 may use the header information of the request packet in error state to replace the corrupted request packet or to identify the requested data to replace the data structure to which the packet belongs. In another embodiment, the solid state storage controller 104 may use some types of RAID to store data and recover corrupted data. In another embodiment, the ECC correction module 322 sends interrupts and / or messages and the receiving device fails to perform the read operation associated with the requested data packet. The skilled artisan will appreciate that the one or more ECC blocks of the request packet may be corrupted and may include acknowledgments or actions taken as a result of other ECC correction module 322 that determines whether ECC correction module 322 can not correct errors something to do.

The read data pipeline 108 includes a de-packetizer 324 that receives ECC blocks of the request packet directly or indirectly from the ECC correction module 322, and checks and removes one or more packet headers. The de-packetizer 324 can identify packet headers by checking packet identifiers, data length, data location, etc. within the headers. In one embodiment, the header includes a hash code that can be used to verify that the packet delivered to the read data pipeline 108 is a request packet. The de-packetizer 324 also removes headers from the request packet added by the packetizer 302. The de-packetizer 324 may be instructed not to operate on certain packets but may pass them ahead of time without modification. An example may be a container label where the header information is requested during the rebuild process requiring index reorganization. Other examples include movement of various types of packets intended to be used within the solid state storage device 102. In another embodiment, the operation of the de-packetizing device 324 may depend on the packet type.

The read data pipeline 108 includes an alignment module 326 that receives data from the de-packetizer 324 and removes the undesired data. In one embodiment, the read command transmitted to the solid state storage medium 110 receives the data of the packet. The device requesting the data may not need all the data in the retrieved packet and the sorting module 326 removes the unwanted data. If all of the data in the retrieved page is request data, the sorting module 326 does not remove any data.

The sorting module 326 reformats the data into data segments of the data structure in a form suitable for the device requesting the data segment before forwarding the data segment to the next page. Generally, when data is processed by the read data pipeline 108, the size of the data segments or packets is changed at various stages. The sorting module 326 uses the received data to format the data into data segments suitable for being sent to the requesting device 155 and connected to form a response. For example, data from a portion of a first data packet may be combined with data from a portion of a second data packet. If the data segment is larger than the data requested by the requesting device 155, the sorting module 326 may discard the undesired data.

In one embodiment, the read data pipeline 108 includes a read synchronization buffer 328 that buffers one or more request packets read from the solid state storage medium 110 before being processed by the read data pipeline 108 do. Read sync buffer 328 is at the boundary between the solid state store clock domain and the local bus clock domain and provides buffering to evaluate the clock domain difference.

In another embodiment, the read data pipeline 108 includes an output buffer 330 that receives request packets from the sort module 326 and stores the packets before sending them to the requesting device 155. The output buffer 330 is coupled between the data segments when they are received from the stages of the read data pipeline 108 and when the data segments are transferred to other parts of the solid state storage controller 104 or to the requesting device 155 . ≪ / RTI > The output buffer 330 also receives data from the read data pipeline 108 at a higher rate than the data bus 204 can withstand the read data pipeline 108 to improve the efficiency of operation of the data bus 204. [ Lt; / RTI > data.

In one embodiment, the read data pipeline 108 receives one or more encrypted request packets from the ECC correction module 322 and sends the one or more request packets to the solid state storage device (e. G. 102 to decrypt the one or more request packets using a unique encryption key. Generally, the encryption key used to decrypt the data by the media decryption module 332 is the same as the encryption key used by the media encryption module 318. In another embodiment, the solid state storage medium 110 may have more than two partitions and the solid state storage controller 104 may include two or more solid Behave as if they were state storage controllers 104. In this embodiment, a unique media encryption key may be used for each partition.

In another embodiment, the read data pipeline 108 includes a decryption module 334 that decrypts the data segment formatted by the de-packetizer 324 before sending the data segment to the output buffer 330. The data segment may be decrypted using the received encryption key with a read request that initiates the retrieval of the requested packet received by the read synchronization buffer 328. [ The decryption module 334 may decrypt the first packet with the received encryption key along with the read request for the first packet and then decrypt the second packet with a different encryption key or to decrypt the read data pipeline 108 without decryption, It is possible to pass the second packet to the next stage of the second packet. When a packet is stored as a non-secret cryptographic temporary value, the temporary value is used with the encryption key to decrypt the data packet. The encryption key may be received from a client 114, a computer, a key manager, or other device that manages the encryption key used by the solid state storage controller 104.

In another embodiment, the read data pipeline 108 includes a decompression module 336 that decompresses the data segments formatted by the de-packetizing device 324. [ In one embodiment, the decompression module 336 uses compression information stored in one or both of the packet header and the container label to select a routine that is complementary to that used by the compression module 312 to compress the data . In another embodiment, the decompression routine used in decompression module 336 is indicated by the device requesting the decompressed data segment. In another embodiment, the decompression module 336 selects decompression routines according to default settings based on each data structure type or data structure class. The first packet of the first object may enable overriding the default decompression routine and the second packet of the second data structure of the same data structure class and data structure type may use the default decompression routine, The third packet of the third data structure of the structure class and data structure type can not use decompression.

In another embodiment, the read data pipeline 108 includes a read program module 338 that includes one or more user definable functions within the read data pipeline 108. The read program module 338 has features similar to the write program module 310 and allows the user to provide custom functions to the read data pipeline 108. [ The read program module 338 may be located as shown in Figure 3 and may be located at another location within the read data pipeline 108 or may be located at a number of locations within the read data pipeline 108 And < / RTI > Moreover, there may be multiple read program modules 338 within multiple locations within the read data pipeline 108 that operate independently. Those skilled in the art will recognize other forms of read program modules 338 in the read data pipeline 108. [ The stages of the read data pipeline 108, such as the write data pipeline 106, may be relocated and the skilled artisan will recognize other orders of stages within the read data pipeline 108. [

The solid state storage controller 104 includes control and status registers 340 and corresponding control queues 342. Control and status registers 340 and control queues 342 facilitate control and sequencing and sub-instructions associated with data being processed in the write data pipeline 106 and the read data pipeline 108. For example, the data segment of the packetizer 302 may have one or more corresponding control commands or instructions in the control queue 342 associated with the ECC generator 304. When a data segment is packetized, several instructions or instructions may be executed within the packetization device 302. [ Other instructions or indications may be passed through the control and status registers 340 to the next control queue 342 when the newly formed data packet generated from the data segment is passed to the next stage.

Instructions or instructions are simultaneously loaded into control queues 342 for packets forwarded to the write data pipeline 106 having respective pipeline stages to pull out the appropriate instructions or instructions when the individual packets are executed by this stage . Likewise, the instructions or instructions may be stored in control queues 342 for packets requested from the read data pipeline 108 having respective pipeline stages to pull out the appropriate instruction or instruction when an individual packet is executed by this stage Can be loaded at the same time. Those skilled in the art will recognize other features and functions of control and status registers 340 and control queues 342. [

The solid state storage controller 104 and / or the solid state storage device 102 may also include a bank interleave controller 344, a synchronization buffer 346, a storage bus controller 348, and a multiplexer (not shown) "MUX") < / RTI >

Bank interleaving

4 is a schematic block diagram illustrating one embodiment 400 of a bank interleave controller 344 in a solid state storage controller 104 in accordance with the present invention. The bank interleave controller 344 is coupled to the control and status registers 340 and the storage I / O bus 210 via a MUX 350, a storage bus controller 348, and a synchronization buffer 346, And is connected to the storage control bus 212. The bank interleave controller 344 is coupled to the banks 214 of the solid state storage medium 110, the bank controllers 418a-n, the bus arbiter 420 and the state MUX 350, The read agent 402, the write agent 404, the erase agent 406, the management agent 408, the read queues 410a-n, the write queues 412a-n, the erasure queues 414a- And cues 416a-n. The storage bus controller 348 includes a mapping module 424 having a remapping module 430, a status capture module 426, and a NAND bus controller 428, as described below.

The bank interleave controller 344 directs one or more instructions to two or more queues in the solid state storage controller 104 and a second type of instruction is executed while a first type of instruction is executed for one bank 214a. Adjusts the execution of instructions stored in the queues between the banks 214 of the solid state storage medium 110 to such an extent that it is executed on the two banks 214b. One or more commands are separated into queues by command type. Each bank 214 of the solid state storage medium 110 has a corresponding set of queues in the bank interleave controller 344 and each set of queues includes one queue for each command type.

The bank interleave controller 344 coordinates the execution of instructions stored in the queues between the banks 214 of the solid state storage medium 110. [ For example, a second type of instruction is executed for the second bank 214b while the first type of instruction is being executed for one bank 214a. In general, the command types and queue types include read and write commands and queues 410 and 412, but may also include other commands or queues that are storage medium specific. For example, in the embodiment shown in FIG. 4, erase and management queues 414 and 416 are included and will be suitable for flash memory, NRAM, MRAM, DRAM, PRAM, and the like.

For other types of solid state storage media 110, other types of instructions and corresponding queues may be included without departing from the scope of the present invention. The flexible nature of the FPGA solid state storage controller 104 allows the flexibility of the storage medium. The bank interleave controller 344, the storage bus controller 348 and the MUX 350 are coupled to the data pipelines 106 and 108 and other solid state storage controllers 104 ) ≪ / RTI > functionality without significantly affecting the media type.

4, the bank interleave controller 344 includes, for each bank 214, a read queue 410 for reading data from the solid state storage medium 110, a solid state storage medium < RTI ID = 0.0 > An erase queue 414 for erasing the erase blocks of the solid state storage device, and a management queue 416 for management instructions. The bank interleave controller 344 also includes corresponding read, write, erase, and management agents 402, 404, 406, 408. In other embodiments, control and status registers 340 and control cues 342 or similar elements may be used to store instructions for data transmitted to banks 214 of solid state storage medium 110 without bank interleave controller 344. [ .

The agents 402, 404, 406, 408, in one embodiment, direct the appropriate type of command that is scheduled for the particular bank 214a to the correct queue for the bank 214a. For example, read agent 402 may receive a read command for bank-1 214b and direct a read command to bank-1 read queue 410b. The recording agent 404 may receive a write command to write data to the bank-0 214a location of the solid state storage medium 110 and then write a write command to the bank-0 write queue 412a Lt; / RTI > Similarly, the cancellation agent 406 may receive an erase command to erase the erase block of bank-1 214b and then pass an erase command to the bank-1 erase queue 414b. The command agent 408 generally receives management instructions, status requests, and similar commands, such as a reset command or a request to read the type register of the bank 214, such as bank-01 214a. The management agent 408 sends an administrative command to the bank-0 management queue 416a.

The agents 402, 404, 406 and 408 also generally monitor the status of the queues 410, 412, 414 and 416 and the queues 410, 412, 414 and 416 are complete, Or other messages when it is unable to do so. In one embodiment, the agents 402, 404, 406, 408 receive the command and generate a corresponding sub-command. In one embodiment, agents 402, 404, 406, 408 receive commands via control and status registers 340 and provide corresponding sub-commands to be forwarded to queues 410, 412, 414, . Those of skill in the art will recognize other functions of the agents 402, 404, 406, 408.

Cues 410, 412, 414, and 416 typically receive instructions and store instructions until it is required to be transmitted to solid state storage banks 214. In a typical embodiment, the queues 410, 412, 414, and 416 are first-in, first-out ("FIFO") registers or similar devices that operate in a FIFO. In another embodiment, the queues 410, 412, 414, and 416 store instructions in order of matching data, order of importance, or other criteria.

The bank controllers 418 generally receive instructions from the queues 410, 412, 414, 416 and generate appropriate sub-instructions. For example, the bank-0 write queue 412a may receive instructions to write one page of data packets to the bank-0 214a. The bank-0 controller 418a may receive one or more write sub-commands for each data packet stored in the write buffer 320 to be able to receive the write command at the appropriate time and to be written to the page of the bank-0 214a . For example, the bank-0 controller 418a checks the status of the bank-0 214a and the solid state storage array 216, selects a suitable location for recording one or more data packets, To clean input buffers in array 216, to move one or more data packets to input buffers, to program input buffers to selected locations, to verify that data has been programmed correctly, and if a program failure occurs, Intercepting the physical location 224, retrying the write to the same physical location, and retrying the write to a different physical location. Furthermore, along with the exemplary write command, the storage bus controller 348 is configured such that one or more instructions are mapped to a first physical address for storage I / O buses 210a, and for storage I / O buses 210b Increasing to each storage I / O buses 210a-n having the logical address of the instruction mapped to the second physical address, and the like as further described below.

In general, the bus arbiter 420 selects one of the bank controllers 418 and fetches the sub-instructions from the output queues within the bank controllers 418 and transfers them to the storage bus controller 418 in a sequence that optimizes the performance of the banks 214. [ RTI ID = 0.0 > 348 < / RTI > In another embodiment, the bus arbiter 420 can respond to high level interrupts and change the normal selection criteria. In another embodiment, the master controller 224 may control the bus arbiter 420 via the control and status registers 340. Those skilled in the art will appreciate other means by which the bus arbiter 420 may control and interleave sequences of instructions from the bank controllers 418 to the solid state storage medium 110.

The bus arbiter 420 generally coordinates the selection of the appropriate command and the corresponding data when the command type is requested from the bank controllers 418 and sends commands and data to the storage bus controller 348. The bus arbiter 420 generally also sends a command to the storage control bus 212 to select the appropriate bank 214. For the case of another solid state storage medium 110 having a flash memory or asynchronous bidirectional serial storage I / O bus 210, only one instruction (control information) or set of data can be transmitted at a time. For example, when a write command or data is transferred to solid state storage medium 110 on storage I / O bus 210, a read command, read data, erase command, management command, It can not be transmitted to the I / O bus 210. For example, when data is being read from the storage I / O bus 210, the data can not be written to the solid state storage medium 110.

For example, during a write operation to bank-0, bus arbiter 420 may have a write command or series write sub-command at the top of its queue causing storage bus controller 348 to execute the following sequence 0.0 > 418a < / RTI > Bus arbiter 420 selects bank-0 214a via storage control bus 212 and clears input buffers of solid state storage elements 216, 218, 220 associated with bank- And sends a write command to the storage bus controller 348 that constitutes the write command by sending a command to acknowledge the state of the solid state storage elements 216, 218, 220 associated with the bank-0 214a Forward. The storage bus controller 348 is then coupled to the storage I / O bus 348 having a physical address including the address of a logical erase block for each physical erased solid state storage element 216a-m, respectively, when mapped from a logical erase block address. O bus < RTI ID = 0.0 > 210 < / RTI > The storage bus controller 348 then multiplexes the write buffer 320 through the write sync buffer 308 to the storage I / O bus 210 via the MUX 350 and streams the write data to the appropriate page. When the page is filled, then the storage bus controller 348 causes the solid state storage elements 216a-m associated with the bank-0 214a to be placed in the memory cells in the solid state storage elements 216a- Lt; / RTI > Finally, the storage bus controller 348 checks the status to ensure that the page has been programmed correctly.

The read operation is similar to the above-described recording example. During a read operation, bus interposer 420, or other element of bank interleave controller 344, typically receives data and corresponding status information and transmits data to read data pipeline 108, And transfers the status information to the registers 340. In general, a read data command forwarded from the bus arbiter 420 to the storage bus controller 348 causes the MUX 350 to read the read data for the I / O bus 210 into the read data pipeline 108. [ And will send state information to the appropriate control and status registers 340 via state MUX 422. [

Bus arbiter 420 coordinates various command types and data connection modes so that only the appropriate command type or equivalent data is loaded on the bus at any given time. If the bus arbiter 420 selects a write command and the write subcommand and corresponding data to be written to the solid state storage medium 110, the bus arbiter 420 may be coupled to the storage I / O bus 210 It will not allow other command types for. Advantageously, the bus arbiter 420 is operable to adjust the execution of various commands to the bus, with the aim of minimizing or eliminating the idle time of the buses, Timing information such as execution time is used.

The master controller 224 via the bus arbiter 420 generally uses the expected completion time of the instructions stored in the queues 410, 412, 414 and 416, together with the status information, When being executed for one bank 214a, another sub-command of another instruction is executed for the other banks 214b-n. When one instruction is fully executed for the bank 214a, the bus arbiter 420 directs another instruction to the bank 214a. The bus arbiter 420 may also adjust commands stored in the queues 410, 412, 414, and 416 with other commands not stored in the queues 410, 412, 414,

For example, the erase command may be sent externally to erase a group of erase blocks in the solid state storage medium 110. For example, The erase command may take 10 to 1000 times more execution time than the write or read command, or it may take 10 to 100 times more execution time than the program command. For N banks 214, the bank interleave controller 244 may divide the erase command into N instructions to erase the virtual erase blocks of the bank 214a, respectively. While bank-0 214a is executing an erase command, bus arbiter 420 may select another command for execution on other banks 214b-n. Bus arbiter 420 may also operate in conjunction with other elements such as storage bus controller 348, master controller 224, etc. to coordinate instruction execution between the buses. Execution of instructions using bus arbiter 420, bank controllers 418, queues 410, 412, 414 and 416, and agents 402, 404, 406 and 408 of bank interleave controller 344 Can dramatically increase performance across other solid state storage systems without bank interleaving.

In one embodiment, the solid state controller 104 includes one bank interleave controller 344 that handles all storage elements 216, 218, 220 of the solid state storage medium 110. In another embodiment, the solid state controller 104 includes a bank interleave controller 344 for each column of storage elements 216a-m, 218a-m, 220a-m. For example, one bank interleave controller 344 handles one column of storage elements (SSS 0.0-SSS N.0.02 216a, 218a, ... 220a), and the second bank interleave controller 344 2 columns of storage elements (SSS 0.1-SSS N.1 216b, 218b, ... 220b).

Storage specific elements

The solid state storage controller 104 includes a synchronization buffer 346 that is transmitted from the solid state storage medium 110 and buffers received commands and status messages. The synchronization buffer 346 is located at the boundary between the solid state storage clock domain and the local bus clock domain and provides buffering for evaluating the clock domain difference. The synchronization buffer 346, the write synchronization buffer 308, and the read synchronization buffer 328 may be independent or may work together to buffer data, commands, status messages, and the like. In one embodiment, the synchronization buffer 346 is located where there are the smallest number of signals traversing the clock domains. Those skilled in the art will recognize that synchronization between clock domains may be arbitrarily shifted to other locations within the solid state storage device 102 to optimize the design performance of some aspects.

The solid state storage controller 104 is coupled to the solid state storage medium 110 and receives instructions from the solid state storage medium 110 to determine the states received from the solid state storage medium 110 based on the instructions for the data read therefrom and the type of solid state storage medium & And a storage bus controller 348 that interprets and translates the messages. For example, the storage bus controller 348 may have different timing requirements for different types of storage devices, storage devices having different performance characteristics, storage devices from different manufacturers, and the like. The storage bus controller 348 also sends control commands to the storage control bus 212.

In one embodiment, the solid state storage controller 104 includes a MUX 350 that includes one array of multiplexers 350a-n, wherein each multiplexer is coupled to the solid state storage array 216 in one column . For example, the multiplexer 350a is coupled to the solid state storage elements 216a, 218a, and 220a. The MUX 350 routes commands from the storage bus controller 348 to the solid state storage medium 110 via the data and storage I / O buses 210 from the write data pipeline 106, Via the read data pipeline 108 and the storage bus controller 348, the synchronization buffer 346, and the bank interleave controller 344 from the solid state storage medium 110 via the controller 210 and the control and status registers 340 RTI ID = 0.0 > and / or < / RTI >

In one embodiment, the solid state storage controller 104 includes a MUX 350 for each column of solid state promoters (e.g., SSS 0.0 216a, SSS 1.0 218a, SSS N.0220a). The MUX 350 combines the data from the write data pipeline 106 with the instructions sent to the solid state storage medium 110 via the storage I / O bus 210 and sends the read data to the read data pipeline 108 To separate the data being processed. The packets stored in the write buffer 320 are transferred to the MUX 350 for each column of solid state storage elements SSS 0.x to SSS Nx 216,218, 0.x to SSS Nx 216, 218, and 220, respectively, from the write buffer 320 via the write synchronization buffer 308. The instruction and the read data are received by the MUXes 350 from the storage I / O bus 210. The MUXs 350 also indicate status messages to the storage bus controller 348. [

The storage bus controller 348 includes a mapping module 424. The mapping module 424 maps the logical address of the erase block to one or more physical addresses of the erase block. For example, a solid state storage medium 110 having 20 storage elements (e.g., SSS 0.0 to SSS 0.M 216) in one arrangement per bank 214a may be configured to store 20 physical addresses of the erase block, It may have a logical address for a specific erase block mapped to one physical address per device. Because the storage elements are connected in parallel, the erase blocks in the same location of each storage element in the storage elements 216a, 218a, 220a of one column will share the physical address. (For example, in the storage element (SSS 0.0 216a)) instead of all erase blocks in the row (e.g., storage elements SSS 0.0, 1.0, ... N.0 216a, 218a, 220a) To select a block, one bank (in this case bank-0 214a) is selected.

If one erase block is damaged or unreachable, this logical to physical mapping of erase blocks is advantageous because the mapping can be changed to map another erase block. This alleviates loss of the entire virtual erase block when there is a defect in the erase block of one device. The remapping module 430 changes the mapping of the logical address of the erase block to one or more physical addresses of the virtual erase block (spread across storage elements of such arrangement). For example, the virtual erase block 1 is connected to the erase block 1 of the storage element (SSS 0.06 216a), to the erase block 1 of the storage element (SSS 0.1 216b), ..., and to the storage element 0 And the virtual erase block 2 may be mapped to the erase block 2 of the storage element SSS 1.0 218a, the erase block 2 of the storage element SSS 1.1 218b, , The storage element (1.M 218m), and the like. Or for each storage element in this arrangement up to the erase block 1 of the storage element NM 220m, the virtual erase block 1 is connected to the erase block of the storage element (SSS 0.06 02a) (1) to the storage element (SSS 0.1 216b) of the storage element (SSS 0.1 216b) and the storage element (SSS 1.0 218a) of the storage element May be mapped from each storage element of one array to one erase block so as to include up to block 1, ..., and storage element 1.M 218m.

If the erase block 1 of the storage element SSS0.02 216a experiences damage due to damage, wear or the like, or can not be used with some adition, The logical to physical mapping for the logical address pointed to the erase block 1 can be changed. If the pre-erase block (referred to as erase block 221) of the storage element (SSS 0.0 216a) is available and not currently mapped, then the remapping module 430 is in the erase block 1 ), The erase block 1 of the storage element SSS 0.2 (not shown), ..., and the storage element 0.M 216m. The mapping module 424 or the remapping module 430 may be configured to perform a predetermined procedure such as erasure block 1, virtual erase block 2, storage block erase block 2, And may map erase blocks of storage elements 216, 218, 220 in a different order based on some other criteria.

In one embodiment, erase blocks may be grouped into connection times. Grouping by connection time, which means the time to execute an instruction, such as programming (writing) data into pages of specific erase blocks, can make the instruction completion even, and as a result, Are not limited by the latest erase block. In other embodiments, erase blocks may be grouped by wear level, soundness, and the like. Those of skill in the art will recognize other factors to consider when mapping or remapping erase blocks.

In one embodiment, the storage bus controller 348 includes a status capture module 426 that receives status messages from the solid state storage medium 110 and sends status messages to the status MUX 422. In another embodiment, the storage bus controller 348 includes a NAND bus controller 428 when the solid state storage medium 110 is a flash memory. The NAND bus controller 428 directs instructions from the read and write data pipelines 106 and 108 to the correct location of the solid state storage medium 110 and adjusts the timing of command execution based on features of the flash memory, do. If the solid state storage medium 110 is another solid state storage type, the NAND bus controller 428 may be replaced by a particular bus controller for the storage type. Those skilled in the art will recognize other functions of the NAND bus controller 428.

Power Outage Management

FIG. 5A shows an embodiment of a power management device 122. FIG. In one embodiment, the power management device 122 may include a monitor module 510 and a power loss module 520. In another embodiment, the power loss module 520 includes an identification module 512, a termination module 514, a corruption module 516, and a completion module 518.

The monitor module 510, in one embodiment, initiates a power loss mode in the non-volatile storage 102 in response to a first power source that fails to provide power beyond a predetermined threshold via the first power connection 130 . The power loss mode is a mode of operation in which, in one embodiment, the power management device 122 prepares the storage device 102 to pause within the power hold time provided by the second power supply 124. [ The power loss module 520, in one embodiment, controls the execution of operations within the process for the non-volatile storage device 102 during the power loss mode to allow operations within the necessary processes to be performed.

In one embodiment, power above a predetermined threshold is sufficient for the storage device 102. Sufficient power is, in one embodiment, the power that meets the requirements for storage device 102 to operate properly. The predetermined threshold is, in other embodiments, set to an insufficient power level for the storage device 102 or higher. Insufficient power is power that does not meet the requirements for storage device 102. When DC is expected, high AC or high frequency components and power with too low a voltage or current are examples of insufficient power. As described above, in one embodiment, the storage device 102 is configured to automatically receive or otherwise draw power from the second power supply 124 when the power from the first power source falls below a predetermined threshold . The predetermined threshold is, in one embodiment, the engineering limit determined by the characteristics of the second power supply 124 and corresponding circuits.

The first power source, in one embodiment, is the source of power used by the non-volatile storage device 102 during normal operation, which provides a substantially continuous supply of power that is not unexpectedly uninterrupted during normal operation. For example, in typical embodiments, a computer system (i.e., client 114 or the like) to which storage device 102 is attached is a first power source and may be a bus such as PCI, PCIe, AGP, Through a motherboard, such as through a slot connection, or through an external port, such as a USB port, a FireWire port, an eSATAp port, or the like. In another embodiment, the first power source is a standard electrical output.

In one embodiment, the monitor module 510 monitors the first power connection 130 directly to determine when the power from the first power source falls below a predetermined threshold. For example, the monitor module 510 may include a power sensor, a current sensor, and / or other suitable sensors for use in determining whether the non-volatile storage device 102 receives sufficient external power. In other embodiments, the monitor module 510 may be informed by other devices in the non-volatile storage device 102 that the non-volatile storage device 102 has lost external power.

In one embodiment, the monitor module 510 includes analog circuitry responsive to a loss of power from the first power connection 130. For example, the first power connection 130 and the second power supply 124 may be configured such that the second power supply 124 is fully charged (e.g., when the second power supply 124 is made of capacitors) And the first power connection 130 may be arranged in parallel to supply power to the storage device 102. In a parallel configuration, the second power supply 124 naturally begins to provide power in the event of a failure of the first power connection 130, and the storage device 102 is configured to power up the power from the second power supply 124 Naturally. The monitor module 510 circuitry may also provide appropriate isolation to ensure that power from the second power supply 124 is delivered to the storage device 102; For example, a diode may be used to ensure power flow from the second power supply 124 to the storage device 102, not to the failed first power supply, in the event of a failure of the first power supply. Methods for proper insulation will be understood by those skilled in the art in light of the present invention.

The monitor module 510 may in this embodiment detect power distribution and in response thereto initiate a power loss mode to cause the power management device 122 to operate other modules (such as current sensors, voltage sensors, ≪ / RTI > or the like) sensing elements. In another embodiment, the monitor module 510 can sense a power distribution signal and can be configured to power up the storage device 102 from the first power connection 130 to the second power supply 124, Can be activated.

The monitor module 510 may, in one embodiment, initiate a power loss mode by communicating directly or indirectly with the power loss module 520 and / or other modules that have entered the power loss mode. For example, in various embodiments, the monitor module 122 may set a status register, send a power loss mode command, send a power loss signal, send a power loss interrupt, Initiates the process, places / places the storage device in a power loss state, or otherwise notifies the power loss mode of the power loss module 520.

The power loss module 520 may be coupled to the storage device 102 during the power loss mode to ensure that, in one embodiment, essential operations, such as operations approved by the client 114 or the like, To control the execution of operations within the process. Operations within the process, in one embodiment, include operations in which the storage device 102 is currently executing. In other embodiments, operations within the process are queues for execution on the storage device 102 and operations within the write data pipeline 106 and / or the read data pipeline 108, or the like, . In the illustrated embodiment, the power loss module 520 includes an identification module 512, a termination module 514, and a corruption module 516.

The identification module 512 is configured to determine whether the external power is lost, under the predetermined threshold, or otherwise insufficient in one embodiment, and in response to the monitor module 510 determining whether to enter the power loss mode, And identifies one or more non-essential acts on the storage device 102. Non-essential operations are those operations that can be terminated, stopped, or stopped without causing data corruption or data loss to the storage device 102. The essential operations include data corruption, data loss to the storage device 102, or communications that are inconsistent between the storage device 102 and the client 114 (i.e., data that is not handled properly and does not match the notification To send a notification to the client 114). The identification module 512 may further determine whether to perform nonessential operations, or whether they are queued and waiting to be executed.

Termination module 514, in one embodiment, terminates nonessential operations identified by identification module 512. [ Termination module 514 may be configured to, in various embodiments, delete and / or erase non-essential operations, commands, and instructions that form a queue, or by blocking non-essential operations currently being performed on storage device 102 You can terminate nonessential actions. In one embodiment, the termination module 514 is configured such that power to the storage device 102 is powered off (i.e., once the power hold time expires, the second power supply 124 is reduced) ). In another embodiment, the termination module 514 terminates nonessential operations in a manner such that nonessential operations are not executed or restarted once the storage device 102 is powered back up after a power loss. For example, in one embodiment, the termination module 514 terminates nonessential operations without leaving a record of nonessential operations that have been terminated, such that the storage device 102 executes nonessential operations that have been terminated Do not do it or get power without restarting.

In one embodiment, the identification module 512 also manages the power budget for the storage device 102 while the storage device 102 is operating with the second power supply 124. The identification module 512 may determine, for example, how much power is available, how much power is needed and preferred for all operations that occur to the storage device 102. Thus, the operations may be refined and executed in order of priority to perform operations in the process, at least essentially, within the power maintenance time. In one embodiment, if the identification module 512 determines that there is insufficient power to perform all write operations (i.e., program operations for non-volatile solid state storage) due to possible errors or failures, The identification module 512 will probably record this information to the user or some or all of the recording operations after the power is restored to provide notification to the lost system.

In one embodiment, non-essential operations include erase operations that erase non-volatile memory 110 in non-volatile storage 102 and / or erase operations that read data in non-volatile storage 102 . The erase operation can occur, for example, as part of an unnecessary data collection operation that reuses space in a solid state storage device, such as a flash memory device. Non-essential operations may also include generating a hash key for data in non-volatile storage 102, operations such as decompressing the data read from the storage, or other operations. Non-essential operations may, in other embodiments, include write (or program) operations in which the non-volatile storage device 102 did not send a notification to the client 114. [ In one embodiment, the user or system designer specifies that certain actions are necessary and that no actions are necessary.

In certain embodiments, the termination module 514 terminates operations that are not essential based on how much power they require. For example, erase operations of solid state storage devices tend to consume a significant amount of power. Termination module 514 may expedite the erase operations to conserve power. Conversely, read operations require relatively little power. Termination module 514 may only begin to terminate the read operations only after the erase operations have ended or after a similar one.

In one embodiment, the identification module 512 determines the priority of the operations as a priority based on the importance of performing the operation. For example, program operations for the notified data may be given the highest priority, while the erase operation may be given the lowest priority. The termination module 514 may begin to terminate the operations with the lowest priority and may move the priority list of operations and may not terminate any essential operations. Thus, the termination module 514, which initiates operation with the lowest priority, determines whether operation is required. If not, this operation is terminated. If operation is required, operation is not terminated and termination module 514 moves to the next operation for consideration.

In certain embodiments, the identification module 512 may also determine the priority of non-essential operations in the course of execution based on the amount of energy required to complete the non-essential operation. For example, a 90% complete erase operation may be given a lower priority than a 5% complete erase operation; Thus, an erase operation of 90% may be allowed to complete, while a 5% complete erase operation is halted when a power failure is detected. In one embodiment, the amount of energy required for operation may change over time during which the operation is performed over time.

Termination module 514, in one embodiment, terminates nonessential operations identified by identification module 512. [ As noted above, the termination module 514 may be one that has been prioritized by the identification module 512 for termination prior to other operations (such as power-concentrated erase operations or proprietary grooming operations) The operations can be terminated. In one embodiment, the termination module 514 terminates non-essential operations by identifying memory regions or elements that are actuated / executed as discussed in more detail with respect to FIG. 6 and resetting memory regions or devices . As used herein, a memory region or element is a physical or logical entity of a non-volatile memory 110 that is capable of being executed in a physical section, reset to a command or signal, terminated, stopped, pending, Points to a section.

By terminating nonessential operations, the power management device 122 can ensure that the power is used for the necessary write operations and other essential operations, such that essential operations can be performed within the power hold time. Moreover, the power management device 122 may thus reduce the overall amount of power required by the second power supply 124 to provide. The designers therefore use smaller capacitors that provide power, which can save space in the storage device 102, for example, reduce costs, and improve reliability, so that all received and notified data is unexpected It is permissible to choose to maintain performance to ensure that it is preserved and protected from power outages.

In one embodiment, the termination module 514 determines whether a particular non-essential operation is queuing or running. Termination module 514 may delete non-essential queuing operations by removing operations from the queue to ensure that operations are not being performed. Or, in addition, the termination module 514 may cancel running operations to prevent the executing operations from consuming additional power. In certain embodiments, as noted above, termination module 514 allows other things to be completed while terminating some nonessential operations between processes.

Damage module 516 identifies data received by storage device 102 that, in one embodiment, is written to non-volatile memory 110 that should be considered corrupted or damaged. Such data may be, for example, data in the record data pipeline 106. The corruption module 516 ensures that the data that is believed to be corrupted is not stored in the non-volatile memory 110 and also assures that the client 114 is aware that the data has not been stored, Is not notified of a successful storage.

In certain embodiments, the corruption module 516 and the termination module 514 record the actions taken once the monitor module 510 detects a power failure. For example, the termination module 514 may record whether any nonessential tasks have been canceled before they begin execution, and whether any nonessential tasks have been terminated during execution. The corruption module 516 may record information as to what data was determined to be corrupted. Other modules of the power management device 122 may similarly have their activities, or their subservices, to assist in determining that the storage device 102, client 114, or other related entity has occurred during an unexpected outage. Set can be recorded.

In one embodiment, the corruption module 516 may detect that the power failure signal received by the storage device 102 initiated at some specific time in the past (e. G., 5 microseconds) before the power failure signal is received by the monitor module 510 It is anticipated that all data should be corrupted and not stored in the non-volatile storage 110. The present specification may be described by standards such as PCI, PCI-e, or the like, or by the client 114, the storage device 102, the vendor, the manufacturer, In another embodiment, corruption module 516 evaluates the data in the write data pipeline 106 before a predetermined stage of corruption.

Figure 5B shows an example of a timetable associated with data corruption. At time zero, a power disturbance occurs. This point is called disturbance time. There is a resulting period of time when the resulting power disturbance and power disturb signals are received by the monitor module 510, when the monitor module 510 detects a power disturbance, or the like. This period is called the impairment period here. Although the present specification can provide exemplary damage periods (such as the 5 microseconds mentioned above), the damage period is not limited to this and can be varied based on the assumptions of standards, developers, manufacturers, designers, etc. have.

The impairment period detects a power disturbance (which appears to have occurred at 5 microseconds), generates a signal indicating that there was a power disturbance (which appears to occur at 3 microseconds) Is the result of the time required for the monitor module 510 to receive the power disturb signal. Generally, the corruption module 516 prevents new data from entering the write data pipeline 106 once it is determined that there has been a power disturbance when new data is considered corrupted. However, the corrupted data is moved to the write data pipeline 106 during the corruption period.

Therefore, all data received after the corruption time is considered corrupted and should not be stored. For example, the corruption module 516 may determine whether the monitor module 510 has received an outage signal at time t, and the corruption module 516 may always set the corruption time at t-5 microseconds. Damage module 516 may therefore conclude that all data received after the damage time of t-5 microseconds is corrupted. In such an embodiment, damage module 516 identifies all write operations (i.e., program operations for the flash memory, etc.) received after t-5 microseconds and determines where they are in the write data pipeline 106 And skips the recording operations. Damage module 516 may, in various embodiments, skip write operations by canceling them, skipping them, cleaning them, blocking them, or otherwise not executing them.

In one embodiment, the power management device 122 also includes a completion module 518. In some implementations, certain operations associated with the stages of the write data pipeline 106 will not or will not allow a continuous flow of data through such a pipeline until the buffer associated with that stage is filled. For example, an ECC stage such as ECC generator 304 of FIG. 3 may require a buffer filled before generating an ECC value. Likewise, an output buffer such as write buffer 320, write synchronization buffer 308, or the like must be populated before data is moved out of the output buffer and moved to non-volatile storage 110. In one embodiment, if the buffer is partially filled under normal conditions, the stage associated with the buffer will wait until the buffer is filled before the operations associated with this buffer are executed. The buffers referred to herein may be physical buffers, or may simply be temporary storage locations such as registers, DRAM locations, or others. In another embodiment, the packetizer 302 may pass a packet to another stage in the record data pipeline 106 until the packet is complete, until one group of packets is complete, or the like . Similarly, the write buffer 320, in certain embodiments, can not transfer data to the storage device 102 until pages, logical pages, a group of pages, logical pages, or the like are complete .

In the event of a power outage, data is moved through the write data pipeline 106, even if the buffer, packet, or page is not filled in one or more stages, to flush the data to non-volatile memory 110, Can be useful. The completion module 518 flushes the data through the write data pipeline 106 and from the partially filled data buffer to the non-volatile memory 110. In one embodiment, the completion module 518 identifies partially filled buffers, packets, and / or unfilled pages, and stores the pad data < RTI ID = 0.0 > To fill the buffers.

The completion module 518, in one embodiment, may be a pad data that ensures that the storage device 102 and / or the client 114 can identify the fill and that the pad data is not part of the actual data Lt; / RTI > In one embodiment, completion module 518 uses a specific header, token, marker, pattern, or other identifier to identify the filling data. In other embodiments, completion module 518 may use buffered data, data, and / or data that is present in the unfilled space of the buffer to complete the partially filled buffer, packet, and / And / or flushes the page. For example, an unfilled or empty buffer may store all binary 1's, all binary 0's, discarded or unneeded data, data from previous processing, or the like. Completion module 518, in one embodiment, identifies data that is present in the unfilled area of the buffer as filling data. Completion module 518 may use a particular pattern, flag or other indicator, or other methods known to those skilled in the art in light of the present invention.

The completion module 518 uses a different identifier to identify whether a particular header, footer, token, marker, pattern, or power management device 122 has successfully completed the necessary operations in the power loss mode . In one embodiment, successful completion of the essential operations is accomplished by the completion module 518 successfully flushing the write data coming from the write operations through the write data pipeline 106 and into the non-volatile memory 110 or the like . The indicator is, in one embodiment, the same indicator as described above for identifying the data to fill. In another embodiment, completion module 518 uses an individual indicator to identify a successful execution of operations that are required during a power loss mode.

FIG. 6 illustrates one embodiment 600 of power management device 122 and non-volatile memory 110. FIG. As noted above, the power management device 122 may be part of the storage controller 104. The power management device 122 and non-volatile memory 110 may be part of a physically identical storage device 102. Power management device 122 may perform the operations described with respect to FIG. 5A. In the illustrated embodiment, the power management device 122 includes a monitor module 510 and a power loss module 520. The power loss module 520 may include an identification module 512, a termination module 514, a corruption module 516, and / or a completion module 518 in other embodiments. The power management device 122 also communicates with the non-volatile memory 110 such that the power management device 122 communicates signals directly or indirectly to the non-volatile memory 110. [ The power management device 122 may enable, for example, to transmit a control signal to the non-volatile memory 110. [

FIG. 6 illustrates one embodiment of a structure for non-volatile memory 110. FIG. Non-volatile memory 110 may include channels 632a and 632b. The channels may include a plurality of banks, for example, channel 632a includes banks 630a and 630b, and channel 632b includes banks 650a and 650b. In certain embodiments, chips 634a-c and 654a-c include a plurality of die 636a-f and 656a-f. In certain embodiments, one die in each chip 634a-c and 654a-c is used to form the bank. As shown, the bank 630a surrounds the dies 636a, 636c, and 636e. Bank 630b includes dies 636b, 636d, and 636f. Banks 650a and 650b similarly comprise one die in chips 654a-c. In one embodiment, the non-volatile memory 110 of Figure 6 is substantially similar to the solid state storage medium 110 of Figure 2 described above.

It will be appreciated by those skilled in the art that the embodiment shown in FIG. 6 is merely an example of a structure for non-volatile memory 110 such as flash, and numerous other structures are possible or possible. FIG. 6 illustrates a simplified version of non-volatile memory 110 to focus on features of non-volatile memory 110 in a manner that helps to understand the present invention. More details on non-volatile memory 110 implementation may be found in "Apparatus, System, and Method for Managing Interfaces ", which is hereby incorporated by reference herein, No. 11 / 952,095, filed December 6, 2007, by David Flynn et al.

As noted above, the termination module 514 may determine the memory region or device in which the operation is being performed and may terminate the non-essential operation identified by the identification module 512 by resetting the memory region or device . As used herein, a memory area or element refers to a physical section of non-volatile memory 110 that can be reset by a reset command. A reset command is an instruction that causes all operations being performed on a memory area, such as writing, erasing, and reading, to terminate. In one embodiment, each die 636a-f and 656a-f may be independently reset such that each die 636a-f and 656a-f constitutes a particular memory region or element. The reset operation causes operation to a particular die, which is the subject of the reset operation, to terminate the process.

In certain embodiments, as described herein, operations occur on a bank basis. For example, the erase operation, in one embodiment, is performed in a logical erase block on a plurality of dice forming a bank. In such embodiments, the memory region or element may be a bank, and the reset operation is transmitted to all the die in the bank at substantially the same time. The reset operation itself may be an instruction or a plurality of instructions; In such embodiments, each die in the bank is reset, which stops erase operations for each physical block in each die of the logical erase block.

In another embodiment, the termination module 514 may reset substantially all of the non-volatile memory 110 simultaneously. For example, in one embodiment, the storage device 102 may simultaneously schedule erase operations for each bank 630a, 630b, 650a, 650b and the termination module 514 may terminate these scheduled erase operations The reset command may be sent to each of the banks 630a, 630b, 650a, and 650b in the non-volatile memory 110 to be reset.

In such an embodiment, the termination module 514 may send a reset command to the particular die 636a-f or 656a-f via the bus. This allows the termination module 514 to reset non-essential operations (such as erase) and to simultaneously resume memory areas that allow programming operations (i.e., data storage write operations) for other memory areas do. In one embodiment, the termination module 514 terminates execution of operations by issuing a reset signal, and terminates operations from the command queue, or otherwise skips operations to return pending operations (i.e., These operations in FIG.

Certain nonessential operations may be terminated without the use of a reset command. For example, as described above, non-essential operations in the command queue can be skipped simply by either not starting them at all, deleting non-essential operations, marking to prevent cleaning, have. Because these operations are never initiated, die 636a-f or 656a-f need not be reset to terminate operation. Other non-essential operations that are not performed on die 636a-f and 656a-f may likewise be terminated without a reset command even when they are being executed; For example, if the stage of the write data pipeline 106 is generating a hash key for the data when a power failure is detected, the hash generation operation may be initiated by a reset operation sent to the die 636a-f and 656a- . ≪ / RTI > In certain embodiments, only the program / write, read, and erase operations in the process of execution at particular dies 636a-f and 656a-f end with a reset command.

In certain embodiments, the termination module 514 may stop or otherwise suspend specific areas / sections / modules / sub-elements of the storage device 102. For example, the termination module 514 may suspend all physical devices / devices and / or logic modules that execute the read data pipeline 108. In another embodiment, termination module 514 may stop or otherwise terminate the read DMA engine, or other sub-elements associated with nonessential operations. Termination module 514 may also stop one or more CPUs operating in storage device 102; For example, the storage device 102 may have a multi-core CPU. In such an embodiment, termination module 514 may suspend one or more cores in the CPU that power management device 122 is not using.

Termination module 514 may also monitor and ensure that activities not related to the operations of power management device 122 do not occur in the core that supports power management device 122. [ In certain embodiments, the power management device 122 may be implemented in hardware separate from the CPU such that the termination module 514 may simply stop the CPU (or CPUs) to conserve power. Termination module 514 may suspend the read data pipeline 108 and the CPU by stopping individual clocks. Those skilled in the art will appreciate other ways of stopping the read data pipeline 108, the read DMA engine, the CPU, and / or other sub-elements of the storage device 102.

In certain embodiments, as described in the bank interleave application, certain operations may occur for the bank level; For example, data is programmed (i.e., written or stored) on dies 636a, 636c, and 636e during program operation affecting bank 630a. Banks 630a-b and 650a-b may be implemented as logical erase blocks (which comprise n number of physical erase blocks when there are n dies in the banks) Physical pages), logical pages (consisting of physical erase blocks), and the like. 6, bank 630a includes a logical erase block consisting of three physical erase blocks (from dies 636a, 636c, and 636e) and three physical erase blocks from die 636a, 636c, and 636e To provide logical pages of data comprised of pages.

In such embodiments, termination module 514 may send a reset command over the bus to die (such as die 636a, 636c, and 636e) that are connected in parallel to bank 630a. In such an embodiment, this group of dies 636a, 636c, and 636e will be reset at the same time and will effectively stop the operations occurring at each die 636a, 636c, and 636e. Thus, because the erase operation occurs in the logical erase block including the physical erase blocks in the three physical dies 636a, 636c, and 636e, the reset operation can be used to simultaneously stop the erase operation in the process for the logical erase block May be physically transmitted to the three physical erase blocks in the dies 636a, 636c, and 636e. Similarly, in other embodiments, termination module 514 may send a reset operation to all dies 636a-f, 656a-f to reset the entire non-volatile memory 110 simultaneously.

In one possible example, the monitor module 510 may determine if the storage device 102 has lost power. The identification module 512 determines whether there is an erase operation occurring in the non-volatile memory 110 corresponding to the logical erase block in the bank 650a. The termination module 514 sends a reset command to the bank 650a which causes the dies 656a, 656c, and 656e to be reset and thus terminates the erase operation. A similar pattern may occur for other erase operations and read operations that are pending for the non-volatile memory 110 after the storage device 102 has lost power. In addition, the banks may be independent of each other such that operations occurring in one bank may be terminated or stopped without affecting the operations of the other banks in the storage device 102.

In certain embodiments, program, erase, and read operations do not occur at the bank level as described above; In certain configurations, program, erase, and read operations occur individually on each die 636a-f and 656a-f. In such embodiments, the reset operation may be sent to the affected die; For example, erasing of a physical erase block on die 636b may be terminated by a termination module 514 that sends a reset command to die 636b.

Other methods may be taken to terminate the nonessential operations that are performed as identified by the identification module 512. [ In one embodiment, the termination module 514 terminates nonessential operations that are in execution by queuing nonessential operations or queuing to execute them. Certain non-volatile memory devices 110 may allow the performing operations to cease. In such embodiments, the termination module 514 may send commands to resume nonessential operations without sending successive commands and to suspend nonessential operations, effectively causing this operation to be canceled. In other embodiments, the termination module 514 may send a command to stop nonessential operations, wait until all essential program actions have been completed, and then send one or more resume commands to the various suspended operations Lt; / RTI >

Figure 7 shows one embodiment 700 of a write data pipeline 106 for power management device 122 and storage device 102. [ In one embodiment, the write data pipeline 106 is substantially similar to the write data pipeline 106 described above with respect to FIG. In the illustrated embodiment 700, the write data pipeline 106 includes an input buffer 306, a compression module 312, an encryption module 314, a packetizer 302, an ECC generator 304, And a buffer 320. In other embodiments, the write data pipeline 106 includes a write program module 310, an unwanted data collector bypass 316, a media encryption module 318, a write synchronization buffer 308, and / or other stages , ≪ / RTI >

As shown in FIG. 3, the write data pipeline 106 may be executed as part of the solid state storage (SSS) controller 104. The power management device 122, in one embodiment, may also be implemented as part of the SSS controller 104. In one embodiment, the power management device 122 may be run independently, but may communicate with the SSS controller 104. The power management device 122 may be integrated with the SSS controller 104, in other embodiments.

As discussed above, corruption module 516, in certain embodiments, is received via a PCI-e connection (or other connection depending on execution) and is received after a power failure (generally referred to herein as corrupted data) Identify data that is believed to be corrupt. The corruption module 516, in one embodiment, also ensures that the client 114 knows or should know that data that is deemed to be corrupted has not been stored in the storage device 102. In one embodiment, corruption module 516 determines the location of the corrupted data of the oldest piece in the write data pipeline 106. The corrupted data of the oldest piece is at the beginning of the received data after the corruption period has begun. Any data that leaves the corrupted data of the oldest piece and returns to the beginning of the write data pipeline 106 (e.g., input buffer 306) is considered corrupted and is removed from the write data pipeline 106.

In one embodiment, the corruption module 516 may cause the storage device 102 to delay returning to the client 114 to send the notification until after the period of time used to calculate the corruption time has elapsed. As discussed above, in certain embodiments that rely on the structure of the storage device 102 and the write data pipeline 106, the corruption module 516 may detect that the monitor module 510 has detected a power outage, It can be assumed that all the data received in the above is damaged. Thus, 5 microseconds is the period used to calculate the damage time. Thus, corruption module 516 may specify that the notification is not sent to client 114 for up to 5 microseconds after data is received by storage device 102. As a result, in certain embodiments, data is stored until the storage device 102 ensures that the data is not corrupted by power failure and has not yet been detected and / or communicated to the storage device 102 Is never notified.

In one embodiment, the corruption module 516 sends a notification once the data is managed by the buffer controller 208, but leaves the buffers prior to the data coming into the write data pipeline 106. For example, data may be transferred to buffers in the storage device 102 by a direct memory access ("DMA") engine, and then such data may be transferred to one or more buffer controllers 208, To the pipeline 106.

In one embodiment, the buffer controller 208 allows the buffer that receives data from the DMA engine to be filled, waits for the end of the corruption time, and then forwards the notification to the client 114. Once the buffer has been filled, once the period has elapsed, it is known whether the power outage has corrupted all or part of the data in the buffer and the data can be notified securely. If a power failure has occurred, the data may be removed from the buffer without being transferred to the write data pipeline 106. Moreover, if a power outage has occurred, the notification may not be transmitted to the client 114 indicating that the data has been stored. In accordance with the best practice, therefore, the client 114 must assume that the data has not been stored. In another embodiment, the potential risk of data corruption in the write data pipeline 106 may be acceptable and so the buffer controller 208 allows the buffer to be filled, the delay is not granted for the impairment time, The storage device 102 sends a notification to the client 114. [ In certain embodiments, the storage device 102 may be configured to insert a damage avoidance delay by default and not to insert a damage avoidance delay.

As a result, in such an embodiment, the corruption module 516 can prevent data corrupted by power outages from entering the write data pipeline 106 and that the storage device 102 is not corrupted It is possible to further prevent the storage device 102 from transmitting the notification until after it can be assumed.

In another embodiment, the corruption module 516 stops corrupted data in the write data pipeline 106 at the choke point. The choke point indicates that any data lying above the choke point (i. E., Between the choke point and the input buffer 306, including data in the input buffer 306) is considered to be damaged And the location of the record data pipeline 106. The location of the choke point may be determined at a rate at which data travels through the write data pipeline 106 and / or during the time period used to determine the damage time. For example, damage module 516 may assume that ECC generator 304 is in the farthest data that can be moved to write data pipeline 106, within 5 microseconds after the damage time. Thus, the ECC generator 304, in the exemplary embodiment, is the choke point of the write data pipeline 106. [ In the event of a power outage, the corruption module 516 may be further from the data in the ECC generator 304 and above the write data pipeline (i.e., the media encryption module 314, the packetizing device 302, Data pipeline 106, etc.), any data may be prevented from moving through the write data pipeline 106 and into the non-volatile memory 110. In certain embodiments, the corruption module 516 interrupts operations occurring in the write data pipeline 106 above the choke point.

In another embodiment, the location of the choke point may be determined by the location at which the record data pipeline 106 has sufficient information to write data to the non-volatile memory 110. [ For example, in one embodiment, once the packetizer 302 has header metadata added to the completed packet, the record data pipeline 106 further processes the packet (i.e., the ECC generator 304) Volatile memory 110) that has sufficient information to write a packet. The packet is, in one embodiment, the smallest writable unit of data in the write data pipeline 106. In this exemplary embodiment, the packetizer 302 is a choke point. In another embodiment, the ECC chunk or codeword is the smallest writable unit of data in the write data pipeline 106, and the ECC generator 304 may be a choke point. In one embodiment, the characteristic of the second power supply 124 is that it provides a power hold time that is sufficiently long enough for data to pass through the write data pipeline 106 from the choke point and is written to the non-volatile memory 110 .

In certain embodiments, once the data has been completely moved through the choke point in the write data pipeline 106, the corruption module 516 sends a notification of the data. Thus, the corruption data can be stopped, and actions acting on the corruption data can be stopped before the notification is sent. As a result, the client 114 does not give notice until the data in this pipeline is good and is not corrupted, in order to be stored or stored.

In certain embodiments, the data may be organized into atomic data units. For example, an atomic data unit may be a set of data, files, documents, and / or data associated with one or more logical block addresses (adjacent or non-adjacent logical block addresses) of a packet, page, logical page, logical packet, Or other group of related data. In such embodiments, the corruption module 516 may delay sending notifications until the entire atomic data unit has passed through the choke point. For example, a portion of a file may be passed through a choke point and is therefore known to be undamaged data; However, the last half of the file has not yet been passed through the choke point and can therefore contain corrupted data. In contrast to sending a notification when only a portion of the atomic data unit is moved, the corruption module 516 may wait until the entire atomic data unit has passed through the choke point before sending the notification. In one embodiment, corruption module 516 partially discards corrupted atomic data units. In another embodiment, the corruption module 516 is configured such that both the undamaged portion of the atomic data unit, or the undamaged portion and the corrupted portion of the atomic data unit are passed through the write data pipeline 106 and the non-volatile memory 110 ). ≪ / RTI > In certain embodiments, when the atomic data unit may include partial data or corrupted data, the power management device 122 may include an indicator with stored data to indicate the atomic data unit in the appropriate state have.

The corruption module 516 may be further responsible for stopping the flow of data to the write data pipeline 106 after a power failure has been detected. Thus, irrespective of whether the corrupted data is handled outside of the write data pipeline 106 or within the write data pipeline 106, the corruption module 516 may be configured so that after the power failure is detected, (106).

Completion module 518 may also act with write data pipeline 106 to ensure that undamaged and unreported data is moved through the write data pipeline 106 and stored in non-volatile memory 110 have. The modules / stages in the write data pipeline 106 may use buffers to support their operations. In certain embodiments, modules (such as modules 302-314) perform only operations once the associated buffer is filled. For example, the ECC generator 304 may wait until the buffer is filled and then generate an ECC value for the entire buffer. In one embodiment, the buffer controller 208 manages the flow of data through the buffers in the write data pipeline 106. Likewise, the write data pipeline 106 may include one or more control queues 342 for the stages in the write data pipeline 106, as described above with respect to FIG.

During normal operation, the write data pipeline 106 continues to stream data through the write data pipeline 106 to such an extent that buffers are always filled. However, in the case of a power failure, the data flow to the write data pipeline 106 may be stopped when one or more buffers in the write data pipeline 106 are only partially filled. For example, as noted above, the corruption module 516 may remove corrupted data from the write data pipeline 106 and prevent new data from flowing to the storage device 102. As a result, one or more buffers in the write data pipeline 106 may be partially filled. If the data is not moved through the write data pipeline 106, the data will be lost at the end of the power hold time once the second power supply 124 is exhausted.

In certain embodiments, the completion module 518 flushes the data through the partially filled buffers in the write data pipeline 106 during the power loss mode. The completion module 518, in one embodiment, fills partially filled buffers with the filling data. In other embodiments, as described above, the completion module 518 may flush the data without adding the filling data by using the existing values stored in the unfilled portion of the buffer as filling data, or the like have. As a result, the data and fill are operated in the buffer, moved therefrom, and moved through the write data pipeline 106. The buffers used in the write data pipeline 106 may not all be the same size; In such embodiments, the completion module 518 may monitor data as data is moved through the write data pipeline 106 and may flush additional buffers at any point where the buffer is partially filled.

In certain embodiments, the completion module 518 uses a particular marker, indicator, or header to identify data that fills in future data to prevent it from being inaccurate with the actual data. In certain embodiments, the pad sequence is configured with a value of one when "1 " is in a state prior to programming of such cells in which non-volatile memory 110 cells have occurred. For example, in a flash memory, program operations convert ones to zeros. By using a pad sequence consisting of 1s, the power required to convert 1s to 0s can be conserved. In a related embodiment, the ones that constitute the pad data do not need to be transmitted before starting program operation when the cells are already in the 1 state.

In certain embodiments, when data is moved from the write data pipeline 106 through the storage I / O bus 210 and into the non-volatile memory 110, the indicator indicates whether the data has been properly written Lt; / RTI > In certain embodiments, the indicator is inserted into the header of the packet for data and indicates whether the data in the packet that precedes the packet with the indicator has been properly written. Thus, if the packet is successfully programmed, the header of the successive packet is programmed with an indicator that the last programmed packet has been successfully programmed.

In other embodiments, the indicator is placed at the end of the packet in the footer and indicates whether the packet containing the indicator is properly recorded. In one embodiment, this is done by moving the data one bit ahead of the data into the header space. Thus, if the header is a 64-bit header, the move reduces the header space to 63 bits and adds a bit to the footer. This leaves a bit at the end of the packet that can be used as an indicator. This method maintains proper alignment while allowing each packet to indicate its own state, and in embodiments it may be sensitive to border alignment.

The indicator may include a packet filling and may be used to identify if the data is not complete accordingly and can not be used by the system. In certain embodiments, when storage device 102 is powered back after a power failure, the indicator is used to help rebuild indexes and validity maps for non-volatile memory 110.

In certain embodiments, one indicator is inserted into each atomic data unit. As mentioned above, the indicator may be placed into a footer at the end of the last packet in the atomic data unit. Thus, the indicator may indicate whether the data for the entire atomic data unit has been properly recorded. If, for example, a power failure causes only a portion of the atomic data unit to be written, and the last packet is filled as described above, the indicator will indicate that the entire atomic data unit was not properly written. Moreover, as discussed above, in certain embodiments, the notification will be sent to the client 114, in certain embodiments.

In one embodiment, the corrupted data is allowed to proceed through the write data pipeline 106 to flush the reported good data going to the non-volatile memory 110. The corrupted data can be identified by setting an indicator as described above, which flags the data as invalid / corrupted. In related embodiments, other types of indicators, such as specialized packets, headers, unique character streams, markers, and similar methods known to those skilled in the art, It can be replaced with the indicator described above to invalidate the data. In all such cases, the corrupted data should never be communicated to the client 114.

As described above with respect to FIG. 5, the completion module 518 is configured to determine whether a unique header, footer, token, marker, pattern, or power management device 122 is present in the record data pipeline 106 or A similar indicator is used to successfully flush the write data and to indicate whether it has successfully completed essential operations in a power loss mode, such as successfully storing data in the non-volatile memory 110 during power hold time. The indicator is, in one embodiment, the same indicator described above that identifies the corrupted data, the data to fill, or the like. In another embodiment, completion module 518 uses an individual indicator to identify a successful execution of operations that are required during a power loss mode.

In one embodiment, the atomic data unit is associated with a plurality of non-contiguous / non-contiguous or corrupted logical block addresses or other indicators that the write data pipeline 106 treats as a single atomic data unit. As used herein, writing non-contiguous / non-contiguous or damaged logical blocks to a single recording operation is referred to as atomic recording. In one embodiment, the hardware controller processes operations in a received order and the software drivers of client 114 transmit operations to the hardware controller for single atomic writes, such that the write data pipeline 106 typically includes atom The recording operation can be processed. Because the hardware is processed in order, this ensures that different logical block addresses or other identifiers for a given atomic record go through the write data pipeline 106 to the non-volatile memory 110 together. In one embodiment, because the termination module 514 does not terminate the notified write operations, the atomic writes reported are successfully stored in the non-volatile memory 110, and if the client 114 does not receive the notification, (114) may detect if atomic writing has failed due to power loss or the like. The client 114 may, in one embodiment, cancel, reprocess, or otherwise handle failed atomic records and / or other failed or terminated operations during recovery if power is once recovered.

In one embodiment, the software driver in client 114 may mark blocks of atomic writes having metadata flags indicating whether a particular block is part of atomic writes. One example of metadata marking relies on the log record / attach protocol of non-volatile memory 110, along with metadata flags, or the like. The use of an attachment-only log to store data and the prevention of arbitrary interleaved blocks enables the atomic write membership metadata to be a single bit. In one embodiment, the flag bit may be zero if the block is not a member of atomic recording, then the bit may be one or vice versa. If the block is a member of atomic recording and is the last block of atomic recording, in one embodiment, the metadata flag may be zero to indicate whether the block is the last block of atomic recording. In another embodiment, different hardware instructions may be sent to mark different headers for atomic recording, such as the first block of atomic records, intermediate member blocks of atomic recording, tail of atomic recording, or the like.

In one embodiment, storage controller 104, power management device 122, or the like, in a deterministic direction, during recovery from power loss or other failure of client 114 or storage device 102, (For example, in one embodiment, the beginning of the log is the tail, the end of the log is the head, and the data is always added to the head). In one embodiment, the power management device 122 scans from the head of the log toward the tail of the load. In other embodiments, the power management device 122 may scan from the tail of the log toward the head of the log, once from the tail and from the head to the tail once, or otherwise scan the log for recovery can do. For atomic recording recovery, in one embodiment, when scanning heads in the tail, if the metadata flag bit is zero then the block is either a single block atomic record or a non-atomic write block. In one embodiment, once the metadata flag bit changes from 0 to 1, the previous scanned block and the potentially scanned current block are members of the atomic record. The power management device 122 continues to scan the log, in one embodiment, until the metadata flag changes back to zero at this point in the log, and the previous scanned block is the last member of the atomic record, Is the first block stored for recording.

In one embodiment, the non-volatile memory 110 uses a log-based, attach-only record structure recording system where new records continue in front of the log (i.e., at the head of the log). In another embodiment, the storage controller 104 re-uses invalid, invalid, or corrupted blocks of logs using a garbage collection system, a rumor, a cleaner agent, or the like. The storage controller 104, in another embodiment, uses a forward map to map logical block addresses to physical addresses to facilitate use of attachment-only recording structures and unnecessary data collection.

The storage controller 104, in another embodiment, uses a data structure such as an inflate tree, or the like, to track write operations within the process during normal operation of the storage device 102. [ The inflate tree is, in one embodiment, a data structure that maintains a record of block storage requests received by storage device 102 but not yet completed (in particular write requests). The power management device 122, in one embodiment, ensures that for a single block write, the write is guaranteed to be complete even if power is lost.

In the illustrated embodiment 700, the packetization device 302 includes an incomplete packet 702 and a completion packet 704. In one embodiment, if the incomplete packet 702 is at the end of the atomic data unit, the corruption module 516 may send a notification to the client 114 about the data in the incomplete packet 702 and the completion packet 704 have. During a power loss mode, in one embodiment, completion module 518 flushes incomplete packet 702 from packetizer 302. As described above, in certain embodiments, completion module 518 may add a marker indicating the end of valid data in unfinished packet 702, add data to fill packet 702, / Else, or otherwise flush incomplete packets 702 from the packetization device 302.

In another embodiment, if the incomplete packet 704 is at the end of the atomic data unit and the incomplete packet 702 is from an incomplete atomic data unit, the corruption module 516 sends the data of the completion packet 704 to the client But does not notify client 114 of the data in incomplete packet 702. [ During the power loss mode, in one embodiment, the termination module 514 may discard the incomplete packet 702 with the unacknowledged data, and may send one or more (e.g., one or more) packets 702 associated with the incomplete packet 702 Operations can be skipped.

In the illustrated embodiment, the write buffer 320 includes one incomplete page 708 and two complete pages 710,712. In one embodiment, pages 708, 710, 712 include logical pages as described above. The completion module 518 flushes one or both of the packets 702 and 704, in one embodiment, through the ECC generator 304 and into the write buffer 320 during the power loss mode.

In one embodiment, the write buffer 320 generally writes completed pages 710 and 712 to the non-volatile memory 110 during a power loss mode. In another embodiment, the termination module 514 may terminate / terminate or reset one or more non-essential operations for the non-volatile memory 110 such that the write buffer 320 may be a non-volatile memory 710 < / RTI > The completion module 518 flushes the incomplete page 708 from the write buffer 320 to the non-volatile memory 110 in one embodiment so that the non-volatile memory 110 is in an incomplete page 708 ). As described above, in various embodiments, the completion module 518 may add a marker indicating the end of valid data in the incomplete page 708, and may add data to fill the incomplete page 708 And may otherwise flush the incomplete page 708 from the write buffer 320, if not.

FIG. 8 illustrates an embodiment of a method 800 for power loss management of storage device 102. FIG. The method 800 begins and the monitor module 510 determines 802 if the power from the first power connection 130 is below a predetermined threshold. In the illustrated embodiment, if the monitor module 510 determines 802 that the power from the first power connection 130 is not lower than the predetermined threshold 802, then the monitor module 510 receives the power from the first power connection 130 Continuous monitoring of the amount of power drawn (802).

In the depicted embodiment, if the monitor module 510 determines 802 that the power from the first power connection 130 is below a predetermined threshold 802, then the monitor module 510 may determine that the power loss mode (804). The storage device 102 receives 806 power from the second power supply 124 during at least the power hold time during the power loss mode. The power loss module 520, in the illustrated embodiment, controls (808) the execution of operations within the process for the storage device 102 during the power loss mode so that operations within the essential processes are performed within the power maintenance time, The method 800 ends.

Figure 9 illustrates one embodiment of a method 900 for improved storage operation during a power failure. The method 900 begins with monitoring 902 power to the storage device 102. In one embodiment, the monitor module 510 monitors power to the storage device 102. The method 900 also includes determining (904) whether power to the storage device 102 has been powered off, falls below a predetermined threshold, or the like.

If power to the storage device 102 has not been interrupted, the monitor module 510 continues to monitor power to the storage device 102 for interrupts. In the case of interception, the method includes identifying (906) incomplete operations for storage device (102). In one embodiment, identification module 512 identifies unfinished operations (906). In certain embodiments, the identification module 512 processes only erase operations, read operations, and program operations. In certain embodiments, other types of operations are also identified.

In the illustrated embodiment, if unfinished operations are read or erase operations, then the identification module 512 determines if any read operations and erase operations are currently being performed (i.e., those currently occurring for non-volatile memory 110) (908). ≪ / RTI > For these read and erase operations that are currently being executed, the end module 514 resets (910) the affected memory region and sends a reset command to cancel the associated operation. As discussed above, the termination module 514 may first perform these actions in accordance with the system, and may instead be selected to allow certain actions that are almost complete to be completed.

If unfinished read / erase operations are not currently being performed, termination module 514 may simply cause operations to be canceled (914) or otherwise skipped. For example, operations may queue up with one or more command queues and wait execution. Termination module 514 may remove read and erase operations from the queues such that they are not executed. Or termination module 514 may cause operations to be ignored or skipped; That is, the operations may be left in the queue, but may not be selected for execution. In other embodiments, the termination module 514 may ignore one or more non-essential command queues that maintain non-essential operations and may operate for execution from one or more mandatory command queues that maintain essential operations, or the like Can be selected.

If the incomplete operation is a program operation, the identification module 512 may determine 912 whether a notification has been sent to the client 114 or not. If a notification has been sent, the termination module 514 may be selected to cancel the queued operation as described above or to reset the affected memory area. In other embodiments, program operations may be allowed to be completed regardless of whether a notification has been sent or not, if they were in the storage device 102.

If a notification has been sent, program operation is allowed to complete (916). As a result, the data associated with program operation is transferred to the non-volatile memory 110 as reported to the client 114. As discussed above, the corruption module 516 may remove corrupted data from the write data pipeline 106 as part of the method 900. Similarly, the completion module 518 may flush partially filled buffers to ensure that the programmed data is moved through the write data pipeline 106. As discussed above, the corruption module 516 and / or the completion module 518 may cause the indicator to be set, which identifies the corrupted data in the storage device 102.

By reducing the number of operations performed by the non-volatile storage device 102 during a power outage, the size, cost, and complexity of the second power supply 124 can be reduced. In certain embodiments, the focus is on low-cost / high-power operations such as erasures that consume significant power, which is not particularly critical. The system 100 can further distinguish between essential programs (those for which notifications are sent to the client 114) and those that are not essential (those for which the notifications are not sent).

The invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes coming within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims (22)

A method for power loss management of a storage device,
The method comprising:
Initiating a power loss mode in the non-volatile data storage device in response to a first power source that fails to provide power to the non-volatile data storage device above a predetermined threshold;
Receiving power for the non-volatile data storage device from a second power source during at least a power hold-up time during the power loss mode; And
Volatile data storage device so that essential in-process operations within the power maintenance time are performed on one or more non-volatile memory elements of the non-volatile data storage device. Volatile memory devices that are currently processing the non-volatile data storage device, and adjusting the execution of operations in the process on the non-volatile data storage device during the power loss mode,
Wherein the operations in the essential process include acknowledged write operations and the operations in the non-essential process include an erase operation. The method according to claim 1,
Wherein adjusting the performance of the operations in the process further comprises: determining, based on the amount of energy required to complete one or more non-essential operations, the one or more non-essential operations from operations in the process on the non-volatile data storage device And terminating the power management of the storage device. 3. The method of claim 2,
Wherein the one or more non-essential operations include unrecognized write operations, erase operations, and read operations to one or more clients of the non-volatile data storage device. 3. The method of claim 2,
And disconnecting power to the non-volatile data storage device without performing the one or more non-essential operations. The method according to claim 1,
Wherein adjusting the execution of operations in the process comprises skipping the one or more non-essential operations in one or more instruction queues such that one or more non-essential operations are not performed in the non-volatile data storage device. A method for power loss management of a device. The method according to claim 1,
Wherein the operations in the essential process include verified write operations to one or more clients of the non-volatile data storage device. delete The method according to claim 1,
Wherein adjusting the performance of operations within the process further comprises executing operations within the essential process on the one or more non-volatile memory devices in response to resetting the one or more non-volatile memory devices. A method for managing power loss in a wireless network. The method according to claim 1,
Wherein the one or more non-volatile memory elements comprise substantially all non-volatile memory elements of the non-volatile data storage device. The method according to claim 1,
Wherein the one or more non-volatile memory elements comprise one or more banks of non-volatile memory elements currently processing non-essential operations from operations within the process,
Each bank including a plurality of non-volatile memory dies. 11. The method of claim 10,
Wherein the non-essential operation includes an erase operation. The method according to claim 1,
Wherein adjusting the performance of operations within the process further comprises reordering the selection of operations within the process to perform operations within the essential process within the power maintenance time. The method according to claim 1,
Further comprising stopping operation of one or more sub-elements of the non-volatile data storage device in response to initiating the power loss mode,
Wherein stopping the operation of the one or more child elements conserves energy of the second power source to provide the power hold time. The method according to claim 1,
Volatile memory associated with operations within a process essential to the non-volatile memory of the non-volatile data storage device to store incomplete data buffered within the power-hold time, Further comprising flushing the unfinished data. ≪ Desc / Clms Page number 19 > The method according to claim 1,
Further comprising storing, in the non-volatile data storage device, an identifier from which the non-volatile data storage device starts the power loss mode,
Wherein the identifier is stored for retrieval from the non-volatile data storage device in response to a recovery of power to the non-volatile data storage device. The method according to claim 1,
Further comprising storing in the non-volatile data storage device an identifier that identifies whether operations in the essential process have completed execution within the power hold time,
Wherein the identifier is stored for retrieval from the non-volatile data storage device in response to a recovery of power to the non-volatile data storage device. The method according to claim 1,
Wherein the second power supply comprises one or more capacitors capable of storing sufficient energy to provide the power hold time to perform operations within the essential process. An apparatus for power loss management of a storage device,
The apparatus comprises:
A monitor module for initiating a power loss mode in the non-volatile data storage device in response to a first power source that fails to provide power higher than a predetermined threshold for the non-volatile data storage device;
A second power supply for supplying power to the nonvolatile data storage device during at least the power holding time during the power loss mode; And
And a power loss module that regulates the execution of operations in the process on the non-volatile data storage device during the power loss mode such that operations within an essential process are performed within the power hold time,
Wherein the power loss module is operable to execute operations in the process by resetting one or more non-volatile memory elements of the non-volatile data storage device that currently processes the non-essential operations from operations within the process to terminate non- Lt; / RTI >
Operations within the essential process are operations that must be performed to avoid data corruption, data loss, and inconsistent communications on the non-volatile data storage device,
The non-essential operation may be a terminated, stopped, or paused operation that can be terminated, stopped, or paused without causing data corruption or data loss on the non-volatile data storage device. Device for power loss management. delete A system for power loss management of a storage device,
The system comprises:
An adapter configured to accept two or more non-volatile data storage devices;
A first power source electrically coupled to the adapter to supply power to the at least two non-volatile data storage devices;
At least one monitor module for initiating a power loss mode in the non-volatile data storage devices in response to the first power source failing to supply power above a predetermined threshold;
At least one second power source electrically coupled to the adapter, each second power source configured to supply power to at least one of the non-volatile data storage devices during at least a power hold time during the power loss mode, One second power supply; And
Volatile data storage devices, wherein operations within the non-volatile data storage devices that are currently in operation are not necessarily required to be performed on one or more non-volatile memory elements of the non-volatile data storage devices, And one or more power loss modules for resetting memory elements to regulate the execution of operations within the process on the non-volatile data storage devices during the power loss mode,
Wherein the operations in the essential process include identified write operations and the operations in the non-essential process include an erase operation. 21. The method of claim 20,
Wherein the one or more power loss modules include a separate power loss module for each non-volatile storage device,
Each separate power loss module being configured to regulate the execution of operations within a process on the associated non-volatile data storage device. 21. The method of claim 20,
Wherein the one or more power loss modules comprise a single system power loss module configured to regulate the execution of operations within a process on the at least two non-volatile data storage devices.

Images