US9921913B2 - Flushing host cache data before rebuilding degraded redundant virtual disk - Google Patents
Flushing host cache data before rebuilding degraded redundant virtual disk Download PDFInfo
- Publication number
- US9921913B2 US9921913B2 US15/153,491 US201615153491A US9921913B2 US 9921913 B2 US9921913 B2 US 9921913B2 US 201615153491 A US201615153491 A US 201615153491A US 9921913 B2 US9921913 B2 US 9921913B2
- Authority
- US
- United States
- Prior art keywords
- virtual disk
- rebuild
- lba
- valid
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
- G06F11/1092—Rebuilding, e.g. when physically replacing a failing disk
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
- G06F11/1084—Degraded mode, e.g. caused by single or multiple storage removals or disk failures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
- G06F12/0815—Cache consistency protocols
- G06F12/0831—Cache consistency protocols using a bus scheme, e.g. with bus monitoring or watching means
- G06F12/0833—Cache consistency protocols using a bus scheme, e.g. with bus monitoring or watching means in combination with broadcast means (e.g. for invalidation or updating)
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
- G06F12/0868—Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/12—Replacement control
- G06F12/121—Replacement control using replacement algorithms
- G06F12/128—Replacement control using replacement algorithms adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/12—Replacement control
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
- G06F2211/10—Indexing scheme relating to G06F11/10
- G06F2211/1002—Indexing scheme relating to G06F11/1076
- G06F2211/1009—Cache, i.e. caches used in RAID system with parity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/22—Employing cache memory using specific memory technology
- G06F2212/224—Disk storage
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/62—Details of cache specific to multiprocessor cache arrangements
- G06F2212/621—Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
-
- G06F2212/69—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/70—Details relating to dynamic memory management
Definitions
- Disclosed subject matter is in the field of data storage systems and methods and, more particularly, data storage systems employing a RAID controller and a hot spare drive.
- An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes, thereby allowing users to take advantage of the value of the information.
- information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated.
- the variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications.
- an information handling system may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
- RAID redundant array of independent drives
- the data storage systems of at least some information handling systems employ redundant array of independent drives (RAID) technology to enable the widespread use of low cost persistent mass storage devices without a corresponding decrease in reliability.
- RAID technology may employ a plurality of hard disk drives (HDDs), in combination with data redundancy, parity information and/or other form(s) of error checking information, or a combination thereof, to provide a “virtual disk.”
- HDDs hard disk drives
- User data and error checking information may be distributed among the plurality of HDDs of a virtual disk.
- the physical drive on which the parity data is stored changes from one stripe to the next so that each physical drive stores approximately 1/N of the user data and 1/N of the parity data and where the ratio of user data to parity data is (N ⁇ 1):1.
- RAID-based storage systems may employ one or more redundant physical storage devices that are available to store data from a physical storage device that has exhibited one or more failures. Because these redundant physical storage devices are generally configured such that they can be swapped into a given virtual disk without powering down and without reloading an operating system, they may be referred to herein as hot spare drives or, more simply, hot spares (HSPs). A virtual disk that includes one or more HSPs may be referred to herein as a redundant virtual disk.
- a rebuild process involving the hot spare drive may be initiated to restore the virtual RAID disk to a NORMAL state.
- the rebuild process requires the storage controller to retrieve data from each of the non-failing physical drives, compute the data that was stored on the physical drive that failed, and store the resulting data on the hot spare drive.
- Disclosed subject matter address problems associated with processes for rebuilding data in degraded virtual RAID disks.
- a storage management method includes receiving, by a host system, a degrade signal indicating a DEGRADED state of a virtual disk mapped to or otherwise associated with the host. Responsive to receiving the degrade signal, rebuild-flush operations may be performed prior to actual rebuilding.
- the rebuild-flush operations may include writing, to the virtual disk and also to the HSP associated with the virtual disk, valid-modified, i.e., “dirty” data cached in a host storage cache associated with the virtual disk.
- valid-unmodified “clean” storage cache data associated with the virtual disk may be written to the HSP only.
- the DEGRADED virtual disk may be rebuilt by performing rebuild operations for each virtual disk logical block address (LBA), i.e., each LBA included in the virtual disk.
- the rebuild operations may include simply skipping to the next LBA for any virtual disk LBA cached, whether valid-unmodified or valid modified, in the host storage cache.
- rebuilding may include regenerating a stripe of user data associated with the LBA and storing the stripe of rebuilt user data resulting from said regenerating to the HSP.
- hash table data indicating virtual disk LBAs cached in the host storage cache may be maintained.
- the hash table data may be maintained in two hash tables, one for valid-unmodified cache data and one for valid-modified cache data.
- a storage driver may distinguish between valid-modified data, which may be flushed to the virtual disk as well as the HSP, and valid-unmodified data, which may be stored to the HSP only, i.e., not stored to the virtual disk as part of the rebuild.
- the rebuild-flush operations may include locking storage cache lines associated with the virtual disk to prevent eviction during rebuild, generating a rebuild-flush I/O request targeting the virtual disk LBAs.
- Generating the rebuild-flush I/O request may include setting, in an I/O structure corresponding to the rebuild-flush I/O request, a rebuild-flush flag indicating whether a corresponding virtual disk LBA is a valid-unmodified LBA.
- the rebuild-flush I/O request may be submitted to flush valid data in the host storage cache associated with the DEGRADED virtual disk instead of rebuilding and recalculating the applicable LBAs.
- Processing of the rebuild-flush I/O request may be performed by storage controller code in accordance with the flush-rebuild flag to write valid-unmodified virtual disk LBAs to the HSP and to write valid-modified virtual disk LBAs to the virtual disk and the HSP.
- the DEGRADED virtual disk may be characterized as comprised of three rebuild regions: completed rebuild regions, current rebuild regions, and pending rebuild regions.
- I/O transaction handling during rebuilding may be performed in accordance with the rebuild region targeted by the I/O transaction. If the I/O transaction LBA maps to a completed rebuild region, the I/O transaction may be completed in conventional RAID or redundant manner. If the I/O transaction LBA maps to a current rebuild region, the I/O transaction may be queued to execute when rebuilding of the current rebuild region completes.
- the transaction may be processed “pre-rebuild” by determining whether the I/O transaction LBA matches an LBA in the hash table, i.e., whether the I/O transaction LBA is a cached LBA. If so, processing the I/O transaction may include processing the transaction by direct access of the applicable physical disks including the applicable hard disk drive(s) and the HSP. If the I/O transaction LBA does not match a hash table LBA, the transaction may be processed by regenerating a stripe associated with the I/O transaction LBA before processing the I/O transaction.
- the rebuild assist module beneficially reduces rebuild time by a potentially substantial amount by reducing the amount of storage that must be rebuilt and recalculated.
- an information handling system includes a central processing unit, a storage cache device for use in providing a host storage cache for a virtual disk, and a memory including processor-executable storage instructions.
- the instructions when executed by the CPU, may cause the CPU to perform all or some of the method operations described above.
- the host storage cache may comprise a solid state drive storage cache device and storage cache metadata.
- the virtual disk may be a RAID virtual disk implemented on a group of HDDs and exposed to the host by a storage controller that supports a hot spare drive.
- an information handling platform includes a host information handling system as described in the preceding paragraph in combination with a logical storage volume or LUN (logical unit number) exposed by a storage controller and a plurality of hard disk drives and a host spare drive.
- the information handling platform may flush a host storage cache as described with respect to disclosed methods following a degrade event, in which a hard disk drive reports an unrecoverable error, and prior to rebuilding the entire degraded disk.
- FIG. 1 illustrates a block diagram of an information handling system platform including an information handling system and a virtualized storage resource
- FIG. 2A and FIG. 2B illustrate a flow diagram of a method of rebuilding a degraded virtual disk
- FIG. 3 illustrates a block diagram of hash tables for determining which logical block addresses are cached in a storage cache
- FIG. 4 illustrates a flow diagram of a method of handling I/O transactions while a degraded virtual disk is rebuilding.
- references within the specification to “one embodiment,” “an embodiment,” “at least one embodiment”, or “some embodiments” and the like indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter.
- the appearance of such phrases in various places within the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
- various features are described which may be exhibited by some embodiments and not by others.
- various requirements are described which may be requirements for some embodiments but not for other embodiments.
- Information handling systems that include or are coupled to a redundant virtual RAID disk may be configured as a host system that interfaces with a storage subsystem.
- the host may employ a storage cache, analogous to an L1 data cache employed by a processor, that stores recently accessed and/or frequently accessed user data in a storage medium that has substantially lower access latency than the virtual RAID disk.
- Non-limiting examples of devices suitable for providing a host cache include, dynamic random access memory (DRAM), non-volatile dual in-line memory module (NVDIMM), and non-volatile memory express (NVMe) devices.
- DRAM dynamic random access memory
- NVDIMM non-volatile dual in-line memory module
- NVMe non-volatile memory express
- an IHS that includes a storage subsystem and a host cache initiates a rebuild
- the host cache is likely to include user data that is valid.
- a conventional rebuild process only accesses data from the virtual RAID disk. It is not unusual for an IHS initiating a rebuild of a degraded 1 TB HDD to have, as an example, 100 GB of valid user data stored in a host cache.
- the IHS may include rebuild assistance modules, implemented in software, hardware, firmware, or a combination thereof, to achieve faster rebuild processes.
- the rebuild assist modules may be configured to execute at the cache layer or in RAID drivers and controllers, e.g., as part of a driver for the host cache.
- the storage controller may propagate the event to the host cache module.
- the rebuild assist module may detect the event and lock all valid host cache data associated with the DEGRADED virtual RAID disk. The rebuild assist module may then flush valid host cache data associated with the virtual RAID disk before proceeding with a rebuild.
- Flushing the host cache of valid data associated with a particular virtual RAID disk may include flushing valid-unmodified data to the HSP and flushing valid-modified data to the physical drives of the virtual disk as well as to the HSP.
- the rebuild assist module may be particularly effective in reducing rebuild time when the storage cache resides on the host system and employs solid state memory or storage including, as examples, DRAM, NVDIMM, or NVMe.
- FIG. 1 illustrates an information handling system platform 100 , also referred to herein as information handling platform 100 , suitable for implementing an efficient rebuild process for a degraded virtual disk.
- the information handling platform 100 illustrated in FIG. 1 includes a host information handling system 101 , also referred to herein as host 101 , generating I/O transactions 140 targeting a LUN 150 .
- the host 101 of FIG. 1 encompasses an application program 102 executing within an operating system (OS) 103 installed on an information handling system 110 .
- OS operating system
- the information handling system 110 includes one or more central processing units (CPUs) 111 coupled to system memory 112 , in which the application program 102 and the operating system 103 have been stored for execution by CPU(s) 111 .
- the information handling system 110 illustrated in FIG. 1 further includes a chip set 113 that provides one or more I/O interfaces that couple external devices including the external storage represented by LUN 150 to host 101 .
- the LUN 150 illustrated in FIG. 1 is implemented on a group of three (3) hard disk drives 160 and an HSP 170 .
- a storage controller 152 exposes LUN 150 and virtual disk 151 to host 101 .
- the virtual disk 151 of FIG. 1 is a RAID virtual disk maintained by a RAID controller 154 underlying storage controller 152 .
- the RAID controller 154 may support any one or more levels of RAID appropriate for the application and configuration.
- the RAID controller 154 of FIG. 1 may, as a non-limiting example, implement virtual disk 151 as a RAID 5 virtual disk employing the three HDDs 160 with two of the three HDDs 160 storing data and the remaining HDD 160 storing parity for any given data stripe, where the HDD 160 containing parity rotates among the three HDDs 160 depending on the stripe.
- the information handling platform 100 illustrated in FIG. 1 includes a storage cache 130 configured to cache I/O transactions 140 generated by host 101 .
- the storage cache 130 of FIG. 1 includes one or more cache storage devices 134 and cache metadata 136 maintained by a storage cache module 120 in operating system 103 .
- Each of the one or more storage cache devices 134 may be implemented with a storage medium having a significantly lower access latency than the HDDs 160 that back virtual disk 151 .
- each one or more storage cache device 134 comprises a nonvolatile memory express (NVMe), peripheral component interconnect express (PCIe) solid state drive (SSD).
- NVMe nonvolatile memory express
- PCIe peripheral component interconnect express
- SSD solid state drive
- storage cache 130 Due to a higher per-byte cost of solid state devices relative to HDDs, the storage capacity of storage cache 130 has, historically, been significantly lower than the storage capacity of LUN 150 . Anecdotal evidence, however, suggests that installations with oversized storage caches, i.e., installations employing significantly higher cache/HDD ratios, are becoming increasingly common. Coupled with the ever-increasing capacities of even modestly implemented LUNs, the resulting storage cache may be quite large.
- HDDs such as the HDDs 160 of FIG. 1
- the reliability of HDDs is inherently lower than the reliability of the solid state devices in which storage cache 130 is typically implemented. While RAID configurations address the most significant risk associated with an HDD failure, i.e., loss of user data, the time required to rebuild a RAID virtual disk following an HDD degrade is undesirably high and will become even more so as terabyte scale HDDs give way to petabyte scale HDDS and beyond.
- the information handling platform 100 of FIG. 1 addresses the rebuild delay associated with large capacity RAID virtual disks at least in part by leveraging the potentially sizeable amount of valid data residing in the storage cache 130 to reduce the number of HDD accesses and the number of parity calculations the rebuild process must perform.
- each individually flushable segment or grain of user data stored in storage cache 130 may be referred to as a cache line.
- the byte size of a cache line is an implementation detail that may vary among different embodiments.
- the cache line size may be selected in conjunction with the size of individually addressable segments of storage in HDDs 160 , e.g., the amount of storage associated with a single LBA.
- Each LBA may represent 512 bytes, 4 Kbytes, or another suitable value.
- each cache line has capacity to store N LBAs where N is an integer greater than 0. For purposes of the following discussion, N is assumed to be 1 unless indicated otherwise.
- each cache line may be characterized in one of three cache coherency states: (1) valid-unmodified, (2) valid-modified, and (3) invalid.
- storage cache 130 may be described as typically containing, at any moment in time, at least some invalid cache data (ICD) and at least some valid cache data, including at least some valid-unmodified data and at least some valid-modified data.
- ICD invalid cache data
- the information handling platform 100 of FIG. 1 enables and supports storage cache 130 and virtual disk 151 with a storage cache module 120 and a RAID controller driver 122 underlying OS 103 and with the RAID controller 154 underlying storage controller 152 .
- the illustrated information handling platform 100 provides rebuild assistance modules for each of these three resources.
- FIG. 1 illustrates three rebuild assistance modules including a rebuild assist module 121 within storage cache module 120 , a driver rebuild module 123 within RAID controller driver 122 , and a controller rebuild module 155 within RAID controller 154 .
- FIG. 2 A description of operations performed by these modules, in conjunction with other resources of the information handling platform 100 , is illustrated in FIG. 2 .
- FIG. 2 illustrates a flow diagram of a process 200 for providing rebuild assistance to a storage controller rebuilding a virtual disk 151 following a virtual disk degrade.
- a degrade or degrade event refers to a state transition of virtual disk 151 from NORMAL to DEGRADED, where the DEGRADED state is a state in which virtual disk 151 has at least one fault or error and from which virtual disk 151 can be rebuilt, i.e., restored to a functional state with no loss of user data.
- a degrade of virtual disk 151 may occur following detection of an unrecoverable error on one of the underlying HDDs 160 .
- the virtual disk 151 may be restored to a NORMAL RAID 5 state of redundancy.
- the process 200 illustrated in FIG. 2 begins whenever the state of a virtual disk 151 transitions ( 202 ) to DEGRADED.
- the storage controller 152 may send ( 204 ) a degrade notification to host 101 .
- the degrade notification may include information identifying the degraded virtual disk 151 associated with the degrade event.
- the process 200 of FIG. 2 responds to receiving the degrade notification by “locking” ( 206 ) valid storage cache lines corresponding to the degraded virtual disk.
- Locking storage cache lines containing valid data, whether modified or unmodified, prevents the applicable cache lines from being evicted from the host cache while the rebuild is in progress.
- Each storage cache line of storage cache 130 may include, within its metadata 136 , one or more fields or attributes (not explicitly depicted).
- the metadata attributes may include an eviction attribute indicating whether the corresponding cache line in storage cache 130 might be invalidated or otherwise evicted.
- the eviction attribute may be indicated in one or more bits of information in metadata 136 . In a single bit embodiment of the eviction attribute, the attribute simply identifies cache lines ineligible to be invalidated or otherwise evicted.
- the RAID controller driver 122 may maintain one or more hash table data structures mapping the LBAs of the degraded virtual disk to storage cache lines containing valid cache data.
- the hash tables 301 of FIG. 3 distinguish between storage cache lines containing valid-unmodified (VU) data and storage cache lines containing valid-modified (VM) data by including a first hash table 301 - 1 identifying degraded virtual disk LBAs cached in the host storage cache with VU data and a second hash table 301 - 2 identifying degraded virtual disk LBAs cached in the host storage cache with VM data.
- Each hash table 301 maps an LBA to a corresponding cache line of the host storage cache, i.e., the host storage cache entry corresponding to the applicable LBA. As the entries in the host storage cache change, the hash table may be updated to reflect the current state of the storage cache at any time.
- RAID controller driver 122 also defines a special purpose flag, referred to herein as the rebuild-flush flag, in the request structure used for I/O transactions 140 , e.g., a bio struct (Block I/O structure) flag in embodiments employing a Linux-based OS 103 .
- the rebuild-flush flag may be asserted to distinguish between storage cache lines that contain VU data and storage cache lines that contain VM data.
- the process 200 illustrated in FIG. 2 sets (operation 208 ) the rebuild-flush flag and initiates (operation 210 ) a rebuild-flush I/O request to write VU data associated with the degraded virtual disk 151 to the HSP 170 and to write back VM data associated with the degraded virtual disk 151 to HDDs 160 as well as HSP 170 .
- the rebuild-flush I/O request distinguishes (operation 212 ) between cache lines containing VU data and cache lines containing VM data.
- rebuild assist module 121 For storage cache lines associated with the degraded virtual disk 151 containing VU data, rebuild assist module 121 sends (operation 220 ) a special purpose control command, referred to herein as the VU transfer command, to the driver rebuild module 123 of RAID controller driver 122 and updates (operation 214 ) a Flush List that contains the LBAs of all valid cache data, whether modified or unmodified, flushed back to LUN 150 .
- VU transfer command a special purpose control command
- VU transfer command is then processed (operation 222 ) by the driver rebuild module 123 and the VU data is sent (operation 224 ) to the controller rebuild module 155 of RAID controller 154 .
- the rebuild assist module 121 issues a write (operation 230 ) of VM data associated with the degraded virtual disk 151 and updates (operation 214 ) the Flush List with the corresponding LBAs.
- the driver rebuild module 123 responds to the VM write by sending (operation 232 ) the VM data to the controller rebuild module 155 in RAID controller 154 .
- the RAID controller 154 writes (operation 240 ) VM data to virtual disk 151 and updates (operation 242 ) the hash table.
- the RAID controller 154 also writes (operation 244 ) VU data to HSP 170 , and only to HSP 170 , and updates the hash table.
- the rebuild following the rebuild-flush operations described above includes an iterative process in which each LBA associated with the degraded virtual disk 151 is checked against the hash table. If (operation 252 ) the current LBA is found in the hash table, the LBA was processed during the rebuild-flush operations described above and the applicable user data was stored to the HSP 170 . Accordingly, no further action is required and processor 200 advances to the next LBA (operation 260 ) If on the other hand, the current LBA is not found in the hash table, data and parity are regenerated (operation 256 ) and written to HSP 170 before advancing to the next LBA. This processing continues until all LBAs associated with the degraded virtual disk 151 have been processed against the hash table, at which point the rebuild has completed (block 262 ).
- FIG. 4 illustrates a method 400 for performing I/O transactions while the previously described rebuilding is in progress.
- a determination (operation 404 ) of whether the transaction is a read or a write is made. If the I/O transaction is a read, a determination (operation 406 ) of whether the I/O transaction targets an LBA that has already been rebuilt as part of the rebuild flush is made, in which case the I/O is executed (operation 408 ) as if the array is in redundant mode.
- operation 410 If (operation 410 ) the I/O transaction targets a region currently being rebuilt, the I/O transaction is serialized (operation 412 ) for execution after the rebuild completes. If the I/O targets a pending rebuild region, i.e., a region yet to be rebuilt, method 400 proceeds to operation 414 where a determination of whether the applicable LBA is in the hash table is made. If so, method 400 performs the I/O transaction by accessing (operation 416 ) physical disks including the HSP 170 . If the I/O region targets an LBA that is not in the hash table, the I/O transaction may be performed by regenerating (operation 420 ) the data, completing the I/O request, and writing the data to the HSP 170 .
- the method 400 illustrated in FIG. 4 branches from operation 404 to operation 430 where a determination is made of whether the I/O transaction targets an LBA that has already been rebuilt as part of the rebuild flush, in which case the I/O is executed (operation 432 ) as if the array is in redundant mode. If (operation 434 ) the I/O transaction targets a region currently being rebuilt, the I/O transaction is serialized (operation 436 ) for execution after the rebuild completes.
- method 400 branches from operation 434 to operation 438 where the method determines whether the requested I/O region is part of the rebuild-flush hash table. If so, the method 400 illustrated in FIG. 4 performs (operation 440 ) a write/modify of the data by accessing HSP 170 and member physical disks and thereafter acknowledging the I/O. If the I/O transaction is not in the hash table, the illustrated method 400 applies (operation 412 ) normal regeneration operations and completes the I/O request, writes the generated stripe to HSP 170 and updates the hash table.
- Any one or more processes or methods described above, including processes and methods associated with FIGS. 2A, 2B, and 4 flow diagrams, may be embodied as a computer readable storage medium or, more simply, a computer readable medium including processor-executable program instructions, also referred to as program code or software, that, when executed by the processor, cause the processor to perform or otherwise result in the performance of the applicable operations.
- processor-executable program instructions also referred to as program code or software
- a computer readable medium which may also be referred to as computer readable memory or computer readable storage, encompasses volatile and non-volatile media, memory, and storage, whether programmable or not, whether randomly accessible or not, and whether implemented in a semiconductor, ferro-magnetic, optical, organic, or other suitable medium.
- Information handling systems may include two or more different types of computer readable media and, in such systems, program code may be stored, in whole or in part, in two or more different types of computer readable media.
- Program code for effecting described operations may be written in any appropriate combination of programming languages and encompasses human readable program code including source code as well as machine readable code including object code.
- Program code may be executed by a general purpose processor, a special purpose processor, including, as non-limiting examples, a graphics processor, a service processor, or an embedded processor or controller.
- Disclosed subject matter may be implemented in any appropriate combination of software, firmware, and hardware.
- Terms including circuit(s), chip(s), processor(s), device(s), computer(s), desktop(s), laptop(s), system(s), and network(s) suggest at least some hardware or structural element(s), but may encompass non-transient intangible elements including program instruction(s) and one or more data structures including one or more databases.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
Description
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/153,491 US9921913B2 (en) | 2016-05-12 | 2016-05-12 | Flushing host cache data before rebuilding degraded redundant virtual disk |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/153,491 US9921913B2 (en) | 2016-05-12 | 2016-05-12 | Flushing host cache data before rebuilding degraded redundant virtual disk |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170329674A1 US20170329674A1 (en) | 2017-11-16 |
US9921913B2 true US9921913B2 (en) | 2018-03-20 |
Family
ID=60297053
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/153,491 Active 2036-09-16 US9921913B2 (en) | 2016-05-12 | 2016-05-12 | Flushing host cache data before rebuilding degraded redundant virtual disk |
Country Status (1)
Country | Link |
---|---|
US (1) | US9921913B2 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108572795B (en) * | 2017-12-21 | 2021-05-25 | 北京金山云网络技术有限公司 | Capacity expansion method, device, equipment and storage medium based on built storage virtualization |
US10891244B2 (en) | 2018-10-25 | 2021-01-12 | Dell Products, L.P. | Method and apparatus for redundant array of independent drives parity quality of service improvements |
CN111240903A (en) * | 2019-11-04 | 2020-06-05 | 华为技术有限公司 | Data recovery method and related equipment |
US11372754B2 (en) | 2020-06-12 | 2022-06-28 | Western Digital Technologies, Inc. | Storage system and method for enabling a software-defined dynamic storage response |
US11734117B2 (en) * | 2021-04-29 | 2023-08-22 | Vast Data Ltd. | Data recovery in a storage system |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070220313A1 (en) * | 2006-03-03 | 2007-09-20 | Hitachi, Ltd. | Storage control device and data recovery method for storage control device |
US20070300034A1 (en) * | 2006-06-27 | 2007-12-27 | Fujitsu Limited | Virtual storage control apparatus |
US20090125680A1 (en) * | 2007-11-09 | 2009-05-14 | Hitachi, Ltd. | Storage System for Restoring Data Stored in Failed Storage Device |
US7650475B2 (en) * | 2006-07-05 | 2010-01-19 | Hitachi, Ltd. | Storage system and method for managing data using the same |
US7873600B2 (en) * | 2007-01-24 | 2011-01-18 | Hitachi, Ltd. | Storage control device to backup data stored in virtual volume |
US20110191538A1 (en) * | 2010-02-04 | 2011-08-04 | Fujitsu Limited | Storage device, method for restoring data in storage device and storage controller |
US20140025990A1 (en) * | 2012-07-23 | 2014-01-23 | Hitachi, Ltd. | Storage system and data management method |
US8943359B2 (en) * | 2012-10-05 | 2015-01-27 | Lsi Corporation | Common hot spare for multiple RAID groups |
US20170004047A1 (en) * | 2015-06-30 | 2017-01-05 | Symantec Corporation | Consolidated full backup of a restored virtual machine |
-
2016
- 2016-05-12 US US15/153,491 patent/US9921913B2/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070220313A1 (en) * | 2006-03-03 | 2007-09-20 | Hitachi, Ltd. | Storage control device and data recovery method for storage control device |
US20070300034A1 (en) * | 2006-06-27 | 2007-12-27 | Fujitsu Limited | Virtual storage control apparatus |
US7673096B2 (en) * | 2006-06-27 | 2010-03-02 | Fujitsu Limited | Control apparatus for controlling virtual storage |
US7650475B2 (en) * | 2006-07-05 | 2010-01-19 | Hitachi, Ltd. | Storage system and method for managing data using the same |
US7873600B2 (en) * | 2007-01-24 | 2011-01-18 | Hitachi, Ltd. | Storage control device to backup data stored in virtual volume |
US20090125680A1 (en) * | 2007-11-09 | 2009-05-14 | Hitachi, Ltd. | Storage System for Restoring Data Stored in Failed Storage Device |
US20110191538A1 (en) * | 2010-02-04 | 2011-08-04 | Fujitsu Limited | Storage device, method for restoring data in storage device and storage controller |
US20140025990A1 (en) * | 2012-07-23 | 2014-01-23 | Hitachi, Ltd. | Storage system and data management method |
US8943359B2 (en) * | 2012-10-05 | 2015-01-27 | Lsi Corporation | Common hot spare for multiple RAID groups |
US20170004047A1 (en) * | 2015-06-30 | 2017-01-05 | Symantec Corporation | Consolidated full backup of a restored virtual machine |
Also Published As
Publication number | Publication date |
---|---|
US20170329674A1 (en) | 2017-11-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9836404B2 (en) | Write mirroring to storage class memory devices | |
US10698818B2 (en) | Storage controller caching using symmetric storage class memory devices | |
US8966170B2 (en) | Elastic cache of redundant cache data | |
US8635407B2 (en) | Direct memory address for solid-state drives | |
US9690493B2 (en) | Two-level system main memory | |
US8930619B2 (en) | Method and apparatus for efficiently destaging sequential I/O streams | |
US8539150B2 (en) | Storage system and management method of control information using a cache memory with multiple cache partitions | |
US9361177B2 (en) | Methods and apparatus for managing error codes for storage systems coupled with external storage systems | |
US8250283B1 (en) | Write-distribute command for RAID mirroring | |
US20190324859A1 (en) | Method and Apparatus for Restoring Data after Power Failure for An Open-Channel Solid State Drive | |
US20150095696A1 (en) | Second-level raid cache splicing | |
US9037793B1 (en) | Managing data storage | |
US9047200B2 (en) | Dynamic redundancy mapping of cache data in flash-based caching systems | |
US9053038B2 (en) | Method and apparatus for efficient read cache operation | |
US20220137835A1 (en) | Systems and methods for parity-based failure protection for storage devices | |
US9921913B2 (en) | Flushing host cache data before rebuilding degraded redundant virtual disk | |
US20150074371A1 (en) | Storage array system and non-transitory recording medium storing control program | |
US20180089088A1 (en) | Apparatus and method for persisting blocks of data and metadata in a non-volatile memory (nvm) cache | |
US8990542B2 (en) | Efficient metadata protection system for data storage | |
TW201107981A (en) | Method and apparatus for protecting the integrity of cached data in a direct-attached storage (DAS) system | |
US9298636B1 (en) | Managing data storage | |
US11023321B2 (en) | Raid storage-device-assisted deferred parity data update system | |
US11315028B2 (en) | Method and apparatus for increasing the accuracy of predicting future IO operations on a storage system | |
TWI782847B (en) | Method and apparatus for performing pipeline-based accessing management in a storage server | |
US10872041B2 (en) | Method and apparatus for journal aware cache management |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DELL PRODUCTS L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SREEDHAR M, DEEPU SYAM;AGARWAL, SANDEEP;P.K., KRISHNA KUMAR;AND OTHERS;SIGNING DATES FROM 20160509 TO 20160513;REEL/FRAME:038637/0586 |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, NORTH CAROLINA Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (ABL);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039643/0953 Effective date: 20160808 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT, TEXAS Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (NOTES);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039644/0084 Effective date: 20160808 Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH CAROLINA Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (TERM LOAN);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039719/0889 Effective date: 20160808 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., A Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (NOTES);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039644/0084 Effective date: 20160808 Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, NO Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (ABL);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039643/0953 Effective date: 20160808 Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH Free format text: SUPPLEMENT TO PATENT SECURITY AGREEMENT (TERM LOAN);ASSIGNORS:AVENTAIL LLC;DELL PRODUCTS L.P.;DELL SOFTWARE INC.;AND OTHERS;REEL/FRAME:039719/0889 Effective date: 20160808 |
|
AS | Assignment |
Owner name: AVENTAIL LLC, CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (ABL);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:040013/0733 Effective date: 20160907 Owner name: WYSE TECHNOLOGY L.L.C., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (ABL);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:040013/0733 Effective date: 20160907 Owner name: DELL SOFTWARE INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (ABL);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:040013/0733 Effective date: 20160907 Owner name: FORCE10 NETWORKS, INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (ABL);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:040013/0733 Effective date: 20160907 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SEC. INT. IN PATENTS (ABL);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:040013/0733 Effective date: 20160907 |
|
AS | Assignment |
Owner name: WYSE TECHNOLOGY L.L.C., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (NOTES);ASSIGNOR:BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT;REEL/FRAME:040026/0710 Effective date: 20160907 Owner name: WYSE TECHNOLOGY L.L.C., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (TL);ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:040027/0329 Effective date: 20160907 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SEC. INT. IN PATENTS (TL);ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:040027/0329 Effective date: 20160907 Owner name: FORCE10 NETWORKS, INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (TL);ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:040027/0329 Effective date: 20160907 Owner name: FORCE10 NETWORKS, INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (NOTES);ASSIGNOR:BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT;REEL/FRAME:040026/0710 Effective date: 20160907 Owner name: DELL SOFTWARE INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (NOTES);ASSIGNOR:BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT;REEL/FRAME:040026/0710 Effective date: 20160907 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SEC. INT. IN PATENTS (NOTES);ASSIGNOR:BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT;REEL/FRAME:040026/0710 Effective date: 20160907 Owner name: DELL SOFTWARE INC., CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (TL);ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:040027/0329 Effective date: 20160907 Owner name: AVENTAIL LLC, CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (NOTES);ASSIGNOR:BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT;REEL/FRAME:040026/0710 Effective date: 20160907 Owner name: AVENTAIL LLC, CALIFORNIA Free format text: RELEASE OF SEC. INT. IN PATENTS (TL);ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:040027/0329 Effective date: 20160907 |
|
AS | Assignment |
Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT, TEXAS Free format text: SECURITY AGREEMENT;ASSIGNORS:ASAP SOFTWARE EXPRESS, INC.;AVENTAIL LLC;CREDANT TECHNOLOGIES, INC.;AND OTHERS;REEL/FRAME:040136/0001 Effective date: 20160907 Owner name: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT, NORTH CAROLINA Free format text: SECURITY AGREEMENT;ASSIGNORS:ASAP SOFTWARE EXPRESS, INC.;AVENTAIL LLC;CREDANT TECHNOLOGIES, INC.;AND OTHERS;REEL/FRAME:040134/0001 Effective date: 20160907 Owner name: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLAT Free format text: SECURITY AGREEMENT;ASSIGNORS:ASAP SOFTWARE EXPRESS, INC.;AVENTAIL LLC;CREDANT TECHNOLOGIES, INC.;AND OTHERS;REEL/FRAME:040134/0001 Effective date: 20160907 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., A Free format text: SECURITY AGREEMENT;ASSIGNORS:ASAP SOFTWARE EXPRESS, INC.;AVENTAIL LLC;CREDANT TECHNOLOGIES, INC.;AND OTHERS;REEL/FRAME:040136/0001 Effective date: 20160907 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., T Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES, INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:049452/0223 Effective date: 20190320 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., TEXAS Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES, INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:049452/0223 Effective date: 20190320 |
|
AS | Assignment |
Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., TEXAS Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:053546/0001 Effective date: 20200409 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
AS | Assignment |
Owner name: WYSE TECHNOLOGY L.L.C., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: SCALEIO LLC, MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: MOZY, INC., WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: MAGINATICS LLC, CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: FORCE10 NETWORKS, INC., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: EMC IP HOLDING COMPANY LLC, TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: EMC CORPORATION, MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL SYSTEMS CORPORATION, TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL SOFTWARE INC., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL MARKETING L.P., TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL INTERNATIONAL, L.L.C., TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: DELL USA L.P., TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: CREDANT TECHNOLOGIES, INC., TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: AVENTAIL LLC, CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 Owner name: ASAP SOFTWARE EXPRESS, INC., ILLINOIS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058216/0001 Effective date: 20211101 |
|
AS | Assignment |
Owner name: SCALEIO LLC, MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC), MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL INTERNATIONAL L.L.C., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL USA L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 Owner name: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061324/0001 Effective date: 20220329 |
|
AS | Assignment |
Owner name: SCALEIO LLC, MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC), MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL INTERNATIONAL L.L.C., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL USA L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 Owner name: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.), TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:061753/0001 Effective date: 20220329 |