CN105446890B

CN105446890B - Intelligent data deployment

Info

Publication number: CN105446890B
Application number: CN201510599243.6A
Authority: CN
Inventors: 萨克西纳·倪马尔·拉吉; 莫松·布鲁斯·凯恩斯; 罗宾森·欧文·迈克尔
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2014-09-18
Filing date: 2015-09-18
Publication date: 2020-08-25
Anticipated expiration: 2035-09-18
Also published as: CN105446890A; KR20160033643A; US9851909B2; KR102414500B1; US20160085467A1

Abstract

A method is provided for mapping a storage volume to a plurality of pools of storage devices, the storage devices being designated by a host having a host identification. The data storage volume has a volume identification and a plurality of zones. The method comprises the following steps: the method further includes assigning a first pool of storage devices to the storage volume based on the host identification, and determining, for the first pool of storage devices, a mapping value based on the host identification and the volume identification. The method also includes determining a storage index based on the mapping value and one or more of the plurality of zones, and mapping a portion of the zone to a first pool of storage based on the storage index.

Description

Intelligent data deployment

Technical Field

The present disclosure relates to a system for data storage, and more particularly, to a rack-mount storage system for network data storage utilizing hash mapping and redundancy.

Background

The demand for storage capacity is increasing. Existing databases and virtual storage systems are typically capable of holding large amounts of data. As the demand for storage capacity increases, the manageability, performance, and reliable accessibility of data in databases becomes important. However, up to now, the memory volume management of physical storage from multiple network storage devices and current databases using a single virtual storage are not intelligently managed, slower than needed, and not reliable enough for efficient data center applications.

Disclosure of Invention

Certain embodiments of the present invention provide a network-based storage system.

The illustrative embodiments provide a method of mapping storage volumes to a plurality of pools of storage devices designated by a host having a host identification. The data storage volume has a volume identification and a plurality of extents (extents). The method comprises the following steps: a first pool of storage devices is assigned to the storage volume based on the host identification, and for the storage devices of the first pool, a mapping value is determined based on the host identification and the volume identification. The method also includes: determining a storage index based on the mapping value and one or more of the plurality of zones, mapping a portion of the zones to storage of the first pool based on the storage index.

Another exemplary embodiment provides a network storage system. The system includes a host having a host identity and mapping a storage volume having a volume identity and a plurality of zones. The system also includes a plurality of pools of storage devices to distribute at least a portion of a region of a storage volume designated by a host. The system further comprises: a processor to assign a first pool of the plurality of pools of storage devices to a storage volume based on a host identification; for a first pool of assignments of storage devices, determining a mapping value based on a host identification and a volume identification; determining a storage index based on the mapping value and the one or more zones; and mapping at least a portion of the region to an assigned first pool of storage based on the storage index.

Yet another exemplary embodiment provides a method of mapping a storage volume to a plurality of storage devices specified by a host having a host identification. The data storage volume has a volume identification and a plurality of zones. The method comprises the following steps: for the plurality of storage devices, a hash value is determined based on a host identification and a volume identification, a storage device index is obtained based on one or more of the plurality of zones, and a storage device address of one of the storage devices in a first pool from the plurality of pools of storage devices is assigned based on the storage device index. The method also includes mapping a portion of the plurality of zones to a first pool of the plurality of pools from a storage device using the assigned storage device address, inserting redundancy behind the portion of the plurality of zones, and mapping a remaining portion of the plurality of zones behind the inserted redundancy.

Drawings

The features and utilities described in the foregoing summary, as well as the following detailed description of certain embodiments of the present general inventive concept, will be better understood when read in conjunction with the appended drawings.

FIG. 1 is a diagram illustrating an embodiment of a network storage system.

Fig. 2 illustrates an exemplary system such as the system shown in fig. 1.

FIG. 3A illustrates an operational flow diagram of a mapping process for mapping logical volumes into logical blocks of a storage device, according to an embodiment of the present invention.

FIG. 3B illustrates an operational flow diagram of an allocation process for allocating storage space or volumes in a storage device according to an embodiment of the present invention.

Fig. 4 illustrates an exemplary address mapping stream according to the present invention.

FIG. 5 illustrates a first allocation example, where multiple hosts allocate multiple volumes through a mapping function in a pool of storage devices.

FIG. 6 illustrates a second allocation example in which multiple hosts allocate multiple volumes through a mapping function in a pool of storage devices without redundancy.

FIG. 7 illustrates a third allocation example, where multiple hosts allocate multiple volumes through a mapping function in a pool of storage devices with redundancy.

FIG. 8 illustrates a fourth allocation example in which multiple hosts allocate multiple volumes through a mapping function in multiple pools of storage with thin provisioning (in provisioning) and redundancy.

FIG. 9 illustrates a fifth allocation example in which multiple hosts allocate multiple volumes through a mapping function in a pool of storage devices using thin provisioning and redundancy.

FIG. 10 illustrates a first exemplary Mean Time Between Failure (MTBF) failure report utilizing the mapping process of FIG. 3 and RAID10 redundancy.

Fig. 11 illustrates a second exemplary MTBF failure report utilizing the mapping process of fig. 3 and RAID6 redundancy.

For the purpose of illustrating the general inventive concept, certain embodiments are shown in the drawings. It should be understood, however, that the present invention is not limited to the arrangements and instrumentality shown in the attached drawings.

Detailed Description

Reference will now be made in detail to the embodiments of the present general inventive concept, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. For the purpose of illustrating the general inventive concept, embodiments are described below while referring to the accompanying drawings.

Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings.

Advantages and features of the present invention and methods of accomplishing the same may be understood more readily by reference to the following detailed description and the accompanying drawings. The general inventive concept may, however, be embodied in many different forms or implemented in various ways, and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and will fully convey the general inventive concept to those skilled in the art. The general inventive concept is defined by the claims. In the drawings, the thickness of layers and regions are exaggerated for visual clarity.

Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of the terms "a" and "an" and "the" and similar referents in the context of describing the invention (e.g., in the context of the following claims) is to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to,") unless otherwise noted.

It should also be apparent to one of ordinary skill in the art that the system shown in the figures is a model that may be similar to a real system. Some of the described models and logic structures can be implemented in software executed by a microprocessor or similar device or can be implemented using hardware including various components, such as an application specific integrated circuit ("ASIC"). Terms such as "processor" may include or refer to both hardware and/or software. No specific meaning is implied or should be inferred simply due to the use of capitalization.

Similarly, as used herein, the term "component" or "model" means, but is not limited to, a software or hardware component such as a Field Programmable Gate Array (FPGA) or ASIC that performs certain tasks. A component or module may advantageously be configured to reside in an accessible storage medium and configured to execute on one or more processors. Thus, a component or module may include, by way of example, a component (e.g., software component, object-oriented software component, class component, and task component), a processor, a function, an attribute, a program, a subroutine, a segment of program code, a driver, firmware, microcode, circuitry, data, database, data structure, table, array, and variable. The functionality provided for in the components and/or modules may be combined into fewer components and/or modules or further separated into additional components and/or modules.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, unless otherwise defined, all terms defined in commonly used dictionaries should have their ordinary meanings.

Embodiments of the inventive concepts relate to a method of allocating a host requested logical volume.

FIG. 1 is a diagram illustrating an embodiment of a network storage system 100. The system 100 includes a host 104 that requests a storage volume (volume of storage) through a switching network (switching network) 108. The system 100 also includes a processing complex 112 also having one or more processors, the processing complex 112 virtualizing data on one or more storage devices into virtual block storage volumes residing on application servers, depending on the performance and/or requests imposed by the application. For example, in some embodiments, as discussed in detail below, the processing complex 112 executes one or more mapping functions to map virtualized volumes throughout a pool of storage devices, while providing redundancy capabilities to the virtualized volumes to enhance performance and availability transparent to the application servers. In some embodiments, the virtualized volume is visible to the host 104 as a standard logical volume with a logical unit number.

In the illustrated embodiment, the processing complex 112 includes M processors 112.1 through 112.M (with a generic index 112.i representing the ith processor). In some embodiments, processing complex 112 includes two Broadcom multicores XLPII 964 of a system-on-chip (SoC) joined by an inter-chip interconnect (ICI), not shown.

A boot complex 116 is coupled to the processing complex 112 to provide the processing complex 112 with a basic input/output system (BIOS) and boot firmware code. For example, in some embodiments, during power-on with software conventions and optional constraint options (strap-options), the processing complex 112 boots with a single kernel image and boots in two socket shared memory CC-NUMA (cache coherent non-uniform memory access) modules.

The processing complex 112 is also coupled to a channel memory 120 and a controller complex 124. In some embodiments, channel memory 120 includes multiple 72-bit DDR3 channel memory controller interfaces for each of the M processors 112i, which provide half bidirectional read or write memory transaction bandwidth of approximately 400 Gbps. In some embodiments, channel memory 120 also has single-error and double-error detection (SEC-DED) and nibble error correction capabilities to enable enhanced reliability of system 100. In some embodiments, channel memory 120 includes one or more 8GB Dynamic Random Access Memories (DRAMs).

The controller complex 124 may have one or more controllers to communicate with the memory complex 128, as performance demands dictate. In the illustrated embodiment, the controller complex 124 includes N controllers 124.1 through 124.N (with a generic index 124.i representing the ith controller). In some embodiments, the controller complex 124 also includes a pair of switches (not shown). For example, in some embodiments, the switch is a PEX 8796PCIe 3.0 switch.

The memory complex 128 includes multiple pools of memory devices 132. In the illustrated embodiment, the memory complex 128 includes P pools 128.1 through 128.P (with a universal index 128.i representing the ith pool), each of the P pools including K storage devices 132, the storage devices 132 including devices 132.i.1 through 132.i.K (with a universal index 132.i.j representing the jth storage device of the ith pool). In some embodiments, storage 132 is one or more solid state drives having 1 Terabyte (TB) of storage. Although the same number of storage devices 132 are shown throughout the P pools, different numbers of storage devices 132 may be used throughout the P pools in other embodiments depending on the application. In some embodiments, storage 132 comprises a continuous range of logical block addresses.

Fig. 2 illustrates an exemplary system 200 of the system 100 as shown in fig. 1. The exemplary system 200 shows N hosts 204(204.0 to 204.N-1 with a generic index 204.i representing the (i +1) th host). Each host 204.i may access multiple logical volumes 208 (e.g., Q volumes). In some embodiments, the Q volumes may not be the same size. In the illustrated embodiment, host 0(204.0) has Q logical volumes 208.0.1 through 208.0.Q (with a common index 208.i.j representing the (i +1) th host or the jth volume of host (i + 1)). The exemplary system 200 also shows a mapping function 212, described in detail below, that maps the logical volumes 208 to virtualized volumes in a plurality of pools 216 of storage 220. Specifically, in the illustrated embodiment, there are four (P ═ 4) pools 216.0 through 216.3, each pool 216 having ten (K ═ 10) storage devices. For example, the pool 216.0 has ten storage devices 220.0.0 through 220.0.9 (with the common index 220, i.j representing the jth storage device of the (i +1) th pool). In the embodiment illustrated in fig. 2, the logical volume 208.0.1 visible by host 0(204.0) may be virtualized, shared, or distributed throughout some of the storage devices 220.0.0-220.0.9. In some embodiments, the number of storage devices used for volume distribution or virtualization is predetermined, and the predetermined number of storage devices available within a pool (such as pool 216.0 for example) is a power of two. In the illustrated embodiment, the number of storage devices 220.0.0 through 220.0.9 used for volume distribution or virtualization is predetermined to be eight, which is the maximum number of storage devices equal to a power of two available within a pool 216.0 of ten storage devices.

In some embodiments, each storage 220.0.i in pool 216 is also grouped in a plurality of storage units, stripes, or zones (extents). Each region is typically a power of two bytes in size and is similar in length to one or more gigabytes, e.g., 1 GB. The zones in the pool are also typically of the same size. In some embodiments, each zone is further subdivided into sectors (not shown). In other embodiments, each sector includes a string of power of two consecutive bytes, similar in size to sixteen Kilobytes (KB), with an optimized value of four kilobytes, which becomes the page size of storage 220. For example, a 4 Kilobyte (KB) logical block operation from the host will result in a physical read or write on storage device 220. In some embodiments, a sector is an allocated unit of area for the storage device 220; that is, throughout a pool, zones are striped, and each storage device 220 receives sector-sized blocks of data. In other embodiments, the sectors are contained in a single storage device 220, and all sectors in the pool are of the same size. A portion of the zones may be evenly distributed or mapped into a predetermined number of storage devices or drives 220 based on a storage device or drive index, where the predetermined number may be, for example, a power of two. In some embodiments, a zone may include a plurality of contiguous memory blocks.

FIG. 3A illustrates an operational flow diagram of a mapping function or mapping process 300 for mapping a storage volume or logical volume into multiple pools (e.g., pools 216.0-216.3 of FIG. 2) of logical blocks or storage devices, similar to mapping function 212 of FIG. 2, according to an embodiment of the present invention. In step 304, the mapping process 300 receives a request for a logical volume from a host (e.g., host 0(204.0) of FIG. 2). In some embodiments, the host is identified with a host identification (e.g., host 0(204.0) of fig. 2 has a host Identification (ID) of "0"). The request may also include the size of the requested logical volume, which is also associated with the logical volume identification. In step 308, the mapping process 300 determines the host identity and logical volume identity from the request. Based on the host identity, mapping process 300 accesses a volume identification table at step 312, which will be described in detail below (e.g., volume table 440 of FIG. 4). In some embodiments, each host (such as host 204.0 of FIG. 2) is assigned a particular volume identification table. At step 316, the mapping process 300 determines the volume size (e.g., from the request). At step 320, based at least in part on the volume size, the mapping process 300 allocates storage volumes for the request in one or more storage devices similar to the storage devices 132.1.1-132.1.8 of pool 1 of FIG. 1 or the storage devices 220.0.0-220.0.7 of pool 0 of FIG. 2. The mapping process 300 also optionally distributes or inserts redundancy among the assigned storage volumes at step 324. For example, in some embodiments, such as those used in redundant array of inexpensive (independent) disks levels 1 and 0 or RAID10, mapping process 300 may include striping and mirroring of the allocated storage volumes. In some embodiments, similar to RAID6, mapping process 300 may insert parity blocks into the assigned storage volume. While RAID10 and RAID6 have been described, other redundancy techniques such as RAID2, RAID4, RAID5, etc. may be used as desired.

FIG. 3B illustrates an operational flow diagram of an allocation process 350 for allocating storage space or volumes in a storage device similar to step 320 of the mapping process 300 of FIG. 3A, according to an embodiment of the present invention. In step 354, the allocation process 350 receives an allocation request. In the illustrated embodiment, for example, from the request as discussed for mapping process 300 of FIG. 3A, the allocation request includes a volume size, a host identification (e.g., host 0(204.0) of FIG. 2 has a host identification of "0"), and a logical volume identification (e.g., volume 0.1(208.0.1) of FIG. 2 has a logical volume identification of "0.1"). In step 358, the allocation process 350 uses a mapping function to generate a pool identification based on the host identification and the logical volume identification. In some embodiments, the mapping function (e.g., 212 of fig. 2) is a hash function (hash function) that randomly determines a mapping value or a hash value (hash value). In some embodiments, the pool identification identifies and assigns which pool of storage 216 that will use and allocate the requested storage space, e.g., pool 216.0 of storage 220.0.0-220.0.9 of FIG. 2.

After the pool identification has been generated, the allocation process 350 determines whether the identified pool has the requested available storage space based on the volume size in step 362. If it is determined at step 362 that the requested volume size is not available in the identified pool, the allocation process 350 generates a new pool identification and determines at step 366 that the newly identified pool has the requested storage space based on the volume size. If it is determined at step 362 that the requested tape size is available in the identified pool, the allocation process 350 creates an entry for the requested roll in the identified pool at step 370. If, at step 366, it is determined that the newly identified pool has the requested storage space based on the volume size, the allocation process 350 likewise proceeds to step 370 to create an entry for the requested volume in the newly identified pool. However, if at step 366 it is determined based on the volume size that the newly identified pool cannot provide the requested storage space, then the allocation process 350 returns an error message indicating that the requested storage space is not available.

Fig. 4 illustrates an exemplary address map flow 400 similar to the assignment process 350 of fig. 3B in accordance with the present invention. The address mirror flow 400 shows a host address 404. Host address 404 lists host identification 408 (e.g., host 204.0 of fig. 2), logical volume identification 412 identifying the volume as seen by host 204, zone address or zone 416, and Logical Block Address (LBA) displacement address 420. As discussed above with respect to fig. 3B, the mapping pool identification 424 is determined by the host identification 408 and the logical volume identification 412 using the mapping function 212 (which may be a random hash function, for example). In some embodiments, the mapping function 212 evenly distributes the pool identification 424, e.g., to reduce or mitigate hot spotting (hot spotting) -two or more hosts 204 are assigned to and active in the same pool. In some embodiments, the address map flow 400, and in particular, the pool identification 424, is determined using the assignment process 350 of FIG. 3B.

In the embodiment illustrated in FIG. 4, pool identification 424 refers to an exemplary pool 428 (e.g., pool 216.0 of FIG. 2). The pool 428 includes ten storage devices 432.0 through 432.9. Each of storage devices 432.0 through 432.9 is subdivided into 2^w1 strips 436.0.0 to 436.0.2^w-1 (with the generic index 436.i.j representing the (j +1) th stripe of the (i +1) th storage device, and with a pass representing the generic stripe in the (i +1) th storage deviceWith index 436. i). For example, stripe 436.9 refers to the stripe in storage 432.9 and stripe 436.1 refers to the stripe in storage 432.1. Similarly, stripe 436.4.2 refers to stripe No. 3 in storage device 432.4.

Each identified pool is also associated with a volume table that holds records of addresses or entries used by the requested volume. In the illustrated embodiment, pool 428 is associated with volume table 440. In the illustrated embodiment, the volume table 440 lists two columns — a device identification (SS ID)440.1 and a stripe identification (PS ID) 440.2. In some embodiments, the device identification 440.1 identifies which of the storage devices 432.0 through 432.9 will be used by the requesting volume, and the stripe identification 440.2 identifies which stripe of the identified storage device will be the starting stripe for the requesting volume. The device identification 440.1 and stripe identification 440.2 form a drive index that identifies the stripe address of the device in the pool associated with the requested volume, as described in detail below. In addition, in some embodiments, volume table 440 may also include other optional information, such as, for example, available regions in the pool defined by the beginning of the device identification and stripe identification.

FIG. 4 also shows that region 416 is mapped to pool 428 by treating storage 432.0 through 432.9 with pool 428 as a two-dimensional array or table. The two-dimensional array thus comprises a plurality of coordinates, indices or drive indices, where throughout the plurality of storage devices 432.0 to 432.9, a column represents an SS ID and a row represents a PS ID or stripe 436.i. In the illustrated embodiment, zones 416 are mapped to storage devices 432.0 through 432.9 as a contiguous grouping of sectors or stripes 436.i. As such, zone 416 may start with any SS ID and PS ID coordinates or drive index in the identified pool (e.g., pool 428), as determined by the mathematical relationship between the number of storage devices 432 in pool 428, the number of bytes in stripe 436.i.j, and the number of stripes 436.i in zone 416. The zone 416 is thus placed in the identified pool in an adjacent manner. If the number of stripes 436.i in zone 416 is equal to the number of storage devices 432 in the pool, then zone 416 begins at 432.0. If the number of strips 436.i in a zone (e.g., zone 416) is less than or greater than the number of storage devices 432.0 through 432.9 in pool 428, then address map stream 400 causes the effect of continuous winding along the length of volume table 440. In the illustrated embodiment, zone 416 points to SS ID 440.1.3 and PSID 440.2.3. In this case, the start address, coordinates, or drive index of the requested volume is identified by SS ID 440.1.3 and PS ID 440.2.3, resulting in drive index 436.4.2, where the start address, coordinates, or drive index of the requested volume has an index of storage 432.4 and two offsets into stripe 436.2. Thus, in some embodiments, the index of the storage device or the drive index may be determined based on the mapping value and one or more zones (e.g., zone 416), or by determining a storage device identification of at least one of the storage devices.

FIG. 5 illustrates a first allocation example 500 in which a plurality of hosts 504(504.0 through 504.4) allocate a plurality of volumes 506 (volume A through volume E) in a pool of storage devices 516(516.0 through 516.9) through a mapping function 508. In the first allocation example 500, the mapping function 508 is similar to the mapping process 300 of fig. 3. In the illustrated embodiment, hosts 504.0-504.4 evenly distribute mapping requested volumes (volume a-volume E) to configured volumes 506.0-506.4, respectively, throughout storage devices 516.0-516.9. In the illustrated embodiment, the standard configuration is applied to volumes 506 (volumes a through E) requested by hosts 504.0 through 504.4 without redundancy.

FIG. 6 illustrates a second allocation example 600 in which multiple hosts 604(604.0 through 604.4) allocate multiple volumes 606 (volume A through volume E) in a pool of storage devices 616(616.0 through 616.9) without redundancy through a mapping function 608. In the second allocation example 600, mapping function 608 is similar to mapping process 300 of FIG. 3 and utilizes thin provisioning to map requested volume 606 (volume A to volume E) to thin provisioned volumes 606.0 to 606.4. In the illustrated embodiment, throughout storage 616.0-616.9, hosts 604.0-604.4 uniformly map requested volumes (volume a-volume E) with one or more additions (additions) 620.0 and 620.1 for thin provisioning volume 606.0-606.4 rollovers (overrows). Depending on the application, further additions may also be used.

Fig. 7 shows a third allocation example 700, where multiple hosts 704 (704.0-704.4) allocate multiple volumes (volume a-volume D) in a pool 712 of storage 716 (716.0-716.9) by a mapping function 708, where storage 716 (716.0-716.9) has redundancy in the same pool 712, and pool 712 is subdivided into a first sub-pool 712.1 and a second sub-pool or mirror sub-pool 712. m. In the third allocation example 700, the mapping function 708 is similar to the mapping process 300 of FIG. 3 and utilizes thin provisioning to map the requested volumes (volume A to volume D) into thin provisioned volumes 706.0 to 706.3. In the third allocation example 700, redundancy is also embedded or applied. In the illustrated embodiment, the redundancy of the application includes RAID10, which also includes striping and mirroring. In this case, redundancy includes striping the thin provisioned volumes 706.0 through 706.3 in pool 712.1 and mirroring the thin provisioned volumes 706m.0 through 706m.3 in the corresponding pool 712. m. More specifically, throughout the storage 716.0-716.4, with one or more additions for the overturning of the thin-provisioned volumes 706.0-706.3 of the sub-pool 712.1, the hosts 704.0-704.3 map the requested volumes (volume a-volume D) evenly to the thin-provisioned volumes 706.0-706.3 using the drive index as described above. Similarly, thin-provisioned volumes 706.0 through 706.3 and thin-provisioned appendages 720.m are also mirrored with the mirrored drive indexes in mirrored storages 716.5 through 716.9 in sub-pool 712.m, creating multiple mirrored volumes 706m.0 through 706m.3 and multiple thin-provisioned appendages 720m.0 and 720 m.1.

FIG. 8 illustrates a fourth allocation example 800 in which multiple hosts 804(804.0 through 804.4) allocate multiple volumes 806 (volume A through volume E) throughout different pools 812.1 and 812.m through a mapping function 808 in a first pool 812.1 and a different second or mirror pool 812.m using thin provisioning and redundancy. In the fourth allocation example 800, the first pool 812.1 has storage devices 816.0 through 816.9, and the second pool 812.m has mirrored storage devices 816m.0 through 816 m.9. Further, the mapping function 808 is similar to the mapping process 300 of FIG. 3 and maps the requested volumes (volume A to volume E) to the thin provisioned volumes 806.0 through 806.4 of the first pool 812.1. In the illustrated embodiment, the redundancy of the application includes RAID10, which also includes striping and mirroring as described above. In particular, throughout the storage devices 816.0-816.9 in the first pool 812.1, hosts 804.0-804.4 map requested volumes (volume a-volume E) evenly to thin-provisioned volumes 806.0-806.4 with one or more thin-provisioned appends 820.0 and 820.1 that are used to tip thin-provisioned volumes 806.0-806.4 of the first pool 812.1. Similarly, thin-provisioned volumes 806.0-806.4 and thin-provisioned appends 820.0 and 820.1 are mirrored in mirrored storage 816 m.0-816 m.9 in the second pool 812.m, creating a plurality of mirrored thin-provisioned volumes 806 m.0-806 m.4 and a plurality of mirrored thin-provisioned appends 820m.0 and 820 m.1. Thus, in this embodiment, the requested volume 806 is allocated across two different pools 812.1 and 812.m, with pool 812.m being a mirror of pool 812.1.

FIG. 9 illustrates a fifth allocation example 900 in which multiple hosts 904(904.0 through 904.4) allocate multiple volumes 906 (volume A through volume E) through a mapping function 908 using thin provisioning and redundancy across a pool of storage devices 916(916.0 through 916.9). In the fifth allocation example 900, the mapping function 908 is similar to the mapping process 300 of FIG. 3 and maps the requested volumes (volume A through volume E) to the parity-checked volumes 906.0 through 906.4. In the illustrated embodiment, the redundancy of the application includes RAID6, which also includes embedded PQ parity blocks. Further, throughout the storage 916.0-916.9 in the pool, hosts 904.0-904.4 map the requested volumes (volume a-volume E) evenly to the parity-checked volumes 906.0-906.4 with one or more thin-provisioned appends 920.0 and 920.1 that are used to overwrite the parity-checked volumes 906.0-906.4 of the pool. In the illustrated embodiment, a plurality of parity blocks 906pq.0 through 906pq.4, 920pq.0, and 920pq.1 are embedded in the parity-checked volumes 906.0 through 906.4 in the pool and the thin-provisioned append 920.

Fig. 10 illustrates a first exemplary Mean Time Between Failure (MTBF) failure report 1000 utilizing the mapping process 300 of fig. 3 and RAID10 redundancy (including 100% redundancy). In the first exemplary MTBF failure report 1000, pool 1004 of storage devices 1004.0-1004.7 (similar to pool 812.1 of fig. 8) is mirrored in mirrored pool 1008 (similar to pool 812.m of fig. 8) of storage devices 1008.0-1008.7. In the illustrated embodiment, multiple sectors 1004.1 and 1004.3 in pool 1004 having 4 Kilobyte (KB) pages have been determined to be faulty, as have additional multiple mirror sectors 1008.6 and 1008.7 in a different mirror pool 1008. If page level failure reports are used in the recovery data for the failed sector, recovery will not be achieved. However, in accordance with the present invention, in response to having determined that the storage has failed, the system 100 can recover the data of a complete 4KB page 1012 by utilizing sector level failure reports and by accessing (e.g., simultaneously accessing) a pool of storage or drives. In the illustrated embodiment, failed sectors 1004.1 and 1004.3 are replaced by their respective mirror versions (mirrorversion) 1008.1 and 1008.3, while failed mirror sectors 1008.6 and 1008.7 can be replaced by sectors 1004.6 and 1004.7.

Fig. 11 illustrates a second exemplary MTBF failure report 1100 utilizing the mapping process 300 of fig. 3 and RAID6 redundancy (including 25% redundancy). In the second exemplary MTBF failure report 1100, pool 1104 of storage devices 1104.0 through 1104.7 (similar to 906.0 of fig. 9) has its PQ parity block stored in parity pool 1108 of storage devices 1108.0 through 1108.7 (similar to 906pq.0 of fig. 9). In the illustrated embodiment, multiple sectors 1104.1 and 1104.3 in pool 1104 with 4KB pages have failed, as have additional multiple parity sectors 1108.6 and 1108.7 in a different parity pool 1108. In accordance with the present invention, in response to having determined that a storage device (e.g., storage device 220.0.0 of FIG. 2) has failed, system 100 can recover the data of a complete 4KB page 1112 by utilizing sector level failure reports and by accessing (e.g., concurrently accessing) a pool of storage devices or drives (e.g., pool 216.0 of FIG. 2). In the illustrated embodiment, failed sectors 1104.1 and 1104.3 originate from parity sectors 1108.0 and 1108.1.

The present invention has been described in accordance with the embodiments shown, and variations to the embodiments are possible and any variations would be within the spirit and scope of the present invention. For example, the illustrative embodiments may be implemented using hardware, software, a computer-readable medium containing program instructions, or a combination thereof. Software written in accordance with the present invention will be stored in some form of computer readable medium (e.g., memory, hard disk, or CD/DVD-ROM) and executed by a processor.

While the invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiments disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.

In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve suitable results. Other steps may also be provided, or steps may be eliminated, from the described flows, other components may be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A method of mapping a storage volume to a plurality of pools of storage devices, wherein the storage devices are designated by a host having a host identification, and a data storage volume has a volume identification and a plurality of zones, the method comprising:

assigning a first pool of storage devices to a storage volume based on a host identification;

for a first pool storage device, determining a mapping value based on a host identity and a volume identity;

determining a storage index based on the mapped value and one or more of the plurality of zones; and

a portion of the plurality of regions is mapped to a first pool storage based on a storage index.

2. The method of claim 1, wherein the determining of the mapping value comprises: a hash value is determined based on the host identity and the volume identity.

3. The method of claim 2, wherein the determining the hash value comprises: the hash value is randomly determined based on the host identity and the volume identity.

4. The method of claim 1, wherein the method further comprises:

allocating a second pool of storage for the storage volume based on the host identification; and

mapping remaining ones of the plurality of regions to a second pool storage based on a storage index.

5. The method of claim 1, wherein at least one of the storage devices in the first pool of storage devices comprises one or more solid state disks.

6. The method of claim 1, wherein determining a storage index comprises: a storage device identification of at least one of the storage devices in the first pool of storage devices is determined.

7. The method of claim 1, wherein at least one of the storage devices in the first pool storage device comprises a plurality of storage device stripes, and wherein determining the storage device index comprises determining an index of one of the storage device stripes.

8. The method of claim 1, wherein the step of mapping a portion of the plurality of zones comprises: the portions of the plurality of zones are uniformly mapped into a predetermined number of storages in a first pool of storages based on a storage index.

9. The method of claim 8, wherein the predetermined number of storage devices is a power of two.

10. The method of claim 1, wherein the plurality of zones comprises a plurality of contiguous memory blocks.

11. The method of claim 1, further comprising: for a storage volume, redundancy is inserted into a first pool storage.

12. The method of claim 11, wherein the step of inserting redundancy comprises mirroring at least a portion of the storage volume in the first pool storage device.

13. The method of claim 11, wherein inserting redundancy comprises striping at least a portion of the storage volume in the first pool storage device.

14. The method of claim 11, wherein inserting redundancy comprises inserting parity blocks into the first pool storage device.

15. The method of claim 11, the method further comprising:

determining whether one or more storage devices of the first pool of storage devices have failed; and

in response to having determined that the one or more storage devices have failed, data of the one or more storage devices is recovered with inserted redundancy.

16. The method of claim 1, further comprising simultaneously accessing one or more of the first pool storage devices.

17. A network storage system, the network storage system comprising:

a host having a host identity and configured to map a storage volume having a volume identity and a plurality of zones;

a plurality of pools of storage devices configured to distribute at least a portion of the plurality of regions of a storage volume designated by a host;

a processor configured to: assigning a first pool from the plurality of pools of storage to a storage volume based on a host identification; determining, for an assigned pool of storage devices, a mapping value based on a host identification and a volume identification; determining a storage index based on the mapped value and one or more of the plurality of zones; and mapping the at least a portion of the plurality of regions to a first pool of the plurality of pools from storage based on a storage index.

18. The network storage system of claim 17, wherein the mapping value comprises a hash value.

19. The network storage system of claim 18, wherein the hash value is determined randomly.

20. The network storage system of claim 17, wherein the processor is further configured to: assigning a second pool of the plurality of pools from a storage device for the storage volume based on the host identification, and mapping remaining regions of the plurality of regions to the second pool of the plurality of pools from the storage device based on a storage device index.

21. The network storage system of claim 17, wherein one or more of the plurality of pools of storage devices comprise one or more solid state disks.

22. The network storage system of claim 17, wherein the storage index comprises a storage identification of at least one of the storages in the plurality of pools of storages.

23. The network storage system of claim 17, wherein at least one of the storage devices in a first pool of the plurality of pools of storage devices comprises a plurality of storage device stripes, wherein the step of determining a storage device index comprises determining an index of one of the storage device stripes.

24. The network storage system of claim 17, wherein the processor is further configured to: the portions of the plurality of zones are uniformly mapped into a predetermined number of storage devices in a first pool of the plurality of pools from a storage device based on a storage device index.

25. The network storage system of claim 24, wherein the predetermined number of storage devices is a power of two.

26. The network storage system of claim 17, wherein the plurality of zones comprises a plurality of contiguous memory blocks.

27. The network storage system of claim 17, wherein the processor is further configured to: for a storage volume, redundancy is inserted into a first pool of the plurality of pools from a storage device.

28. The network storage system of claim 27, wherein redundancy comprises mirroring at least a portion of a storage volume in a first pool from the plurality of pools of storage devices.

29. The network storage system of claim 27, wherein redundancy comprises striping at least a portion of a storage volume in a first pool from the plurality of pools of storage devices.

30. The network storage system of claim 27, wherein the redundancy comprises parity blocks.

31. The network storage system of claim 17, wherein the processor is further configured to: the method further includes determining whether one or more storage devices of the first pool of storage devices have failed, and, in response to determining that one or more storage devices have failed, recovering data of the one or more storage devices with the inserted redundancy.

32. The network storage system of claim 27, wherein the processor is further configured to access storage devices in a first pool of the plurality of pools from the storage devices simultaneously.

33. A method of mapping a storage volume to a plurality of storage devices, wherein the plurality of storage devices are designated by a host having a host identification, a data storage volume having a volume identification and a plurality of zones, the method comprising:

determining, for the plurality of storage devices, a hash value based on a host identity and a volume identity;

obtaining a storage index based on one or more of the plurality of zones;

assigning a storage address of one of the storages from a first pool of a plurality of pools of storages based on a storage index;

mapping a portion of the plurality of regions to a first pool of the plurality of pools from a storage device using an assigned storage device address;

inserting redundancy behind the portion of the plurality of zones; and

mapping a remaining portion of the plurality of regions after the inserted redundancy.

34. The method of claim 33, wherein the hash value is determined based on the host identity and the volume identity, and the hash value is determined randomly.

35. The method of claim 33, wherein the method further comprises grouping the plurality of storage devices into a plurality of pools of equal number of storage devices.

36. The method of claim 33, wherein one of the storage devices in the first pool storage device comprises one or more solid state disks.

37. The method of claim 33, wherein the step of obtaining the storage index comprises: a storage device identification of at least one of the storage devices from a first pool of the plurality of pools of storage devices is determined from a predetermined volume table.

38. The method of claim 33, wherein at least one of the storage devices in the first pool of the plurality of pools from the storage devices comprises a plurality of storage device stripes, and wherein obtaining the storage device index comprises determining an index of one of the storage device stripes from a predetermined volume table.

39. The method of claim 33, wherein mapping a portion of the plurality of zones comprises mapping the portion of the plurality of zones evenly into a predetermined number of storage devices.

40. The method of claim 39, wherein the predetermined number of storage devices is a power of two.

41. The method of claim 33, wherein the plurality of zones comprises a plurality of contiguous memory blocks.

42. The method of claim 33, wherein inserting redundancy comprises mirroring at least a portion of a storage volume in at least one of the plurality of storage devices.

43. The method of claim 33, wherein inserting redundancy comprises striping at least a portion of a storage volume in at least one of the plurality of storage devices.

44. The method of claim 33, wherein inserting redundancy comprises inserting parity blocks in at least one of the plurality of storage devices.

45. The method of claim 33, the method further comprising:

determining whether one or more storage devices from a first pool of the plurality of pools of storage devices have failed; and

in response to having determined that the one or more storage devices have failed, data of the one or more storage devices is restored with the inserted redundancy.

46. The method of claim 33, further comprising accessing one or more of the plurality of storage devices simultaneously.