JP4852118B2 - Storage device and logical disk management method - Google Patents

Description

  The present invention relates to a storage apparatus and a logical disk management method that can be used continuously even when a temporary failure such as a timeout occurs.

  Conventionally, in a storage apparatus, when a failure occurs in a physical disk that constitutes a logical disk, the physical disk in which the failure has occurred is disconnected and a degeneration operation is performed. For example, in a RAID-configured storage apparatus, if a temporary failure such as a timeout occurs when accessing a physical disk, it is regarded as a failure. Then, the physical disk in which the temporary failure has occurred is disconnected and the degenerate operation is performed.

  However, even a physical disk that has been separated by detecting a temporary failure may not have failed due to subsequent inspection. For example, it may be possible to restore the normal state by turning on the power of the physical disk again. In such a case, although the physical disk is not actually failed, it is treated as a failure, and a problem of increasing the frequency of failure occurs.

Therefore, there is a method in which a physical disk in which a temporary failure has occurred is once disconnected from the logical disk, and then the physical disk is subjected to diagnostic processing. Patent Document 1).
JP 2006-164304 A "Phoenix Technology", [online], [Search June 19, 2008], Internet <URL: http://www.nec.co.jp/products/istorage/words/m036.shtml>

  However, the above-described conventional method also has a problem that the redundancy of the logical disk is lost from the time when the physical disk is disconnected until the logical disk is reconfigured.

  Further, for example, in a RAID-5 configuration storage device, failure of a plurality of physical disks is not supported, and while one physical disk has a failure, a failure has occurred in another physical disk. Can not respond.

  Therefore, if a failure occurs on another physical disk between the time when the physical disk determined to be failed and the time when the logical disk is reconfigured, the logical disk operation stops and data is lost. It will be.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a storage apparatus and a logical disk management method that can be used continuously even when a temporary failure such as a timeout occurs.

In order to solve the above problems, the present invention includes at least one physical disk, a hot spare disk, and a disk controller that constitute a logical disk, and stores data in response to a request from a host device connected via a network. In the storage apparatus, the disk controller detects a temporary failure that occurs in the physical disk, and, when detecting the failure, the disk controller detects a failure in the failed physical disk. Failure recovery means for performing failure recovery processing, data replication means for copying data of the failed physical disk to the hot spare disk, monitoring the failed physical disk for a certain period after the start of failure recovery processing, and the host A method of recording response data of the failed physical disk in response to a command from the device And the recorded response data and reference response data to determine whether or not the failure physical disk is in failure, and if the failure determination means determines that there is a failure, the failure occurrence physical Provided is a storage device comprising a logical disk reconfiguration unit that uses the hot spare disk as a physical disk constituting the logical disk instead of a disk.

  According to the present invention, it is possible to provide a storage apparatus and a logical disk management method that can be used continuously even when a temporary failure such as a timeout occurs.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a schematic diagram showing a configuration of a storage apparatus 10 according to the first embodiment of the present invention. The storage apparatus 10 includes one or more physical disks 21, a hot spare disk 22, and a disk controller 30 that constitute the logical disk 20. The storage device 10 is connected to the host device 5 via a network by SCSI (small computer system interface), FC (Fibre Channel), etc., and stores data in response to a request from the host device 2 Secondary storage device. The physical disk 21 is generally an HDD (hard disk drive), but is not limited to this, and may be a storage device including a semiconductor disk. The hot spare disk 22 replaces a physical disk in which a failure has occurred.

  The disk controller 30 includes a memory 40 and a processor 50, and when a “logical disk management program” stored in the memory 40 is read by the processor 50, a logical disk setting unit 51 and a temporary failure detection unit 52 are included. It functions as a failure recovery unit 53, a data replication unit 54, a disk monitoring unit 55, a failure determination unit 56, a logical disk reconstruction unit 57, and a warning data output unit 58. In FIG. 1, the processing units 51 to 58 are described inside the processor, but this is a representation for convenience. That is, each of the processing units 51 to 58 is programmed as part of the function of the logical disk management program, and is realized by the processor 50 executing the program.

  The memory 40 is a storage device that stores data processed by the disk controller 30. The memory 40 stores preset “reference response data”. Also, “response data” is written in the memory 40 by a disk monitoring unit 55 described later.

  The normal average response time differs depending on the HDD model, and even the same HDD varies depending on the IO load (queuing depth), and a value corresponding to this varies as the reference response data. For example, table data for each HDD model supported by the storage apparatus 10 or a value that can be reliably determined as abnormal assuming any model and environment is used as the reference response data. The logical disk setting unit 51 sets one or more physical disks 21 together as a logical disk 20. This implements a RAID (redundant array of inexpensive disks) function. The logical disk 20 may or may not have redundancy depending on the type of RAID.

  The temporary failure detection unit 52 detects a temporary failure that occurs in the physical disk 21. For example, the temporary failure detection unit 52 detects a temporary failure from a timeout or the like when a write request is received from the host device 5.

  When the temporary failure detection unit 52 detects a temporary failure, the failure recovery unit 53 performs failure recovery processing on a physical disk in which the temporary failure has occurred (hereinafter referred to as a failure-occurring physical disk 21X). is there. For example, the failure recovery unit 53 performs a failure recovery process by device reset, physical disk power off / on, or the like. Note that while the failure recovery unit 53 is performing the failure recovery processing, access from the host device 5 to the failed physical disk 21X is stopped, and the I / O processing is resumed when the failure recovery processing is completed.

  The data replicating unit 54 replicates the data of the failed physical disk 21X on the hot spare disk 22 and mirrors it. Here, since the data of the hot spare disk 22 is a mirror of the failed physical disk 21X, the data replicating unit 54 can copy the entire data from the failed physical disk 21X. In addition, when the logical disk 20 has a redundant RAID configuration, the data replicating unit 54 selects a hot spare disk from member disks other than the failed physical disk 21X among the member disks of the logical disk 20, as shown in FIG. It is also possible to restore the 22 data.

  The disk monitoring unit 55 monitors the failed physical disk 21X for a certain period after the start of the failure recovery processing, and has a function of recording the response data of the failed physical disk 21X in response to a command from the host device 5 in the memory 40. Have. Specifically, the disk monitoring unit 55 performs a certain period (for example, 24 hours) after data restoration to the hot spare disk 22 during the restoration of data to the hot spare disk 22 after the first temporary failure has occurred in the physical disk 21. ), The I / O pattern of the failed physical disk 21X is recorded.

  The failure determination unit 56 compares the response data recorded after the failure recovery process with the reference response data stored in the memory 40 to determine whether or not the failure physical disk 21X has a failure. For example, the failure determination unit 56 determines whether or not the failure-occurring physical disk 21X has a failure from the I / O response delay and other abnormal operations of the physical disk 21.

  Note that when the data replication to the hot spare disk 22 is completed, the failed physical disk 21X and the hot spare disk 22 are operating in a state where the data is mirrored.

  As shown in FIG. 3, when the failure determination unit 56 determines that the failed physical disk 21X has failed, the logical disk reconfiguration unit 57 replaces the failed physical disk 21X with the hot spare disk 22 as a logical disk. It is incorporated as 20 member disks.

  In addition, when the hot spare disk 22 is incorporated as a member disk of the logical disk 20, the logical disk reconstruction unit 57 cancels the monitoring of the failed physical disk 21X by the disk monitoring unit 55 and ends the recording of response data. At this point, the failure of the failed physical disk 21X is confirmed.

  If a temporary failure is detected again on the failed physical disk 21X to be monitored during restoration of data to the hot spare disk 22 or during a certain period after data restoration, the logical disk reconfiguration unit 57 At that time, the failed physical disk 21X is disconnected from the logical disk 20, and the hot spare disk 22 is assigned as a member disk of the logical disk 20.

  If no abnormal operation is observed for the failed physical disk 21X, as shown in FIG. 4, the logical disk reconstruction unit 57 assumes that the failure that has occurred in the failed physical disk 21X is temporary, and the disk The monitoring by the monitoring unit 55 is canceled and the recording of response data is terminated. Then, the logical disk reconstruction unit 57 cancels the mirror configuration of the failed physical disk 21X and the hot spare disk 22. As a result, the logical disk 20 returns to the original state.

  The warning data output unit 58 outputs warning data when the failure determination unit 56 determines that the failed physical disk 21X has a failure.

  Next, the operation of the storage apparatus 10 according to this embodiment will be described with reference to the flowchart of FIG. In the disk controller 30, the temporary failure detection unit 52 is always in operation, and when a temporary failure occurs in the physical disk 21, this is detected by the temporary failure detection unit 52 (S1-Yes). Subsequently, the failure recovery unit 53 executes failure recovery processing for the failed physical disk 21X (S2). In the failure recovery processing, device reset and power on / off are executed.

  When the failure recovery processing of the failed physical disk 21X is started by the failure recovery unit 53, the hot spare disk 22 is assigned to the failed physical disk 21X by the data replication unit 54 (S3). As a result, the hot spare disk 22 is configured as a mirror disk of the failed physical disk 21X. Then, the data replicating unit 54 replicates the data of the failed physical disk 21X to the hot spare disk 22.

  When the failure recovery processing of the failed physical disk 21X is started by the failure recovery unit 53, the response data of the failed physical disk 21X to the command from the host device 5 is recorded in the memory 40 by the disk monitoring unit 55 ( S4).

  Subsequently, the failure determination unit 56 compares the response data recorded after the failure recovery process with the reference response data stored in advance in the memory 40 to determine whether or not the failure physical disk 21X has a failure. (S5).

  If the failure determination unit 56 determines that there is a failure (S5-Yes), the logical disk reconfiguration unit 57 reconfigures the hot spare disk 22 as a member disk of the logical disk 20 instead of the failed physical disk 21X. (S6). When the logical disk 20 is reconfigured, the failed physical disk 21X is disconnected (S7).

  On the other hand, if the failure determination unit 56 does not determine that there is a failure, the failed physical disk 21X is continuously used (S5-No, S8). Since the failed physical disk 21X and the hot spare disk 22 are mirrored, the hot spare disk 22 is incorporated into the logical disk instead of continuing to use the failed physical disk 21X, and the disk that has been determined as the failed disk is the hot spare disk. May be used as

  As described above, in the storage apparatus 10 according to the present embodiment, the disk controller 30 detects the temporary failure that occurs in the physical disk 21, and the occurrence of a failure in which a temporary failure has occurred. A failure recovery unit 53 that performs failure recovery processing on the physical disk 21X and a data replication unit 54 that replicates data of the failed physical disk 21X to the hot spare disk 22 are provided, and the failed physical disk 21X and the hot spare disk 22 are mirrored. By doing so, even if a temporary failure such as a timeout occurs, it can be used continuously.

  Further, the disk controller 30 monitors the failed physical disk 21X for a certain period after the failure recovery process is started (after the failed physical disk 21X and the hot spare disk 22 are mirrored), and the failed physical for the command from the host device 5 is monitored. Since the response data of the disk 21X is recorded, it is possible to provide information for determining whether or not the failure physical disk 21X has failed and what kind of failure has occurred. For example, the system administrator can analyze the cause of failure of the failed physical disk 21X from the log of the response data, and finally until the temporary failure of the failed physical disk 21X separated as a failure. By reproducing the I / O sequence, the cause of the failure can be easily verified.

  In addition, since the failure determination unit 56 that determines whether or not the failure physical disk 21X has failed, the disk controller 30 compares the response data recorded after the failure recovery processing with the reference response data, The presence or absence of a failure can be confirmed without disconnecting the failed physical disk 21X from the logical disk 20. As a result, it is possible to avoid the problem of loss of redundancy of the logical disk 20 that occurs due to the failure determination work.

  If the failure determining unit 56 determines that the failed physical disk 21X is defective, the hot spare disk 22 is incorporated in the member disk of the logical disk 20 instead of the failed physical disk 21X, and it is determined that there is a failure. Even in such a case, the storage apparatus 10 can be used continuously.

  In the conventional storage apparatus, as shown in FIG. 6A, when a failure occurs in the normal physical disk 21 (A1), the failed physical disk 21X is disconnected (A2), and the hot spare disk 22 is logically set as a member disk. The disk 20 is rebuilt (A3). At this time, there is a problem that the redundancy of the logical disk 20 cannot be maintained until the rebuilding of the hot spare disk 22 is completed (A4).

  On the other hand, as shown in FIG. 6B, when a failure occurs in the normal physical disk 21 (B1), the storage apparatus 10 according to this embodiment transfers the data of the failed physical disk 21X to the hot spare disk 22. (B2), and the failed disk 21X and the hot spare disk 22 are mirrored for a certain period. Then, an I / O pattern for the command from the host device 5 during the mirror operation is recorded (B3), and it is determined whether or not the failure physical disk 21X has a failure (B4 / B5). Therefore, the logical disk 20 can be used without losing redundancy because the failure physical disk 21X is mirrored with the hot spare disk 22 without being disconnected only when a temporary failure occurs.

  Further, when the failure determination unit 56 determines that the failed physical disk 21X has failed, the disk controller 30 outputs warning data so that the failed physical disk 21X can be used continuously or the hot spare disk 22 can be used. It is possible to prompt the system administrator to determine whether to switch.

A modification of monitoring by the disk monitoring unit 55 and failure determination by the failure determination unit 56 will be described.
(Monitoring method 1: I / O pattern monitoring)
In the monitoring method 1, the I / O pattern of all commands (or at least the latest several tens of seconds) during the monitoring period issued to the failed physical disk 21X is recorded in the memory. Here, the command I / O pattern includes a command type (CDB image), an issue time, a completion time, and a completion status (sense information). In the sense information, error recovery information such as failure in executing the command at once but success in retrying in the HDD is recorded.

  Also, in the monitoring method 1, the failure determination unit 56 determines that a failure has occurred when the ratio of commands that have been successfully retried to all issued commands exceeds a predetermined threshold. (No line break) Even if the determination based on the above-described determination criterion is not performed, if it is determined that a failure has occurred by another monitoring method, information on the I / O pattern is required. For example, when the failure physical disk 21X is collected and a reproduction test is performed, a command I / O pattern is required to reproduce the command sequence up to the temporary failure. Therefore, it is desirable to use this I / O pattern recording together even when the monitoring method described below is performed.

(Monitoring method 2: Response time monitoring)
In the monitoring method 2, detailed statistics of the response time to the copy command to the hot spare disk 22 or the command from the host device 5 are recorded in the memory. If there is sufficient capacity to store the above I / O pattern records over the entire monitoring period, this information need only be statistically processed.

  In the monitoring method 2, the failure determination unit 56 compares the response time obtained by the disk monitoring unit 55 with the reference response time stored in advance in the memory 40, and according to a ratio exceeding a predetermined threshold value. It is determined whether or not a failure has occurred. For example, if a command whose response time exceeds 1 second exceeds 10%, it is determined that a failure has occurred.

(Monitoring method 3: Throughput monitoring)
In the monitoring method 3, the read throughput is measured for a full-face read request to the failed physical disk 21X when copying data to the hot spare disk 22. However, since the failed physical disk 21X receives a command from the host device 5 in parallel, the correction process for the data change by this is separately executed.

  The failure determination unit 56 holds the throughput performance that the failure-occurring physical disk 21X should have as table data, and determines whether or not there is a failure from the performance difference between the table data and the actually measured value. For example, if this performance difference falls below a certain threshold value (for example, 50%), it is determined that there is a failure.

(Monitoring method 4: Error information monitoring by SMART function)
In the monitoring method 4, error information obtained by the SMART function of the HDD is acquired. A general HDD is equipped with a SMART function, and the SMART function monitors an internal error of the HDD itself. Since this error information can be referred to from the outside, the disk monitoring unit 55 acquires it. Error information that can be acquired by the SMART function includes a read error rate, a write error rate, a seek error rate, the number of remaining alternate sectors, a spin-up time, a G list update frequency, a device temperature, and the like. However, since the spin-up time is recorded when the power is turned on, it should not be monitored by the disk monitoring unit 55.

  Further, the failure determination unit 56 periodically refers to each parameter other than the spin-up time (for example, every 1 minute), and determines that a failure occurs when the value of each referenced parameter or its increment exceeds a threshold value. .

(Monitoring method 5: Monitoring error information similar to the first temporary failure)
In the monitoring method 5, the first temporary detection is performed based on a determination criterion similar to the determination criterion for the temporary failure detection unit 52 to detect a temporary failure (for example, a retry was made due to a timeout or error response, but a retry was made). It is monitored whether error information similar to the failure can be obtained again. The failure determination unit 56 determines that a failure has occurred when the disk monitoring unit 55 obtains again error information similar to the first temporary failure.

With this monitoring method 5, when a temporary failure that does not generally occur occurs in the physical disk 21, it is very rare that the second temporary failure is detected. Can be reduced.
Note that the monitoring method 5 can be used together with the monitoring method 1 to collect a detailed log of the I / O pattern, thereby easily investigating the cause of the failure.

<Others>
The present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine a component suitably in different embodiment.

  Note that the method described in the above embodiment includes a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a magneto-optical disk (MO) as programs that can be executed by a computer. ), Stored in a storage medium such as a semiconductor memory or a semiconductor disk, and distributed.

  In addition, as long as the storage medium can store a program and can be read by a computer, the storage format may be any form.

  In addition, an OS (operating system) running on a computer based on an instruction of a program installed in the computer from a storage medium, MW (middleware) such as database management software, network software, and the like realize the above-described embodiment. A part of each process may be executed.

  Further, the storage medium in the present invention is not limited to a medium independent of a computer, but also includes a storage medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

  Further, the number of storage media is not limited to one, and the case where the processing in the above embodiment is executed from a plurality of media is also included in the storage media in the present invention, and the media configuration may be any configuration.

  The computer according to the present invention executes each process in the above-described embodiment based on a program stored in a storage medium, and is a single device such as a personal computer or a system in which a plurality of devices are connected to a network. Any configuration may be used.

  In addition, the computer in the present invention is not limited to a personal computer, but includes an arithmetic processing device, a microcomputer, and the like included in an information processing device, and is a generic term for devices and devices that can realize the functions of the present invention by a program. .

1 is a schematic diagram showing a configuration of a storage apparatus 10 according to a first embodiment of the present invention. It is a schematic diagram for demonstrating the function of the data replication part 54 which concerns on the embodiment. FIG. 6 is a schematic diagram for explaining functions of a logical disk reconstruction unit 57 according to the embodiment. FIG. 6 is a schematic diagram for explaining functions of a logical disk reconstruction unit 57 according to the embodiment. 4 is a flowchart for explaining the operation of the storage apparatus 10 according to the embodiment. It is a schematic diagram for demonstrating the effect of the storage apparatus 10 concerning the embodiment.

  5 ... Host device, 10 ... Storage device, 20 ... Logical disk, 21 ... Physical disk, 22 ... Hot spare disk, 30 ... Disk controller, 40 ... Memory, 50. .. Processor 51... Logical disk setting unit 52. Temporary failure detection unit 53 53 Failure recovery unit 54 Data replication unit 55 Disk monitoring unit 56 Failure determination unit, 57... Logical disk reconstruction unit, 58.

Claims (4)

A storage device that includes one or more physical disks, a hot spare disk, and a disk controller that constitute a logical disk, and stores data in response to a request from a host device connected via a network,
The disk controller is
Temporary failure detection means for detecting a temporary failure occurring in the physical disk;
When the failure is detected, failure recovery means for performing failure recovery processing on the failed physical disk in which the temporary failure has occurred;
Data duplicating means for duplicating the data of the failed physical disk to the hot spare disk ;
Means for monitoring the failed physical disk for a certain period after the start of failure recovery processing, and recording response data of the failed physical disk in response to a command from the host device;
A failure determination means for comparing the recorded response data with reference response data and determining whether the failure physical disk is a failure;
A logical disk reconfiguration unit configured to use the hot spare disk as a physical disk constituting the logical disk instead of the failed physical disk when the failure determination unit determines that a failure has occurred. Storage device.   2. The storage apparatus according to claim 1, wherein when the failure determination unit does not determine that a failure has occurred, the failure physical disk is continuously used. A logical disk management method used in a storage device that includes one or more physical disks, a hot spare disk, and a disk controller that constitute a logical disk, and that stores data in response to a request from a host device connected via a network. And
The disk controller is
When a temporary failure occurring in the physical disk is detected, a failure recovery process is performed on the failed physical disk in which the temporary failure has occurred, and data of the failed physical disk is copied to the hot spare disk,
Monitoring the failed physical disk for a certain period after the start of the failure recovery process, and recording response data of the failed physical disk in response to a command from the host device;
The response data recorded after the failure recovery process is compared with reference response data to determine whether or not the failed physical disk is faulty,
If the result of the determination is that there is a failure, the hot spare disk is used as a physical disk constituting the logical disk instead of the failed physical disk .   4. The logical disk management method according to claim 3, wherein if the result of the determination is that there is no failure, the disk controller continues to use the failed physical disk.

Images