JP2005267056A - Software mirroring disk failure monitoring and recovery system, its failure monitoring and recovery method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a software mirroring disk failure monitoring and a recovery system, its failure monitoring and a recovery method, without requiring the mounting of special hardware. <P>SOLUTION: A system 1 monitoring and recovering a synchronous state of a software mirroring disk is built up, and the system monitors a hard disk drive unit 2 in which software mirroring is carried out. The system has sections 11 and 12 for monitoring mirroring states and errors of disks, sections 13 and 14 for instructing formatting and resynchronous processings to the hard disk drive units, and a section 15 for reporting processing results. These processings can be performed by simple functional expansion from a conventional system by mounting a software module not depending on the mounting of special hardware. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a software mirroring disk failure monitoring / recovery system, a software mirroring disk failure monitoring / recovery method, and a program for executing a software mirroring disk failure monitoring / recovery method.

  One of the important items for maintaining the high availability of computer devices is duplication of hard disk devices. As a duplexing method, software mirroring that does not require special hardware implementation like a disk array device is often used.

  In the conventional technology, when off-track is detected in any of a plurality of disk drives constituting a disk array, the off-track is notified to the array controller, and the servo cylinder on the data surface is re-formatted by re-format processing. As a result, the situation where the off-track exceeds the measurement limit value of the servo cylinder can be resolved, and the disk drive that caused the off-track is restored to the normal operation state without exchanging and repairing the disk drive from the disk array. There is a disk array device that can be used (see Patent Document 1).

  In addition, in a computer system equipped with multiplexed hard disk devices, a hard disk device cannot be read by executing a program for diagnosing soundness on some of the hard disk devices constituting the multiplexed hard disk device. There is a mirror disk diagnostic method that can be detected early during online operation (see Patent Document 2).

If a failure occurs in any of the multiple physical disks that make up the submirror, the physical disk is detached from the submirror as a unit, and when the physical disk is recovered, only the recovered physical disk is resynchronized. Therefore, there is a recording medium control device that can reduce the time required for the resynchronization process (see Patent Document 3).
JP-A-9-167427 JP 2000-029638 A JP 2000-330731 A

Conventionally, if a severe media error occurs on a disk with software mirroring, synchronization of the data block in which the error occurred becomes invalid. To recover the state, replace the disk and re-execute. The method of synchronizing was often adopted. Problems associated with this disk replacement include the following.
1. Expenses (purchase costs) for replacement disk modules.
2. Expenses (personnel costs) are incurred due to human intervention during the exchange.
3. Disk module preparation time and replacement time occur. In some cases, a disk module cannot be obtained, and no immediate action is taken, leaving it in an illegal state for a long time.

  In view of the above-described problems of the prior art, an object of the present invention is to provide a software mirroring disk failure monitoring / recovery system, a failure monitoring / recovery method, and a program for automatically recovering mirroring that do not require special hardware implementation. It is to provide.

The software mirroring disk failure monitoring / recovery system of the present invention
Issue a command to collect the software mirroring status of the disk unit and monitor the data area part where the synchronization error has occurred. If the synchronization error has occurred, the hardware The log storing the error is confirmed, and it is confirmed whether or not the disk media error of the first disk device or the second disk device in the software mirroring configuration is recorded, or the internal statistical information of the disk The disk error check section that confirms that the error count has increased, the disk error check part that identifies the failed disk that caused the synchronization error, and the software mirroring function are stopped, and the disk with the disk media error has occurred. Issue a media format command to the By executing the format, the disk media error occurrence data block is masked and replaced, and the disk format processing unit that confirms that all data areas of the disk can be used normally, and the software mirroring resynchronization command are issued. A mirroring resynchronization processing unit that restores the software mirroring, and a processing result reporting unit that reports each detection content and the execution result of the processing.

  Further, the disk error check unit may notify the error information of the failed disk at the timing when the failed disk causing the synchronization fraud is specified, and arrange a replacement disk.

The load balance system of the present invention is
When the above-mentioned software mirroring disk failure monitoring / recovery system is installed in the disk device under the server of the real server group of the load balance server, and software mirroring recovery processing is performed on the disk device under any of the servers in the real server group, The load balance server detects the server load increase due to the recovery process, suppresses the information allocated to that server, increases the allocation to other servers accordingly, and when the server software mirroring recovery process is completed, Release information suppression.

The software mirroring disk failure monitoring / recovery method of the present invention comprises:
Issue a command to collect the software mirroring status of the disk unit by the mirroring status monitoring unit, and monitor whether there is a data area part where synchronization fraud has occurred. If the error occurred, check the log that stores the hardware error, and check whether the disk media error of the first disk device or the second disk device in the software mirroring configuration has been recorded. Or, by referring to the internal statistical information of the disk, confirm that the error count has increased, identify the failed disk that caused the synchronization error, and use the disk format processing unit to perform software mirroring. Stop the function and generate a disk media error. Issuing a media format command to the specified disk, performing masking and replacement processing of the disk media error occurrence data block, and confirming that all data areas of the disk can be used normally by performing media formatting. The mirroring resynchronization processing unit issues a software mirroring resynchronization command to restore software mirroring, and the processing result reporting unit reports each detection content and the execution result of the processing.

  The step of identifying the failed disk that caused the synchronization error by the disk error check unit notifies the error information of the failed disk and replaces it when the failed disk that caused the synchronization error is identified. The method may further comprise the step of arranging a working disk.

The program of the present invention
A computer or microprocessor is caused to execute the above-described software mirroring disk failure monitoring / recovery method.

  The recovery method of the mirroring system according to the present invention is a software mirroring system using two single hard disk devices. When a mirroring synchronization error is detected due to a medium disk error of one disk, an error disk is automatically formatted. The mirroring system is recovered early by executing and resynchronizing.

  That is, a system for monitoring and recovering the synchronization state of mirroring is constructed, and this system monitors hard disk devices that are software mirrored.

  This system has a part for monitoring the mirroring state / disk error, a part for instructing the disk device to format / resynchronization processing, and a part for reporting the processing result. Any processing can be realized by easy function expansion from a conventional system by implementing a software module independent of special hardware implementation.

  The present invention has the following effects.

  The first effect is that special hardware implementation is unnecessary. It can be easily applied by making the mirroring monitoring / recovery system process resident as a daemon or service on the local operating system. Further, in a remote system, an application such as building an integrated environment for monitoring a plurality of mirror disks under another system is possible.

The second effect is that the following three costs (costs), which have been burdened so far, are reduced or unnecessary by automatically restoring mirroring.
1. 1. Cost of replacement disk module (purchase cost); 2. Expenses (personnel costs) incurred due to human intervention during exchange; Preparation time before replacement and time cost of replacement.

(First Embodiment of the Invention)
FIG. 1 shows the configuration of the software mirroring fault monitoring / recovery system according to the first embodiment of this invention.

  Referring to FIG. 1, the software mirroring failure monitoring / recovery system 1 monitors a disk device 2 having a software mirroring configuration (RAID 1 configuration), and automatically detects the synchronization recovery for the disk device 2 when a synchronization fraud is detected. Instructions for The software mirroring failure monitoring / recovery system 1 includes a mirroring state monitoring unit 11, a disk error check unit 12, a disk format processing unit 13, a mirroring resynchronization processing unit 14, and a processing result reporting unit 15.

  The mirroring state monitoring unit 11 periodically issues a command (instruction) for collecting the software mirroring state of the disk device 2 and monitors whether there is a data area portion in which the synchronization fraud has occurred. If the synchronization fraud has occurred, the disk error check unit 12 confirms the log in which the hardware disk medium error is stored, and the disk medium error of the disk device 21 or the disk device 22 is not recorded. And identify the failed disk that caused the synchronization failure. Alternatively, by referring to the internal statistical information of the disk, a method for confirming that the error count has increased is used to identify the failed disk that caused the synchronization failure.

  The disk format processing unit 13 stops the software mirroring function and issues a media format command to the disk in which a disk media error has occurred. By executing the media format, the disk media error occurrence data block is masked and replaced, and it is confirmed that the entire data area of the disk can be used normally.

  The mirroring resynchronization processing unit 14 issues a software mirroring resynchronization command to restore software mirroring. The processing result reporting unit 15 reports the detection contents of 11 to 14 and the execution result of the processing.

  Next, the operation of the present embodiment will be described in detail with reference to FIGS.

  FIG. 2 shows a flowchart of the operation of the software mirroring failure monitoring / recovery system according to the first embodiment of this invention.

  The mirroring monitoring / recovery system 1 is activated, and the disk device 2 constitutes software mirroring. The mirroring state monitoring unit 11 periodically (polling) issues a mirroring state monitoring command for the disk device 2 (step A1 in FIG. 2). It is confirmed whether or not data area synchronization fraud is detected as a result of the state monitoring command (step A2 in FIG. 2). If no synchronization fraud is detected, the process returns to the periodic monitoring again. If detected, the disk error check unit 12 searches the log storing the hardware error and configures the mirroring. It is confirmed that the media error information of 21 or the disk device 22 is recorded, and the failed disk that caused the synchronization error is specified (step A3 in FIG. 2). As another method of identifying the failed disk of the disk error check unit 12, if the disk is a SCSI disk, internal error statistical information of the disk device 21 or the disk device 22 is collected by periodically issuing a READ_DEFECT / LOG_SENSE command, and the previous time. It is also possible to identify the disk in which a media error has occurred from the difference in the collection information (step A3 in FIG. 2).

  Next, the disk format processing unit 13 issues a media format command (command) to the identified failed disk (step A4 in FIG. 2). By performing the formatting, the media error occurrence data block is masked and replaced, and it is confirmed that the entire data area of the disk can be used normally (step A5 in FIG. 2). If the formatting cannot be completed normally and the disk becomes unusable, the automatic recovery process fails (step A10 in FIG. 2). If the formatting is completed normally, the mirroring resynchronization processing unit 14 issues a resynchronization command (command) (step A6 in FIG. 2). Here, if the processing is interrupted during the resynchronization process, or if a mirror invalid state is detected in the status confirmation after the process, it is determined that the resynchronization has not been completed normally (step A7 in FIG. 2), and automatic recovery is performed. The process fails (step A10 in FIG. 2). If the resynchronization process is completed normally (step A8 in FIG. 2), the mirroring system is restored and reports to that effect (step A9 in FIG. 2). Even if the automatic recovery fails (step A10 in FIG. 2), a mirroring failure is reported (step A11 in FIG. 2), and preparation work for disk replacement is started at that time.

(Second Embodiment of the Invention)
Next, a second embodiment of the present invention will be described with reference to the drawings.

  FIG. 3 shows the configuration of the load balance system according to the second embodiment of the present invention in which the software mirroring failure monitoring / recovery system according to the first embodiment of the present invention is incorporated.

  FIG. 3 shows software mirroring disks in the disk devices 311, 312, 321, 322, 331, 332, 341, 342, 351, 352 under the servers 31 to 35 of the real server group 3 of the load balance server 4 that is a load balancing system. It shows that the failure monitoring / recovery system 1 is incorporated. Processes such as disk formatting (step A4 in FIG. 2) and resynchronization of mirroring (step A6 in FIG. 2) that are performed when mirroring synchronization fraud is detected are subject to system load, particularly the load on the I / O side. These processes are very large and require several hours for these processes, which may reduce system throughput. As one method for alleviating such a problem, the software mirroring disk failure monitoring / recovery system 1 is operated in the load balance system. As a result, when mirroring recovery processing is performed on a disk device under one of the servers in the real server group 3, the load balance server 4 detects an increase in server load due to the recovery processing, and suppresses information to be assigned to that server. Therefore, the allocation to other servers is increased accordingly. When the server mirroring recovery process is complete, release the information suppression.

  Integrated environment for operating the status monitoring and recovery processing of the disk devices 311, 312, 321, 322, 331, 332, 341, 342, 351, 352 under the servers 31 to 35 of the real server group 3 remotely from the load balance server 4 In this case, it is possible to perform processing more efficiently by coordinating with the load balance system, such as performing balance control of the load balance server 4 before starting mirroring recovery processing. .

(Third embodiment of the invention)
Next, a third embodiment of the present invention will be described with reference to the drawings.

  FIG. 4 shows a flowchart of the operation of the software mirroring fault monitoring / restoring system according to the third embodiment of the present invention.

  FIG. 4 shows error information of a failed disk at the timing when a mirroring synchronization fraud is detected (step A2 in FIG. 4) and the failed disk causing the synchronization fraud is identified (step A3 in FIG. 4). Is added (step A12 in FIG. 4), and a step for arranging replacement disks (A13 in FIG. 4) is added. If an error occurs during disk formatting or resynchronization, automatic recovery fails and a notification is given, and the disk is replaced. However, in order to perform disk replacement immediately after the automatic recovery failure, preparation at an early stage is necessary, and steps for the preparation (steps A12 and A13 in FIG. 4) are added.

  Thereby, automatic recovery of disk mirroring and replacement parts arrangement are performed at the same time, and when the automatic recovery fails, replacement work can be started immediately.

  Further, the software mirroring disk failure monitoring / recovery method program according to the embodiment of the present invention described above can cause a computer or microprocessor to execute the software mirroring disk failure monitoring / recovery method.

It is a figure which shows the structure of the software mirroring failure monitoring and recovery system of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the software mirroring failure monitoring and recovery system of the 1st Embodiment of this invention. It is a figure which shows the structure of the load balance system of the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the software mirroring failure monitoring and recovery system of the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Software mirroring disk failure monitoring and recovery system 2 Disk unit of software mirroring configuration 3 Real server group 4 Load balance server 11 Mirroring state monitoring unit 12 Disk error checking unit 13 Disk format processing unit 14 Mirroring resynchronization processing unit 15 Processing result reporting unit 21, 22, 311, 312, 321, 322, 331, 332, 341, 342, 351, 352 Disk unit 31, 32, 33, 34, 35 Server

Claims (6)

A mirroring status monitoring unit that issues a command to collect the software mirroring status of the disk device and monitors whether there is a data area part where synchronization fraud has occurred;
If a synchronization error has occurred, check the log in which the hardware error is stored, and check whether the disk media error of the first disk device or the second disk device in the software mirroring configuration has been recorded. Or by referring to the internal statistical information of the disk to confirm that the error count has increased, and to identify the failed disk that caused the synchronization fraud,
Stop the software mirroring function, issue the media format command to the disk where the disk media error occurred, perform the media format, mask and replace the data block where the disk media error occurred, and the entire data area of the disk A disk format processor to confirm that it can be used normally,
A mirroring resynchronization processor that issues software mirroring resynchronization commands to restore software mirroring;
A software mirroring disk failure monitoring / recovery system comprising a processing result reporting unit for reporting each detected content and a processing execution result.   2. The software mirroring according to claim 1, wherein the disk error check unit further notifies error information of the failed disk and arranges a replacement disk at a timing when the failed disk that caused the synchronization error is identified. Disk failure monitoring and recovery system. The software mirroring disk failure monitoring and recovery system according to claim 1 is incorporated in a disk device under a server of a real server group of a load balance server,
When software mirroring recovery processing is performed on a disk device under any of the servers in the real server group, the load balance server detects an increase in server load due to the recovery processing, and suppresses information assigned to the server, A load balance system that increases the allocation to other servers and releases the information suppression when the server software mirroring recovery process is completed. Issuing a command for collecting the software mirroring status of the disk device by the mirroring status monitoring unit and monitoring whether there is a data area portion in which the synchronization fraud has occurred;
If a synchronization error has occurred by the disk error check unit, the log storing the hardware error is confirmed, and the disk media error of the first disk device or the second disk device in the software mirroring configuration is Confirming that the error count is increasing by checking if it is not recorded or by referring to the internal statistics of the disk, and identifying the failed disk that caused the synchronization fraud;
The disk format processing unit stops the software mirroring function, issues a media format command to the disk where the disk media error has occurred, performs masking and replacement processing of the disk media error occurrence data block by performing media formatting, Confirming that all data areas of the disk are available normally;
Issuing a software mirroring resynchronization command by the mirroring resynchronization processing unit to restore software mirroring;
A software mirroring disk failure monitoring / recovering method comprising a step of reporting each detected content and a process execution result by a process result reporting unit.   The step of identifying the failed disk that caused the synchronization error by the disk error check unit notifies the error information of the failed disk at the timing when the failed disk that caused the synchronization error is identified, and is used for replacement. 5. The software mirroring disk failure monitoring / recovering method according to claim 4, further comprising the step of arranging a disk.   A program for causing a computer or a microprocessor to execute the software mirroring disk failure monitoring / recovering method according to claim 4 or 5.

Images