JP7242041B2 - Gap change detection device, gap change detection method, and gap change detection program - Google Patents

Description

The present invention belongs to the technical field of gap change detection apparatus, gap change detection method, and gap change detection program. More specifically, for example, the technical field of a gap change detection device and a gap change detection method for detecting changes in the length of gaps between ends of a plurality of bones that constitute human joints, and a program for the gap change detection device belongs to

In recent years, research on drugs for treating rheumatoid arthritis has progressed, and drugs that are more effective than conventional drugs have been developed and provided. These drugs are required to be properly used depending on the degree of progression of rheumatoid arthritis, coupled with their high drug prices. Therefore, it is extremely important to accurately recognize the progress of rheumatoid arthritis. At this time, one measure of the progress is the length of the gap in the joint to be diagnosed. For example, in general, the shorter the length, the more advanced rheumatoid arthritis may be diagnosed.

On the other hand, the diagnosis of the length of the gap is often performed using an X-ray image of the joint to be diagnosed. That is, the length of the gap is often finally diagnosed by a doctor who has viewed the X-ray image. Therefore, the X-ray image is required to be clear. As a prior art document corresponding to such a request, for example, Patent Document 1 below can be cited. In the technique described in Patent Document 1, a scintillator member composed of at least two or more types of constituent members is provided on the X-ray incident side, and the constituent members are periodically arranged in the X-ray incident/reflected direction. At least one of the members contains a phosphor that converts X-rays into visible light, and the scintillator member, the optical member, and the photodetector are arranged in this order from the X-ray incident side.

JP 2018-179795 A

However, on the other hand, even if a clear X-ray image can be taken using the technique described in Patent Document 1, for example, when a doctor visually observes it and diagnoses the length of the gap, the length of the gap cannot be determined. However, for rheumatoid arthritis, where changes due to the degree of progression are difficult to discern, (i) it is easy to be subjective, (ii) it is easy to overlook minute changes in the length of the gap, and (iii) special measures are required for accurate diagnosis. Conventionally, it has been pointed out that there are problems such as the need for training and (iv) poor reproducibility.

Therefore, the present invention has been made in view of the above problems, and one example of the problem is to detect changes in the length of the gap from an image of the joint to be diagnosed, for example, subjectively by a doctor. It is an object of the present invention to provide a gap change detection device, a gap change detection method, and a program for the gap change detection device that can accurately and quantitatively detect the gap change without intervening.

In order to solve the above problems, the invention according to claim 1 provides a gap change detection device for detecting a change in the length of a gap between a plurality of objects, wherein an image showing the plurality of objects and the gap is provided. and an acquisition means such as a finger position detection unit that acquires image information corresponding to the image information, based on the acquired image information, the edge of each of the objects in the image corresponding to the image information on the gap side is detected in the image A separation means such as a separation unit that separates in and a change detection unit that detects time-series position changes that are changes along the same time series of the positions of the separated ends using the phase-only correlation method. and change detection means such as a change detection unit for detecting a change in the length of the gap along the time series based on the time-series position changes detected for each of the ends. , provided.

In order to solve the above-mentioned problems, the invention according to claim 6 is a gap change detection device for detecting a change in the length of gaps between a plurality of objects, comprising acquisition means such as a finger position detection unit, A gap detection method performed in a gap change detection device including separation means such as a separation section, position change detection means such as a change detection section, and change detection means such as a change detection section, wherein the plurality of objects and an acquiring step of acquiring image information corresponding to the image in which the gap is shown by the acquiring means, and each of the objects in the image corresponding to the image information by the separating means based on the acquired image information A separation step of separating the gap-side end of the image in the image, and a time-series positional change that is a change along the same time series of the position of each of the separated ends by the positional change detection means a positional change detection step of detecting using a phase-only correlation method; and determining the length of the gap along the time series by the change detecting means based on the time-series positional changes detected for each of the ends. and a change detection step of detecting a change.

In order to solve the above-mentioned problems, the invention according to claim 7 provides a computer included in a gap change detection device for detecting a change in the length of a gap between a plurality of objects, wherein the plurality of objects and the gap are Acquisition means for acquiring image information corresponding to the displayed image, based on the acquired image information, separating ends of the objects on the gap side in the image corresponding to the image information in the image. Separating means, position change detecting means for detecting time-series positional changes, which are changes along the same time series, of the positions of the separated ends using a phase-only correlation method, and for each of the ends It functions as change detection means for detecting a change in the length of the gap along the time series based on the time-series positional changes detected respectively.

According to the invention described in any one of claims 1, 6, and 7, based on the image information corresponding to the image in which the plurality of objects and the gap are shown, the end portion of each object on the side of the gap is detected. is separated, and the time-series positional change for each of the relevant ends is detected using the phase-only correlation method, and the change in the length of the gap is detected based on the detection result, so the change in the length of the gap is detected , can be accurately and quantitatively detected from the image information, for example, without intervention of doctor's subjectivity or the like.

In order to solve the above-mentioned problems, the invention according to claim 2 is the gap change detection device according to claim 1, wherein the object is a bone forming a joint, and the gap is between the bones in the joint. The separation means includes conversion means such as a separation unit that performs morphological conversion processing on the acquired image information, and the separation means is based on the conversion result of the image information by the conversion means and detecting in the image a part of the body that includes the joint, and detecting and separating the end based on the detected part.

According to the invention of claim 2, in addition to the effects of the invention of claim 1, each object is a bone that constitutes a joint, the gap is a gap between bones in the joint, and image information is Based on the transformation result of the morphological transformation processing, body parts including joints are detected, and bone ends are detected and separated based on the detected parts, so that the ends can be detected more accurately. can be separated.

In order to solve the above problems, the invention according to claim 3 is the gap change detection device according to claim 2, wherein the separation means is a binary discriminator such as a binary discriminator using a machine learning method. It further comprises an identification means such as a finger joint detection unit in which a plurality of composite identification means such as a composite identification device in which means are linearly connected are connected in series, and the separation means separates the detected part from the identification means to detect the joints in the image, and to detect and separate the ends based on the detected joints.

According to the third aspect of the invention, in addition to the effects of the second aspect of the invention, a joint is detected by subjecting the detected body part to binary identification processing, and based on the detected joint, Since the ends of the bone are detected and separated, the ends can be detected and separated more accurately.

In order to solve the above problems, the invention according to claim 4 is the gap change detection device according to any one of claims 1 to 3, wherein the position change detection means is arranged in the time series. positional deviation detection means such as a change detection unit for detecting a positional deviation of the object between the corresponding image information using a position-limited correlation method for the entire image corresponding to the image information; is configured to detect the chronological position change while removing the detected positional deviation.

According to the invention of claim 4, in addition to the effect of the invention of any one of claims 1 to 3, positional deviation of an object between pieces of image information corresponding to time series can be detected in the entire image. is detected using the position-only correlation method for , and the time-series position change is detected while removing the detected position shift, so that the time-series position change can be detected more accurately.

In order to solve the above-mentioned problems, the invention according to claim 5 is the gap change detection device according to any one of claims 1 to 4, wherein the separating means is adapted to the acquired image information. Conversion means such as a separation unit for performing morphological conversion processing is provided, and the separation means is configured to separate the ends in the image based on the result of conversion of the image information by the conversion means.

According to the fifth aspect of the invention, in addition to the effect of the invention according to any one of the first to fourth aspects, the edge of the object is determined based on the transformation result of the morphological transformation processing of the acquired image information. Since the parts are each separated in the image, the edges can be separated more accurately.

According to the present invention, the edge of each object on the side of the gap is separated based on image information corresponding to an image in which a plurality of objects and gaps are shown, and the time-series position change of each of the edges is analyzed using phase-only correlation. method, and the change in gap length is detected based on the detection results.

Therefore, the change in the length of the gap between a plurality of objects can be accurately and quantitatively detected from the image information corresponding to the image showing the plurality of objects and the gap, for example, without intervening the subjectivity of the doctor. can do.

1 is a block diagram showing a schematic configuration of a clearance inspection system including a clearance change detection device of an embodiment; FIG. 1 is a block diagram showing a schematic configuration of a gap change detection device according to an embodiment; FIG. FIG. 4A is a diagram illustrating finger position detection processing of the embodiment, FIG. 4A is a diagram illustrating an image to be subjected to the finger position detection processing, and FIG. 4B is a diagram illustrating the first processing of the finger position detection processing; , (c) is a diagram showing the second processing of the finger position detection processing, (d) is a diagram showing the third processing of the finger position detection processing, and (e) is the finger position detection processing (f) is a diagram showing a fifth process of the finger position detection process; (g) is a diagram showing a sixth process of the finger position detection process; ) shows the seventh process of the finger position detection process. FIG. 4A is a block diagram showing a detailed configuration of a finger joint detection unit of the embodiment, and FIG. 4B shows the first processing of the finger joint detection processing. It is a figure, (c) is a figure which shows the 2nd process of the said finger joint detection process, (d) is a figure which shows the detection result by the said finger joint detection process. FIG. 4A is a diagram showing a position correction process of the embodiment, FIG. 4A is a diagram showing a first process of the position correction process, and FIG. and (c) are diagrams showing the second processing of the position correction processing. It is a figure explaining the phase only correlation method of embodiment. It is a figure (I) which shows the groove|channel detection process in the joint separation process of embodiment. FIG. 2(II) shows a groove detection process in the joint separation process of the embodiment, (a) shows a first example of the groove detection process, and (b) shows a second example of the groove detection process. FIG. 3C is a diagram showing a third example of the groove detection process, (d) is a diagram showing a fourth example of the groove detection process, and (e) is a diagram showing the groove detection process. FIG. 4 illustrates the results; FIG. 1(I) shows separation processing in the joint separation processing of the embodiment, FIG. 1(a) is a diagram illustrating the arrangement of pixels in the joint separation processing, and FIG. 1(b) is a diagram illustrating the pixel value of each pixel; is. FIG. 11 is a diagram (I) illustrating morphological transformation processing in the separation processing of the embodiment, (a) is a diagram illustrating the first processing of the morphological transformation processing, and (b) is the second processing of the morphological transformation processing; It is a figure which illustrates. FIG. 2B is a diagram (II) illustrating the morphological transformation processing in the separation processing of the embodiment, (a) is a diagram illustrating the third processing of the morphological transformation processing, and (b) is the fourth processing of the morphological transformation processing; It is a figure which illustrates. FIG. 3(III) illustrates morphological transformation processing in the separation processing of the embodiment, FIG. FIG. 4 is a diagram illustrating a joint separation line; FIG. 2(II) shows separation processing in the joint separation processing of the embodiment, (a) is a diagram illustrating an X-ray image on which a joint separation line is superimposed, and (b) is a diagram using the joint separation line. FIG. 10 is a diagram illustrating joint separation, and FIG. 10C is a diagram illustrating a result of the joint separation processing; It is a figure which illustrates the clearance change detection process of embodiment. FIG. 4A is a diagram illustrating the result of the gap change detection process of the embodiment, FIG. 4A is a diagram illustrating the result of the joint separation process of the embodiment, and FIG. be. FIG. 7 is a diagram showing a display example of a result of gap change detection processing according to the embodiment; It is a figure which shows the result of an Example, (a) is a figure which shows the 1st example of the said result, (b) is a figure which shows the 2nd example of the said result.

Next, a mode for carrying out the present invention will be described with reference to FIGS. 1 to 16. FIG. In the embodiments described below, the length of the gap between the ends of two or more bones forming the joint of the hand is obtained from an image of a human hand taken by an X-ray device, a CT (Computed Tomography) device, or the like. This is an embodiment in which the present invention is applied to a gap change detection device that detects a change in height. The gap change detection device compares/analyzes a plurality of images obtained by imaging the same joint at different times, thereby automatically and quantifying changes in the length of the gap. To detect. In the following description, an image captured by the X-ray device, CT device, or the like is simply referred to as an "image."

FIG. 1 is a block diagram showing a schematic configuration of a gap inspection system including the gap change detection device of the embodiment, FIG. 2 is a block diagram showing a schematic configuration of the gap change detection device, and FIG. 4A and 4B are diagrams showing finger position detection processing of the embodiment, and FIG. Further, FIG. 5 is a diagram for explaining the phase-only correlation method of the embodiment, FIG. 6 is a diagram showing position correction processing of the embodiment, and FIGS. 7 and 8 are groove detection processing in the joint separation processing of the embodiment. , respectively, and FIG. 9 is a diagram (I) showing separation processing in the joint separation processing of the embodiment. 10 to 12 are diagrams respectively illustrating morphological transformation processing in the joint separation processing, and FIG. 13 is a diagram (II) showing separation processing in the joint separation processing of the embodiment. 14 is a diagram illustrating the gap change detection process of the embodiment, FIG. 15 is a diagram illustrating the result of the gap change detection process, and FIG. 16 is a diagram illustrating a display example of the result.

(1) Overall configuration and operation of gap detection system
First, the overall configuration and operation of the gap detection system of the embodiment will be described with reference to FIG. As shown in FIG. 1, a gap detection system SS including the gap change detection device S of the embodiment is configured to include an imaging device SX in addition to the gap change detection device S. As shown in FIG.

In this configuration, the imaging device SX is composed of, for example, an X-ray device or a CT device, and images a joint having a gap to be detected by the gap change detection device S or a human hand including the joint. and outputs the image data Dx to the gap change detection device S. Accordingly, the gap change detection device S automatically and numerically detects the change in the length of the gap based on the image data Dx, and outputs it as detection result data Dout. Here, a so-called Phase-Only Correlation (POC) method is used to detect the change in the length of the gap in the gap change detection device S. This phase-only correlation method will be described in detail later. A detection result corresponding to the detection result data Dout is displayed, for example, on a display (not shown) as described later. In the following description, the phase-only correlation method is simply referred to as "POC method", and the processing using the phase-only correlation method is simply referred to as "POC processing".

Next, the configuration of the gap change detection device S will be specifically described with reference to FIG. As shown in FIG. 2, the gap change detection device S of the embodiment includes a finger position detection unit 1, a finger joint detection unit 2, a position correction unit 3, a separation unit 4, and a change detection unit 5. It is At this time, the finger position detection unit 1, the finger joint detection unit 2, the position correction unit 3, the separation unit 4, and the change detection unit 5 constitute the gap change detection device S, for example, a CPU (Central Processing Unit), a ROM (Read (Only Memory) and RAM (Random Access Memory), etc., or a program corresponding to each process described later is read and executed by the CPU, etc., so that software It may be realized in Further, the finger position detection unit 1 corresponds to an example of the "acquisition means" of the present invention, the finger joint detection unit 2 corresponds to an example of the "identification means" of the present invention, and the separation unit 4 corresponds to the "separation means" of the present invention. The change detection unit 5 corresponds to an example of the "position change detection means", an example of the "change detection means", and an example of the "position deviation detection means" of the present invention. correspond respectively.

In the above configuration, when the finger position detection unit 1 first acquires the image data Dx output from the imaging device SX, the finger in the image corresponding to the image data Dx (that is, the detection target of the change in the gap) is detected. The finger position detection processing of the embodiment for detecting the position of the finger (including the joint that is formed) is executed, and the result is output to the finger joint detection unit 2 as finger position data Dfng. At this time, the finger position detection unit 1 outputs finger position data Dfng for each of the plurality of image data Dx corresponding to the plurality of images obtained by imaging the same joint separated in time. Note that the finger position detection processing will be described later in detail with reference to FIG.

As a result, the finger joint detection unit 2 detects the joints included in the fingers whose positions are indicated by the finger position data Dfng, based on the finger position data Dfng from the finger position detection unit 1. The finger joint detection processing of the embodiment is executed, and the result is output to the position corrector 3 as joint data Djnt. The finger joint detection process will be described later in detail with reference to FIG.

Next, based on the joint data Djnt from the finger joint detection unit 2, the position correction unit 3 moves the joints corresponding to the joint data Djnt apart in time (i.e., acquires them in time series). If there is a positional deviation due to the above-mentioned POC method, the positional correction processing of the embodiment is executed to mutually correct the positional deviation using the POC method, and the result is sent to the separation unit 4 as corrected joint data Dm. Output each. The above position correction processing will be explained in detail later with reference to FIGS. 5 and 6. FIG.

Then, based on the corrected joint data Dm from the position correcting unit 3, the separation unit 4 extracts, from the image corresponding to the corrected joint data Dm, end portions of, for example, two bones forming the joint shown in the image. , and outputs the result to the change detection unit 5 as joint separation data Dsep. By the function of the separation unit 4, two images each including only each end portion constituting the joint are separated from one image showing the joint. The joint separation processing will be described later in detail with reference to FIGS. 7 to 13. FIG.

Finally, based on the joint separation data Dsep from the separation unit 4, the change detection unit 5 detects a change in the distance between the ends (i.e., the gap ) is detected using the POC method, and the result is output as the detection result data Dout to, for example, the display (not shown). The clearance change detection process will be described in detail later with reference to FIGS. 14 to 16. FIG.

(2) Finger position detection processing
Next, the finger position detection processing executed by the finger position detection section 1 will be described with reference to FIGS. 2 and 3. FIG.

As illustrated in FIG. 3, when image data Dx corresponding to an image G (see FIG. 3A) obtained by imaging the right hand with an X-ray device, for example, is acquired, the finger position detection unit 1 first detects As the first process of the finger position detection process, for example, so-called Otsu's binarization process is applied to the image data Dx to generate a binarized image illustrated in FIG. 3(b). Next, as the second processing of the finger position detection processing, the finger position detection unit 1 performs expansion processing and contraction processing in morphological transformation on the binarized image to generate a binarized image from which noise has been removed. (See FIG. 3(c)). After that, as the third process of the finger position detection process, the finger position detection unit 1 performs the same edge detection process and contour approximation process as in the conventional art on the binarized image from which the noise has been removed, and detects each finger and wrist. etc. is detected (see FIG. 3(d)). Next, as the fourth processing of the finger position detection processing, the finger position detection unit 1 performs edge detection processing (that is, partial Processing to detect the maximum and minimum values) is performed, and the tips of each finger (indicated by white circles in FIG. 3(e)) and the portions between the fingers (indicated by white squares in FIG. 3(e)) detect each. Next, as the fifth process of the finger position detection process, the finger position detection unit 1 cuts out an image of each finger based on the positions between the tips and the fingers illustrated in FIG. Then, set the center line of each finger (see FIG. 3(f)). As the sixth process of the finger position detection process, the finger position detection unit 1 performs the process of detecting the opening angle of each finger based on the set center line (see FIG. 3G), and further As the seventh process of the finger position detection process, only the image of the finger portion is finally cut out for each finger as shown in FIG. 3(h). After that, the finger position detection unit 1 outputs each clipped finger image (see FIG. 3(h)) to the finger joint detection unit 2 as the finger position data Dfng. In an experiment conducted by the inventors of the present invention, the time required for the finger position detection process using the image G of 250 pixels×300 pixels was about 0.05 seconds.

(3) Finger joint detection process
Next, the finger joint detection process executed by the finger joint detection unit 2 will be described with reference to FIGS. 2 and 4. FIG. In the finger joint detection processing of the embodiment, an image including the position of the finger indicated by the finger position data Dfng is subjected to feature amount detection processing using a conventional Haar-like feature amount. This is finger joint detection processing for detecting a joint from among. At this time, the feature amount detection processing using the Haar-like feature amount is defined as a feature amount of a local contrast difference in a portion of the image obtained by cutting out a part of the image, and the feature amount are combined for each part to capture the characteristics of the image as a whole.

That is, in the finger joint detection unit 2 of the embodiment, as shown in FIG. 4A, n composite discriminators MI1 to MIn are connected in series (n is the number of connections set in advance). It is configured. Each of the composite classifiers MI1 to MIn includes T binary classifiers (T is a preset optimum number of bonds) are connected linearly. In this configuration, the binary discriminator corresponds to an example of the "binary discriminating means" of the present invention, and each of the composite discriminators MI1 to MIn corresponds to an example of the "composite discriminating means" of the present invention. do.

At this time, each of the binary discriminators, for example, corresponds to the image of the joint formed by the ends of the bones, "white" (corresponding to the end of one of the bones forming the joint) - "black ” (corresponding to the above gap)−“white” (corresponding to the end of the other bone forming the joint) is used to detect the Haar-like feature quantity h represented by the following equation (1). As a result, each of the composite classifiers MI1 to MIn detects the Haar-like feature quantity H shown in the following equation (2).

More specifically, as shown in FIG. 4(a), the decoding discriminator MI1 of the finger joint detection unit 2 to which the finger position data Dfng is input, detects the finger position data by the binary feature quantity detection process. Each portion recognized as a joint is detected from the image corresponding to Dfng (see FIG. 4(b)), and the result is output to the composite discriminator MI2 at the next stage. Thereafter, the composite classifiers MI2 to MIn successively perform binary feature amount detection processing similar to that of the composite classifier MI1. As a result, each joint corresponding to the filter is detected from the original image G, as illustrated by white squares in FIGS. 4(c) and 4(d). The detection result of each joint is output to the position corrector 3 as the joint data Djnt. According to an experiment conducted by the inventors of the present invention, the time required for the finger joint detection process using an image G of 2,280 pixels×2,860 pixels (see FIG. 4(d)) was about 0.4 seconds. Met.

(4) Position correction processing
Next, the position correcting process executed by the position correcting section 3 will be described with reference to FIGS. 2, 5 and 6. FIG. In the position correction processing of the embodiment, based on the joint data Djnt from the finger joint detection unit 2, the positional deviations of the joints corresponding to the joint data Djnt are mutually corrected using the POC method. position correction processing. More specifically, the position correction processing of the embodiment includes first processing for removing high-frequency components in the image G corresponding to the joint data Djnt using a so-called median filter using a local median value, and a second process of correcting misregistration using the POC method. At this time, the high-frequency component is a high-frequency component corresponding to the veins of the bones forming the joint corresponding to the joint data Djnt, and is unnecessary for detecting the change in the length of the gap in the embodiment.

More specifically, as the first process of the position correction process of the embodiment, the position correction unit 3 applies the Median Filter to the image G (an example is shown on the left side of FIG. 5A) corresponding to the joint data Djnt. Applying processing is performed to remove high frequency components corresponding to the bone veins as illustrated on the right side of FIG. 5(b).

Next, as second processing of the position correction processing of the embodiment, the position correction unit 3 corrects the positional deviation caused by the acquisition along the time series to the joint data Djnt as illustrated in FIG. 5B, for example. is included in the image G corresponding to , the positional deviation is mutually corrected using the POC method for the entire image from which the high frequency component has been removed.

Here, the POC method of the embodiment is used not only in the position correction process, but also in the gap change detection process of the embodiment by the change detection unit 5, which will be described later. Therefore, the POC method will be generally described below with reference to FIG.

That is, for example, the image data Dx corresponding to the image G captured most recently is represented as image data f(x, y), and the image data Dx corresponding to the image G captured in the past is represented as image data g(x, y), in the POC process, as shown in FIG. (k) Transform into a spatial signal. At this time, the result of the conversion is represented by the following formula (3).

In the above POC process, next, as shown in FIG. 5, the function F(k _x , _ky ) and the function G(k _x , _ky ) in equation (3) are normalized by dividing by their amplitudes, respectively, Further, by conjugated one of them, a so-called element product (Hadamard Product or Element-Wise Product) is obtained. This element product is represented by the following formula (4).

In this equation (4), “θ _F (k _x , _ky )” is the phase angle of the complex function F (k _x , _ky ), and “θ _G (k _x , _ky )” is the complex function G is the phase angle of (k _x , _ky ). Finally, in the POC process, as shown in FIG. 5, a so-called Inverse Discrete Fourier Transform (hereinafter referred to as "IDFT") process is performed on the element product obtained by the above equation (4). and output as the result of the POC processing, which is given by the following equation (5).

Here, the relationship between the image corresponding to the image data f(x, y) and the image corresponding to the image data g(x, y) is defined as the displacement amount d (horizontal displacement). It does not matter whether it is a positional shift in the vertical direction, or in the oblique direction), the following equation (6) is obtained.

Then, the result of performing the DFT processing (see FIG. 5) on both sides of the equation (6) is the following equation (7).

Therefore, when formula (7) is normalized in the same way as formula (4) above, and the element product thereof is obtained (see FIG. 5), the result corresponds to formula (4) above. , as shown in the following equation (8).

Then, the result of performing the IDFT processing (see FIG. 5) on both sides of the above equation (8) is the following equation (9).

As can be seen from the equation (9), when there is a positional deviation d between the image corresponding to the image data f(x, y) and the image corresponding to the image data g(x, y) ( (see FIG. 5B), a peak appears as a delta (.delta.) function represented by the equation (9) corresponding to the positional deviation d.

Then, as the second process of the position correction process, the position correction unit 3 mutually corrects the positional deviations d corresponding to the positions of the peaks of the delta functions as illustrated in FIG. 5(c). After that, the position correction unit 3 outputs the result of the correction to the separation unit 4 as corrected joint data Dm.

(5) About the joint separation process
Next, the joint separation processing executed by the separation section 4 will be described with reference to FIGS. 2 and 7 to 13. FIG. In the joint separation processing of the embodiment, based on each corrected joint data Dm (that is, each corrected joint data Dm corresponding to each image G obtained along the time series) from the position correction unit 3, each corrected Groove detection processing for detecting the gap in each joint corresponding to the joint data Dm as a groove (Gully), and morphological conversion processing for the image of the detected gap are performed so that the gap is opposite to each other across the gap. and a separation process for separating each epiphysis.

First, groove detection processing in the joint separation processing will be described with reference to FIGS. 7 and 8. FIG. As the groove detection process, the separation unit 4 first sets a straight line in the vertical direction in FIG. The process of applying the filter A1 and the filter B1 to each pixel value of the image G (for example, the luminance value of each pixel constituting the image G) while moving the filter B1 along the straight line is performed by moving the position of the straight line. This is repeated while moving laterally in FIG.

At this time, the filter A1 has a size corresponding to two pixels vertically by three pixels horizontally, and the filter coefficients corresponding to each pixel are "1", "2 , “1”, “-1”, “-2” and “-1”. On the other hand, the filter B1 has a size corresponding to two pixels vertically by three pixels horizontally, and the filter coefficients corresponding to each pixel are "-1", "- 2", "-1", "1", "2" and "1". The position of the filter A1 and the position of the filter B1 are shifted in the movement direction by one horizontal row of pixels as illustrated in FIG. Therefore, the filters A1 and B1 are applied to each pixel value of the image G while maintaining the positional relationship shown in FIG. The filter coefficient is multiplied), and the result is a waveform diagram indicated by symbols A1 and B1 on the right side of FIG. ). After that, the separation unit 4 compares the change in the waveform diagram indicated by symbol A1 with the change in the waveform indicated by symbol B1, and selects the smaller one at the same position as a function F ₁ (m, n) (m and n indicates the position of the pixel.The same applies hereinafter.) is calculated. As a result, a groove image P1 corresponding to the function F ₁ (m, n) is obtained. As a result, the separation unit 4 records image data corresponding to the groove image P1 in a ROM (not shown) or the like.

Next, as the groove detection process, the separating unit 4 moves the filter A2 and the filter B2 illustrated in FIG. is repeated while moving the position of the straight line in the horizontal direction in FIG.

At this time, the filter A2 has a size corresponding to four pixels vertically by three pixels horizontally, and the filter coefficients corresponding to each pixel are "1", "2 , "1", "1", "2", "1", "-1", "-2", "-1", "-1", "-2" and "-1". there is On the other hand, the filter B2 has a size corresponding to four pixels vertically by three pixels horizontally, and the filter coefficients corresponding to each pixel are "-1", "- 2”, “-1”, “-1”, “-2”, “-1”, “1”, “2”, “1”, “1”, “2” and “1” . The position of the filter A2 and the position of the filter B2 are shifted in the moving direction by two horizontal rows of pixels as illustrated in FIG. Therefore, when the filter A2 and the filter B2 are applied to each pixel value of the image G while maintaining the mutual positional relationship shown in FIG. 7 and moving in the vertical direction in FIG. On the right, the waveform changes as indicated by symbols A2 and B2. After that, the separation unit 4 compares the change in the waveform diagram indicated by symbol A2 with the change in the waveform indicated by symbol B2, and calculates a function F ₂ (m, n) that selects the smaller one at the same position. As a result, a groove image P2 corresponding to the function F ₂ (m,n) is obtained. As a result, the separation unit 4 records image data corresponding to the groove image P2 in a ROM (not shown) or the like. At this time, the groove image P2 becomes a groove image in which the groove image P1 is more blurred (i.e., a groove image from which unnecessary noise has been removed; the same shall apply hereinafter), and the groove corresponding to the gap becomes clear on the image. It's getting

Further, as the groove detection process, the separating unit 4 performs a process of applying the filter A3 and the filter B3 to each pixel value of the image G while moving the filter A3 and the filter B3 illustrated in FIG. 7 along the straight line. , while moving the position of the straight line in the horizontal direction in FIG.

At this time, the filter A3 has a size corresponding to 6 pixels vertically by 3 pixels horizontally, and the filter coefficients corresponding to each pixel are, as shown in FIG. ', '1', '1', '2', '1', '1', '2', '1', '-1', '-2', '-1', '-1', "-2", "-1", "-1", "-2" and "-1". On the other hand, the filter B3 has a size corresponding to 6 pixels vertically by 3 pixels horizontally, and the filter coefficient corresponding to each pixel is "-1", "- 2", "-1", "-1", "-2", "-1", "-1", "-2", "-1", "1", "2", "1", "1", "2", "1", "1", "2" and "1". The position of the filter A3 and the position of the filter B3 are shifted in the movement direction by three horizontal rows of pixels as illustrated in FIG. Therefore, when the filter A3 and the filter B3 are applied to each pixel value of the image G while maintaining the positional relationship shown in FIG. 7 and moving them in the vertical direction in FIG. As with A1 and said filter B1 or said filter A2 and said filter B2, a groove image P3 corresponding to the corresponding function F ₃ (m,n) is obtained. As a result, the separation unit 4 records image data corresponding to the groove image P3 in a ROM (not shown) or the like. At this time, the groove image P3 becomes a groove image in which the groove image P2 is further blurred, and the groove corresponding to the gap is becoming clearer on the image.

After that, the separation unit 4 performs the same groove detection processing as in the case of using the filter A1 and the filter B1, etc., for example, by applying a filter having a size and a filter coefficient corresponding to 10 pixels vertically by 3 pixels horizontally as illustrated in FIG. By repeating the process up to the process using A5 and filter B5 and superimposing the finally obtained groove images P1 to P5, a groove image Pall in which the groove portions corresponding to the gaps clearly appear is generated. 8, groove image P1 (see FIGS. 8A and 7) and groove image P5 (see FIG. (b) and FIG. 7), a groove image P7 (see FIG. 8(c). Each filter has a size corresponding to 14 vertical pixels×3 horizontal pixels), and a groove image P9 (see FIG. 8(d). Each filter has a size corresponding to 18 vertical pixels.times.3 horizontal pixels.) is superimposed to obtain a groove image Pall (see FIG. 8(e)) in which the grooves are clearly represented. In the groove image Pall exemplified in FIG. 8(e), the epiphysis EG constituting the joint and the correspondingly formed gap (groove) GP are clearly represented.

Next, separation processing in the joint separation processing will be described with reference to FIGS. 9 to 13. FIG. As the separation process, the separation unit 4 first performs the groove image Pall indicating the grooves (gap) detected by the groove detection process as shown in FIG. Set the image PX of Here, it is assumed that the pixel values of the set pixels PX are the values illustrated in FIG. 9B, for example. As a result, the separation unit 4 performs the first separation process using the morphological transformation process. First, a frame W (Fig. 10 10A) is set in the first column from the left, and the maximum value of the pixel values of the pixels PX to be included in the frame W ("2" in the example shown in FIG. 10A) ) becomes the pixel value at the center of the frame W, the dilation processing in the morphology conversion processing is executed (indicated by the dashed arrow in FIG. 10A). Next, the separation unit 4 shifts the frame W downward by one pixel in FIG. Perform dilation processing so that the maximum value (“6” in the example shown in FIG. 10A) becomes the pixel value in the center of the frame W (indicated by the dashed-dotted line arrow in FIG. 10A) . Further, the separation unit 4 further shifts the frame W downward by one pixel in FIG. A value (“6” in the example shown in FIG. 10(a)) is converted to a pixel value in the center of the frame W (indicated by a dotted arrow in FIG. 10(a)). After that, the separation unit 4 performs the same dilation processing as described above while sequentially shifting the frame W downward by one pixel for all the pixels PX in the first row from the left. As a result, the pixel values of the pixels PX in the first column from the left are illustrated in italics on the right side of FIG. 10(a).

Next, as the second process of the separation process using the morphological transformation process, the separation unit 4 converts the pixel values of the pixels PX in the first column from the left after the dilation process to It is added to the pixel value of the corresponding pixel PX in the second column. As a result, the pixel values of the pixels PX in the second column from the left are the pixel values illustrated in FIG. 10(b). As a result, the pixel values in the first column from the left after expansion processing are reflected (diffused) in the pixel values in the second column from the left.

Next, the separation unit 4 performs the third process of the separation process using the morphological transformation process. The frame W ( 11(a)) is set again in the second column from the left, and the maximum value of the pixel values of the pixels PX to be included in the frame W (in the case illustrated in FIG. 11(a) ("7") is converted to the pixel value at the center of the frame W (indicated by the dashed arrow in FIG. 11(a)). Next, the separation unit 4 shifts the frame W downward by one pixel in FIG. Perform dilation processing so that the maximum value (“12” in the example shown in FIG. 11A) becomes the pixel value at the center of the frame W (indicated by the dashed-dotted line arrow in FIG. 11A) . Further, the separation unit 4 further shifts the frame W downward by one pixel in FIG. A value (“12” in the example shown in FIG. 11(a)) is converted to a pixel value in the center of the frame W (indicated by a dotted arrow in FIG. 11(a)). Thereafter, the separation unit 4 performs the same dilation processing as described above on all the pixels PX in the second column from the left while sequentially shifting the frame W by one pixel downward. The pixel value of each pixel PX of is illustrated in italics on the right side of FIG. 11(a). In the case of FIG. 11A, the pixel value of the pixel PX in the first column from the left that has undergone the dilation process is the pixel value before the dilation process (FIG. 9B). 10(a) left)).

Next, as the fourth processing of the separation processing using the morphological transformation processing, the separation unit 4 converts the pixel values of the pixels PX in the second column from the left after the dilation processing to the left to the pixel value of the corresponding pixel PX in the third column. As a result, the pixel values of the pixels PX in the third column from the left become the pixel values illustrated on the right side of FIG. 11(b). As a result, the pixel values in the second column from the left after dilation processing are reflected (diffused) in the pixel values in the third column from the left. After the state illustrated in FIG. 11B, the pixel value of the pixel PX in the second column from the left is illustrated in FIG. Note that the pixel values of the straight line to which the dilation processing is applied (see the right side of FIG. 10(b) or the left side of FIG. 11(a)) are restored as shown in FIG.

Thereafter, the separation unit 4 repeats the expansion processing and the addition processing to the corresponding pixel value on the right for each column. As a result, the pixel value of each pixel PX finally becomes the pixel value illustrated in FIG. 12(b). After that, the separation unit 4 selects the pixel PXe, which has the highest pixel value among the pixels PX in the first column from the right, as a starting point, and from right to left, extracts the largest pixel value among the vertical three pixels in the column adjacent to the left. By tracing the pixels PX (see hatching in FIG. 12B) (see black arrows in FIG. 12B), a joint separation line SEP is set as a result of the separation processing.

When the joint separation line SEP is set for the image G illustrated in FIG. As illustrated in FIG. 13B, the image G is divided along the joint separation line SEP. Specifically, an image GU containing only one epiphysis and an image GL containing only the other epiphysis are generated (see FIG. 13(c)). After that, the separation unit 4 outputs joint separation data Dsep corresponding to the generated image GU and image GL to the change detection unit 5 as a result of the joint separation processing of the embodiment.

(6) About the gap change detection process
Finally, the gap change detection process executed by the change detection unit 5 will be described with reference to FIGS. 14 to 16. FIG. The gap change detection process of the embodiment is based on the joint separation data Dsep from the separation unit 4 (that is, each joint separation data Dsep corresponding to each image G acquired along the time series). This is gap change detection processing for detecting, using the POC method, changes in the distances between the ends (that is, changes in the length of the gaps) shown in the image corresponding to the data Dsep.

That is, as exemplified in FIG. 14, an image GUn and an image GLn separated (divided) for the joint from the recently captured joint image Gn are obtained as the joint separation data Dsep, while previously captured images When the image GUo and the image GLo obtained by separating (dividing) the joint from the image Go of the joint are obtained as the joint separation data Dsep (see the solid line in FIG. 14), the change detection unit 5 first detects the image GUn and the image GUo A phase-contrast image PIU (delta function image (see formula (9) above), which indicates the deviation between the position of the epiphysis shown in the image GUn and the position of the epiphysis shown in the image GUo, is obtained by the POC processing described below. , and so on)) (see dashed line in FIG. 14). In parallel with this, the change detection unit 5 performs POC processing on the image GLn and the image GLo to indicate the deviation between the position of the epiphysis shown in the image GLn and the position of the epiphysis shown in the image GLo. A phase difference image PIL (image of delta function) is generated (see dashed line in FIG. 14).

Then, based on the peak position in the phase contrast image PIU, the change detection unit 5 detects the deviation between the position of the epiphysis appearing in the image GUn and the position of the epiphysis appearing in the image GUo. 1.793 pixels in the direction and 1.243 pixels in the downward direction". Similarly, based on the peak position in the phase difference image PIL, the change detection unit 5 detects the deviation between the position of the epiphysis shown in the image GLn and the position of the epiphysis shown in the image GLo, for example, " 0.634 pixels to the right in the image and 0.020 pixels to the top. As a result, the change detection unit 5 detects a bone from one epiphysis BUo detected based on the image GUn and the image GUo illustrated in FIG. 15A and the images GLn and GLo illustrated in FIG. Based on the deviation to the end BUn (indicated by the lower right arrow in FIG. 15(b)) and the deviation from the other epiphysis BLo to the epiphysis BLn (indicated by the upper right arrow in FIG. 15(b)), As the final change in the length of the gap, the detection result data Dout is generated, for example, "decrease by 1.26 pixels". A detection result corresponding to the detection result data Dout is displayed on a display D (not shown) as shown in FIG. 16, for example. In this case, for example, an image G showing the positions of the joints detected by the finger joint detection process is displayed as "The Result of Detected", and the gap detected by the gap change detection process is displayed as "Line Chart". It is preferable that the length is indicated by imaging date, and the image G of each joint is indicated as "Knuckles" by imaging date.

Next, an example corresponding to the embodiment will be described with reference to FIG. 17 . In addition, FIG. 17 is a figure which shows the result of the said Example, respectively.

That is, as an example, the inventor of the present invention experimentally confirmed the linearity of the length of the gap detected by the gap change detection device S of the embodiment. That is, as shown in FIGS. 17(a) and 17(b) for different scales, the linearly approximated data and the gap length data detected by the gap change detection device S of the embodiment were compared. By the way, it was found that the data of the gap length detected by the gap change detector S had good linearity. From this, it can be understood that the accuracy of the length of the gap detected by the gap change detection device S of the embodiment is high.

As described above, according to the operation of the gap change detection device S according to the embodiment, each epiphysis is separated based on the image data Dx corresponding to the image G showing the plurality of epiphyses and their gaps. (see FIGS. 7 to 13), time-series positional changes for each epiphysis are detected using the POC method, and changes in the gap length are detected based on the detection results. The change in length can be accurately and quantitatively detected from the image data Dx, for example, without subjective intervention by a doctor.

Further, the finger is detected based on the transformation result of the morphological transformation processing for the image data Dx (see FIG. 3), and the epiphysis is detected and separated based on the position of the detected finger. Edges can be detected and separated.

Further, the detected finger is subjected to binary identification processing to detect the joint (see FIG. 4), and the epiphysis is detected and separated based on the detected joint, so the epiphysis can be detected more accurately. can be separated by

Furthermore, the positional deviation between the image data Dx corresponding to the time series is detected using the POC method for the entire image G, and the positional change in the time series is detected while removing the detected positional deviation ( 5), the change in position can be detected more accurately.

Moreover, since the epiphysis is separated from the image G based on the transformation result of the morphological transformation processing for the acquired image data Dx (see FIGS. 9 to 12), the epiphysis can be separated more accurately.

In the above-described embodiments and examples, the change in the length of the gap between the epiphyses at the joint was used as the object of detection. The present invention can be widely applied to detection.

Further, a program corresponding to each of the finger position detection process, the finger joint detection process, the position correction process, the joint separation process, and the gap change detection process is recorded in a recording medium such as an optical disk or a hard disk, Alternatively, it is possible to obtain the data via a network such as the Internet, read the data to a general-purpose microcomputer or the like, and execute the data, so that the microcomputer or the like functions as the gap change detection device S according to the embodiment. .

INDUSTRIAL APPLICABILITY As described above, the present invention can be used in the field of gap change detectors, and in particular, the present invention can be used to detect changes in the length of epiphyseal gaps in joints when diagnosing rheumatoid arthritis. If applied to the field of change detection devices, a particularly remarkable effect can be obtained.

1 finger position detection unit 2 finger joint detection unit 3 position correction unit 4 separation unit 5 change detection unit S gap change detection device SS gap detection system SX imaging device EG epiphysis GP gap (groove)
SEP Joint separation line Dx Image data Dout Detection result data Dfng Finger position data Djnt Joint data Dm Corrected joint data Dsep Joint separation data G, Gn, GUn, GLn, Go, GUo, GLo Image PIU, PIL Phase contrast image BUo, BUn, BLo, BLn Epiphysis

Claims (7)

In a gap change detection device for detecting a change in gap length between a plurality of objects,
Acquisition means for acquiring image information corresponding to an image showing the plurality of objects and the gap;
Separating means for separating, based on the obtained image information, an end portion of each of the objects on the gap side in the image corresponding to the image information;
Position change detection means for detecting time-series position changes, which are changes along the same time series, of the positions of the separated ends using a phase-only correlation method;
change detection means for detecting a change in the length of the gap along the time series based on the time-series position change detected for each of the ends;
A gap change detection device comprising: The gap change detection device according to claim 1,
the object is a bone forming a joint, the gap is a gap between the bones in the joint,
The separating means comprises transforming means for performing morphological transformation processing on the acquired image information,
The separating means detects a body part including the joint in the image based on the conversion result of the image information by the converting means, and detects and separates the end based on the detected part. A gap change detection device characterized by: In the gap change detection device according to claim 2,
The separation means further comprises identification means in which a plurality of composite identification means are connected in series, and a plurality of composite identification means are formed by linearly connecting binary identification means using a machine learning method,
The separating means performs binary identification processing by the identifying means on the detected parts to detect the joints in the image, and detects and separates the ends based on the detected joints. A gap change detection device characterized by: In the gap change detection device according to any one of claims 1 to 3,
The positional change detection means includes positional deviation detection means for detecting a positional deviation of the object between the image information corresponding to the time series by using a position only correlation method for the entire image corresponding to the image information. ,
The gap change detection device, wherein the position change detection means detects the time-series position change while removing the detected position shift. In the gap change detection device according to any one of claims 1 to 4,
The separating means comprises transforming means for performing morphological transformation processing on the acquired image information,
The gap change detecting device, wherein the separation means separates the ends in the image based on the result of conversion of the image information by the conversion means. A gap change detection apparatus for detecting a change in gap length between a plurality of objects, the gap change detection apparatus comprising an acquisition means, a separation means, a position change detection means, and a change detection means. A gap detection method comprising:
an acquisition step of acquiring image information corresponding to an image showing the plurality of objects and the gap by the acquisition means;
a separating step of separating, by the separating means, an end portion of each of the objects on the gap side in the image corresponding to the acquired image information, based on the acquired image information;
a position change detection step of detecting a time-series position change, which is a change along the same time series, of the positions of the separated ends using a phase-only correlation method by the position change detection means;
a change detection step of detecting a change in the length of the gap along the time series by the change detection means based on the time-series position change detected for each of the ends;
A gap change detection method, comprising: a computer included in a gap change detection device for detecting a change in gap length between a plurality of objects;
Acquisition means for acquiring image information corresponding to an image showing the plurality of objects and the gap;
Separating means for separating, based on the obtained image information, an end portion of each of the objects on the gap side in the image corresponding to the image information, in the image;
position change detection means for detecting time-series position changes, which are changes along the same time series, of the positions of the separated ends using a phase-only correlation method; and
change detection means for detecting a change in the length of the gap along the time series based on the time-series position change detected for each of the ends;
A gap change detection program characterized by functioning as

Images