[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2024128818A1 - System for quantifying lung disease of lung sample by using computer vision and machine learning, and method for quantifying lung disease by using system - Google Patents

System for quantifying lung disease of lung sample by using computer vision and machine learning, and method for quantifying lung disease by using system Download PDF

Info

Publication number
WO2024128818A1
WO2024128818A1 PCT/KR2023/020600 KR2023020600W WO2024128818A1 WO 2024128818 A1 WO2024128818 A1 WO 2024128818A1 KR 2023020600 W KR2023020600 W KR 2023020600W WO 2024128818 A1 WO2024128818 A1 WO 2024128818A1
Authority
WO
WIPO (PCT)
Prior art keywords
emphysema
lung
unit
area
image
Prior art date
Application number
PCT/KR2023/020600
Other languages
French (fr)
Korean (ko)
Inventor
김진현
송대현
성주현
김아라
백승주
권민욱
김지연
김태규
최현주
최우식
이승환
Original Assignee
경상국립대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020220174910A external-priority patent/KR20240092297A/en
Priority claimed from KR1020220174559A external-priority patent/KR20240091538A/en
Application filed by 경상국립대학교산학협력단 filed Critical 경상국립대학교산학협력단
Publication of WO2024128818A1 publication Critical patent/WO2024128818A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A61B10/02—Instruments for taking cell samples or for biopsy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders

Definitions

  • the present invention relates to a system for quantifying lung disease in lung samples and a method for quantifying lung disease using the same. It is configured to detect lung disease lesions in images of lung samples and use them to confirm the pneumonia rate and emphysema rate, based on expert opinion.
  • This relates to a lung disease quantification system for lung samples using computer vision and machine learning capable of standard quantification and a lung disease quantification method using the same.
  • Fine dust and ultrafine dust are known to have different components depending on the circumstances in which they are created and the surrounding environment, and the components of fine dust may include small particles, liquids, organic compounds, metals, and soil.
  • the concentration of heavy metals, known as hazardous substances, in the air around industrial complexes is high, so the impact of these heavy metals on health is likely to be greater.
  • the Ministry of Environment classifies those with an aerodynamic diameter of less than 10 ⁇ m as ultrafine dust 10 (PM10), and those with an aerodynamic diameter of less than 2.5 ⁇ m as ultrafine dust 2.5 (PM 2.5) (Ministry of Environment, Framework Act on Environmental Policy).
  • the primary route through which fine dust enters the body is the respiratory tract, and first, respiratory epithelial cells and alveolar macrophages are induced to produce inflammatory cytokines to remove foreign substances.
  • an inflammatory response occurs due to cytokines in the lung tissue, immune cells called Tcells act, reactive oxygen species (ROS) increase, and oxidative stress is induced by reactive oxygen species in the lung tissue. Oxidative stress again causes inflammation in lung tissue cells and DNA damage.
  • ROS reactive oxygen species
  • lung diseases caused by fine heavy metals are diagnosed by experts using lung photographs using X-ray or CT.
  • the lung disease includes pneumonia and emphysema.
  • Lung disease is diagnosed by calculating the area of diseased tissue compared to normal tissue. In this process, the expert's diagnosis has the disadvantage that it relies on each expert's experience and intuition, so there is significant inter-expert variability, and visual evaluation also takes a lot of time.
  • the present invention used machine learning to solve these shortcomings, which can provide more stable predictions at a much lower cost and thus can assist clinical pathologists much more easily when calculating ratios.
  • the present invention was created to solve the above problems, and the purpose of the present invention is to propose a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
  • the purpose of the present invention is to provide a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
  • An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging
  • a lung interstitium extraction unit 30 that extracts a lung interstitium area from the lung image from which the bronchioles have been removed;
  • a segmented image generator 40 that divides the lung image from which the lung interstitium area is extracted and generates a segmented image
  • a cell nucleus recognition unit 50 that recognizes a cell nucleus in the segmented image
  • a cell nucleus counting unit 60 that counts the number of recognized cell nuclei
  • a distribution map generator 70 that visualizes lung images according to the counted number of cell nuclei and generates a cell nucleus distribution map
  • a threshold comparison unit 80 that compares the counted number of cell nuclei with a threshold value
  • a pneumonia determination unit 90 that determines pneumonia when the number of cell nuclei counted is high compared to the threshold
  • an area calculation unit 100 that calculates a lung interstitium area in the segmented image determined to be pneumonia
  • An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
  • a bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
  • a pulmonary interstitium extraction step in which the pulmonary interstitium extraction unit 30 extracts a pulmonary interstitium area from the lung image from which the bronchioles have been removed;
  • a segmented image generation step in which the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and then generates a segmented image;
  • the pneumonia rate calculation unit 110 is characterized in that it includes a pneumonia rate calculation step in which the pneumonia rate of the lung is calculated.
  • An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging
  • a binarization processing unit 30 that binarizes the lung image from which the bronchioles have been removed;
  • An air layer calculation unit 40 that calculates the area of the entire air layer in the binarized lung image
  • an alveolar removal unit 50 that removes the air layer of alveoli and extracts emphysema from the binarized lung image
  • a coordinate confirmation unit 60 that checks the extracted coordinates of emphysema
  • An emphysema detection unit 70 that detects emphysema whose coordinates have been confirmed;
  • An emphysema calculator 80 that extracts the detected emphysema and calculates the area
  • It is characterized in that it includes an emphysema quantification unit 90 that quantifies the emphysema rate of the extracted emphysema.
  • An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
  • a bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
  • An air layer calculation step in which the air layer calculation unit 40 calculates the area of the entire air layer in the binarized lung image
  • An alveolar removal step in which the alveolar removal unit 50 removes the air layer of the alveoli and extracts emphysema from the binarized lung image;
  • It is characterized in that it includes an emphysema quantification step in which the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
  • the present invention can present a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
  • the present invention provides a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
  • Figure 1 is a schematic diagram of the pneumonia quantification system of the present invention.
  • Figure 2 is a flowchart of the present invention's pneumonia quantification method.
  • Figure 3 is a diagram showing a biopsy method for obtaining a lung specimen in the image acquisition step (S10) according to an embodiment of the present invention.
  • Figure 4 is a diagram confirming the disease characteristics in Figure 3 according to an embodiment of the present invention.
  • Figure 5 is a split image generated in the split image generation step (S40) according to an embodiment of the present invention.
  • Figure 6 is an image showing the cell nucleus recognition unit 50 and the cell nucleus counting unit 60 performing the cell nucleus recognition step (S50) and the cell nucleus counting step (S60) according to an embodiment of the present invention.
  • Figure 7 is an image of the pneumonia determination unit 90 performing the pneumonia determination step (S90) according to an embodiment of the present invention.
  • Figure 8 is a graph showing the distribution of the number of nuclei per unit area according to an embodiment of the present invention, (a) sample a, which the doctor determines to be 30%, and (b) sample b, which the doctor determines to be 90%.
  • Figure 9 relates to the distribution of the proportion of pneumonia in the samples of experts 1 and 2 according to an embodiment of the present invention, (a) when a patch containing 40 or more nuclei in a unit area is judged to be pneumonia, (b) unit (c) A patch containing more than 50 nuclei per unit area was judged to be pneumonia, and (c) a patch containing more than 60 nuclei per unit area was judged to be pneumonia.
  • Figure 10 is a quantified value of the pneumonia incidence rate based on the results quantified for each sample in Figure 9 according to an embodiment of the present invention.
  • Figure 11 shows the deviation between the pneumonia incidence rate quantification results performed through the present invention and the quantification results performed through Expert 1 (a) and Expert 2 (b) according to an embodiment of the present invention.
  • Figure 12 shows the results of pneumonia rates quantified by expert 1 according to an embodiment of the present invention.
  • Figure 13 shows the results of pneumonia rates quantified by expert 2 according to an embodiment of the present invention.
  • Figure 14 shows the quantitative results of pneumonia incidence (more than 50 unit area) performed through the present invention according to an embodiment of the present invention and the deviation of the quantitative results performed through Expert 1 (a) and Expert 2 (b).
  • Figure 15 is a configuration diagram of the emphysema quantification system of the present invention.
  • Figure 16 is a more detailed configuration diagram of the emphysema quantification system of the present invention.
  • Figure 17 is a flowchart of the emphysema quantification method of the present invention.
  • FIG. 18 is a more detailed flowchart of the emphysema quantification method of the present invention.
  • Figure 19 is a diagram showing a biopsy method for obtaining a lung specimen in the image acquisition step (S10) according to an embodiment of the present invention.
  • Figure 20 is a diagram confirming the disease characteristics in Figure 17 according to an embodiment of the present invention.
  • Figure 21 is a diagram showing the binarization processing unit 30 step-by-step binarization processing of the lung image from which the bronchioles have been removed in the third step (S30) binarization processing step according to an embodiment of the present invention. .
  • Figure 22 is a graph showing the comparison of results with expert 1 for the emphysema analogy when steps 12 (a) to 15 (d) among the 15 steps (Steps) of Figure 21 in experimental verification according to an embodiment of the present invention. .
  • Figure 23 is a graph showing the comparison of results with expert 2 for the emphysema analogy when steps 12 (a) to 15 (d) among the 15 steps (Steps) of Figure 21 in experimental verification according to an embodiment of the present invention.
  • Figure 24 shows the deviation of the quantification results performed by Expert 1 and Expert 2 at steps 12 to 15 among the 15 steps of Figure 21 according to an embodiment of the present invention. It is shown.
  • Figure 25 shows the results of emphysema incidence rate quantification performed through the present invention at step 12 to step 15 among the 15 steps (Steps) of Figure 21 according to an embodiment of the present invention, and expert 1 (a) and expert 2 (b) This shows the deviation of the quantification results performed through .
  • Figure 26 is a mechanical measurement result of emphysema quantification by expert 1 when steps 12 to 15 of the 15 steps (Steps) of Figure 21 are performed according to an embodiment of the present invention.
  • Figure 27 is a mechanical measurement result of emphysema quantification by expert 2 at steps 12 to 15 among the 15 steps (Steps) of Figure 21 according to an embodiment of the present invention.
  • Pneumonia and emphysema can be evaluated qualitatively and quantitatively through chest X-ray or CT.
  • the method according to the present invention reflected qualitative judgment based on the expert's experience and intuition, and applied the same quantitative method to all samples to absolutely quantify pneumonia and emphysema.
  • the present invention determined qualitative standards based on expert judgment to quantify pneumonia and emphysema. In addition, in order to verify the accuracy of the quantified method according to qualitative standards, verification was conducted according to the judgment of two experts.
  • pneumonia was determined by the number of nuclei in the cells and quantified according to the density of the nuclei.
  • the density quantified most similar to the expert's judgment is when the number of nuclei per unit area is 50 or more. Therefore, based on this, quantification standards were set and expert judgment and comparative analysis were conducted.
  • the two experts judged the pneumonia rate to be an intermediate rate (45 to 70) the difference between the two experts was small and stable. Meanwhile, quantification at a low ratio (0 to 45) showed that cases with high nuclear density were judged to be pneumonia, while quantification at high ratios (70 to 90) showed that cases with low nuclear density were judged to be pneumonia.
  • emphysema was quantified by removing normal alveoli according to the erode stage and detecting only the emphysema area.
  • the quantification criteria were determined based on the qualitative judgment of experts regarding the results we detected. As a result, the deviation from expert 1's quantification results was lowest at erosion steps 14 and 12 in the normal alveolar removal stage.
  • the results quantified by two experts at a low ratio (0 to 10) showed the smallest deviation from the quantified results. In step 15, the higher the ratio, the smaller the deviation from the results quantified in step 12. This showed that the two experts judged that even a small air gap was emphysema because the degree of emphysema in the sample was highly quantified.
  • the higher the degree of emphysema the greater the deviation from our results, showing that experts performed unstable quantification at a high rate.
  • the quantification method proposed in the present invention can analyze the quantification judgment of experts.
  • quantifying pneumonia and emphysema it was possible to identify experts' judgment trends and differences between experts. And although experts said they quantified it according to the absolute standards of pathology, it appeared to reflect the experts' visual experience and intuition.
  • This invention proposed a method for absolute quantification of pneumonia and emphysema in lung samples using machine learning and classical computer vision techniques. They showed that quantifying pneumonia and emphysema for each sample based on methods developed by experts can lead to different results. It also showed that the expert's qualitative judgment could vary depending on the degree of pneumonia and emphysema in the sample. Therefore, our quantification method can be a standard for quantification and can be used as a tool to correct the quantitative evaluation results of experts when they quantitatively evaluate the results.
  • the pneumonia quantification system for lung specimens using computer vision and machine learning includes an image acquisition unit (10), a bronchioles removal unit (20), a lung interstitial extraction unit (30), and a segmented image.
  • the image acquisition unit 10 biopsies the lung and acquires an image of the lung through microscopic imaging.
  • the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium.
  • One lung image represents one slide, and one slide consists of 2 to 4 sections.
  • the bronchiole removal unit 20 recognizes the bronchiole in the lung image and removes the bronchiole.
  • the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
  • the bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing.
  • the bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
  • a 5x5 partition is created and the bronchioles are labeled directly within this partition.
  • One partition is 1,640 wide and 1,440 long.
  • the total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
  • the lung interstitium extraction unit 30 extracts the lung interstitium area from the lung image from which the bronchiole has been removed.
  • the lung interstitium extraction unit 30 labels the lung interstitium area with pixels of the same color and then extracts the labeled pixel area.
  • the connected pulmonary interstitial area is labeled with pixels of the same color using connected components of OpenCV, one of the computer vision technologies, and the pixel area of this label is extracted.
  • the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and generates a segmented image.
  • the split image generator 40 prepares the split images to have the same width and length.
  • the sample is divided to facilitate visual identification of the nucleus.
  • image_slicer the sample is divided into 82 and 72 partitions, respectively, with size 100x100.
  • the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
  • the cell nucleus can be recognized using YOLOv5.
  • YOLOv5 learning black nuclei are captured in the partition and the labeled data are all made of the same class.
  • one section is divided into several partitions to detect nuclei.
  • One partition is divided into 82 and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels.
  • the total number of partitions is 1,399,248, of which 100 partitions are labeled, 90 training data and 10 test data are used to train YOLOv5.
  • the cell nucleus counting unit 60 counts the number of recognized cell nuclei.
  • the cell nucleus counting unit 60 counts the number of cell nuclei recognized in the segmented image through the learning.
  • the distribution map generator 70 generates a cell nucleus distribution map by visualizing the lung image according to the counted number of cell nuclei.
  • the distribution map generator 70 selects all segmented images suspected of having pneumonia and generates a distribution map of the cell nuclei based on the segmented images, as shown in FIG. 8.
  • the threshold comparison unit 80 compares the counted number of cell nuclei and the threshold.
  • the method used by experts calculates the overall ratio by calculating the number of cell nuclei per unit area.
  • unit area was used as the size of each segmented image. The important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
  • the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is high compared to the threshold.
  • Figure 8 shows an example of the distribution of the number of nuclei per unit area.
  • This distribution diagram follows a Gaussian distribution, and this distribution diagram alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each partition was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found.
  • the area calculation unit 100 calculates the lung interstitium area in the segmented image determined to be pneumonia.
  • the area calculation unit 100 calculates the lung interstitium area in the segmented image as a dense nucleus on a pixel basis.
  • the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lung.
  • the pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below.
  • a P is the area of the pulmonary interstitial area calculated by the area calculation unit 100
  • a INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30.
  • the method for quantifying pneumonia in lung specimens using computer vision and machine learning according to the present invention quantifies pneumonia using the pneumonia quantification system (1).
  • the first step (S10) is the image acquisition step.
  • the image acquisition unit 10 biopsies the lung and acquires a lung image through microscopic imaging. As shown in FIG. 3, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
  • the second step (S20) is the bronchioles removal step.
  • the bronchioles removal unit 20 recognizes the bronchioles in the lung image and then removes the bronchioles. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
  • the bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing.
  • the bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
  • a 5x5 partition is created and the bronchioles are labeled directly within this partition.
  • One partition is 1,640 wide and 1,440 long.
  • the total number of partitions is 5,925, and labeling is performed on 50, 40 are used as training data, and the remaining 10 are used as test data.
  • the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
  • the third step (S30) is the pulmonary interstitium extraction step.
  • the lung interstitium extraction unit 30 extracts the lung interstitium area from the lung image from which the bronchiole has been removed. More specifically, the lung interstitium extraction unit 30 labels the lung interstitium area with pixels of the same color and then extracts the labeled pixel area.
  • the connected pulmonary interstitial area is labeled with pixels of the same color using connected components of OpenCV, one of the computer vision technologies, and the pixel area of this label is extracted.
  • the fourth step (S40) is a segmented image generation step.
  • the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and generates a segmented image. More specifically, the split image generator 40 prepares the split images to have the same width and length.
  • the sample is divided to facilitate visual identification of the nucleus.
  • image_slicer the sample is divided into 82 and 72 partitions, respectively, with size 100x100.
  • the fifth step (S50) is the cell nucleus recognition step.
  • the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
  • the cell nucleus can be recognized using YOLOv5.
  • YOLOv5 learning black nuclei are captured in the partition and the labeled data are all made of the same class.
  • To detect nuclei in high-quality specimen images one section is divided into several partitions to detect nuclei.
  • One partition is divided into 82 and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels. The total number of partitions is 1,399,248, 100 partitions are labeled, 90 training data and 10 test data are used to train YOLOv5.
  • the sixth step (S60) is the cell nucleus counting step.
  • the cell nucleus counting unit 60 counts the number of recognized cell nuclei.
  • the seventh step (S70) is the distribution map generation step.
  • the distribution map generating unit 70 In the seventh step (S70), the distribution map generating unit 70 generates a cell nucleus distribution map by visualizing the lung image according to the counted number of cell nuclei. More specifically, the distribution map generator 70 selects all segmented images suspected of having pneumonia and generates a distribution map of the cell nuclei based on the segmented images, as shown in FIG. 8.
  • the eighth step (S80) is a threshold comparison step.
  • the threshold value comparison unit 80 compares the counted number of cell nuclei and the threshold value.
  • the method used by experts calculates the overall ratio by calculating the number of cell nuclei per unit area.
  • unit area was used as the size of each segmented image. The important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
  • the ninth step (S90) is the pneumonia determination step.
  • the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is greater than the threshold value.
  • Figure 8 shows an example of the distribution of the number of nuclei per unit area.
  • This distribution diagram follows a Gaussian distribution, and this distribution diagram alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each partition was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found.
  • the tenth step (S10) is the area calculation step.
  • the area calculation unit 100 calculates the lung interstitium area in the segmented image determined to be pneumonia. More specifically, the area calculation unit 100 calculates the lung interstitium area in the segmented image as a dense nucleus on a pixel basis.
  • the 11th step (S110) is the pneumonia rate calculation step.
  • the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lung. More specifically, the pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below.
  • a P is the area of the pulmonary interstitial area calculated by the area calculation unit 100
  • a INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30.
  • mice Seven-week-old C57BL/6 mice were selected as experimental animals and exposed to mouse lung tissue samples and diluted nickel, chromium, manganese, and cadmium at concentrations of 50nM, 20nM, 10nM, and 10nM, respectively. A total of 70 samples were collected through the nasal cavity, alone or in combination, once a day for 4 weeks. This experiment was conducted at the University of Ulsan SPF (no specific pathogens added) and was reviewed by the Animal Invention Center of the University of Ulsan (IACUC) (IACUC No. BSK-21-030).
  • IACUC Animal Invention Center of the University of Ulsan
  • the 70 specimens collected consisted of each slide.
  • Figure 3 shows one slide, and one slide consists of 2-4 sections. Therefore, a total of 70 slides were sampled and cross-sectional data were collected from 237 slides. Of these, only 70 sections used by actual doctors were used.
  • One section is a high-quality image measuring 8200 wide and 7200 long.
  • the rate of pneumonia was quantified, and the method for confirming the rate of pneumonia through a sample is as follows.
  • the ratio is quantified by calculating the area of the interstitial area where many nuclei are concentrated compared to the total interstitial area of the specimen excluding the air passage layer.
  • the index for calculating the rate of pneumonia is as follows [Equation 1].
  • a P is the area of the pulmonary interstitial area calculated by the area calculation unit 100
  • a INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30.
  • YOLACT learning requires bronchioles labels. Because the size of the bronchioles is relatively small compared to the overall section, we divide them into 5x5 partitions and label the bronchioles directly within these partitions. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data.
  • the bronchial regions within every section are segmented and classified.
  • YOLOv5 learning black nuclei are captured in the segmented image, and all label data are made of the same class.
  • One section is divided into several divisions to detect nuclei.
  • One segmented image is divided into 82 horizontal and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels.
  • the total number of segmented images was 1,399,248, of which 100 segmented images were labeled, with 90 training data and 10 test data.
  • YOLOv5 learning is performed to determine the number of detected nuclei for each segmented image.
  • the method used by experts is to calculate the overall ratio by counting the number of cell nuclei per unit area.
  • the unit area is also used as the size of each divided image.
  • the important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
  • Figure 8 shows an example of the distribution of the number of nuclei per unit area.
  • This distribution chart follows a Gaussian distribution, and this distribution chart alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each segmented image was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found.
  • the area of the lung interstitium area calculated by the area calculation unit 100 and the area of the lung interstitium area extracted by the lung interstitium extraction unit 30 are calculated as dense nuclei in pixel units.
  • the final pneumonia rate is quantified by calculating the area of the lung interstitium where the nuclei are concentrated compared to the total lung interstitium area.
  • Figure 9 shows the results of experts 1 and 2 quantifying the pneumonia rate according to the quantification of each sample and the number of nuclei in each segmented image.
  • the x-axis represents the number of samples and the y-axis represents the rate of pneumonia for each sample.
  • Figure 9(a) shows that most of our results are calculated higher than those quantified by experts, meaning that experts consider pneumonia when the criterion for quantifying pneumonia is concentrated in more than 40 nuclei per unit area. .
  • Figure 9(b) can be judged to be more similar to that of the expert than Figure 9(a).
  • the error was the smallest among the 60 ranges quantified by experts.
  • our quantification results were smaller than those of experts. This means that even a concentration of less than 50 nuclei per unit area, when quantified by experts at a high rate, is considered pneumonia.
  • Figure 10 confirms that the rate of pneumonia quantified had the smallest deviation between the two experts when there were more than 50 nuclei per unit area. This can be confirmed through Figure 9(b).
  • the results of quantifying the rate of pneumonia when there were more than 50 nuclei per unit area were most similar to the quantitative inventions of experts, and our quantification method was absolute because it quantified the rate of pneumonia by identifying the number of nuclei present in the sample. . Therefore, the rate of pneumonia can be quantified using absolute standards based on qualitative analysis based on the experience and intuition of experts. Below, we compare and analyze the quantification results of experts based on quantification based on more than 50 nuclei per unit area.
  • the quantification method provided by the present invention provides observations and considerations of traditional quantification methods for human and mouse samples.
  • the sample data of the present invention was collected for the purpose of pathology invention together with Changwon Gyeongsang National University Hospital and consists of a total of 237 lung sample images.
  • To evaluate the accuracy of the proposed method for quantifying pneumonia and emphysema we collected data on the results of the invention for quantifying pneumonia and emphysema as judged by two independent pathologists.
  • data judged by a pathology expert were collected, and all were judged by a pathology expert at Changwon Gyeongsang National University Hospital.
  • the image size of each neural network model used in the present invention is YOLOv5 640x640, YOLACT is 1,640x1, 440 pixel size (neural network input size), and RGB 32-bit image.
  • the equipment used in the YOLOv5 and YOLACT experiments was Ubuntu 20.04 LTS and the GPU was GeForce RTX 3090 Ti GPU, and the OS was implemented in python. And the equipment used in the experiment using computer vision was implemented in python using Windows 10 as the OS and Geforce RTX 2080 Ti GPU as the GPU.
  • Figure 11 shows the difference between the pneumonia results quantified by the expert and the results of the expert's qualitative analysis of the sample. Deviation was calculated using the sliding window method based on the proportion of pneumonia quantified by experts.
  • Figure 11 shows the results of a comparative analysis of the pneumonia rate quantified by experts based on the pneumonia rate quantified by us.
  • the degree of pneumonia quantified by the two experts was closest to the machine's judgment, in the mid-range of 45 to 70. This means that the two experts quantified the severity of pneumonia more stably in the mid-range (45-70) than in the relatively low range (0-45) and high range (70-90).
  • the deviation from the quantitative result was the smallest when it was 60 or more in the range from 0 to 20. In the range of 20 to 60, the deviation from the quantitative result was smallest when it was 50 or more. And in the range of 60 or more, the deviation from the quantified results was the smallest when there were 40 or more.
  • Figures 12 and 13 also show that experts tend to judge pneumonia as having a low nuclear concentration per unit area within the high range.
  • Figure 12 shows the quantified results according to the pneumonia rate quantified by expert 1, and most similar results to the quantified results were shown when the nuclear density distribution was 40 or more. Expert 1 even considered that most cases of pneumonia were less than 40 per unit area.
  • Figure 30 also shows that in the range where the pneumonia rate quantified by expert 2 is high, cases where the nuclear density per unit area is less than 40 are considered pneumonia.
  • Figure 14 shows the deviation of expert results based on the quantification results according to the present invention (proportion of pneumonia quantified when there are 50 or more per unit area).
  • the x-axis represents the rate of pneumonia and the y-axis represents the deviation.
  • Figure 14(a) shows the deviation from expert 1 based on the results of the present invention, and the expert quantified pneumonia at a higher level than the results of the present invention in the pneumonia degree range of 50 or higher.
  • Figure 14(b) shows the deviation from expert 2 based on the results of the present invention, and the expert quantified pneumonia at a higher level than the results of the present invention in the pneumonia degree range of 50 or higher.
  • a bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole.
  • a pulmonary interstitium extraction step in which the pulmonary interstitium extraction unit 30 extracts the pulmonary interstitium area from the lung image from which the bronchioles have been removed.
  • a segmented image generation step in which the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and then generates a segmented image.
  • S50 A cell nucleus recognition step in which the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
  • S80 A threshold comparison step in which the threshold comparison unit 80 compares the counted number of cell nuclei with the threshold value.
  • the emphysema quantification system for lung specimens using computer vision and machine learning includes an image acquisition unit (10), a bronchioles removal unit (20), a binarization processing unit (30), and an air layer. It is characterized by comprising a calculation unit 40, an alveolar removal unit 50, a coordinate confirmation unit 60, an emphysema detection unit 70, an emphysema calculation unit 80, and an emphysema quantification unit 90.
  • the image acquisition unit 10 biopsies the lung and acquires an image of the lung through microscopic imaging.
  • the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium.
  • One lung image represents one slide, and one slide consists of 2 to 4 sections.
  • the bronchiole removal unit 20 recognizes the bronchiole in the lung image and removes the bronchiole.
  • the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
  • the bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing.
  • the bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
  • a 5x5 partition is created and the bronchioles are labeled directly within this partition.
  • One partition is 1,640 wide and 1,440 long.
  • the total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
  • the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed.
  • the binarization processing unit 30 processes the pulmonary interstitium and bronchiole areas with the first color (1), and processes the alveoli and emphysema areas with the second color (0).
  • the binarized lung image that has passed through the binarization processing unit 30 is processed step by step as shown in FIG. 21 to search for emphysema, removing general alveoli and leaving emphysema.
  • the stages reflected the opinion that steps 13 to 15 were most appropriate through consultation with pathology experts.
  • the interstitial and bronchial regions are colored black using the OpenCV library. Additionally, alveoli and emphysema are treated in white.
  • the air layer calculation unit 40 calculates the total air layer area in the binarized lung image.
  • the air layer calculation unit 40 calculates the sum of the pixel area of the label designation unit 41 that specifies a label in the binarized lung image and the designated label. It consists of an air layer pixel extraction unit 42 that extracts the entire air layer pixel area.
  • OpenCV's connectedComponent is used to label air layers in a sample. Afterwards, the total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
  • the alveolar removal unit 50 removes the air layer of alveoli from the binarized lung image and extracts emphysema.
  • Normal alveoli are smaller and relatively round in shape than those of emphysema or lung bronchioles. Therefore, if the pulmonary interstitial layer becomes thicker, the alveoli disappear due to the thickened interstitial layer, and only emphysema or bronchioles remain. In other words, because the bronchioles were previously removed, if the pulmonary interstitial layer is thickened, only the emphysema has an air layer, and this part is judged to be emphysema.
  • OpenCV's binarization process is used to perform a step-by-step erosion process for the air layer region.
  • the step-by-step erode process the normal alveoli are removed, leaving only the features of emphysema.
  • the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema.
  • the coordinate confirmation unit 60 includes an emphysema designation unit 61 that specifies a label for emphysema extracted from the binarized lung image, and a center coordinate in the designated emphysema label. It consists of a central coordinate extraction unit 62 that extracts.
  • the connected component of OpenCV is used to label the remaining emphysema features of the sample. Afterwards, the center coordinate is extracted from the specified label.
  • the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed.
  • the emphysema detection unit 70 is an image mapping unit ( 71) and an emphysema color designation unit 72 that designates a color to the emphysema area based on the coordinates of the center of the emphysema.
  • the image mapping unit 71 maps the center coordinates of emphysema detected by the coordinate confirmation unit 60 to the original lung image to calculate the actual size of emphysema in the emphysema calculation unit 80 below.
  • the emphysema color designation unit 72 colors only the identified emphysema to be distinguished.
  • the center coordinates of the confirmed emphysema are mapped to the original image of the image acquisition unit 10 to the original lung image. Afterwards, a color is assigned to the emphysema area using the BFS (Breadth First Search) algorithm based on the center coordinates.
  • BFS Bitth First Search
  • the emphysema calculator 80 extracts the detected emphysema and calculates the area.
  • the emphysema calculation unit 80 checks the emphysema area detected by the emphysema detection unit 70, and then configures the emphysema area designation unit 81 to label the emphysema area and the labeled emphysema area. It consists of an emphysema pixel extraction unit 82 that extracts the area of emphysema pixels by calculating the sum of pixels in the area.
  • inRange is used to extract the area of emphysema detected by the BFS (Breadth First Search) algorithm.
  • BFS Bitth First Search
  • the extracted emphysema region is labeled using OpenCV's connectedComponents.
  • the area of the emphysema pixel is extracted by calculating the sum of the pixel areas of the designated labels.
  • Emphysema can be detected by substituting the center coordinates extracted in the previous step into the original image and coloring the emphysema area. Nodes are set to pixels and black areas are considered gaps. At this time, it was set to search the white area. The extraction coordinates are set to the first node and the search area is set to change from white to red pixels.
  • the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
  • the emphysema determination unit 90 calculates the emphysema rate according to [Equation 1] below.
  • Emphysema rate A a / A emp
  • a a is the area of the total air layer calculated by the air layer calculator 40
  • a emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
  • the area of the total air layer extracted by the air layer calculator 40 and the detected emphysema area calculated by the emphysema calculator 80 are calculated on a pixel basis.
  • the final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
  • the method for quantifying emphysema in lung samples using computer vision and machine learning according to the present invention quantifies emphysema using the emphysema quantification system (1), as shown in FIGS. 17 and 18.
  • the first step (S10) is the image acquisition step.
  • the image acquisition unit 10 biopsies the lung and acquires a lung image taken with a microscope. As shown in FIG. 17, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
  • the second step (S20) is the bronchioles removal step.
  • the bronchioles removal unit 20 recognizes the bronchioles in the lung image and then removes the bronchioles. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
  • the bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing.
  • the bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
  • a 5x5 partition is created and the bronchioles are labeled directly within this partition.
  • One partition is 1,640 wide and 1,440 long.
  • the total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
  • the third step (S30) is a binarization processing step.
  • the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed. More specifically, the binarization processing unit 30 processes the pulmonary interstitium and bronchiole areas with the first color (1), and processes the alveoli and emphysema areas with the second color (0).
  • the binarized lung image that has passed through the binarization processing unit 30 is processed step by step as shown in FIG. 21 to search for emphysema, removing general alveoli and leaving behind emphysema.
  • the stages reflected the opinion that steps 13 to 15 were most appropriate through consultation with pathology experts.
  • the interstitial and bronchial regions are colored black using the OpenCV library. Additionally, alveoli and emphysema are treated in white.
  • the fourth step (S40) is the air space calculation step.
  • the air layer calculation unit 40 calculates the total area of the air layer in the binarized lung image. More specifically, in the fourth step (S40), the air layer calculation unit 40 includes a label designation unit 41 that specifies a label in the binarized lung image and the designated label. It consists of an air layer pixel extraction unit 42 that extracts the entire air layer pixel area by calculating the sum of the pixel areas.
  • OpenCV's connectedComponent is used to label air layers in a sample. Afterwards, the total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
  • the fifth step (S50) is the alveolar removal step.
  • the alveolar removal unit 50 removes the air layer of the alveoli from the binarized lung image and extracts emphysema.
  • Normal alveoli are smaller and relatively round in shape than those of emphysema or lung bronchioles. Therefore, if the pulmonary interstitial layer becomes thicker, the alveoli disappear due to the thickened interstitial layer, and only emphysema or bronchioles remain. In other words, because the bronchioles were previously removed, if the pulmonary interstitial layer is thickened, only the emphysema has an air layer, and this part is judged to be emphysema.
  • OpenCV's binarization process is used to perform a step-by-step erosion process for the air layer region.
  • the step-by-step erode process the normal alveoli are removed, leaving only the features of emphysema.
  • the sixth step (S60) is the coordinate confirmation step.
  • the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema. More specifically, the sixth step (S60) is an emphysema designation step (S61) in which the emphysema designation unit 61 assigns a label to the emphysema extracted from the binarized lung image and central coordinate extraction. It consists of a center coordinate extraction step (S62) in which the unit 62 extracts the center coordinate from the designated emphysema label.
  • the connected component of OpenCV is used to label the remaining emphysema features of the sample. Afterwards, the center coordinate is extracted from the specified label.
  • the seventh step (S70) is the emphysema detection step.
  • the emphysema detection unit 70 detects emphysema whose coordinates are confirmed. More specifically, in the seventh step (S70), the image mapping unit 71 matches the center coordinates of emphysema confirmed by the coordinate confirmation unit 60 to the lung image acquired by the image acquisition unit 10 to the original lung image. It consists of an image mapping step (S71) and an emphysema color designation step (S72) in which the emphysema color designation unit 72 assigns a color to the emphysema area based on the center coordinates of the emphysema.
  • the image mapping unit 71 maps the center coordinates of emphysema detected by the coordinate confirmation unit 60 to the original lung image and actualizes them in the emphysema calculation unit 80 below.
  • the size of emphysema is calculated.
  • the emphysema color designation unit 72 colors only the identified emphysema to distinguish it.
  • the center coordinates of the confirmed emphysema are mapped to the original image of the image acquisition unit 10 to the original lung image. Afterwards, a color is assigned to the emphysema area using the BFS (Breadth First Search) algorithm based on the center coordinates.
  • BFS Bitth First Search
  • the eighth step (S80) is the emphysema calculation step.
  • the emphysema calculation unit 80 extracts the detected emphysema and calculates the area.
  • the eighth step (S80) is an emphysema area designation step in which the emphysema area designation unit 81 confirms the emphysema area detected by the emphysema detection unit 70 and then labels the emphysema area ( S81) and an emphysema pixel extraction step (S82) in which the emphysema pixel extraction unit 82 extracts the area of the emphysema pixel by calculating the sum of pixels of the labeled emphysema area.
  • inRange is used to extract the area of emphysema detected by the BFS (Breadth First Search) algorithm.
  • BFS Bitth First Search
  • the extracted emphysema region is labeled using OpenCV's connectedComponents.
  • the area of the emphysema pixel is extracted by calculating the sum of the pixel areas of the designated labels.
  • Emphysema can be detected by substituting the center coordinates extracted in the previous step into the original image and coloring the emphysema area. Nodes are set to pixels and black areas are considered gaps. At this time, it was set to search the white area. The extraction coordinates are set to the first node and the search area is set to change from white to red pixels.
  • the ninth step (S90) is the emphysema quantification step.
  • the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
  • the emphysema determination unit 90 calculates the emphysema rate according to [Equation 1] below.
  • Emphysema rate A a / A emp
  • a a is the area of the total air layer calculated by the air layer calculator 40
  • a emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
  • the area of the total air layer extracted by the air layer calculator 40 and the detected emphysema area calculated by the emphysema calculator 80 are calculated on a pixel basis.
  • the final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
  • mice Seven-week-old C57BL/6 mice were selected as experimental animals and exposed to mouse lung tissue samples and diluted nickel, chromium, manganese, and cadmium at concentrations of 50nM, 20nM, 10nM, and 10nM, respectively. A total of 70 samples were collected through the nasal cavity, alone or in combination, once a day for 4 weeks. This experiment was conducted at the University of Ulsan SPF (no specific pathogens added) and was reviewed by the Animal Invention Center of the University of Ulsan (IACUC) (IACUC No. BSK-21-030).
  • IACUC Animal Invention Center of the University of Ulsan
  • the 70 specimens collected consisted of each slide.
  • Figure 17 shows one slide, and one slide consists of 2-4 sections. Therefore, a total of 70 slides were sampled and cross-sectional data were collected from 237 slides. Of these, only 70 sections actually used by doctors were used.
  • One section is a high-quality image measuring 8200 wide and 7200 long.
  • the rate of emphysema was quantified, and the method for confirming the rate of emphysema through a sample is as follows.
  • the proportion of emphysema is quantified by calculating the air space of emphysema compared to the total air space excluding bronchioles.
  • Emphysema rate A a / A emp
  • a a is the area of the total air layer calculated by the air layer calculator 40
  • a emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
  • YOLACT learning requires bronchioles labels. Because the size of the bronchioles is relatively small compared to the overall section, we divide them into 5x5 partitions and label the bronchioles directly within these partitions. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data.
  • the bronchial regions within every section are segmented and classified.
  • the interstitial and bronchial regions are colored black. Treats alveoli and emphysema with white color.
  • the total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
  • OpenCV's erosion is used to perform a step-by-step erosion process on the air layer region.
  • the normal alveoli are removed, leaving only the features of emphysema.
  • the center coordinates of the confirmed emphysema are mapped to the original image extracted in 2-5) to the original lung image. Based on the coordinates of the center, the emphysema area is assigned a color using the BFS (Breadth First Search) algorithm.
  • BFS Bitth First Search
  • the total area of the air layer extracted in 2-3) total air layer area calculation and the area of emphysema extracted in 2-7) emphysema area extraction are calculated in pixel units.
  • the final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
  • Figures 22 and 23 show the results of emphysema quantified at each stage based on quantification by two experts.
  • the x-axis represents the number of samples, and the y-axis represents the proportion of emphysema for each sample.
  • Figure 22 shows quantitative results based on Expert 1. Samples quantified at the same ratio were grouped and compared. For Expert 1, 0% for samples 1 to 2, 1% for samples 3 to 13, 2% for samples 14 to 20, 5% for samples 21 to 45, 10% for samples 46 to 59, and 1% for samples 60 to 64. It was quantified as 15%, samples 65 to 69 as 20%, and sample 70 as 30%.
  • Figure 23 shows the quantified results based on expert 2. Analyzing the graph in Figure 23, samples quantified at the same ratio were grouped and compared. For Expert 2, Sample 1 was quantified as 2%, Samples 2 to 11 as 5%, Samples 12 to 13 as 7%, Samples 14 to 26 as 10%, and Samples 27 to 35 as 10%. It was quantified as 15%, samples 36 to 54 were 20%, samples 55 to 59 were 25%, samples 60 to 67 were 30%, samples 68 to 69 were 35%, and sample 70 was 50%.
  • the expert's mean, variance, and standard deviation were calculated as shown in Figure 24.
  • Expert 1 showed that the deviation decreased with each step, while expert 2 showed that the deviation increased with each step.
  • Expert 1 quantified the degree of emphysema as low, and Expert 2 quantified it as high.
  • Expert 1 showed the smallest deviation from the results according to the present invention at step 14 and Expert 2 at step 12.
  • emphysema was detected in the same way for all samples and quantified on an absolute basis. Therefore, below, the results according to the present invention are assumed to be standard quantification and the results of expert quantification are compared and analyzed.
  • the stage with the smallest deviation from the quantified results at each stage is selected based on the proportion of emphysema quantified by the expert for each sample.
  • the present invention can present a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
  • the present invention provides a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
  • a bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole.
  • Emphysema detection step in which the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed.
  • An emphysema color designation step in which the emphysema color designation unit 72 assigns a color to the emphysema area based on the coordinates of the center of the emphysema.
  • Emphysema calculation step in which the emphysema calculation unit 80 extracts the detected emphysema and calculates the area.
  • An emphysema area designation step in which the emphysema area designation unit 81 confirms the emphysema area detected by the emphysema detection unit 70 and then labels the emphysema area.
  • An emphysema pixel extraction step in which the emphysema pixel extraction unit 82 extracts the area of the emphysema pixel by calculating the sum of pixels of the labeled emphysema area.
  • Emphysema quantification step in which the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Databases & Information Systems (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to a system for quantifying lung disease of a lung sample, and a method for quantifying lung disease by using the same, and, more particularly, to a system for quantifying lung disease of a lung sample by using computer vision and machine learning, and a method for quantifying lung disease by using same, the system detecting pneumonia and emphysematous lesions from an image of a lung sample and using same to identify pneumonia incidence and emphysema incidence, thereby enabling standard quantification of expert opinions.

Description

컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐질환 정량화 시스템 및 이를 이용한 폐질환 정량화 방법Lung disease quantification system for lung samples using computer vision and machine learning and lung disease quantification method using the same
본 발명은 폐 검체의 폐질환 정량화 시스템 및 이를 이용한 폐질환 정량화 방법에 관한 것으로, 폐 검체의 이미지에서 폐질환 병변을 감지하고 이를 이용하여 폐렴율 및 폐기종율을 확인하도록 구성되어 전문가의 소견에 대한 표준 정량화가 가능한 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐질환 정량화 시스템 및 이를 이용한 폐질환 정량화 방법에 관한 것이다. The present invention relates to a system for quantifying lung disease in lung samples and a method for quantifying lung disease using the same. It is configured to detect lung disease lesions in images of lung samples and use them to confirm the pneumonia rate and emphysema rate, based on expert opinion. This relates to a lung disease quantification system for lung samples using computer vision and machine learning capable of standard quantification and a lung disease quantification method using the same.
미세먼지와 초미세먼지는 만들어지는 상황과 주변 환경에 따라 서로 다른 성분을 가지고 있는 것으로 알려져 있으며, 미세먼지의 구성성분에는 작은 입자, 액체, 유기화합물, 금속, 흙 등이 포함될 수 있다. 특히 산업단지 주변 대기 중 유해물질로 알려진 중금속의 농도가 높아 이들 중금속이 건강에 미치는 영향이 더 클 가능성이 높다.Fine dust and ultrafine dust are known to have different components depending on the circumstances in which they are created and the surrounding environment, and the components of fine dust may include small particles, liquids, organic compounds, metals, and soil. In particular, the concentration of heavy metals, known as hazardous substances, in the air around industrial complexes is high, so the impact of these heavy metals on health is likely to be greater.
환경부는 공기역학적 직경이 10㎛ 미만인 것을 초미세먼지 10(PM10), 2.5㎛ 미만인 것을 초미세먼지 2.5(PM 2.5)로 분류(환경부, 환경정책기본법)한다. 미세먼지가 체내로 유입되는 1차 경로는 호흡기이며, 먼저 호흡기 상피세포와 폐포 대식세포가 염증성 사이토카인을 생성하도록 유도하여 외부 물질을 제거한다. 그 결과, 폐 조직의 사이토카인에 의해 염증 반응이 일어나고, Tcell 이라는 면역 세포가 작용하여 활성산소종(ROS)이 증가하고, 폐 조직의 활성산소에 의해 산화 스트레스가 유발된다. 산화 스트레스는 다시 폐 조직 세포에 염증을 일으키고 DNA 손상을 일으킨다. 이러한 반응의 지속적인 발현은 폐 기능 감소, 폐기종, 만성 폐쇄성 폐질환 악화, 만성 염증, 폐암으로 이어질 수 있다.The Ministry of Environment classifies those with an aerodynamic diameter of less than 10㎛ as ultrafine dust 10 (PM10), and those with an aerodynamic diameter of less than 2.5㎛ as ultrafine dust 2.5 (PM 2.5) (Ministry of Environment, Framework Act on Environmental Policy). The primary route through which fine dust enters the body is the respiratory tract, and first, respiratory epithelial cells and alveolar macrophages are induced to produce inflammatory cytokines to remove foreign substances. As a result, an inflammatory response occurs due to cytokines in the lung tissue, immune cells called Tcells act, reactive oxygen species (ROS) increase, and oxidative stress is induced by reactive oxygen species in the lung tissue. Oxidative stress again causes inflammation in lung tissue cells and DNA damage. Continued expression of this reaction can lead to decreased lung function, emphysema, exacerbation of chronic obstructive pulmonary disease, chronic inflammation, and lung cancer.
병리학에서는 미세 중금속에 의해 발생하는 폐질환을 X-ray나 CT를 사용한 폐 사진을 가지고 전문가가 진단한다. 상기 폐질환은 폐렴(pneumonia) 및 폐기종(pneumonia)을 포함한다. 폐질환은 정상적인 조직에 대비 질병에 해당하는 조직의 면적을 계산하여 질병의 비율을 진단한다. 이러한 과정에서 전문가의 진단에는 전문가마다의 경험과 직관에 의존하기에 전문가 간 변동성이 심하고, 시각적 평가 또한 시간이 많이 걸린다는 단점이 있다. In pathology, lung diseases caused by fine heavy metals are diagnosed by experts using lung photographs using X-ray or CT. The lung disease includes pneumonia and emphysema. Lung disease is diagnosed by calculating the area of diseased tissue compared to normal tissue. In this process, the expert's diagnosis has the disadvantage that it relies on each expert's experience and intuition, so there is significant inter-expert variability, and visual evaluation also takes a lot of time.
따라서 본 발명은 이러한 단점을 해결하기 위해 기계 학습을 사용하였는데 이는 훨씬 저렴한 비용으로 보다 안정적인 예측을 제고할 수 있으므로 임상 병리학자들이 비율을 계산할 때 훨씬 더 용이하게 보조해 줄 수 있다.Therefore, the present invention used machine learning to solve these shortcomings, which can provide more stable predictions at a much lower cost and thus can assist clinical pathologists much more easily when calculating ratios.
본 발명은 상기의 문제점을 해결하기 위해서 안출된 것으로서, 본 발명의 목적은 머신러닝을 이용하여 폐 표본의 폐질환의 절대 기준에 따른 정량화 방법을 제시하고자 한다. The present invention was created to solve the above problems, and the purpose of the present invention is to propose a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
또한, 본 발명의 목적은 폐 표본에서 폐질환의 절대 정량화를 기반으로 인간의 경험과 직관에 기반 한 정량화 결정을 수정할 수 있는 시스템을 제공하고자 한다.Additionally, the purpose of the present invention is to provide a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
발명이 해결하고자 하는 기술적 과제들은 이상에서 언급한 기술적 과제들로 제한되지 않으며, 언급되지 않은 또 다른 기술적 과제들은 아래의 기재로부터 본 발명이 속하는 기술분야에서 통상의 지식을 가진 자에게 명확하게 이해될 수 있을 것이다.The technical problems to be solved by the invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.
(1) 폐렴 정량화 시스템(1) Pneumonia quantification system
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템은,The system for quantifying pneumonia in lung samples using computer vision and machine learning according to the present invention,
폐를 생검하여 현미경 촬영을 통해 폐이미지 획득하는 이미지획득부(10);An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging;
상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지제거부(20);A bronchiole removal unit (20) that recognizes bronchiole in the lung image and then removes the bronchiole;
상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출하는 폐간질추출부(30);A lung interstitium extraction unit 30 that extracts a lung interstitium area from the lung image from which the bronchioles have been removed;
상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성하는 분할이미지생성부(40);a segmented image generator 40 that divides the lung image from which the lung interstitium area is extracted and generates a segmented image;
상기 분할이미지에서 세포핵(cell nucleus)을 인식하는 세포핵인식부(50);A cell nucleus recognition unit 50 that recognizes a cell nucleus in the segmented image;
상기 인식된 세포핵(cell nucleus)의 개수를 집계하는 세포핵집계부(60); A cell nucleus counting unit 60 that counts the number of recognized cell nuclei;
상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus) 분포도를 생성하는 분포도생성부(70);a distribution map generator 70 that visualizes lung images according to the counted number of cell nuclei and generates a cell nucleus distribution map;
상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교하는 임계값비교부(80);a threshold comparison unit 80 that compares the counted number of cell nuclei with a threshold value;
상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단하는 폐렴판단부(90);a pneumonia determination unit 90 that determines pneumonia when the number of cell nuclei counted is high compared to the threshold;
상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산하는 면적계산부(100);an area calculation unit 100 that calculates a lung interstitium area in the segmented image determined to be pneumonia;
폐의 폐렴율을 산출하는 폐렴율산출부(110);를 포함하는 것을 특징으로 한다. It is characterized by including a pneumonia rate calculation unit 110 that calculates the pneumonia rate of the lung.
(2) 폐렴 정량화 방법(2) Pneumonia quantification method
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법은,The method for quantifying pneumonia in lung specimens using computer vision and machine learning according to the present invention,
이미지획득부(10)가 폐를 생검한 현미경 촬영을 통해 폐이미지를 획득하는 이미지 획득 단계;An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계;A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
폐간질추출부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질 (interstitium) 영역을 추출하는 폐간질 추출 단계; A pulmonary interstitium extraction step in which the pulmonary interstitium extraction unit 30 extracts a pulmonary interstitium area from the lung image from which the bronchioles have been removed;
분할이미지생성부(40)가 상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성하는 분할이미지 생성 단계;A segmented image generation step in which the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and then generates a segmented image;
세포핵인식부(50)가 상기 분할이미지에서 세포핵(cell nucleus)을 인식하는 세포핵 인식 단계;A cell nucleus recognition step in which the cell nucleus recognition unit 50 recognizes a cell nucleus in the segmented image;
세포핵집계부(60)가 상기 인식된 세포핵(cell nucleus)의 개수를 집계하는 세포핵 집계 단계; A cell nucleus counting step in which the cell nucleus counting unit 60 counts the number of the recognized cell nuclei;
분포도생성부(70)가 상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus) 분포도를 생성하는 분포도 생성 단계;A distribution map generating step in which the distribution map generator 70 generates a cell nucleus distribution map by visualizing a lung image according to the counted number of cell nuclei;
임계값비교부(80)가 상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교하는 임계값 비교 단계;A threshold comparison step in which the threshold comparison unit 80 compares the counted number of cell nuclei with a threshold value;
폐렴판단부(90)가 상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단하는 폐렴 판단 단계;A pneumonia determination step in which the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is high compared to the threshold;
면적계산부(100)가 상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산하는 면적 계산 단계;An area calculation step in which the area calculation unit 100 calculates a lung interstitium area in the segmented image determined to be pneumonia;
폐렴율산출부(110)가 폐의 폐렴율을 산출하는 폐렴율 산출 단계;를 포함하는 것을 특징으로 한다.The pneumonia rate calculation unit 110 is characterized in that it includes a pneumonia rate calculation step in which the pneumonia rate of the lung is calculated.
(3) 폐기종 정량화 시스템(3) Emphysema quantification system
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템은,The system for quantifying emphysema in lung samples using computer vision and machine learning according to the present invention,
폐를 생검하여 현미경 촬영을 통해 폐이미지 획득하는 이미지획득부(10);An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging;
상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지제거부(20);A bronchiole removal unit (20) that recognizes bronchiole in the lung image and then removes the bronchiole;
상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리하는 이진화처리부(30);A binarization processing unit 30 that binarizes the lung image from which the bronchioles have been removed;
상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산하는 공기층계산부(40);An air layer calculation unit 40 that calculates the area of the entire air layer in the binarized lung image;
상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출하는 폐포제거부(50);an alveolar removal unit 50 that removes the air layer of alveoli and extracts emphysema from the binarized lung image;
상기 추출된 폐기종의 좌표를 확인하는 좌표확인부(60);A coordinate confirmation unit 60 that checks the extracted coordinates of emphysema;
상기 좌표가 확인 된 폐기종을 감지하는 폐기종감지부(70);An emphysema detection unit 70 that detects emphysema whose coordinates have been confirmed;
상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종계산부(80);An emphysema calculator 80 that extracts the detected emphysema and calculates the area;
상기 추출된 폐기종의 폐기종율을 정량화하는 폐기종정량부(90);를 포함하는 것을 특징으로 한다. It is characterized in that it includes an emphysema quantification unit 90 that quantifies the emphysema rate of the extracted emphysema.
(4) 폐기종 정량화 방법(4) Emphysema quantification method
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법은,The method for quantifying emphysema in lung samples using computer vision and machine learning according to the present invention,
이미지획득부(10)가 폐를 생검한 현미경 촬영을 통해 폐이미지를 획득하는 이미지 획득 단계;An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계;A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
이진화처리부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리하는 이진화 처리 단계;A binarization processing step in which the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed;
공기층계산부(40)가 상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산하는 공기층 계산 단계;An air layer calculation step in which the air layer calculation unit 40 calculates the area of the entire air layer in the binarized lung image;
폐포제거부(50)가 상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출하는 폐포 제거 단계;An alveolar removal step in which the alveolar removal unit 50 removes the air layer of the alveoli and extracts emphysema from the binarized lung image;
좌표확인부(60)가 상기 추출된 폐기종의 좌표를 확인하는 좌표 확인 단계;A coordinate confirmation step in which the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema;
폐기종감지부(70)가 상기 좌표가 확인 된 폐기종을 감지하는 폐기종 감지 단계;An emphysema detection step in which the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed;
폐기종계산부(80)가 상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종 계산 단계;An emphysema calculation step in which the emphysema calculation unit 80 extracts the detected emphysema and calculates an area;
폐기종정량부(90)가 상기 추출된 폐기종의 폐기종율을 정량화하는 폐기종 정량 단계;를 포함하는 것을 특징으로 한다.It is characterized in that it includes an emphysema quantification step in which the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
상기 과제의 해결 수단에 의해, 본 발명은 머신러닝을 이용하여 폐 표본의 폐질환의 절대 기준에 따른 정량화 방법을 제시할 수 있다.As a means of solving the above problem, the present invention can present a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
또한, 본 발명은 폐 표본에서 폐질환의 절대 정량화를 기반으로 인간의 경험과 직관에 기반 한 정량화 결정을 수정할 수 있는 시스템을 제공한다.Additionally, the present invention provides a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
도 1은 본 발명인 폐렴 정량화 시스템의 구성도이다. Figure 1 is a schematic diagram of the pneumonia quantification system of the present invention.
도 2는 본 발명인 폐렴 정량화 방법의 순서도이다.Figure 2 is a flowchart of the present invention's pneumonia quantification method.
도 3은 본 발명의 일실시예에 따라 이미지 획득 단계(S10)에서 폐 표본을 획득하기 위한 생검 방법을 나타낸 도면이다.Figure 3 is a diagram showing a biopsy method for obtaining a lung specimen in the image acquisition step (S10) according to an embodiment of the present invention.
도 4는 본 발명의 일실시예에 따라 도 3에서 질병 특성을 확인한 도면이다.Figure 4 is a diagram confirming the disease characteristics in Figure 3 according to an embodiment of the present invention.
도 5는 본 발명의 일실시예에 따라 분할이미지 생성 단계(S40)에서 생성된 분할이미지이다.Figure 5 is a split image generated in the split image generation step (S40) according to an embodiment of the present invention.
도 6은 본 발명의 일실시예에 따라 세포핵인식부(50) 및 세포핵집계부(60)가 세포핵 인식 단계(S50) 및 세포핵 집계 단계(S60)를 수행하는 이미지이다.Figure 6 is an image showing the cell nucleus recognition unit 50 and the cell nucleus counting unit 60 performing the cell nucleus recognition step (S50) and the cell nucleus counting step (S60) according to an embodiment of the present invention.
도 7은 본 발명의 일실시예에 따라 폐렴판단부(90)가 폐렴 판단 단계(S90)을 수행하는 이미지이다.Figure 7 is an image of the pneumonia determination unit 90 performing the pneumonia determination step (S90) according to an embodiment of the present invention.
도 8은 본 발명의 일실시예에 따라 단위 면적당 핵 수 분포도를 나타낸 그래프로, (a) 의사가 30%로 결정하는 표본 a이고, (b) 의사가 90%로 결정하는 표본 b이다.Figure 8 is a graph showing the distribution of the number of nuclei per unit area according to an embodiment of the present invention, (a) sample a, which the doctor determines to be 30%, and (b) sample b, which the doctor determines to be 90%.
도 9는 본 발명의 일실시예에 따라 전문가 1과 2의 샘플에 대한 폐렴의 비율 분포에 관한 것으로, (a) 단위면적에 40개 이상의 핵을 포함한 패치를 폐렴으로 판단 경우, (b) 단위면적에 50개 이상의 핵을 포함한 패치를 폐렴으로 판단한 경우, (c) 단위면적에 60개 이상의 핵을 포함한 패치를 폐렴으로 판단한 경우이다.Figure 9 relates to the distribution of the proportion of pneumonia in the samples of experts 1 and 2 according to an embodiment of the present invention, (a) when a patch containing 40 or more nuclei in a unit area is judged to be pneumonia, (b) unit (c) A patch containing more than 50 nuclei per unit area was judged to be pneumonia, and (c) a patch containing more than 60 nuclei per unit area was judged to be pneumonia.
도 10은 본 발명의 일실시예에 따라 도 9의 각 샘플에 대해 정량한 결과를 바탕으로 폐렴 발병률을 정량화 한 값이다.Figure 10 is a quantified value of the pneumonia incidence rate based on the results quantified for each sample in Figure 9 according to an embodiment of the present invention.
도 11은 본 발명의 일실시예에 따라 본 발명을 통해 수행한 폐렴 발병률 정량화 결과와 전문가1(a) 및 전문가 2(b)를 통해 수행한 정량화 결과의 편차를 나타낸 것이다. Figure 11 shows the deviation between the pneumonia incidence rate quantification results performed through the present invention and the quantification results performed through Expert 1 (a) and Expert 2 (b) according to an embodiment of the present invention.
도 12는 본 발명의 일실시예에 따라 전문가 1에 의해 정량화된 폐렴 비율의 결과이다.Figure 12 shows the results of pneumonia rates quantified by expert 1 according to an embodiment of the present invention.
도 13은 본 발명의 일실시예에 따라 전문가 2에 의해 정량화된 폐렴 비율의 결과이다.Figure 13 shows the results of pneumonia rates quantified by expert 2 according to an embodiment of the present invention.
도 14는 본 발명의 일실시예에 따라 본 발명을 통해 수행한 폐렴 발병률 정량 결과(50 단위 면적 이상)와 전문가1(a) 및 전문가 2(b)를 통해 수행한 정량 결과 편차이다.Figure 14 shows the quantitative results of pneumonia incidence (more than 50 unit area) performed through the present invention according to an embodiment of the present invention and the deviation of the quantitative results performed through Expert 1 (a) and Expert 2 (b).
도 15는 본 발명인 폐기종 정량화 시스템의 구성도이다. Figure 15 is a configuration diagram of the emphysema quantification system of the present invention.
도 16은 본 발명인 폐기종 정량화 시스템의 보다 구체적인 구성도이다. Figure 16 is a more detailed configuration diagram of the emphysema quantification system of the present invention.
도 17은 본 발명인 폐기종 정량화 방법의 순서도이다.Figure 17 is a flowchart of the emphysema quantification method of the present invention.
도 18은 본 발명인 폐기종 정량화 방법의 보다 구체적인 순서도이다.Figure 18 is a more detailed flowchart of the emphysema quantification method of the present invention.
도 19는 본 발명의 일실시예에 따라 이미지 획득 단계(S10)에서 폐 표본을 획득하기 위한 생검 방법을 나타낸 도면이다.Figure 19 is a diagram showing a biopsy method for obtaining a lung specimen in the image acquisition step (S10) according to an embodiment of the present invention.
도 20은 본 발명의 일실시예에 따라 도 17에서 질병 특성을 확인한 도면이다.Figure 20 is a diagram confirming the disease characteristics in Figure 17 according to an embodiment of the present invention.
도 21은 본 발명의 일실시예에 따라 제3단계(S30) 이진화 처리 단계에서 이진화처리부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지를 단계별로 이진화(binarization) 처리하는 것을 나타낸 도면이다.Figure 21 is a diagram showing the binarization processing unit 30 step-by-step binarization processing of the lung image from which the bronchioles have been removed in the third step (S30) binarization processing step according to an embodiment of the present invention. .
도 22은 본 발명의 일실시예에 따라 실험 검증에서 도 21의 15 단계(Steps) 중에서 step 12(a) 내지 step 15(d)일 때 폐기종 비유에 대한 전문가 1과의 결과 비교를 나타낸 그래프이다.Figure 22 is a graph showing the comparison of results with expert 1 for the emphysema analogy when steps 12 (a) to 15 (d) among the 15 steps (Steps) of Figure 21 in experimental verification according to an embodiment of the present invention. .
도 23은 본 발명의 일실시예에 따라 실험 검증에서 도 21의 15 단계(Steps) 중에서 step 12(a) 내지 step 15(d)일 때 폐기종 비유에 대한 전문가 2와의 결과 비교를 나타낸 그래프이다.Figure 23 is a graph showing the comparison of results with expert 2 for the emphysema analogy when steps 12 (a) to 15 (d) among the 15 steps (Steps) of Figure 21 in experimental verification according to an embodiment of the present invention.
도 24은 본 발명의 일실시예에 따라 도 21의 15 단계(Steps) 중에서 step 12 내지 step 15일 때, 전문가 1(Expert 1) 및 전문가 2(Expert 2)를 통해 수행한 정량화 결과의 편차를 나타낸 것이다. Figure 24 shows the deviation of the quantification results performed by Expert 1 and Expert 2 at steps 12 to 15 among the 15 steps of Figure 21 according to an embodiment of the present invention. It is shown.
도 25은 본 발명의 일실시예에 따라 도 21의 15 단계(Steps) 중에서 step 12 내지 step 15일 때, 본 발명을 통해 수행한 폐기종 발병률 정량화 결과와 전문가1(a) 및 전문가 2(b)를 통해 수행한 정량화 결과의 편차를 나타낸 것이다. Figure 25 shows the results of emphysema incidence rate quantification performed through the present invention at step 12 to step 15 among the 15 steps (Steps) of Figure 21 according to an embodiment of the present invention, and expert 1 (a) and expert 2 (b) This shows the deviation of the quantification results performed through .
도 26는 본 발명의 일실시예에 따라 도 21의 15 단계(Steps) 중에서 step 12 내지 step 15일 때, 전문가 1에 의한 폐기종 정량화의 기계 측정 결과이다.Figure 26 is a mechanical measurement result of emphysema quantification by expert 1 when steps 12 to 15 of the 15 steps (Steps) of Figure 21 are performed according to an embodiment of the present invention.
도 27은 본 발명의 일실시예에 따라 도 21의 15 단계(Steps) 중에서 step 12 내지 step 15일 때, 전문가 2에 의한 폐기종 정량화의 기계 측정 결과이다.Figure 27 is a mechanical measurement result of emphysema quantification by expert 2 at steps 12 to 15 among the 15 steps (Steps) of Figure 21 according to an embodiment of the present invention.
본 명세서에서 사용되는 용어에 대해 간략히 설명하고, 본 발명에 대해 구체적으로 설명하기로 한다.The terms used in this specification will be briefly explained, and the present invention will be described in detail.
본 발명에서 사용되는 용어는 본 발명에서의 기능을 고려하면서 가능한 현재 널리 사용되는 일반적인 용어들을 선택하였으나, 이는 당 분야에 종사하는 기술자의 의도 또는 판례, 새로운 기술의 출현 등에 따라 달라질 수 있다. 따라서 본 발명에서 사용되는 용어는 단순한 용어의 명칭이 아닌, 그 용어가 가지는 의미와 본 발명의 전반에 걸친 내용을 토대로 정의되어야 한다.The terms used in the present invention are general terms that are currently widely used as much as possible while considering the functions in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.
명세서 전체에서 어떤 부분이 어떤 구성요소를 “포함”한다고 할 때, 이는 특별히 반대되는 기재가 없는 한 다른 구성요소를 제외하는 것이 아니라 다른 구성요소를 더 포함할 수 있음을 의미한다.When it is said that a part “includes” a certain element throughout the specification, this means that it does not exclude other elements, but may further include other elements, unless specifically stated to the contrary.
아래에서는 첨부한 도면을 참고하여 본 발명의 실시예에 대하여 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자가 용이하게 실시할 수 있도록 상세히 설명한다. 그러나 본 발명은 여러 가지 상이한 형태로 구현될 수 있으며 여기에서 설명하는 실시예에 한정되지 않는다.Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein.
본 발명에 대한 해결하고자 하는 과제, 과제의 해결 수단, 발명의 효과를 포함한 구체적인 사항들은 다음에 기재할 실시 예 및 도면들에 포함되어 있다. 본 발명의 이점 및 특징, 그리고 그것들을 달성하는 방법은 첨부되는 도면과 함께 상세하게 후술되어 있는 실시 예들을 참조하면 명확해질 것이다.Specific details, including the problem to be solved by the present invention, the means for solving the problem, and the effect of the invention, are included in the examples and drawings described below. The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings.
폐렴과 폐기종은 흉부 X선이나 CT를 통해 정성적, 정량적으로 평가할 수 있다. 그러나 미세먼지 속 미세 중금속이 폐에 미치는 미시적인 영향을 X선과 CT를 통해 분석하는 것은 어렵다. 즉, 미세 중금속과 폐질환의 상관관계를 거시적 관점에서 분석하기는 어렵다. 따라서, 본 발명은 마우스의 폐 샘플에서 폐렴의 비율을 정량화하는 방법을 제안했다. 본 발명에 따른 방법은 전문가의 경험과 직관에 근거한 정성적 판단을 반영하였고, 모든 검체에 동일한 정량 방법을 적용하여 폐렴과 폐기종을 절대적으로 정량화하였다.Pneumonia and emphysema can be evaluated qualitatively and quantitatively through chest X-ray or CT. However, it is difficult to analyze the microscopic effects of fine heavy metals in fine dust on the lungs through X-rays and CT. In other words, it is difficult to analyze the correlation between fine heavy metals and lung disease from a macroscopic perspective. Therefore, the present invention proposed a method to quantify the rate of pneumonia in mouse lung samples. The method according to the present invention reflected qualitative judgment based on the expert's experience and intuition, and applied the same quantitative method to all samples to absolutely quantify pneumonia and emphysema.
본 발명은 폐렴과 폐기종을 정량화하기 위해 전문가의 판단에 따라 정성적 기준을 결정했다. 또한 정성적 기준에 따라 정량화된 방법의 정확성을 검증하기 위해 두 전문가의 판단에 따라 검증을 진행하였다.The present invention determined qualitative standards based on expert judgment to quantify pneumonia and emphysema. In addition, in order to verify the accuracy of the quantified method according to qualitative standards, verification was conducted according to the judgment of two experts.
우선, 폐렴은 세포의 핵 수를 확인하고 핵의 밀도에 따라 정량화했다. 두 전문가가 시료에 대해 정량한 결과와 밀도별로 정량한 결과를 비교할 때 전문가의 판단과 가장 유사하게 정량한 밀도는 단위면적당 핵수가 50개 이상일 때이다. 따라서 이를 바탕으로 정량화 기준을 설정하고 전문가의 판단과 비교분석을 진행하였다. 두 전문가가 폐렴의 비율을 중간 비율(45~70)로 판단했을 때 두 전문가의 차이는 작고 안정적이었다. 한편, 낮은 비율(0~45)의 정량화는 핵의 밀도가 높은 경우 폐렴으로 판단되는 것으로 나타났고, 높은 비율(70~90)의 정량화는 핵의 밀도가 낮은 경우 폐렴으로 판단했다.First, pneumonia was determined by the number of nuclei in the cells and quantified according to the density of the nuclei. When comparing the quantitative results of a sample by two experts and the quantitative results by density, the density quantified most similar to the expert's judgment is when the number of nuclei per unit area is 50 or more. Therefore, based on this, quantification standards were set and expert judgment and comparative analysis were conducted. When the two experts judged the pneumonia rate to be an intermediate rate (45 to 70), the difference between the two experts was small and stable. Meanwhile, quantification at a low ratio (0 to 45) showed that cases with high nuclear density were judged to be pneumonia, while quantification at high ratios (70 to 90) showed that cases with low nuclear density were judged to be pneumonia.
다음으로, 폐기종은 침식(erode) 단계에 따라 정상적인 폐포를 제거하고, 폐기종 부위만 검출하여 정량화하였다. 정량화 기준은 우리가 탐지한 결과에 대한 전문가의 정성적 판단에 따라 결정되었다. 그 결과, 정상적인 폐포 제거 단계에서 침식 step 14와 step 12에서 전문가 1의 정량화 결과와 편차가 가장 낮았다. 그러나 두 명의 전문가가 낮은 비율(0~10)로 정량화한 결과가 정량화된 결과와 가장 작은 편차를 보였다. step 15에서는 비율이 높을수록 step 12에서 정량화한 결과와의 편차가 작아졌다. 이는 샘플에 대해 폐기종의 정도가 높게 정량화되어 있어 작은 공기층도 폐기종이라고 두 전문가가 판단한 것으로 나타났다. 또한, 폐기종의 정도가 높을수록 우리와의 편차가 커져 전문가들이 높은 비율로 불안정한 정량을 수행함을 알 수 있었다.Next, emphysema was quantified by removing normal alveoli according to the erode stage and detecting only the emphysema area. The quantification criteria were determined based on the qualitative judgment of experts regarding the results we detected. As a result, the deviation from expert 1's quantification results was lowest at erosion steps 14 and 12 in the normal alveolar removal stage. However, the results quantified by two experts at a low ratio (0 to 10) showed the smallest deviation from the quantified results. In step 15, the higher the ratio, the smaller the deviation from the results quantified in step 12. This showed that the two experts judged that even a small air gap was emphysema because the degree of emphysema in the sample was highly quantified. In addition, the higher the degree of emphysema, the greater the deviation from our results, showing that experts performed unstable quantification at a high rate.
본 발명에서 제안하는 정량화 방법은 전문가의 정량화 판단을 분석할 수 있다. 폐렴 및 폐기종을 정량화할 때 전문가의 판단 경향과 전문가 간의 편차를 파악할 수 있었다. 그리고 전문가들은 병리학의 절대적 기준에 따라 수치화했다고 하지만 전문가들의 시각적 경험과 직관이 반영된 것으로 나타났다.The quantification method proposed in the present invention can analyze the quantification judgment of experts. When quantifying pneumonia and emphysema, it was possible to identify experts' judgment trends and differences between experts. And although experts said they quantified it according to the absolute standards of pathology, it appeared to reflect the experts' visual experience and intuition.
이 발명은 기계 학습과 고전적인 컴퓨터 비전 기술을 사용하여 폐 샘플에서 폐렴 및 폐기종의 절대 정량화 방법을 제안했다. 전문가들이 개발된 방법을 기반으로 각 샘플에 대해 폐렴과 폐기종을 정량화하면 결과가 달라질 수 있음을 보여주었다. 또한 표본 내 폐렴과 폐기종의 정도에 따라 전문가의 정성적 판단도 달라질 수 있음을 보여주었다. 따라서 당사의 정량화 방법은 정량화의 기준이 될 수 있으며, 전문가가 정량적으로 평가할 때 전문가의 정량적 평가 결과를 수정하는 도구로 사용될 수 있다.This invention proposed a method for absolute quantification of pneumonia and emphysema in lung samples using machine learning and classical computer vision techniques. They showed that quantifying pneumonia and emphysema for each sample based on methods developed by experts can lead to different results. It also showed that the expert's qualitative judgment could vary depending on the degree of pneumonia and emphysema in the sample. Therefore, our quantification method can be a standard for quantification and can be used as a tool to correct the quantitative evaluation results of experts when they quantitatively evaluate the results.
이하, 첨부된 도면을 참조하여 본 발명을 보다 상세히 설명하기로 한다. Hereinafter, the present invention will be described in more detail with reference to the attached drawings.
(1) 폐렴 정량화 시스템(1) Pneumonia quantification system
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템은, 도 1에 나타난 바와 같이, 이미지획득부(10), 세기관지제거부(20), 폐간질추출부(30), 분할이미지생성부(40), 세포핵인식부(50), 세포핵집계부(60), 분포도생성부(70), 임계값비교부(80), 폐렴판단부(90), 면적계산부(100) 및 폐렴율산출부(110)를 포함하는 것을 특징으로 한다. As shown in Figure 1, the pneumonia quantification system for lung specimens using computer vision and machine learning according to the present invention includes an image acquisition unit (10), a bronchioles removal unit (20), a lung interstitial extraction unit (30), and a segmented image. Generation unit 40, cell nucleus recognition unit 50, cell nucleus counting unit 60, distribution map generation unit 70, threshold comparison unit 80, pneumonia determination unit 90, area calculation unit 100, and pneumonia It is characterized by including a rate calculation unit 110.
먼저, 상기 이미지획득부(10)는 폐를 생검하여 현미경 촬영을 통해 폐이미지 획득한다. First, the image acquisition unit 10 biopsies the lung and acquires an image of the lung through microscopic imaging.
도 3에 나타난 바와 같이, 상기 이미지획득부(10)는 니켈, 크롬 망간 및 카드뮴 등의 다양한 중금속에 노출된 폐 생검 표분을 획득한다. 하나의 폐이미지는 하나의 슬라이드를 나타내며, 하나의 슬라이드는 2 내지 4개의 섹션으로 구성된다. As shown in FIG. 3, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
다음으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거한다. Next, the bronchiole removal unit 20 recognizes the bronchiole in the lung image and removes the bronchiole.
보다 구체적으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식하여 상기 세기관지(bronchiole)를 제거하는 학습을 위해 세기관지를 레이블링(labeling)해야 한다. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
상기 세기관지제거부(20)는 상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 한다. 상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중, 일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용한다. 상기 세기관지제거부(20)는 상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성한다.The bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing. The bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
일실시예로, 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링을 수행하고 40개는 트레인 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다. 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류되고, 파티션 이미지를 모아서 다시 하나의 이미지로 만든다.In one embodiment, because the size of the bronchioles is relatively small compared to the entire section, a 5x5 partition is created and the bronchioles are labeled directly within this partition. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
다음으로, 상기 폐간질추출부(30)는 상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출한다. Next, the lung interstitium extraction unit 30 extracts the lung interstitium area from the lung image from which the bronchiole has been removed.
보다 구체적으로, 상기 폐간질추출부(30)는 상기 폐간질(interstitium) 영역을 동일한 색상의 픽셀로 레이블링(labeling) 한 후 상기 레이블링(labeling) 된 픽셀 영역을 추출한다.More specifically, the lung interstitium extraction unit 30 labels the lung interstitium area with pixels of the same color and then extracts the labeled pixel area.
일실시예로, 컴퓨터 비전 기술 중 하나인 OpenCV의 connected Components 를 사용하여 연결된 상기 폐간질영역을 동일한 색상의 픽셀로 레이블링하고 이 레이블의 픽셀 영역을 추출한다. In one embodiment, the connected pulmonary interstitial area is labeled with pixels of the same color using connected components of OpenCV, one of the computer vision technologies, and the pixel area of this label is extracted.
다음으로, 상기 분할이미지생성부(40)는 상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성한다.Next, the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and generates a segmented image.
보다 구체적으로, 상기 분할이미지생성부(40)는 상기 분할이미지의 너비와 길이가 동일하도록 마련한다.More specifically, the split image generator 40 prepares the split images to have the same width and length.
일실시예로, 표본을 분할하여 핵을 시각적으로 식별이 용이하도록 한다. image_slicer를 사용하여 표본을 각각 82 개 및 72 개 파티션으로 나누어 100x100 크기의 파티션으로 나눈다.In one embodiment, the sample is divided to facilitate visual identification of the nucleus. Using image_slicer, the sample is divided into 82 and 72 partitions, respectively, with size 100x100.
다음으로, 상기 세포핵인식부(50)는 상기 분할이미지에서 세포핵(cell nucleus)을 인식한다.Next, the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
일실시예로, YOLOv5를 사용하여 상기 세포핵을 인식할 수 있다. In one embodiment, the cell nucleus can be recognized using YOLOv5.
YOLOv5 학습의 경우 파티션에 검은색 핵이 캡처되고 레이블 데이터는 모두 동일한 클래스로 만들어진다. 고화질 표본 이미지에서 핵을 검출하기 위해 한 섹션을 여러 파티션으로 나누어 핵을 검출한다. 하나의 파티션은 너비와 길이가 동일한 82 개와 72 개의 파티션으로 나뉘며 너비는 100픽셀, 그 길이는 100픽셀이다. 총 파티션 수는 1,399,248 개로 그 중 100 개의 파티션에 레이블이 지정되었으며 학습 데이터는 90개, 테스트 데이터는 10개로 구성되어 YOLOv5 학습을 진행한다.For YOLOv5 learning, black nuclei are captured in the partition and the labeled data are all made of the same class. To detect nuclei in high-quality specimen images, one section is divided into several partitions to detect nuclei. One partition is divided into 82 and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels. The total number of partitions is 1,399,248, of which 100 partitions are labeled, 90 training data and 10 test data are used to train YOLOv5.
다음으로, 상기 세포핵집계부(60)는 상기 인식된 세포핵(cell nucleus)의 개수를 집계한다. 상기 세포핵집계부(60)는 상기 학습을 통해 상기 분할이미지에서 인식된 상기 세포핵(cell nucleus)의 개수를 집계한다. Next, the cell nucleus counting unit 60 counts the number of recognized cell nuclei. The cell nucleus counting unit 60 counts the number of cell nuclei recognized in the segmented image through the learning.
다음으로, 상기 분포도생성부(70)는 상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus) 분포도를 생성한다.Next, the distribution map generator 70 generates a cell nucleus distribution map by visualizing the lung image according to the counted number of cell nuclei.
보다 구체적으로, 상기 분포도생성부(70)는 폐렴이 의심되는 모든 분할이미지를 선택하고, 도 8과 같이 상기 분할이미지에 의한 상기 세포핵의 분포도를 생성한다.More specifically, the distribution map generator 70 selects all segmented images suspected of having pneumonia and generates a distribution map of the cell nuclei based on the segmented images, as shown in FIG. 8.
다음으로, 상기 임계값비교부(80)는 상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교한다.Next, the threshold comparison unit 80 compares the counted number of cell nuclei and the threshold.
일반적으로, 전문가들이 사용하는 방법은 단위 면적당 세포핵의 수를 계산하여 전체 비율을 계산하였다. 본 발명에서는 각 분할이미지의 크기로 단위면적을 사용하였다. 여기서 중요한 점은 단위면적당 세포핵의 개수가 폐렴의 유무를 판별하는 기준이 된다는 점이다. 그러나 전통적으로 폐렴을 결정하는 단위 면적의 세포 밀도에 대한 기준은 없다. 따라서 본 발명에서는 사용하는 단위 면적에 다양한 밀도를 전문가의 판단과 비교하여 적용하여 가장 적절한 폐렴을 결정하는 단위 면적당 세포핵의 수를 결정하였다.Generally, the method used by experts calculates the overall ratio by calculating the number of cell nuclei per unit area. In the present invention, unit area was used as the size of each segmented image. The important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
다음으로, 상기 폐렴판단부(90)는 상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단한다.Next, the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is high compared to the threshold.
일실시예로, 도 8은 단위 면적당 핵 수 분포도의 예를 보여준다. 이 분포도는 가우스 분포를 따르며, 이 분포도만으로는 이상치, 즉 폐렴을 특정하는 단위 면적당 핵 수를 지정할 수 없다. 따라서 전문가들이 정량화한 폐렴의 비율에 따라 각 파티션에 존재하는 핵의 수를 분석하였다. 예를 들어 전문가가 폐렴을 30% 라고 하면 위의 분포도에서 상위 30%를 차지하는 단위면적당 최소 핵 수를 찾아 폐렴을 특정하는 단위면적당 핵 수를 찾아냈다. 따라서 전문가의 경험과 직관을 바탕으로 정량화를 진행하기 위해서는 한 파티션에 40개 이상, 50개 이상, 60개 이상의 핵이 있을 때 폐렴의 정도를 정량화하는 것이 전문가의 것과 가장 유사함을 확인하였다. 경험과 직관에 따라서 40개, 50개, 60개 이상의 핵이 농축된 구획에 대해 발효도를 계산하였다.In one embodiment, Figure 8 shows an example of the distribution of the number of nuclei per unit area. This distribution diagram follows a Gaussian distribution, and this distribution diagram alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each partition was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found. Therefore, in order to perform quantification based on the expert's experience and intuition, it was confirmed that quantifying the degree of pneumonia when there are more than 40, more than 50, or more than 60 nuclei in one partition is most similar to that of the expert. According to experience and intuition, the degree of fermentation was calculated for compartments in which more than 40, 50, and 60 nuclei were concentrated.
다음으로, 상기 면적계산부(100)는 상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산한다.Next, the area calculation unit 100 calculates the lung interstitium area in the segmented image determined to be pneumonia.
*보다 구체적으로, 상기 면적계산부(100)는 분할이미지에서 폐간질(interstitium) 영역을 픽셀 단위로 조밀한 핵으로 계산한다.*More specifically, the area calculation unit 100 calculates the lung interstitium area in the segmented image as a dense nucleus on a pixel basis.
다음으로, 상기 폐렴율산출부(110)는 폐의 폐렴율을 산출한다.Next, the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lung.
보다 구체적으로, 상기 폐렴율산출부(110)는 아래의 [식 1]에 의해 상기 폐렴율을 산출한다.More specifically, the pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below.
[식 1][Equation 1]
폐렴율 = AP / AINT Pneumonia rate = A P / A INT
(여기서, AP 는 상기 면적계산부(100)에서 계산된 폐간질 영역의 면적이고, AINT 는 폐간질추출부(30)에서 추출된 폐간질 영역의 면적).(Here, A P is the area of the pulmonary interstitial area calculated by the area calculation unit 100, and A INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30).
(2) 폐렴 정량화 방법(2) Pneumonia quantification method
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법은, 도 2에 나타난 바와 같이, 상기 폐렴 정량화 시스템(1)을 이용하여 폐렴을 정량화한다. As shown in FIG. 2, the method for quantifying pneumonia in lung specimens using computer vision and machine learning according to the present invention quantifies pneumonia using the pneumonia quantification system (1).
먼저, 제1단계(S10)는 이미지 획득 단계이다.First, the first step (S10) is the image acquisition step.
상기 제1단계(S10)는 상기 이미지획득부(10)가 폐를 생검하여 현미경 촬영을 통해 폐이미지를 획득한다. 도 3에 나타난 바와 같이, 상기 이미지획득부(10)는 니켈, 크롬 망간 및 카드뮴 등의 다양한 중금속에 노출된 폐 생검 표분을 획득한다. 하나의 폐이미지는 하나의 슬라이드를 나타내며, 하나의 슬라이드는 2 내지 4개의 섹션으로 구성된다. In the first step (S10), the image acquisition unit 10 biopsies the lung and acquires a lung image through microscopic imaging. As shown in FIG. 3, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
다음으로, 제2단계(S20)는 세기관지 제거 단계이다.Next, the second step (S20) is the bronchioles removal step.
상기 제2단계(S20)는 상기 세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거한다. 보다 구체적으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식하여 상기 세기관지(bronchiole)를 제거하는 학습을 위해 세기관지를 레이블링(labeling)해야 한다. In the second step (S20), the bronchioles removal unit 20 recognizes the bronchioles in the lung image and then removes the bronchioles. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
상기 세기관지제거부(20)는 상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 한다. 상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중, 일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용한다. 상기 세기관지제거부(20)는 상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성한다.The bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing. The bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
일실시예로, 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링을 수행하고 40개는 훈련 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다. 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류되고, 파티션 이미지를 모아서 다시 하나의 이미지로 만든다.In one embodiment, because the size of the bronchioles is relatively small compared to the entire section, a 5x5 partition is created and the bronchioles are labeled directly within this partition. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as training data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
다음으로, 제3단계(S30)는 폐간질 추출 단계이다.Next, the third step (S30) is the pulmonary interstitium extraction step.
상기 제3단계(S30)는 상기 폐간질추출부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출한다. 보다 구체적으로, 상기 폐간질추출부(30)는 상기 폐간질(interstitium) 영역을 동일한 색상의 픽셀로 레이블링(labeling) 한 후 상기 레이블링(labeling) 된 픽셀 영역을 추출한다.In the third step (S30), the lung interstitium extraction unit 30 extracts the lung interstitium area from the lung image from which the bronchiole has been removed. More specifically, the lung interstitium extraction unit 30 labels the lung interstitium area with pixels of the same color and then extracts the labeled pixel area.
일실시예로, 컴퓨터 비전 기술 중 하나인 OpenCV의 connected Components 를 사용하여 연결된 상기 폐간질영역을 동일한 색상의 픽셀로 레이블링하고 이 레이블의 픽셀 영역을 추출한다. In one embodiment, the connected pulmonary interstitial area is labeled with pixels of the same color using connected components of OpenCV, one of the computer vision technologies, and the pixel area of this label is extracted.
다음으로, 제4단계(S40)는 분할이미지 생성 단계이다.Next, the fourth step (S40) is a segmented image generation step.
상기 제4단계(S40)는 상기 분할이미지생성부(40)가 상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성한다. 보다 구체적으로, 상기 분할이미지생성부(40)는 상기 분할이미지의 너비와 길이가 동일하도록 마련한다.In the fourth step (S40), the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and generates a segmented image. More specifically, the split image generator 40 prepares the split images to have the same width and length.
일실시예로, 표본을 분할하여 핵을 시각적으로 식별이 용이하도록 한다. image_slicer를 사용하여 표본을 각각 82 개 및 72 개 파티션으로 나누어 100x100 크기의 파티션으로 나눈다.In one embodiment, the sample is divided to facilitate visual identification of the nucleus. Using image_slicer, the sample is divided into 82 and 72 partitions, respectively, with size 100x100.
다음으로, 제5단계(S50)는 세포핵 인식 단계이다.Next, the fifth step (S50) is the cell nucleus recognition step.
상기 제5단계(S50)는 상기 세포핵인식부(50)가 상기 분할이미지에서 세포핵(cell nucleus)을 인식한다. In the fifth step (S50), the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
일실시예로, YOLOv5를 사용하여 상기 세포핵을 인식할 수 있다. YOLOv5 학습의 경우 파티션에 검은색 핵이 캡처되고 레이블 데이터는 모두 동일한 클래스로 만들어진다. 고화질 표본 이미지에서 핵을 검출하기 위해 한 섹션을 여러 파티션으로 나누어 핵을 검출한다. 하나의 파티션은 너비와 길이가 동일한 크기의 82개 및 72개의 파티션으로 나뉘며 너비는 100픽셀, 그 길이는 100픽셀이다. 총 파티션 수는 1,399,248 개로 100 개의 파티션에 레이블이 지정되었으며 학습 데이터는 90개, 테스트 데이터는 10개로 구성되어 YOLOv5 학습을 진행한다.In one embodiment, the cell nucleus can be recognized using YOLOv5. For YOLOv5 learning, black nuclei are captured in the partition and the labeled data are all made of the same class. To detect nuclei in high-quality specimen images, one section is divided into several partitions to detect nuclei. One partition is divided into 82 and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels. The total number of partitions is 1,399,248, 100 partitions are labeled, 90 training data and 10 test data are used to train YOLOv5.
다음으로, 제6단계(S60)는 세포핵 집계 단계이다.Next, the sixth step (S60) is the cell nucleus counting step.
상기 제6단계(S60)는 상기 세포핵집계부(60)가 상기 인식된 세포핵(cell nucleus)의 개수를 집계한다. In the sixth step (S60), the cell nucleus counting unit 60 counts the number of recognized cell nuclei.
다음으로, 제7단계(S70)는 분포도 생성 단계이다.Next, the seventh step (S70) is the distribution map generation step.
상기 제7단계(S70)는 상기 분포도생성부(70)가 상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus)분포도를 생성한다. 보다 구체적으로, 상기 분포도생성부(70)는 폐렴이 의심되는 모든 분할이미지를 선택하고, 도 8과 같이 상기 분할이미지에 의한 상기 세포핵의 분포도를 생성한다.In the seventh step (S70), the distribution map generating unit 70 generates a cell nucleus distribution map by visualizing the lung image according to the counted number of cell nuclei. More specifically, the distribution map generator 70 selects all segmented images suspected of having pneumonia and generates a distribution map of the cell nuclei based on the segmented images, as shown in FIG. 8.
다음으로, 제8단계(S80)는 임계값 비교 단계이다.Next, the eighth step (S80) is a threshold comparison step.
상기 제8단계(S80)는 상기 임계값비교부(80)가 상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교한다. In the eighth step (S80), the threshold value comparison unit 80 compares the counted number of cell nuclei and the threshold value.
일반적으로, 전문가들이 사용하는 방법은 단위 면적당 세포핵의 수를 계산하여 전체 비율을 계산하였다. 본 발명에서는 각 분할이미지의 크기로 단위면적을 사용하였다. 여기서 중요한 점은 단위면적당 세포핵의 개수가 폐렴의 유무를 판별하는 기준이 된다는 점이다. 그러나 전통적으로 폐렴을 결정하는 단위 면적의 세포 밀도에 대한 기준은 없다. 따라서 본 발명에서는 사용하는 단위 면적에 다양한 밀도를 전문가의 판단과 비교하여 적용하여 가장 적절한 폐렴을 결정하는 단위 면적당 세포핵의 수를 결정하였다.Generally, the method used by experts calculates the overall ratio by calculating the number of cell nuclei per unit area. In the present invention, unit area was used as the size of each segmented image. The important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
다음으로, 제9단계(S90)는 폐렴 판단 단계이다.Next, the ninth step (S90) is the pneumonia determination step.
상기 제9단계(S90)는 상기 폐렴판단부(90)가 상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단한다. In the ninth step (S90), the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is greater than the threshold value.
일실시예로, 도 8은 단위 면적당 핵 수 분포도의 예를 보여준다. 이 분포도는 가우스 분포를 따르며, 이 분포도만으로는 이상치, 즉 폐렴을 특정하는 단위 면적당 핵 수를 지정할 수 없다. 따라서 전문가들이 정량화한 폐렴의 비율에 따라 각 파티션에 존재하는 핵의 수를 분석하였다. 예를 들어 전문가가 폐렴을 30% 라고 하면 위의 분포도에서 상위 30%를 차지하는 단위면적당 최소 핵 수를 찾아 폐렴을 특정하는 단위면적당 핵 수를 찾아냈다. 따라서 전문가의 경험과 직관을 바탕으로 정량화를 진행하기 위해서는 한 파티션에 40개 이상, 50개 이상, 60개 이상의 핵이 있을 때 폐렴의 정도를 정량화하는 것이 전문가의 것과 가장 유사함을 확인하였다. 경험과 직관에 따라서 40개, 50개, 60개 이상의 핵이 농축된 구획에 대해 발효도를 계산하였다.In one embodiment, Figure 8 shows an example of the distribution of the number of nuclei per unit area. This distribution diagram follows a Gaussian distribution, and this distribution diagram alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each partition was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found. Therefore, in order to perform quantification based on the expert's experience and intuition, it was confirmed that quantifying the degree of pneumonia when there are more than 40, more than 50, or more than 60 nuclei in one partition is most similar to that of the expert. According to experience and intuition, the degree of fermentation was calculated for compartments in which more than 40, 50, and 60 nuclei were concentrated.
다음으로, 제10단계(S10)는 면적 계산 단계이다.Next, the tenth step (S10) is the area calculation step.
상기 제10단계(S10)는 상기 면적계산부(100)가 상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산한다. 보다 구체적으로, 상기 면적계산부(100)는 분할이미지에서 폐간질(interstitium) 영역을 픽셀 단위로 조밀한 핵으로 계산한다.In the tenth step (S10), the area calculation unit 100 calculates the lung interstitium area in the segmented image determined to be pneumonia. More specifically, the area calculation unit 100 calculates the lung interstitium area in the segmented image as a dense nucleus on a pixel basis.
다음으로, 제11단계(S110)는 폐렴율 산출 단계이다. Next, the 11th step (S110) is the pneumonia rate calculation step.
상기 제11단계(S110)는 상기 폐렴율산출부(110)가 폐의 폐렴율을 산출한다. 보다 구체적으로, 상기 폐렴율산출부(110)는 아래의 [식 1]에 의해 상기 폐렴율을 산출한다.In the 11th step (S110), the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lung. More specifically, the pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below.
[식 1][Equation 1]
폐렴율 = AP / AINT Pneumonia rate = A P / A INT
(여기서, AP 는 상기 면적계산부(100)에서 계산된 폐간질 영역의 면적이고, AINT 는 폐간질추출부(30)에서 추출된 폐간질 영역의 면적).(Here, A P is the area of the pulmonary interstitial area calculated by the area calculation unit 100, and A INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30).
이하, 실시예를 통하여 본 발명을 보다 상세하게 설명한다. 본 발명의 목적, 특징, 장점은 이하의 실시예를 통하여 쉽게 이해될 것이다. 본 발명은 여기서 설명하는 실시예에 한정되지 않고, 다른 형태로 구체화될 수도 있다. 여기서 소개되는 실시예는 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자에게 본 발명의 사상이 충분히 전달될 수 있도록 하기 위해 제공되는 것이다. 따라서 이하의 실시예에 의해 본 발명이 제한되어서는 안 된다.Hereinafter, the present invention will be described in more detail through examples. The purpose, features, and advantages of the present invention will be easily understood through the following examples. The present invention is not limited to the embodiments described herein and may be embodied in other forms. The embodiments introduced here are provided to enable the idea of the present invention to be sufficiently conveyed to those skilled in the art to which the present invention pertains. Therefore, the present invention should not be limited by the following examples.
1) 생검 조직의 데이터 수집1) Data collection of biopsy tissue
7주령 C57BL/6 마우스를 실험 동물로 선택하여 마우스 폐 조직 샘플, 희석된 니켈, 크롬, 망간 및 카드뮴을 각각 50nM , 20nM , 10nM , 10nM 농도로 채취 하여 노출시켰다. 1일 1회 4주 동안 단독으로 복합적으로 비강을 통해 총 70개의 샘플을 채취하였다. 본 실험은 울산대학교 SPF(특정 병원체 무첨가)에서 진행되었으며, 울산대학교 동물발명소(IACUC)(IACUC No. BSK-21-030)의 심사를 받았다.Seven-week-old C57BL/6 mice were selected as experimental animals and exposed to mouse lung tissue samples and diluted nickel, chromium, manganese, and cadmium at concentrations of 50nM, 20nM, 10nM, and 10nM, respectively. A total of 70 samples were collected through the nasal cavity, alone or in combination, once a day for 4 weeks. This experiment was conducted at the University of Ulsan SPF (no specific pathogens added) and was reviewed by the Animal Invention Center of the University of Ulsan (IACUC) (IACUC No. BSK-21-030).
수집된 70개의 표본은 각 슬라이드로 구성되었다. 도 3은 하나의 슬라이드를 보여주고 하나의 슬라이드는 2-4개의 섹션으로 구성되어 있다. 따라서 총 70개의 슬라이드에서 표본을 추출하고 237장에서 단면 데이터를 수집했다. 이 중 실제 의사가 사용하는 섹션 70개만 사용했다. 한 섹션은 너비 8200, 길이 7200 크기의 고화질 이미지이다.The 70 specimens collected consisted of each slide. Figure 3 shows one slide, and one slide consists of 2-4 sections. Therefore, a total of 70 slides were sampled and cross-sectional data were collected from 237 slides. Of these, only 70 sections used by actual doctors were used. One section is a high-quality image measuring 8200 wide and 7200 long.
2) 폐렴 비율 측정2) Measurement of pneumonia rate
본 발명에서는 폐렴의 비율을 정량화하였으며, 검체를 통해 폐렴의 비율을 확인하는 방법은 다음과 같다. 비율은 공기통과층을 제외한 시편의 전체 간극 면적 대비 다수의 핵이 집중된 간질 영역의 면적 을 계산하여 정량화 한다.In the present invention, the rate of pneumonia was quantified, and the method for confirming the rate of pneumonia through a sample is as follows. The ratio is quantified by calculating the area of the interstitial area where many nuclei are concentrated compared to the total interstitial area of the specimen excluding the air passage layer.
폐렴의 비율을 계산하는 지표는 아래 [식 1]과 같다.The index for calculating the rate of pneumonia is as follows [Equation 1].
[식 1][Equation 1]
폐렴율 = AP / AINT Pneumonia rate = A P / A INT
(여기서, AP 는 상기 면적계산부(100)에서 계산된 폐간질 영역의 면적이고, AINT 는 폐간질추출부(30)에서 추출된 폐간질 영역의 면적).(Here, A P is the area of the pulmonary interstitial area calculated by the area calculation unit 100, and A INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30).
2-1) YOLACT를 사용하여 말단 폐포 조직 제거2-1) Distal alveolar tissue removal using YOLACT
YOLACT 학습에는 세기관지 레이블이 필요하다. 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440 이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링(labeling)을 수행하고 40개는 트레인 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다.YOLACT learning requires bronchioles labels. Because the size of the bronchioles is relatively small compared to the overall section, we divide them into 5x5 partitions and label the bronchioles directly within these partitions. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data.
YOLACT 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류된다.After learning YOLACT, the bronchial regions within every section are segmented and classified.
파티션 이미지를 모아서 다시 하나의 이미지로 만든다.Collect partition images and create a single image again.
2-2) 전체 조직공간의 픽셀 영역 계산 2-2) Calculation of pixel area of entire tissue space
컴퓨터 비전 기술 중 하나인 OpenCV의 connected Components를 사용하여 연결된 간극을 동일한 색상의 픽셀로 레이블링(labeling)하고, 이 레이블의 픽셀 영역을 추출하였다.Using OpenCV's connected components, one of the computer vision technologies, connected gaps were labeled with pixels of the same color, and the pixel area of these labels was extracted.
2-3) YOLOv5를 사용하여 핵 감지2-3) Nucleus detection using YOLOv5
YOLOv5 학습의 경우 분할이미지에 검은색 핵이 캡처되고 레이블 데이터는 모두 동일한 클래스로 만들어진다. 고화질 표본 이미지에서 핵을 검출하기 위해 한 섹션을 여러 분할로 나누어 핵을 검출한다. 하나의 분할이미지는 너비와 길이가 동일한 크기의 가로 82개 및 72개의 파티션으로 나뉘며 너비는 100 픽셀, 그 길이는 100 픽셀이다. 총 분할이미지 수는 1,399,248 개로 그 중 100 개의 분할이미지에 레이블이 지정되었으며 학습 데이터는 90 개, 테스트 데이터는 10 개로 구성되었다. 이후, YOLOv5 학습을 진행하여 각 분할이미지에 대해 감지된 핵의 수를 결정한다. In the case of YOLOv5 learning, black nuclei are captured in the segmented image, and all label data are made of the same class. To detect nuclei in high-quality specimen images, one section is divided into several divisions to detect nuclei. One segmented image is divided into 82 horizontal and 72 partitions of equal width and length, with a width of 100 pixels and a length of 100 pixels. The total number of segmented images was 1,399,248, of which 100 segmented images were labeled, with 90 training data and 10 test data. Afterwards, YOLOv5 learning is performed to determine the number of detected nuclei for each segmented image.
2-4) 폐렴이 의심되는 모든 분할이미지를 선택2-4) Select all segmented images suspected of pneumonia
전문가들이 사용하는 방법은 단위 면적당 세포핵의 수를 계산하여 전체 비율을 계산하는 것이다. 본 발명에서는 각 분할이미지의 크기로도 단위면적을 사용하였다. 여기서 중요한 점은 단위면적당 세포핵의 개수가 폐렴의 유무를 판별하는 기준이 된다는 점이다. 그러나 전통적으로 폐렴을 결정하는 단위 면적의 세포 밀도에 대한 기준은 없다. 따라서 본 발명에서는 사용하는 단위 면적에 다양한 밀도를 전문가의 판단과 비교하여 적용하여 가장 적절한 폐렴을 결정하는 단위 면적당 세포핵의 수를 결정하였다.The method used by experts is to calculate the overall ratio by counting the number of cell nuclei per unit area. In the present invention, the unit area is also used as the size of each divided image. The important point here is that the number of cell nuclei per unit area is the standard for determining the presence or absence of pneumonia. However, there is no standard for cell density per unit area that traditionally determines pneumonia. Therefore, in the present invention, the number of cell nuclei per unit area that determines the most appropriate pneumonia was determined by applying various densities to the unit area used and comparing them with expert judgment.
다음과 같은 방법을 사용하여 특정 폐렴에 가장 적합한 단위 면적당 세포 수를 구했다. 도 8은 단위 면적당 핵 수 분포도의 예를 보여준다. 이 분포도는 가우스 분포를 따르며, 이 분포도만으로는 이상치, 즉 폐렴을 특정 하는 단위 면적당 핵 수를 지정할 수 없다. 따라서 전문가들이 정량화한 폐렴의 비율에 따라 각 분할이미지에 존재하는 핵의 수를 분석하였다. 예를 들어 전문가가 폐렴을 30%라고 하면 위의 분포도에서 상위 30%를 차지하는 단위면적당 최소 핵 수를 찾아 폐렴을 특정 하는 단위면적당 핵 수를 찾아냈다. 따라서 전문가의 경험과 직관을 바탕으로 정량화를 진행하기 위해서는 한 분할이미지에 40개 이상, 50개 이상, 60개 이상의 핵이 있을 때 폐렴의 정도를 정량화하는 것이 전문가의 것과 가장 유사함을 확인하였다. 경험과 직관에 따라서 40개, 50개 및 60개 이상의 핵이 농축된 구획에 대해 발효도를 계산하였다.The number of cells per unit area most appropriate for a specific pneumonia was determined using the following method. Figure 8 shows an example of the distribution of the number of nuclei per unit area. This distribution chart follows a Gaussian distribution, and this distribution chart alone cannot specify the number of nuclei per unit area that identifies an outlier, that is, pneumonia. Therefore, the number of nuclei present in each segmented image was analyzed according to the proportion of pneumonia quantified by experts. For example, if an expert says that pneumonia is 30%, the minimum number of nuclei per unit area that accounts for the top 30% in the distribution chart above was found, and the number of nuclei per unit area that specifies pneumonia was found. Therefore, in order to proceed with quantification based on the expert's experience and intuition, it was confirmed that quantifying the degree of pneumonia was most similar to that of the expert when there were more than 40, more than 50, or more than 60 nuclei in one segmented image. According to experience and intuition, the degree of fermentation was calculated for compartments enriched with more than 40, 50, and 60 nuclei.
2-5) 폐렴의 비율 정량화2-5) Quantification of the rate of pneumonia
상기 면적계산부(100)에서 계산된 폐간질(interstitium) 영역의 면적과 폐간질추출부(30)에서 추출된 폐간질(interstitium) 영역의 면적을 픽셀 단위로 조밀한 핵으로 계산한다.The area of the lung interstitium area calculated by the area calculation unit 100 and the area of the lung interstitium area extracted by the lung interstitium extraction unit 30 are calculated as dense nuclei in pixel units.
최종 폐렴의 비율은 전체 폐간질(interstitium) 면적 대비 핵이 집중된 폐간질(interstitium) 면적을 계산하여 정량화한다.The final pneumonia rate is quantified by calculating the area of the lung interstitium where the nuclei are concentrated compared to the total lung interstitium area.
3) 실험 검증3) Experimental verification
본 발명에서는 제안된 폐렴 및 폐기종 정량 방법의 정확성을 검증하기 위해 병리학자의 판단과 우리 방법의 결과를 비교 분석한다.In the present invention, we compare and analyze the results of our method with the judgment of a pathologist to verify the accuracy of the proposed method for quantifying pneumonia and emphysema.
전문가 1과 전문가 2가 각 샘플에 대해 정량한 결과를 바탕으로 하나의 분할이미지에서 최소 40, 50 및 60개 핵의 폐렴 발병률을 정량화했다. 도 9는 전문가 1, 2의 샘플별 정량화 및 분할이미지별 핵 수에 따른 폐렴 비율 정량화 결과를 나타낸 것이다. 그래프에서 x축은 샘플의 수를 나타내고 y축은 각 샘플에 대한 폐렴의 비율을 나타낸다. 도 9(a)는 우리 결과의 대부분이 전문가가 정량화한 것보다 높게 계산되었음을 보여주며, 이는 폐렴을 정량화하는 기준이 단위 면적당 40개 이상의 핵에 집중되어 있을 때 전문가들이 폐렴을 고려한다는 것을 의미한다.Based on the quantitative results for each sample by Expert 1 and Expert 2, the pneumonia incidence rate of at least 40, 50, and 60 nuclei in one segmented image was quantified. Figure 9 shows the results of experts 1 and 2 quantifying the pneumonia rate according to the quantification of each sample and the number of nuclei in each segmented image. In the graph, the x-axis represents the number of samples and the y-axis represents the rate of pneumonia for each sample. Figure 9(a) shows that most of our results are calculated higher than those quantified by experts, meaning that experts consider pneumonia when the criterion for quantifying pneumonia is concentrated in more than 40 nuclei per unit area. .
한편, 도 9(b)는 도 9(a) 보다 전문가의 것과 유사하다고 판단할 수 있다. 특히, 오차는 전문가들이 정량화한 60 개 범위 중 가장 작았다. 그러나 전문가들이 폐렴을 60 이상으로 정량화한 범위에서 우리의 정량화 결과는 전문가보다 작았다. 이는 전문가가 높은 비율로 정량화할 때 단위 면적당 50 개 미만의 핵 농도도 폐렴으로 간주된다는 것을 의미한다. Meanwhile, Figure 9(b) can be judged to be more similar to that of the expert than Figure 9(a). In particular, the error was the smallest among the 60 ranges quantified by experts. However, in the range where experts quantified pneumonia as 60 or higher, our quantification results were smaller than those of experts. This means that even a concentration of less than 50 nuclei per unit area, when quantified by experts at a high rate, is considered pneumonia.
도 10은 단위 면적당 핵이 50 개 이상일 때 정량한 폐렴의 비율이 두 전문가의 편차가 가장 작은 것을 확인했다. 이는 도 9(b)를 통해 확인할 수 있다. 단위 면적당 50 개 이상의 핵이 있을 때 폐렴의 비율을 정량화한 결과는 전문가의 정량적 발명과 가장 유사했으며, 우리의 정량화 방법은 샘플에 존재하는 핵의 수를 식별하여 폐렴의 비율을 정량화했기 때문에 절대적이다. 따라서 전문가의 경험과 직관을 바탕으로 한 정성적 분석을 바탕으로 절대적 기준을 통해 폐렴의 비율을 정량화할 수 있다. 아래는 단위 면적당 50 개 이상의 핵을 기준으로 한 정량화를 기반으로 전문가의 정량화 결과를 비교 분석한다.Figure 10 confirms that the rate of pneumonia quantified had the smallest deviation between the two experts when there were more than 50 nuclei per unit area. This can be confirmed through Figure 9(b). The results of quantifying the rate of pneumonia when there were more than 50 nuclei per unit area were most similar to the quantitative inventions of experts, and our quantification method was absolute because it quantified the rate of pneumonia by identifying the number of nuclei present in the sample. . Therefore, the rate of pneumonia can be quantified using absolute standards based on qualitative analysis based on the experience and intuition of experts. Below, we compare and analyze the quantification results of experts based on quantification based on more than 50 nuclei per unit area.
4) 본 발명의 정량화 방법에 의한 인간의 결정 평가4) Evaluation of human decisions by the quantification method of the present invention
본 발명에서 제공하는 정량화 방법은 인간의 마우스 샘플에 대한 전통적인 정량화 방법의 관찰 및 고려를 제공한다.The quantification method provided by the present invention provides observations and considerations of traditional quantification methods for human and mouse samples.
4-1) 데이터 및 실험 환경4-1) Data and experiment environment
본 발명의 샘플 데이터는 창원 경상 대학교병원과 함께 병리학 발명을 목적으로 수집한 것으로 총 237 개의 폐샘플 이미지로 구성되어 있다. 제안된 폐렴 및 폐기종 정량 방법의 정확도를 평가하기 위해 두 명의 독립적인 병리학 전문가가 판단한 폐렴 및 폐기종 정량 발명의 결과 데이터를 수집하였다. 본 발명의 결과와 비교하기 위해 병리학 전문가가 판단한 데이터를 수집하였으며, 모두 창원 경상 대학교병원 병리학 전문가가 판단하였다.The sample data of the present invention was collected for the purpose of pathology invention together with Changwon Gyeongsang National University Hospital and consists of a total of 237 lung sample images. To evaluate the accuracy of the proposed method for quantifying pneumonia and emphysema, we collected data on the results of the invention for quantifying pneumonia and emphysema as judged by two independent pathologists. In order to compare the results of the present invention, data judged by a pathology expert were collected, and all were judged by a pathology expert at Changwon Gyeongsang National University Hospital.
본 발명에 사용된 각 신경망 모델의 이미지 크기는 YOLOv5 640x640, YOLACT는 1,640x1, 440 픽셀 크기(신경망 입력 크기), RGB 32비트 이미지이다. YOLOv5 및 YOLACT 실험에 사용된 장비는 Ubuntu 20.04 LTS와 GPU를 GeForce RTX 3090 Ti GPU로 사용하여 python으로 OS를 구현하였다. 그리고 컴퓨터 비전을 이용한 실험에 사용된 장비는 OS는 Windows 10으로, GPU는 Geforce RTX 2080 Ti GPU 를 사용하여 python으로 구현 하였다.The image size of each neural network model used in the present invention is YOLOv5 640x640, YOLACT is 1,640x1, 440 pixel size (neural network input size), and RGB 32-bit image. The equipment used in the YOLOv5 and YOLACT experiments was Ubuntu 20.04 LTS and the GPU was GeForce RTX 3090 Ti GPU, and the OS was implemented in python. And the equipment used in the experiment using computer vision was implemented in python using Windows 10 as the OS and Geforce RTX 2080 Ti GPU as the GPU.
4-2) 폐렴의 정량 결과 분석4-2) Quantitative result analysis of pneumonia
도 11은 전문가가 정량한 폐렴 결과와 전문가가 검체를 정성적으로 분석한 결과의 편차를 나타낸 것이다. 편차는 전문가에 의해 정량화된 폐렴의 비율을 기반으로 슬라이딩 윈도우 방법을 사용하여 계산되었다. 도 11은 우리가 정량한 폐렴의 비율을 기준으로 전문가들이 정량한 폐렴의 비율을 비교분석한 결과이다.Figure 11 shows the difference between the pneumonia results quantified by the expert and the results of the expert's qualitative analysis of the sample. Deviation was calculated using the sliding window method based on the proportion of pneumonia quantified by experts. Figure 11 shows the results of a comparative analysis of the pneumonia rate quantified by experts based on the pneumonia rate quantified by us.
두 전문가가 정량화한 폐렴 정도는 45~70의 중간 범위에서 기계의 판단에 가장 가까웠다. 이는 두 전문가가 상대적으로 낮은 범위(0 ~ 45) 및 높은 범위(70 ~ 90) 보다 중간 범위(45~70)에서 안정적으로 폐렴 정도를 정량화했다는 의미다. The degree of pneumonia quantified by the two experts was closest to the machine's judgment, in the mid-range of 45 to 70. This means that the two experts quantified the severity of pneumonia more stably in the mid-range (45-70) than in the relatively low range (0-45) and high range (70-90).
도 11에서 전문가 1이 정량한 폐렴의 비율과 우리가 정량한 폐렴의 비율을 비교하면 단위면적에 50 개 이상의 핵이 있을 때 정량된 결과와의 편차가 0~30 범위에서 가장 작았다. 그리고 30 개 이상의 범위에서 단위 면적에 40 개 이상의 핵이 있을 때 정량화된 결과로부터의 편차가 가장 작았다.In Figure 11, comparing the proportion of pneumonia quantified by expert 1 and the proportion of pneumonia quantified by us, when there were more than 50 nuclei in a unit area, the deviation from the quantified result was the smallest in the range of 0 to 30. And in the range of more than 30, the deviation from the quantified results was the smallest when there were more than 40 nuclei per unit area.
또한, 도 11에서 전문가 2에 의해 정량화된 폐렴의 비율과 우리가 정량한 폐렴의 비율을 비교할 때, 0~20까지의 범위에서 60 이상일 때 정량 결과로부터의 편차가 가장 작았다. 20~60의 범위에서 정량 결과와의 편차는 50 이상일 때 가장 작았다. 그리고 60 개 이상의 범위 에서는 40 개 이상일 때 정량화된 결과와의 편차가 가장 작았다.In addition, when comparing the proportion of pneumonia quantified by expert 2 and the proportion of pneumonia quantified by us in Figure 11, the deviation from the quantitative result was the smallest when it was 60 or more in the range from 0 to 20. In the range of 20 to 60, the deviation from the quantitative result was smallest when it was 50 or more. And in the range of 60 or more, the deviation from the quantified results was the smallest when there were 40 or more.
두 전문가는 폐렴 비율이 낮은 범위에서 단위 면적당 핵 밀도가 높은 경우를 폐렴, 반대로 폐렴이 높은 범위에서 단위 면적당 핵 밀도가 낮은 범위를 폐렴이라고 판단했다.The two experts judged that cases with a high nuclear density per unit area in a low pneumonia rate range were pneumonia, and conversely, those in a range with a high pneumonia rate and low nuclear density per unit area were judged to be pneumonia.
도 12 및 도 13은 또한 전문가들이 폐렴을 높은 범위 내에서 단위 면적당 핵 농도가 낮은 것으로 판단하는 경향이 있음을 보여준다. Figures 12 and 13 also show that experts tend to judge pneumonia as having a low nuclear concentration per unit area within the high range.
도 12는 전문가 1에 의해 정량화된 폐렴 비율에 따라 정량화된 결과를 나타낸 것으로, 핵의 밀도 분포가 40 이상일 때 정량화된 결과와 유사한 결과가 대부분 나타났다. 심지어 전문가 1은 폐렴이 대부분 단위 면적당 40 개 미만으로 간주했다. 도 30에서도 전문가 2에 의해 정량화된 폐렴의 비율이 높은 범위에서 단위면적당 핵밀도가 40 미만인 경우를 폐렴으로 본다는 것을 보여주고 있다.Figure 12 shows the quantified results according to the pneumonia rate quantified by expert 1, and most similar results to the quantified results were shown when the nuclear density distribution was 40 or more. Expert 1 even considered that most cases of pneumonia were less than 40 per unit area. Figure 30 also shows that in the range where the pneumonia rate quantified by expert 2 is high, cases where the nuclear density per unit area is less than 40 are considered pneumonia.
여기서 두 전문가가 폐렴을 정량화할 때 폐렴의 비율이 낮을수록 전문가가 폐간질(interstitium)과 핵 사이의 밀도를 더 높게 정량화하는 경향이 있음을 발견했다. 반대로, 폐렴의 비율이 높을수록 폐간질(interstitium)과 핵 사이의 밀도의 정량화가 낮아짐을 발견했다.Here, we found that when two experts quantified pneumonia, the lower the proportion of pneumonia, the higher the expert tended to quantify the density between the lung interstitium and the nucleus. Conversely, we found that the higher the rate of pneumonia, the lower the quantification of density between the lung interstitium and the nucleus.
도 14는 본 발명에 따른 정량화 결과(단위 면적당 50개 이상인 경우 정량화한 폐렴의 비율)를 기준으로 전문가 결과의 편차를 나타낸다. x 축은 폐렴의 비율을 나타내고 y 축은 편차를 나타낸다. 도 14(a)는 본 발명의 결과를 바탕으로 전문가 1과의 편차를 나타내며, 전문가는 폐렴도 50 이상 구간에서 본 발명 결과보다 높은 수준의 폐렴을 정량화하였다. 도 14(b)는 본 발명의 결과를 바탕으로 전문가 2와의 편차를 나타내며, 전문가는 폐렴도 50 이상 구간에서 본 발명 결과보다 높은 수준의 폐렴을 정량화하였다.Figure 14 shows the deviation of expert results based on the quantification results according to the present invention (proportion of pneumonia quantified when there are 50 or more per unit area). The x-axis represents the rate of pneumonia and the y-axis represents the deviation. Figure 14(a) shows the deviation from expert 1 based on the results of the present invention, and the expert quantified pneumonia at a higher level than the results of the present invention in the pneumonia degree range of 50 or higher. Figure 14(b) shows the deviation from expert 2 based on the results of the present invention, and the expert quantified pneumonia at a higher level than the results of the present invention in the pneumonia degree range of 50 or higher.
이는 폐렴의 비율을 정량화할 때 큰 문제로 이어질 수 있다. 문제는 폐렴의 정도가 높은 검체에서 폐렴 증상이 나타나지 않는 부분도 폐렴으로 진단할 수 있다는 점이다. 이를 통해 표본의 특성에 따라 전문가의 정성적 판단도 달라질 수 있음을 확인하였다.This can lead to major problems when quantifying rates of pneumonia. The problem is that in samples with a high degree of pneumonia, even parts that do not show symptoms of pneumonia can be diagnosed as pneumonia. Through this, it was confirmed that experts' qualitative judgments may vary depending on the characteristics of the sample.
상기 부호의 설명을 아래와 같이 정리한다.The explanation of the above symbols is summarized as follows.
1. 폐렴 정량화 시스템 1. Pneumonia Quantification System
10. 이미지획득부10. Image acquisition department
20. 세기관지제거부20. Bronchiole removal unit
30. 폐간질추출부30. Pulmonary interstitial extract
40. 분할이미지생성부40. Segmented image generation unit
50. 세포핵인식부50. Cell nuclear recognition unit
60. 세포핵집계부60. Cell nucleus aggregation unit
70. 분포도생성부70. Distribution map generator
80. 임계값비교부80. Threshold comparison unit
90. 폐렴판단부90. Pneumonia diagnosis department
100. 면적계산부100. Area calculation section
110. 폐렴율산출부110. Pneumonia rate calculation department
S10. 이미지획득부(10)가 폐를 생검한 현미경 촬영을 통해 폐이미지를 획득하는 이미지 획득 단계S10. An image acquisition step in which the image acquisition unit 10 acquires lung images through microscopic imaging of lung biopsies.
S20. 세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계S20. A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole.
S30. 폐간질추출부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출하는 폐간질 추출 단계S30. A pulmonary interstitium extraction step in which the pulmonary interstitium extraction unit 30 extracts the pulmonary interstitium area from the lung image from which the bronchioles have been removed.
S40. 분할이미지생성부(40)가 상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성하는 분할이미지 생성 단계S40. A segmented image generation step in which the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and then generates a segmented image.
S50. 세포핵인식부(50)가 상기 분할이미지에서 세포핵(cell nucleus)을 인식하는 세포핵 인식 단계S50. A cell nucleus recognition step in which the cell nucleus recognition unit 50 recognizes the cell nucleus in the segmented image.
S60. 세포핵집계부(60)가 상기 인식된 세포핵(cell nucleus)의 개수를 집계하는 세포핵 집계 단계S60. A cell nucleus counting step in which the cell nucleus counting unit 60 counts the number of the recognized cell nuclei.
S70. 분포도생성부(70)가 상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus)분포도를 생성하는 분포도 생성 단계S70. A distribution map generation step in which the distribution map generator 70 generates a cell nucleus distribution map by visualizing the lung image according to the aggregated number of cell nuclei.
S80. 임계값비교부(80)가 상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교하는 임계값 비교 단계S80. A threshold comparison step in which the threshold comparison unit 80 compares the counted number of cell nuclei with the threshold value.
S90. 폐렴판단부(90)가 상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단하는 폐렴 판단 단계S90. A pneumonia determination step in which the pneumonia determination unit 90 determines pneumonia when the number of aggregated cell nuclei is high compared to the threshold value.
S100. 면적계산부(100)가 상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산하는 면적 계산 단계S100. An area calculation step in which the area calculation unit 100 calculates the lung interstitium area in the segmented image determined to be pneumonia.
S110. 폐렴율산출부(110)가 폐의 폐렴율을 산출하는 폐렴율 산출 단계S110. Pneumonia rate calculation step in which the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lungs
(3) 폐기종 정량화 시스템(3) Emphysema quantification system
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템은, 도 15 및 도 16에 나타난 바와 같이, 이미지획득부(10), 세기관지제거부(20), 이진화처리부(30), 공기층계산부(40), 폐포제거부(50), 좌표확인부(60), 폐기종감지부(70), 폐기종계산부(80) 및 폐기종정량부(90)를 포함하는 것을 특징으로 한다. As shown in Figures 15 and 16, the emphysema quantification system for lung specimens using computer vision and machine learning according to the present invention includes an image acquisition unit (10), a bronchioles removal unit (20), a binarization processing unit (30), and an air layer. It is characterized by comprising a calculation unit 40, an alveolar removal unit 50, a coordinate confirmation unit 60, an emphysema detection unit 70, an emphysema calculation unit 80, and an emphysema quantification unit 90.
먼저, 상기 이미지획득부(10)는 폐를 생검하여 현미경 촬영을 통해 폐이미지 획득한다. First, the image acquisition unit 10 biopsies the lung and acquires an image of the lung through microscopic imaging.
도 17에 나타난 바와 같이, 상기 이미지획득부(10)는 니켈, 크롬 망간 및 카드뮴 등의 다양한 중금속에 노출된 폐 생검 표분을 획득한다. 하나의 폐이미지는 하나의 슬라이드를 나타내며, 하나의 슬라이드는 2 내지 4개의 섹션으로 구성된다. As shown in FIG. 17, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
다음으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거한다. Next, the bronchiole removal unit 20 recognizes the bronchiole in the lung image and removes the bronchiole.
보다 구체적으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식하여 상기 세기관지(bronchiole)를 제거하는 학습을 위해 세기관지를 레이블링(labeling)해야 한다. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
상기 세기관지제거부(20)는 상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 한다. 상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중, 일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용한다. 상기 세기관지제거부(20)는 상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성한다.The bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing. The bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
일실시예로, 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링을 수행하고 40개는 트레인 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다. 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류되고, 파티션 이미지를 모아서 다시 하나의 이미지로 만든다.In one embodiment, because the size of the bronchioles is relatively small compared to the entire section, a 5x5 partition is created and the bronchioles are labeled directly within this partition. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
다음으로, 상기 이진화처리부(30)는 상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리한다. Next, the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed.
보다 구체적으로, 상기 이진화처리부(30)는 폐간질(interstitium) 및 세기관지(bronchiole) 영역은 제1색(1)으로 처리하고, 폐포와 폐기종 영역은 제2색(0)으로 처리한다. More specifically, the binarization processing unit 30 processes the pulmonary interstitium and bronchiole areas with the first color (1), and processes the alveoli and emphysema areas with the second color (0).
폐기종에 비해 일반 폐포는 면적이 작고 모양이 불규칙하다. 또한 폐기종은 불규칙한 모양을 하고 있다. 다만 일반 폐포보다 면적이 넓은 것이 특징이다. 따라서, 상기 이진화처리부(30)를 거친 이진화된 폐이미지는 폐기종 탐색을 위해서 도 21에 나타난 바와 같이 단계별로 진행되어 일반적인 폐포를 제거하고 폐기종을 남긴다. 단계는 병리학 전문가와의 협의를 통해 step 13 내지 15가 가장 적절하다는 의견을 반영했다.Compared to emphysema, normal alveoli are smaller in area and irregular in shape. Additionally, emphysema has an irregular shape. However, their characteristic feature is that they have a larger area than regular alveoli. Therefore, the binarized lung image that has passed through the binarization processing unit 30 is processed step by step as shown in FIG. 21 to search for emphysema, removing general alveoli and leaving emphysema. The stages reflected the opinion that steps 13 to 15 were most appropriate through consultation with pathology experts.
도 21은 이진화한 이미지에서 검은색 부분의 경계를 넓힘으로써 정상적인 폐포를 삭제하고, 경계를 넓혀도 남아 있는 부분을 폐기종으로 판단하게 된다. 즉, 폐기종으로 판단할 가능성이 있는 공기주머니를 선택하는 부분이다. 다만, 의사마다 페기종으로 판단할 수 있는 불규칙한 공기 주머니의 크기가 다르기 때문에, 이 단계에서 의사의 체험적인(heuristic) 판단을 참고하게 된다. 본 발명은 이 참고에 따라서 step 12-15를 추출함으로써 의사의 체험적인(heuristic) 정량적인 판단을 존중하여 절대적인 정량화를 시도한다.In Figure 21, normal alveoli are deleted by widening the border of the black area in the binarized image, and even after widening the border, the remaining part is judged to be emphysema. In other words, this is the part where air sacs that are likely to be diagnosed as emphysema are selected. However, since the size of irregular air sacs that can be diagnosed as emphysema varies depending on the doctor, the doctor's heuristic judgment is referred to at this stage. The present invention attempts absolute quantification by respecting the doctor's heuristic quantitative judgment by extracting steps 12-15 according to this reference.
일실시예로, OpenCV 라이브러리를 사용하여 간질 및 세기관지 영역은 검은색으로 처리된다. 또한, 폐포와 폐기종을 흰색으로 처리한다.In one embodiment, the interstitial and bronchial regions are colored black using the OpenCV library. Additionally, alveoli and emphysema are treated in white.
다음으로, 상기 공기층계산부(40)는 상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산한다.Next, the air layer calculation unit 40 calculates the total air layer area in the binarized lung image.
보다 구체적으로, 상기 공기층계산부(40)는 상기 이진화(binarization) 처리 된 폐이미지에서 레이블(label)을 지정하는 레이블지정부(41) 및 상기 지정된 레이블(label)의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출하는 공기층픽셀추출부(42)로 구성된다.More specifically, the air layer calculation unit 40 calculates the sum of the pixel area of the label designation unit 41 that specifies a label in the binarized lung image and the designated label. It consists of an air layer pixel extraction unit 42 that extracts the entire air layer pixel area.
일실시예로, OpenCV의 connectedComponent 를 사용하여 표본의 공기층에 레이블을 지정한다. 이후, 상기 지정된 레이블의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출한다.In one example, OpenCV's connectedComponent is used to label air layers in a sample. Afterwards, the total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
다음으로, 상기 폐포제거부(50)는 상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출한다.Next, the alveolar removal unit 50 removes the air layer of alveoli from the binarized lung image and extracts emphysema.
정상적인 폐포는 폐기종이나 폐의 세기관지보다 크기가 작고 비교적 둥근 모양이다. 따라서 폐간질 층을 점점 두껍게 하면 폐포는 두껍게 된 간질층에 의해 사라지고 폐기종이나 세기관지만 남는다. 즉, 앞서 세기관지는 제거되었기 때문에 폐간질 층을 두껍게 하면 폐기종만이 공기층을 갖게 되어 이 부분을 폐기종으로 판단한다. Normal alveoli are smaller and relatively round in shape than those of emphysema or lung bronchioles. Therefore, if the pulmonary interstitial layer becomes thicker, the alveoli disappear due to the thickened interstitial layer, and only emphysema or bronchioles remain. In other words, because the bronchioles were previously removed, if the pulmonary interstitial layer is thickened, only the emphysema has an air layer, and this part is judged to be emphysema.
일실시예로, OpenCV의 이진화처리 과정은 공기층 영역에 대해 단계별 침식(erode) 과정을 수행하는 데 사용된다. 단계별 침식(erode) 과정의 결과, 일반 폐포 가 제거되고 폐기종의 특징만 남는다.In one embodiment, OpenCV's binarization process is used to perform a step-by-step erosion process for the air layer region. As a result of the step-by-step erode process, the normal alveoli are removed, leaving only the features of emphysema.
다음으로, 상기 좌표확인부(60)는 상기 추출된 폐기종의 좌표를 확인한다. Next, the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema.
보다 구체적으로, 상기 좌표확인부(60)는 상기 이진화(binarization) 처리 된 폐이미지에서 추출된 폐기종에 레이블(label)을 지정하는 폐기종지정부(61) 및 상기 지정된 폐기종 레이블(label)에서 중심 좌표를 추출하는 중심좌표추출부(62)로 구성된다.More specifically, the coordinate confirmation unit 60 includes an emphysema designation unit 61 that specifies a label for emphysema extracted from the binarized lung image, and a center coordinate in the designated emphysema label. It consists of a central coordinate extraction unit 62 that extracts.
일실시예로, OpenCV의 연결된 구성 요소를 사용하여 표본의 나머지 폐기종 기능에 레이블을 지정한다. 이후, 상기 지정된 레이블에서 중심 좌표를 추출한다.In one embodiment, the connected component of OpenCV is used to label the remaining emphysema features of the sample. Afterwards, the center coordinate is extracted from the specified label.
다음으로, 상기 폐기종감지부(70)는 상기 좌표가 확인 된 폐기종을 감지한다.Next, the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed.
보다 구체적으로, 상기 폐기종감지부(70)는 상기 이미지획득부(10)에서 획득한 폐이미지에 상기 좌표확인부(60)에서 확인 된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑하는 이미지맵핑부(71) 및 상기 폐기종의 중심 좌표를 기반으로 상기 폐기종 영역에 색을 지정하는 폐기종색지정부(72)로 구성된다. More specifically, the emphysema detection unit 70 is an image mapping unit ( 71) and an emphysema color designation unit 72 that designates a color to the emphysema area based on the coordinates of the center of the emphysema.
보다 구체적으로, 상기 이미지맵핑부(71)는 상기 좌표확인부(60)에서 감지된 폐기종의 중심좌표를 원본 폐이미지에 맵핑 시켜 아래 폐기종계산부(80)에서 실제 폐기종의 크기를 계산하게 된다. More specifically, the image mapping unit 71 maps the center coordinates of emphysema detected by the coordinate confirmation unit 60 to the original lung image to calculate the actual size of emphysema in the emphysema calculation unit 80 below.
또한, 상기 폐기종색지정부(72)는 상기 구별된 폐기종만을 색깔을 입혀 구별되도록 한다. In addition, the emphysema color designation unit 72 colors only the identified emphysema to be distinguished.
일실시예로, 상기 이미지획득부(10)의 원본 이미지에 상기 확인된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑한다. 이후, 상기 중심 좌표를 기반으로 BFS(Breadth First Search) 알고리즘을 사용하여 폐기종 영역에 색을 지정한다.In one embodiment, the center coordinates of the confirmed emphysema are mapped to the original image of the image acquisition unit 10 to the original lung image. Afterwards, a color is assigned to the emphysema area using the BFS (Breadth First Search) algorithm based on the center coordinates.
다음으로, 상기 폐기종계산부(80)는 상기 감지된 폐기종을 추출 후 면적을 계산한다.Next, the emphysema calculator 80 extracts the detected emphysema and calculates the area.
보다 구체적으로, 상기 폐기종계산부(80)는 상기 폐기종감지부(70)에 의해 감지된 폐기종 영역을 확인 후, 상기 폐기종 영역에 레이블을 지정하는 폐기종영역지정부(81) 및 상기 레이블이 지정된 폐기종 영역의 픽셀 합을 계산하여 폐기종 픽셀의 영역을 추출하는 폐기종픽셀추출부(82)로 구성된다.More specifically, the emphysema calculation unit 80 checks the emphysema area detected by the emphysema detection unit 70, and then configures the emphysema area designation unit 81 to label the emphysema area and the labeled emphysema area. It consists of an emphysema pixel extraction unit 82 that extracts the area of emphysema pixels by calculating the sum of pixels in the area.
일실시예로, inRange 를 사용하여 BFS(Breadth First Search) 알고리즘에 의해 감지된 폐기종의 영역을 추출한다. 다음으로, 상기 추출된 폐기종 영역은 OpenCV 의 connectedComponents를 사용하여 레이블을 지정한다. 이후, 상기 지정된 레이블의 픽셀 영역의 합을 계산하여 폐기종 픽셀의 영역을 추출한다.In one embodiment, inRange is used to extract the area of emphysema detected by the BFS (Breadth First Search) algorithm. Next, the extracted emphysema region is labeled using OpenCV's connectedComponents. Afterwards, the area of the emphysema pixel is extracted by calculating the sum of the pixel areas of the designated labels.
폐기종은 이전 단계에서 추출한 상기 중심 좌표를 상기 원본 이미지에 대입하고 폐기종 영역을 채색하여 감지할 수 있다. 노드는 픽셀로 설정되고 검은색 영역은 간극으로 간주된다 . 이때 흰색 영역을 검색하도록 설정되었다. 추출 좌표는 첫 번째 노드로 설정하고 탐색 영역은 흰색에서 빨간색 픽셀로 변경되도록 설정한다.Emphysema can be detected by substituting the center coordinates extracted in the previous step into the original image and coloring the emphysema area. Nodes are set to pixels and black areas are considered gaps. At this time, it was set to search the white area. The extraction coordinates are set to the first node and the search area is set to change from white to red pixels.
다음으로, 상기 폐기종정량부(90)는 상기 추출된 폐기종의 폐기종율을 정량화한다.Next, the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
보다 구체적으로, 상기 폐기종정량부(90)는 아래의 [식 1]에 의해 상기 폐기종율을 산출한다.More specifically, the emphysema determination unit 90 calculates the emphysema rate according to [Equation 1] below.
[식 1][Equation 1]
폐기종율 = Aa / Aemp Emphysema rate = A a / A emp
(여기서, Aa 는 상기 공기층계산부(40)에 의해 계산된 전체 공기층의 면적이고, Aemp 는 Aa 와 상기 폐기종계산부(80)에 의해 계산된 상기 폐기종 면적의 합).(Here, A a is the area of the total air layer calculated by the air layer calculator 40, and A emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
일실시예로, 상기 공기층계산부(40)에 의해 추출된 전체 공기층의 면적과 상기 폐기종계산부(80)에 의해 계산된 감지된 폐기종 면적은 픽셀 단위로 계산한다. 전체 공기층의 면적에 대한 폐기종의 면적을 계산하여 최종 폐기종의 비율을 정량화한다.In one embodiment, the area of the total air layer extracted by the air layer calculator 40 and the detected emphysema area calculated by the emphysema calculator 80 are calculated on a pixel basis. The final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
(4) 폐기종 정량화 방법(4) Emphysema quantification method
본 발명에 따른 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법은, 도 17 및 도 18에 나타난 바와 같이, 상기 폐기종 정량화 시스템(1)을 이용하여 폐기종을 정량화한다. The method for quantifying emphysema in lung samples using computer vision and machine learning according to the present invention quantifies emphysema using the emphysema quantification system (1), as shown in FIGS. 17 and 18.
먼저, 제1단계(S10)는 이미지 획득 단계이다.First, the first step (S10) is the image acquisition step.
상기 제1단계(S10)는 상기 이미지획득부(10)가 폐를 생검하여 현미경 촬영한 폐이미지를 획득한다. 도 17에 나타난 바와 같이, 상기 이미지획득부(10)는 니켈, 크롬 망간 및 카드뮴 등의 다양한 중금속에 노출된 폐 생검 표분을 획득한다. 하나의 폐이미지는 하나의 슬라이드를 나타내며, 하나의 슬라이드는 2 내지 4개의 섹션으로 구성된다. In the first step (S10), the image acquisition unit 10 biopsies the lung and acquires a lung image taken with a microscope. As shown in FIG. 17, the image acquisition unit 10 acquires lung biopsy samples exposed to various heavy metals such as nickel, chromium manganese, and cadmium. One lung image represents one slide, and one slide consists of 2 to 4 sections.
다음으로, 제2단계(S20)는 세기관지 제거 단계이다.Next, the second step (S20) is the bronchioles removal step.
상기 제2단계(S20)는 상기 세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거한다. 보다 구체적으로, 상기 세기관지제거부(20)는 상기 폐이미지에서 세기관지(bronchiole)를 인식하여 상기 세기관지(bronchiole)를 제거하는 학습을 위해 세기관지를 레이블링(labeling)해야 한다. In the second step (S20), the bronchioles removal unit 20 recognizes the bronchioles in the lung image and then removes the bronchioles. More specifically, the bronchiole removal unit 20 must label the bronchioles in order to recognize them in the lung image and learn to remove the bronchiole.
상기 세기관지제거부(20)는 상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 한다. 상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중, 일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용한다. 상기 세기관지제거부(20)는 상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성한다.The bronchiole removal unit 20 divides the lung image into predetermined sizes to set partitions, and labels some of the set partitions as bronchioles areas. Among the set partitions, some of the partitions that are not labeled as the bronchial region are used as learning data for learning, and the rest are used as test data for testing. The bronchiole removal unit 20 regathers the partitions to be divided after labeling of the bronchiole region is completed and regenerates one lung image.
일실시예로, 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링을 수행하고 40개는 트레인 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다. 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류되고, 파티션 이미지를 모아서 다시 하나의 이미지로 만든다.In one embodiment, because the size of the bronchioles is relatively small compared to the entire section, a 5x5 partition is created and the bronchioles are labeled directly within this partition. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data. After learning, the bronchiolar regions within all sections are segmented and classified, and the partition images are collected to form a single image.
다음으로, 제3단계(S30)는 이진화 처리 단계이다.Next, the third step (S30) is a binarization processing step.
상기 제3단계(S30)는 상기 이진화처리부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리한다. 보다 구체적으로, 상기 이진화처리부(30)는 폐간질(interstitium) 및 세기관지(bronchiole) 영역은 제1색(1)으로 처리하고, 폐포와 폐기종 영역은 제2색(0)으로 처리한다. In the third step (S30), the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed. More specifically, the binarization processing unit 30 processes the pulmonary interstitium and bronchiole areas with the first color (1), and processes the alveoli and emphysema areas with the second color (0).
폐기종에 비해 일반 폐포는 면적이 작고 모양이 불규칙하다. 또한 폐기종은 불규칙한 모양을 하고 있다. 다만 일반 폐포보다 면적이 넓은 것이 특징이다. 따라서, 상기 이진화처리부(30)를 거친 이진화 폐이미지는 폐기종 탐색을 위해서 도 21에 나타난 바와 같이 단계별로 진행되어 일반적인 폐포를 제거하고 폐기종을 남긴다. 단계는 병리학 전문가와의 협의를 통해 step 13 내지 15가 가장 적절하다는 의견을 반영했다.Compared to emphysema, normal alveoli are smaller in area and irregular in shape. Additionally, emphysema has an irregular shape. However, their characteristic feature is that they have a larger area than regular alveoli. Therefore, the binarized lung image that has passed through the binarization processing unit 30 is processed step by step as shown in FIG. 21 to search for emphysema, removing general alveoli and leaving behind emphysema. The stages reflected the opinion that steps 13 to 15 were most appropriate through consultation with pathology experts.
일실시예로, OpenCV 라이브러리를 사용하여 간질 및 세기관지 영역은 검은색으로 처리된다. 또한, 폐포와 폐기종을 흰색으로 처리한다.In one embodiment, the interstitial and bronchial regions are colored black using the OpenCV library. Additionally, alveoli and emphysema are treated in white.
다음으로, 제4단계(S40)는 공기층 계산 단계이다.Next, the fourth step (S40) is the air space calculation step.
상기 제4단계(S40)는 상기 공기층계산부(40)가 상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산한다. 보다 구체적으로, 상기 제4단계(S40)는 상기 공기층계산부(40)는 상기 이진화(binarization) 처리 된 폐이미지에서 레이블(label)을 지정하는 레이블지정부(41) 및 상기 지정된 레이블(label)의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출하는 공기층픽셀추출부(42)로 구성된다.In the fourth step (S40), the air layer calculation unit 40 calculates the total area of the air layer in the binarized lung image. More specifically, in the fourth step (S40), the air layer calculation unit 40 includes a label designation unit 41 that specifies a label in the binarized lung image and the designated label. It consists of an air layer pixel extraction unit 42 that extracts the entire air layer pixel area by calculating the sum of the pixel areas.
일실시예로, OpenCV의 connectedComponent 를 사용하여 표본의 공기층에 레이블을 지정한다. 이후, 상기 지정된 레이블의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출한다.In one example, OpenCV's connectedComponent is used to label air layers in a sample. Afterwards, the total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
다음으로, 제5단계(S50)는 폐포 제거 단계이다.Next, the fifth step (S50) is the alveolar removal step.
상기 제5단계(S50)는 상기 폐포제거부(50)가 상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출한다. In the fifth step (S50), the alveolar removal unit 50 removes the air layer of the alveoli from the binarized lung image and extracts emphysema.
정상적인 폐포는 폐기종이나 폐의 세기관지보다 크기가 작고 비교적 둥근 모양이다. 따라서 폐간질 층을 점점 두껍게 하면 폐포는 두껍게 된 간질층에 의해 사라지고 폐기종이나 세기관지만 남는다. 즉, 앞서 세기관지는 제거되었기 때문에 폐간질 층을 두껍게 하면 폐기종만이 공기층을 갖게 되어 이 부분을 폐기종으로 판단한다. Normal alveoli are smaller and relatively round in shape than those of emphysema or lung bronchioles. Therefore, if the pulmonary interstitial layer becomes thicker, the alveoli disappear due to the thickened interstitial layer, and only emphysema or bronchioles remain. In other words, because the bronchioles were previously removed, if the pulmonary interstitial layer is thickened, only the emphysema has an air layer, and this part is judged to be emphysema.
일실시예로, OpenCV의 이진화처리 과정은 공기층 영역에 대해 단계별 침식(erode) 과정을 수행하는 데 사용된다. 단계별 침식(erode) 과정의 결과, 일반 폐포 가 제거되고 폐기종의 특징만 남는다.In one embodiment, OpenCV's binarization process is used to perform a step-by-step erosion process for the air layer region. As a result of the step-by-step erode process, the normal alveoli are removed, leaving only the features of emphysema.
다음으로, 제6단계(S60)는 좌표 확인 단계이다.Next, the sixth step (S60) is the coordinate confirmation step.
상기 제6단계(S60)는 상기 좌표확인부(60)가 상기 추출된 폐기종의 좌표를 확인한다. 보다 구체적으로, 상기 제6단계(S60)는 폐기종지정부(61)가 상기 이진화(binarization) 처리 된 폐이미지에서 추출된 폐기종에 레이블(label)을 지정하는 폐기종 지정 단계(S61) 및 중심좌표추출부(62)가 상기 지정된 폐기종 레이블(label)에서 중심 좌표를 추출하는 중심 좌표 추출 단계(S62)로 구성된다.In the sixth step (S60), the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema. More specifically, the sixth step (S60) is an emphysema designation step (S61) in which the emphysema designation unit 61 assigns a label to the emphysema extracted from the binarized lung image and central coordinate extraction. It consists of a center coordinate extraction step (S62) in which the unit 62 extracts the center coordinate from the designated emphysema label.
일실시예로, OpenCV의 연결된 구성 요소를 사용하여 표본의 나머지 폐기종 기능에 레이블을 지정한다. 이후, 상기 지정된 레이블에서 중심 좌표를 추출한다.In one embodiment, the connected component of OpenCV is used to label the remaining emphysema features of the sample. Afterwards, the center coordinate is extracted from the specified label.
다음으로, 제7단계(S70)는 폐기종 감지 단계이다.Next, the seventh step (S70) is the emphysema detection step.
상기 제7단계(S70)는 상기 폐기종감지부(70)가 상기 좌표가 확인 된 폐기종을 감지하는 폐기종 감지한다. 보다 구체적으로, 상기 제7단계(S70)는 이미지맵핑부(71)가 상기 이미지획득부(10)에서 획득한 폐이미지에 상기 좌표확인부(60)에서 확인 된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑하는 이미지 맵핑 단계(S71) 및 폐기종색지정부(72)가 상기 폐기종의 중심 좌표를 기반으로 상기 폐기종 영역에 색을 지정하는 폐기종 색 지정 단계(S72)로 구성된다. In the seventh step (S70), the emphysema detection unit 70 detects emphysema whose coordinates are confirmed. More specifically, in the seventh step (S70), the image mapping unit 71 matches the center coordinates of emphysema confirmed by the coordinate confirmation unit 60 to the lung image acquired by the image acquisition unit 10 to the original lung image. It consists of an image mapping step (S71) and an emphysema color designation step (S72) in which the emphysema color designation unit 72 assigns a color to the emphysema area based on the center coordinates of the emphysema.
보다 구체적으로, 상기 이미지 맵핑 단계(S71)는 상기 이미지맵핑부(71)가 상기 좌표확인부(60)에서 감지된 폐기종의 중심좌표를 원본 폐이미지에 맵핑 시켜 아래 폐기종계산부(80)에서 실제 폐기종의 크기를 계산하게 된다. More specifically, in the image mapping step (S71), the image mapping unit 71 maps the center coordinates of emphysema detected by the coordinate confirmation unit 60 to the original lung image and actualizes them in the emphysema calculation unit 80 below. The size of emphysema is calculated.
또한, 상기 폐기종 색 지정 단계(S72)는 상기 폐기종색지정부(72)가 상기 구별된 폐기종만을 색깔을 입혀 구별되도록 한다. In addition, in the emphysema color designation step (S72), the emphysema color designation unit 72 colors only the identified emphysema to distinguish it.
일실시예로, 상기 이미지획득부(10)의 원본 이미지에 상기 확인된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑한다. 이후, 상기 중심 좌표를 기반으로 BFS(Breadth First Search) 알고리즘을 사용하여 폐기종 영역에 색을 지정한다.In one embodiment, the center coordinates of the confirmed emphysema are mapped to the original image of the image acquisition unit 10 to the original lung image. Afterwards, a color is assigned to the emphysema area using the BFS (Breadth First Search) algorithm based on the center coordinates.
다음으로, 제8단계(S80)는 폐기종 계산 단계이다.Next, the eighth step (S80) is the emphysema calculation step.
상기 제8단계(S80)는 상기 폐기종계산부(80)가 상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종 계산한다. In the eighth step (S80), the emphysema calculation unit 80 extracts the detected emphysema and calculates the area.
보다 구체적으로, 상기 제8단계(S80)는 폐기종영역지정부(81)가 상기 폐기종감지부(70)에 의해 감지된 폐기종 영역을 확인 후, 상기 폐기종 영역에 레이블을 지정하는 폐기종 영역 지정 단계(S81) 및 폐기종픽셀추출부(82)가 상기 레이블이 지정된 폐기종 영역의 픽셀 합을 계산하여 폐기종 픽셀의 영역을 추출하는 폐기종 픽셀 추출 단계(S82)로 구성된다.More specifically, the eighth step (S80) is an emphysema area designation step in which the emphysema area designation unit 81 confirms the emphysema area detected by the emphysema detection unit 70 and then labels the emphysema area ( S81) and an emphysema pixel extraction step (S82) in which the emphysema pixel extraction unit 82 extracts the area of the emphysema pixel by calculating the sum of pixels of the labeled emphysema area.
일실시예로, inRange 를 사용하여 BFS(Breadth First Search) 알고리즘에 의해 감지된 폐기종의 영역을 추출한다. 다음으로, 상기 추출된 폐기종 영역은 OpenCV 의 connectedComponents를 사용하여 레이블을 지정한다. 이후, 상기 지정된 레이블의 픽셀 영역의 합을 계산하여 폐기종 픽셀의 영역을 추출한다.In one embodiment, inRange is used to extract the area of emphysema detected by the BFS (Breadth First Search) algorithm. Next, the extracted emphysema region is labeled using OpenCV's connectedComponents. Afterwards, the area of the emphysema pixel is extracted by calculating the sum of the pixel areas of the designated labels.
폐기종은 이전 단계에서 추출한 상기 중심 좌표를 상기 원본 이미지에 대입하고 폐기종 영역을 채색하여 감지할 수 있다. 노드는 픽셀로 설정되고 검은색 영역은 간극으로 간주된다 . 이때 흰색 영역을 검색하도록 설정되었다. 추출 좌표는 첫 번째 노드로 설정하고 탐색 영역은 흰색에서 빨간색 픽셀로 변경되도록 설정한다.Emphysema can be detected by substituting the center coordinates extracted in the previous step into the original image and coloring the emphysema area. Nodes are set to pixels and black areas are considered gaps. At this time, it was set to search the white area. The extraction coordinates are set to the first node and the search area is set to change from white to red pixels.
다음으로, 제9단계(S90)는 폐기종 정량 단계이다.Next, the ninth step (S90) is the emphysema quantification step.
상기 제9단계(S90)는 상기 폐기종정량부(90)가 상기 추출된 폐기종의 폐기종율을 정량화한다. In the ninth step (S90), the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
보다 구체적으로, 상기 폐기종정량부(90)는 아래의 [식 1]에 의해 상기 폐기종율을 산출한다.More specifically, the emphysema determination unit 90 calculates the emphysema rate according to [Equation 1] below.
[식 1][Equation 1]
폐기종율 = Aa / Aemp Emphysema rate = A a / A emp
(여기서, Aa 는 상기 공기층계산부(40)에 의해 계산된 전체 공기층의 면적이고, Aemp 는 Aa 와 상기 폐기종계산부(80)에 의해 계산된 상기 폐기종 면적의 합).(Here, A a is the area of the total air layer calculated by the air layer calculator 40, and A emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
일실시예로, 상기 공기층계산부(40)에 의해 추출된 전체 공기층의 면적과 상기 폐기종계산부(80)에 의해 계산된 감지된 폐기종 면적은 픽셀 단위로 계산한다. 전체 공기층의 면적에 대한 폐기종의 면적을 계산하여 최종 폐기종의 비율을 정량화한다.In one embodiment, the area of the total air layer extracted by the air layer calculator 40 and the detected emphysema area calculated by the emphysema calculator 80 are calculated on a pixel basis. The final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
이하, 실시예를 통하여 본 발명을 보다 상세하게 설명한다. 본 발명의 목적, 특징, 장점은 이하의 실시예를 통하여 쉽게 이해될 것이다. 본 발명은 여기서 설명하는 실시예에 한정되지 않고, 다른 형태로 구체화될 수도 있다. 여기서 소개되는 실시예는 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자에게 본 발명의 사상이 충분히 전달될 수 있도록 하기 위해 제공되는 것이다. 따라서 이하의 실시예에 의해 본 발명이 제한되어서는 안 된다.Hereinafter, the present invention will be described in more detail through examples. The purpose, features, and advantages of the present invention will be easily understood through the following examples. The present invention is not limited to the embodiments described herein and may be embodied in other forms. The embodiments introduced here are provided to enable the idea of the present invention to be sufficiently conveyed to those skilled in the art. Therefore, the present invention should not be limited by the following examples.
1) 생검 조직의 데이터 수집1) Data collection of biopsy tissue
7주령 C57BL/6 마우스를 실험 동물로 선택하여 마우스 폐 조직 샘플, 희석된 니켈, 크롬, 망간 및 카드뮴을 각각 50nM , 20nM , 10nM , 10nM 농도로 채취 하여 노출시켰다. 1일 1회 4주 동안 단독으로 복합적으로 비강을 통해 총 70개의 샘플을 채취하였다. 본 실험은 울산대학교 SPF(특정 병원체 무첨가)에서 진행되었으며, 울산대학교 동물발명소(IACUC)(IACUC No. BSK-21-030)의 심사를 받았다.Seven-week-old C57BL/6 mice were selected as experimental animals and exposed to mouse lung tissue samples and diluted nickel, chromium, manganese, and cadmium at concentrations of 50nM, 20nM, 10nM, and 10nM, respectively. A total of 70 samples were collected through the nasal cavity, alone or in combination, once a day for 4 weeks. This experiment was conducted at the University of Ulsan SPF (no specific pathogens added) and was reviewed by the Animal Invention Center of the University of Ulsan (IACUC) (IACUC No. BSK-21-030).
수집된 70개의 표본은 각 슬라이드로 구성되었다. 도 17은 하나의 슬라이드를 보여주고 하나의 슬라이드는 2-4개의 섹션으로 구성되어 있다. 따라서 총 70개의 슬라이드에서 표본을 추출하고 237장에서 단면 데이터를 수집했다. 이 중 실제 의사가 사용하는 섹션 70개만 사용했다. 한 섹션은 너비 8200, 길이 7200 크기의 고화질 이미지이다.The 70 specimens collected consisted of each slide. Figure 17 shows one slide, and one slide consists of 2-4 sections. Therefore, a total of 70 slides were sampled and cross-sectional data were collected from 237 slides. Of these, only 70 sections actually used by doctors were used. One section is a high-quality image measuring 8200 wide and 7200 long.
2) 폐기종 비율 측정2) Measurement of emphysema rate
본 발명에서는 폐기종의 비율을 정량화하였으며, 검체를 통해 폐기종의 비율을 확인하는 방법은 다음과 같다. 폐기종의 비율은 세기관지를 제외한 전체 공기층 대비 폐기종의 공기층을 계산하여 정량화한다.In the present invention, the rate of emphysema was quantified, and the method for confirming the rate of emphysema through a sample is as follows. The proportion of emphysema is quantified by calculating the air space of emphysema compared to the total air space excluding bronchioles.
폐기종의 비율을 계산하는 지표는 아래 [식 1]과 같다.The index for calculating the rate of emphysema is as follows [Equation 1].
[식 1][Equation 1]
폐기종율 = Aa / Aemp Emphysema rate = A a / A emp
(여기서, Aa 는 상기 공기층계산부(40)에 의해 계산된 전체 공기층의 면적이고, Aemp 는 Aa 와 상기 폐기종계산부(80)에 의해 계산된 상기 폐기종 면적의 합).(Here, A a is the area of the total air layer calculated by the air layer calculator 40, and A emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
2-1) YOLACT를 사용하여 말단 폐포 조직 제거2-1) Distal alveolar tissue removal using YOLACT
YOLACT 학습에는 세기관지 레이블이 필요하다. 세기관지의 크기는 전체 섹션에 비해 상대적으로 작기 때문에 파티션을 5x5로 분할하고 이 파티션 내에서 직접 세기관지 레이블을 지정한다. 하나의 파티션은 너비가 1,640, 길이가 1,440 이다. 총 파티션 수는 5,925개이며 50개에 대해 레이블링(labeling)을 수행하고 40개는 트레인 데이터로 사용하고 나머지 10개는 테스트 데이터로 사용한다.YOLACT learning requires bronchioles labels. Because the size of the bronchioles is relatively small compared to the overall section, we divide them into 5x5 partitions and label the bronchioles directly within these partitions. One partition is 1,640 wide and 1,440 long. The total number of partitions is 5,925, and labeling is performed on 50, 40 are used as train data, and the remaining 10 are used as test data.
YOLACT 학습 후 모든 섹션 내의 세기관지 영역이 세분화되고 분류된다.After learning YOLACT, the bronchial regions within every section are segmented and classified.
파티션 이미지를 모아서 다시 하나의 이미지로 만든다.Collect partition images and create a single image again.
2-2) 이미지 이진화 전처리2-2) Image binarization preprocessing
OpenCV 라이브러리를 사용하여 간질 및 세기관지 영역은 검은색으로 처리된다. 폐포와 폐기종을 흰색으로 치료한다.Using the OpenCV library, the interstitial and bronchial regions are colored black. Treats alveoli and emphysema with white color.
2-3) 총 공기층 면적 계산2-3) Calculation of total air space area
OpenCV의 connectedComponent 를 사용하여 표본의 공기층에 레이블을 지정한다. 지정된 레이블의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출한다.Use OpenCV's connectedComponent to label the air layers in the sample. The total air layer pixel area is extracted by calculating the sum of the pixel areas of the specified labels.
2-4) 일반 폐포의 공기층을 제거2-4) Removal of air layer in normal alveoli
OpenCV의 침식은 공기층 영역에 대해 단계별 침식 과정을 수행하는 데 사용된다. 단계별 침식(erode) 과정의 결과, 일반 폐포 가 제거되고 폐기종의 특징만 남는다.OpenCV's erosion is used to perform a step-by-step erosion process on the air layer region. As a result of the step-by-step erode process, the normal alveoli are removed, leaving only the features of emphysema.
2-5) 폐기종의 중심 좌표 추출2-5) Extraction of center coordinates of emphysema
OpenCV의 연결된 구성 요소를 사용하여 표본의 나머지 폐기종 기능에 레이블을 지정한다. 지정된 레이블에서 중심 좌표를 추출한다.Label the remaining emphysema features of the sample using OpenCV's connected component. Extracts the center coordinates from the specified label.
2-6) 폐기종 감지2-6) Emphysema detection
원본 이미지에 2-5)에서 추출한 원본 이미지에 상기 확인된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑한다. 중심 좌표를 기반으로 BFS(Breadth First Search) 알고리즘을 사용하여 폐기종 영역에 색을 지정한다.The center coordinates of the confirmed emphysema are mapped to the original image extracted in 2-5) to the original lung image. Based on the coordinates of the center, the emphysema area is assigned a color using the BFS (Breadth First Search) algorithm.
2-7) 폐기종 부위 추출2-7) Extraction of emphysema area
inRange 를 사용하여 BFS(Breadth First Search) 알고리즘에 의해 감지된 폐기종의 영역을 추출한다. 추출된 폐기종 영역 의 경우 OpenCV 의 connectedComponents 를 사용하여 레이블을 지정한다. 지정된 레이블의 픽셀 영역의 합을 계산하여 폐기종 픽셀의 영역을 추출한다.Use inRange to extract the area of emphysema detected by the BFS (Breadth First Search) algorithm. For the extracted emphysema region, labels are assigned using OpenCV's connectedComponents. The area of the emphysema pixel is extracted by calculating the sum of the pixel areas of the specified label.
2-8) 폐기종의 비율을 정량화2-8) Quantify the rate of emphysema
2-3) 총 공기층 면적 계산에서 추출된 공기층의 전체 면적과 2-7) 폐기종 부위 추출에서 추출된 폐기종의 면적은 픽셀 단위로 계산된다. 전체 공기층의 면적에 대한 폐기종의 면적을 계산하여 최종 폐기종의 비율을 정량화한다.The total area of the air layer extracted in 2-3) total air layer area calculation and the area of emphysema extracted in 2-7) emphysema area extraction are calculated in pixel units. The final proportion of emphysema is quantified by calculating the area of emphysema relative to the area of the entire air space.
3) 실험 검증3) Experimental verification
본 발명에서는 제안된 폐기종 및 폐기종 정량 방법의 정확성을 검증하기 위해 병리학자의 판단과 우리 방법의 결과를 비교 분석한다.In the present invention, we compare and analyze the results of our method with the judgment of a pathologist to verify the accuracy of the proposed method for quantifying emphysema and emphysema.
여기서, 각 샘플에 대한 폐기종의 비율을 정량화하기 위해 전문가의 자문을 받았다. 그 결과, 본 발명에 따른 방법이 step 12 ~ 15의 침식(erode) 과정에서 전문가의 정성적 연구와 가장 유사한 결과를 얻었다는 결론을 얻었다. 도 22 내지 도 23는 두 전문가의 정량화를 바탕으로 각 단계에서 정량화된 폐기종의 결과를 보여준다. x축은 샘플의 수를 의미하며, y축은 각 샘플에 대한 폐기종의 비율을 의미한다.Here, experts were consulted to quantify the proportion of emphysema for each sample. As a result, it was concluded that the method according to the present invention obtained results most similar to the expert's qualitative research in the erode process of steps 12 to 15. Figures 22 and 23 show the results of emphysema quantified at each stage based on quantification by two experts. The x-axis represents the number of samples, and the y-axis represents the proportion of emphysema for each sample.
도 22은 전문가 1(Expert 1)을 기준으로 한 정량 결과를 나타낸 것이다. 동일한 비율로 정량된 샘플을 그룹화하여 비교하였다. 전문가 1의 경우 샘플 1~2는 0%, 샘플 3~13은 1%, 샘플 14~20은 2%, 샘플 21~45는 5%, 샘플 46~ 59개는 10%, 샘플 60~64는 15%, 샘플 65~69는 20%, 샘플 70은 30%로 정량화했다.Figure 22 shows quantitative results based on Expert 1. Samples quantified at the same ratio were grouped and compared. For Expert 1, 0% for samples 1 to 2, 1% for samples 3 to 13, 2% for samples 14 to 20, 5% for samples 21 to 45, 10% for samples 46 to 59, and 1% for samples 60 to 64. It was quantified as 15%, samples 65 to 69 as 20%, and sample 70 as 30%.
도 23의 그래프를 보면 각 단계에서 정량화 결과에 대한 추세선의 기울기는 동일하였다. 그러나 전문가와 약간의 편차가 있다. 이는 침식(erode) 단계가 높을수록 폐기종의 검출 영역이 더 작기 때문이다. 그리고 전문가 1을 기준으로 우리의 결과를 비교할 때, 도 22(c)와 도 22(d)는 도 22(a)와 도 22(b) 보다 전문가 1의 결과로부터 더 적은 편차를 보였다. 도 22의 (a)와 (b) 에서는 추세선이 전문가의 결정 분포보다 높게 존재하고, 도 22의 (c)와 (d)에서는 추세선이 전문가의 결정 분포 위에 존재한다.Looking at the graph in Figure 23, the slope of the trend line for the quantification results at each stage was the same. However, there are some differences from experts. This is because the higher the erosion stage, the smaller the detection area for emphysema. And when comparing our results based on Expert 1, Figures 22(c) and 22(d) showed less deviation from the results of Expert 1 than Figures 22(a) and 22(b). In Figures 22 (a) and (b), the trend line exists higher than the expert's decision distribution, and in Figures 22 (c) and (d), the trend line exists above the expert's decision distribution.
도 23는 전문가 2를 기준으로 정량화된 결과를 보여준다. 도 23의 그래프를 분석하면, 동일한 비율로 정량화된 샘플을 그룹화하여 비교했다. 전문가 2의 경우 샘플 1은 2%, 샘플 2~11은 5%, 샘플 12~13은 7%, 샘플 14~26은 10%, 샘플 27~35는 10% 로 정량화 하였다. 15%로 정량화, 샘플 36~54는 20%, 샘플 55~59는 25%, 샘플 60~67은 30%, 샘플 68~69는 35%, 샘플 70은 50%으로 정량화 하였다. Figure 23 shows the quantified results based on expert 2. Analyzing the graph in Figure 23, samples quantified at the same ratio were grouped and compared. For Expert 2, Sample 1 was quantified as 2%, Samples 2 to 11 as 5%, Samples 12 to 13 as 7%, Samples 14 to 26 as 10%, and Samples 27 to 35 as 10%. It was quantified as 15%, samples 36 to 54 were 20%, samples 55 to 59 were 25%, samples 60 to 67 were 30%, samples 68 to 69 were 35%, and sample 70 was 50%.
도 23의 그래프 에서 각 단계의 정량화 결과에 대한 추세선의 기울기는 동일하였다. 그러나 전문가 2는 표본을 높게 정량화하는 경향이 있어 우리와 상당한 편차를 보인다. 전문가 2는 도 23의 (a)와 (b)보다 (c)와 (d)에서 더 큰 편차를 보였다. 특히, 도 23의 (c)와 (d)를 보면 전문가들이 정량화한 비율이 20 이상의 범위에서 본 발명에 따른 정량화 값과 크게 차이가 나는 것을 알 수 있다. 반대로, 도 23의 (a)와 (b)는 전문가 정량화 비율이 20 미만의 범위에서 우리로부터 가장 작은 편차를 가짐을 보여준다.In the graph of Figure 23, the slope of the trend line for the quantification results of each stage was the same. However, Expert 2 tends to quantify the sample highly, showing significant deviation from us. Expert 2 showed greater deviation in (c) and (d) than in (a) and (b) of Figure 23. In particular, looking at (c) and (d) of Figure 23, it can be seen that the ratio quantified by experts is significantly different from the quantification value according to the present invention in the range of 20 or more. Conversely, Figure 23(a) and (b) show that the expert quantification ratio has the smallest deviation from us in the range of less than 20.
수치해석을 위해 전문가의 평균, 분산, 표준편차를 도 24과 같이 계산하였다. 전문가 1은 각 단계가 증가함에 따라 편차가 감소하는 반면 전문가 2는 각 단계가 증가함에 따라 편차가 증가하는 것으로 나타났다. 이를 통해 전문가 1은 폐기종의 정도를 낮게, 전문가 2는 높게 수량화했음을 알 수 있다. 또한, 전문가 1은 step 14, 전문가 2는 step 12에서 본 발명에 따른 결과로부터 가장 작은 편차를 보였다. 본 발명에서는 모든 샘플에 대해 동일한 방식으로 폐기종을 검출하고 절대 기준으로 정량화했다. 따라서 아래는 본 발명에 따른 결과를 표준 정량화로 가정하고 전문가 정량화 결과를 비교 분석하였다.For numerical analysis, the expert's mean, variance, and standard deviation were calculated as shown in Figure 24. Expert 1 showed that the deviation decreased with each step, while expert 2 showed that the deviation increased with each step. Through this, it can be seen that Expert 1 quantified the degree of emphysema as low, and Expert 2 quantified it as high. Additionally, Expert 1 showed the smallest deviation from the results according to the present invention at step 14 and Expert 2 at step 12. In the present invention, emphysema was detected in the same way for all samples and quantified on an absolute basis. Therefore, below, the results according to the present invention are assumed to be standard quantification and the results of expert quantification are compared and analyzed.
4) 폐기종의 정량 결과 분석4) Quantitative result analysis of emphysema
전문가에 의해 정량화된 폐기종의 비율을 분석하기 위해 본 발명에서 정량화한 폐기종의 비율을 기준으로 전문가와의 편차를 통해 분석하였다. 폐기종의 정량화와 동일한 방법을 적용하여 모든 샘플을 정량화했다. 도 25은 슬라이딩 윈도우 방법을 사용하여 각 폐기종 비율 범위의 평균 편차를 계산했다. 또한 10개의 창에 5개의 슬라이딩 창으로 슬라이딩 창을 수행하였다.In order to analyze the proportion of emphysema quantified by experts, the difference from the expert was analyzed based on the proportion of emphysema quantified in the present invention. All samples were quantified by applying the same method as for quantification of emphysema. Figure 25 calculates the mean deviation for each emphysema rate range using the sliding window method. Additionally, sliding windows were performed with 5 sliding windows in 10 windows.
폐기종을 정량화할 때 두 전문가는 낮은 범위에서는 우리와의 편차가 상대적으로 작았지만 범위가 높을수록 우리와의 편차가 크다는 것을 확인했다. 도 25의 (a)를 참조하면, 전문가 1은 폐기종 범위 0-20에서 오차가 6보다 작았으나 20을 초과하는 지점에서 점차 오차가 6 이상으로 증가함을 확인하였다. 유사하게 도 25(b) 에서 모든 단계의 결과를 고려할 때 낮은 범위에서는 편차가 작지만 높은 범위에서는 편차가 증가함을 알 수 있다. 폐기종 비율이 높을수록 폐기종이 의심되는 경우가 많으며, 이는 두 전문가가 높은 섹션에서 편차가 더 크기 때문에 문제가 될 수 있다. 그리고 모든 표본에 동일한 방법을 적용하여 정량화한 결과, 전문가들은 다른 표본에 대해 동일한 비율을 정량화하더라도 전문가가 다른 것으로 나타났다.When quantifying emphysema, the two experts confirmed that the deviation from us was relatively small at low ranges, but the deviation from us was large at higher ranges. Referring to (a) of Figure 25, Expert 1 confirmed that the error was less than 6 in the emphysema range of 0-20, but that the error gradually increased to 6 or more at the point exceeding 20. Similarly, when considering the results of all steps in Figure 25(b), it can be seen that the deviation is small in the low range, but the deviation increases in the high range. The higher the emphysema rate, the more often emphysema is suspected, which can be problematic because the two experts have greater variation in the higher sections. And as a result of quantification by applying the same method to all samples, it was found that experts were different even if they quantified the same ratio for different samples.
다음 도 26 및 도 27에서, 각 샘플에 대해 전문가가 정량화한 폐기종의 비율을 기반으로 각 단계에서 정량화된 결과에서 가장 작은 편차가 있는 단계를 선택한다. In the following Figures 26 and 27, the stage with the smallest deviation from the quantified results at each stage is selected based on the proportion of emphysema quantified by the expert for each sample.
도 26를 참조하면, 총 70개의 샘플 중 전문가 1이 10개 미만 범위에서 45개, 10개 이상 20개 미만 범위에서 19개, 20개 초과 범위에서 6개를 정량화한 것을 알 수 있다. 10 미만의 범위 내에서 전문가가 정량화 결과와 가장 유사하게 정량화한 단계는 15단계로 33개 샘플이 이에 해당한다. 그리고 10개 이상 20개 미만의 범위에서 정량화 결과와 가장 유사한 정량화 단계는 step 12/로 8개의 샘플이 이에 해당한다. 그리고 20개 이상의 범위에서는 전문가가 정량화한 표본의 수가 적기 때문에 분석하기에 충분하지 않다고 판단되어 생략한다.Referring to Figure 26, it can be seen that out of a total of 70 samples, Expert 1 quantified 45 in the range of less than 10, 19 in the range of 10 to less than 20, and 6 in the range of more than 20. Within the range of less than 10, the quantification level most similar to the quantification result by experts was level 15, which corresponds to 33 samples. And the quantification step most similar to the quantification result in the range of 10 to 20 is step 12/, which corresponds to 8 samples. In addition, in the range of more than 20, the number of samples quantified by experts is small, so it is judged to be insufficient for analysis and is omitted.
도 27을 참조하면, 총 70개의 샘플 중 전문가 2는 10 미만 범위에서 13개, 10개 이상 20개 미만 범위에서 22개, 20개 초과 범위에서 35개를 정량화한 것을 알 수 있다. 10 미만의 범위 내에서 전문가 2가 정량화 결과와 가장 유사하게 정량화한 단계는 step 15로 7개의 샘플이 이에 해당한다. 그리고 10개 이상 20개 미만의 범위에서 정량화 결과와 가장 유사한 정량화 단계는 step 13으로 15개 샘플이 이에 해당한다. 마지막으로 20개 이상의 범위에서 전문가와 가장 유사하게 정량화하는 단계는 step 12로 25개 샘플이 이에 해당한다.Referring to Figure 27, it can be seen that out of a total of 70 samples, Expert 2 quantified 13 in the range of less than 10, 22 in the range of 10 to 20, and 35 in the range of more than 20. The quantification step that was most similar to the quantification result by expert 2 within a range of less than 10 was step 15, which corresponds to 7 samples. And the quantification step most similar to the quantification result in the range of 10 to 20 is step 13, which corresponds to 15 samples. Lastly, the step that quantifies in a range of 20 or more most similar to that of experts is step 12, which corresponds to 25 samples.
다음 분석에 따르면 전문가 1은 전체 샘플의 절반 이상을 낮은 속도로 정량화하는 경향이 있었고 전문가 2는 전체 샘플의 절반 이상을 높은 속도로 정량화하는 경향이 있었다. 그리고 두 전문가는 낮은 비율로 step 15의 결과와 높은 비율로 step 12의 결과와 가장 유사했다. step이 높을수록 감지할 폐기종의 크기가 커진다. 따라서 전문가들은 폐기종을 폐기종의 발생률이 낮은 표본의 비교적 큰 구멍으로 간주한다. 또한, 폐기종의 발병률이 높은 표본의 비교적 작은 구멍을 폐기종으로 간주한다.According to the following analysis, Expert 1 tended to quantify more than half of all samples at a low rate, and Expert 2 tended to quantify more than half of all samples at a high rate. And the two experts were most similar to the results of step 15 at a low rate and the results of step 12 at a high rate. The higher the step, the larger the size of emphysema to be detected. Therefore, experts consider emphysema to be relatively large cavities in specimens with a low incidence of emphysema. Additionally, relatively small cavities in specimens with a high incidence of emphysema are considered to be emphysema.
이를 통해 전문가들은 폐기종 비율이 낮은 표본과 폐기종 비율이 높은 표본에 따라 전문가의 정성적 판단이 다를 수 있음을 확인했다. 또한, 절대기준에서 정량화된 결과를 바탕으로 전문가의 정량화 결과를 비교하였다. 그 결과 경험과 직관을 바탕으로 정량화를 진행했기 때문에 결과가 다를 수 있다고 전문가들은 판단했다. 따라서 폐 샘플에서 폐기종의 절대 정량화를 기반으로 인간의 경험과 직관에 기반한 정량화 판단을 수정할 수 있는 도구를 제안한다.Through this, experts confirmed that qualitative judgments of experts may differ depending on samples with a low rate of emphysema and samples with a high rate of emphysema. In addition, the experts' quantification results were compared based on the results quantified on an absolute standard. As a result, experts determined that the results may vary because the quantification was based on experience and intuition. Therefore, based on absolute quantification of emphysema in lung samples, we propose a tool that can correct quantification judgments based on human experience and intuition.
상기 과제의 해결 수단에 의해, 본 발명은 머신러닝을 이용하여 폐 표본의 폐질환의 절대 기준에 따른 정량화 방법을 제시할 수 있다.As a means of solving the above problem, the present invention can present a method for quantifying lung disease in lung samples according to absolute standards using machine learning.
또한, 본 발명은 폐 표본에서 폐질환의 절대 정량화를 기반으로 인간의 경험과 직관에 기반 한 정량화 결정을 수정할 수 있는 시스템을 제공한다.Additionally, the present invention provides a system that can modify quantification decisions based on human experience and intuition based on absolute quantification of lung disease in lung specimens.
이와 같이, 상술한 본 발명의 기술적 구성은 본 발명이 속하는 기술분야의 당업자가 본 발명의 그 기술적 사상이나 필수적 특징을 변경하지 않고서 다른 구체적인 형태로 실시될 수 있다는 것을 이해할 수 있을 것이다.As such, a person skilled in the art will understand that the technical configuration of the present invention described above can be implemented in other specific forms without changing the technical idea or essential features of the present invention.
그러므로 이상에서 기술한 실시예들은 모든 면에서 예시적인 것이며 한정적인 것이 아닌 것으로서 이해되어야 하고, 본 발명의 범위는 상기 상세한 설명보다는 후술하는 특허청구범위에 의하여 나타나며, 특허청구범위의 의미 및 범위 그리고 그 등가 개념으로부터 도출되는 모든 변경 또는 변형된 형태가 본 발명의 범위에 포함되는 것으로 해석되어야 한다.Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive, and the scope of the present invention is indicated by the claims described later rather than the detailed description above, and the meaning and scope of the claims and their All changes or modified forms derived from the equivalent concept should be construed as falling within the scope of the present invention.
마찬가지로, 상기 부호의 설명을 아래와 같이 정리한다.Likewise, the explanation of the above symbols is organized as follows.
1. 폐기종 정량화 시스템 1. Emphysema Quantification System
10. 이미지획득부10. Image acquisition department
20. 세기관지제거부20. Bronchiole removal unit
30. 이진화처리부30. Binarization processing unit
40. 공기층계산부40. Air layer calculation unit
41. 레이블지정부41. Labeling department
42. 공기층픽셀추출부42. Air layer pixel extraction unit
50. 폐포제거부50. Alveolar removal unit
60. 좌표확인부60. Coordinate confirmation unit
61. 폐기종지정부61. Emphysema Designation Department
62. 중심좌표추출부62. Central coordinate extraction unit
70. 폐기종감지부70. Emphysema detection unit
71. 이미지맵핑부71. Image mapping department
72. 폐기종색지정부72. Department of Emphysema and Colorectal Designation
80. 폐기종계산부80. Emphysema calculator
81. 폐기종영역지정부81. Emphysema Area Designation Department
82. 폐기종픽셀추출부82. Emphysema pixel extraction unit
90. 폐기종정량부90. Emphysema Quantification Department
S10. 이미지획득부(10)가 폐를 생검하여 현미경 촬영한 폐이미지를 획득하는 이미지 획득 단계S10. An image acquisition step in which the image acquisition unit 10 biopsies the lung and acquires a lung image taken with a microscope.
S20. 세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계S20. A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole.
S30. 이진화처리부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리하는 이진화 처리 단계S30. A binarization processing step in which the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed.
S40. 공기층계산부(40)가 상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산하는 공기층 계산 단계S40. An air layer calculation step in which the air layer calculation unit 40 calculates the area of the entire air layer from the binarized lung image.
S41. 레이블지정부(41)가 상기 이진화(binarization) 처리 된 폐이미지에서 레이블(label)을 지정하는 레이블 지정 단계S41. A labeling step in which the labeling unit 41 assigns a label to the binarized waste image.
S42. 공기층픽셀추출부(42)가 상기 지정된 레이블(label)의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출하는 공기층 픽셀 추출 단계S42. An air layer pixel extraction step in which the air layer pixel extraction unit 42 extracts the entire air layer pixel area by calculating the sum of the pixel areas of the designated label.
S50. 폐포제거부(50)가 상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출하는 폐포 제거 단계S50. An alveolar removal step in which the alveolar removal unit 50 removes the air layer of the alveoli and extracts emphysema from the binarized lung image.
S60. 좌표확인부(60)가 상기 추출된 폐기종의 좌표를 확인하는 좌표 확인 단계S60. Coordinate confirmation step in which the coordinate confirmation unit 60 confirms the coordinates of the extracted emphysema.
S61. 폐기종지정부(61)가 상기 이진화(binarization) 처리 된 폐이미지에서 추출된 폐기종에 레이블(label)을 지정하는 폐기종 지정 단계S61. An emphysema designation step in which the emphysema designation unit 61 assigns a label to the emphysema extracted from the binarized lung image.
S62. 중심좌표추출부(62)가 상기 지정된 폐기종 레이블(label)에서 중심 좌표를 추출하는 중심 좌표 추출 단계S62. A center coordinate extraction step in which the center coordinate extraction unit 62 extracts the center coordinate from the designated emphysema label.
S70. 폐기종감지부(70)가 상기 좌표가 확인 된 폐기종을 감지하는 폐기종 감지 단계S70. Emphysema detection step in which the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed.
S71. 이미지맵핑부(71)가 상기 이미지획득부(10)에서 획득한 폐이미지에 상기 좌표확인부(60)에서 확인 된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑하는 이미지 맵핑 단계S71. An image mapping step in which the image mapping unit 71 maps the center coordinates of emphysema identified in the coordinate confirmation unit 60 to the lung image acquired by the image acquisition unit 10 to the original lung image.
S72. 폐기종색지정부(72)가 상기 폐기종의 중심 좌표를 기반으로 상기 폐기종 영역에 색을 지정하는 폐기종 색 지정 단계S72. An emphysema color designation step in which the emphysema color designation unit 72 assigns a color to the emphysema area based on the coordinates of the center of the emphysema.
S80. 폐기종계산부(80)가 상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종 계산 단계S80. Emphysema calculation step in which the emphysema calculation unit 80 extracts the detected emphysema and calculates the area.
S81. 폐기종영역지정부(81)가 상기 폐기종감지부(70)에 의해 감지된 폐기종 영역을 확인 후, 상기 폐기종 영역에 레이블을 지정하는 폐기종 영역 지정 단계S81. An emphysema area designation step in which the emphysema area designation unit 81 confirms the emphysema area detected by the emphysema detection unit 70 and then labels the emphysema area.
S82. 폐기종픽셀추출부(82)가 상기 레이블이 지정된 폐기종 영역의 픽셀 합을 계산하여 폐기종 픽셀의 영역을 추출하는 폐기종 픽셀 추출 단계S82. An emphysema pixel extraction step in which the emphysema pixel extraction unit 82 extracts the area of the emphysema pixel by calculating the sum of pixels of the labeled emphysema area.
S90. 폐기종정량부(90)가 상기 추출된 폐기종의 폐기종율을 정량화하는 폐기종 정량 단계S90. Emphysema quantification step in which the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.

Claims (34)

  1. 폐를 생검하여 현미경 촬영을 통해 폐이미지 획득하는 이미지획득부(10);An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging;
    상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지제거부(20);A bronchiole removal unit (20) that recognizes bronchiole in the lung image and then removes the bronchiole;
    상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출하는 폐간질추출부(30);A lung interstitium extraction unit 30 that extracts a lung interstitium area from the lung image from which the bronchioles have been removed;
    상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성하는 분할이미지생성부(40);a segmented image generator 40 that divides the lung image from which the lung interstitium area is extracted and generates a segmented image;
    상기 분할이미지에서 세포핵(cell nucleus)을 인식하는 세포핵인식부(50);A cell nucleus recognition unit 50 that recognizes a cell nucleus in the segmented image;
    상기 인식된 세포핵(cell nucleus)의 개수를 집계하는 세포핵집계부(60); A cell nucleus counting unit 60 that counts the number of recognized cell nuclei;
    상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus) 분포도를 생성하는 분포도생성부(70);a distribution map generator 70 that visualizes lung images according to the counted number of cell nuclei and generates a cell nucleus distribution map;
    상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교하는 임계값비교부(80);a threshold comparison unit 80 that compares the counted number of cell nuclei with a threshold value;
    상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단하는 폐렴판단부(90);a pneumonia determination unit 90 that determines pneumonia when the number of cell nuclei counted is high compared to the threshold;
    상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산하는 면적계산부(100);an area calculation unit 100 that calculates a lung interstitium area in the segmented image determined to be pneumonia;
    폐의 폐렴율을 산출하는 폐렴율산출부(110);를 포함하는 것을 특징으로 하는,Characterized by comprising a pneumonia rate calculation unit 110 that calculates the pneumonia rate of the lung,
    컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A system for quantifying pneumonia in lung specimens using computer vision and machine learning.
  2. 제 1항에 있어서,According to clause 1,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A pneumonia quantification system for lung specimens using computer vision and machine learning, characterized in that the lung image is divided into a certain size, a partition is set, and some of the set partitions are labeled as bronchial regions.
  3. 제 2항에 있어서,According to clause 2,
    상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중,Among the partitions not labeled as the bronchial region among the set partitions,
    일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A system for quantifying pneumonia in lung samples using computer vision and machine learning, where some are used as learning data for learning and the rest are used as test data for testing.
  4. 제 2항에 있어서,According to clause 2,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A pneumonia quantification system for lung specimens using computer vision and machine learning, characterized in that after the labeling of the bronchiolar region is completed, the partitions to be divided are re-gathered to regenerate one lung image.
  5. 제 1항에 있어서,According to clause 1,
    상기 폐간질추출부(30)는,The lung interstitial extraction unit 30,
    상기 폐간질(interstitium) 영역을 동일한 색상의 픽셀로 레이블링(labeling) 한 후 상기 레이블링(labeling) 된 픽셀 영역을 추출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A pneumonia quantification system for lung specimens using computer vision and machine learning, characterized in that the pulmonary interstitium area is labeled with pixels of the same color and then the labeled pixel area is extracted.
  6. 제 1항에 있어서,According to clause 1,
    상기 분할이미지생성부(40)는,The split image generator 40,
    상기 분할이미지의 너비와 길이가 동일하도록 마련하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템.A system for quantifying pneumonia in lung specimens using computer vision and machine learning, characterized in that the width and length of the segmented images are the same.
  7. 제 1항에 있어서,According to clause 1,
    상기 폐렴율산출부(110)는 아래의 [식 1]에 의해 상기 폐렴율을 산출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 시스템 :The pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below. A pneumonia quantification system for lung specimens using computer vision and machine learning:
    [식 1][Equation 1]
    폐렴율 = AP / AINT Pneumonia rate = A P / A INT
    (여기서, AP 는 상기 면적계산부(100)에서 계산된 폐간질 영역의 면적이고, AINT 는 폐간질추출부(30)에서 추출된 폐간질 영역의 면적).(Here, A P is the area of the pulmonary interstitial area calculated by the area calculation unit 100, and A INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30).
  8. 이미지획득부(10)가 폐를 생검한 현미경 촬영을 통해 폐이미지를 획득하는 이미지 획득 단계;An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
    세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계;A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
    폐간질추출부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지에서 폐간질(interstitium) 영역을 추출하는 폐간질 추출 단계; A pulmonary interstitium extraction step in which the pulmonary interstitium extraction unit 30 extracts a pulmonary interstitium area from the lung image from which the bronchioles have been removed;
    분할이미지생성부(40)가 상기 폐간질(interstitium) 영역이 추출된 폐이미지를 분할 후 분할이미지를 생성하는 분할이미지 생성 단계;A segmented image generation step in which the segmented image generator 40 divides the lung image from which the lung interstitium area is extracted and then generates a segmented image;
    세포핵인식부(50)가 상기 분할이미지에서 세포핵(cell nucleus)을 인식하는 세포핵 인식 단계;A cell nucleus recognition step in which the cell nucleus recognition unit 50 recognizes a cell nucleus in the segmented image;
    세포핵집계부(60)가 상기 인식된 세포핵(cell nucleus)의 개수를 집계하는 세포핵 집계 단계; A cell nucleus counting step in which the cell nucleus counting unit 60 counts the number of the recognized cell nuclei;
    분포도생성부(70)가 상기 집계된 세포핵(cell nucleus)의 개수에 따라 폐이미지를 시각화하여 세포핵(cell nucleus) 분포도를 생성하는 분포도 생성 단계;A distribution map generating step in which the distribution map generator 70 generates a cell nucleus distribution map by visualizing a lung image according to the counted number of cell nuclei;
    임계값비교부(80)가 상기 집계 된 세포핵(cell nucleus)의 개수와 임계값을 비교하는 임계값 비교 단계;A threshold comparison step in which the threshold comparison unit 80 compares the counted number of cell nuclei with a threshold value;
    폐렴판단부(90)가 상기 임계값과 비교하여 상기 집계 된 세포핵(cell nucleus)의 개수가 많은 경우 폐렴으로 판단하는 폐렴 판단 단계;A pneumonia determination step in which the pneumonia determination unit 90 determines pneumonia when the number of cell nuclei counted is high compared to the threshold;
    면적계산부(100)가 상기 폐렴으로 판단된 분할이미지에서 폐간질(interstitium) 영역을 계산하는 면적 계산 단계;An area calculation step in which the area calculation unit 100 calculates a lung interstitium area in the segmented image determined to be pneumonia;
    폐렴율산출부(110)가 폐의 폐렴율을 산출하는 폐렴율 산출 단계;를 포함하는 것을 특징으로 하는,Characterized in that it includes a pneumonia rate calculation step in which the pneumonia rate calculation unit 110 calculates the pneumonia rate of the lung.
    컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.Method for quantifying pneumonia in lung specimens using computer vision and machine learning.
  9. 제 8항에 있어서,According to clause 8,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.A method for quantifying pneumonia in a lung sample using computer vision and machine learning, characterized in that the lung image is divided into a certain size to set a partition, and some of the set partitions are labeled as a bronchial region.
  10. 제 9항에 있어서,According to clause 9,
    상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중,Among the partitions not labeled as the bronchial region among the set partitions,
    일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.A method for quantifying pneumonia in lung samples using computer vision and machine learning, characterized by using some as learning data for learning and using the rest as test data for testing.
  11. 제 9항에 있어서,According to clause 9,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.A method for quantifying pneumonia in a lung sample using computer vision and machine learning, characterized in that after the labeling of the bronchiolar region is completed, the partitions to be divided are re-gathered to regenerate one lung image.
  12. 제 8항에 있어서,According to clause 8,
    상기 폐간질추출부(30)는,The lung interstitial extraction unit 30,
    상기 폐간질(interstitium) 영역을 동일한 색상의 픽셀로 레이블링(labeling) 한 후 상기 레이블링(labeling) 된 픽셀 영역을 추출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.A method for quantifying pneumonia in a lung specimen using computer vision and machine learning, characterized in that the lung interstitium area is labeled with pixels of the same color and then the labeled pixel area is extracted.
  13. 제 8항에 있어서,According to clause 8,
    상기 분할이미지생성부(40)는,The split image generator 40,
    상기 분할이미지의 너비와 길이가 동일하도록 마련하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법.A method for quantifying pneumonia in a lung specimen using computer vision and machine learning, characterized in that the width and length of the segmented image are the same.
  14. 제 8항에 있어서,According to clause 8,
    상기 폐렴율산출부(110)는 아래의 [식 1]에 의해 상기 폐렴율을 산출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐렴 정량화 방법 :The pneumonia rate calculation unit 110 calculates the pneumonia rate according to [Equation 1] below. Method for quantifying pneumonia in a lung sample using computer vision and machine learning:
    [식 1][Equation 1]
    폐렴율 = AP / AINT Pneumonia rate = A P / A INT
    (여기서, AP 는 상기 면적계산부(100)에서 계산된 폐간질 영역의 면적이고, AINT 는 폐간질추출부(30)에서 추출된 폐간질 영역의 면적).(Here, A P is the area of the pulmonary interstitial area calculated by the area calculation unit 100, and A INT is the area of the pulmonary interstitial area extracted by the pulmonary interstitial extraction unit 30).
  15. 폐를 생검하여 현미경 촬영을 통해 폐이미지 획득하는 이미지획득부(10);An image acquisition unit (10) that biopsies the lung and acquires a lung image through microscopic imaging;
    상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지제거부(20);A bronchiole removal unit (20) that recognizes bronchiole in the lung image and then removes the bronchiole;
    상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리하는 이진화처리부(30);A binarization processing unit 30 that binarizes the lung image from which the bronchioles have been removed;
    상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산하는 공기층계산부(40);An air layer calculation unit 40 that calculates the area of the entire air layer in the binarized lung image;
    상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출하는 폐포제거부(50);an alveolar removal unit 50 that removes the air layer of alveoli and extracts emphysema from the binarized lung image;
    상기 추출된 폐기종의 좌표를 확인하는 좌표확인부(60);A coordinate confirmation unit 60 that checks the extracted coordinates of emphysema;
    상기 좌표가 확인 된 폐기종을 감지하는 폐기종감지부(70);An emphysema detection unit 70 that detects emphysema whose coordinates have been confirmed;
    상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종계산부(80);An emphysema calculator 80 that extracts the detected emphysema and calculates the area;
    상기 추출된 폐기종의 폐기종율을 정량화하는 폐기종정량부(90);를 포함하는 것을 특징으로 하는,Characterized in that it includes an emphysema quantification unit (90) that quantifies the emphysema rate of the extracted emphysema.
    컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.A system for quantifying emphysema in lung samples using computer vision and machine learning.
  16. 제 15항에 있어서,According to clause 15,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.An emphysema quantification system for lung specimens using computer vision and machine learning, characterized in that the lung image is divided into a certain size, a partition is set, and some of the set partitions are labeled as bronchial regions.
  17. 제 16항에 있어서,According to clause 16,
    상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중,Among the partitions not labeled as the bronchial region among the set partitions,
    일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.A system for quantifying emphysema in lung samples using computer vision and machine learning, where some are used as learning data for learning and the rest are used as test data for testing.
  18. 제 16항에 있어서,According to clause 16,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.Emphysema quantification system for lung specimens using computer vision and machine learning, characterized in that after labeling of the bronchiolar region is completed, the partitions to be divided are regathered to regenerate one lung image.
  19. 제 15항에 있어서,According to clause 15,
    상기 이진화처리부(30)는,The binarization processing unit 30,
    폐간질(interstitium) 및 세기관지(bronchiole) 영역은 제1색(1)으로 처리하고,The pulmonary interstitium and bronchiole areas are treated with the first color (1),
    폐포와 폐기종 영역은 제2색(0)으로 처리하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.Emphysema quantification system for lung specimens using computer vision and machine learning, characterized in that alveoli and emphysema areas are treated with a second color (0).
  20. 제 15항에 있어서,According to clause 15,
    상기 공기층계산부(40)는,The air layer calculation unit 40,
    상기 이진화(binarization) 처리 된 폐이미지에서 레이블(label)을 지정하는 레이블지정부(41); 및A labeling unit 41 that specifies a label on the binarized waste image; and
    상기 지정된 레이블(label)의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출하는 공기층픽셀추출부(42);로 구성되어 전체 공기층의 면적을 계산하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.An air layer pixel extraction unit 42 that extracts the entire air layer pixel area by calculating the sum of the pixel areas of the specified label; computer vision and machine learning, characterized in that it calculates the area of the entire air layer. Emphysema quantification system for used lung specimens.
  21. 제 15항에 있어서,According to clause 15,
    상기 좌표확인부(60)는,The coordinate confirmation unit 60,
    상기 이진화(binarization) 처리 된 폐이미지에서 추출된 폐기종에 레이블(label)을 지정하는 폐기종지정부(61); 및An emphysema designation unit 61 that assigns a label to emphysema extracted from the binarized lung image; and
    상기 지정된 폐기종 레이블(label)에서 중심 좌표를 추출하는 중심좌표추출부(62);로 구성되어 상기 추출된 폐기종의 좌표를 확인하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.Quantifying emphysema in lung specimens using computer vision and machine learning, characterized in that it consists of a central coordinate extraction unit 62 that extracts the central coordinates from the designated emphysema label and confirms the extracted coordinates of the emphysema. system.
  22. 제 15항에 있어서,According to clause 15,
    상기 폐기종감지부(70)는,The emphysema detection unit 70,
    상기 이미지획득부(10)에서 획득한 폐이미지에 상기 좌표확인부(60)에서 확인 된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑하는 이미지맵핑부(71); 및An image mapping unit 71 that maps the center coordinates of emphysema identified by the coordinate confirmation unit 60 to the lung image acquired by the image acquisition unit 10 to the original lung image; and
    상기 폐기종의 중심 좌표를 기반으로 상기 폐기종 영역에 색을 지정하는 폐기종색지정부(72);로 구성되어 상기 좌표가 확인 된 폐기종을 감지하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.An emphysema color designation unit (72) that specifies a color to the emphysema area based on the coordinates of the center of the emphysema, and is characterized in that it detects emphysema whose coordinates have been identified, of a lung specimen using computer vision and machine learning. Emphysema quantification system.
  23. 제 15항에 있어서,According to clause 15,
    상기 폐기종계산부(80)는,The emphysema calculator 80,
    상기 폐기종감지부(70)에 의해 감지된 폐기종 영역을 확인 후, 상기 폐기종 영역에 레이블을 지정하는 폐기종영역지정부(81); 및An emphysema area designation unit (81) that confirms the emphysema area detected by the emphysema detection unit (70) and then assigns a label to the emphysema area; and
    상기 레이블이 지정된 폐기종 영역의 픽셀 합을 계산하여 폐기종 픽셀의 영역을 추출하는 폐기종픽셀추출부(82);로 구성되어 폐기종을 추출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템.An emphysema pixel extraction unit 82 that extracts the area of the emphysema pixel by calculating the pixel sum of the labeled emphysema area, and extracts the emphysema. Emphysema of the lung specimen using computer vision and machine learning. Quantification system.
  24. 제 15항에 있어서,According to clause 15,
    상기 폐기종정량부(90)는 아래의 [식 1]에 의해 상기 폐기종율을 정량화하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 시스템 :The emphysema quantification unit 90 is a system for quantifying emphysema in lung samples using computer vision and machine learning, wherein the emphysema quantification unit 90 quantifies the emphysema rate according to [Equation 1] below:
    [식 1][Equation 1]
    폐기종율 = Aa / Aemp Emphysema rate = A a / A emp
    (여기서, Aa 는 상기 공기층계산부(40)에 의해 계산된 전체 공기층의 면적이고, Aemp 는 Aa 와 상기 폐기종계산부(80)에 의해 계산된 상기 폐기종 면적의 합).(Here, A a is the area of the total air layer calculated by the air layer calculator 40, and A emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
  25. 이미지획득부(10)가 폐를 생검한 현미경 촬영을 통해 폐이미지를 획득하는 이미지 획득 단계;An image acquisition step in which the image acquisition unit 10 acquires a lung image through microscopic imaging of a lung biopsy;
    세기관지제거부(20)가 상기 폐이미지에서 세기관지(bronchiole)를 인식 후 상기 세기관지(bronchiole)를 제거하는 세기관지 제거 단계;A bronchiole removal step in which the bronchiole removal unit 20 recognizes a bronchiole in the lung image and then removes the bronchiole;
    이진화처리부(30)가 상기 세기관지(bronchiole)가 제거된 폐이미지를 이진화(binarization) 처리하는 이진화 처리 단계;A binarization processing step in which the binarization processing unit 30 binarizes the lung image from which the bronchioles have been removed;
    공기층계산부(40)가 상기 이진화(binarization) 처리 된 폐이미지에서 전체 공기층의 면적을 계산하는 공기층 계산 단계;An air layer calculation step in which the air layer calculation unit 40 calculates the area of the entire air layer in the binarized lung image;
    폐포제거부(50)가 상기 이진화(binarization) 처리 된 폐이미지에서 폐포의 공기층을 제거하고 폐기종을 추출하는 폐포 제거 단계;An alveolar removal step in which the alveolar removal unit 50 removes the air layer of the alveoli and extracts emphysema from the binarized lung image;
    좌표확인부(60)가 상기 추출된 폐기종의 좌표를 확인하는 좌표 확인 단계;A coordinate confirmation step in which the coordinate confirmation unit 60 confirms the extracted coordinates of emphysema;
    폐기종감지부(70)가 상기 좌표가 확인 된 폐기종을 감지하는 폐기종 감지 단계;An emphysema detection step in which the emphysema detection unit 70 detects emphysema whose coordinates have been confirmed;
    폐기종계산부(80)가 상기 감지된 폐기종을 추출 후 면적을 계산하는 폐기종 계산 단계;An emphysema calculation step in which the emphysema calculation unit 80 extracts the detected emphysema and calculates an area;
    폐기종정량부(90)가 상기 추출된 폐기종의 폐기종율을 정량화하는 폐기종 정량 단계;를 포함하는 것을 특징으로 하는,Characterized in that it includes an emphysema quantification step in which the emphysema quantification unit 90 quantifies the emphysema rate of the extracted emphysema.
    컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.Method for quantifying emphysema in lung specimens using computer vision and machine learning.
  26. 제 25항에 있어서,According to clause 25,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 폐이미지를 일정 크기로 분할하여 파티션을 설정하고, 상기 설정된 파티션 중 일부에 세기관지영역으로 레이블링(labeling) 하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.A method for quantifying emphysema in a lung sample using computer vision and machine learning, characterized in that the lung image is divided into a certain size, a partition is set, and some of the set partitions are labeled as a bronchial region.
  27. 제 26항에 있어서,According to clause 26,
    상기 설정된 파티션 중에서 상기 세기관지영역으로 레이블링(labeling) 되지 않은 파티션 중,Among the partitions not labeled as the bronchial region among the set partitions,
    일부는 학습을 위한 학습데이터로 사용하고, 나머지는 테스트를 위한 테스트데이터로 사용하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.A method for quantifying emphysema in lung samples using computer vision and machine learning, characterized by using some as learning data for learning and using the rest as test data for testing.
  28. 제 26항에 있어서,According to clause 26,
    상기 세기관지제거부(20)는,The bronchioles removal unit (20),
    상기 세기관지영역 레이블링(labeling)이 완료된 후 상기 분할될 파티션을 다시 모아 하나의 폐이미지를 재생성하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.A method for quantifying emphysema in a lung specimen using computer vision and machine learning, characterized in that after the labeling of the bronchiolar region is completed, the partitions to be divided are regathered to regenerate a single lung image.
  29. 제 25항에 있어서,According to clause 25,
    상기 이진화처리부(30)는,The binarization processing unit 30,
    폐간질(interstitium) 및 세기관지(bronchiole) 영역은 제1색(1)으로 처리하고,The pulmonary interstitium and bronchiole areas are treated with the first color (1),
    폐포와 폐기종 영역은 제2색(0)으로 처리하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.A method for quantifying emphysema in lung specimens using computer vision and machine learning, characterized in that the alveoli and emphysema areas are treated with a second color (0).
  30. 제 25항에 있어서,According to clause 25,
    상기 공기층계산부(40)는,The air layer calculation unit 40,
    레이블지정부(41)가 상기 이진화(binarization) 처리 된 폐이미지에서 레이블(label)을 지정하는 레이블 지정 단계; 및A labeling step in which the labeling unit 41 assigns a label to the binarized waste image; and
    공기층픽셀추출부(42)가 상기 지정된 레이블(label)의 픽셀 영역의 합을 계산하여 전체 공기층 픽셀 영역을 추출하는 공기층 픽셀 추출 단계;로 구성되어 전체 공기층의 면적을 계산하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.An air layer pixel extraction step in which the air layer pixel extraction unit 42 extracts the entire air layer pixel area by calculating the sum of the pixel areas of the designated label, and calculating the area of the entire air layer. Method for quantifying emphysema in lung specimens using vision and machine learning.
  31. 제 25항에 있어서,According to clause 25,
    상기 좌표확인부(60)는,The coordinate confirmation unit 60,
    폐기종지정부(61)가 상기 이진화(binarization) 처리 된 폐이미지에서 추출된 폐기종에 레이블(label)을 지정하는 폐기종 지정 단계; 및An emphysema designation step in which the emphysema designation unit 61 assigns a label to emphysema extracted from the binarized lung image; and
    중심좌표추출부(62)가 상기 지정된 폐기종 레이블(label)에서 중심 좌표를 추출하는 중심 좌표 추출 단계;로 구성되어 상기 추출된 폐기종의 좌표를 확인하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.A central coordinate extraction step in which the central coordinate extraction unit 62 extracts the central coordinates from the designated emphysema label, and confirming the extracted coordinates of the emphysema using computer vision and machine learning. Method for quantifying emphysema in lung specimens.
  32. 제 25항에 있어서,According to clause 25,
    상기 폐기종감지부(70)는,The emphysema detection unit 70,
    이미지맵핑부(71)가 상기 이미지획득부(10)에서 획득한 폐이미지에 상기 좌표확인부(60)에서 확인 된 폐기종의 중심 좌표를 원본 폐이미지에 맵핑하는 이미지 맵핑 단계; 및An image mapping step in which the image mapping unit 71 maps the center coordinates of emphysema identified by the coordinate confirmation unit 60 to the lung image acquired by the image acquisition unit 10 to the original lung image; and
    폐기종색지정부(72)가 상기 폐기종의 중심 좌표를 기반으로 상기 폐기종 영역에 색을 지정하는 폐기종 색 지정 단계;로 구성되어 상기 좌표가 확인 된 폐기종을 감지하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.An emphysema color designation step in which the emphysema color designation unit 72 assigns a color to the emphysema area based on the center coordinates of the emphysema; computer vision and machine learning, characterized in that detecting emphysema for which the coordinates have been identified. Method for quantifying emphysema in lung specimens using .
  33. 제 25항에 있어서,According to clause 25,
    상기 폐기종계산부(80)는,The emphysema calculator 80,
    폐기종영역지정부(81)가 상기 폐기종감지부(70)에 의해 감지된 폐기종 영역을 확인 후, 상기 폐기종 영역에 레이블을 지정하는 폐기종 영역 지정 단계; 및An emphysema area designation step in which the emphysema area designation unit 81 confirms the emphysema area detected by the emphysema detection unit 70 and then assigns a label to the emphysema area; and
    폐기종픽셀추출부(82)가 상기 레이블이 지정된 폐기종 영역의 픽셀 합을 계산하여 폐기종 픽셀의 영역을 추출하는 폐기종 픽셀 추출 단계;로 구성되어 폐기종을 추출하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법.An emphysema pixel extraction step in which the emphysema pixel extraction unit 82 extracts the area of the emphysema pixel by calculating the sum of pixels of the labeled emphysema area; computer vision and machine learning, characterized in that the emphysema is extracted. Methods for quantifying emphysema in lung specimens used.
  34. 제 25항에 있어서,According to clause 25,
    상기 폐기종정량부(90)는 아래의 [식 1]에 의해 상기 폐기종율을 정량화하는 것을 특징으로 하는, 컴퓨터 비전과 머신러닝을 사용한 폐 검체의 폐기종 정량화 방법 :The emphysema quantification unit 90 is a method for quantifying emphysema in a lung sample using computer vision and machine learning, wherein the emphysema quantification unit 90 quantifies the emphysema rate according to [Equation 1] below:
    [식 1][Equation 1]
    폐기종율 = Aa / Aemp Emphysema rate = A a / A emp
    (여기서, Aa 는 상기 공기층계산부(40)에 의해 계산된 전체 공기층의 면적이고, Aemp 는 Aa 와 상기 폐기종계산부(80)에 의해 계산된 상기 폐기종 면적의 합).(Here, A a is the area of the total air layer calculated by the air layer calculator 40, and A emp is the sum of A a and the emphysema area calculated by the emphysema calculator 80).
PCT/KR2023/020600 2022-12-14 2023-12-14 System for quantifying lung disease of lung sample by using computer vision and machine learning, and method for quantifying lung disease by using system WO2024128818A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR1020220174910A KR20240092297A (en) 2022-12-14 2022-12-14 System for quantifying emphysema in lung samples using computer vision and machine learning and method for quantifying emphysema thereof
KR1020220174559A KR20240091538A (en) 2022-12-14 2022-12-14 System for quantifying pneumonia in lung samples using computer vision and machine learning and method for quantifying pneumonia thereof
KR10-2022-0174559 2022-12-14
KR10-2022-0174910 2022-12-14

Publications (1)

Publication Number Publication Date
WO2024128818A1 true WO2024128818A1 (en) 2024-06-20

Family

ID=91486077

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2023/020600 WO2024128818A1 (en) 2022-12-14 2023-12-14 System for quantifying lung disease of lung sample by using computer vision and machine learning, and method for quantifying lung disease by using system

Country Status (1)

Country Link
WO (1) WO2024128818A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101794578B1 (en) * 2017-06-07 2017-11-07 (주)크레아소프트 Method for predicting disease, recording medium and apparatus for performing the method
KR20190068607A (en) * 2016-10-21 2019-06-18 난토믹스, 엘엘씨 Digital histopathology and dissection of brown rice
KR20190071724A (en) * 2016-10-12 2019-06-24 테라리콘, 인코포레이티드 System and method for medical image interpretation
JP2020018705A (en) * 2018-08-02 2020-02-06 キヤノンメディカルシステムズ株式会社 Medical image processing device, image formation method and image formation program
US20200388029A1 (en) * 2017-11-30 2020-12-10 The Research Foundation For The State University Of New York System and Method to Quantify Tumor-Infiltrating Lymphocytes (TILs) for Clinical Pathology Analysis Based on Prediction, Spatial Analysis, Molecular Correlation, and Reconstruction of TIL Information Identified in Digitized Tissue Images
KR20220085000A (en) * 2020-12-14 2022-06-21 주식회사 뷰노 Method to read lesion

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190071724A (en) * 2016-10-12 2019-06-24 테라리콘, 인코포레이티드 System and method for medical image interpretation
KR20190068607A (en) * 2016-10-21 2019-06-18 난토믹스, 엘엘씨 Digital histopathology and dissection of brown rice
KR101794578B1 (en) * 2017-06-07 2017-11-07 (주)크레아소프트 Method for predicting disease, recording medium and apparatus for performing the method
US20200388029A1 (en) * 2017-11-30 2020-12-10 The Research Foundation For The State University Of New York System and Method to Quantify Tumor-Infiltrating Lymphocytes (TILs) for Clinical Pathology Analysis Based on Prediction, Spatial Analysis, Molecular Correlation, and Reconstruction of TIL Information Identified in Digitized Tissue Images
JP2020018705A (en) * 2018-08-02 2020-02-06 キヤノンメディカルシステムズ株式会社 Medical image processing device, image formation method and image formation program
KR20220085000A (en) * 2020-12-14 2022-06-21 주식회사 뷰노 Method to read lesion

Similar Documents

Publication Publication Date Title
WO2020106010A1 (en) Image analysis system and analysis method
WO2015111840A1 (en) Device and method for inserting advertisement by using frame clustering
WO2021071189A1 (en) Platform for analyzing heavy electric equipment health and analysis method using same
WO2020050499A1 (en) Method for acquiring object information and apparatus for performing same
WO2021132851A1 (en) Electronic device, scalp care system, and control method for same
WO2020101108A1 (en) Artificial-intelligence model platform and method for operating artificial-intelligence model platform
WO2020138803A1 (en) Image analysis device and method
WO2019039912A1 (en) Diagnosis assistance system and control method therefor
WO2020185003A1 (en) Method for displaying ultrasonic image, ultrasonic diagnostic device, and computer program product
WO2018143707A1 (en) Makeup evaluation system and operation method thereof
WO2023080379A1 (en) Disease onset information generating apparatus based on time-dependent correlation using polygenic risk score and method therefor
WO2023191206A1 (en) Exploratory data analysis automation system and method based on variable attributes
WO2023033275A1 (en) Method and system for generating personalized biological age prediction model
WO2017217656A1 (en) Method and apparatus for video compression and computer program therefor
WO2015102391A1 (en) Image generating method for analyzing golf swing posture of user through depth image analysis, and method and device for analyzing golf swing posture using same
WO2023080766A1 (en) Apparatus for generating disease-specific risk gene mutation information using time-varying covariate-based prs model, and method therefor
WO2024128818A1 (en) System for quantifying lung disease of lung sample by using computer vision and machine learning, and method for quantifying lung disease by using system
WO2021215787A1 (en) Wireless ip camera detection system and method
US20200216916A1 (en) Method for estimating additive and dominant genetic effects of single methylation polymorphisms (smps) on quantitative traits
WO2020060161A1 (en) Statistical analysis system and statistical analysis method using conversational interface
WO2020251299A1 (en) Line detection method
WO2022145544A1 (en) Method for providing diagnosis assistance information, and device performing same
WO2025089631A1 (en) Apparatus and method for providing alarm according to behavior of driver in driver monitoring system
WO2014185714A1 (en) Method for improving medical image quality and apparatus therefor
WO2024005464A1 (en) Data clinic method, computer program in which data clinic method is stored, and computing device that performs data clinic method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23904017

Country of ref document: EP

Kind code of ref document: A1