KR20210063116A - Method and apparatus for detecting nuclear region using artificial neural network - Google Patents

Abstract

An objective of the present invention is to provide a method and an apparatus which accurately detect segmentation of a nuclear instance. According to one embodiment of the present invention, the method comprises: a step of acquiring a plurality of nuclear instance images including a first image and a second image; a step of displaying a marker based on a probability map of a nucleus for the first image, and displaying a marker based on a probability map of a nucleus for the second image; a step of inputting the first image and the second image with the displayed markers into a convolution-based neural network to estimate a nuclear instance region included in the first image and a nuclear instance region included in the second image; and a step of using output data of the neural network for the nuclear instance region included in the first image and the nuclear instance region included in the second image to determine whether a nuclear instance included in the first image and a nuclear instance included in the second image are the same.

Description

Method and device for detecting nuclear region using artificial neural network { METHOD AND APPARATUS FOR DETECTING NUCLEAR REGION USING ARTIFICIAL NEURAL NETWORK }

The present invention relates to a method and apparatus for detecting a nuclear region, and more particularly, to a technology for detecting and correcting a divided nuclear region.

Counting the number of mitotic cells is a pathology diagnosis task generally performed by a pathologist. The pathologist counts mitotic cells by observing the stained tissue slide through a high-magnification microscope or by observing the tissue slide image scanned by the digital pathology instrument through a computing device.

To improve this, various methods for detecting nuclear regions have been developed. However, there is still a limitation in that an adjacent nuclear region having an ambiguous boundary cannot be accurately detected during the nuclear division process.

An object of the present invention is to provide a method and apparatus for accurately detecting segmentation of nuclear instances in a method for detecting a region of a divided nucleus.

However, these problems are exemplary, and the scope of the present invention is not limited thereby.

A method for detecting a nuclear region according to an embodiment of the present invention includes acquiring a plurality of nuclear instance images including a first image and a second image, and a marker based on a probability map of a nucleus with respect to the first image. and displaying a marker based on the probability map of the nucleus with respect to the second image, and inputting the first image and the second image on which the marker is displayed into a convolution-based neural network to the first image estimating a region of a nuclear instance included in , and a region of a nuclear instance included in the second image, the neural network for a region of a nuclear instance included in the first image and a region of a nuclear instance included in the second image. and determining whether the nuclear instances included in the first image and the nuclear instances included in the second image are identical by using the output data.

In one embodiment, the displaying of the marker comprises inputting the first image and the second image to a third neural network based on convolution to generate a probability map of the nucleus for the probability of the existence of a nucleus per unit area in the image, The method may include displaying the marker on a unit area corresponding to a local maximum among unit areas in each image.

In one embodiment, the displaying of the marker may include displaying a marker based on a probability map of a nucleus for each of the plurality of nuclear instance images, generating a plurality of image pairs based on the location of the marker and acquiring one image pair from among the plurality of image pairs, wherein the one image pair may include the first image and the second image.

In an embodiment, each of the plurality of image pairs may include at least two images in which a position of a marker included in each image is closer than a reference value.

In an embodiment, the one image pair may be randomly obtained from among the plurality of image pairs.

In an embodiment, the neural network includes a convolutional layer including a plurality of convolutional layers and a deconvolutional layer including a plurality of deconvolutional layers, and the step of estimating the region of the nuclear instance includes: via the convolutional layer, The method may include extracting a feature of a nuclear instance and estimating a region of a segmented nuclear instance through the deconvolution layer.

In an embodiment, the output data of the neural network is a feature map for the first image and a feature map for the second image, and the step of determining whether the nuclear instances are identical includes the first image Estimating the degree of identity between the nuclear instances included in the first image and the nuclear instances included in the second image based on the overlap of the Dice coefficients of the feature map and the feature map with respect to the second image may include the step of

In an embodiment, the determining of the identity may further include merging the first image and the second image when the identity value is equal to or greater than a reference value.

A nuclear region detection apparatus according to another embodiment of the present invention includes a processor, wherein the processor acquires a plurality of nuclear instance images including a first image and a second image, and a probability of a nucleus with respect to the first image A marker is displayed based on a map, a marker is displayed based on a probability map of a nucleus with respect to the second image, and the first image and the second image on which the marker is displayed are combined with a convolution-based neural network. to estimate the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image, and the region of the nuclear instance included in the first image and the nuclear instance included in the second image It is possible to determine whether the nuclear instance included in the first image and the nuclear instance included in the second image are identical by using the output data of the neural network for the region.

In an embodiment, the processor inputs the first image and the second image to a convolution-based third neural network to generate a probability map of a nucleus for a probability that a nucleus exists per unit area in the image, and a unit in each image. The marker may be displayed in a unit region corresponding to a local maximum among regions.

In one embodiment, the processor displays a marker based on a probability map of a nucleus for each of the plurality of nuclear instance images, generates a plurality of image pairs based on the positions of the markers, and the plurality of image pairs One image pair may be obtained from among the one image pair, and the one image pair may include the first image and the second image.

In an embodiment, each of the plurality of image pairs may include at least two images in which a position of a marker included in each image is closer than a reference value.

In an embodiment, the one image pair may be randomly selected from among the plurality of image pairs.

In an embodiment, the neural network includes a convolutional layer including a plurality of convolutional layers and a deconvolutional layer including a plurality of deconvolutional layers, and the processor extracts a feature of a nuclear instance through the convolutional layer, An area of a segmented nuclear instance may be estimated through the deconvolution layer.

In an embodiment, the output data of the neural network is a feature map for the first image and a feature map for the second image, and the processor includes: a feature map for the first image and the second image An identity value between the nuclear instance included in the first image and the nuclear instance included in the second image may be estimated based on the feature map for the image.

In an embodiment, the processor may merge the first image and the second image when the identity value is equal to or greater than a reference value.

Other aspects, features, and advantages other than those described above will become apparent from the detailed contents, claims, and drawings for carrying out the following invention.

1 is a conceptual diagram illustrating a method for detecting a nuclear region according to an embodiment of the present invention.
2 is a diagram for explaining the configuration and operation of a nuclear region detection apparatus according to an embodiment of the present invention.
3 is a flowchart of a method for detecting a nuclear region according to an embodiment of the present invention.
4 is a diagram for explaining a nuclear instance image according to an embodiment of the present invention.
5 is a diagram for explaining the configuration and operation of a neural network for detecting a nuclear instance region according to an embodiment of the present invention.
6 is a diagram for explaining a method of determining the identity of detected nuclear instances according to an embodiment of the present invention.

Hereinafter, various embodiments of the present disclosure will be described in connection with the accompanying drawings. Various embodiments of the present disclosure may have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and related detailed descriptions are described. However, this is not intended to limit the various embodiments of the present disclosure to specific embodiments, and it should be understood that all changes and/or equivalents or substitutes included in the spirit and scope of the various embodiments of the present disclosure are included. In connection with the description of the drawings, similar reference numerals have been used for similar elements.

Expressions such as "include" or "may include" that may be used in various embodiments of the present disclosure indicate the existence of a corresponding function, operation, or component that is disclosed, and an additional one or more functions, operations, or It does not limit the components, etc. In addition, in various embodiments of the present disclosure, terms such as "include" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, It is to be understood that it does not preclude the possibility of the presence or addition of one or more other features or numbers, steps, actions, components, parts, or combinations thereof.

In various embodiments of the present disclosure, expressions such as "or" include any and all combinations of words listed together. For example, "A or B" may include A, may include B, or may include both A and B.

Expressions such as "first", "second", "first", or "second" used in various embodiments of the present disclosure may modify various elements of various embodiments, but do not limit the corresponding elements. Does not. For example, the expressions do not limit the order and/or importance of corresponding elements. The above expressions may be used to distinguish one component from another component. For example, a first user device and a second user device are both user devices and represent different user devices. For example, without departing from the scope of the rights of various embodiments of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, the component is directly connected to or may be connected to the other component, but the component and It should be understood that new other components may exist between the other components. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it will be understood that no new other component exists between the component and the other component. Should be able to.

The terms used in various embodiments of the present disclosure are only used to describe a specific embodiment, and are not intended to limit the various embodiments of the present disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise.

Unless otherwise defined, all terms including technical or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present disclosure belong.

Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in various embodiments of the present disclosure, ideal or excessively formal It is not interpreted in meaning.

1 is a conceptual diagram illustrating a method for detecting a nuclear region according to an embodiment of the present invention. In pathological imaging, cell atypism and nuclear homogeneity are important factors in diagnosing the grade and stage of cancer.

1 (a) is an original pathological image, (b) is an exemplary view of detecting a nuclear region using a conventional technique, (c) is a method for detecting a nuclear region according to an embodiment of the present invention to be described later It is an exemplary diagram in which a nuclear region is detected, and (d) is a diagram showing an actual nuclear region.

As shown in (b), individual nuclei were identified by conventionally detecting the nuclear region, background region, and border of the cell. Although such a prior art exhibits excellent performance in nuclear extraction of cells, there is a limitation in that it is impossible to accurately distinguish a plurality of adjacent instances. In the prior art, a process is performed so that a background between a plurality of detected instances is necessarily made, and even if a plurality of instances are attached, a process is performed between each instancer to be a background. Accordingly, even if each of the plurality of detected instances is included in the same nucleus, it may be identified as being included in each different nucleus. On the other hand, when using the method for detecting a nuclear region according to some embodiments of the present invention, which will be described later, as shown in FIG.

More specifically, in the method for detecting a nuclear region according to some embodiments of the present invention, in order to increase the accuracy of detecting a nuclear region, an operation of detecting a nuclear instance and merging or separating the nuclear instance using a neural network may be repeatedly performed. The aforementioned nuclear instance means a region in which a nucleus is determined to exist, and in the present embodiment, when it is determined that each nuclear instance detected using the neural network is included in the same nucleus, each nuclear instance may be merged. In addition, in an embodiment, the above-described neural network may further include a decoding layer, and a segmentaion of a nuclear instance may be estimated through the above-described decoding layer. In addition, the weight matrix of the above-described neural network may be updated using the segmentation of the estimated nuclear instance. Through the above-described process, the neural network according to some embodiments of the present invention may perform learning using various input data, thereby minimizing the occurrence of an overfitting problem.

The nuclear region detection method according to an embodiment may first acquire a plurality of nuclear instance images including a first image and a second image. Thereafter, a marker may be displayed on the first image based on the probability map of the nucleus, and the marker may be displayed on the second image based on the probability map of the nucleus. Thereafter, the first image and the second image marked with the marker may be input to a convolution-based neural network to estimate the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image. In addition, the sameness between the nuclear instance included in the first image and the nuclear instance included in the second image using output data of the neural network for the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image can determine whether A more detailed description will be given later in the related drawings.

2 is a diagram for explaining the configuration and operation of a nuclear region detection apparatus according to an embodiment of the present invention.

The nuclear region detection apparatus 100 according to an embodiment may include a memory 103 , a processor 104 , a communication module 102 , an input/output interface 101 , and a system bus.

The memory 103 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Also, an operating system and at least one program code may be stored in the memory 103 . These software components may be loaded from a computer-readable recording medium separate from the memory 103 using a drive mechanism. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 103 through the communication module 102 rather than a computer-readable recording medium. For example, the at least one program may be loaded into the memory 103 based on a program installed by files provided through a network by developers or a file distribution system that distributes installation files of applications.

The processor 104 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 104 by the memory 103 or the communication module 102 . For example, the processor 104 may be configured to execute instructions received according to program code stored in a recording device, such as the memory 103 .

In one embodiment, the processor 104 acquires a plurality of nuclear instance images including a first image and a second image, displays a marker on the first image based on a probability map of the nucleus, and a second A marker is displayed based on the probability map of the nucleus with respect to the image, and the first image and the second image marked with the marker are input to a convolution-based neural network, and the region and the second image of the nuclear instance included in the first image. estimating the nuclear instance region included in , and using the output data of the neural network for the nuclear instance region included in the first image and the nuclear instance region included in the second image, the nuclear instance region included in the first image and the second nuclear instance region included in the second image It is possible to determine whether the nuclear instances included in the image are identical.

The communication module 102 may provide a function for the external server and the nuclear region detection apparatus 100 to communicate with each other through a network. Of course, the communication module 102 may provide a function for communicating with other users and other servers. For example, a request generated by the processor 104 of the nuclear region detection apparatus 100 according to a program code stored in a recording device such as the memory 103 is transmitted to an external server through a network under the control of the communication module 102 . can be Conversely, the control signal, command, content, file, etc. provided under the control of the processor of the external server is transmitted through the communication module and the network through the communication module 102 of the nuclear region detection device 100 to the nuclear region detection device 100 ) can be received. For example, a control signal or command of an external server received through the communication module 102 may be transmitted to the processor 104 or the memory 103 , and the content or file may be further transmitted by the nuclear region detection device 100 . It may be stored in a storage medium that may include.

The communication method is not limited, and not only a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network may include, but also short-range wireless communication between devices may be included. For example, the network is a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. may include any one or more of the networks of Further, the network may include, but is not limited to, any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. .

The input/output interface 101 may be a means for interfacing with an input/output device. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input/output interface 101 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. As a more specific example, the processor 104 of the nuclear region detection apparatus 100 processes a command of a computer program loaded in the memory 103, and a service screen or content configured using data provided by an external server is an input/output interface. may be displayed on the display through 101 .

Also, in other embodiments, the nuclear region detection apparatus 100 may include more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the nuclear region detection apparatus 100 may further include other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database.

3 is a flowchart of a method for detecting a nuclear region according to an embodiment of the present invention. Hereinafter, it is assumed that an apparatus for performing a nuclear region detection method described below is a nuclear region detection apparatus.

In operation S110, the nuclear region detection apparatus may acquire a plurality of nuclear instance images including the first image and the second image. The nuclear instance image may be a pathological image including at least one nuclear instance. For example, the plurality of nuclear instance images may include MoNuSeg data including multi-tissue Haematoxylin Eosin (H&E) staining images and/or Triple-Negative Breast Cancer (TNBC) data of negative breast cancer tissues.

In operation S120, the nuclear region detection apparatus may display the marker on the first image and display the marker on the second image. In more detail, the nuclear region detection apparatus may generate a nuclear probability map with respect to the first image and the second image, and display a marker corresponding to the location of the nucleus on each image using the generated nuclear probability map. can do. In an embodiment, the apparatus for detecting a nuclear region may display a local maximum of the generated nuclear probability map as a marker.

In operation S130, the first image and the second image marked with the marker may be input to the convolution-based neural network. In an embodiment, the apparatus for detecting a nuclear region may generate an image pair including a first image and a second image by using a plurality of nuclear instance images. The above-described image pairs may be generated based on specified criteria. In an embodiment, the apparatus for detecting a nuclear region may generate an image pair including two images in an order in which positions of markers displayed on a plurality of nuclear instance images are close to each other. However, this is only an example of generating the image pair, and the reference for generating the image pair and the number of images included in the image pair are not limited thereto.

Also, in another embodiment, a new image pair may be generated with a different combination from the previously generated image pair combination, and this may be used as training data for the neural network. As described above, by learning the neural network in various combinations for a plurality of nuclear instance images, it is possible to secure a sufficient amount of learning data and to prevent the occurrence of an overfitting problem.

Also, a plurality of image pairs generated according to an embodiment may be sequentially input to the neural network. In an embodiment, the image pair input to the neural network may be randomly selected. However, it should be noted that this is only an example of a method of selecting an image pair, and a criterion for selecting an image pair used as input data for the neural network of the present invention is not limited thereto.

In operation S140 , the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image may be estimated. In an embodiment, a neural network used to detect a nuclear region may include a convolutional layer and a deconvolutional layer. In an embodiment, the convolutional layer may extract a feature of the nuclear instance for extracting the nuclear instance. Also, in an embodiment, the deconvolution layer may detect a nuclear instance included in the input data. For example, the convolutional layer may include 16 layers performing a convolution operation, and the deconvolution layer may include 16 layers performing a deconvolution operation. However, it should be noted that this is only an example and the number of layers is not limited thereto.

In operation S150 , output data of the neural network for the nuclear instance region included in the first image and the nuclear instance region included in the second image may be acquired. In an embodiment, the nuclear region detection apparatus may output a feature map of the first image and a feature map of the second image. Also, the output data of the neural network may include segmentation information of the nuclear instance included in the first image and segmentation information of the nuclear instance included in the second image. Segmentation of a nuclear instance refers to a segmented portion of a nucleus generated by nuclear fission. However, it should be noted that the output data form of the neural network according to some embodiments of the present invention may be various forms that can be easily designed and changed by those skilled in the art.

In operation S160, it may be determined whether the nuclear instance included in the first image is identical to the nuclear instance included in the second image using the output data of the neural network. The nuclear instance device may acquire the segmentation of the nuclear instance detected by the neural network, and may compare the segmentation of the nuclear instance of the first image included in one image pair and the segmentation of the nuclear instance of the second image.

For example, the nuclear instance device may calculate the overlap between the segmentation of the nuclear instance of the first image and the segmentation of the nuclear instance of the second image by using a Dice coefficient. In an embodiment, the apparatus for detecting a nuclear region may determine whether each image includes an instance of the same nucleus using the above-described redundancy.

Thereafter, when the segmentation overlap of the nuclear instances between the first image and the second image is higher than the specified reference value, it is assumed that the nuclear instances included in the first image and the nuclear instances included in the second image are included in the same nucleus. In this case, the nuclear instances of the first image and the second image are merged. The image including the merged nuclear instance may be used again as input data and/or training data of the neural network to increase the accuracy of the nuclear region detection of the neural network.

On the other hand, when the segmentation overlap of the nuclear instances between the first image and the second image is lower than the reference value, it can be assumed that the nuclear instances included in the first image and the nuclear instances included in the second image are included in different nuclei. Thereafter, the above-described first image and second image are not included in the same image pair. In this case, it goes without saying that the first image and the second image may be paired with different images, respectively, and used as input data and/or learning data of the neural network.

Hereinafter, data input to the neural network will be described in detail with reference to FIG. 4 . 4 (a) is an exemplary view of a probability map of a nucleus obtained through the prior art, (b) is an exemplary view of an image in which a marker is displayed based on the local maximum value of the probability map, (c) is a segmented nuclear instance It is an exemplary diagram of a result of detecting a region.

As shown in (a), the nuclear region detection apparatus according to some embodiments of the present invention may generate a probability map for a plurality of nuclear instance images. The probability map of nuclei can be estimated using a zoning network. For example, U-net may be used as the zoning network, but it should be noted that the type of zoning network according to an embodiment of the present invention is not limited thereto.

Thereafter, as shown in (b), the nuclear region detection apparatus may display a local maximum as a marker using the generated probability map of the nucleus. In this case, each region marked with the marker may then be detected as an individual nuclear instance region, merged with another nuclear instance region, or separated into a plurality of nuclear instance regions through a nuclear region detection step.

A neural network for detecting a nuclear region according to some embodiments of the present invention may use an image including a marker as input data. In this case, the above-described neural network may use an image pair composed of a plurality of images as input data, and the above-described image pair may be composed of a plurality of images according to a specified criterion. In an embodiment, the image pair may be generated based on positions of markers included in a plurality of images. For example, an image pair including at least two images may be generated according to an order in which the positions of the markers are close. In this case, the image data input to the neural network may be an RGB image patch including a marker channel. (c) is an example of an image in which a nuclear instance region is detected by the nuclear region detection device.

5 is a diagram for explaining a method of determining the identity between nuclear instances according to an embodiment of the present invention.

In an embodiment, the neural network 210 for detecting the nuclear instance region may use the image pair 200 including the first image 201 and the second image 202 as input data and/or training data. Thereafter, the neural network 210 detecting the nuclear instance region may output data 300a and 300b in which the nuclear instance region included in the first image 201 and the second image 202 is detected.

In an embodiment, the image pair 200 that is input data of the neural network may be generated in an order in which the positions of markers displayed on the images are close to each other. In one embodiment, as a result of detecting the nuclear instance region through the neural network 210 based on the marker displayed in the first image 201 and the marker displayed in the first image 202 , the nuclear region detection apparatus results in the first image 201 ) and the nuclear instance included in the second image 202 are included in the same nucleus, the first image 201 and the second image 202 may be merged ( 300a ). The data 300a merged according to the above-described method may be input to the neural network 210 by forming an image pair with another third image.

In an optional embodiment, when the nuclear region detection apparatus determines that the nuclear instances included in the first image 201 and the second image 202 are included in different nuclei (300b), the first image 201 and the second image 202 may be input back to the neural network 210 by forming a pair of images with different images, respectively. In this case, the first image 201 and the second image 202 may not be included in the same image pair after that.

In an embodiment, the neural network 210 for detecting the nuclear instance region may include an encoding layer 220 and a decoding layer 230 . In an embodiment, the encoding layer 220 may extract a feature of a nuclear instance included in each image and output a feature map of each image. In addition, the decoding layer 230 may estimate the region of the nuclear instance included in each image, and estimate whether the nuclear instance included in each image is included in the same nucleus using a dice coefficient to be described later. A more detailed description will be given below with reference to FIG. 6 .

FIG. 6 is a diagram for describing the neural network shown in FIG. 5 in more detail. The neural network for detecting a nuclear region according to an embodiment of the present invention may further determine whether nuclear instances included in the input image pair are identical.

In more detail, the nuclear region detection apparatus may use the image pair 200 including the first image 201 and the second image 202 as input data. The input data may be input to the convolutional layers 221a, 221b, 223a, and 223b first. Thereafter, at the ends 223a and 223b of the encoding section for extracting the feature maps of the first image 201 and the second image 202, the above-described first image is a classifier that determines whether nuclear instances included in each image are identical. The feature maps of the first image 201 and the second image 202 may be transmitted.

In addition, the convolutional layers 221a, 221b, 223a, and 223b and the deconvolutional layers 231a, 231b, 232a, and 232b may perform skip connections 250a and 250b, and channel unit matrix multiplication to share weights between layers (222, 240) can be performed. In this case, the deconvolution layers 231a , 231b , 232a , and 232b may estimate the nuclear instance regions included in the first image 201 and the second image 202 . In more detail, the deconvolution layers 231a , 231b , 232a , and 232b may estimate segmentation regions of nuclear instances included in the first image 201 and the second image 202 .

은

=

+

로 정의될 수 있다. 여기서 분할 손실

는 수식 1과 같이 정의된다.In an embodiment, the classifier may include an additional convolutional layer 260 and a fully connected layer 261 , but it should be noted that the neural network structure of the classifier is only an example and the structure of the neural network is not limited thereto. In more detail, the feature map extracted by the convolutional layers 221a, 221b, 223a, and 223b may be transmitted to an additional convolutional layer 260 and a fully-connected layer 261 for estimating a class. . Based on the class estimated through this and the overlap of the nuclear instance regions 311 and 312 estimated in the deconvolution layers 231a, 231b, 232a, and 232b, the nuclear instance of the first image 201 and the second image 202 ) may be computed as an identity 320 between nuclear instances. The loss function of the neural network described above

silver

=

+

Can be defined as split loss here

is defined as in Equation 1.

[Equation 1]

과

는 두 가지 입력 피처이다.

는 ·에 대한 세그먼트 예측이고,

과

는

과

에 대한 세그먼트 기준입니다.

는

와

사이의 교차 엔트로피이다. 분류 손실은

로 정의되며, 여기서

는 분류 예측

와 세그먼트 예측 사이의 다이스 계수

사이의 곱이다.where Θ is the set of parameters,

and

are two input features.

is the segment prediction for

and

Is

and

Segment criteria for .

Is

Wow

is the cross entropy between Classification loss is

is defined as, where

is the classification prediction

dice coefficient between and segment prediction

is the product between

[Equation 2]

손실은 인스턴스 분할의 일관성이 클래스 예측에 영향을 주도록 할 수 있다. 즉, 분할이 일관된 경우 두 인스턴스가 모두 동일한 핵에 있다고 판단 될 가능성이 높아지고, 분할이 일관되지 않으면 확률이 줄어 든다. the above

The loss can cause the consistency of instance partitioning to affect class prediction. That is, if the splits are consistent, the probability that both instances will be judged to be in the same nucleus increases, and if the splits are inconsistent, the probability decreases.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium is a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magnetic-optical medium such as a floptical disk, and a ROM. It may include a hardware device specially configured to store and execute program instructions, such as, RAM, flash memory, and the like.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and used by those skilled in the art of computer software. Examples of the computer program may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The specific implementations described in the present invention are only examples and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections. It may be referred to as a connection, or circuit connections. In addition, if there is no specific mention such as "essential", "important", etc., it may not be an essential component for the application of the present invention.

As described above, the present invention has been described with reference to the embodiments shown in the drawings, but these are only exemplary, and those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. . Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (17)

acquiring a plurality of nuclear instance images including a first image and a second image;
displaying a marker based on a probability map of a nucleus with respect to the first image, and displaying a marker based on a probability map of a nucleus with respect to the second image;
estimating a region of a nuclear instance included in the first image and a region of a nuclear instance included in the second image by inputting the first image and the second image marked with a marker into a convolution-based neural network; and
Between the nuclear instance included in the first image and the nuclear instance included in the second image using the output data of the neural network for the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image Determining whether or not the same; including
Methods for detecting nuclear regions.
The method of claim 1,
The step of displaying the marker,
By inputting the first image and the second image to a third neural network based on convolution, a probability map of nuclei with respect to the probability that nuclei exist per unit region in the image is generated, and a local maximum value among unit regions in each image is generated. ) including the step of displaying the marker in a unit area corresponding to
Methods for detecting nuclear regions.
The method of claim 1,
The step of displaying the marker,
displaying a marker based on a probability map of a nucleus for each of the plurality of nuclear instance images;
generating a plurality of image pairs based on the positions of the markers; and
Including; acquiring one image pair from among the plurality of image pairs;
The one image pair is
comprising the first image and the second image,
Methods for detecting nuclear regions.
The method of claim 3,
Each of the plurality of image pairs,
At least two images in which the position of the marker included in each image is closer than the reference value,
Methods for detecting nuclear regions.
The method of claim 3,
The one image pair is
randomly obtained from among the plurality of image pairs,
Methods for detecting nuclear regions.
The method of claim 1,
The neural network is
A convolutional layer including a plurality of convolutional layers and a deconvolutional layer including a plurality of deconvolutional layers,
The step of estimating the area of the nuclear instance comprises:
extracting a feature of a nuclear instance through the convolutional layer; and
estimating the region of a segmented nuclear instance through the deconvolutional layer,
Methods for detecting nuclear regions.
The method of claim 1,
The output data of the neural network is,
a feature map for the first image and a feature map for the second image,
The step of determining whether the nuclear instances are identical,
Between the nuclear instance included in the first image and the nuclear instance included in the second image based on the degree of overlap of the Dice coefficients of the feature map for the first image and the feature map for the second image estimating a measure of identity;
Methods for detecting nuclear regions.
The method of claim 7,
The step of determining whether the identity is
Further comprising the step of merging the first image and the second image when the identity value is greater than a reference value,
Methods for detecting nuclear regions.
processor; including;
The processor,
Acquire a plurality of nuclear instance images including a first image and a second image, display a marker based on a probability map of a nucleus with respect to the first image, and a probability map of a nucleus with respect to the second image A marker is displayed based on , and the first image and the second image on which the marker is displayed are input to a convolution-based neural network, and the region of the nuclear instance included in the first image and the region of the nuclear instance included in the second image. estimating a nuclear instance region, and using the output data of the neural network for the nuclear instance region included in the first image and the nuclear instance region included in the second image, the nuclear instance included in the first image and the second nuclear instance region To determine whether the nuclear instances included in the image are identical,
nuclear region detection device.
The method of claim 9,
The processor,
By inputting the first image and the second image to a third neural network based on convolution, a probability map of nuclei with respect to the probability that nuclei exist per unit region in the image is generated, and a local maximum value among unit regions in each image is generated. ) to display the marker in the unit area corresponding to,
nuclear region detection device.
The method of claim 9,
The processor,
For each of the plurality of nuclear instance images, a marker is displayed based on the probability map of the nucleus, a plurality of image pairs are generated based on the position of the marker, and one image pair of the plurality of image pairs is obtained,
The one image pair is
comprising the first image and the second image,
nuclear region detection device.
The method of claim 11,
Each of the plurality of image pairs,
At least two images in which the position of the marker included in each image is closer than the reference value,
nuclear region detection device.
The method of claim 11,
The one image pair is
randomly selected from among the plurality of image pairs,
nuclear region detection device.
The method of claim 9,
The neural network is
A convolutional layer including a plurality of convolutional layers and a deconvolutional layer including a plurality of deconvolutional layers,
The processor,
extracting a feature of a nuclear instance through the convolutional layer, and estimating an area of a segmented nuclear instance through the deconvolutional layer,
nuclear region detection device.
The method of claim 9,
The output data of the neural network is,
a feature map for the first image and a feature map for the second image,
The processor,
Estimating the identity value between the nuclear instance included in the first image and the nuclear instance included in the second image based on the feature map for the first image and the feature map for the second image,
nuclear region detection device.
The method of claim 15,
The processor,
Merging the first image and the second image when the identity value is greater than or equal to a reference value,
nuclear region detection device.
A computer program stored in a recording medium for executing the method of any one of claims 1 to 8 using a computer.

Images