JP2020124220A - Image processing apparatus - Google Patents

Abstract

To provide an image processing apparatus capable of suppressing poor analysis when calculating an interaction in a cell or between cells.SOLUTION: An image processing apparatus comprises: a cell image acquisition part that acquires a cell image obtained by imaging cells; a feature amount calculation part that calculates a plurality of types of feature amounts of the cell image acquired by the cell image acquisition part; and a correlation extraction part that extracts a specific correlation from the plurality of correlations between the feature amounts calculated by the feature amount calculation part, based on the likelihood of the feature amounts.SELECTED DRAWING: None

Description

The present invention relates to an image processing device.

In biological science, medicine, etc., it is known that the state of health and diseases of organisms is related to the state of cells and organelles within cells, for example. Therefore, analyzing these relationships is one means for solving problems in various fields such as biological science and medicine. In addition, analysis of the transmission pathway of information transmitted between cells or in cells can be useful for research on biosensors for industrial use, pharmaceuticals for disease prevention, and the like. As various analysis techniques for cells, tissue pieces, etc., for example, a technique using image processing is known (see, for example, Patent Document 1).

U.S. Pat. No. 0228069

However, when calculating the interaction between cells or between cells, if the information obtained by image-processing the image containing the imaged cells is used, the amount of information obtained is large, so in order to obtain the correlation, An enormous amount of calculation is required, which may result in poor analysis.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus capable of suppressing poor analysis.

In order to solve the above problems, one aspect of the present invention calculates a plurality of types of feature amounts of a cell image acquisition unit that acquires a cell image in which cells are imaged, and the cell image that the cell image acquisition unit acquires. And a correlation extraction unit that extracts a specific correlation from among the plurality of correlations between the feature amounts calculated by the feature amount calculation unit based on the likelihood of the feature amount. And an image processing apparatus including.

According to the present invention, analysis failure due to image processing can be suppressed.

It is a schematic diagram which shows an example of a structure of the microscope observation system by embodiment of this invention. It is a block diagram showing an example of functional composition of each part with which an image processing device of this embodiment is provided. It is a flow chart which shows an example of a calculation procedure of a calculation part of this embodiment. It is a schematic diagram which shows an example of the calculation result of the feature-value by the feature-value calculation part of this embodiment. It is a figure which shows an example of the matrix of the feature-value for every cell of this embodiment. It is a schematic diagram which shows an example of the correlation between the feature-values of this embodiment. It is a schematic diagram which shows an example of the specific correlation of this embodiment. It is a flow chart which shows an example of a procedure of cross validation by a correlation extraction part of this embodiment. It is a schematic diagram which shows an example of the calculation procedure of the average likelihood by the correlation extraction part of this embodiment. It is a graph which shows an example of the relation of the average likelihood and the regularization parameter of this embodiment. It is a schematic diagram which shows an example of the extraction result of the correlation of the feature-value by the correlation extraction part of this embodiment. It is a table which shows an example of the protein annotation database of this embodiment. It is a table showing an example of a feature quantity annotation database of this embodiment. It is a figure which shows an example of the comparative type of the feature-value network of this embodiment. It is a figure which shows an example of the determination procedure of the regularization parameter using the biological knowledge of this embodiment. It is a figure which shows the relationship of a knowledge base network and a prediction network regarding the presence or absence of connection of the feature-value network of this embodiment.

[Embodiment]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing an example of the configuration of a microscope observation system 1 according to an embodiment of the present invention.

The microscope observation system 1 performs image processing on an image acquired by imaging a cell or the like. In the following description, an image acquired by imaging a cell or the like is also simply referred to as a cell image. The microscope observation system 1 includes an image processing device 10, a microscope device 20, and a display unit 30.

The microscope device 20 is a biological microscope and includes an electric stage 21 and an imaging unit 22. The electric stage 21 can arbitrarily move the position of the imaging target in a predetermined direction (for example, a certain direction within a horizontal two-dimensional plane). The image pickup unit 22 includes an image pickup device such as a CCD (Charge-Coupled Device) or a CMOS (Complementary MOS), and picks up an image of an image pickup target on the electric stage 21. It should be noted that the microscope apparatus 20 may not be provided with the electric stage 21, and the stage may not operate in the predetermined direction.

More specifically, the microscope device 20 has functions such as a differential interference microscope (DIC), a phase contrast microscope, a fluorescence microscope, a confocal microscope, and a super-resolution microscope. The microscope apparatus 20 images the culture container placed on the electric stage 21. The culture container is, for example, a well plate WP. The microscope apparatus 20 irradiates cells cultured in a large number of wells W included in the well plate WP with light, thereby capturing the transmitted light transmitted through the cells as an image of the cells. This makes it possible to acquire images such as a transmission DIC image of cells, a phase difference image, a dark field image, and a bright field image. Further, by irradiating the cells with excitation light that excites the fluorescent substance, the fluorescence emitted from the biological substance is captured as an image of the cells. In addition, the microscope device 20 captures the fluorescence emitted from the coloring substance itself taken into the biological substance or the fluorescence emitted by the substance having the chromophore bound to the biological substance as an image of the cell described above. You may. Thereby, the microscope observation system 1 can acquire a fluorescence image, a confocal image, and a super-resolution image. The method of acquiring the image of the cell is not limited to the optical microscope. For example, an electron microscope may be used. Further, as the image including cells described below, images obtained by different methods may be used to obtain the correlation. That is, the type of image containing cells may be appropriately selected. The cells in the present embodiment are, for example, primary culture cells, established culture cells, cells of tissue sections, and the like. In order to observe the cells, the sample to be observed may be an aggregate of cells, a tissue sample, an organ, or a solid (animal etc.), and an image containing the cells may be acquired. The state of the cells is not particularly limited, and may be a living state or a fixed state, and may be "in-vivo" or "in-vitro". Of course, the information of the living state and the fixed information may be combined.
The cell state may be appropriately selected according to the purpose. For example, fixed or unfixed may be selected depending on the type of discrimination in the intracellular structure (for example, protein or organelle). In addition, when acquiring the dynamic behavior of cells with fixed cells, a plurality of fixed cells under different conditions are created and the dynamic behavior is acquired. In the intracellular structure, the type of discrimination is not limited to the nucleus.
Further, in order to observe the cells, the cells may be previously treated and then observed. Of course, in order to observe the cells, the cells may be observed without being treated. When observing the cells, the cells may be stained by immunostaining and then observed. For example, the staining solution to be used may be selected depending on the type of discrimination in the intracellular nuclear structure. As for the dyeing method, any dyeing method can be used. For example, there are various special stains mainly used for tissue staining, and hybridization utilizing binding of base sequences.
Alternatively, cells may be treated with a luminescent protein (for example, a luminescent protein expressed from an introduced gene (luciferase gene or the like)) and observed. For example, the photoprotein to be used may be selected depending on the type of discrimination in the intracellular nuclear structure. Further, the pretreatment for analyzing the correlation acquisition, such as the means for observing these cells and/or the method for staining the cells, may be appropriately selected according to the purpose. For example, when obtaining the dynamic behavior of a cell, the optimal method is used to obtain the dynamic information of the cell, and when obtaining intracellular signal transduction, the optimal method is used to obtain the information regarding the intracellular signal transduction. It doesn't matter. The pretreatments selected for each purpose may be different. Further, the number of types of preprocessing selected for each purpose may be reduced. For example, even if the method for obtaining the dynamic behavior of a cell and the method for obtaining intracellular signal transduction are different from each other, it is complicated to obtain each information by different methods. In addition, when it is sufficient to acquire the respective information, different methods may be used in common, unlike the optimum method.

The well plate WP has a plurality of wells W. In this example, the well plate WP has 96 wells W of 12×8. The cells are cultured in the well W under specific experimental conditions. The specific experimental conditions include temperature, humidity, culture period, time elapsed after stimulation is applied, type and strength of stimulation applied, presence or absence of stimulation, induction of biological characteristics, and the like. The stimulus is, for example, a physical stimulus such as electric, sound wave, magnetism, light, or a chemical stimulus by administration of a substance or a drug. The biological characteristic is a characteristic indicating the stage of cell differentiation, morphology, cell number, and the like.

FIG. 2 is a block diagram showing an example of the functional configuration of each unit included in the image processing apparatus 10 according to this embodiment. The image processing device 10 is a computer device that analyzes an image acquired by the microscope device 20. The image processing device 10 includes a calculation unit 100, a storage unit 200, and a result output unit 300. The image processed by the image processing apparatus 10 is not limited to the image captured by the microscope apparatus 20, but may be, for example, an image previously stored in the storage unit 200 included in the image processing apparatus 10 or an unillustrated image. It may be an image stored in advance in an external storage device.

The arithmetic unit 100 functions by the processor executing a program stored in the storage unit 200. Further, some or all of the functional units of the arithmetic unit 110 may be configured by hardware such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit). The calculation unit 100 includes a cell image acquisition unit 101, a feature amount calculation unit 102, a noise component removal unit 103, and a correlation extraction unit 104.

The cell image acquisition unit 101 acquires the cell image captured by the imaging unit 22 and supplies the acquired cell image to 102. Here, the cell images acquired by the cell image acquisition unit 101 include a plurality of images in which the culture state of the cells is captured in time series and a plurality of images in which the cells are cultured under various experimental conditions.

The feature amount calculation unit 102 calculates a plurality of types of feature amounts of the cell image supplied by the cell image acquisition unit 101. This feature amount includes the brightness, area, variance, etc. of the cell image.
That is, it is a feature derived from the information acquired from the imaged cell image. For example, the brightness distribution in the acquired image is calculated. Using multiple images that are different in time series or due to changes in cell state such as differentiation, changes in the calculated luminance distribution over a predetermined period of time, or changes due to changes in cell state such as differentiation in the calculated luminance distribution, It is also possible to obtain position information indicating a change in brightness different from that and use the change in brightness as the feature amount. In this case, it is possible to use a plurality of images which are not limited to the change in time and have different changes in cell states such as differentiation. Further, the information of the positions indicating different changes in brightness may be used as the feature amount. For example, the behavior of cells within a predetermined time, or the behavior associated with changes in cell states such as cell differentiation may be used, and changes in the shape of cells within a predetermined time or changes in cell states such as differentiation of cell shapes may occur. It may be accompanied by a change. Further, if no change within a predetermined time or a change associated with a change in cell state such as differentiation is not recognized from the imaged cell image, no change may be made or the feature amount may be used.
The noise component removal unit 103 removes a noise component (noise) from the feature amount calculated by the feature amount calculation unit 102.
The correlation extraction unit 104 determines, for the feature quantity after the noise component removal unit 103 has removed the noise component, a plurality of correlations between the feature quantities calculated by the feature quantity calculation unit 102 based on the likelihood of the feature quantity. From them, we extract a specific correlation. Here, the likelihood is a numerical value indicating the likelihood of estimating the predetermined condition from the result when the result is calculated according to the predetermined condition. Further, the likelihood is a cost function that represents the likelihood of the parameter to be estimated when the data follows the probability model.
The result output unit 300 outputs the calculation result of the calculation unit 100 to the display unit 30. The result output unit 300 may output the calculation result of the calculation unit 100 to an output device other than the display unit 30, a storage device, or the like.
The display unit 30 displays the calculation result output by the result output unit 300.
A specific calculation procedure of the calculation unit 100 described above will be described with reference to FIG.

FIG. 3 is a flowchart showing an example of the calculation procedure of the calculation unit 100 of this embodiment. The calculation procedure shown here is an example, and the calculation procedure may be omitted or the calculation procedure may be added.
The cell image acquisition unit 101 acquires a cell image (step S10). The cell image includes images of a plurality of types of living tissues having different sizes such as genes, proteins, and organelles. In addition, the cell image includes cell shape information. Further, dynamic behavior is acquired from cell images having different time series or different cell states such as differentiation. Therefore, different scales of multiple sizes from the minute structure in the cell to the cell shape, or a predetermined time, or a time series different from the cell image in a certain cell state such as differentiation, or differentiation, etc. It can also be referred to as a multi-scale analysis because it is analyzed using the scales having different dimensions in cell images having different cell states.

The feature amount calculation unit 102 extracts an image of cells included in the cell image acquired in step S10 for each cell (step S20). The feature amount calculation unit 102 extracts an image of a cell by performing image processing on the cell image by a known method. In this example, the feature amount calculation unit 102 extracts a cell image by performing contour extraction of the image, pattern matching, or the like.

Next, the feature amount calculation unit 102 determines the type of cell in the image of the cell extracted in step S20 (step S30). Further, the feature amount calculation unit 102 determines the constituent elements of the cells included in the image of the cells extracted in step S20 based on the determination result in step S30 (step S40). Here, the constituent elements of cells include organelles such as cell nuclei, lysosomes, Golgi apparatus, mitochondria, and proteins that constitute organelles. Although the cell type is determined in step S30, the cell type may not be determined. In this case, if the type of cell to be introduced is determined in advance, that information may be used. Of course, it is not necessary to specify the cell type.

Next, the feature amount calculation unit 102 calculates the feature amount of the image for each constituent element of the cell determined in step S40 (step S50). The feature amount includes a brightness value of a pixel, an area of a certain area in an image, a variance value of brightness of the pixel, and the like. In addition, there are a plurality of types of feature quantities depending on the constituent elements of cells. As an example, the feature quantity of the image of the cell nucleus includes the total brightness value in the nucleus, the area of the nucleus, and the like. The feature quantity of the cytoplasm image includes the total brightness value in the cytoplasm, the area of the cytoplasm, and the like. Further, the feature amount of the image of the entire cell includes the intracellular total brightness value, the area of the cell, and the like. Further, the feature amount of the mitochondrial image includes the fragmentation rate. The feature amount calculation unit 102 may calculate the feature amount by normalizing it to a value between 0 (zero) and 1, for example.

Further, the feature amount calculation unit 102 may calculate the feature amount based on the information on the experimental condition for the cell associated with the cell image. For example, in the case of a cell image captured when an antibody is made to react with a cell, the feature amount calculation unit 102 may calculate a unique feature amount when making the antibody react. Further, in the case of a cell image taken when a cell is stained or when a fluorescent protein is added to the cell, the feature amount calculation unit 102 stains the cell or when a fluorescent protein is added to the cell. You may calculate the characteristic amount peculiar to.
In these cases, the storage unit 200 may include the experimental condition storage unit 202. The experimental condition storage unit 202 stores information on experimental conditions for cells associated with cell images for each cell image.

The feature amount calculation unit 102 supplies the feature amount calculated in step S50 to the noise component removal unit 103.

The noise component removal unit 103 removes a noise component from the feature amount calculated in step S50 (step S60). Specifically, the noise component removal unit 103 acquires information indicating the normal range or abnormal range of the feature amount. The information indicating the normal range or the abnormal range of the feature amount is predetermined based on the characteristics of the cell captured in the cell image. For example, the normal range of the total nuclear intensity value of the feature amount of the image of the cell nucleus is determined based on the characteristics of the image of the cell nucleus. The noise component removing unit 103 removes the calculated feature amount as a noise component when the calculated feature amount is not included in the normal range. Here, the noise component removing unit 103 removes the feature amount as a noise component for each cell. Specifically, a plurality of feature values may be calculated for a certain cell. For example, for a certain cell, the total intracellular brightness value, the total nuclear brightness value, and the area of the nucleus may be calculated as the feature values. In this case, when removing the total intracellular brightness value as a noise component from a certain cell, the noise component removing unit 103 also removes the total nuclear brightness value of the cell and the area of the nucleus. That is, the noise component removal unit 103 also removes other feature amounts of this cell when at least one feature amount out of the plurality of feature amounts calculated for a cell is not included in the normal range.

That is, the noise component removal unit 103 captures a noise component in the cell image from the feature amounts supplied to the correlation extraction unit 104 based on the information indicating the characteristics of the cell captured in the cell image. Remove for each cell. With this configuration, the noise component removing unit 103 can exclude the feature amount in cell units when there is a feature amount having a relatively low reliability. That is, the noise component removing unit 103 can improve the reliability of the feature amount.

Further, when the calculated feature amount is included in this normal range, the noise component removal unit 103 supplies the feature amount to the correlation extraction unit 104. The noise component removing unit 103 is not an essential component and may be omitted depending on the state of the cell image and the state of calculation of the feature amount.

The correlation extraction unit 104 extracts a specific correlation from a plurality of correlations between the feature amounts calculated by the feature amount calculation unit 102 based on the likelihood of the feature amount using sparse estimation ( Step S70). Hereinafter, the process performed by the correlation extraction unit 104 will be described more specifically.

The correlation extracting unit 104 acquires the feature amount after the noise component is removed in step S60. The feature amount is calculated for each cell by the feature amount calculation unit 102 in step S50. The calculation result of the characteristic amount of a certain protein by the characteristic amount calculation unit 102 will be described with reference to FIG.

FIG. 4 is a schematic diagram showing an example of the calculation result of the characteristic amount by the characteristic amount calculation unit 102 of this embodiment. The feature amount calculation unit 102 calculates a plurality of feature amounts for the protein 1 for each cell and for each time. In this example, the feature amount calculation unit 102 calculates the feature amount of N cells from cell 1 to cell N. Further, in this example, the feature amount calculation unit 102 calculates the feature amount at seven times from time 1 to time 7. Further, in this example, the feature amount calculation unit 102 calculates K types of feature amounts from the feature amount k1 to the feature amount kK. That is, in this example, the feature amount calculation unit 102 calculates the feature amount in the directions of the three axes. Here, the axis in the cell direction is described as the axis Nc, the axis in the time direction as the axis N, and the axis in the feature amount direction as the axis d1.

The K types of feature quantities from the feature quantity k1 to the feature quantity kK are combinations of the feature quantities of the protein 1. Regarding the proteins other than the protein 1 or the constituent elements in the cell other than the protein 1, the types and combinations of the characteristic amounts may differ.

FIG. 5 is a diagram showing an example of the matrix X of the feature amount for each cell according to the present embodiment. The feature amount of a certain cell can also be shown by a matrix X shown in FIG. 5 with the axis N in the row direction and the axis d in the column direction. In FIG. 5, each element of the matrix X is shown by the average value of the cell population, but it is also possible to use a statistic such as the median or the mode. Of course, the matrix X of the feature amount for each cell may be used.

FIG. 6 is a schematic diagram showing an example of the correlation between the feature amounts of this embodiment. The correlation extraction unit 104 extracts a specific correlation from the plurality of correlations between the feature amounts calculated by the feature amount calculation unit 102.
FIG. 7 is a schematic diagram showing an example of the specific correlation according to the present embodiment. Here, the specific correlation is a correlation selected by a mathematical operation from a plurality of correlations between feature amounts. The number of specific correlations is smaller than the number of the plurality of correlations between the feature quantities before being extracted by the correlation extraction unit 104. That is, the correlation extraction unit 104 makes the number of correlations between feature amounts sparse. The correlation extraction unit 104 reduces the number of correlations between feature quantities by using sparse estimation for the correlations between feature quantities. An example of sparse estimation by the correlation extracting unit 104 will be described.

The likelihood in the Gaussian model used by the correlation extraction unit 104 for sparse estimation is shown in Expression (1).

In this equation (1), the term “φ(Λ)” is a regularization term. The function form of the regularization term has various forms depending on the attribute of the correlation to be extracted. In particular, it is known that an accuracy matrix having sparse components can be obtained by adding the L1 regularization term as in Expression (2) when assuming a Gaussian model. Λ in the equation (2) represents the strength of regularization, and the larger this is, the more easily the components of the accuracy matrix become sparse.

Further, for example, it is possible to obtain a sparse submatrix by setting the regularization term as in Expression (3).

[Sparse estimation by Graphical Lasso]
Here, an example of sparse estimation in the case where the regularization term is a functional form by Graphical Lasso will be described. Graphical Lasso is an efficient algorithm for estimating an accuracy matrix from a Gaussian model with L1 regularization. For example, in JEROME FRIEDMAN, TREVOR HASTIE, and ROBERT TIBSHIRANI, Biostatistics (2008), 9, 3 432-441, “Sparse inverse covariance esteation with the graph”.
First, the equation (1) when the equation (2) is adopted, the likelihood with L1 regularization of the Gaussian model is differentiated by the accuracy matrix.

Next, on the condition that the precision matrix is a positive definite matrix, the diagonal components are obtained as in Expression (5).

Next, focusing on a specific variable i, the variance covariance matrix, the precision matrix, and the sample covariance matrix are divided into block matrices as shown in equation (6) to obtain equation (4) as shown in equation (7). Can be transformed.

Expression (7) can be reduced to Expression (8), which is a general Lasso regression optimization problem, and can be obtained by an existing solver or the like.

If the optimal solution β* (beta asterisk) of the equation (8) satisfying the condition of the equation (9) is obtained, the equation (11) derived from the equations (6), (7), and (10) is obtained. By using Expression (12), each component of the block matrix of Expression (6) can be obtained. However, I in Expression (10) is an identity matrix.

The operations of the above equations (5) to (12) are repeated by exchanging rows and columns.

As described above, in the image processing apparatus 10, the correlation extraction unit 104 performs the sparse estimation to reduce the number of correlations between feature amounts. Therefore, according to the image processing apparatus 10, the analysis can be performed in a state where the number of correlations between the feature amounts is reduced. That is, according to the image processing apparatus 10, it is possible to reduce the calculation amount of the analysis of the correlation of the feature amounts. Since it is possible to reduce the number of correlations between feature amounts using the likelihood as an index, it is possible to suppress analysis failure due to, for example, a person arbitrarily reducing the calculation amount of the analysis of the feature amount correlations. it can. In addition, the feature amount was acquired from the image containing the cells in order to obtain the correlation of the feature amount. In addition to the information such as luminance information directly derived from the image, information other than the image information is also used as the feature amount. For example, the location of the brightness information in the cell (nucleus, nuclear membrane, cytoplasm) is estimated from the shape of the image. Further, for example, the location (nucleus, nuclear membrane, cytoplasm) of the stained portion (the portion having the brightness information) of the stained cell is estimated. As described above, in addition to the information directly derived from the image information, the information amount estimated from the image information is used to acquire the correlation of the feature amounts, resulting in a huge amount of calculation. Therefore, since the correlation extraction unit 104 performs the sparse estimation, the number of correlations between the feature amounts can be sparse, and thus the amount of calculation can be prevented from becoming huge.
Further, for example, to explain the case of acquiring the intracellular correlation, when acquiring the intracellular correlation, for example, it may be possible to acquire a plurality of cells in the image, in the plurality of cells, It is possible to obtain the correlation. In this case, when the correlation in a plurality of cells is acquired, the correlation in a plurality of cells can be acquired as compared with the case where the correlation of a single cell is acquired. The accuracy of the route can be improved. When the correlation of a plurality of cells is obtained in order to improve the accuracy, the amount of calculation performed to obtain the correlation becomes enormous. In this case, since the amount of calculation can be reduced using the likelihood as an index, it is possible to suppress poor analysis. Further, for example, when the case of acquiring the correlation between cells is described as an example, when acquiring the correlation between cells, for example, it may be possible to acquire a plurality of cells in the image, and for the plurality of cells, between the cells It is possible to obtain the correlation. In this case, a given cell may be correlated with a plurality of cells, and cells other than the given cell may also be correlated with a plurality of cells. Further, in this case, by acquiring the correlation between the cells, it is possible to improve the accuracy of the signal transduction pathway between the cells, which is calculated as the acquisition of the correlation. When the correlation of a plurality of cells is obtained in order to improve the accuracy, the amount of calculation performed to obtain the correlation becomes enormous. In this case, since the amount of calculation can be reduced using the likelihood as an index, it is possible to suppress poor analysis.
Further, the feature amount calculated by the feature amount calculation unit 102 is involved in the signal transduction in the cell when the signal transduction in the cell after the cell receives the signal from the extracellular is obtained as a correlation, for example. The type of protein to be extracted may be extracted as the feature amount. That is, for example, it may be the type of substance involved in intracellular signal transduction, or the change in the shape of the cell as a result of intracellular signal transduction. The substance involved in intracellular signal transduction may be identified by NMR or the like, or may be a method of analogizing the interacting partner from the staining solution used.

[Determination of regularization parameters]
Next, the determination of the regularization parameter in the sparse estimation performed by the correlation extraction unit 104 will be described. Sparse estimation has the advantage that the model is relatively flexible and that it can be adjusted for human and computer interpretation. On the other hand, in sparse estimation, there is a problem that the model cannot be uniquely determined due to the regularization parameter dependence. A procedure for determining a regularization parameter for solving this problem will be shown. Examples of the parameter determining procedure include a parameter determining procedure by cross validation and a parameter determining procedure using biological knowledge. Of these, the procedure for determining parameters by cross validation will be described.

[Λ estimation by cross validation]
FIG. 8 is a flowchart showing an example of the procedure of cross validation by the correlation extraction unit 104 of this embodiment.
The correlation extraction unit 104 acquires the matrix X, the regularization parameter λ, and the division number K (step S100). The correlation extraction unit 104 divides the matrix X based on the division number K acquired in step S100 (step S110).
Next, the correlation extraction unit 104 processes a double loop including a loop for the regularization parameter λ (step S120) and a loop for the number of divisions K (step S130). In the loop of the division number K, the correlation extraction unit 104 calculates the likelihood Lk using the matrix Xk and the matrix X/k (step S140). In the loop of the regularization parameter λ, the correlation extraction unit 104 calculates the average likelihood Lλ of the likelihood Lk in the regularization parameter λ (step S150). Details of the procedure of calculating the average likelihood Lλ by the correlation extracting unit 104 will be described with reference to FIG. 9.

FIG. 9 is a schematic diagram showing an example of the procedure for calculating the average likelihood Lλ by the correlation extraction unit 104 of this embodiment. The correlation extraction unit 104 divides the matrix X into K pieces in the cell direction Nc in the loop of the division number K described above. The correlation extracting unit 104 calculates the likelihood Lk for the matrix X divided into K pieces.

Specifically, the correlation extraction unit 104 calculates the likelihood L1 for the elements included in the calculation frame W1 shown in FIG. 9 (k=1). Further, the correlation extraction unit 104 calculates the likelihood L2 for the elements included in the calculation frame W2-1 and the calculation frame W2-2 shown in FIG. 9 (k=2) for a certain regularization parameter λ. Further, the correlation extracting unit 104 calculates the likelihood LK for the elements included in the calculation frame WK shown in FIG. 9(k=K). That is, the correlation extraction unit 104 calculates the likelihood Lk from k=1 to k=K for each regularization parameter λ. The correlation extraction unit 104 calculates the average likelihood Lλ by the equation (13) for the likelihood Lk calculated for each regularization parameter λ.

Next, the correlation extraction unit 104 calculates the regularization parameter λ whose average likelihood Lλ indicates the maximum value Max(Lλ) as the regularization parameter λ of the calculation target (step S160), and the regularization parameter λ The calculation ends. An example of the procedure of calculating the regularization parameter λ by the correlation extracting unit 104 will be described.

In this example, the correlation extraction unit 104 calculates the likelihood Lλ when the regularization parameter λ is changed from 0 to 1. Specifically, the correlation extraction unit 104 sets the regularization parameter λ to a certain value from 0 to 1 and calculates the likelihood Lk shown in FIG. 9. The correlation extraction unit 104 calculates the average of the likelihoods Lk, that is, the average likelihood Lλ, and generates the correspondence between the regularization parameter λ and the average likelihood Lλ. FIG. 10 shows the correspondence between the regularization parameter λ generated by the correlation extraction unit 104 and the average likelihood Lλ.

FIG. 10 is a graph showing an example of the relationship between the average likelihood Lλ and the regularization parameter λ of this embodiment. In the case of the example shown in FIG. 10, the average likelihood Lλ shows the maximum value in the regularization parameter λa. In this case, the correlation extracting unit 104 calculates the regularization parameter λa as the regularization parameter λ to be calculated.

In step S110 described above, when the matrix X is divided into K pieces by the division number K, the correlation extraction part 104 divides the matrix X in the cell direction Nc. In other words, the correlation extraction unit 104 extracts the specific correlation by obtaining the likelihood L of the feature amount based on the number of cells captured in the cell image. Here, the correlation extraction unit 104 can also divide the matrix X in the time direction N. However, in order for the correlation extraction unit 104 to divide the matrix X in the time direction N, a plurality of cell images are required in the time direction N. That is, in order for the correlation extraction unit 104 to divide the matrix X in the time direction N, a plurality of cell images taken at different times are necessary. On the other hand, when the correlation extraction unit 104 divides the matrix X into the cell directions Nc, a plurality of cell images captured at different times are unnecessary. That is, the correlation extraction unit 104 of the present embodiment can calculate the regularization parameter λ from one cell image, for example, even if there are no cell images captured at different times.

[Biological interpretation of sparse estimation results]
Next, the biological interpretation of the sparse estimation result by the correlation extracting unit 104 will be described with reference to FIGS. 11 to 13. A biological interpretation can be added to the sparse estimation result by the correlation extracting unit 104 based on the biological information. In this example, the biological information will be described as being stored in the storage unit 200 in advance, but the present invention is not limited to this. The biological information may be supplied from outside the microscope observation system 1 via a network or a portable storage medium. The biological information includes an intracellular component annotation database, a feature amount annotation database, and the like. The intracellular constituents are the constituents of the cell. The elements constituting the cells include, for example, proteins, genes and compounds. Hereinafter, a specific example of the procedure of biological interpretation of the sparse estimation result by the correlation extraction unit 104 will be described.

FIG. 11 is a schematic diagram showing an example of the extraction result of the correlation of the feature amounts by the correlation extraction unit 104 of this embodiment. In this example, the correlation extraction unit 104 extracts the correlations of the feature amounts for the six types of proteins, protein A to protein F, and mitochondria.

In the case of this example, a cell image includes cells including protein A to protein F and mitochondria. The characteristic amount calculation unit 102 calculates the characteristic amounts of the types corresponding to the proteins A to F and mitochondria. That is, the feature amount calculation unit 102 calculates, out of a plurality of types of feature amounts, the feature amount of the type corresponding to the type of the constituent element in the cell included in the cell imaged in the cell image.

Specifically, the correlation extraction unit 104 extracts three types of correlation R1, correlation R2, and correlation R3 between the characteristic amount of the protein A and the characteristic amount of the protein D. .. Here, the correlation R1 is a correlation between the feature amount A3 of the protein A and the feature amount D2 of the protein D. The correlation R2 is a correlation between the characteristic amount A1 of the protein A and the characteristic amount D3 of the protein D. The correlation R3 is a correlation between the characteristic amount A4 of the protein A and the characteristic amount D1 of the protein D. For example, the feature amount A1 is a variance value of the image of the protein A. The feature amount D3 is the brightness value of the image of protein D.

Further, the correlation extraction unit 104 extracts the correlation R4 between the characteristic amount of the protein A and the characteristic amount of mitochondria. Here, the correlation R4 is a correlation between the characteristic amount A2 of the protein A and the characteristic amount M1 of mitochondria. The feature amount A2 is, for example, the ratio of the total nuclear intensity value to the total intracellular intensity value of the image of protein A. For example, the feature amount M1 is a mitochondrial fragmentation rate shown by the mitochondrial image.

FIG. 12 is a table showing an example of the intracellular component annotation database of this embodiment. This intracellular component annotation database associates the type of intracellular component with the function of the intracellular component. In the example of the present embodiment, the intracellular component annotation database is stored in the storage unit 200 in advance. Specifically, in the intracellular component annotation database, the type “protein A” is associated with the function “transcription factor (promotion)”. Further, in the intracellular component annotation database, the type “protein B” is associated with the function “kinase”.

FIG. 13 is a table showing an example of the feature quantity annotation database of this embodiment. This feature amount annotation database associates NW elements, feature amounts, changing directions of feature amounts, and information indicating biological meaning. In the example of the present embodiment, the feature amount annotation database is stored in the storage unit 200 in advance. Specifically, in the feature amount annotation database, the NW element “transcription factor (promotion)”, the feature amount “total nuclear brightness value/intracellular total brightness value”, feature amount change “UP”, and biology The physical meaning “transcriptional promotion” is associated with each other. Further, in the feature amount annotation database, the NW element “mitochondria”, the feature amount “fragmentation rate”, the feature amount change “UP”, and the biological meaning “mitosis, oxidative stress increase, mitophagy enhancement” Are associated with each other. The feature amount annotation database is a specific example of the type storage unit 201 included in the storage unit 200. That is, the type storage unit 201 stores the types of intracellular constituent elements included in the cells captured in the cell image and the types of feature amounts in association with each other.

When the extracted correlation R4, that is, the correlation between the nuclear total luminance value to the intracellular total luminance value of the image of protein A and the mitochondrial fragmentation rate is highly correlated, the following is performed. Biological interpretation can be performed.

That is, it is determined that the function of protein A is “transcription factor (promotion)” based on the intracellular component annotation database. Further, based on the feature amount annotation database, when the feature amount “total nuclear brightness value/intracellular total brightness value” associated with “transcription factor (promotion)” indicates a feature amount change “UP”, the organism It can be determined that the biological meaning is “promoting transcription”. Further, based on the feature amount annotation database, when the feature amount “fragmentation rate” associated with “mitochondria” indicates a feature amount change “UP”, its biological meaning is “mitogenesis, increase in oxidative stress, It can be determined that it is “Mito fuzzy enhancement”.

Based on these determination results, the following biological interpretation of the correlation R4 can be added. That is, (1) protein A promotes transcription of proteins involved in mitochondrial division. (2) Protein A promotes transcription of proteins that enhance mitophage.

As described above, according to the image processing apparatus 10, it is possible to give a suggestion to the biological interpretation of the correlation based on the extraction result of the correlation between the feature amounts of the cells and the biological information. ..
From the feature amount of the cell used to acquire the correlation, biological information of the feature amount is created. That is, the biological information of the feature amount of the cell used to acquire the correlation is created. This allows a biological interpretation of the extracted correlation.

[Feature Feature Network Feature Extraction Using Sparse Estimation Results]
The elements of the feature amount network can be extracted using the sparse estimation result by the correlation extraction unit 104. Here, the feature amount network refers to a network of correlations between feature amounts. Elements of the feature amount network include nodes, edges, subgraphs (clusters), and links. The features of the feature amount network include the presence/absence of hubs and the presence/absence of clusters. For example, whether or not a node has a hub can be determined based on the value of the partial correlation matrix. Here, the hub is a feature amount having a relatively large number of correlations with other feature amounts.

When a hub exists in a certain node, it is considered that the feature quantity that is the hub or the node including the hub has a biologically important meaning. Therefore, the discovery of the presence of the hub may lead to the discovery of important proteins and important feature quantities. That is, by utilizing the sparse estimation result by the correlation extraction unit 104, it is possible to contribute to the discovery of important proteins and important feature quantities.
Therefore, the correlation extraction unit 104 can specify one or more elements from the calculated feature amount network by using the sparse estimation, from the plurality of elements forming the feature amount network. With this, it is possible to extract an element to be considered with priority when acquiring the correlation. By extracting, it can be taken into consideration when acquiring other correlations.
Of course, the correlation extraction unit 104 can specify one or more element groups from the calculated feature amount network by using the sparse estimation, from the plurality of element groups forming the feature amount network. .. As the element group, for example, a plurality of proteins having the same secondary structure of the protein may be one element group.

[Detection of changes between feature networks]
A change between a plurality of feature amount networks can be detected by using the sparse estimation result by the correlation extraction unit 104.
FIG. 14 is a diagram showing an example of a comparative type of the feature quantity network of this embodiment. The feature networks can be compared with each other in each grain size from micro to macro. Specifically, a change in edge granularity can be detected based on the change in the relationship. This edge granularity change detection has d(d-1)/d scores. Further, it is possible to detect a change in node granularity based on a change in a variable. This node granularity change detection has d scores. A change in cluster grain size can be detected based on a change in EGONET. This cluster granularity change detection has d scores. It is possible to detect changes in network granularity based on changes in the network. This network granularity change detection has a score of one.

The detection of changes in node granularity is performed by, for example, the stochastic neighborhood method. The change in node granularity can be detected by the probabilistic neighborhood method by calculating the neighborhood probability between the i-th element and the j-th element of the dissimilarity matrix and scoring the neighborhood probability. In addition, by arranging scores for a certain index of a certain protein for each protein and for each index, the change in the score can be graphed. For example, Tsushioshi Ide, Spiros Papadimitriou, and Michel Vlachos are described in Proceedings of the Event International Conference on Data Mining (IC28) 52, Oct.

The change in cluster grain size is detected by EGONET. Here, EGONET refers to a relationship between a focused node, a node related to the focused node, and an edge formed by these nodes, focusing on individual nodes (Ego) in a feature quantity network composed of nodes and edges. FIG. According to EGONET, the edge of the target node is extracted from all the edges included in the feature amount network. Therefore, comparing EGONETs is easier than comparing feature amount networks. This is useful for visualizing the correlation structure around the node of interest.

As a modification of the change detection between the feature amount networks, there is a quantitative comparison between two or more feature amount networks. For example, a feature network for cells taken from the tissue of a person suffering from a disease can be compared with a feature network for cells taken from the tissue of a healthy person. In this case, it is possible to discover the structure of the feature amount network peculiar to a certain disease by comparing the feature amount networks. Further, in the present embodiment, since the correlation extraction unit 104 uses sparse estimation to derive a sparse feature amount network that is easy to interpret, comparison of feature amount networks is easier than when sparse estimation is not used. is there.

Note that network learning by SVM, random forest, or the like may be used in comparison between feature amount networks. Further, during this network learning, the regularization parameter may be adjusted in order to improve the classification performance.

[Identification of proteins with the same function by comparing feature networks]
When a certain protein P1 and another certain protein P2 have the same function, there is a difference between the edge between the protein P1 and the other protein Pn and the edge between the protein P2 and the other protein Pn. May not be. That is, when the protein P1 and the protein P2 are the same protein, the structures of the feature amount networks are similar to each other. By utilizing this characteristic, it is possible to identify a protein having the same function. For example, at the time of sparse estimation by the correlation extraction unit 104, by extracting the one in which only the protein P1 is excluded from the feature amount network and the one in which only the protein P2 is excluded from the feature amount network, identification of the functional protein is performed. It can be performed.

[Determination of regularization parameter using biological knowledge]
FIG. 15 is a diagram showing an example of a procedure for determining a regularization parameter using the biological knowledge of this embodiment. In this example, the binding of proteins based on biological knowledge is predefined. Specifically, based on biological knowledge, a bond "should have a bond" and a bond "should not exist" between a feature amount of one protein and a feature amount of another protein. And are previously defined as a knowledge database. In the present embodiment, this predefined knowledge database may be stored in the storage unit 200.

The regularization parameter λ is determined by calculating the distance between this knowledge database and the feature amount network indicated by the sparse estimation result by the correlation extraction unit 104, that is, Fscore. This Fscore is expressed as in Expression (16) by the accuracy (Precision) shown in Expression (14) and the reproducibility (Recall) shown in Expression (15). Fscore depends on the regularization parameter, takes the maximum value when the estimated network is closest to the knowledge database, and adopts the regularization parameter at this time. Further, as shown in FIG. 15, the range of regularization parameters can be narrowed even when the knowledge database is not sufficient, that is, when the matrix components are not completely labeled.

FIG. 16 shows the relationship between TP, FP, and FN shown in equations (14) and (15).
FIG. 16 is a diagram showing the relationship between the knowledge base network and the prediction network regarding the presence/absence of connection of feature amount networks according to the present embodiment. Note that the precision is a ratio of an element that is actually an edge among elements predicted to be an edge for a certain element of the feature amount network. Further, the reproducibility (Recall) is a ratio of a certain element of the feature amount network to an element that is actually an edge and an element that is predicted to be an edge.

As described above, the correlation extracting unit 104 may extract the specific correlation by obtaining the likelihood of the feature amount based on the information indicating the characteristics of the cell captured in the cell image.
It should be noted that the correlations extracted by the above-described embodiment may be properly verified using academic literature or a database. In addition, when the database is divided into a plurality of pieces, information from the divided databases may be selected as appropriate. In this case, the divided databases may be divided according to types such as genes and proteins. Further, the extracted correlation may be verified by combining the verification results of a plurality of databases.

A program for executing each process of the image processing apparatus 10 according to the embodiment of the present invention is recorded in a computer-readable recording medium, and the program recorded in the recording medium is read into a computer system and executed. Therefore, the various processes described above may be performed.

The “computer system” here may include an OS and hardware such as peripheral devices. Further, the “computer system” also includes a homepage providing environment (or display environment) if a WWW system is used. The "computer-readable recording medium" is a writable nonvolatile memory such as a flexible disk, a magneto-optical disk, a ROM, a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, or the like. Storage device.

Further, the "computer-readable recording medium" means a volatile memory (for example, a DRAM (Dynamic) in a computer system that serves as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), which also holds the program for a certain period of time. Further, the program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the functions described above in combination with a program already recorded in the computer system.

The embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes a design and the like within a range not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1... Microscope observation system, 10... Image processing apparatus, 20... Microscope apparatus, 101... Cell image acquisition part, 102... Feature amount calculation part, 103... Noise component removal part, 104... Correlation extraction part

In order to solve the above problems, one embodiment of the present invention is a cell image acquisition unit that acquires a cell image including at least one cell, and a plurality of constituent elements that configure the cell included in the cell image , A feature amount of each of the predetermined components is calculated, and a feature amount calculation unit that calculates a plurality of correlations between the feature amounts based on the feature amount, and a likelihood is calculated from each of the plurality of correlations. An image processing apparatus including: a correlation extraction unit that extracts a specific correlation from the plurality of correlations based on the likelihood .

Claims (1)

A cell image acquisition unit that acquires a cell image of a cell imaged,
A feature amount calculation unit that calculates a plurality of types of feature amounts of the cell image acquired by the cell image acquisition unit,
Based on the likelihood of the feature amount, from among the plurality of correlations between the feature amounts calculated by the feature amount calculation unit, a correlation extraction unit that extracts a specific correlation,
An image processing apparatus including.

Images