JP6376196B2 - Cell evaluation apparatus, cell evaluation method and program - Google Patents

Description

  The present invention relates to a cell evaluation apparatus, a cell evaluation method, and a program.

  Stem cells such as ES (Embryonic Stem) cells and iPS (induced Pluripotent Stem) cells can be differentiated into all tissues and can be proliferated indefinitely. . In addition, human ES cells and iPS cells are differentiated and cultured into nerve cells.

  When stem cells are differentiated and cultured into neurons, it is known that externally unique and characteristic structures appear in the cells during the differentiation process of neurons. Therefore, in evaluating the differentiation state into nerve cells, it is effective to evaluate a unique structure in such a differentiation process into nerve cells.

  The state of the structure as described above is evaluated visually. However, the visual evaluation requires human labor, and because the subject's subjectivity is large and cannot be quantified, it is difficult to give reproducibility to the evaluation results. In addition, for example, since the cells are taken out from the incubator and observed, the quality of the cells during the observation is deteriorated and the quality becomes unstable.

  Thus, for example, a technique is known in which the presence of a protein corresponding to a structure in a nerve cell is confirmed by staining with a fluorescent antibody (see, for example, Non-Patent Document 1). With this method, the structure can be identified by the state of the stained cells, and therefore the evaluation result can be given reproducibility.

Leif Dehmelt, Gunnar Poplawski, Eric Hwang and Shelley Halpain `` NeuriteQuant: An open source toolkit for high content screens of neuronal Morphogenesis ''

  However, the technique of Non-Patent Document 1 requires a relatively long time and effort for the work for staining cells, and thus a human burden is large. Further, in the technique of Non-Patent Document 1, since cells are stained and invasion occurs, the cells to be observed cannot be continuously used for differentiation culture.

  The present invention has been made in view of such circumstances, and an object thereof is to make it possible to evaluate the differentiation state into nerve cells stably and non-invasively while reducing human labor.

To solve the problems described above, the cell evaluation apparatus according to one aspect of the present invention includes a storage unit for storing omnifocal image in a state in focus at a position in the thickness direction of the cells, supervised classification process wherein a region extracted from the whole focus image, the common region between the extracted region based on the density difference in the entire focus image, and the Rosetta extraction unit that extracts as Russia Zetta, extracted of the Rosetta state by And a rosetta correspondence determination unit.

Further, the cell evaluation method according to one aspect of the present invention is extracted, and a storage step of storing the entire focus image fine state cell of focused at a position in the thickness direction, from said omnifocal image by supervised classification process a region, said region shared by the extracted region based on the density difference in the entire focus image, Rosetta corresponding steps of: determining the Rosetta extracting the state of the extracted the rosette as b Zetta With.

The program of an aspect of the present invention is extracted into a computer, a storage step of storing the entire focus image fine state in focus at a position in the thickness direction of the cells, from said omnifocal image by supervised classification process a region, said common region with the extracted region based on the density difference in the entire focus image, Rosetta extracting as Russia Zetta, the extracted Rosetta corresponding determining step of determining a state of the Rosetta were It is for execution.

  As described above, according to the present invention, it is possible to obtain an effect that it is possible to evaluate the differentiation state into nerve cells stably and non-invasively while reducing human labor.

It is a figure which shows the structural example of the incubator in embodiment of this invention from the front. It is a figure which shows the structural example of the incubator in this embodiment from a plane. It is a figure which shows the structural example of the incubator of this embodiment. It is a figure which shows the relationship between the differentiation process from a stem cell to a nerve cell, and the phenomenon which arises in a cell. It is a figure which shows the rosetta and neurite which appear in the differentiation process to a nerve cell. It is a flowchart which shows the example of a procedure for the incubator of this embodiment to evaluate the differentiation state of the cell differentiation-induced to a transcellular. It is a figure which shows the example of a more detailed procedure for the incubator of this embodiment to determine the state of a rosetta. It is a figure which shows the example of a more detailed procedure for the incubator of this embodiment to determine the state of a neurite. It is a figure which shows typically the production method of the omnifocal image by the omnifocal image generation part of this embodiment. It is a figure which shows the example of a procedure of the image process performed in order for the rosette extraction part of this embodiment to extract a rosette. It is a figure which shows the example of a result about the soft matching which the rosette extraction part of this embodiment performs. It is a figure which shows the example of a result when the rosette extraction part of this embodiment performs the process which extracts the area | region where a density difference is low from an omnifocal image. It is a figure which shows the example of a result of the noise removal about the area | region where the density difference by the rosetta extraction part of this embodiment is low. It is a figure which shows the example of a result of the rosette extraction process which a rosette extraction part performs using the image after soft matching of this embodiment, and the extraction image of an area | region where a density difference is low. It is a figure which shows the example of the neurite, dead cell, and living cell which the neurite extraction part of this embodiment extracts from a phase difference image. It is a figure which shows the example of a process sequence for the neurite extraction part of this embodiment to extract a neurite, a dead cell, and a living cell. It is a figure which shows the example of a process result of the soft matching which the neurite extraction part of this embodiment performs corresponding to extraction of a dead cell and a nerve cell. Example of results of closing and binarization performed by the neurite extraction unit for extraction of living cells and difference image generation and binarization performed by the transprotrusion extraction unit for extraction of neurites FIG. It is a figure which shows the example of a result of the binarization and common object extraction process which the neurite extraction part of this embodiment performs for extraction of a living cell. It is a figure which shows the example of an image utilized when the neurite extraction part of this embodiment performs the difference image generation performed for extraction of a nerve cell. It is a figure which shows the example of a result of the process from the thinning to neurite extraction which the neurite extraction part of this embodiment performs for extraction of a nerve cell. It is a figure which shows the example of a process result of the dilation and the difference image generation which the neurite extraction part of this embodiment performs for extraction of a dead cell. It is a figure which shows the example of a result of the common object extraction process which the neurite extraction part of this embodiment performs for extraction of a dead cell. It is a figure which shows the example of the area | region division which a neurite extraction part performs when extracting a dead cell and a living cell.

[Configuration example of incubator]
With reference to FIGS. 1-3, the incubator 11 with which the cell evaluation apparatus in embodiment of this invention is applied is demonstrated.
1 and 2 show an example of the structure of the incubator 11. FIG. 1 is a front view of the incubator 11, and FIG. 2 is a plan view of the incubator 11. FIG. 3 shows a configuration example of the incubator 11.

  The incubator 11 shown in these drawings has an upper casing 12 and a lower casing 13. In the assembled state of the incubator 11, the upper casing 12 is placed on the lower casing 13. Note that the internal space between the upper casing 12 and the lower casing 13 is vertically divided by a base plate 14.

  First, inside the upper casing 12, a temperature-controlled room 15 for culturing cells is provided. The temperature-controlled room 15 includes a temperature adjusting device 15a and a humidity adjusting device 15b, and the temperature-controlled room 15 is maintained in an environment suitable for cell culture (for example, an atmosphere having a temperature of 37 ° C. and a humidity of 90%) ( In FIGS. 1 and 2, the temperature adjustment device 15a, the humidity adjustment device 15b, the recovery device 51, the seeding device 52, and the medium exchange device 53 are not shown.

  A large door 16, a middle door 17 and a small door 18 are arranged on the front surface of the temperature-controlled room 15. The large door 16 covers the front surfaces of the upper casing 12 and the lower casing 13. The middle door 17 covers the front surface of the upper casing 12 and isolates the environment between the temperature-controlled room 15 and the outside when the large door 16 is opened. The small door 18 is a door for carrying in and out a culture vessel 19 for culturing cells, and is attached to the middle door 17. It is possible to suppress environmental changes in the temperature-controlled room 15 by carrying the culture container 19 in and out of the small door 18. The large door 16, the middle door 17, and the small door 18 are maintained airtight by the packings P1, P2, and P3, respectively.

  In the temperature-controlled room 15, a stocker 21, an observation unit 22, a container transfer device 23, and a transfer table 24 are arranged. Here, the conveyance stand 24 is disposed in front of the small door 18, and carries the culture container 19 in and out of the small door 18.

  The stocker 21 is disposed on the left side of the temperature-controlled room 15 when viewed from the front surface of the upper casing 12 (the lower side in FIG. 3). The stocker 21 has a plurality of shelves, and each shelf of the stocker 21 can store a plurality of culture vessels 19. Each culture container 19 contains cells to be cultured together with a medium.

The observation unit 22 is disposed on the right side of the temperature-controlled room 15 when viewed from the front surface of the upper casing 12.
Here, the observation unit 22 is disposed by being fitted into the opening of the base plate 14 of the upper casing 12. The observation unit 22 includes a sample stage 31, a stand arm 32 projecting above the sample stage 31, and a main body portion 33 incorporating a microscopic optical system for phase difference observation and an imaging device 33a. The sample stage 31 and the stand arm 32 are disposed in the temperature-controlled room 15, while the main body portion 33 is accommodated in the lower casing 13.

  The sample stage 31 is made of a translucent material, and the culture vessel 19 can be placed thereon. The sample stage 31 is configured to be movable in the horizontal direction, and the position of the culture vessel 19 placed on the upper surface can be adjusted. Further, the stand arm 32 incorporates an LED light source 33b. And the imaging device 33a can acquire the microscope image of a cell by imaging the cell of the culture container 19 permeate | transmitted and illuminated by the stand arm 32 from the upper side of the sample stand 31 via a microscopic optical system.

  The container transport device 23 is disposed in the center of the temperature-controlled room 15 when viewed from the front surface of the upper casing 12. The container transport device 23 delivers the culture container 19 between the stocker 21, the sample table 31 of the observation unit 22, and the transport table 24.

  The container transfer device 23 includes a vertical robot 34 having an articulated arm, a rotary stage 35, a mini stage 36, and an arm unit 37. The rotary stage 35 is attached to the tip of the vertical robot 34 via a rotary shaft 35a so as to be capable of rotating 180 ° in the horizontal direction. Therefore, the rotary stage 35 can make the arm portions 37 face the stocker 21, the sample table 31, and the transport table 24.

  The mini stage 36 is attached to the rotation stage 35 so as to be slidable in the horizontal direction. An arm part 37 that holds the culture vessel 19 is attached to the mini stage 36.

  Next, the lower casing 13 will be described. Inside the lower casing 13, the main body portion 33 of the observation unit 22 and the control device 40 of the incubator 11 are accommodated.

  The control device 40 is connected to the temperature adjustment device 15a, the humidity adjustment device 15b, the observation unit 22, the container transport device 23, the collection device 51, the seeding device 52, and the medium exchange device 53, respectively. The control device 40 comprehensively controls each part of the incubator 11 according to a predetermined program.

  As an example, the control device 40 controls the temperature adjusting device 15a and the humidity adjusting device 15b, respectively, to maintain the inside of the temperature-controlled room 15 at a predetermined environmental condition. The control device 40 controls the observation unit 22, the container transport device 23, the collection device 51, the seeding device 52, the culture medium exchange device 53, etc., and automatically executes the observation sequence of the culture vessel 19. Furthermore, the control device 40 executes a culture state evaluation process for evaluating the culture state of the cells based on the image acquired in the observation sequence.

  The incubator 11 of this embodiment is used, for example, to differentiate and culture human-derived skin cells into nerve cells. In FIG. 3, the functional structural example of the control apparatus 40 corresponding to the differentiation culture to a nerve cell is shown. The operation of each functional unit in the control device 40 is realized by causing a CPU (Central Processing Unit) forming the control device 40 to execute a program, for example.

  In response to the differentiation culture into nerve cells, the control device 40 includes a storage unit 41, an image input unit 42, an omnifocal image generation unit 43, a rosette extraction unit 44, a neurite extraction unit 45, a rosette correspondence determination unit 46, a nerve A protrusion correspondence determination unit 47, a comprehensive determination unit 48, and an apparatus control unit 49 are provided.

The storage unit 41 stores various data required by the control device 40.
The image input unit 42 inputs a captured image captured by the imaging device 33a. In the present embodiment, the imaging device 33a captures a microscope image observed by a microscopic optical system for phase difference observation, that is, a phase difference image obtained by converting the phase difference of light into contrast. Further, the imaging device 33a changes the focal position of the imaging position on the xy plane determined in advance along the Z direction (the depth direction of the sample), performs the captured image at each focal position, and outputs the captured image. . Each of the captured images obtained in this way is a single focus image. The image input unit 42 inputs single focus images as these phase difference images.

The omnifocal image generation unit 43 generates an omnifocal image using a plurality of single focus images input by the image input unit 42. Specifically, the omnifocal image generation unit 43 takes out a region in focus at, for example, a position in the cell thickness direction, and generates an omnifocal image so as to synthesize these regions. Such an omnifocal image is also referred to as an EDF (Extend Depth of Focus) image, for example.
For example, in the process of differentiation from stem cells to neurons, changes in the undulations on the cell surface occur as structures called rosetta appear. For this reason, in a single-focus image, a region where the focus is high and the contrast is high and a region where the focus is low and the contrast is low are mixed. Since the position of the structure in the image is indefinite, it is difficult to obtain an appropriate determination result even if the single focus image is used for evaluating the differentiation state of the nerve cell. Therefore, in the present embodiment, the omnifocal image is used for evaluation when the rosette 100 is extracted. Thereby, an appropriate evaluation result can be obtained.

  The rosetta extraction unit 44 extracts a rosette, which is one of the structures that uniquely appear in the cells during the differentiation process into nerve cells, from the omnifocal image (first original image).

  The neurite extraction unit 45 (structure extraction unit) extracts a neurite, which is one of the structures in the process of differentiation into nerve cells, from the single focus image (second original image). In the differentiation process in which neurites appear, in addition to neurites, dead cells (hereinafter also referred to as “dead cells”) and living cell bodies (hereinafter also referred to as “live cells”) coexist. It becomes a state to do. Therefore, the neurite extraction unit 45 distinguishes and extracts neurites, dead cells, and live cells.

The rosette correspondence determination unit 46 determines the state of the rosette extracted by the rosette extraction unit 44. In addition, the rosette correspondence determination unit 46 determines a predetermined item related to cell differentiation induction based on the determined state of the rosette.
The neurite correspondence determination unit (structure correspondence determination unit) 47 determines the state of the rosette extracted by the neurite extraction unit 45. In addition, the neurite correspondence determination unit 47 determines a predetermined matter related to cell differentiation induction based on the determined neurite state.
The comprehensive determination unit 48 determines a predetermined matter related to cell differentiation induction based on the determination result of the rosette correspondence determination unit 46 and the determination result of the neurite correspondence determination unit 47.

  The device control unit 49 controls the operations of the imaging device 33 a, the temperature adjustment device 15 a, the humidity adjustment device 15 b, the container transport device 23, the collection device 51, the seeding device 52, and the culture medium exchange device 53.

[Rosetta and neurite]
The incubator 11 of this embodiment evaluates the differentiation state of the nerve cell based on the state of rosetta and neurite that appear in the differentiation process into the nerve cell. Therefore, the rosette and the neurite will be described with reference to FIGS.

FIG. 4 shows the process of differentiation of stem iPS (induced Pluripotent Stem) cells into nerve cells over time. When culture is performed to induce differentiation of iPS cells into nerve cells, iPS cells differentiate in the order of neuroepithelial cells, neural progenitor cells, and mature neurons.
Rosetta appears when differentiating from neuroepithelial cells to neural progenitor cells.

FIG. 5A shows an example of a phase contrast image obtained by observing a cell in which the rosette 100 appears with a phase contrast microscope.
For example, the rosette 100 has a shape in which cylindrical cells are arranged radially. The Rosetta 100 has a characteristic shape and appears as an inherent structure in the process of differentiation into nerve cells. Therefore, the appearance of Rosetta 100 has been confirmed, indicating that the differentiation induction of stem cells such as iPS cells into neurons has been successful.
Considering that the Rosetta 100 has the above properties, it can be said that it is effective to evaluate the state of the Rosetta 100 in evaluating the differentiation state into nerve cells. Therefore, in this embodiment, when evaluating the differentiation state into nerve cells, first, the state of the rosetta 100 is determined.

In addition, a neurite appears in the process of differentiation from a neural progenitor cell into a mature neuron, and a phenomenon occurs in which the neurite extends.
In addition, although the example using the iPS cell which is a stem cell is shown in FIG. 4, it uses other stem cells such as ES (Embryonic Stem) cells or other types of differentiated cells by direct reprogramming, etc. Also when differentiation culture is performed, the differentiation process is the same as in FIG.

FIG. 5B shows the structure of the nerve cell 200. The nerve cell 200 has a structure having a cell body 210 and a neurite 300 (axon 220 and dendrite 230) extending from the cell body 210.
The cell body 210 has a nucleus and has a shape that rises round.
The axon 220 has a function of outputting a signal. The axon 220 is thin, has a uniform thickness, is long, and has almost no endoplasmic reticulum or ribosome. The extension speed of the axon 220 is fast.
The dendrite 230 has a function of receiving a signal. The dendrite 230 is relatively thick, but becomes thinner toward the tip. In addition, the extension speed is slow.
The nerve cell 200 is also one of the structures that appear uniquely in the differentiation process of the nerve cell. Moreover, in the neuron in the culturing process, a large number of neurons 200 are overlapped, and it becomes difficult to distinguish the axon 220 and the dendrite 230. Therefore, in this embodiment, the axon 220 and the dendrite 230 are extracted as an extended portion in the nerve cell 200 as the neurite 300 without being distinguished. In addition, the living cell body 210 in the nerve cell 200 is also extracted as a living cell.

[Evaluation procedure]
FIG. 6 shows a procedure example for the incubator 11 to evaluate the differentiation state of cells induced to differentiate into nerve cells.
First, in the incubator 11, human-derived iPS cells or ES cells are accommodated in the culture vessel 19 as stem cells that are induced to undergo differentiation induction into neurons (step S101). Alternatively, other types of differentiated cells may be accommodated in the culture vessel 19 for the purpose of inducing differentiation by direct reprogramming (step S102).

  The stem cells accommodated in the culture vessel 19 in step S101 or S102 are subjected to differentiation induction culture into neuroepithelial cells (step S103). Then, after differentiation into neuroepithelial cells, Rosetta 100 appears in the process of differentiation into progenitor cells (step S104). In addition, the cells in which the rosetta 100 has thus appeared can be grown by differentiation culture by passage.

  The control device 40 performs image processing for extracting the rosette 100 using the image (first captured image) captured by the imaging device 33a (step S105). For this purpose, the control device 40 causes the imaging device 33a to capture an image of the rosette 100 that has appeared in the differentiation process cells. As described above, the image picked up by the image pickup device 33a is a phase difference image observed by a phase contrast microscope, and is a plurality of single focus images each having a different focal position. In addition, the control device 40 performs control so that the imaging device 33a captures a plurality of single-focus images, for example, at each imaging time at a predetermined time interval.

And the image input part 42 in the control apparatus 40 inputs the several single focus image imaged by the imaging device 33a. Further, the omnifocal image generation unit 43 sequentially generates one omnifocal image using a plurality of single focus images captured at the same imaging time. Thereby, the omnifocal image generation unit 43 generates a plurality of omnifocal images corresponding to each imaging time. The plurality of omnifocal images generated in this way are so-called time-lapse images according to time series.
Then, the rosette extraction unit 44 extracts the rosette 100 from the omnifocal image by the image processing as step S105.

  Next, the rosette correspondence determination unit 46 uses the rosette 100 extracted in step S105 to determine the state of the rosette 100 that has appeared in the cell (step S106). The determination items here are various and not particularly limited. For example, various determination items are determined on the basis of the quality and quantity of the rosette 100 and the period required for the appearance.

  As an example, the rosette correspondence determination unit 46 determines the position, differentiation state, differentiation efficiency, and the like of the appeared rosetta 100 (step S107). As the differentiation state, specifically, the shape or quality of the individual or the whole of the rosette 100 is determined. In addition, the differentiation efficiency here can be obtained, for example, as the number of appearances of the rosetta 100 with respect to the number of colonies that have appeared.

  Further, the rosette correspondence determination unit 46 determines the next process for the cell in which the rosette 100 appears (step S108) and predicts the future (step S109) based on the determined state of the rosette 100. That is, the rosette correspondence determination unit 46 can further make a determination regarding a predetermined matter related to cell differentiation culture based on the determined state of the rosette 100.

As the next process determined by the rosette correspondence determination unit 46 in step S108, the following may be mentioned.
That is, the rosette correspondence determination unit 46 can determine that the determination result of the state of the rosette 100 should be used as lot management information as the next step.
In addition, the rosette correspondence determination unit 46 can determine that the cell in which the rosette 100 appears is collected and used as a transplant sample as the next step. When it is determined that the cells should be collected in this way, the device control unit 49 of the control device 40 controls the collection device 51 to collect the cells.
In addition, the rosette correspondence determination unit 46 can determine that the cell in which the rosette 100 appears should be cryopreserved as the next step. The cryopreserved cells are used, for example, for transplantation, storage for mass production purposes, or lot management.
In addition, the rosette correspondence determination unit 46 can determine that the cells in which the rosette 100 appears should be sorted and examined as the next step. In addition, when it determines that it should test | inspect in this way, various test | inspections and analysis are performed (step S110). In this inspection and analysis, the cells may be collected or may be performed using the captured image without being collected.
In addition, the rosette correspondence determination unit 46 can determine that the culture of the cell in which the rosette 100 appears should be continued as the next step.

  Further, the rosette correspondence determination unit 46 can perform the future prediction as step S109 as follows. That is, the rosette correspondence determination unit 46 can predict the yield of differentiation into neurons. In addition, the rosette correspondence determination unit 46 can predict what kind of neurons are differentiated, such as dopaminergic and glutamic acid.

Further, when the differentiation-inducing culture into neuroepithelial cells in step S103 is continued, the process further proceeds to differentiation into neural progenitor cells (step S111). At the stage of differentiation into neural progenitor cells, neurospheroids appear (step S112). This neurospheroid can also be propagated by passage.
In addition, after the appearance of neurospheroids, differentiation induction culture into nerve cells is performed (step S113). In the process of differentiation induction culture in step S113, neurite 200 appears (step S114). When the neurite 200 appears in this manner, a phenomenon occurs in which the protrusions (axons 220 and dendrites 230) in the neurite 200 expand.

  As described above, the control device 40 causes the imaging device 33a to capture an image of the nerve cell in which the neurite 300 is extended, and extracts the neurite 300 using the captured image (second captured image) ( Step S115). In the image processing for neurite extraction, the image input unit 42 inputs a single focus image at each imaging time. And the neurite extraction part 45 extracts the neurite 300 from the phase difference image as a produced | generated single focus image.

  Next, the neurite correspondence determination unit 47 uses the neurite 300 extracted in step S115 to determine the state of the neurite 300 that appears in the cell (step S116). Items used as criteria for this determination are, for example, the process elongation rate, the number of living cells, the number of dead cells, the increase in dead cells, the growth rate of neurites 300 and living cells, and the like. For example, the neurite correspondence determination unit 47 determines a predetermined determination item based on these items. Note that the determination items related to the neurite 300 are various and not particularly limited.

  As an example, the neurite correspondence determination unit 47 determines the differentiation state and quality of the neurite 300 (step S117).

In addition, the neurite correspondence determination unit 47 determines the next process for the nerve cell in which the neurite 300 appears (step S118).
As the next process determined by the neurite correspondence determination unit 47 in step S118, the following may be mentioned.
For example, the neurite correspondence determination unit 47 can determine that the determination result of the current state of the neurite 300 should be used as lot management information as the next step.
Further, the neurite correspondence determination unit 47 can determine that the neuron in which the neurite 300 appears is collected and used as a transplant sample as the next step.
In addition, the neurite correspondence determination unit 47 can determine that the neuron in which the neurite 300 appears should be sorted and examined as the next step.
In addition, the neurite correspondence determination unit 47 can determine that the neuron in which the neurite 300 appears should be used for screening, assay, and the like as the next step.

  Further, the comprehensive determination unit 48 performs a comprehensive determination using the determination result of the rosette correspondence determination unit 46 in steps S107 and S109 and the determination result of the neurite correspondence determination unit 47 in step S117 (step S119).

For example, as a result of the comprehensive determination, the comprehensive determination unit 48 can output the differentiation state and the differentiation efficiency in the differentiation process so far including the appearance of the rosette 100 and the neurite 300 (step S120).
Further, the comprehensive determination unit 48 can determine the next process for the nerve cell at the stage where the neurite 300 appears as the comprehensive determination result (step S118).

Further, the comprehensive determination unit 48 can create a quality database reflecting the comprehensive determination result (step S121). The quality database stores, for example, the differentiation state of the rosetta 100, the differentiation state of the neurite 300, the culture conditions until the neurite 300 appears from the rosetta 100, the period of time, and the like. Also, profiles of genes and protein markers are stored. In addition, test results after final differentiation can be stored. This quality database is stored in the storage unit 41.
Then, the contents of the quality database can be fed back to the next culture step, for example (step S122). Specifically, the comprehensive determination unit 48 sets various items in the next culture step based on the contents of the quality database stored in the storage unit 41. And the apparatus control part 49 performs the notification of the setting item to a user, or temperature, humidity adjustment, seeding | inoculation, culture medium exchange, etc. according to the set item automatically. Thereby, the feedback of the comprehensive determination result to the next culture process is realized.

  In addition, after the neurite 300 appears, differentiation induction culture into mature neurons is performed (step S123). Then, the mature nerve cells are sorted for inspection, for example (step S124). In addition, cells that have differentiated into mature neurons can be used for medical treatment or the like (step S125).

  The flowchart in FIG. 7 shows a more detailed procedure example for the incubator 11 to determine the state of the rosette 100. The procedure shown in this figure corresponds to the procedure including steps S104 to S109, for example, in correspondence with FIG.

In determining the state of the rosetta 100, cells in a stage where differentiation has progressed to neuroepithelial cells are cultured (step S201). And the control apparatus 40 switches the illumination irradiated to the culture container 19 to the thing suitable for appearance of a rosetta, for example (step S202). Moreover, the apparatus control part 49 in the control apparatus 40 performs collection | recovery and sowing (step S203).
And the image input part 42 in the control apparatus 40 inputs the captured image of the cell in the stage immediately after seeding (step S204). The rosette extraction unit 44 performs image processing for extracting a colony from the omnifocal image (original image) generated by the omnifocal image generation unit 43 using the captured images (a plurality of single-focus images) (step S205). ). The colony corresponds to the earliest state of Rosetta 100.
In step S205, the rosette extraction unit 44 processes each of a plurality of omnifocal images as time-lapse images. This also applies to image processing steps described later in FIG.

Next, the rosette correspondence determination unit 46 predicts differentiation efficiency based on the extracted colony quality, size, texture, and other states (step S206). Specifically, the rosette correspondence determination unit 46 predicts, for example, the number of successfully extracted rosettes out of the total number of extracted colonies.
In addition, the rosette correspondence determination unit 46 predicts, for example, the density of the rosetta 100 and the number of days until it differentiates into the rosetta 100 in addition to the prediction result of the differentiation efficiency described above, and continues the culture based on the prediction result. It is determined whether it should be discarded or not (step S207).
If it is determined that the device should be discarded, the device control unit 49 controls, for example, the container transport device 23 so that the neurons cultured so far are discarded (step S208). On the other hand, when it is determined that the culture should be continued, the device control unit 49 controls a predetermined device in the upper casing 12 so that the culture is continued as it is (step S209).

  When the culture is continued in step S209, the image input unit 42 inputs a captured image of cells that appear at the timing immediately before the medium replacement (step S210). The rosette extraction unit 44 performs image processing for extracting the rosette from the omnifocal image generated using the input captured image (step S211).

  Next, the apparatus control unit 49 performs medium replacement at an appropriate timing (step S212). Next, the image input unit 42 inputs a captured image of the cell immediately after the medium exchange (step S213), and the rosetta extraction unit 44 uses a colony from the omnifocal image generated using the input captured image. The image processing for extracting is executed (step S214).

  The cells at the stage of step S213 are further cultured (step S215). Then, the cell at the timing immediately before the subsequent land replacement is imaged, and the image input unit 42 inputs the captured image (step S216). The rosette extraction unit 44 performs image processing for extracting the rosette 100 from the omnifocal image generated using the input captured image (step S217).

Then, the rosette correspondence determination unit 46 determines the state of the rosette based on the colony extracted in step S214 and the rosette 100 extracted in step S217 (step S218). Specifically, the rosette correspondence determination unit 46 determines whether or not the rosette 100 is formed, for example. Further, when the rosette 100 is formed, the quality is determined.
In addition, the rosette correspondence determination unit 46 determines again whether or not to continue the subsequent culture based on the state of the rosette 100 determined in step S218 (step S219).
When it is determined that the rosette correspondence determination unit 46 should be discarded, the cells that have been targeted for determination are discarded (step S220), and when it is determined that the culture should be continued, medium replacement is performed. After performing, the culture is continued (step S221).

  When the medium is exchanged in step S221, the rosetta 100 is in the state of appearance of the cells immediately after the medium exchange (step S222). In the process of further culturing the cells (step S223), the image input unit 42 inputs the captured image of the cell at the timing immediately before the collection (step S224), and the rosette extraction unit 44 inputs the input imaging. Image processing for extracting the rosette 100 generated using the image is executed (step S225).

  Then, the rosette correspondence determination unit 46 determines the state of the rosette 100 extracted in step S225 (step S226). At this stage, the rosette correspondence determination unit 46 determines the quality of the rosette 100, for example.

Next, the rosette correspondence determination unit 46 determines whether to continue or discard the subsequent process based on the determination result of step S226 (step S227).
When it is determined that the rosette correspondence determination unit 46 should be discarded, the cell that has been the determination target is discarded (step S228).
If it is determined in step S228 that the culture should be continued, the determination content is further divided into two.
One determination is to shift to a process for differentiation into neural progenitor cells, for example, because the Rosetta 100 is sufficiently grown. In this case, the device control unit 49 causes the collection device 51 to collect the cells for the next step as differentiation into neural progenitor cells (step S229).
Another determination is that the process proceeds to a process for further growing the rosette 100 because, for example, the rosette 100 is not sufficiently grown. In this case, the apparatus control unit 49 controls the collection apparatus 51 and the seeding apparatus 52 so that the collection and seeding processes in step S203 are performed.

  The flowchart of FIG. 8 shows a more detailed procedure example for determining the state of the neurite 300. The procedure shown in this figure corresponds to the procedure including steps S114 to S118, for example, in correspondence with FIG.

  In determining the state of the neurite 300, cells that have differentiated to neural progenitor cells are cultured (step S301). And the apparatus control part 49 in the control apparatus 40 performs collection | recovery and sowing to each of the collection | recovery apparatus 51 and the sowing apparatus 52 (step S302), and continues culture | cultivation (step S303).

  In the course of culturing in step S303, the image input unit 42 inputs a captured image of a nerve cell in which the neurite 300 appears, that is, a single focus image as a phase difference image (step S304). In addition, the neurite extraction unit 45 executes image processing for extracting the neurite 300 from the input single focus image (original image) (step S305).

  Here, in the cell in which the neurite 300 appears, as described above, the dead cell (dead cell) and the living cell (live cell) other than the neurite 300 are mixed. Yes. These dead cells and living cells are also effective materials for determining the differentiation state of nerve cells. Therefore, depending on the image processing in step S305, not only the neurite 300 but also dead cells and living cells are extracted.

  Then, the neurite correspondence determination unit 47, for example, the length of the extended neurite 300, the neurite 300, the live cell, and the dead cell based on the neurite 300, the dead cell, and the live cell extracted in step S305. Recognize the number of cells and the growth rate of neurite 300 and living cells. Also, using this recognition result, the growth state, differentiation state, etc. of the nerve cell are determined (step S306).

In addition, based on these determination results, the neurite correspondence determination unit 47 further determines subsequent processes in the same step S306.
Specifically, the neurite correspondence determination unit 47 can determine that the nerve cells cultured so far should be discarded. When determined in this way, the device control unit 49 controls, for example, the container transport device 23 so as to discard the nerve cells cultured so far (step S307). In addition, when it is determined that the medium is to be changed and the next process is to be performed, the apparatus control unit 49 causes the medium exchange apparatus 53 to perform the medium replacement (step S308) and then proceeds to the next process. (Step S309). Alternatively, when it is determined that the next process is to be performed without exchanging the culture medium, the apparatus control unit 49 controls a predetermined apparatus in the upper casing 12, for example, so that the process proceeds to the next appropriate process.

[Image processing for Rosetta extraction]
Next, a specific example of image processing for extracting the rosette 100 will be described with reference to FIGS.
As described above, the original image used to extract the rosetta 100 is a phase difference image obtained by imaging a cell through a phase contrast microscope, and is an omnifocal image that is focused on the entire area of the screen. is there. Further, the omnifocal image is formed corresponding to each imaging time of a predetermined time interval, for example. That is, in the present embodiment, a time-lapse image is formed by a plurality of omnifocal images in time series.

  FIG. 9 schematically shows a procedure for generating an omnifocal image by time lapse. For example, at the imaging time t0, the device control unit 49 of the control device 40 causes the imaging device 33a to capture N single-focus images Psf-1 to Psf-N captured at different focal positions. As described above, the single focus images Psf-1 to Psf-N are phase difference images obtained by imaging cells observed through the phase contrast microscope. The image input unit 42 in the control device 40 inputs these single focus images Psf-1 to Psf-N. Then, the omnifocal image generation unit 43 generates an omnifocal image Paf-0 corresponding to the imaging time t0 by, for example, extracting and synthesizing an image region with high contrast from the single focus images Psf-1 to Psf-N. To do.

Similarly, at the imaging time t1 when a predetermined time has elapsed from the imaging time t0, N single-focus images Psf-1 to Psf-N are imaged by the imaging device 33a. Then, the omnifocal image generation unit 43 uses the single focus images Psf-1 to Psf-N input by the image input unit 42 to generate the omnifocal image Paf-1 corresponding to the imaging time t1 as described above. Generate.
Thereafter, similarly, the control device 40 generates the omnifocal image Paf-N using the single focus images Psf-1 to Psf-N captured by the image capturing device 33a every predetermined time.

  As described above, the incubator 11 according to the present embodiment generates the omnifocal image Paf based on the phase difference image every predetermined time. In this way, a time-lapse image is formed by the omnifocal image Paf generated at regular intervals. Then, the rosette extraction unit 44 extracts the rosetta 100 from each of the omnifocal images Paf generated in this way.

The original image to be processed when the neurite extraction unit 45 extracts the neurite 300 may be a single-focus image, but is an omnifocal image Paf formed as a time-lapse image as shown in FIG. Also good. Since the neurite 300 is not as thick as the rosetta 100, even a single focus image can be extracted with sufficient accuracy. However, if the omnifocal image Paf is used, the extraction accuracy is further improved.
However, in the case of the neurite 300, the extension rate is considerably faster than the growth rate of the rosetta 100. For this reason, the time interval of the time-lapse image is set to about 6 hours for extracting the rosette 100, whereas it is set to about 30 minutes to 1 hour for extracting the neurite 300.

  Note that the omnifocal image generation unit 43 may generate the omnifocal image Paf at the timing when the single focus images Psf-1 to Psf-N corresponding to each imaging time are input. In addition, the single-focus images Psf-1 to Psf-N input corresponding to each imaging time are stored in the storage unit 41, and then read out from the storage unit 41 at a predetermined timing thereafter. The omnifocal image Paf may be generated using Psf-1 to Psf-N.

  Next, the flowchart of FIG. 10 shows an example of an image processing procedure executed by the rosette extraction unit 44 in order to extract the rosette 100. The process shown in this figure corresponds to, for example, step S105 in FIG. 6 or steps S205, S211, S214, S217, and S225 in FIG. Further, the processing shown in this figure is for an omnifocal image generated at one imaging time in FIG.

  First, the rosette extraction unit 44 in the control device 40 inputs the omnifocal image generated for the rosette extraction as an original image (step S401). The rosette extraction unit 44 may input the omnifocal image generated by the omnifocal image generation unit 43 and stored in the storage unit 41, for example.

Next, the rosette extraction unit 44 executes soft matching for extracting an image region having a feature corresponding to the rosette 100 from the original image (step S402). Soft matching is one of supervised classifications.
FIG. 11 shows an example of the result of soft matching as step S403. FIG. 11 shows the original image. In the original image that is a phase difference image, the rosette 100 that appears in the cell is observed as a dark grouped region. Focusing on this point, the soft matching in step S403 is to determine a texture for extracting a dark region in advance and extract a region corresponding to this texture from the original image. In FIG. 11, a black area indicated by a white arrow is an area extracted by the soft matching in step S403. The extracted region is a region that is highly likely to be the rosette 100.

In addition, the circular structure that is characteristic of the rosette 100 in the uniformly dark region in the original image as the phase difference image of the rosette 100 has a small density difference in the surrounding area.
Therefore, in parallel with the soft matching in step S403, the rosette extraction unit 44 extracts an area having a density difference equal to or less than a certain value from the original image as the phase difference image (steps S403 to S407).

For this purpose, the rosette extraction unit 44 first performs an erosion process on the original image (step S403).
FIG. 12A shows an example of the original image generated in step S402. FIG. 12B shows an image obtained by eroding the original image in step S403. The eroded image is in a state in which a dark part is enlarged in the original image.

  Next, the rosetta extraction unit 44 generates a difference image obtained by subtracting the original image of FIG. 12A from the image subjected to erosion of FIG. 12B (step S404). FIG. 12C shows the difference image generated in step S404. Next, the rosetta extraction unit 44 performs dilation on the difference image generated in step S404 (step S405). FIG. 12D shows an image that has been subjected to the dilation in step S405. In the image of FIG. 12D generated in this way, the absolute value of the density difference is enlarged.

  Next, the rosetta extraction unit 44 binarizes the image after dilation in FIG. 12D using a predetermined threshold (step S406). The image in FIG. 13A shows the result of binarizing the image in FIG. 12D in step S406. In this figure, the portion indicated by black having the lowest luminance is an area (object) extracted as having a density difference of a certain value or less.

  In addition, in order to further remove noise, the rosette extraction unit 44 deletes objects in the binarized image of FIG. 13A whose size is not more than a certain value (step S407). As a specific example, in the binarized image of FIG. 13A, an object having a small size exists at a position indicated by a broken line. Depending on step S407, for example, such an object is deleted. As a result, the image shown in FIG. 13B is obtained. In the region shown by the broken line in the image of FIG. 13B, the objects whose density is below a certain level and whose size is below a certain level are deleted and do not exist.

  Next, the rosetta extraction unit 44 extracts a portion (object product) in which objects are common between the image subjected to the soft matching in step S403 and the image subjected to the processing in steps S403 to S407 (step S408). ).

14A shows the same image as that in FIG. 11 as the image subjected to the soft matching in step S402, and FIG. 14B shows the image subjected to the processing in steps S403 to S407 as shown in FIG. Shows the same image.
As the processing in step S408, the rosette extraction unit 44 extracts a matching portion between the object shown in the image of FIG. 11 and the object shown in FIG. That is, the rosetta extraction unit 44 extracts a region where the coordinates of the portion shown in black in the image of FIG. 11 and the portion shown in black in the image of FIG. FIG. 14C shows the region extracted in step S409 in black as indicated by a white arrow. The region thus extracted is the rosetta 100.
Then, the rosette extraction unit 44 outputs the image (rosette extraction image) indicated by the rosette extracted in step S408 in this way (step S409).

Note that the processing shown in the flowchart of FIG. 10 is performed on the original image as one omnifocal image generated corresponding to one imaging time in FIG. 9. As can be understood from the above description, the omnifocal image forms a time-lapse image by a set for each imaging time.
Accordingly, the rosette extraction unit 44 performs the processing described with reference to FIG. 10 for each omnifocal image corresponding to each imaging time. Then, the rosette correspondence determination unit 46 can make a determination based on a change in state according to the passage of time, for example, by comparing the states of the rosette 100 extracted from the focus images at the respective imaging times.

[Image processing for neurite extraction]
Next, an example of an image processing procedure executed by the neurite extraction unit 45 in order to extract the neurite 300 will be described.
As described above, the neurite extraction unit 45 extracts not only the neurite 300 but also dead cells and live cells. Therefore, the features of the neurite 300, dead cells, and living cells observed in the original image that is the phase difference image will be described with reference to FIG.
Note that the phase difference image used for extraction of the neurite 300 is not an omnifocal image but a single focus image. The rosette 100 has a height because it is a three-dimensional structure in which cells are gathered, but in the case of the neurite 300, the height is not so high. Therefore, the neurite 300 can be extracted with sufficiently high accuracy without using an omnifocal image.

  FIG. 15A shows an example of a single focus image obtained by imaging neural progenitor cells in which neurites 300 appear. FIG. 15B shows the neurite 300, dead cell 240, and live cell 250 existing in the single focus image of FIG. The living cell 250 is a living cell other than the neurite 300, and specifically, is a living cell as a cell body 210. The dead cell 240 includes the cell body 210 and the neurite 300 in a dead state.

The single focus image is a phase difference image captured through a phase contrast microscope. The phase difference image has a feature that, for example, a portion with a large undulation is bright and a portion with a small undulation is dark.
As understood from FIGS. 15A and 15B, the neurite 300 is observed as a linear dark part in the phase difference image. Therefore, the neurite extraction unit 45 extracts, as the neurite 300, a thin linear portion whose luminance is lower than that of the surroundings in the single focus image (first original image).
Moreover, the dead cell 240 is observed as a granular portion in which a bright small circle is bordered by a dark line in the phase difference image. Therefore, the neurite extraction unit 45 extracts, as a dead cell 240, a granular portion having a luminance of a certain level or more and a luminance gradient of the edge of the single focus image.
In addition, the living cells 250 are observed as a relatively large and dark lump in the phase difference image and as a part in a state where the periphery is brightly bordered. Therefore, the neurite extraction unit 45 extracts, as a living cell 250, a portion of the single focus image whose luminance is below a certain level, whose area is above a certain level and whose edge luminance is above a certain level.

The flowchart of FIG. 16 shows an example of a processing procedure for the neurite extraction unit 45 to extract the neurite 300, dead cells 240, and live cells 250.
First, the neurite extraction unit 45 inputs, as an original image, a single focus image imaged by the imaging device 33a for extraction of the neurite 300 (step S501). At this time, the neurite extraction unit 45 may input a single focus image that is imaged by the imaging device 33a and stored in the storage unit 41, for example, in order to extract the neurite 300.

  The neurite extraction unit 45 executes soft matching for extracting an image region having a feature corresponding to the dead cell 240 from the input phase difference image as the original image (step S502). As the soft matching in step S502, the neurite extraction unit 45 performs a process of extracting, as an object, a portion whose periphery is surrounded by a bright outline, for example.

FIG. 17 shows an example of the result of soft matching processing in step S502.
FIG. 17A shows an example of the original image input in step S501. Through soft matching in step S502, the neurite extraction unit 45 extracts, as an object, an area surrounded by black in FIG. 17B from the original image in FIG. The object extracted in this way is likely to be a dead cell 240.

  Further, the neurite extraction unit 45 performs closing image processing on the original image input in step S501 to extract the living cells 250 (step S503), and further binarizes the closed image (step S503). S504).

FIGS. 18A to 18C show the results of closing and binarization image processing in steps S503 and S504.
FIG. 18A is an example of the original image input in step S501. When the neurite extraction unit 45 performs closing on the original image in FIG. 18A, an image shown in FIG. 18B is obtained. Depending on the closing process, narrow black grooves are filled. As a result, the black mass portion corresponding to the living cell 250 is preserved. However, for example, a thin black portion corresponding to the neurite 300 or a bright portion around a small dark circle such as the dead cell 240 has a surrounding density. Filled with.
Next, the neurite extraction unit 45 binarizes the image of FIG. 18B in step S504, thereby extracting a black area as an object. The object extracted in this way is a region indicated by black in FIG. The object extracted in this way is highly likely to be a living cell 250.

  Further, the neurite extraction unit 45 generates a difference image obtained by subtracting the original image input in step S501 from the image subjected to the closing in step S503 in order to extract the neurite 300 (step S505). Next, the neurite extraction unit 45 binarizes this difference image (step S506).

  FIGS. 18A, 18B, 18D, and 18E show the results of closing and binarization image processing in steps S505 and S506. That is, the neurite extraction unit 45 obtains the difference of the original image in FIG. 18A with respect to the image in FIG. 18B closed in step S503 in step S505, whereby the difference image shown in FIG. Is obtained. The difference image generated in this way is, for example, an image from which the dark mass corresponding to the living cells 250 in the original image has been removed. And the neurite extraction part 45 binarizes this difference image by step S506, and the image shown in FIG.18 (e) is obtained. The white portion in the image shown in FIG. 18E is an object indicating a dark and thin portion in the original image as the original image, and is highly likely to be a neurite 300.

  Further, when extracting the living cells 250, the neurite extraction unit 45 binarizes the original image input in step S501 (step S507). In addition, the neurite extraction unit 45 extracts an image portion (product of objects) in which the object is common between the image binarized in step S507 and the binarized image in step S504 (step S508).

  FIG. 19 shows an image example corresponding to the image processing in steps S507 and S508. FIG. 19A shows an example of the original image input in step S501. FIG. 19B shows a state in which the image obtained by binarizing the original image of FIG. 19A and the image binarized by step S504 are overlaid in step S507.

  In FIG. 19B, white portions are portions classified into white gradations in the image binarized in step S507. In FIG. 19B, the black portion having the lowest luminance is an object corresponding to the portion classified into the black gradation in the image binarized in step S504. This black object corresponds to a dark block area in the original image.

  Then, the neurite extraction unit 45 executes the process of step S508, so that among objects in the image binarized by step S504 as shown in FIG. 19B, there is no bright border around the object. Are excluded. The processing result of step S508 is shown in FIG. Specifically, the four objects extracted in the region surrounded by the broken line in FIG. 19B are excluded in FIG. 19C because their surroundings are not bright in the original image. ing. As described above, as a result of the processing in step S508, only the dark block in the original image whose periphery is brightly bordered is extracted as an object. This means that, for example, the object corresponding to the living cell 250 is narrowed down from the binarized image in step S504. That is, depending on step S508, an object corresponding to the living cell 250 is extracted with high accuracy.

  Also, the neurite extraction unit 45 extracts an object corresponding to the neurite 300 from the image binarized in step S506 as follows. As described with reference to FIG. 18E, the portion of the object classified into the white gradation in the image binarized in step S506 is a dark and thin portion in the original image, and thus may be a neurite 300. Is expensive. However, at the stage of the image binarized in step S506, there is still a high possibility that a relatively large amount of noise components such as dead cells 240 and live cells 250 are mixed in the object.

  Therefore, the neurite extraction unit 45 generates a difference image by subtracting the image processed by the soft matching in step S502 from the image binarized in step S506 in order to extract the object corresponding to the neurite 300. (Step S509).

  In the image processed by the soft matching in step S502, as described above, an image region having a feature corresponding to the dead cell 240 is extracted as an object. Therefore, the difference image generated in step S509 is an image having contents obtained by excluding the object corresponding to the dead cell 240 from the object of the image binarized in step S506.

  Furthermore, the neurite extraction unit 45 further generates a difference image obtained by subtracting the image of the object extracted in step S508 from the difference image generated in step S509 (step S510). The object extracted in step S508 corresponds to the living cell 250 as described above. Therefore, the object corresponding to the living cell 250 is further excluded from the difference image generated in step S510. That is, the difference image generated in step S <b> 510 has content excluding objects that are likely to be dead cells 240 and live cells 250 from objects that are candidates for the neurite 300.

FIG. 20A shows an example of an image binarized in step S506. FIG. 20B shows an image obtained by performing soft matching in step S502 using the original image that is the generation source of the image in FIG. FIG. 20C is an image showing the object extracted in step S508.
For example, the processing in step S509 and step S510 corresponds to processing for subtracting both the image shown in FIG. 20B and the image shown in FIG. 20C from the image shown in FIG.

Next, the neurite extraction unit 45 further executes processing for narrowing down objects corresponding to the neurite 300 from the difference image generated in step S510 as follows.
As described above, the neurite 300 is observed as a thin and dark portion in the original image as the phase difference image. However, when the length is less than a certain value, the neurite 300 is not the neurite 300 but, for example, a living cell 250 or the like. Is likely. Therefore, the neurite 300 can be extracted with higher accuracy by excluding objects with a length equal to or less than a certain length from the objects shown in the difference image generated in step S510.

  That is, in other words, the neurite extraction unit 45 executes thinning processing on the candidate object of the neurite 300 shown in the difference image generated in step S510 (step S511). Note that the thinning method is not particularly limited here.

FIG. 21A shows an example of the content of the difference image generated in step S510. In this figure, the portion indicated by black having the lowest luminance is an object extracted as a candidate for the neurite 300.
When the neurite extraction unit 45 thins the object of the image in FIG. 21A, the image shown in FIG. 21B is obtained. As can be seen from FIG. 21B, the object having a certain thickness in FIG. 21A is converted into a linear shape by thinning in step S511. Since the object is linearized in this way, the object can be quantified by the length.

  Therefore, the neurite extraction unit 45 selects only objects whose length is equal to or greater than a certain length from the thinned objects in step S511 (step S512). Specifically, the neurite extraction unit 45 deletes, for example, thinned objects whose length is less than a preset threshold value, and leaves only those objects that are equal to or greater than the threshold value.

Next, the neurite extraction unit 45 restores the object selected in step S512 to the original thickness (step S513).
FIG. 21C shows an image example showing the object restored in step S512. Here, the object existing in the area enclosed by the broken line in FIG. 21A is deleted in step S512 because its length is less than a certain length, so in the restoration stage of FIG. 21C. Does not exist. In this way, in the image at the stage restored in step S513, noise corresponding to the dead cell 240 and the live cell 250 is removed, and the object corresponding to the neurite 300 is extracted with high accuracy.
Then, the neurite extraction unit 45 outputs an image indicating the object restored in step S513 as a result of extracting the neurite 300 (step S514).

  As described above, depending on the soft matching in step S502, an image obtained by extracting, as an object, a portion whose periphery is surrounded by a bright outline as a portion corresponding to the dead cell 240 is obtained. In addition, the neurite extraction unit 45 performs the following processing in order to extract dead cells 240 with higher accuracy.

  That is, the neurite extraction unit 45 performs dilation (expansion) on the original image input in step S501 (step S515), and further, the original image input in step S501 from the image subjected to dilation in step S515. A difference image is generated by subtracting (step S516).

  Next, the neurite extraction unit 45 binarizes the difference image generated in step S516 (step S517). Then, the neurite extraction unit 45 extracts, as an object, a common part between the object binarized in step S517 and the object extracted by the soft matching in step S502 (step S518).

  FIG. 22 shows an example of an image corresponding to the processing of step S515 and step S516. 22A is an example of the original image input in step S501, and FIG. 22B is an image obtained by dilating the original image in FIG. 22A in step S516. FIG. 22C is an image obtained by subtracting the image of FIG. 22A from the image of FIG. 22A in step S516. In the image shown in FIG. 22C, a portion having a large density gradient is emphasized.

FIG. 23 shows an example of an image corresponding to the processing in steps S517 and S518.
FIG. 23A shows an example in which the difference image shown in FIG. 22C is binarized in step S517 and a portion classified into white gradation is extracted as an object.
FIG. 23C is an image showing the object extracted as a common part between the object shown in FIG. 23A and the object shown in FIG. 23B in step S508. As can be seen by comparing FIG. 23B and FIG. 23C, in the stage of step S518, objects as dead cells 240 are further narrowed down from the objects extracted by the soft matching.

  As understood from the above description, the object extracted in step S508 corresponds to the living cell 250. However, in the stage of step S508, the objects corresponding to the plurality of living cells 250 are extracted as one object because they are close to each other even though they are actually separate. There may be. The same is true for the dead cell 240 object extracted in step S518.

For example, in determining the state of the neurite 300, the number of live cells 250 and dead cells 240 is an effective determination factor. For this reason, it is preferable that the number of objects corresponding to each of the live cell 250 and the dead cell 240 in the extraction result is as accurate as possible.
Therefore, the neurite extraction unit 45 performs image processing for region division on the object of the living cell 250 extracted in step S508 (step S519). In other words, the neurite extraction unit 45 divides the object, which is estimated as a plurality of living cells 250 extracted as a single overlapping cell, corresponding to each of the plurality of living cells 250. As a result, the object of the living cell 250 that is in contact with each other and extracted as one object is divided into a plurality of objects. In addition, the neurite extraction unit 45 performs image processing for area division on the dead cell 240 object extracted in step S518 (step S521). As a result, the dead cell 240 object that is extracted as one object in contact with each other is divided into a plurality of objects.

The algorithm for area division in step S519 or S521 is not particularly limited, but a moisture boundary separation algorithm is known as one of the effective algorithms.
FIGS. 24A and 24B schematically show region division by moisture boundary separation. For example, FIG. 24A illustrates a state in which two circular objects OBJ1 and OBJ2 are in contact with each other so as to overlap each other. In this case, the object OBJ1 and the object OBJ2 are separated as shown in FIG. 24 (b) by performing the divided areas by the moisture boundary separation algorithm.

In addition, the living cells 250 are larger than the dead cells 240, and their shapes are relatively diverse. On the other hand, dead cells 240 are relatively small, and exist in a nearly circular state on the original image.
Therefore, the neurite extraction unit 45 performs region division adapted to each of the live cell 250 and the dead cell 240 in steps S519 and S521 as follows.

  Specifically, FIGS. 24C and 24D show an example of region division for the object of the living cell 250 in step S519. FIG. 24A is an image before area division (that is, an image showing the object extracted in step S508), and FIG. 24B is an image after area division.

  In FIG. 24 (a), the part estimated that there is a possibility that the object of the two living cells 250 is in contact (is a watershed) is, for example, a part enclosed by a broken line A and a broken line B. Here, when two objects are overlapped at a part indicated by a broken line A, a dividing line (watershed) when these objects are divided becomes long to some extent. On the other hand, the length of the watershed between the objects when the two objects overlap each other at the part indicated by the broken line B is considerably shorter than the part indicated by the broken line A.

  As described above, the living cell 250 has a feature that its shape is various. From this point, there is a high possibility that an object including a part enclosed by a broken line A is actually one living cell 250. On the other hand, the part indicated by the broken line B is likely to be a part where two existing objects overlap in reality. Therefore, in the region division in step S519 corresponding to the living cell 250, the neurite extraction unit 45 only when the length of the watershed (division line) when the overlapping portion is divided is equal to or less than a certain value. Make a split. Thus, when the region division is performed on the image shown in FIG. 24C, the result is as shown in FIG. That is, the area indicated by the broken line A is not divided, and the area indicated by the broken line B is divided.

On the other hand, the dead cell 240 is relatively small as described above, and the shape of the dead cell 240 is almost circular. Therefore, when dividing the region in step S519 corresponding to the living cell 250, the neurite extraction unit 45 divides all the parts detected as being a watershed, for example, regardless of their lengths.
FIG. 24E is an example of an image showing the object extracted in step S518. In this image, a portion indicated by a broken line C is estimated as a watershed. Therefore, as shown in FIG. 24F, the neurite extraction unit 45 performs region division so as to divide the portion indicated by the broken line C in step S521.

  Then, the neurite extraction unit 45 outputs the extraction result of the object after performing the divided region in step S519 as the extraction result of the live cell 250 (step S520). Further, the neurite extraction unit 45 outputs the extraction result of the object after performing the divided region in step S521 as the extraction result of the dead cell 240 (step S522).

  Note that the processing shown in the flowchart of FIG. 16 is also performed on the original image as a single focus image generated corresponding to one imaging time in FIG. Therefore, the neurite extraction unit 45 executes the process of FIG. 16 corresponding to each single focus image that forms a time-lapse image.

Thus, in the present embodiment, the rosette 100 and the neurite 300 are extracted from the image, and the state of the rosetta 100 and the neurite 300 is determined based on the extraction result. That is, in the present embodiment, the incubator 11 automatically extracts the rosetta 100 and the neurite 300 and various determinations based on the extraction result. Thereby, in this embodiment, reduction of human labor is achieved.
In this embodiment, since the state of the rosette is determined based on the content of the image obtained by imaging the cell, the differentiation state of the nerve cell can be stably evaluated non-invasively. In the present embodiment, a phase difference image is adopted as an image format in which the rosette 100 and the neurite 300 can be effectively extracted non-invasively.
In this embodiment, since the cells in the environment cultured in the incubator 11 are imaged, it is not necessary to interrupt the culture for imaging and take it out of the incubator. This stabilizes the quality of the cells in the culture process without deterioration.

In the present embodiment, the neurite 300 is also extracted together with the rosetta 100, and the cell state is comprehensively determined based on the state of these structures. For example, the state of the cell is determined only by the rosette 100. May be. However, a more reliable determination result can be expected when the extraction result of the neurite 300 is also used in combination.
Further, the structure to be extracted together with the rosette 100 may be other than the neurite 300 such as a neural tube. Further, after extracting the rosette 100 and the neurite 300, other structures may also be extracted, and the cell state may be determined based on these extraction results.

3 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system and executed to extract a structure. A state determination or the like may be performed.
Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The inventions described in the above embodiments are organized and disclosed as supplementary notes.

(Appendix 1)
An image input unit for inputting a first captured image obtained by imaging a cell in a neural cell differentiation process;
An omnifocal image generation unit that generates an omnifocal image in a focused state at a position in the cell thickness direction as a first original image based on at least the first captured image;
A rosetta extraction unit that extracts a region common to a region having a luminance distribution below a certain level in the first original image and a region having a density difference below a certain level in the first original image as a rosette appearing in a differentiation process When,
A rosette correspondence determination unit that determines the state of the extracted rosette;
A cell evaluation apparatus comprising:

(Appendix 2)
The Rosetta correspondence determination unit
Making a determination regarding the process or future prediction associated with the differentiation of the cell based on the determined Rosetta state;
The cell evaluation apparatus according to Appendix 1.

(Appendix 3)
The image input unit
Input a second captured image that is a single-focus image obtained by imaging cells in the differentiation process in which a predetermined structure other than the rosetta appears,
A structure extraction unit that extracts the predetermined structure from a second original image based on the second captured image;
A structure correspondence determining unit that determines the state of the extracted structure;
The cell evaluation apparatus according to Supplementary Note 1 or Supplementary Note 2.

(Appendix 4)
The structure correspondence determination unit
Determining the differentiation state of the structure or the quality of the structure related to the differentiation induction of the cell based on the determined state of the structure;
The cell evaluation apparatus according to Appendix 3.

(Appendix 5)
Based on the determination result of the Rosetta correspondence determination unit and the determination result of the structure correspondence determination unit, further includes a comprehensive determination unit that performs determination on the differentiation state or differentiation efficiency related to the differentiation culture of the cell,
The cell evaluation apparatus according to Supplementary Note 3 or Supplementary Note 4.

(Appendix 6)
The image input unit
Inputting the first captured image obtained by capturing a phase difference image obtained by observing the cells with a phase contrast microscope;
The cell evaluation apparatus according to any one of appendix 1 to appendix 5.

(Appendix 7)
The omnifocal image generation unit includes:
Acquiring the first captured image as a plurality of single-focus images captured by changing the focal position of the cell on the predetermined xy plane along the depth direction of the cell; 7. The cell evaluation apparatus according to claim 6, wherein each of the plurality of single-focus images is combined to generate at least the first original image as an omnifocal image in a focused state at a position in the cell thickness direction. .

(Appendix 8)
An image input step for inputting a captured image obtained by imaging cells in the process of differentiation of nerve cells;
An omnifocal image generation step of generating an omnifocal image in a focused state at a position in the cell thickness direction as a first original image based on at least the captured image;
A rosetta extraction step of extracting a common area between a region having a luminance distribution below a certain level in the first original image and a region having a density difference below a certain level in the first original image as a rosette appearing in the differentiation process When,
Rosette correspondence determination step for determining the state of the extracted rosette;
A cell evaluation method comprising:

(Appendix 9)
On the computer,
An image input step for inputting a captured image obtained by imaging a cell in the process of neural cell differentiation,
An omnifocal image generating step for generating an omnifocal image in a focused state at a position in the thickness direction of the cell as a first original image based on at least the captured image;
A rosetta extraction step of extracting a common area between a region having a luminance distribution below a certain level in the first original image and a region having a density difference below a certain level in the first original image as a rosette appearing in the differentiation process ,
Rosette correspondence determination step for determining the state of the extracted rosette,
A program for running

DESCRIPTION OF SYMBOLS 11 Incubator 12 Upper casing 13 Lower casing 14 Base plate 15 Temperature-controlled room 15a Temperature adjusting device 15b Humidity adjusting device 16 Large door 17 Middle door 18 Small door 19 Culture vessel 21 Stocker 22 Observation unit 23 Container transfer device 24 Transfer stand 31 Sample stand 32 Stand Arm 33 Main body part 33a Imaging device 33b LED light source 34 Vertical robot 35 Rotating stage 35a Rotating shaft 36 Mini stage 37 Arm unit 40 Controller 41 Storage unit 42 Image input unit 43 All-focus image generating unit 44 Rosetta extracting unit 45 Neurite extracting unit 46 Rosetta Correspondence Determining Unit 47 Neurite Corresponding Determining Unit 48 Comprehensive Determining Unit 49 Device Control Unit 51 Recovery Device 52 Seeding Device 53 Medium Exchange Device 100 Rosetta 200 Neuronal Cell 210 Cell Body 220 Axon 23 Dendritic 240 dead cells 250 cells 300 neurite

Claims (12)

A storage unit for storing the entire focus image in a state in focus at a position in the thickness direction of the cells,
Said the supervised classification process is extracted from the entire focus image region, wherein the common area between the extracted region based on the density difference in the entire focus image, and the Rosetta extraction unit that extracts as a B Zetta,
A rosette correspondence determination unit that determines the state of the extracted rosette;
A cell evaluation apparatus comprising: The storage unit stores a plurality of omnifocal images focused in all regions of an image generated using a plurality of single-focus images captured at predetermined time intervals;
The rosette extraction unit uses, as a rosetta, a region common to a region extracted from the plurality of omnifocal images by the supervised classification process and a region extracted based on a density difference in the plurality of omnifocal images. The cell evaluation apparatus according to claim 1, which is extracted. The cell evaluation apparatus according to claim 1, wherein the region extracted from the omnifocal image by the supervised classification process has a luminance distribution of a certain value or less. The Rosetta correspondence determination unit performs a determination regarding a process or future prediction related to the differentiation of the cell based on the determined Rosetta state.
The cell evaluation apparatus according to any one of claims 1 to 3. The storage unit stores a single-focus image obtained by imaging cells of a structure other than the rosetta,
The cell evaluation apparatus comprises:
A structure extraction unit for extracting the structure from the single focus image;
A structure correspondence determining unit that determines the state of the extracted structure;
The cell evaluation apparatus according to any one of claims 1 to 4. The structure correspondence determining unit performs determination on the differentiation state of the structure or the quality of the structure related to the differentiation induction of the cell based on the determined state of the structure.
The cell evaluation apparatus according to claim 5. Based on the determination result of the Rosetta correspondence determination unit and the determination result of the structure correspondence determination unit, further includes a comprehensive determination unit that performs determination on the differentiation state or differentiation efficiency related to the differentiation culture of the cell,
The cell evaluation apparatus according to claim 5 or 6. The storage unit stores the omnifocal image obtained by capturing a phase difference image obtained by observing the cells with a phase contrast microscope.
The cell evaluation apparatus according to any one of claims 1 to 7. The cell evaluation apparatus according to any one of claims 1 to 8, wherein the supervised classification process is soft matching. The cell evaluation device according to any one of claims 1 to 9, wherein the region extracted based on the density difference in the omnifocal image is a region having a density difference equal to or less than a fixed value in the omnifocal image. A storage step of storing an omnifocal image in a focused state at a position in a cell thickness direction;
A rosetta extraction step for extracting, as a rosette, a common area between the region extracted from the omnifocal image by supervised classification processing and the region extracted based on the density difference in the omnifocal image;
Rosette correspondence determination step for determining the state of the extracted rosette;
A cell evaluation method comprising: On the computer,
A storage step of storing an omnifocal image in a focused state at a position in a cell thickness direction;
A rosetta extraction step for extracting, as a rosette, a common area between the region extracted from the omnifocal image by supervised classification processing and the region extracted based on the density difference in the omnifocal image;
Rosette correspondence determination step for determining the state of the extracted rosette;
A program for running