WO2015162665A1

WO2015162665A1 - Surgery assistance system, projector system, medical image projection method

Info

Publication number: WO2015162665A1
Application number: PCT/JP2014/061160
Authority: WO
Inventors: 俊哉中口
Original assignee: アドバンストヘルスケア株式会社; 国立大学法人千葉大学
Priority date: 2014-04-21
Filing date: 2014-04-21
Publication date: 2015-10-29
Also published as: JPWO2015162665A1; JP5987998B2

Abstract

Provided is a surgery assistance system that can carry out registration of a projector completely automatically. The surgery assistance system comprises a camera (1), a projector (2), a registration tracker (3), and an image processing device (4). A medical image is acquired in advance. The registration tracker (3) is placed on a specific position on the surface of a body. The camera (1) captures an image of the registration tracker (3). The image processing device (4) carries out registration on the basis of an image (m_m) of the registration tracker (3) captured by the camera (1) and registration tracker information (ct) acquired at the same time as when the medical image which was acquired in advance was acquired. The image processing device also acquires target information (p_g) which the projector uses to project onto the object of projection. On the basis of the target information (p_g), the projector (2) projects a target image onto the object of projection.

Description

Surgery support system, projector system, and medical image projection method

The present invention is a surgical support technology using a projector system.

In recent years, technology related to project mapping has attracted attention. By projecting unreal images on the walls of Tokyo Station Station and Osaka Castle, you can create a fantastic space. In addition, it is possible to change the cityscape by projecting artistic images on the wall of the building. Events that use project mapping can delight the eyes of the audience and can be expected to attract a large amount of customers. As a result, a great economic effect can be expected.

Also, since it can project unrealistic images and entertain the eyes of the audience, it is also applied to the field of entertainment (Patent Document 1).

JP 2014-010362 A

On the other hand, surgery support technology using a projector is also being researched in the medical field. Project medical images taken with CT and MRI onto the body surface and organs. Projection makes it easy to grasp the internal structure intuitively and can reduce surgical errors.

At this time, the registration (registration) of the projection image and the projection target is important. Various registration methods have been studied, and there is no established technology.

For example, an image of a body surface is acquired together with a medical image in advance, the body surface image is projected onto the body surface, alignment is performed, and position matching is confirmed, and then a target image (projection target image based on the medical image) Project.

By the way, in the above-mentioned project mapping technology, a video engineer with specialized knowledge uses a special machine. On the other hand, surgeons often lack expertise in video technology. If there is a defect in the registration, not only will it not be possible to provide appropriate surgical support, but conversely, the surgical site may be confused.

¡Slightly unsuccessful surgery is not allowed. The surgeon is forced more than expected. Therefore, it is not preferable to impose any additional burden on the surgeon.

This invention solves the said subject, and aims at providing the surgery assistance system which performs registration of a projector fully automatically.

The surgical operation support system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, and the image processing device is based on the camera. Registration is performed based on a captured image of the registration tracker and registration tracker information acquired at the same time as acquiring a medical image in advance, and target information for the projector to project onto a projection target is acquired. .

The surgical operation support system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, and the image processing device includes: A camera coordinate system setting function unit for setting a viewed three-dimensional coordinate system, a projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector, a positional relationship between the camera and the projector, and the camera coordinates A registration tracker information conversion function unit that acquires registration tracker information of the projector coordinate system based on the registration tracker information of the system, and registration tracker information and medical image coordinates of the medical image coordinate system acquired in advance A medical image input function unit for inputting system target information; Target imaging that acquires target information of the projector coordinate system based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system and the registration tracker information of the projector coordinate system An image acquisition function unit; and a dimension conversion function unit that converts target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates.

With the above configuration, projector registration can be performed fully automatically.

In the above invention, preferably, the camera captures the registration tracker, and the image processing apparatus acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera. It has a registration tracker shooting function section.

In the above invention, preferably, the registration tracker is provided with a registration marker, the camera photographs the registration marker, and the image processing device is configured to capture the camera based on a captured image of the registration marker. The registration marker information of the coordinate system is acquired, and the registration tracker information of the camera coordinate system is obtained based on the positional relationship between the registration tracker and the registration marker and the registration marker information of the camera coordinate system. A registration marker photographing function unit to be acquired is included.

In the above invention, preferably, the projector projects a target image, which is an image related to a target inside the projection target, onto the projection target based on the target information.

In the above invention, preferably, the registration tracker information of the medical image coordinate system is acquired in a state where the registration tracker is installed at a predetermined position on the surface of the projection target.

The surgery support system of the present invention further includes a unit moving mechanism for moving a unit including the camera and the projector.

In the above invention, the registration tracker is a polyhedron.

In the above invention, the image processing apparatus includes a movement detection function unit that repeatedly outputs a control command when the movement of the camera is detected.

This makes it possible to project the target image from multiple directions. Thereby, it can be avoided that the surgeon becomes an obstacle to projection. Furthermore, a pseudo three-dimensional image is obtained.

A projector system of the present invention that solves the above problems includes a camera, a projector, a registration tracker installed on a surface of a projection target, and an image processing device, and the image processing device is viewed from the camera. A camera coordinate system setting function unit for setting a three-dimensional coordinate system, a projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector, a positional relationship between the camera and the projector, and the camera coordinate system A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on the registration tracker information of Pre-image input function to input target information Based on the positional relationship between the registration tracker information of the previous image coordinate system and the target information of the previous image coordinate system, and the registration tracker information of the projector coordinate system. A target captured image acquisition function unit, and a dimension conversion function unit that converts target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates.

The projector system of the present invention can be applied to uses other than surgical support.

In the medical image projection method of the present invention that solves the above-described problem, the medical image creation device creates registration tracker information simultaneously with the creation of the medical image related to the target, and is installed on the surface of the camera, the projector, and the projection target. A unit including the camera and the projector of the surgery support system including the registration tracker and the image processing apparatus is installed so as to correspond to the projection target, and the camera is a surface of the projection target. The registration tracker installed in the camera is photographed, and the registration is performed based on the registration tracker information acquired at the same time when the image processing apparatus acquires the medical track and the captured image of the registration tracker. To the target inside the projection target Acquires target information is image information, the projector, based on said target information, to project the target image on a projection target.

In the medical image projection method of the present invention that solves the above problems, the medical image creation device creates registration tracker information in the medical image coordinate system and target information in the medical image coordinate system in advance, and includes a camera, a projector, Of the surgical operation support system including a registration tracker installed on the surface of the projection target and an image processing apparatus, a unit including the camera and the projector is installed to correspond to the projection target, and the camera Image a registration tracker installed on the surface of the projection target, and the image processing device acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera, The positional relationship between the camera and the projector and the registration of the camera coordinate system The registration tracker information of the projector coordinate system is acquired based on the translation tracker information, the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system, and the projector coordinates Based on the registration tracker information of the system, acquires target information of the projector coordinate system, converts the target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates, and the projector converts the target information into the two-dimensional target information Based on this, the target image is projected onto the projection target.

In the above invention, a unit including the camera and the projector is moved, and each time the unit is moved, photographing by the camera, image processing by the image processing apparatus, and projection by the projector are repeated.

The operation support system of the present invention can perform registration of a projector fully automatically. As a result, the burden on the operator is greatly reduced.

Functional block diagram of system (first embodiment) Registration tracker Registration tracker (variation) Operation concept diagram of this system System processing flow diagram Illustration of operations when creating medical images Operation explanatory diagram at the time of target image projection Functional block diagram of system (second embodiment) System processing flow diagram Pseudo three-dimensional image principle explanatory diagram Pseudo three-dimensional image operation explanatory diagram Functional Block Diagram of System (Second Embodiment) (Modification) Registration tracker (variation) Operation explanatory diagram at the time of target image projection (third embodiment)

<First Embodiment>
~ Configuration ~
FIG. 1 is a functional block diagram of the system. The structure which concerns on the surgery assistance system of this embodiment is demonstrated. The surgery support system includes a camera 1, a projector 2, a registration tracker 3, and an image processing device 4.

Camera 1 and projector 2 are unit 5. Therefore, the positional relationship between the camera 1 and the projector 2 is known and stored in the image processing device 4.

The optical characteristics (lens characteristics) of the camera 1 and the projector 2 are known.

The camera 1 photographs the registration tracker 3. The projector 2 projects the target image onto the projection target. The target image is an image related to a target inside the projection target, and is acquired based on a medical image acquired separately.

FIG. 2 is an example of a registration tracker. The registration tracker 3 is preferably a polyhedron (see the second embodiment). In the figure, it is a hexahedron.

When creating a medical image, the registration tracker 3 is installed at an arbitrary position on the body surface (see FIG. 6). However, it is installed at the same position when the target image is projected (see FIG. 7).

In addition, when imaging the registration tracker 3 as a CT image, it is preferable that the registration tracker 3 is a material (for example, wood) harder than the living skin tissue.

The 3D shape information of the registration tracker 3 is known and stored in the image processing apparatus 4.

A reflection type registration marker 6 is provided on the surface of the registration tracker 3. The registration marker 6 is provided on at least one surface so as to face the camera 1, but it is preferable that the registration marker 6 be provided on a plurality of surfaces as shown (see the second embodiment).

The positional relationship of the registration marker 6 in the registration tracker 3 is known and stored in the image processing apparatus 4.

FIG. 3 shows a modification of the registration tracker. The registration tracker 3 in FIG. 2 is a hexahedron, whereas the modified example has a plate shape. That is, it need not be a polyhedron. In FIG. 2, the registration marker 6 is provided on the surface of the registration tracker 3, whereas in the modified example, the four corners of the registration tracker 3 function as the registration marker 6.

That is, the registration tracker is not limited to this as long as the three-dimensional shape information can be acquired and can be identified by the camera.

Returning to FIG. 1, the functional units 41 to 47 of the image processing apparatus 4 will be described.

The camera coordinate system setting function unit 41 sets a three-dimensional coordinate system viewed from the camera 1 based on the optical characteristics of the camera 1.

The projector coordinate system setting function unit 42 sets a three-dimensional coordinate system viewed from the projector 2 based on the optical characteristics of the projector 2.

The registration marker photographing function unit 43 obtains registration marker information Mm (for example, three-dimensional coordinates of four points at the four corners of the marker) in the camera coordinate system based on the photographed image information m _m (two-dimensional) of the registration marker 6. get. Furthermore, based on the positional relationship between the registration tracker 3 and the registration marker 6 and the registration marker information Mm in the camera coordinate system, the registration tracker information Mt in the camera coordinate system (for example, 4 at the four corners of the opposing surface). 3D coordinates of the point) are acquired.

For convenience of explanation, only one surface of the registration tracker is used. However, when there are registration markers on a plurality of surfaces (see FIG. 2), the camera 1 identifies the registration markers on the plurality of surfaces. This improves the accuracy of registration.

The registration tracker information conversion function unit 44 acquires registration tracker information Pt in the projector coordinate system based on the positional relationship between the camera 1 and the projector 2 and the registration tracker information Mt in the camera coordinate system.

The medical image input function unit 45 inputs registration tracker information CTt of the medical image coordinate system and target information CTg of the medical image coordinate system acquired in advance.

The target photographed image acquisition function unit 46 calculates the projector based on the positional relationship between the registration tracker information CTt in the medical image coordinate system and the target information CTg in the medical image coordinate system, and the registration tracker information Pt in the projector coordinate system. Get target information Pg of coordinate system.

Dimensional transform function unit 47 converts the target information of the projector coordinate system from the three-dimensional coordinates Pg in two-dimensional coordinates p _g.

~ Operation ~
FIG. 4 is a conceptual diagram of the operation of this system, and FIG. 5 is a processing flow diagram of the system. The operation | movement which concerns on the surgery assistance system of this embodiment is demonstrated.

In advance, a medical image coordinate system registration tracker information CTt and a medical image coordinate system target information CTg are created by a medical image creation device (see FIG. 6) (step S1). Medical images include CT images and MRI images. Three-dimensional information can be obtained by overlapping two-dimensional images in layers.

FIG. 6 is an operation explanatory diagram when creating a medical image. The illustration shows a CT scanning device. At the time of CT scan, the registration tracker 3 is installed at an arbitrary position on the body surface. However, it is necessary to set the same position as the installation position of the registration tracker 3 at the time of projecting the target image (see FIG. 7). Therefore, consideration is given so as not to hinder target image projection.

Usually, medical images are created the day before surgery. After creating the medical image, the registration tracker 3 is marked on the body surface, and then the registration tracker 3 is removed.

The medical image coordinate system registration tracker information CTt and the medical image coordinate system target information CTg are input to the image processing apparatus 4.

Next, the operation during the operation will be described. FIG. 7 is an explanatory diagram of an operation at the time of projecting a target image.

The installation position of the registration tracker 3 is marked on the patient's body surface. The registration tracker 3 is installed at a predetermined position.

The unit 5 including the camera 1 and the projector 2 is installed so as to correspond to the projection target (predetermined range on the patient body surface) (step S2). More specifically, the projector 2 can project the target image onto the projection target so that the camera 1 can photograph the registration tracker 3.

The camera 1 photographs the registration tracker 3 installed on the surface of the projection target (step S3).

The image processing apparatus 4 performs registration based on the image taken by the registration tracker by the camera and the registration tracker information acquired at the same time as acquiring the medical image, and the inside of the projection target (under the body surface) Target information, which is image information related to the target at, is acquired (step S4). Details of the image processing will be described below.

First, registration tracker information Mt in the camera coordinate system is acquired based on an image taken by the registration tracker 3 by the camera 1 (step S41).

The camera 1 identifies the registration marker 6. The actual registration DOO marker 6 to form a dimension known square (see FIG. 2), registration preparative marker 6 in the captured image is a two-dimensional form a quadrangle m _m was crushed. Based on the degree of deformation, registration marker information Mm (three-dimensional coordinates) in the camera coordinate system is acquired.

Also, the positional relationship between the registration tracker 3 and the registration marker 6 is known, whereby registration tracker information Mt in the camera coordinate system can be acquired.

Furthermore, the positional relationship between the camera 1 and the projector 2 is known, and registration tracker information Pt in the projector coordinate system is acquired (step S42).

This completes registration. For confirmation, a test image may be projected onto the registration tracker 3.

The positional relationship between the registration tracker information CTt in the medical image coordinate system and the target information CTg in the medical image coordinate system is known, and the target information Pg in the projector coordinate system is acquired (step S43).

The projection image of the projector 2 is two-dimensional. Therefore, to convert the target information of the projector coordinate system from the three-dimensional coordinates Pg in two-dimensional coordinates p _g (step S44).

The projector 2 based on the 2-dimensional target information p _g, projecting a target image on a projection target (step S5).

The surgeon can easily grasp the internal structure intuitively from the target image. Thereby, an operation is started.

~ Effect ~
As described above, in this system, projector registration can be performed fully automatically. There is no risk of disruption of the surgical site due to a registration failure. As a result, the burden on the operator for introducing the system is greatly reduced, and the operator can concentrate on the operation.

Furthermore, with the support of the target image, the surgeon can easily grasp the internal structure intuitively. As a result, the burden on the surgeon for the surgery is greatly reduced, and a reduction in surgical mistakes can be expected.

Second Embodiment
~ Configuration / Operation ~
FIG. 8 is a functional block diagram of the system of the second embodiment. The structure which concerns on the surgery assistance system of this embodiment is demonstrated. The operation support system includes the camera 1, the projector 2, the registration tracker 3, and the image processing device 4 in common with the first embodiment. The second embodiment further includes a unit moving mechanism 7 that moves the unit 5.

For example, in FIG. 7, the unit moving mechanism 7 is a rail that moves the unit 5 and a driving device. The rail is provided in an annular shape on the upper part of the operating table. For understanding, a partial rail cross section is displayed. As the unit 5 makes a round on the rail, the projector 2 can project from all directions.

The image processing apparatus 4 further includes a movement detection function unit 48. The movement detection function unit 48 receives movement information from the unit moving mechanism 7 and outputs a control command repeatedly when detecting movement of the camera 1.

FIG. 9 is a processing flowchart of the system of the second embodiment. The operation | movement which concerns on the surgery assistance system of this embodiment is demonstrated. The operations from steps S1 to S5 are the same as those in the first embodiment. In the second embodiment, the process related to step S6 is further performed.

When the unit moving mechanism 7 moves the unit 5, the position of the projector 2 changes and it is necessary to register again. The image processing device 4 inputs the movement information output by the unit moving mechanism 7 and repeatedly outputs a control command (step S6).

Again, the camera 1 images a registration tracker installed on the surface of the projection target (step S3), and the image processing apparatus 4 performs registration and acquires target information for projection (step S4). The projector 2 projects the target image onto the projection target (step S5), and a series of operations are repeated.

The registration tracker 3 is a polyhedron, and there are registration markers on a plurality of surfaces (see FIG. 2). When the unit 5 moves greatly, the camera 1 recognizes the registration marker on the other surface, and the image processing apparatus 4 performs registration again.

~ Effect ~
During the operation, the position of the operator may change, and the operator may become an obstacle and the target image may not be projected onto the projection target.

In this embodiment, the unit 5 is moved as the operator position changes. Since registration is performed in real time, the target image can be projected again from another direction. Thereby, it can be avoided that the surgeon becomes an obstacle to projection.

Furthermore, in this embodiment, a pseudo three-dimensional image is obtained. This will be described in detail below.

FIG. 10 is a diagram illustrating the principle of a pseudo three-dimensional image. When there are two objects on the same line of sight and the line of sight is moved to the left and right, a near object appears to move greatly and a distant object appears to move slightly. This phenomenon is called visual motion parallax. Thereby, the depth can be recognized.

FIG. 11 is an explanatory diagram of an operation for obtaining a pseudo 3D image. As the unit 5 moves, the projection direction from the projector 2 changes. As a result, a phenomenon similar to visual motion parallax occurs. That is, the object in the deep part from the body surface moves little, and the object near the body surface moves greatly. In the illustrated example, the movement of the sternum is larger than the movement of the spine. Thereby, the depth can be intuitively recognized.

Although the projection image from the projector 2 is two-dimensional information, a pseudo three-dimensional image is obtained by the movement of the unit 5.

~ Modification ~
FIG. 12 is a functional block diagram according to a modification. In FIG. 8, the movement detection function unit 48 inputs movement information from the unit moving mechanism 7, whereas in FIG. 12, the movement detection function unit 48 inputs movement information from the registration marker photographing function unit 43.

That is, when the unit 5 moves, the field of view of the camera 1 also moves, and the camera 1 cannot recognize a known registration marker. The image processing device 4 repeatedly outputs a control command based on the change in the captured image by the camera 1 (step S6).

7 and 8, the unit is moved by the unit moving mechanism 7, but the unit 5 can be made portable by downsizing the camera and the projector. For example, the same effect as in the second embodiment can be obtained when the surgical assistant moves with the unit 5.

FIG. 13 shows a modification of the registration tracker. In the second embodiment, it has been described that the registration tracker 3 is preferably a polyhedron. In FIG. 2, it is a hexahedron, whereas in FIG. 13, it is a dodecahedron. This improves the registration accuracy.

<Third Embodiment>
The present invention is a surgery support system, but is not limited to medical treatment or surgery, and can be applied to a case where the objective is to project an internal structure by a projector.

FIG. 14 is an example of application to a non-destructive inspection of a projector system. The system configuration and system processing flow are the same as those of the surgery support system.

For example, after an earthquake, the internal rebar may be damaged even if there is no damage to the concrete structure by visual confirmation. On the other hand, ultrasonic inspection of the reinforcing bar is performed from the concrete surface to confirm that the reinforcing bar is healthy. The operation of the application example will be described below.

First, 3D CAD information of concrete structures and reinforcements is created in advance based on the design drawings. Further, the installation position of the registration tracker 3 is determined in advance and added to the three-dimensional CAD information (step S1). As a result, the positional relationship between the registration tracker information CDt in the three-dimensional CAD coordinate system and the target information CDg in the three-dimensional CAD coordinate system becomes known.

¡Registration tracker 3 is installed at a predetermined position on the concrete structure surface. Further, the unit 5 including the camera 1 and the projector 2 is installed so as to correspond to the projection target (inspection range of the concrete structure) (step S2). More specifically, the projector 2 can project the target image onto the projection target so that the camera 1 can photograph the registration tracker 3. Whether the camera 1 is properly capturing the registration tracker 3 may be confirmed by a monitor.

As soon as the above preparations are completed, the bar arrangement image is projected onto the concrete structure. The camera 1 images the registration tracker 3 installed on the surface of the projection target (step S3), and the image processing apparatus 4 performs registration and acquires target information for projection (step S4). 2 projects a target image (reinforcement image) onto a projection target (step S5).

Furthermore, a pseudo three-dimensional image may be obtained by moving the unit 5 (step S6).

Referring to the projected image, estimate the position of the target rebar and conduct ultrasonic survey. By visualizing the bar arrangement, it is possible to identify the survey target and improve the search accuracy.

Investigators can concentrate on the investigation without troublesome projection work because the registration of the projector can be performed fully automatically at the investigation site.

DESCRIPTION OF SYMBOLS 1 Camera 2 Projector 3 Registration tracker 4 Image processing apparatus 5 Unit 6 Registration marker 7 Unit moving mechanism 41 Camera coordinate system setting function part
42 Projector coordinate system setting function unit 43 Registration marker imaging function unit 44 Registration tracker information conversion function unit 45 Medical image input function unit 46 Target captured image acquisition function unit 47 Dimension conversion function unit 48 Movement detection function unit

Claims

A camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device,
The image processing apparatus includes:
Based on the image captured by the registration tracker by the camera and the registration tracker information acquired at the same time as acquiring the medical image in advance, registration is performed,
A surgery support system, wherein the projector acquires target information for projecting onto a projection target.
A camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device,
The image processing apparatus includes:
A camera coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the camera;
A projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector;
A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on a positional relationship between the camera and the projector and registration tracker information of the camera coordinate system;
A medical image input function unit for inputting registration tracker information of a medical image coordinate system acquired in advance and target information of a medical image coordinate system;
Target imaging that acquires target information of the projector coordinate system based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system and the registration tracker information of the projector coordinate system An image acquisition function unit;
A dimension conversion function unit for converting target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates;
A surgical operation support system characterized by comprising:
The camera shoots the registration tracker,
The image processing apparatus includes:
The surgery support system according to claim 1, further comprising: a registration tracker imaging function unit that acquires registration tracker information of the camera coordinate system based on a captured image of the registration tracker by the camera.
The registration tracker is provided with a registration marker,
The camera photographs the registration marker,
The image processing apparatus includes:
The registration marker information of the camera coordinate system is acquired based on the photographed image of the registration marker, and based on the positional relationship between the registration tracker and the registration marker, and the registration marker information of the camera coordinate system The surgery support system according to claim 1, further comprising a registration marker imaging function unit that acquires registration tracker information of the camera coordinate system.
The surgery support system according to claim 1, wherein the projector projects a target image, which is an image related to a target inside the projection target, onto the projection target based on the target information.
The surgery support system according to claim 1 or 2, wherein the registration tracker information of the medical image coordinate system is acquired in a state where the registration tracker is installed at a predetermined position on the surface of the projection target.
The surgery support system according to any one of claims 1 to 6, further comprising a unit moving mechanism for moving a unit including the camera and the projector.
The surgery support system according to any one of claims 1 to 7, wherein the registration tracker is a polyhedron.
The image processing apparatus includes:
The surgery support system according to any one of claims 1 to 8, further comprising a movement detection function unit that repeatedly outputs a control command when movement of the camera is detected.
A camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device,
The image processing apparatus includes:
A camera coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the camera;
A projector coordinate system setting function unit for setting a three-dimensional coordinate system viewed from the projector;
A registration tracker information conversion function unit for acquiring registration tracker information of the projector coordinate system based on a positional relationship between the camera and the projector and registration tracker information of the camera coordinate system;
A pre-image input function unit for inputting registration tracker information of the pre-image coordinate system acquired in advance and target information of the pre-image coordinate system;
Target that acquires target information of the projector coordinate system based on the positional relationship between the registration tracker information of the prior image coordinate system and the target information of the prior image coordinate system and the registration tracker information of the projector coordinate system A captured image acquisition function unit;
A dimension conversion function unit for converting target information of the projector coordinate system from three-dimensional coordinates to two-dimensional coordinates;
A projector system comprising:
The medical image creation device creates registration tracker information at the same time as the medical image creation for the target,
Of the surgical support system including a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, a unit including the camera and the projector corresponds to the projection target. Installed as
The camera shoots a registration tracker installed on the surface of the projection target,
The image processing apparatus is
Based on the image captured by the registration tracker by the camera and the registration tracker information acquired at the same time as acquiring the medical image in advance, registration is performed,
Get target information that is image information about the target inside the projection target,
A medical image projecting method, wherein the projector projects a target image onto a projection target based on the target information.
The medical image creation device creates registration tracker information in the medical image coordinate system and target information in the medical image coordinate system in advance,
Of the surgical support system including a camera, a projector, a registration tracker installed on the surface of the projection target, and an image processing device, a unit including the camera and the projector corresponds to the projection target. Installed as
The camera shoots a registration tracker installed on the surface of the projection target,
The image processing apparatus is
Based on the captured image of the registration tracker by the camera, obtain registration tracker information of the camera coordinate system,
Based on the positional relationship between the camera and the projector and the registration tracker information of the camera coordinate system, obtain registration tracker information of the projector coordinate system,
Based on the positional relationship between the registration tracker information of the medical image coordinate system and the target information of the medical image coordinate system, and the registration tracker information of the projector coordinate system, obtain target information of the projector coordinate system,
Convert the target information of the projector coordinate system from 3D coordinates to 2D coordinates,
A medical image projecting method, wherein the projector projects a target image onto a projection target based on the two-dimensional target information.
A unit including the camera and the projector is moved;
Each time the unit moves,
The medical image projection method according to claim 11 or 12, wherein photographing by the camera, image processing by the image processing device, and projection by the projector are repeated.