Keywords

1 Introduction

In recent years, many groups have conducted extensive research on surgical operation navigation systems. Often, they develop navigation systems in the field of orthopedic surgery because bones have few variations. Navigation systems have also been suggested for neurosurgery and otolaryngology [1, 2], which deal with fairly immobile organs that are surrounded by bones. Surgical navigation systems focusing on the kidney and liver are also being explored; these include endoscopic and/or laparoscopic surgery systems [3, 4] and robot surgery systems [5].

We have supported doctors by designing a sensor-based surgical operation navigator for the liver and brain [6,7,8,9,10,11,12,13]. For this, it was essential to accurately measure the surgical area with a depth camera to obtain a depth image. To accomplish this, last year we built a new surgical area-measuring robot-mechanical system and assessed the correlation between the distance the robot traveled and the change in distance of the depth image [14,15,16].

For example, this robot-mechanical system allows a surgeon to raise multiple cameras simultaneously to insert a microscope into the surgical area. Then, when the microsurgery is finished, the surgeon can lower the cameras back into the surgical area. The robot knows the distance that the group of cameras was raised or lowered, so it can achieve consistency in the precision of the depth image before, during, and after the cameras are moved. The system captures images from multiple angles in the surgical area because during long surgeries, the surgeon’s head and arms block parts of the surgical area. Using one camera proved insufficient for acquiring an accurate depth image, so we increased the number of cameras and controlled their infrared pattern emissions to acquire stable and accurate depth images.

The robotic-mechanical system is constructed in such a way that there is a pole next to the surgical area and the system is mounted directly onto the operation bed, which obstructs various tasks performed during surgery. Therefore, in this study, we mounted the pole on the operation bed by the patient’s feet, and using it as a base, we designed and built a new robotic-mechanical system to support the group of cameras in the surgical area. Here we will evaluate the measurement precision.

In Sect. 2 of this study, we will compare the old and new surgical area-measuring robotic-mechanical systems. In Sect. 3, we will evaluate the surgical area-imaging robotics-mechanical system in an operating room that has two operating lights. Finally, in Sect. 4, we will summarize the assessment test and discuss future projects.

2 Comparison of the Old and New Robotic-Mechanical Systems

First, we will introduce the specifications of the Intel Real Sense SR300, and then we will introduce the old and new surgical area measuring robotic-mechanical systems.

2.1 Intel Real Sense SR300

In this study, we used three Intel Real Sense SR300 cameras to acquire serial depth images of the surgical area (Fig. 1, Table 1).

Fig. 1.
figure 1

Real Sense SR300.

Full size image
Table 1. Real Sense SR300 specifications.
Full size table

2.2 Previous Surgical Area-Imaging Robotic-Mechanical System

We constructed a robotic-mechanical system that gives doctors an unobstructed view of the surgical area while being able to freely raise and lower a group of cameras (Fig. 2(a), (b), (c)) [14,15,16]. In this system, the three cameras alternate emission of an infrared pattern, and upon receiving the signal, the system was able to acquire a stable depth image [16].

Fig. 2.
figure 2

(a), (b) Previous robotic-mechanical system (includes the display in the front of the depth image from the three cameras), (c) aerial view of the basic layout.

Full size image

However, when the system was tested, we discovered a flaw. The poles that attach the robotic-mechanical system to the operating bed would often obstruct access to the patient. Therefore, to solve this problem, we built a robotic-mechanical system with the supporting pole located by the patient’s feet.

2.3 The New Model of the Surgical Area-Imaging Robotic-Mechanical System

We constructed a robotic-mechanical system supported by a pole that does not obstruct the doctor, assistant, or nurse from accessing the patient (Fig. 3(a), (b), (c)). Then we evaluated the robotic-mechanical system’s vibrations when the cameras are raised or lowered using the change in the depth image. We will describe the test results in the next section.

Fig. 3.
figure 3

(a), (b), (c) The new robotic-mechanical system (including the placement of the three cameras and robotic slider), (d) aerial view of the basic layout.

Full size image

3 Assessment of the Surgical Area-Imaging Robotic-Mechanical System in the Operating Room

We attached the new surgical area-imaging robotic-mechanical system to an operating bed in an operating room with two shadow-less operating lights. We verified that the center Intel Real Sense SR300 depth camera could acquire a stable depth image when the operating lights were off. Then, using the robotic slider, we raised and lowered the camera approximately 50 cm, 100 cm, and 200 cm to verify that the depth image also changed by 50 cm, 100 cm, and 200 cm, accordingly. Here, we averaged a depth of 30 pixels in the depth image for the depth at each time point. The sampling time was set to 30 frames per second. Then, we evaluated the extent of the robotic-mechanical system’s vibrations by the change in depth over time. Next, we performed the same experiment, but with the operating lights on using the same evaluation described above.

3.1 Change in Depth Over Time (Without Operating Lights)

First, we raised the camera 50 mm, 100 mm, and 200 mm from the starting point (310 mm), and then we lowered it 200 mm, 100 mm, and 50 mm. The resulting graphs shown in Fig. 4 reveal that the depth of the top camera of the surgical area-imaging robotic-mechanical system increases and decreases are approximately equal.

Fig. 4.
figure 4

Graph of the change in depth recorded by the top depth camera of the surgical area-imaging robotic-mechanical system while the operating lights were off (The frame rate is 30 fps).

Full size image

3.2 Vibrations While Raising the Robotic-Mechanical System (Without Operating Lights)

The results for when we raised the camera 50 mm, 100 mm, and 200 mm are shown in Fig. 5(a), (b), (c). There was almost no vibration when it was raised 50 mm and 100 mm. When it was raised 200 mm, however, it vibrated at most 1 s. This shows that the differences in the depth images before and after raising the camera is significant. However, typical surgical operations last 3–4 h and laparotomies take about 20 min, so this vibration has very little effect on the surgical operation navigation.

Fig. 5.
figure 5

Graph of vibrations, evaluated as the change in depth recorded when the robotic-mechanical system’s camera was raised (a) 50 mm, (b) 100 mm, and (c) 200 mm with the operating lights off (The vibration time is less than 30 frames < 1 s).

Full size image

3.3 Vibrations While Lowering the Robotic-Mechanical System (Without Operating Lights)

The results from when we lowered the camera 50 mm, 100 mm, and 200 mm are shown in Fig. 6(a), (b), (c). There was almost no vibration when it was lowered 50 mm and 100 mm, however, when it was lowered 200 mm, it vibrated when it started moving. Fortunately, the vibration time is less than 30 frames < 1 s, which is quite smaller than real periods of several surgeries.

Fig. 6.
figure 6

Graph of vibrations, evaluated as the change in depth recorded when the robotic-mechanical system’s camera lowered (a) 50 mm, (b) 100 mm, and (c) 200 mm with the operating lights off (The vibration time is less than 30 frames < 1 s).

Full size image

3.4 Change in Depth Over Time (with Two Operating Lights)

First, we raised the camera 50 mm, 100 mm, and 200 mm from the starting point (310 mm), and then we lowered it 200 mm, 100 mm, and 50 mm. The resulting graph, which is shown in Fig. 7, reveals that the depth of the top camera of the surgical area-imaging robotic-mechanical system increases and decreases are approximately equally. The depth is more accurate when the camera is on a flat plain and the operating lights are on, which means the operating lights have a positive effect on sensing.

Fig. 7.
figure 7

Graph of the change in depth recorded by the top depth camera of the surgical area-imaging robotic-mechanical system while the operating lights were on.

Full size image

3.5 Vibrations While Raising the Robotic-Mechanical System (with Operating Lights)

The graph of when we raised the camera 50 mm, 100 mm, and 200 mm is shown in Fig. 8(a), (b), (c). There was almost no vibration when it was raised 50 mm and 100 mm, however, when raised 200 mm, there were vibrations, which lasted for a shorter time compared to when the operating lights were off, and the depth was more constant after the vibrations stopped. This means the operating lights have a positive effect on sensing. From these results, we can see that the depth image comparison before and after raising the camera is significant. Typical surgical operations last 3–4 h and laparotomies take about 20 min, so this vibration has very little effect on the surgical operation navigation.

Fig. 8.
figure 8

Graph of vibrations, evaluated as the change in depth recorded when the robotic-mechanical system’s camera was raised (a) 50 mm, (b) 100 mm, and (c) 200 mm while the operating lights were on.

Full size image

3.6 Vibrations While Lowering the Robotic-Mechanical System (with Operating Lights)

The results when we lowered the camera 50 mm, 100 mm, and 200 mm are shown in Fig. 9(a), (b), (c). There was almost no vibration when it was lowered 50 mm, 100 mm, or 200 mm, and depth is more accurate when the camera is on a flat plain and the operating lights are on, which means the illumination from the operating lights has a positive effect on sensing.

Fig. 9.
figure 9

Graph of vibrations, evaluated as the change in depth recorded when the robotic-mechanical system’s camera was lowered (a) 50 mm, (b) 100 mm, and (c) 200 mm while the operating lights were on.

Full size image

4 Concluding Remarks

In this study, we built a new surgical area-measuring robotic-mechanical system with a pole attached to the operating bed that does not obstruct access to the patient. Then, we evaluated the usability of the system by graphing the change in depth measured by the top camera. Our results showed that under two lit operating room lights, even if the robotic slider raised or lowered the camera, there were no abnormal vibrations and the system was able to stably acquire a depth image of the target object.

In future studies, we will examine calculating depth by averaging depths at randomly selected 30 pixels (instead of averaging depths at a fixed set of 30 pixels), which we believe will cancel out errors and provide better results. After that, we plan to change the number of frames to obtain a more stable average depth value (and individual depth values).

We will strengthen the theoretical formulas for moving the robotic slider and camera up and down based on the geometric properties of the robotic-mechanical system and its targets (various organs) as well as the geometric properties of the position of the three cameras and their targets (various organs). In addition, we will calibrate the relationship between the movement of the robot and the movement of the average depth value. We believe this will allow us to more precisely measure the target object.