@phdthesis{ nicolai2016,
    author = {Philip Nicolai},
    title =  {{A 3D camera-based system concept for safe and intuitive use of a surgical robot system}},
    school = {{Karlsruhe Institute for Technology (KIT)}},
    year =   2016,
    month =  {June},
    type = {Ph.D.} dissertation,
    note = {Published as book with ISBN 978-3-8439-2845-8 by Dr. Hut Verlag, Munich},
    }

Within the last decades, surgical robot systems have been integrated operation rooms worldwide. However, in current robotic procedures, the surgical personnel has to devote a significant part of attention in order to ensure and monitor seamless functioning of the robot system.
To overcome this limitation, this thesis explores the feasibility of developing a system for safe and intuitive use of surgical robots, based on state-of-the-art range imaging cameras and newly developed algorithms. A novel concept for an Operating Room (OR) monitoring system is proposed that can perceive the environment of a surgical robot using multiple 3D cameras and detect potentially harmful situations between the robot, its surroundings and the persons in its vicinity, i.e. the OR personnel and the patient. Such a system is realized in a generic way in order to be applicable to different surgical robot systems. It is optimized for a low spatial footprint for not interfering with the OR personnel and their actions in already crowded ORs. Furthermore, the system provides intuitive feedback to the OR personnel whenever safety-critical events are detected, without drawing on their attention otherwise.
The realized system was extensively evaluated using the OP:Sense surgical research platform. Based on the proposed approach of establishing a virtual safety zone around each robot arm, the system was shown to reliably detect and therefore avoid impending collisions, without requiring information about the trajectory of the robot. To ensure the applicability of use within the operating room, the effects of sterile draping on range imaging cameras were analyzed. A filtering method was put forward to eliminate these effects within the realized ToF camera system, allowing for successful detection of impending collisions even for draped robots.
The results indicate that a 3D-camera-based supervision system can effectively contribute to the safety of use of surgical robot systems in the OR, allowing the OR personnel to completely focus on their medical tasks. The proposed methods contribute to scene supervision for human-robot cooperation and show the feasibility of the approach.

	Forward propagation of semantic labelling (full scene)
	Forward propagation of semantic labelling (single ToF camera only)
	Projection of surgical instruments
	Collision detection with moving human and static robots