Shared control of flexible needles for robot-assisted biopsies
Advisor: Alexandre Krupa
Co-Supervisor: Claudio Pacchierotti
Please also send your list of marks (even preliminary) of your Master 2 or engineer formation.
Needle insertion in soft-tissue is a minimally invasive surgical (MIS) procedure used for diagnostic and therapeutic purposes, and it is one of the many surgical procedures that may greatly benefit from the use of teleoperated robotic systems. Hence, researchers have been constantly trying to develop new techniques and systems able to improve its safety and accuracy. Flexible needles and haptic feedback are two of these technological advancements. Flexible needles provide the clinician with enhanced steering capabilities, and haptic feedback enables the clinician to receive information about the forces exerted by the needle on the soft tissue being penetrated.
In our previous works, we studied different approaches to automatically steer a flexible needle actuated by a robotic arm, in order to accurately position its tip on a desired target by visual servoing and 3D ultrasound imaging. However, for reasons of safety and responsibility, it would be beneficial to provide clinicians with direct control of the motion of the medical instrument.
We propose to study innovative teleoperation systems for steering flexible needles, exploiting grounded and ungrounded haptic stimuli for our vision-based needle insertion system, with the final objective of maximizing the information provided, the clinician comfort, and the medical procedure’s safety and effectiveness. The project will proceed by developing four main key aspects:
– Perception of multiple haptic stimuli. At first, we will study the effectiveness of combining multiple haptic stimuli, focusing on force, vibrations, normal indentation, and skin stretch. We will focus on stimuli being able to provide multi-directional information, applied to different parts of the body, such as the hand, wrist, and forearm.
– Visual servoing. We propose to develop new ways of assistance solutions where the clinician will keep total or partial manual control of the needle positioning. This could be achieved by sharing different degrees of liberty of the needle between the robot and the clinician through teleoperation.
– Shared control with haptic guidance. To help clinicians steer the needle toward the target, we will study how to provide effective guiding haptic stimuli. Haptic feedback will be used to enforce active constrains aimed at safely positioning the needle without damaging the tissues and also to provide guiding information extracted from the current image.
– Safety and stability. We will work to improve existing stability control approaches to take into account for the additional tactile stimuli, focusing on time-domain energy-based techniques, with the objective of maximizing transparency while guaranteeing the overall safety of the system.
These methods will be developed, tested, and validated on the Rainbow team’s medical robotics platform. It consists of an ultrasound station equipped with 2D or 3D ultrasound probes that can be attached on an anthropomorphic robot with 6 degrees of freedom (DoF) equipped with a force sensor. A second 6 DoF manipulator robot, also equipped with a force sensor, will be used to operate the needle. Haptic stimuli can be transmitted to the user by a 6 DoF haptic interface (Virtuose 6D device arm from Haption) and wearable haptic devices able to apply vibrations and/or skin stretch to the user’s arms/hands. All this equipment is already available on the team’s robotic platform.
Figure. Robotic platform for needle steering: 6-dof robot holding an 3D ultrasound probe, 6-dof robot actuating
a flexible needle, Haption Virtuose 6D haptic device.
 N. Abolhassani, R. V. Patel, and M. Moallem, “Needle insertion into soft tissue: A survey,” Medical engineering & physics, vol. 29, no. 4, pp. 413–431, 2007.
 C. Papalazarou, P. M. Rongen, et al., “Surgical needle reconstruction using small-angle multi-view x-ray,” in Proc. IEEE International Conference on Image Processing (ICIP), pp. 4193–4196, 2010.
 V. Duindam, R. Alterovitz, S. Sastry, and K. Goldberg, “Screwbased motion planning for bevel-tip flexible needles in 3d environments with obstacles,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 2483–2488, 2008.
 K. Hauser, R. Alterovitz, N. Chentanez, A. M. Okamura, and K. Goldberg, “Feedback control for steering needles through 3d deformable tissue using helical paths,” in Proc. Robotics: Science and Systems (RSS), vol. 37, 2009.
 M. Abayazid, R. J. Roesthuis, R. Reilink, and S. Misra, “Integrating deflection models and image feedback for real-time flexible needle steering,” IEEE Transactions on Robotics, vol. 29, no. 2, pp. 542–553, 2013.
 M. Abayazid, M. Kemp, and S. Misra, “3D flexible needle steering in soft-tissue phantoms using Fiber Bragg Grating sensors,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 5823–5829, 2013.
 J. Troccaz and Y. Delnondedieu, “Semi-active guiding systems in surgery. a two-dof prototype of the passive arm with dynamic constraints (padyc),” Mechatronics, vol. 6, no. 4, pp. 399 – 421, 1996. Mechatronics in Surgery.
 M. Jakopec, F. Rodriguez y Baena, S. Harris, P. Gomes, J. Cobb, and B. L. Davies, “The hands-on orthopaedic robot ”acrobot”: Early clinical trials of total knee replacement surgery,” IEEE Transactions on Robotics and Automation, vol. 19, no. 5, pp. 902– 911, 2003.
 D. Prattichizzo, C. Pacchierotti, and G. Rosati, “Cutaneous force feedback as a sensory subtraction technique in haptics,” IEEE Transactions on Haptics, vol. 5, no. 4, pp. 289–300, 2012.
 S. E. Salcudean, S. Ku, and G. Bell, “Performance measurement in scaled teleoperation for microsurgery,” in Proc. First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery, pp. 789–798, 1997.
 P. Chatelain, A. Krupa, N. Navab. 3D ultrasound-guided robotic steering of a flexible needle via visual servoing. In IEEE Int. Conf. on Robotics and Automation, ICRA’15, pp. 2250-2255, Seattle, WA, May 2015.
 J. Chevrie, A. Krupa, M. Babel. Needle steering fusing direct base manipulation and tip-based control. In IEEE Int. Conf. on Robotics and Automation, ICRA’16, pp. 4450-4455, Stockholm, Sweden, May 2016.
 J. Chevrie, A. Krupa, M. Babel. Online prediction of needle shape deformation in moving soft tissues from visual feedback. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’16, pp. 2375-2380, Daejeon, South Korea, October 2016.
The candidate must have an excellent track of record and a Master Degree (or equivalent) in Robotics, Computer Science, or Biomedical Engineering. It is also imperative to master programming in C and/or C++. In addition, the candidate should be comfortable with as much following items as possible:
- Robot kinematics;
- Teleoperation control techniques;
- Multimodal interaction (e.g. visual, haptics, audio);
- Evaluation methods and controlled users studies.
The candidate must also have good communication skills and a good English level (oral and written).