PhD and PostDoc in Virtual Reality and Haptics

PhD and PostDoc Proposal

Shared control of flexible needles for robot-assisted biopsies

Team: Rainbow, IRISA/Inria Rennes

Advisors: Alexandre KrupaMarie BabelClaudio Pacchierotti

Main contact: Alexandre Krupa, – Marie Babel,

Project site:

Application: Applications consist of a cover letter describing how your research experience is relevant to the position and a resume sent to and marie.babel@irisa.frPlease also send your list of marks (even preliminary) of your Master 2 or engineer formation.


Needle insertion in soft-tissue is a minimally invasive surgical (MIS) procedure used for diagnostic
and therapeutic purposes, and it is one of the many surgical procedures that may greatly benefit from
the use of teleoperated robotic systems. Hence, researchers have been constantly trying to develop
new techniques and systems able to improve its safety and accuracy. Flexible needles and haptic
feedback are two of these technological advancements. Flexible needles provide the clinician with
enhanced steering capabilities, and haptic feedback enables the clinician to receive information about
the forces exerted by the needle on the soft tissue being penetrated.

In our previous works, we studied different approaches to automatically steer a flexible needle
actuated by a robotic arm, in order to accurately position its tip on a desired target by visual servoing
and 3D ultrasound imaging. However, for reasons of safety and responsibility, it would be beneficial
to provide clinicians with direct control of the motion of the medical instrument.

We propose to study innovative teleoperation systems for steering flexible needles, exploiting
grounded and ungrounded haptic stimuli for our vision-based needle insertion system, with the final
objective of maximizing the information provided, the clinician comfort, and the medical procedure’s
safety and effectiveness.

The project will proceed by developing four main key aspects:

  • Perception of multiple haptic stimuli. At first, we will study the effectiveness of combining multiple
    haptic stimuli, focusing on force, vibrations, normal indentation, and skin stretch. We will focus on
    stimuli being able to provide multi-directional information, applied to different parts of the body, such
    as the hand, wrist, and forearm.
  • Visual servoing. We propose to develop new ways of assistance solutions where the clinician will
    keep total or partial manual control of the needle positioning. This could be achieved by sharing
    different degrees of liberty of the needle between the robot and the clinician through
  • Shared control with haptic guidance. To help clinicians steer the needle toward the target, we will
    study how to provide effective guiding haptic stimuli. Haptic feedback will be used to enforce active
    constrains aimed at safely positioning the needle without damaging the tissues and also to provide
    guiding information extracted from the current image.
  • Safety and stability. We will work to improve existing stability control approaches to take into
    account for the additional tactile stimuli, focusing on time-domain energy-based techniques, with the
    objective of maximizing transparency while guaranteeing the overall safety of the system.

Robotic platform for needle steering: 6-dof robot holding an 3D ultrasound probe, 6-dof robot actuating a flexible needle, Haption Virtuose 6D haptic device.

We are looking for excellent, highly-motivated students interested in Mixed Reality and haptics, with a computer science background and previous experience in computer programming (C++). Experience in using VR/AR tools and systems (e.g., Unity 3D, ARToolkit, Oculus Rift, Hololens) is considered a plus.

The researcher will be required to develop novel rendering algorithms in VR/AR, design novel interaction techniques, and conduct of several user studies and experiments with human participants. We also plan exchanges and meetings with academic and industrial partners in different European countries, so the student should be ready to travel.

Keywords: Medical robotics, shared control, visual servoing, haptic feedback, ultrasound imaging, needle insertion


[1] N. Abolhassani, R. V. Patel, and M. Moallem, “Needle insertion into soft tissue: A survey,” Medical engineering & physics, vol. 29, no. 4, pp. 413–431, 2007.

[2] C. Papalazarou, P. M. Rongen, et al., “Surgical needle reconstruction using small-angle multi-view x-ray,” in Proc. IEEE International Conference on Image Processing (ICIP), pp. 4193–4196, 2010.

[3] V. Duindam, R. Alterovitz, S. Sastry, and K. Goldberg, “Screwbased motion planning for bevel-tip flexible needles in 3d environments with obstacles,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 2483–2488, 2008.

[4] K. Hauser, R. Alterovitz, N. Chentanez, A. M. Okamura, and K. Goldberg, “Feedback control for steering needles through 3d deformable tissue using helical paths,” in Proc. Robotics: Science and Systems (RSS), vol. 37, 2009.

[5] M. Abayazid, R. J. Roesthuis, R. Reilink, and S. Misra, “Integrating deflection models and image feedback for real-time flexible needle steering,” IEEE Transactions on Robotics, vol. 29, no. 2, pp. 542–553, 2013.

[6] M. Abayazid, M. Kemp, and S. Misra, “3D flexible needle steering in soft-tissue phantoms using Fiber Bragg Grating sensors,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 5823–5829, 2013.

[7] J. Troccaz and Y. Delnondedieu, “Semi-active guiding systems in surgery. a two-dof prototype of the passive arm with dynamic constraints (padyc),” Mechatronics, vol. 6, no. 4, pp. 399 – 421, 1996. Mechatronics in Surgery.

[8] M. Jakopec, F. Rodriguez y Baena, S. Harris, P. Gomes, J. Cobb, and B. L. Davies, “The hands-on orthopaedic robot ”acrobot”: Early clinical trials of total knee replacement surgery,” IEEE Transactions on Robotics and Automation, vol. 19, no. 5, pp. 902– 911, 2003.

[9] D. Prattichizzo, C. Pacchierotti, and G. Rosati, “Cutaneous force feedback as a sensory subtraction technique in haptics,” IEEE Transactions on Haptics, vol. 5, no. 4, pp. 289–300, 2012.

[10] S. E. Salcudean, S. Ku, and G. Bell, “Performance measurement in scaled teleoperation for microsurgery,” in Proc. First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery, pp. 789–798, 1997.

[11] P. Chatelain, A. Krupa, N. Navab. 3D ultrasound-guided robotic steering of a flexible needle via visual servoing. In IEEE Int. Conf. on Robotics and Automation, ICRA’15, pp. 2250-2255, Seattle, WA, May 2015.

[12] J. Chevrie, A. Krupa, M. Babel. Needle steering fusing direct base manipulation and tip-based control. In IEEE Int. Conf. on Robotics and Automation, ICRA’16, pp. 4450-4455, Stockholm, Sweden, May 2016.

[13] J. Chevrie, A. Krupa, M. Babel. Online prediction of needle shape deformation in moving soft tissues from visual feedback. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’16, pp. 2375-2380, Daejeon, South Korea, October 2016.


The PhD position will be for 3 years, and will start in October 2018.

Place: IRISA-Inria Rennes, Campus de Beulieu, Rennes, France


Comments are closed.