Tele-robotic pinching for intra-operative palpation

Transcripción

06/04/2011
TTelerobotic pinching for
l b ti i hi f
intraoperative palpation
Departamento de Inge
eniería de Sistemas y Automáticaa
(1‐10‐2010 – 31‐3‐2011)
Jesús Manuel Gómez de Gabriel
ISRG
School of Systems Engineering
University of Reading
Contents
• Short CV & Previous works
• Introduction
• Haptic feedback in surgical robotics
– Touch feedback
– Force feedback
•
•
•
•
•
•
•
•
Goal
Master System
Interaction with virtual environments
Virtual prototyping
Slave System
Control System
Experiments
Conclusions
1
06/04/2011
Short CV
• Jesús M. Gómez de Gabriel, is an Associate Professor at the Department of Systems Engineering and Automation, University of Málaga (Spain)
University of Málaga (Spain)
• Engineering and Ph.D. degrees in Computer Science from the UMA in 1990 and 1999, respectively.
• Has led a project on telerobotic surgery and participated in different medical robotics projects.
• Current research interests include mechatronics education, medical robotics, and indoor mobile robotics.
• Currently in a Sabbatical Year in the SSE
y
(UoR) during course 2010‐11.
Engineering School at UMA
Previous works on medical robotics
• ERM cameraman robot.
– Now transfered to SENER
– Clinical trials
– Adaptive control and surgeon/patient interaction/compliance • Telerobotic system
y
for
minimally invasive
surgery.
– DPI 2003‐08263 project
– Interfaces for delayed
teleoperation
2
06/04/2011
Introduction
• Minimally Invasive Surgery techniques (MIS)
– Small incisions in the patient using specialized instruments,
– Reduction on clinical complications and hospitalization time
– Abdominal MIS Techniques known as laparoscopic.
– Constraints and motion inversion • Robotics & MIS: – Thanks to the special constraints (instrument and motion restriction), today, commercial robotic systems for surgery can be found.
– Can enhance the performance of these tools by means of scaling, filtering and other aids.
Introduction
• Lack of tactile feedback in MIS.
– The surgeon loses tactile feedback, but can feel forces at the instrument handle
– Standard MIS training provides the surgeons the skill to do manual tasks with video feedback only.
– Force or tactile feedback can be necessary/important for many manipulation tasks,
– Growing interest of the surgeons in recovering the sense of touch, for grasping and touching the patient tissues.
• Haptic feedback in MIS robotics
– With robots we can loose force feedback too
– No tactile feedback is commonly used
3
06/04/2011
Haptic feedback in surgical robotics
• Haptics generally describes touch feedback (Okamura, 2009), which may include kinesthetic (force) and cutaneous (tactile) feedback.
feedback
• Graphical display can be used as a sensorial substitution system
• Special master system: Impedance type device is the most popular because of the lower costs (there are no force sensors) and high responsiveness to human inputs.
Tactile feedback in surgical robotics
• Useful for exploratory tasks such as palpation, in which distributed pressure or deformation information can be
distributed pressure or deformation information can be used to identify hard lumps in surrounding soft tissue. • Also experiments show how tactile feedback induces reduced grasping force in robot‐assisted surgery (King, 2009)
• It remains difficult to design both tactile sensors and displays that are compatible with the surgical
displays that are compatible with the surgical environment.
4
06/04/2011
Force feedback in surgical robotics
• Easy to implement and process.
• Haptics and special manipulators for needle insertion p
p
p
control (Zarrad, 2007) considering the changes in environment stiffness.
• Force Feedback in dentistry Learning (San Diego, 2008) • Teleoperated palpation for calcified vessel detection (Gwilliam, 2009).
• Force controlled telerobotic grasper (Rosen, 1999) can identify automatically different kinds of animal tissues
identify automatically different kinds of animal tissues.
• Special wheeled haptic probe for the identification of soft tissues abnormalities (Liu, 2009)
Soft tissue mechanics
• Experimental measures (Rosen, 1999)
• Dual Maxwell Model with nonlinear functions (Liu, 2009)
5
06/04/2011
Surgeon Grasping Mechanics
• Thirty‐one surgeons of varying skill were recorded performing three different surgical tasks (Brown, 2004)
– Force
• The mean force applied to the tool handles during tissue grasps was 8.52 N ± 2.77 N;
• maximum force was 68.17 N.
– Frequency
• Ninety‐five percent of the handle angle frequency content was below 1.98 Hz ± 0.98 Hz. • Mean percentage of Fg that lies below 5 Hz, during tissue grasps: 99.35% ±
99 35% 1.35% 1 35%
– Time
• Average grasp time was 2.29 s ± 1.65 s, and 95% of all grasps were held for 8.86 s ± 7.06 s or less.
• The average maximum grasp time was 13.37 s ± 11.42 s.
Goal of this work
• Goal: Design (Build) the robotic instruments for the task of intraoperative lump detection in laparoscopic surgery. • Moveable and overlapped organs (Two or more fingers).
• No grasping needed.
6
06/04/2011
First Experiments
• Some preliminary experiments have been done in order to check if the proposed task can be performed
proposed task can be performed without tactile feedback.
• They showed the feasibility of a force feedback control system by means of a set of haptic devices without touch sensors.
• Rigid fixed inclusions (bones) convert grasping force into displacement force.
Master System
• Master system already developed at the THRILL Lab:
– 2 to 3 Fingers (3 active DOF each)
– 2 or 3D immersive display
– C++ environment with Haptic, Math and graphics libraries
7
06/04/2011
Interaction with virtual environments (I)
• Simulators available used for surgery and medical applications (Halvorsen, 2005).
• Laparoscopic simulators used for training
L
i i l
df
i i on the use of laparoscopic instruments rarely use force feedback.
• Difficult environment modelling.
– Mechanical properties of the tissues
– Flexible tissues, dynamic (cut, suture,…),
• Preoperative planning. Models with data from p
g g( ,
,
the real patient. Medical imaging (CT, MRI, etc.)
• 3D/2D graphical simulation
• Force feedback commonly used in orthopaedic surgery training.
Interaction with virtual environments (II)
• Same question: Is it possible to successfully achieve the proposed task without tactile
achieve the proposed task without tactile feedback ?
– Simulation for the validation of the idea. Not for training
– Kinesthetic feedback only (force feedback) and visual
• Experiments (4 Videos):
– Moving/fixed and
– visible/hidden inclusions 8
06/04/2011
Interaction with virtual environments (III)
Hidden
Fixed
Mobile
Visible
Virtual prototyping
• It is necessary to reduce the number of joints and sensors to a minimum.
• The virtual prototyping of the different instruments allows testing The virtual prototyping of the different instruments allows testing
the physical and sensorial constraints of the proposed instruments.
Master
Slave
Video display
Video camera
Virtual instrument
Position
Multi finger h i d i
haptic device
Force
Intrument
Position
Mechanical constraints
Constraints
Forces
Feedback
Forces
+
Meassured
Forces
Sensorial constraints
Interaction
Forces
Fingers with position control
Force sensors
• Different virtual prototypes for the robotic instrument have been tested, and full information about the task execution (positions, and measured forces) has been recorded for analysis.
9
06/04/2011
Virtual prototypes (I)
• Virtual prototypes are composed of master position and slave force sensing constraints (Virtual fixtures).
• Positions are constrained before sending to the slave.
P ii
i db f
di
h l
• Force readings are filtered before feeding back to the haptics.
• The forces imposed by the virtual constraints are implemented using virtual high stiffness springs.
• Instrument dynamics has not been considered.
FY0
FX0
Y0
• General prototype:
G
l
t t
FY1
– Unconstrained. FX1
Y1
• 3 DOF each finger
• 3 axis force sensing
• Figure shows horizontal XY plane
X0
X1
Virtual Prototypes (II)
– Parallel horizontal grasper (3 Dofs)
•
•
•
•
SSame Y coordinate.
Y
di t
Horizontal plane (Z = 0).
Grasping only force sensing
Figure shows horizontal XY plane
Y0
Master Position
Slave Position
K
F0
F1
(Y0+Y1)/2
K
Y1
10
06/04/2011
Virtual Prototypes (III)
– Parallel horizontal symmetric grasper (2 Dofs)
•
•
•
•
•
SSame Y coordinate.
Y
di t
Horizontal plane (Z = 0)
Symmetry along X axis
X axis only force sensing
Figure shows horizontal XY plane
Master Position
Y0
F1
Slave Position
(Y0+Y1)/2
F0
Y1
‐X1
X0
X=0
‐X0
X1
Slave system
Version 1.0
Two slave fingers design 1.0 with external force sensors
3 DOFs per finger.
L l
Local proportional position control servos 1 KHz update rate.
i
l
ii
l
1 KH
d
10 bits position resolution (≈ 0.3 degrees)
Low cost but high unmeasured compliance
•
•
•
•
•
11
06/04/2011
Slave System
Version 1.1
Camera
Camera boom
Monitor
Mirror
Master
Haptic
Supporting
frame
Base
Hanging
Fingers
Tissues
Experimental Bilateral Teleoperation Setup
Slave System
Force sensing
•
•
•
•
•
Thanks to previous development from TRHILL Lab on its force sensor.
750g Micro‐load cells
g
On board signal conditioning and data acquisition 10 bits 1KHz
Rearranged for uncoupled serial configuration.
Last finger link body.
FZ
Fingertip
FY
FX
3
DAQ
2
1
Sensor base
12
06/04/2011
Slave System
Version 1.2
• Same components as 1.0
• Shortened axis distances by changing the servo links.
– Less compliance
– Better cartesian space resolution.
• Smaller
Smaller workspace due to workspace due to
shorter links and angle limits.
Bilateral Control System
• Classical implementation of a bilateral Force‐Position control system
Xm
Surgeon
Fh
‐
Haptic Master manipulator
(Impedance)
Force Sensor
Patient
Xs
‐
Position control
Kf
Fs
Slave device
(Admitance)
• Master
Master workstation is based on a set of haptic devices with workstation is based on a set of haptic devices with
impedance control
• Slave setup as a impedance control (position control and force sensors).
• A video feedback channel has also been added to the system.
• Better than standard position‐position control scheme in non‐
contact motion.
• Problem of transparency and stability.
13
06/04/2011
Bilateral control system (II)
• Bilateral control system model
• Force feedback gain depends on the overall dynamics. • In medical robotics teleoperation experiments, the changes in the environment stiffness can be abrupt and large in
the environment stiffness can be abrupt and large in magnitude (e.g. touching a muscle, an organ, a bone, another instrument or a rib for cardiothoracic surgery)
• Stiffness estimation can be difficult due to organ motion (breathing or displacements)
Bilateral control system (II)
• Possible Improvements to be implemented: Adaptive Position‐
Position control system (Poignet, 2009; Zarrad, 2007)
– Environment Stiffness estimator (Needs force sensing at the slave)
Environment Stiffness estimator (Needs force sensing at the slave)
– Active observer and Kalman filter
Dynamic gain and p p p
haptic palpation ???
14
06/04/2011
Virtual prototype experiments
• Task goal/scoring:
• To find a rigid object (crystal T fi d i id bj t (
t l
ball 1.6 mm dia.) inside a three‐sections foam “organ”.
• Guess the approximate size/shape of the object.
• Reposition the organ for better handling
better handling.
Virtual prototype experiments (I)
• Free motion
– 6 Dofs
– Full force feedback
Full force feedback
• Results:
– Easy to find inclusion
– Easy to guess the size
– Fair organ manipulation
http://www.youtube.com/watch?v=ETZayk2op3I
15
06/04/2011
Virtual prototype experiments (II)
• Parallel grasper
– 3 Dofs
– Grasping force
Grasping force
feedback
• Results:
– Fair to find inclusion
F i
fi d i l i
– No shape guessing
– Limited organ manipulation
http://www.youtube.com/watch?v=1Q0c1juUY3E
Virtual prototype experiments (III)
• Symmetrical parallel
grasper
– 2
2 Dofs
Dofs
– Grasping force
feedback
• Results:
– Hard to find inclusion
Hard to find inclusion
– No guessing of size/shape
– Bad organ manipulation
http://www.youtube.com/watch?v=v‐wBtc9hCYA
16
06/04/2011
Data analysis
• To be done:
– Second force sensor reading and filtering
Second force sensor reading and filtering
– Tissue dentification assistance?
Left Finger force
Finger distance
Open
Open
closed
Slave system
Version 2.0
Lower compliance
4x spatial resolution
Higher torque
Different kinematic
Same control system and protocol
• More expensive than 1.x
• To be tested
•
•
•
•
•
17
06/04/2011
Conclusions
• This is an ongoing work. No final conclusions yet.
• The foam phantom used is much more elastic than real animal tissues (sausage) Needs improvement
tissues (sausage). Needs improvement.
• Processing of the sensorial information for some way of dynamic gain and tissue identification has to be implemented.
• Although it may not be necessary, we miss a touch sensor/display system.
• This system allows the study of the performance of different instruments for different telemanipulation tasks without a physical implementation.
• Also, the analysis of the obtained information during the trials Al
h
l i f h b i di f
i d i
h i l
provides a better understanding of the use of the instruments, so further optimizations of the prototype can be made.
References
•
•
•
•
•
•
•
•
•
•
Bogado Torres, J.M., 2007, “Control bilateral de robots treleoperados por convergencia de estados”, PhD Thesis, Universidad Politécnica de Madrid.
Brown, J.D. et al, “Quantifying Surgeon Grasping Mechanics in Laparoscopy Using the Blue DRAGON y
,
gy
Medicine Meets Virtual Reality, Newport Beach, y,
p
,
System”, Studies in Health Technology and Informatics ‐
CA, January 2004.
J.C. Gwilliam, M. Mahvash, B. Vagvolgyi, A. Vacharat, D D. Yuh, and A. M. Okamura, “Effects of Haptic and Graphical Force Feedback on Teleoperated Palpation”, 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, May 12‐17, 2009
Halvorsen, F.H. et al. 2005, “Simulators in Surgery”, Minimally Invasive Therapy 14:4‐5; pp. 214‐223.
King, C.‐H.; Culjat, M.O.; Franco, M.L.; Lewis, C.E.; Dutson, E.P.; Grundfest, W.S.; Bisley, J.W.; , "Tactile
Feedback Induces Reduced Grasping Force in Robot‐Assisted Surgery," Haptics, IEEE Transactions on , vol.2, no.2, pp.103‐110, April‐June 2009
Hongbin Liu Elhage, O. Dasgupta, P. Challacombe, B. Murphy, D. Seneviratne, L. Althoefer, K., “A haptic probe for soft tissue abnormality identification during minimally invasive surgery”, Reconfigurable Mechanisms and Robots, 2009. ReMAR 2009. ASME/IFToMM International Conference on, 22‐24 June 2009
Okamura, A. M., 2009, “Haptic Feedback in Robot‐Assited Minimally Invasive Surgery”, Curr. Opin. Urol, January 2009.
Poignet, P, et al. 2009. “Some control related issues in mini‐robotics for endoluminal surgery”, 31st
Annual Intl. Conf. Of the IEEE EMB, pp. 6850‐6855
Rosen, J. Hannaford, B. MacFarlane, M.P. Sinanan, M.N., “Force controlled and teleoperated endoscopic grasper for minimally invasive surgery‐experimental performance evaluation”, Biomedical Engineering, IEEE Transactions on, 46:10; 1999
Zarrad, W. et al 2007, “Stability and Transparency Analysis of a Haptic Feedback Controller for Medical Applications”, 46th IEEE Conference on Decision and Control New Orleans.
18
06/04/2011
Thanks and acknowledgements
• Special Thanks to Prof. William Harwin, Víctor Becerra, Rui Loureiro and the University of Reading
Becerra, Rui
and the University of Reading
• Thrill Lab: Alastair Barrow, Brian Tse and Balazs Janko
• Ministerio de Educación y Ciencia of Spain
• SSE’s Workshop staff
• University of Málaga
Contact
Jesús M. Gomez de Gabriel
Dt I
Dto. Ingeniería de Sistemas y Automática
i í d Si t
A t áti
Universidad de Málaga
E‐mail: [email protected]
Web: www.isa.uma.es/c10/degabriel
/ / g
Blog: www.hombremecatronico.es
YouTube: www.youtube.com/roboticario
19

Tele-robotic pinching for intra-operative palpation

Transcripción

Documentos relacionados

5 awards for research performed by undergraduate students of

Sin título de diapositiva - Departamento de Matemáticas de la UAM

free of charge interpreter service