By substituting mechanical instruments for human
fingers, robotic tools give surgeons a new way to perform
medical procedures with great precision in small spaces.
But as the surgeon directs these tools from a computer
console, an important component is lost: the sense of
Johns Hopkins researchers are trying to change that by
adding such sensations, known as haptic feedback, to
medical robotic systems. "Haptic" refers to the sense of
"The surgeons have asked for this kind of feedback,"
said Allison Okamura, an associate professor of
engineering, "so we're using our understanding of
haptic technology to try to give surgeons back the sense of
touch that they lose when they use robotic medical
Okamura is a leading researcher in human-machine
interaction, particularly involving mechanical devices that
convey touchlike sensations to a human operator. In recent
years, she has focused on medical applications as a
participant in the National Science Foundation Engineering
Research Center for Computer-Integrated Surgical Systems
and Technology, based at Johns Hopkins. With funding from
the National Institutes of Health, she has established a
collaboration with Intuitive Surgical, maker of the da
Vinci robotic system used in many hospitals for heart and
In the da Vinci system, a surgeon sits at a computer
console, looks through a three-dimensional video display of
the surgery site and moves finger controls that direct the
motion of robotic tools inside the patient. Currently, this
system does not send haptic feedback to the surgeon to
convey what the mechanical tool "feels" inside the body.
Okamura's team seeks to add these sensations to the da
Vinci and similar machines.
Through the arrangement with Intuitive Surgical,
Okamura's lab has acquired da Vinci hardware and software
that allow her to conduct experiments toward achieving that
goal. For example, the da Vinci's tools can be directed to
tie sutures, but if the operator causes the tools to pull
too hard, the thread can break. The Johns Hopkins
researchers want the human operator to be able to feel
resistance when too much force is applied.
"The sense of touch is important to surgeons," Okamura
said. "They like to feel what's happening when they're
working inside the body. They feel a 'pop' when a needle
pokes through tissue. They can feel for calcification.
Their sense of touch helps tell them where they are within
the body. In robotic procedures and other types of
minimally invasive surgery, surgeons insert long tools
between their hands and the patient. This approach may have
some medical benefits, but for the surgeon, there's a loss
of dexterity. It's like operating with chopsticks that have
grippers on the end."
To address this, Okamura's team is experimenting with
several techniques that could give some of those sensations
back to the surgeons. One option is to attach to the
robotic tools force sensors capable of conveying to the
human operator how much force the machine is applying
during surgery. Another idea is to create mathematical
computer models that represent the moves made by the
robotic tools, and then use this data to send haptic
feedback to the operator.
Both approaches have advantages and drawbacks. Force
sensors may be highly accurate, but they are expensive and
would have to be made of sterile, biocompatible materials
to be used in medical robots. Computer models could be less
expensive but might not respond quickly enough. "I'm
exploring both approaches to see which produces the best
results," Okamura said. "The most important thing is that
the haptic feedback sent to the human operator must feel
right because the fingers aren't easily fooled."
While this research continues, Okamura's team has
developed an interim system that instead sends "haptic"
information to the eyes. When a surgeon is using a robotic
tool to tie a suture, for example, a colored circle follows
the image of the tool in the visual display, indicating how
much force is being using. A red light may signal that too
much force is being applied, and the thread is likely to
break. Green and yellow lights may indicate that the right
amount of force is being used or that the tool is edging
toward excessive force.
Okamura's team has already published a journal article
describing an early version of this visual haptic feedback
project and is continuing to refine the system.
A video showing some of this research is online at