Office of News and Information
Johns Hopkins University
901 South Bond Street, Suite 540
Baltimore, Maryland 21231
Phone: 443-287-9960 | Fax: 443-287-9920
December 8, 2006
FOR IMMEDIATE RELEASE
CONTACT: Phil Sneiderman
Computer Scientists Unravel 'Language of
Data Collected From Robotic Medical Tools
Could Improve Operating Room Skills
Borrowing ideas from speech recognition research,
Johns Hopkins computer scientists are building
mathematical models to represent the safest and most
effective ways to perform surgery, including tasks such as
suturing, dissecting and joining tissue.
The team's long-term goal is to develop an objective
way of evaluating a surgeon's work and to help doctors
improve their operating room skills. Ultimately, the
research also could enable robotic surgical tools to
perform with greater precision.
The project, supported by a three-year National
Science Foundation grant, has yielded promising early
results in modeling suturing work. The researchers
performed the suturing with the help of a robotic surgical
device, which recorded the movements and made them
available for computer analysis.
"Surgery is a skilled activity, and it has a
structure that can be taught and acquired," said Gregory
D. Hager, a professor of
computer science in the university's Whiting School of
Engineering and principal investigator on the project. "We
can think of that structure as the language of surgery.'
To develop mathematical models for this language, we're
borrowing techniques from speech recognition technology
and applying them to motion recognition and skills
Complicated surgical tasks, Hager said, unfold in a
series of steps that resemble the way that words,
sentences and paragraphs are used to convey language. "In
speech recognition research, we break these down to their
most basic sounds, called phonemes," he said. "Following
that example, our team wants to break surgical procedures
down to simple gestures that can be represented
mathematically by computer software."
'Language of surgery' researchers collect data from
this da Vinci
robotic surgical system operated by David Yuh, a cardiac surgeon
Johns Hopkins Hospital. Standing are team members Gregory Hager,
Shafran, Henry Lin and Sanjeev Khudanpur.
Photo by Will Kirk
With that information in hand, the computer
scientists hope to be able to recognize when a surgical
task is being performed well and also to identify which
movements can lead to operating room problems. Just as a
speech recognition program might call attention to poor
pronunciation or improper syntax, the system being
developed by Hager's team might identify surgical
movements that are imprecise or too time-consuming.
But to get to that point, computers first must become
fluent in the "language" of surgery. This will require
computers to absorb data concerning the best ways to
complete surgical tasks. As a first step, the researchers
have begun collecting data recorded by Intuitive
Surgical's da Vinci Surgical Systems. These systems allow
a surgeon, seated at a computer workstation, to guide
robotic tools to perform minimally invasive procedures
involving the heart, the prostate and other organs.
Although only a tiny fraction of hospital operations
involve the da Vinci, the device's value to Hager's team
is that all of the robot's surgical movements can be
digitally recorded and processed.
In a paper presented at the Medical Image Computing
and Computer-Assisted Intervention Conference in October
2005, Hager's team announced that it had developed a way
to use data from the da Vinci to mathematically model
surgical tasks such as suturing, a key first step in
deciphering the language of surgery. The lead author,
Johns Hopkins graduate student Henry C. Lin, received the
conference award for best student paper.
"Now, we're acquiring enough data to go from words'
to sentences,'" said Hager, who is deputy director of the
Foundation Engineering Research Center for
Computer-Integrated Surgical Systems and Technology,
based at Johns Hopkins. "One of our goals for the next few
years is to develop a large vocabulary that we can use to
represent the motions in surgical tasks."
When a surgeon operates the controls of a da Vinci
robotic system, the
device records these hand movements. Computer scientists are
this data in their effort to understand the 'language of
Photo by Will Kirk
The team also hopes to incorporate video data from
the da Vinci and possibly from minimally invasive
procedures performed directly by surgeons. In such
operations, surgeons insert instruments and a tiny camera
into small incisions to complete a medical procedure. The
video data from the camera could contribute data to the
team's efforts to identify effective surgical methods.
Hager's Johns Hopkins collaborators include David D.
Yuh, a cardiac surgeon from the School of Medicine. "It is
fascinating to break down the surgical skills we take for
granted into their fundamental components," Yuh said.
"Hopefully, a better understanding of how we learn to
operate will help more efficiently train future surgeons.
With the significantly reduced number of hours surgical
residents are permitted to be in the hospital, surgical
training programs need to streamline their training
methods now more than ever. This research work represents
a strong effort toward this."
Hager's other collaborators include Sanjeev
Khudanpur, a Johns Hopkins assistant professor of
electrical and computer engineering, and Izhak Shafran,
who was a postdoctoral fellow affiliated with the
university's Center for Language and Speech Processing and
who is now an assistant professor at the Oregon Graduate
Cardiac surgeon David Yuh controls the da Vinci
robotic surgical system
as computer scientists Izhak Shafran and Gregory Hager
Photo by Will Kirk
Hager cautioned that the project is not intended to
produce a "Big Brother" system that would critique a
surgeon's every move. "We're trying to find ways to help
them become better at what they do," he said. "It's not a
new idea. In sports and dance, people are studying the
mechanics of movement to see what produces the best
possible performance. By understanding the underlying
structures, we can become better at what we do. I think
surgery's no different."
A video showing some of this research is online at
Digital photos of the researchers available; contact Phil
Gregory Hager's Lab Page
NSF Center for Computer-Integrated Surgical Systems and
Johns Hopkins Department of Computer Science
Johns Hopkins University news releases can be found on the
World Wide Web at
Information on automatic e-mail delivery
of science and medical news releases is available at the