Headlines at Hopkins: news releases from across
university Headlines
News by Topic: news releases organized by
subject News by Topic
News by School: news releases organized by the 
university's 9 schools & divisions News by School
Events Open to the Public (campus-wide) Events Open
to the Public
Blue Jay Sports: Hopkins Athletic Center Blue Jay Sports
Search News Site Search the Site

Contacting the News Staff: directory of
press officers Contacting
News Staff
Receive News Via Email (listservs) Receive News
Via Email
RSS News Feeds RSS News Feeds
Resources for Journalists Resources for Journalists

Virtually Live@Hopkins: audio and video news Virtually
Hopkins in the News: news clips about Hopkins Hopkins in
the News

Faculty Experts: searchable resource organized by 
topic Faculty Experts
Faculty and Administrator Photos Faculty and
Faculty with Homepages Faculty with Homepages

JHUNIVERSE Homepage JHUniverse Homepage
Headlines at Hopkins
News Release

Office of News and Information
Johns Hopkins University
901 South Bond Street, Suite 540
Baltimore, Maryland 21231
Phone: 443-287-9960 | Fax: 443-287-9920

December 5, 2006
CONTACT: Lisa De Nike
(443) 287-9960

We're No. 1
JHU/Illinois Team Wins Bandwidth Bowl

A team from The Johns Hopkins University and the University of Illinois at Chicago's National Center for Data Mining recently won the seventh annual Bandwidth Challenge, moving the equivalent of 2,000 CDs full of data more than 1,000 miles in less than 20 minutes.

The team transported 1.3 terabytes of data from Chicago to the Tampa, Fla., site of SC 06, the international conference for high performance computing, networking and storage. They achieved a sustained data transfer rate of 8 gigabytes per second over a 10 gigabyte per second link. The team's peak rate was 9.18 gigabytes per second.

"Two or three years ago, the only way we were able to distribute such large data sets would be to ship three whole computers around the world by Federal Express," said Alexander Szalay, Alumni Centennial Professor in the Henry A. Rowland Department of Physics and Astronomy at Johns Hopkins.

Teams from California Institute of Technology and University of Tokyo came in second and third.

Szalay, a theoretical astrophysicist, also is an expert in the manipulation of the huge banks of data that are produced by today's research in fields as diverse as astronomy and environmental science. He said events like the Nov. 16 competition are important for more than just bragging rights; they have implications for how researchers in many fields collaborate on huge projects from locations all around the globe.

"Moving data at such speeds opens up whole new ways of approaching scientific problems," he said. Moving the same amount of data using an 800 kilobits per second DSL connection would have taken almost six months, he said. "Using the more typical upload speed of 384 kilobits per second, it would take almost a year," he said.

The data set involved in the competition was from the Sloan Digital Sky Survey, an astronomy project in which Szalay is a key player. It is collecting more than 40 terabytes of raw data, consisting of five color digital images of the entire northern sky. When compressed, the date used in the Bandwidth Challenge consisted of 60 files of about 23 gigabytes, totaling 1.3 terabytes.

The technology that made the transfer possible was an open source, high performance network transport protocol called UDT that the National Center for Data Mining developed several years ago. Since then it has been downloaded more than 8,000 times, using an open source, peer-to-peer storage system called Sector (also developed by NCDM). Sector was built using UDT, and was designed to distribute large e-science data sets. The team used a 10 gigabyte per second network called PacketNet, which was provided by National Lambda Rail.

In the past, UDT and other technologies could move data at high speeds, but faced challenges when utilized to transport data from disk to disk over long distances, said Szalay. He said that Sector allowed the team to transport data from disk to disk quite easily.

"During the challenge, we copied the data over in 20 minutes from disks in Chicago to disks in Florida, ready to be used immediately," he said. "Previously, we would have had to ship disks around, which is a cumbersome process involving a lot of work on either end. Though other teams have transmitted data from memory to memory at a rate faster than ours, our disk-to-disk transmission does a better job of resembling real applications, which is why it is so impressive."

The Johns Hopkins team — Szalay, research scientist Ani Thakar, computer systems administrator Jan vandenBerg and systems administrator Alainna Wonders — and the researchers at University of Illinois would eventually like to use such network speeds as part of their everyday research.

Michael McCarty, chief network officer for Johns Hopkins, said he believes that such collaboration could lead to benefits not only for Johns Hopkins researchers but for the community at large.

"We have been excited to have an opportunity to work with Alex and other research scientists across Hopkins whose research creates new challenges for us as well. Today, we have partnerships with the state of Maryland, several Maryland universities and the Internet 2 community that we have been able to leverage to help make efforts such as this possible," McCarty said.

Related Web sites

arrow Sloan Digital Sky Survey data distribution using Sector and UDT
arrow Alex Szalay

Johns Hopkins University news releases can be found on the World Wide Web at http://www.jhu.edu/news_info/news/
   Information on automatic e-mail delivery of science and medical news releases is available at the same address.

arrow Go to Headlines@HopkinsHome Page