Measuring and Understanding High-Speed Networks

***Refreshments at 11:50am***

Abstract:

Systems and networks have grown increasingly complex over the past four decades. The four ARPANET hosts of 1969 have been transformed into a global Internet of staggering size, connectivity, and usage. This evolution has led to the Internet displaying emergent behavior, which necessitate empirical measurement, rather than intuition, to understand. The networking literature has a rich tradition of these measurements, focusing upon either the structure of the Internet or its traffic characteristics. However, existing techniques can no longer obtain instantaneous traffic measurements with accuracy and reproducibility at high-speeds --- an inadvertent casualty of the architectural evolution of networks and systems over the past decade.

To address this, we design and implement BiFocals, a novel class of high-speed and high-precision network instrumentation. We apply our tool to perform the first exact packet-timing measurements of a network path, capturing 10 Gigabit Ethernet packets in flight on optical fiber over the wide-area. Through principled design, we improve timing precision by two to six orders of magnitude over existing techniques. Our observations contest some common assumptions about the behavior of wide-area networks and the relationship between their input and output traffic flows. Further, we identify and characterize emergent packet chains as a mechanism to explain previously observed anomalous packet loss on receiver endpoints of such networks.

I conclude by discussing our current efforts toward the next-generation of BiFocals instrumentation, as well as progress in achieving massive improvements in cost, size, and convenience that presage more wide-spread adoption of this technique. Finally, given such an instrument, I outline some open questions in networking research that should thus prove tractable.