Ryan Kim

In a dimly lighted chamber festooned with wires and hidden in one of California?s largest data centres, Tim Pozar is changing the shape of the Internet. He is using what Internet engineers refer to as a ?meet-me room?. The room itself is enclosed in a building full of computers and routers. What Pozar does there is to informally wire together the networks of different businesses that want to freely share their Internet traffic.

The practice is known as peering, and it goes back to the earliest days of the Internet, when organisations would directly connect their networks instead of paying yet another company to route data traffic. Originally, the companies that owned the backbone of the Internet shared traffic. In recent years, however, the practice has increased to the point where some researchers who study the way global networks are put together believe that peering is changing the fundamental shape of the Internet, with serious consequences for its stability and security. Others see the vast increase in traffic staying within a structure that has remained essentially the same.

What is clear is that today a significant portion of Internet traffic does not flow through the backbone networks of giant Internet companies like AT&T and Level 3. Instead, it has begun to cascade in torrents of data on the edges of the network, as if a river in flood were carving new channels.

Some of this traffic coursing through new channels passes through public peering points like Pozar?s. And some flows through so-called dark networks, private channels created to move information more cheaply and efficiently within a business or any kind of organisation. For instance, Google has privately built such a network so that video and search data need not pass through so many points to get to customers.

Everyone agrees that the shape of the network is changing rapidly, driven by a variety of factors, including content delivery networks that have pushed both data and applications to the edge of the network; the growing popularity of smartphones leading to the emergence of the wireless Internet; and the explosion of streaming video as the Internet?s predominant data type.

?When we started releasing data publicly, we measured it in petabytes of traffic,? said Doug Webster, a Cisco Systems market executive who is responsible for an annual report by the firm that charts changes in the Internet. ?Then a couple of years ago, we had to start measuring them in zettabytes, and now we?re measuring them in what we call yottabytes.? One petabyte is equivalent to 1 million gigabytes. A zettabyte is a million petabytes. And a yottabyte is a thousand zettabytes. The company estimates that video will account for 90% of all Internet traffic by 2013.

The staggering growth of video is figuring prominently in political and business debates like the one over the principle of network neutrality.

A study presented last year by Arbor Networks suggesting that traffic flows were moving away from the core of the network touched off a spirited controversy. The study was based on an analysis of two years of Internet traffic data collected by 110 large and geographically diverse cable operators, international transit backbones, regional networks and content providers.

Arbor?s Internet Observatory Report concluded that today the majority of Internet traffic by volume flows directly between large content providers like Google and consumer networks like Comcast. It also described what it referred to as the rise of so-called hyper giants?monstrous portals that have become the focal point for much of the network?s traffic: ?Out of the 40,000 routed end sites in the Internet, 30 large companies??hyper giants? like Limelight, Facebook, Google, Microsoft and YouTube?now generate and consume a disproportionate 30% of all Internet traffic,? the researchers noted.

The changes are not happening just because of the growth of the hyper giants. At the San Francisco data centre 365 Main, Pozar?s SFMIX peering location, or fabric, as it is called, now connects just 13 networks and content providers. But elsewhere in the world, huge peering fabrics are beginning to emerge. As a result, the ?edge? of the Internet is thickening, and that may be adding resilience to the network.

?The rise of these highly connected data centres around the world is changing our model of the Internet,? said Jon M Kleinberg, a computer scientist and network theorist at Cornell University. However, he added that the rise of giant distributed data centres built by Google, Amazon, Microsoft, IBM and others as part of the development of cloud computing services is increasing the part of the network that constitutes a so-called dark Internet, making it harder for researchers to build a complete model.

All of these changes have sparked a debate about the big picture. What does the Internet look like now? And is it stronger or weaker in terms of its resistance to failure because of random problems or actual attack? Researchers have come up with a dizzying array of models to explain the consequences of the changing shape of the Internet. Some describe the interconnections of the underlying physical wires. Others analyse patterns of data flow. And still others look at abstract connections like Web page links that Google and other search engine companies analyse as part of the search process. Such models are of great interest to social scientists, who can watch how people connect with each other, and entrepreneurs, who can find new ways to profit from the Internet. They are also of increasing interest to government and law enforcement organisations trying to secure the Net and use it as a surveillance tool.