Where the cloud meets the ground

Font Size

Print

Feedback

Email

Discuss

Related Articles

ED attaches properties of Raju and associatesRajus shared spoils of fictitious profitsHP bullish on Indian servers mktAssam offers land to Infosys and Wipro

: It is almost as easy as plugging in a laser printer. Up to 2,500 servers—in essence, souped-up PCs—are crammed into a 40-foot shipping container. A truck places the container inside a bare steel-and-concrete building. Workers quickly connect it to the electric grid, the computer network and a water supply for cooling. The necessary software is downloaded automatically. Within four days all the servers are ready to dish up videos, send emails or crunch a firm’s customer data.

This is Microsoft’s new data centre in Northlake, a suburb of Chicago, one of the world’s most modern, biggest and most expensive, covering 5,00,000 square feet and costing $500 million. One day it will hold 4,00,000 servers. The entire first floor will be filled with 200 containers like this one. Michael Manos, the head of Microsoft’s data centres, is really excited about these containers. They solve many of the problems that tend to crop up when putting up huge data centres: how to package and transport servers cheaply, how to limit their appetite for energy and how to install them only when they are needed to avoid leaving expensive assets idle.

But containers are not the only innovation of which Manos is proud. Microsoft’s data centres in Chicago and across the world are equipped with software that tells him exactly how much power each application consumes and how much carbon it emits. “We’re building a global information utility,” he says.

Engineers must have spoken with similar passion when the first moving assembly lines were installed in car factories almost a century ago, and Microsoft’s data centre in Northlake, just like Henry Ford’s first large factory in Highland Park, Michigan, may one day be seen as a symbol of a new industrial era.

Before Ford revolutionised carmaking, automobiles were put together by teams of highly skilled craftsmen in custom-built workshops. Similarly, most corporate data centres today house armies of ‘systems administrators’, the craftsmen of the information age. There are an estimated 7,000 such data centres in America alone, most of them one-off designs that have grown over the years, reflecting the history of both technology and the particular use to which it is being put. It is no surprise that they are egregiously inefficient. On average only 6% of server capacity is used, according to a study by McKinsey and the Uptime Institute, a think-tank. Nearly 30% are no longer in use at all, but no one has bothered to remove them. Often nobody knows which application is running on which server. A widely used method to find out is: “Let’s pull the plug and see who calls.”

Limited technology and misplaced incentives are to blame. Windows, the most pervasive operating system used in data centres, allows only one application to run on any one server because otherwise it might crash. So IT departments just kept adding machines when new applications were needed, leading to a condition known as ‘server sprawl’. This made sense at the time: servers were cheap, and ever-rising electricity bills were generally charged to a company’s facilities budget rather than to IT.

To understand the technology needed to industrialise data centres, it helps to look at the history of electricity. It was only after the widespread deployment of the ‘rotary converter’, a device that transforms one kind of current into another, that different power plants and generators could be assembled into a universal grid. Similarly, a technology called ‘virtualisation’ now allows physically separate computer systems to act as one.

The origins of virtualisation go back to the 1960s, when IBM developed the technology so that its customers could make better use of their mainframes. Yet it lingered in obscurity until VMware, now one of the world’s biggest software firms, applied it to the commodity computers in today’s data centres. It did that by developing a small program called hypervisor, a sort of electronic traffic cop that controls access to a computer’s processor and memory. It allows servers to be split into several ‘virtual machines’, each of which can run its own operating system and application.

“In a way, we’re cleaning up Microsoft’s sins,” says Paul Maritz, VMware’s boss and a Microsoft veteran, “and in doing so we’re separating the computing workload from the hardware.” VMware and its competitors, which now include Microsoft, hope eventually to turn a data centre into a single pool of computing, storage and networking resources that can be allocated as needed. Such a ‘real-time infrastructure’, as Thomas Bittman of Gartner calls it, is still years off. But the necessary software is starting to become available. Perhaps surprisingly, it is Amazon, a big online retailer, that shows where things are heading. In 2006 it started offering a computing utility called Amazon Web Services (AWS). Anybody with a credit card can start, say, a virtual machine on Amazon’s vast computer system to run an application, such as a Web-based service. Developers can quickly add extra machines when needed and shut them down if there is no demand. And the service is cheap: a virtual machine, for instance, starts at 10 cents per hour.

If Amazon has become a cloud-computing pioneer, it is because it sees itself as a technology company. As it branched out into more and more retail categories, it had to develop a sophisticated computing platform which it is now offering as a service for a fee. “Of course this has nothing to do with selling books,” says Adam Selipsky, in charge of product management at AWS, “but it has a lot to do with the same technology we are using to sell books.”

Yet Amazon is not the only big online company to offer the use of industrial-scale data centres. Google is said to be operating a global network of about three dozen data centres loaded with more than 2 million servers. Microsoft is investing billions and adding up to 35,000 servers a month. Other internet giants, such as Yahoo!, are also busy building huge server farms.

In some places this has led to a veritable data-centre construction boom. Half a dozen are being built in Quincy, a hamlet in the middle of America’s Washington state, close to the Columbia River. The attraction is that its dams produce plenty of low-cost power, which apart from IT gear is the main input for these computing farms. On average, cooling takes as much power as computing. Microsoft’s new data centre near Chicago, for instance, has three substations with a total capacity of 198 mw, as much as a small aluminium smelter.

But cheap electricity is only one, albeit important, criterion for choosing the site of a data centre. Microsoft currently feeds 35 sets of data into an electronic map of the world, including internet connectivity, the availability of IT workers, even the air quality, to see where conditions are favourable and which places should be avoided. Apparently Siberia comes out well.

Many chief information officers would love to take their IT infrastructure out to sea and perhaps drown it there. Even as demand for corporate computing continues to increase, IT budgets are being cut. At the same time many firms’ existing IT infrastructure is bursting at the seams. According to IDC, a market-research firm, a quarter of corporate data centres in America have run out of space for more servers.

So IDC thinks that many data centres will be consolidated and given a big makeover. The industry itself is taking the lead. For example, Hewlett-Packard (HP) used to have 85 data centres with 19,000 IT workers worldwide, but expects to cut this down to six facilities in America with just 8,000 employees by the end of this year.

Other large organisations are following suit. Using VMware’s software, BT, a telecoms firm, has cut the number of servers in its 57 data centres across the world from 16,000 to 10,000 yet increased their workload. The US Marine Corps is reducing the number of its IT sites from 175 to about 100. Both organisations are also starting to build internal clouds so they can move applications around. Ever more firms are expected to start building similar in-house, or ‘private’, clouds.

But to what extent will companies outsource their computing to ‘public’ clouds, such as Amazon’s? James Staten of Forrester Research, a market-research firm, says the economics are compelling, particularly for smaller firms. Cloud providers, he says, have more expertise in running data centres and benefit from a larger infrastructure.

As a result, says Staten, cloud computing has not yet moved much beyond the early-adopter phase. Stefan van Overtveldt, the man in charge of transforming BT’s IT infrastructure, thinks that to attract more customers, service providers will have to offer ‘virtual private clouds’, fenced off within a public cloud. BT plans to offer these as a service for firms that quickly need extra capacity. So there will be not just one cloud but a number of different sorts: private ones and public ones, which themselves will divide into general-purpose and specialised ones. Cisco, a leading maker of networking gear, is already talking of an ‘intercloud’, a federation of all kinds of clouds, in the same way that the internet is a network of networks. And all of those clouds will be full of applications and services.