Celebrate A New Performance Standard

Updated: Jun 27 2003, 05:30am hrs
On May 20, 2003, Hewlett-Packard posted a TPC-C benchmark of more than 700,000 transactions per minute, the second highest ever performance in transaction processing. What is amazing about this is that the event also marks a single-digit price performance ($9/tpmC) at this level of performance.

Transaction Processing Council, an independent organisation, has been developing industry benchmarks by conducting measurements and publishing them regularly, for more than a decade. These benchmarks attempt to characterise different scenarios. The most important benchmark, the TPC-C benchmark, characterises online transaction processing (OLTP). This benchmark includes a peak performance measure, represented by transactions per minute of TPC-C (tpmC), as well as a price performance measure represented by price/tpmC (typically in dollars).

Currently, TPC-C measures the number of new order transactions per minute that can be generated by a computer system while performing five concurrent transactions of entering an order, monitoring stock levels, delivering an order, recording payment, and checking the order status. The transaction involves generic transactions of a firm with 100,000 catalog items, 3,000 customers spread over 10 sales districts, with an order response time of five seconds or less. That is, a 50,000 tpmC represents an ability to generate 50,000 new orders. For computing costs, the cost of basic hardware, operating system, database management system (DBMS) and TP monitor for a 24X7 operation are included, and maintenance costs for three years.

Just as car racing indirectly influences automotive performance, TPC-C ratings over the years have pushed the envelope of high-end transaction processing through a combination of high-performance servers, operating systems and DBMS; high-speed storage, and networking infrastructure. For example, in September 1992, when the TPC-C benchmark was first published, the top performer had 54 tpmC and a price performance of $188,562/tpmC. For a while, Sun Sparc 8-way processor and HP 9000 H70 servers with nearly 1,000 tpmC and $1,000/tpmC were considered the best. In 1998, the industry had moved to a new peak of 52,871 tpmC and price performance of $135. What is exciting about last months result is a peak performance of 707,112 and price performance of $9.13/tpmC.

Interestingly, the results show some interesting characteristics: o Mainframes are not dead. Yet, transaction processing is not for mainframes alone. Of the total of 120 published benchmarks as of June 2003, there were 23 from IBM, eight from Bull, seven from NEC, seven from Unisys, six from Fujitsu and six from Fujitsu-Siemens, all mainframe manufacturers. There were 41 from HP, 19 from Dell, and one from Sun; there was even one each from Network Appliance and Rack server, and none of them are mainframe guys!

* With HP bagging 41 and IBM bagging 23 of the 120 benchmarks, 50 per cent of the high-end OLTP market is held by the Top Two a clear case of consolidation.

* Of the 120 benchmarks, Microsoft SQL Server DBMS product accounts for 86 (Oracle and IBM DB2 getting 18 and three respectively). MS Windows accounts for 90 (IBM AIX, HP-UX, Sun Solaris, and Compaq True OS accounting for 12, five, four & four respectively). Within months of the launch, Windows Server 2003 has 38 out of 90 benchmarks, demonstrating the maturity of Microsoft products.

* The fastest price performance benchmark from HP is based on Itanium 2 microprocessor-based system, thus disproving the myth that commodity processors are not ready for OLTP.

* The lonely Red Hat Linux-based benchmark with $17.32/tpmC is nearly twice as expensive as compared to the best-performing HP $9.13/tpmC (that too with a peak performance of 707,132 tpmC compared to Linux performance of 138,362). So, Linux does not always save money.

* Perhaps the most interesting feature is a sure win for commodity products: Intel Itanium, Microsoft Windows Server 200 and Microsoft SQL Server products. They were dubbed not ready for prime time. Surely and steadily, their time has come.

It may not be celebration time for some vendors. Vendors like HP, Intel & Microsoft may celebrate, about which I am not too interested. What is truly interesting is that the end customer has something real to celebrate about a whopping 700,000+ tpmC at $9/tpmC!

The author is Director of the Indian Institute of Information Technology, Bangalore. Views expressed herein are personal. He can be reached at askss@iiitb.ac.in