IF YOU were about to start playing a game of online poker, you might want to think again. Humankind has just been beaten at yet another game. This time, it’s ‘Heads-Up No-Limit Texas Hold’em poker’. An artificial intelligence (AI) system developed at Carnegie Mellon University (CMU) in the US recently racked up $1,766,250 worth of chips against four of the world’s top professional players after a marathon 20-day poker binge.
Among the claimed breakthroughs was the computer’s success in outbluffing its human opponents—a new milestone in the ongoing contest between man and machine.
The latest ‘achievement’ was one of the last remaining pastimes where humans still held sway after chess, the quiz show Jeopardy! and the boardgame Go fell to the robots. The first game that humans lost to machines was ‘backgammon’. In 1979, the world backgammon champion was beaten by Hans Berliner’s BKG 9.8 program.
In 1997, Gary Kasparov, who was the reigning world chess champion, lost to IBM’s Deep Blue program. Kasparov remarked that he could ‘smell’ a new form of intelligence across the table from him. Other games that have since fallen to the machines are Checkers, Othello, Scrabble, the general knowledge quiz Jeopardy! and even the classic arcade game Pong.
Most recently, the ancient Chinese boardgame of Go fell to the machines. In March last year, one of the leading Go players on the planet, Lee Sedol, was beaten 4-1 by Google’s AlphaGo program. Industry experts are calling the latest poker loss a milestone moment for AI. Let us see why. Tuomas Sandholm, professor of computer science at CMU, was quoted as saying by Financial Times that the poker win is “the last frontier” in the creeping series of victories that intelligent machines have recorded in human games.Last year’s Go victory was hailed as the ultimate test in a mind-stretching strategy game. Poker, however, tests different mental muscles since it involves strategising using imperfect information in a way that is more akin to the real world.
Go was described as the Mount Everest of boardgames. It is far more complex than chess or many other games. However, it is less of a challenge than poker.
You May Also Want To Watch:
[jwplayer amKJJFsM-DE6UeepY]
Like the real world, poker is a game of uncertainty. Players don’t know what cards the other players have or what cards will be dealt in the future. In a game like chess or Go, by comparison, all the players can see the board. Everyone has complete information. This makes chess and Go much easier to program than poker. Poker also requires understanding the psychology of the other players. Are they bluffing? Should you fold? Should you bluff?
Finally, poker involves betting. When should you bet? What should you bet? This again adds to the challenge of writing a poker program that plays as well as or better than humans. “This is not just about poker. The algorithms we have developed… can take any imperfect information situation and output a good strategy for that setting,” Sandholm, who developed the system with Noam Brown, a PhD student, was quoted as saying.
The technology could be used to compete against humans in business negotiations, military strategy, and the high-frequency trading systems used by the biggest banks, he said. The AI system, called Libratus, played the so-called Heads Up No-Limit Texas Hold ‘em version of poker—considered the most challenging form—over nearly three weeks of gruelling 10-hour days. Its early performance left the humans with hope of ultimate victory. However, the software gradually patched up the holes in its strategy and won with what Sandholm described as “a very large margin of victory and highly statistically significant”.
“Half-way through the challenge even we really thought we were going to win,” said Daniel McAulay, one of the professional players. “We really got a beat-down.” The CMU team used a supercomputer each night to analyse the day’s games and improve the software. Rather than try to study its opponents’ winning tactics, the system examined the weaknesses in its own game and patched up the three most obvious failures each day. That eventually enabled it to outflank most of its opponents’ tactics, which Sandholm called “the most psychologically devastating” aspect of the system for the humans playing against it.
The main improvement over other poker-playing programmes involved the way the computer approached the end game in each hand. Previous systems have adopted a single strategy to play out with each hand, but Libratus used an extra feedback loop to respond in real time to the humans on the other side of the table.
The technology behind Libratus will almost certainly be spun off into a new start-up company and developed for commercial use, Sandholm, who has studied negotiating strategy for 27 years, was quoted as saying. One of his earlier programmes is used by two-thirds of US transplant centres to decide which patients should receive new kidneys.
