A computer’s newfound ‘intuition’ beats world poker champs


(CNN)Poker expert Martin Sturc may have finally met his match: a computer program called DeepStack.

“You always have to be one step ahead,” said Sturc, president of the Austrian Pokersport Association. “When I realized he was adapting his gameplan, I had to adapt.”
Using a type of artificial intelligence its creators describe as “intuition,” DeepStack defeated 10 out of 11 professional players from the International Federation of Poker in a game of heads-up no-limit Texas Hold ’em. Each player played 3,000 hands against DeepStack.
    Sturc was the 11th player. DeepStack won by a margin that was too small to decide the overall winner; statistically, Sturc may be the one player who is still better than DeepStack.
    “It’s a huge compliment,” he said. “It feels good that there’s a small chance that I’m still at least on the same level as artificial intelligence.”
    The team of computer scientists that invented DeepStack say the program mirrors real-life decision-making more than traditional AI, and it may have uses beyond the poker table.

    A high-tech digital world

    To understand why DeepStack is so special, you have to understand the nature of the game, said Michael Bowling, one of DeepStack’s creators and the corresponding author on a paper published today in the journal Science.
    Bowling said that in most AI games, such as the chess-playing Deep Blue and Google’s AlphaGo, the board has all the information you need to make a decision.
    “There isn’t some knowledge in one of the other players’ heads that you wish you knew,” said Bowling, a computer science professor at the University of Alberta.
    But in Texas Hold ’em, the players can’t see each other’s cards. They all have a different view of the game. According to Bowling, this more closely mirrors the decisions we make in everyday life, which are often based on incomplete information.
    “It’s actually a rare moment when we say, ‘Oh, yeah, I have all the information I need to make this decision,’ ” Bowling said.
    Online poker sites have been around for years, and poker-playing AI is not necessarily new. But heads-up no-limit Texas Hold ’em is a different game altogether. Because there are no limits on the sizes of individual bets, the number of possible decisions a player can make is astronomically high: 1 followed by 160 zeroes.
    “(That’s) more than there are atoms in the universe,” Bowling said.
    A common estimation for the number of atoms in the observable universe is roughly 1 followed by 80 zeroes. When poker bets are fixed — a variation called heads-up limit — the number of zeroes drops to just 14.
    With so many possible choices, this version of Texas Hold ’em would be too complex for most computer programs.
    DeepStack “is different from ‘good old-fashioned AI,’ ” said Vlad Firoiu, a doctoral candidate at MIT’s Computer Science and Artificial Intelligence Laboratory, who recently developed an AI for “Super Smash Bros. Melee” that beat top players of the popular Nintendo fighting game. Firoiu was not involved in DeepStack.
    Whereas traditional AI calculated all the possible outcomes of a game beforehand, that was not possible or practical for a game with so many possible choices. So Bowling and other modern gamemakers have turned to a newer type of algorithm called deep learning. The more examples they feed a computer, the smarter it gets at handling situations it has never seen before.
    “Where we get (intuition) in DeepStack is very similar to how a human might get it … through experience,” Bowling said.

    Your intuition, it will lead you in the right direction


    Although games such as poker have allowed computer scientists to measure and benchmark their AI programs, Bowling and his colleagues also look forward to a future in which their algorithm can be adapted to real-world situations — like medical care.
    Bowling has started developing a mathematical model to improve care in patients with diabetes, but “we’re miles away from thinking about real trials on people,” he said.
    But Huettel doesn’t see poker and medical decision making as similar problems.
    “Poker is a zero-sum game,” he said. “Medical decision making usually involves aligned interests.”
    In medicine, he added, even if we know the best treatments, “the challenge will be getting people to follow those rules.”
    MIT’s Firoiu said that although DeepStack’s algorithm may be similar to those of newer games, it also uses many “hacks” that are poker-specific.
    Medicine would present additional challenges because we don’t know exactly how many diseases work, he said, as opposed to the simpler, fixed rules of poker.
    But poker expert Sturc said that DeepStack still has plenty of unreached potential within poker itself. He said it has created not just a new way to train for competitions but a way to reliably compare players before a game.
    Sturc, who studied poker and AI in graduate school, said that his approach to DeepStack was to think like a computer.
    “I didn’t play it 100% like I would a human being,” he said. “DeepStack does a couple of things that human players don’t do.”
    DeepStack would bet more than Sturc expected, making it difficult to tell whether it had a good hand or was simply “bluffing.” Sturc described its style as “unconventional … but very solid.”
    The next time they meet, however, DeepStack might be improved and harder to beat than ever.

    See the latest news and share your comments with CNN Health on Facebook and Twitter.

    One thing DeepStack does not do, Bowling said, is actively learn from an opponent’s strategy during gameplay. But it’s something his team is looking into, said Bowling — who, despite being an expert in AI and machine learning, is not much of a poker player himself.
    “I’m a terrible poker player,” he said, “so I need a computer to play it for me.”

    Read more: www.cnn.com


    Please enter your comment!
    Please enter your name here