Poker is a game of incomplete information, which makes it an ideal candidate for artificial intelligence (AI) research. In poker, players are dealt cards and must make betting decisions based on the value of their hand and the bets made by other players.
The goal of AI in poker is to create a program that can make optimal betting decisions given the information available.
AI poker research began in the 1970s with simple programs that could only play a very limited form of the game. These early programs were not very successful and often lost to human players.
However, they laid the groundwork for more sophisticated AI programs that would come later.
In the 1990s, AI researchers started using artificial neural networks (ANNs) to create programs that could learn to play poker. ANNs are a type of machine learning algorithm that can learn from data.
They are similar to the human brain in that they can learn to recognize patterns and make predictions.
ANN-based poker programs were initially quite successful, but they soon ran into a problem known as the “brittleness problem.” This problem occurs when an AI program is only able to find one or two winning strategies and then sticks to them even when circumstances change.
This can be a serious problem in poker, where the value of cards can change from hand to hand.
To overcome the brittleness problem, AI researchers started using reinforcement learning (RL) algorithms. RL algorithms are designed to learn by trial and error, like humans do.
They start with no knowledge of how to play the game and then gradually improve their skills by playing against other RL algorithms or human opponents.
One of the first RL-based poker programs was called Polaris. Polaris was developed by computer scientist Michael Bowling and his team at the University of Alberta in Canada.
It was designed to play heads-up limit Texas hold’em, which is a simplified form of poker with only two players and fixed betting limits.
Polaris was initially not very good at playing poker, but it quickly improved through self-play. After just two days of training, Polaris was beating some of the best human players in the world at heads-up limit Texas hold’em.
It continued to improve after that and eventually became unbeatable at this form of poker.
Polaris’s success showed that RL could be used to create an AI program that could beat humans at poker. This led to a wave of new RL-based poker programs being developed in the late 2000s and early 2010s.
These programs were designed to play more complex forms of poker, such as no-limit Texas hold’em, which is the most popular form of poker in the world.
The first no-limit Texas hold’em program was called DeepStack. It was developed by computer scientist Neil Burch at the University of Alberta in Canada.
DeepStack was designed to play heads-up no-limit Texas hold’em, which means it was only pitted against one human opponent at a time.
DeepStack was initially not very good at playing no-limit Texas hold’em, but it quickly improved through self-play just like Polaris had before it. After just two days of training, DeepStack was already beating some professional human players at this form of poker.
It continued to improve after that and eventually became unbeatable at heads-up no-limit Texas hold’em as well.
DeepStack’s success showed that RL could be used to create an AI program that could beat humans at even the most complex forms of poker such as no-limit Texas hold’em. This led to even more RL-based poker programs being developed in recent years with the goal of beating humans at this form of poker too.
So far, these programs have been successful in beating even professional human players at no-limit Texas hold’em under tournament conditions (where all players start with an equal amount of chips).
The current state of AI in poker is thus quite impressive: there are now multiple AI programs that can beat even professional human players at various forms of poker such as heads-up limit Texas hold’em and heads-up no-limit Texas hold’em.
