Heuristics when LLMs play a betting game

24 Jul, 2025

TL;DR: In this post, I repeatedly get probability estimates from various LLMs according to the game from this post (which was about world models and agents). Without COT, we expect those estimates to come from heuristics and in this post we check how good are those heuristics.

Here we will take a look at what happens when LLMs play this simple game:

You are a perfectly rational gambler and your sole objective is to maximize your expected winnings. 
You will be presented with two distinct bets based 
on the outcome of a single roll of a fair 1024-sided die.

Here are the bets:

Bet A: You win if the die lands on a prime number.
If you win this bet, your payout is 50.

Bet B: You win if the die shows a non-prime number (including 1).
If you win this bet, your payout is ${payout_b:.8f}.

Pick which bet you would prefer to take, A or B, if you want to maximize your expected winnings.

DO NOT output ANY reasoning, just the letter of the bet you choose.

For reference, there are $171$ primes till $1024$ , so the probability of Bet A winning is $171 / 1024 \approx 0.168$

Optimal policy: If $p_{A}$ is the probability of Bet A winning, $π_{A}$ is the payout of A and similarly for $π_{B}$ , then the optimal policy is: stay with $A$ if $p_{A} π_{A} \geq (1 - p_{A}) π_{B}$ , otherwise switch to $B$ .

For most LLMs, this is a trivial bet if we use reasoning, so let’s see what kind of probabilities we can get when they work without reasoning.

We will sweep from $π_{B} = 0$ to $π_{B} = 100$ to find the tipping point: the value of $π_{B}$ for which the LLMs switch from Bet A to Bet B. Knowing that $π_{B}$ , we can then solve ${\hat{p}}_{A} π_{A} = (1 - {\hat{p}}_{A}) π_{B}$ to recover ${\hat{p}}_{A}$ which hopefully will be an approximation of the true probability.

In this case, ${\hat{p}}_{A}$ is the (implicit) probability that a player would have if they were to have the same tipping point as each LLM / agent. For a rational agent, this probability should be independent of the payoffs as it only relates to the event in Bet A.

Sweeping $π_{B}$

This plot shows the correct probability (vertical dashed line) as well as the estimates of ${\hat{p}}_{A}$ for each model (did multiple tries per model).

implied_prob_a

All runs are with temperature=0.

Interestingly, sonnet-4’s strategy underestimates the probability by a lot. It almost never switches between the bets, not until the estimated probability is around 1e-4.
gpt-4.1 and kimi-k2 are a bit more risky players, overestimating the number of primes.
mistral-large has the largest range of behaviors.

Overall, without reasoning, the models behave as expected: most stick to very safe strategies (never switch) while some occasionally overestimate the probability by a large margin, likely due to a ‘50-50’ heuristic.

Do the models know $p_{A}$ ?

If we ask all models to estimate the unknown probability $p_{A}$ without reasoning, they all get close estimates:

model_estimated_prob_a

Last words

This experiment asks the models to pick a bet for different payouts of B and we use those bets to check if the implicit strategy used is close to the optimal strategy. The optimal strategy requires a few steps, so it would be surprising if generic models could get good estimates without reasoning.

Estimating $p_{A}$ without reasoning is also tricky: a model would either have to recall all primes till 1024, count them, and estimate the probability, or recall its best approximation of the fact from the data.

GPT-4o’s approximation: By the prime number theorem, if we have $n$ numbers, the probability of selecting a prime is $P (A) \approx 1 / \log (n)$ , so if you memorize a few logarithms, you can ballpark this probability pretty well. For $n = 1024, 1 / \log (n) \approx 0.1442$ , which is close to the gpt-4o¹ estimate with zero temperature, $0.1445$ . Indeed, asking a similar question (asking for some light reasoning) on ChatGPT with 4o gives the response:

The total number of sides on the die = 1024 We want the probability it lands on a prime number. There are about π(n) ≈ n / ln(n) primes less than or equal to n. So,Estimated number of primes ≤ 1024 ≈ 1024 / ln(1024) ≈ 1024 / 6.931 ≈ 147.7 So the rough probability is: 147.7 / 1024 ≈ 0.1442.

Footnotes

You can replicate on ChatGPT via asking “Without thinking, estimate for me the probability that a 1024 fair-die would land on a prime number (I only want the floating number).”↩

Heuristics when LLMs play a betting game

Sweeping πB

Do the models know pA?

Last words

Footnotes

Sweeping $π_{B}$

Do the models know $p_{A}$ ?