Ask a chat tool to pick a random number between 1 and 20, and one answer shows up far more often than chance allows. It isn't rolling dice. It's guessing which number sounds random, and that guess is predictable.

The favorite is often 17. It isn't always 17, and it isn't the same for every range, which matters more than the viral posts suggest. Below: what the model is actually doing, a five minute experiment to see it yourself, and one rule for when not to trust the answer.

What the model is doing when it "picks" a number

A chat model writes by predicting. Given everything in the conversation so far, it works out which word (or number) is most likely to come next, picks one, and repeats. That's the whole engine, whether you ask for a poem, a summary or a random number.

So when you type "pick a random number," nothing rolls. There are no hidden dice. The model effectively asks: what does an answer to this request usually look like in text? And it has read an enormous amount of text where people did exactly that.

The result is a guess at what a random answer looks like, not a random draw. There's a little built-in variation, so you won't get the identical number every single time. But that variation sits on top of strong favorites. You get a lopsided spread, not a fair one.

This isn't a malfunction. The model is doing exactly what it was built to do. The surprise is on our side: we expected dice and got a very convincing impression of a person blurting out a number.

Run the experiment yourself

Don't take this on faith. Open a fresh chat in any AI tool and paste this:

Try this prompt
Pick a random number between 1 and 20. Reply with just the number.

Then run it properly:

  1. Start a brand new chat each time, not a follow up in the same conversation. Earlier answers in the same chat influence the next one.
  2. Repeat 8 to 10 times.
  3. Write down every result.

Alternatively, run it once each in ChatGPT, Claude and Gemini and compare.

If the picks were truly random, each of the 20 numbers would have a 5% chance, so ten runs should give you mostly different numbers with maybe one repeat. What people consistently report instead is one or two numbers coming back again and again. For 1 to 20 and 1 to 25, informal tests keep landing on 17.

This bias is older than AI

The model didn't invent the habit. People have it too. In an informal 2007 poll on the science blog Cognitive Daily, 347 readers were asked for a random number between 1 and 20, and almost 18% of them picked 17 (source). Put that next to what a fair draw would give:

Sounding random vs. being random
True random draw, 1 to 20:     every number has a 5% chance
People in the 2007 survey:     17 was chosen 18% of the time

That's more than three times what chance allows. And the Jargon File, a long running glossary of hacker culture from Stanford and MIT, has called 17 "the least random number" for decades, long before any language model existed.

Why 17? It fits almost every rule of human intuition about what random looks like (breakdown here):

The bigger range favorites fit the same profile. 73 and 27 are odd, avoid 5s and 10s, and sit away from the edges.

Now connect it back to how the model works. It learned from text written by humans, including countless moments of people trying to sound random and reaching for exactly these numbers. The AI isn't producing a new kind of failure. It's reproducing a documented human habit that was sitting in its training material all along.

One irony: the claim "AI always picks 17" spreads because it's catchy and simple, which is the same reason 17 wins. The accurate version (a skewed spread that shifts with range and model) is less shareable, but it's what the evidence shows.

When this actually matters

For picking tonight's movie, a biased number is harmless. It becomes a real problem the moment the outcome needs to be unpredictable or fair: a raffle winner, a PIN or password, deciding who goes first when someone might object, picking a sample of customers to survey. Anyone who knows the favorites can predict or game the result.

The fix already exists. Many chat tools can run a small program behind the scenes, often shown as an "analyzing" step or a panel with code in it. A program that calls a real random function gives every number a genuinely equal chance. The skew only shows up when the model answers directly in plain text with no program involved.

Whether a tool does this on its own varies by product, plan and settings, and it changes over time. So don't assume. Ask for it explicitly:

Ask for a real random number
Use code to generate a random number between 1 and 20. Run it, then show me the code you ran and the result.

Then apply this rule to whatever comes back:

When to trust the number
Plain text answer, no code shown:  flavored guesswork. Not for anything that must be fair.
Code visibly ran to produce it:    trust it.
Not sure which happened:           use a dedicated random number generator instead.

A chat tool's random number is a guess at what sounds random, not a real draw, and the only way to know which one you got is to check whether code actually ran. Run the ten chat experiment today to see the skew with your own eyes, and next time you need a fair pick, ask the tool to run code or reach for a proper random number generator.