Fairness, measured

Every tool here does the same thing underneath: it makes a choice at random. That is a claim you can check, so this page checks it — with the method named, the run counts given, and a deliberately broken version measured alongside for comparison.

The measurement

Three checks run against the same code the tools use, from a script committed alongside it. They run on every change, on three versions of Node, and the build fails if any of them drifts outside tolerance. The figures below are written by that script rather than typed into this page, and the script refuses to pass if this page's data file and a fresh run disagree.

What was measured	Runs	Outcomes seen	Worst deviation
shuffle: all 24 permutations of 4 items control = list.sort(() => random() - 0.5), the most copy-pasted line in this product category	240,000	24 of 24	2.54%
shuffle: item-to-position matrix, 20 items each of the 20 items must reach each of the 20 slots equally often	200,000	400 of 400	2.70%
randomInt: uniform over [0, 10) 10 does not divide 2**32	1,000,000	10 of 10	0.56%

“Worst deviation” means the single worst-behaved outcome, not the average — the average is easy to make look good. Take the first row. A four-name list has 24 possible orders, so over 240,000 shuffles each one should appear about 10,000 times. The worst-behaved order was off by 2.54%, which is 254 counts.

That figure only means something next to the amount that chance alone moves it, which is the number most published claims of this kind leave out. At these run counts the ordinary spread of a single outcome's count is about 98. So the worst of 24 outcomes landing roughly 2.6 times that away is unremarkable — it is what an even draw looks like. A shuffle that produced no deviation at all would be the suspicious result, not the reassuring one.

The second row asks a different question, and it is the one that scales. Checking every possible order stops being possible almost immediately: a twenty-name list has more orderings than there are atoms available to count them with. So instead it asks whether each of the twenty names reaches each of the twenty positions equally often — 400 pairings, all of them observed, worst deviation 2.70%.

The one method that really is broken

There is a single line that this whole category copies from one another, and it does not work:

list.sort(() => Math.random() - 0.5)

It looks elegant, it is one line, and it returns a plausible-looking result every single time — which is exactly why it survives review. The problem is that a sorting algorithm calls its comparison function a fixed number of times in a fixed pattern, and it is entitled to assume the answers are consistent. Answering at random breaks that assumption, so the order you get back is decided by the internals of the sorting algorithm rather than by chance. Some orderings come out far more often than others, and which ones depends on the browser.

That is not an argument, it is a measurement. Both methods were run over the same list, for the same 240,000 shuffles, drawing from the same source of randomness with the same starting value. The only thing that differed was the method.

Method	Worst deviation from an even spread
Fisher–Yates — what this site uses Identical on every engine we run this on	2.54%
Sorting with a random comparison Measured on v24.11.1 — see below, this figure moves	350.36%

You cannot even say how unfair it is

Those two rows are not the same kind of number, and the difference is worth more than either figure. Fisher–Yates measures identically everywhere. We run this check on three versions of Node, and it produces 2.54% on all of them — the same seed, the same list, the same stream of random numbers, so there is nothing left for the platform to influence.

The sorting version does not. The same code, the same seed and the same numbers produced 350.36% on v24.11.1 and a figure several times smaller on a newer release — because the newer engine sorts differently, and it is the sort that decides the outcome. That is the mechanism made visible: with a correct shuffle the fairness is a property of the algorithm, and with this one it is a property of whatever happens to be running it. You cannot quote a number for how unfair it is, only for how unfair it was on one engine on one day. For a tool that is supposed to be even-handed, that is a worse position than a large error would be.

The broken version is kept in the codebase on purpose. It is the control: the same check that has to pass for our shuffle has to fail for that one, and if it ever stops failing, the build breaks. A check nobody has proven can fail is a check nobody should trust, so this one is proven to fail every time it runs — on every engine, whatever number it happens to produce there.

What we are careful not to claim

You will read in a lot of places that Math.floor(Math.random() * n) is biased and should never be used. It is not a real defect, and we are not going to pretend it is. The skew is on the order of one part in nine quadrillion — you would need to run a draw for considerably longer than the age of the universe before it made a difference to which name came out of a class register.

We use a different generator anyway, and the honest reason is not that the ordinary one is broken. It is that crypto.getRandomValues costs nothing to use and lets us describe the method precisely, which is worth something on a page like this one.

This paragraph exists because the incentive points the other way. A page like this is more persuasive with more villains in it, and inventing one would be the easiest thing on the site to get away with. It would also be the most expensive: a reader who catches one claim that does not survive checking is right to discount every other claim on the page, including the ones that took real work.

Where the bias actually is

There is a genuine version of that bug, and it is the one this site had to solve. A cryptographic generator hands back a whole number somewhere in a range of 2³² values. If you want a number from 0 to 9 and take the remainder after dividing by ten, you have a problem: 2³² is not a multiple of ten. Six of the ten results are reachable by one more starting value than the other four, so those six come up very slightly more often — every time, forever, in the same direction.

The fix is to throw away the small leftover block at the top of the range and draw again. That is what this site does, and it is what the third measurement above tests. The range chosen for that test is deliberate: ten was picked precisely because 2³² is not a multiple of it. A range that divided evenly would have passed the test whether the fix was there or not, which is the failure this whole page is trying to avoid — a check that cannot reach the thing it is supposed to catch reports success, and success reads as proof.

Reproducing this yourself

None of the above is worth much if you have to take our word for it, so here is everything you need to get the same numbers. The check uses a fixed, fully specified generator so that it produces identical results on every machine and every run — a check over genuinely unpredictable input is unreliable by construction, and an unreliable check gets switched off.

The generator is SplitMix32, started at 1234:

let state = 1234;

function next() {
  state = (state + 0x9e3779b9) | 0;
  let t = state ^ (state >>> 16);
  t = Math.imul(t, 0x21f0aaad);
  t = t ^ (t >>> 15);
  t = Math.imul(t, 0x735a2d97);
  return (t ^ (t >>> 15)) >>> 0;
}

The shuffle is Fisher–Yates, walking the list from the end:

for (let i = list.length - 1; i > 0; i--) {
  const j = randomInt(next, i + 1);   // an unbiased index in 0..i, inclusive
  [list[i], list[j]] = [list[j], list[i]];
}

Shuffle [0, 1, 2, 3] 240,000 times, count how often each of the 24 orderings appears, and take the one furthest from 10,000. Then swap the shuffle for the sorting line above and do it again. The two figures in the table should be the ones you get.

One thing to be clear about: that seeded generator is not what runs when you spin the wheel. It exists so the check is repeatable. In your browser the randomness comes from crypto.getRandomValues, which is the generator your browser ships for security work. What the measurement proves is that the method is even-handed; the generator underneath it is your platform's, and we make no claim about it beyond that it is the one built for this.

Mistakes designed out rather than warned about

Most of the ways a picker can quietly become unfair are not in the shuffle at all. Each of these was met by removing the possibility rather than by being careful around it, which is why none of them needs a warning in the interface:

The way it goes wrong	The usual defence	What this site does instead
The list and the “already picked” pile drift apart, so somebody is dropped from the draw or picked twice	Careful bookkeeping in both places	There is only one list. Being picked is a flag on the entry, so there is no second pile to disagree with it.
On a weighted wheel, rounding error makes the last entry unreachable — it can never win, and nothing on screen says so	Accumulate the fractions carefully	Weights are whole numbers only. A weight of 3 means “count this name three times”, so there are no fractions to accumulate and the last entry is reachable by construction.
A share link is too long, gets cut short, and the person who opens it draws from a shorter list than the one you built	Cut it at a safe character boundary	It refuses to make the link and tells you why. Nothing is ever silently shortened.
A tally of past flips gets read as telling you something about the next one	A note under the tally saying it does not	The tally has nothing of that kind in it — no “expected next”, no balance, no streak score. It reports what happened and stops.

The last one is worth spelling out, because it is the most common misunderstanding about anything that flips or rolls. Six heads in a row does not make tails more likely on the seventh throw. The coin has no memory, and neither does this site. Runs like that are ordinary — in a hundred flips, a run of six is closer to expected than surprising.

What the numbers do and do not cover

They measure the code the tools actually run. The selection logic lives in one set of functions, the tools call those functions, and a check enforces that no part of the interface can make a draw of its own behind their back.
They describe the method, not any single draw. An even-handed method still produces clusters, repeats and runs. If a name comes up twice in a row, that is what randomness looks like — a result that never repeated would be the one worth complaining about.
The wheel's spin does not decide anything. The pick is made first and the wheel is then animated to land on it, which is why the result is also announced as text and why the animation can be skipped entirely without changing anything.
They say nothing about your browser's generator. We use the one your platform provides for security work and treat it as sound. That is a reasonable position, not a measurement we have taken.