Andrzej Uhl, PhD student and Gates Cambridge Scholar at the Institute of Criminology, has conducted new research into ‘noise’ among judges and its consequences for the defendants. He found evidence that different judges varied wildly in the sentences they proscribed, even for identical cases.
This research paper,produced in collaboration with Justin T. Pickett of the University at Albany, was published in Criminology, the field’s flagship journal. In this context, ‘noise’ refers to unsystematic variation between judges, who might choose very different punishments for similar or even identical crimes. Noise differs from systemic bias but is equally unjust. Random assignment of a case to a harsher judge might mean that an offender will spend many more years in prison than he or she would, had the case gone to a more lenient judge.
A dearth of knowledge
Unfortunately, Uhl’s research subject is difficult to study. It is very hard to prove that two cases are similar enough to deserve comparable sentences, so researchers struggle to pin down noise with scientific precision. Other researchers ask judges how they would punish a hypothetical crime and then examine whether their responses differ. Uhl doubted whether busy judges would pay much attention to such hypothetical cases – such studies may mistake inattention for genuine inconsistency. Uhl calls this “cheap talk” – the surveyed judges can give any answer without consequences for themselves or for real defendants. This is very different from real sentencing decisions.
“The previous studies are mostly hypothetical scenarios,” he explains. “You hope that some judge will spend 5 minutes reading a very short description of a hypothetical crime and then tell you what they would do. It's hardly detailed and they have no real stakes in it. We can only hope that they engage with these hypothetical stories.
“And then the other line of research is trying to use the real-life cases and compare them, to see if the ones that are similar attract different punishments. But of course, you can never hold every factor constant and there are no two identical cases.” Even advanced statistical models cannot tell the difference between nuanced case characteristics and genuine noise.
Measuring noise
In response to the research gap, Uhl decided to conduct his own research into noise within criminal justice. He identified a unique, naturally occurring experiment in adjudication - the judicial exam in his home country of Poland. During a gruelling six-hour examination, several hundred judges sentence the same crime based on a detailed case file. There is no doubt that judges consider the case carefully – their careers and future appointments depended on the exam results.
“This is an enormous incentive for judges to pay full attention to the case,” Uhl explains, “and since they all sentence one and the same offender unbeknownst to each other, we can see whether they treat like cases alike.”
Uhl says that the inspiration for his study was Dubner and Levitt’s book ‘Freakonomics’, which taught him that answers to some of social science’s most burning questions are often hiding in plain sight. The exam has been conducted for decades but no one noticed that its byproduct – sentences written under high-stakes conditions - present a unique opportunity to solve a long-standing research puzzle.
“Sometimes the really big social science questions can be explored with datasets that are collected independently of research,” says Uhl. “Once you locate this dataset, sometimes they are much more reliable than any experiments that researchers would conduct.”
The sentencing lottery
The study found enormous variation in what judges chose as the just punishment. When deciding how to punish a single offender, 54% of candidates sentenced the offender to a suspended prison sentence, 30% chose community service, 9% an unsuspended prison term and 5% a fine. Those who agreed on the type of sentence often differed on its severity. Fines varied by orders of magnitude and prison sentences ranged from a few months to several years.And all these sentences were chosen for one and the same crime.
“For an identical case of theft,” Uhl and Pickett wrote, “some judges would discontinue the proceedings all together or impose a fine equivalent to the price of six Big Macs in the local currency. Others would sentence the very same thief to 4 months in prison or 300 hours of community service. For an identical case of burglary, the offender may receive probation, if he was lucky enough to draw one judge, or spend a year in prison, if he was sentenced by another judge.”
In practice, Uhl and Pickett described criminal sentencing as "closer to roulette than to justice".
Next steps
Uhl now wants to expand on the findings. He is particularly keen to gather more data directly from individual judges, about how they perceive themselves and each other. This could help them realise the extent of noise and overcome what Uhl calls the “illusion of agreement”.
“One of the problems with noise is that it's not that visible,” Uhl explains. “You can be the harshest judge in your court or your district, without knowing at all, because you assume that everyone does the same as you. And so, they continue to choose different sentences”.
Once judges and the public realise the extent of the problem, consistency in penalties might become a priority for the criminal justice system.
Noise may be as damaging to the justice system as bias. Uhl’s results provide a basis for further research on this subject, including how noise and bias interact, and how one may provide cover for the other. This research could help make the justice system fairer.
Reference
Uhl, Andrzej, and Justin Pickett. Noise in judicial decision-making: A research note. Criminology; 04 December 2025; DOI: 10.1111/1745-9125.70021