Generative AI in the courtroom: 6% more cases, no quality drop
A GPT-4 assistant rolled out to Pakistani judges raised case throughput by 6% with no measurable change in decision quality. Here is what enterprise teams should read into the result.
The headline number, and what sits behind it
A field experiment with judges in Pakistan, working with a GPT-4 based assistant on real cases, raised the number of matters each judge moved through by about 6%. The companion finding is the one enterprise leaders tend to find more interesting: no measurable drop in decision quality. The study, by Elliott Ash, Sultan Mehmood, and Christoph Riedl, is one of the first clean empirical reads on generative AI in a regulated, high-stakes public function.
For most enterprises, the throughput result is the headline because it is the part of the value calculation that is easiest to defend in a board paper. Quality held steady in a setting where mistakes carry real consequences. That pairing is what makes the Pakistan result worth reading carefully.
What the assistant actually did
The pilot put an LLM powered drafting and research helper into the daily workflow of a randomized subset of judges. The model helped judges summarize case files, surface relevant precedents, and draft orders. Human judges remained the decision-maker on every case. The treatment group was compared against a control group of judges handling the same case mix, over the same period.
This is the kind of boundary that matters for any regulated deployment. The system did not decide. It accelerated the work that surrounds the decision, which is where most of the cost and time in legal, financial, and medical review already sits. The 6% throughput lift came from those surrounding tasks getting cheaper per case.
Why this matters beyond the courtroom
Three patterns travel with this result, and they show up in private-sector regulated work too.
First, throughput scales before quality moves, when quality is the bottleneck. Pakistan's judicial backlog is severe enough that even modest per-case acceleration compounds. Enterprise back-office queues, claims handling, compliance review, and onboarding all share the same shape: more cases than headroom, with quality gates the business will not relax.
Second, the assistant is paired with a human in the loop, not replacing one. The Pakistan deployment kept judges as the accountable actor for every decision. That is the same posture regulators across banking, healthcare, and the public sector have signaled they expect for the foreseeable future. Build the workflow around that posture from day one and the audit story writes itself.
Third, quality measurement is the part most enterprise pilots underinvest in. The Pakistan study pre-registered its outcome measures, including the quality instrument used to check decisions. Without that, a 6% throughput lift with no quality signal would be meaningless. Most internal AI pilots do not run a parallel quality check at all, which is why their results do not generalize.
Reading the number in context
A 6% throughput lift, taken alone, sounds modest. Inside a queue measured in millions of pending matters, modest per-case acceleration compounds quickly: a judge moving through 100 cases a year moves through 106 with the assistant, and that delta adds up across an entire bench over a single term. That compounding effect is the part of the result enterprise teams should plan around, because the same arithmetic applies to claims pipelines, onboarding queues, and compliance review. The acceleration is small per case and large in aggregate.
The experimental design is the other piece worth noting. The team randomized which judges received access and held the case mix, time period, and jurisdiction constant across the treatment and control groups. They also pre-registered the quality instrument before the rollout, which is what allows the "no quality drop" finding to carry weight outside the research community. Most enterprise AI pilots do not run that kind of clean comparison, which is one reason their results struggle to survive contact with a skeptical board.
For organizations looking at a similar deployment, the implication is not that 6% is the target. It is that the result was generated under conditions the business can actually defend: a randomized rollout, a pre-registered outcome measure, a human-in-the-loop posture, and a clear accountability boundary between the assistant and the decision-maker. Run a pilot under those conditions and the throughput number you produce, whatever it is, will be portable. Run it without them and the result will not survive its first review.
What enterprise teams should take from this
For board-level sponsors, the question is not whether generative AI can move the throughput needle in a regulated workflow. The Pakistan result is now part of the public record, and it points in the same direction as the early evidence from tax authorities, healthcare systems, and large financial institutions. The question is whether the deployment will hold up under the quality lens that already exists inside the business.
Three operational choices follow. Instrument the workflow before you turn the model on, so you have a quality baseline. Keep a human in the loop on every decision that has downstream accountability, and write that into policy rather than leaving it to individual practice. Treat the model as an accelerator of the surrounding work, and frame the rollout to leadership in those terms.
Done well, that posture is how an enterprise gets the kind of lift Pakistan's judges saw: real, defensible, and small enough to fit inside the risk envelope the business already accepts.
Book a strategy consultation to map these patterns to your own regulated AI rollout.