Q&A: Professor Adam Chilton Demystifies Statistical Evidence in His New Book
Adam Chilton, Howard G. Krane Professor of Law, recently released a book, Trial by Numbers: A Lawyer’s Guide to Statistical Evidence, which he coauthored with Kyle Rozema, a law professor at Northwestern University. The book aims to give lawyers and other legal professionals a basic understanding of the most common methods of interpreting empirical evidence used in scholarly papers, policy briefs, and expert witness reports.
Chilton recently answered a few questions about his new book, published by Oxford University Press in May 2024.
What inspired you to write Trial by Numbers?
Being able to understand and engage with statistical evidence is increasingly important for nearly every kind of practice area in the legal profession. Data, statistics, and economic evidence play an important role in legal fields ranging from complex civil litigation to criminal defense work to transactional lawyering. But many lawyers do not gain a strong grasp of these subjects as an undergraduate, and law schools often do far too little to teach their students about them. Kyle Rozema and I thus wanted to write a book to make empirical evidence and methods accessible to all members of the legal profession.
How does Trial by Numbers help lawyers without a strong math background engage with statistical evidence?
Most books on empirical methods say they are going to be straightforward and easy to understand, and then within a page or two there are complicated equations and statistics. We wanted to write a book that was about the intuition behind statistical concepts with as few equations or complicated math as possible. In fact, there is only one equation in the entire book—though it does appear quite a few times. We also exclusively use examples that are drawn from legal education, the legal system, and legal practice—which we hope will make the material more engaging and understandable to the average lawyer.
What role do you see statistical evidence playing in the future of legal proceedings?
Trials are already frequently “battles of experts” where both sides put forward expert witnesses that report their own analysis of data or describe bodies of existing research. I don’t see any signs of that reality reversing in the near future. Instead, the increasing availability of data and academic research all suggest that litigation will be more and more focused on arguments between experts that bases their opinions on empirical evidence.
Can you provide an example from the book where statistical analysis played a critical role in a real case?
To teach the basics of regression analysis, we rely on describing (a slightly simplified version) of the facts of the recent case of Students for Fair Admissions v. Harvard. This was the case that ultimately led the US Supreme Court to rule that the use of race-based affirmative action in university admissions violate the Equal Protection Clauses. Although the Supreme Court opinions did not get into the weeds of the data, at the trial level the case was all about a battle of two leading economists that used regression analysis to evaluate whether Harvard had discriminated against students during the admissions process. We used these expert reports as the basis of the way we explain regression.
What are the most common mistakes lawyers make when dealing with statistical evidence, and how does the book help prevent them?
I’m not sure if it’s the most common mistake (I’d want data to make that kind of claim), but one frequent mistake made by lawyers, judges, and juries is to focus on whether a claimed statistical relationship is “statistically significant” instead of worrying about whether the relationship is substantively significant. In other words, some relationships can be statistically significant but trivial in magnitude, but other relationships can fall short of traditional levels of statistical significance but be substantial in size. We should shift our focus in litigation to worry more about effect sizes.
What advice do you have for law students or early-career attorneys who want to improve their understanding of statistical evidence?
Well, the obvious one is to buy and read our book. But another entry point is to find an area where you already have interests—like politics, sports, investing, or even pop culture—and start digging into the empirical work written on those topics that is designed for a popular audience. I can’t tell you how many data people I know that got their start by getting into things like advanced basketball statistics or election predictions. That’s a way to make learning more about statistics feel fun instead of like homework.
What are some challenges lawyers face when presenting statistical evidence in court, and how can they overcome them?
Tables of numbers frequently make people’s eyes glaze over. If possible, it’s extremely useful to turn statistical results into figures and graphs instead of tables and lists of numbers. This makes the results more engaging and easier to understand. The goal should be to make trial exhibits more like New York Times infographics than like the regression tables often found in medical research articles.
Anything else you’d like to share about the book?
We’ve tried to make this the most accessible book introducing empirical methods available. So, I hope it’s helpful not only for members of the legal profession potentially interested in the topic, but for anyone else too.