science in courtrooms

A very interesting article in Science News last week on a topic that I wasn't aware of, the controversy about scientific expert testimony in courtrooms. It seems that prior to 1993, lawyers had basically a free hand to bring expert witnesses into the courtroom. That is, in cases where scientific evidence is important, the jury was responsible for weighing the conclusions of the competing experts and coming to a decision about what evidence was reliable and what was not. Clearly, there is some compelling interest in making sure that juries aren't bamboozled by unscrupulous lawyers and unethical scientists.

In 1993, a Supreme Court Decision called Daubert v. Merrell Dow Pharmaceuticals allowed judges to evaluate the reliability and relevance of scientific evidence, prior to testimony in front of a jury. That is, a judge can decide, for example, that epidemiological evidence of toxicity of a chemical is admissible, but test-tube or animal studies are not. Importantly, since lower-court judges are considered the final arbiters of "evidence" in legal cases (just as juries are the final arbiters of "fact"), their decisions about what is admissible cannot generally be reversed by a higher court. As a result, there is a wide amount of inconsistency about what kinds of scientific evidence is admissible.

Some judges, [Margaret A. ] Berger [of the Brooklyn Law School] notes, "find all animal studies irrelevant"; others allow animal results in conjunction with findings on people. Many judges—but not all—throw out test-tube studies and analyses of chemical structures to gauge the activity of related compounds.

In addition, although judges are generally considered particularly intelligent people (I heard at one point that the IQ of lawyers is typically a bit higher than the IQ of scientists, and judges are generally more intelligent than the average lawyer), their understanding of the principles of scientific research can often be incorrect.

Berger says that few judges understand the scientific method... [J]udges who had applied Daubert standards to evidence "had little understanding of the key concept of hypothesis testing or of the significance of error rates," Berger says.

There are ongoing workshops and so forth that train judges in scientific methods and standards of evidence. One particularly interesting insight comes from philosopher Susan Haack of the University of Miami, who points out that:

this weighing of disparate but often interlocking bits of scientific evidence is little different from assembling forensic and other conventional evidence to establish a defendant's means, motive, and opportunity to have committed some crime.

That is, despite the compelling nature of (for example) null results in epidemiological studies of hundreds of thousands of people, there are many ways that such studies could be looking for the wrong thing or in the wrong population, and that evidence from other areas, including test-tube experiments and animal models, needs to be integrated to find the most probable state of affairs. Also, with regards to the apparent supremacy of epidemiological evidence by many judges, David Michaels of George Washington University points out:

because of the Daubert decision, the work, theories, and interpretation of data by even careful and credible scientists are often barred from trials. Restricting a jury's access to such information can diminish the likelihood that justice will be served, he argues.

This exclusion of science, he adds, might also affect the conduct and stature of research, as "judges essentially tell scientists that certain of their avenues of inquiry are not valued."

A very interesting article, with lots of things to think about, and worry about. From the descriptions of the situation in this article, I guess I'm of the opinion that judges should be able to exclude some scientific evidence, particularly pseudo-scientific evidence, but that the standards by which such exclusion can take place have been excessively lowered.

It's hard to know the best way to present potentially conflicting scientific evidence to a jury of laypeople with little to no knowledge of statistical evidence. Do juries weigh conflicting evidence fairly, or does the fact that evidence can be inconclusive make them throw out scientific evidence entirely? Certainly the system is not served by whole classes of evidence being thrown out, but nor is it likely to be served by allowing a jury to think that there is a controversy present when in fact there really isn't. (Bogus evidence against the addictiveness of cancer by tobacco companies, for example.)