Study: Many courtroom psychology tests are unreliable

When scientific or expert testimony is to be admitted as evidence, courts are supposed to ensure that the evidence is reasonably reliable. That means sorting out the pseudoscience from the real science. Technical evidence should not be admitted unless it involves techniques and theories that are widely accepted by the field in question.

Yet many courts appear to be failing in their responsibility to keep bad scientific evidence out of court.

“There’s huge variability in the psychological tools now being admitted in U.S. courts,” says a psychology professor who co-authored a recent study on these types of tests. “There’s a lot of stuff that looks like it’s junk and should be filtered out by the courts, but it’s not being filtered out,” she added.

These include psychological tests and IQ tests, for example, which could have an enormous impact on the defendant’s trial.

To find out just how variable courts have been in this regard, the psychology professor and other researchers reviewed 876 court cases with psychological evidence that took place in the U.S. between 2016 and 2018.

One thing they found was very interesting: defendants made legal challenges to the use of psychological tests as evidence in less than 3% of the cases.

What were the most common tests being admitted as evidence?

The good news is that the most common test the researchers found being used was the Minnesota Multiphasic Personality Inventory, or MMPI. This is a common test for mental illnesses that is widely accepted in the field.

On the other hand, the second most common test being used was the Rorschach test, or “ink blot” test. First developed in 1921, the test is not generally considered reliable by the psychological community. It does have its defenders, but it is not widely accepted as reliable. Many psychologists say that it is ambiguous and subjective.

There were hundreds of different types of tests used in the court cases. Of those, about a third had never even been reviewed by psychology’s most prominent journals. Of the two-thirds that had been reviewed, only 40% received generally favorable reviews. Nearly 25% were considered unreliable.

Lawyers and judges are not psychological experts, but they still have a duty to ensure that the evidence admitted in criminal cases is credible and based on the highest scientific standards. When questionable psychological testimony is presented, defense attorneys can challenge its admissibility. Judges can also insist that scientific evidence be at least generally accepted in its field.