The real Sasquatch genome scandal

SmalfutThe claim that a group of researchers, led by Melba Ketchum, had sequenced the Sasquatch genome was one of the wackier science stories of 2013.

Unable to find a home for their findings in any established journals, the authors started their own journal–DeNovo–and published their article on the Sasquatch genome there. (So far, this is the only article that has been published in the journal.) They held a press conference in Dallas on October 1st, and the media had a lot of fun covering the story.

To their credit, Ketchum and co-authors have made their article and the supporting data public. You can check out the article here and the supporting data here. They even made available some peer reviews that had been leaked after they submitted their work to Nature and the Journal of Advanced Zoological Exploration in Zoology (I’m not familiar with the latter journal). They also seem to have been very open in discussing their work with other scientists, which is great.

John Timmer provided a great summary of the considerable problems with the Sasquatch study over at Ars Tehnica. Here’s the short version: the sequence produced did not belong to bigfoot. It was an artifact resulting from sample contamination, degradation, and subpar assembly methods.

So OK, a wacky bit of research comes out, it’s rejected during the peer review process, and the post-publication process seems to have worked well too. The media seems to have accepted that this is not solid science, although it made for some great headlines. The system worked, and no real harm was done. Right?

That’s what I thought initially, but in reading about this story something started nagging me. Ketchum is a forensic scientist. She is director of a company called DNA Diagnostics that appears to specialize in animal forensics. Forensics is all about quality control, correct? If elementary quality control measures were not taken in this bigfoot study (and don’t even seem to have been understood), what does that say about the forensic work of this team–and in particular the work of the lead author?

This study was quickly debunked by scientists because it was published, albeit in a vanity journal, to great media fanfare. But forensic evidence is usually presented in court absent any real peer-review process. As the ongoing Annie Dookhan saga illustrates, when the integrity of forensic evidence is compromised, it can have real consequences. People go to prison for crimes they didn’t commit. Other people don’t go to prison because the evidence that should have put them behind bars is thrown out. Murders go unsolved. This is serious stuff.

According to her CV, Ketchum has presented forensic evidence in criminal cases. In death penalty cases, even. Death penalty cases in Texas, which has executed more prisoners since the death penalty was reinstated in 1976 than any other state. If basic quality control procedures were neglected in the presentation of this criminal evidence, as they were in the bigfoot study, what does it mean for the outcomes of the cases involved? (Incidentally, the problems with DNA Diagnostics–the lab Ketchum runs–seem not to be limited to bigfoot research.) What does this say about the quality of scientific evidence being presented in life-or-death cases in general? This reminds me of the incredible problems with expert scientific testimony revealed for shaken-baby syndrome and arson. I found the whole thing very, very unsettling.