Saturday, March 23, 2013
The Hellmann-Zanetti Appeal Court’s DNA Consultancy Looks Even Worse In Face Of The Latest Science
Posted by Fly By Night
[Above: images of typical modern analysis DNA facilities similar to Dr Stefanoni’s in Rome]
The Galati appeal to Cassation comes down very strongly against the work and conclusions of the appeal court’s DNA consultants Vecchiotti and Conti.
Dr. Galati argues that the consultancy should never have happened at appeal level, that its methods were slipshod and out of date, that its conclusions were mainly innuendo that left the prosecution case untouched, and that the consultants should not have refused to test a remaining sample from the large knife collected at Sollecito’s place.
In July 2011, about the midpoint of the appeal trial, I took strong issue with the C&V science and essentially mirrored in advance what Dr. Galati would argue to Cassation nearly a year later. Many other TJMK posters including our legal posters James Raper and Cardiol took issue with legal and other aspects.
With a Supreme Court ruling on the 2nd level (first appeal level) outcome scheduled for early next week, it’s the perfect time to re-examine the role of DNA in that outcome against the latest science. I want to include some excellent observations from our contributing poster “Thoughtful” as expressed in her recently published book Math on Trial.
I’ll start off with an overview of the science of DNA analysis and describe recent developments in analysis approaches, techniques and capabilities. Incidentally, one of my resources for this information is a chapter in “DNA Electrophoresis Protocols for Forensic Genetics” published shortly after the Hellmann verdict for the first appeal (circa early 2012); a chapter in which Carla Vecchiotti is cited as providing technical assistance.
Given Vecchiotti’s involvement in recent academic publications we can be certain that at the time of the Hellmann verdict Vecchiotti was well aware of the rapidly evolving and improving nature of DNA testing procedures and capabilities. And in contrast to her courtroom allegations that Dr. Stefanoni had not followed “internationally established forensic science standards” in her DNA analysis techniques, Vecchiotti has recently contributed to sources claiming that today’s critical challenge is to develop general guidelines for DNA evaluation and promulgate clear and universal laboratory practices while recognizing that a multitude of labs exist, each with its own specific protocols and personnel.
We will return to the Conti-Vecchiotti report shortly, but first let’s have a quick look at the history and state-of-the-art of DNA analysis.
Brief History of DNA Testing
The literature reveals that the USA has never been at the forefront of forensic DNA analysis. The first court cases to successfully employ DNA “fingerprinting” techniques occurred in England during the mid 1980s. A case involving a double rape/homicide of teenage girls in 1986 turned out to be prophetic in that it involved the first use of DNA to exonerate an innocent suspect and also was the first to apply DNA “databases”, issues which still give rise to disputes nearly 30 years later.
Over time, a variety of procedures were developed to extract DNA from biological samples but all worked on the same basic principle of breaching individual cell walls, removing the protein surrounding the DNA, isolating the DNA, and finishing with the purification and quantification of the DNA.
An important milestone in DNA fingerprinting was the development of the Polymerase Chain Reaction (PCR) in 1985. The PCR quickly became an important analytical method for forensic samples because of its sensitivity, specificity, rapid analysis, and ease of automation. PCR amplification technology permitted the analysis of forensic samples with low quantities (less than 1 ng) of extracted DNA, unlike earlier methods that required at least 50 ng.
While PCR was far more sensitive than earlier procedures, problems with mixed DNA samples and DNA degradation led to the use of genetic markers known as Short Tandem Repeats (STR). STR analyses were fast and reactions could be multiplexed permitting multiple loci to be amplified in a single run.
In 1997 the Federal Bureau of Investigation (FBI) Laboratory in the USA launched an effort to establish a set of 13 core STR loci for use within a national DNA database known as CODIS (Combined DNA Index System). Similar sets of STR markers had already been selected by the European Union and elsewhere but, in general, a DNA profile obtained using 12 or more STR loci was found to yield a composite genotype frequency of less than 1 in a quadrillion. This high degree of accuracy results from the hereditary nature of STR distribution and enables a very powerful method for biological identifications.
STR typing of extracted DNA has traditionally been very sensitive to the quantity of input DNA with ideal levels ranging from 0.5 to 2 ng. Either too little or too much DNA could produce imbalanced amplification results resulting in incomprehensible outcomes. The STR process is further complicated by “stutter” in the interpretation of multiple contributor DNA samples. Stutter is an artifact of the PCR process that produces “false alleles” one repeat shorter than a primary allele.
In recent years DNA analysis techniques have evolved rapidly as equipment manufacturers upgrade STR systems to tolerate even the smallest of samples and samples that have been highly degraded. The improved sensitivity of today’s STR kits along with the development of new strategies for the amplification of low levels of DNA now allows samples which previously could not be analyzed to produce viable results.
Low-level DNA samples often contain mixtures of DNA, which has complicated the detection and interpretation process due to stochastic sampling effects that include peak imbalance, enhanced stutter, allele loss (allele drop-out), and un-attributable alleles (allele drop-in). With this in mind, strict guidelines have been developed including a careful determination of analytical thresholds and the use of replicate analyses in a profile to properly interpret low-level mixed-DNA samples. More importantly, new analytical techniques such as laser micro-dissection and fluorescence in-situ hybridization have been developed enabling the identification, capture, and amplification of DNA from individual cells prior to “electrophoresis”, eliminating the problem of mixed profiles altogether.
In addition to today’s far more precise DNA analysis machines and methods there are also compelling arguments for the use of statistical or probabilistic models within the DNA analysis process to augment traditional “consensus allele” electropherogram evaluation approaches. In short, the efforts of both scientists and statisticians are now creating powerful next generation approaches to DNA analyses as we progress through a second decade of highly successful STR typing methodologies.
Logic and Science on Trial
In my 2011 report I challenged Carla Vecchiotti’s contention that Dr. Stefanoni had not followed “internationally established forensic science standards” in her DNA analysis techniques. Vecchiotti herself has conceded to the challenge through her contributions to publications that clearly describe a need to develop generally accepted guidelines for DNA evaluation and to create clear and universal laboratory practices that can be accepted by the diverse population of analytical labs currently operating under divergent operational protocols, all under the direction of professional and expert personnel.
In her excellent and recently published Math on Trial book, contributing poster “Thoughtful” accurately describes how DNA analysis expert Dr. Patrizia Stefanoni proceeded in her laboratory analysis of a small DNA sample found on the blade of a knife confiscated from Raffaele Sollecito’s apartment. Not having what she felt was a sufficient sample to divide for replication of her analysis Dr. Stefanoni took the chance of running her entire sample in a single run.
As is typical of all DNA analyses, Stefanoni proceeded to amplify the results to a point where an electropherogram would reveal meaningful “peaks” and found that a resultant 13 pairs of peaks corresponded precisely to peaks derived from a known sample of Meredith Kercher’s DNA!
In this case it is pointless to attempt to argue that Stefanoni somehow exceeded the amplification limits of her equipment. As outlined in the DNA discussion above, the typical problems associated with an amplification of low levels of DNA are related to peak imbalances, enhanced stutter, allele drop-outs, or allele drop-ins. In this case there was nothing but a perfect match for Meredith that even Carla Vecchiotti and Stefano Conti could not deny in court.
Stefanoni had clearly identified an identical match for Meredith’s DNA on the blade of Sollecito’s kitchen knife, leaving Vecchiotti and Conti no other option than to argue for “contamination” in court. However, it was convincingly demonstrated by Stefanoni and all evidence handlers that from knife collection through laboratory analysis no reasonable opportunity for contamination with Meredith’s DNA existed.
In the first appeal trial, Judge Hellmann was thus presented with exceptionally compelling evidence that Meredith’s DNA was in fact found on the alleged murder weapon that had been confiscated from Raffaele Sollecito’s apartment. Astonishingly, Hellmann rejected this evidence on an expressed assumption of non-compliance with testing techniques established by international scientific community standards; compliance standards that Vecchioti herself admits do not exist via recent academic and scientific publications as discussed above.
As “Thoughtful” carefully explains in Math on Trial, Hellmann’s faulty reasoning in excluding the knife evidence did not end there. Hellmann provided Vecchiotti and Conti with an opportunity to retest any remaining DNA on the knife if they felt it was warranted. Vecchiotti and Conti declined to perform any retests on the basis that that only a few cells might still exist on the knife, thus invalidating any potential results according to a false assumption that “international testing standards” somehow prohibited such low-level DNA tests even though, as outlined in the DNA discussion above, single-cell DNA analysis had at that time already become an acceptable possibility and Vecchiotti knew it.
Hellmann, however, accepted Vecchiotti and Conti’s reasoning by essentially stating that repeating an “invalid” DNA analysis procedure twice can do nothing towards resolving a DNA identification problem because two wrongs do not make a right. In Math on Trial, “Thoughtful” artfully explains the complete failure of logic of Hellmann’s line of reasoning. Hellmann claims that running an experiment independently two separate times and obtaining the same result each time can do absolutely nothing towards increasing the assurance of reliability for an event.
However, “Thoughtful” describes how successfully repeating Stefanoni’s low-level DNA analysis technique could easily carry a probabilistic result from a “not beyond a reasonable doubt” percentage range to a highly convincing 98.5% or higher probability. “Thoughtful’s” arguments in Math on Trial are completely in line with today’s efforts to embed statistical and probabilistic models within the DNA analysis process for a much higher precision and accuracy standard.
Conclusions
In 2011 I concluded that Vecchiotti and Conti’s expert report findings actually boiled down to two primary debates: (1) Issues surrounding the small sample (Low Copy Number ““ LCN) DNA analysis techniques employed by Dr. Stefanoni, and (2) Issues surrounding the probability of excluding all possible sources of contamination from the evidence.
In 2013, on the eve of the Court of Cassation ruling on the first appeal outcomes of the Meredith Kercher murder trial, it appears to me that all issues related to DNA analysis and contamination have been powerfully addressed by both the prosecution and “best available science” considerations.
The errors in Judge Hellmann’s logic and reasoning that set Amanda Knox and Raffaele Sollecito free have been shown to be plentiful and astounding, as evidenced by the few DNA related examples that have been examined in this report. In light of all of the above and the powerful legal arguments raised by the Galati appeal to Cassation, it seems that there can be no other option than to send this appeal outcome back for a thorough lower court re-evaluation.
Posted by Fly By Night on 03/23/13 at 11:05 AM in Evidence & Witnesses, The two knives, DNA and luminol, Appeals 2009-2015, Hellmann critiques, Hoaxers from 2007, More hoaxers
Comments
Thanks FBN for this outstanding piece about the history of and recent developments in DNA testing.
Conti and Vecchiotti never proved there had been any contamination. Alberto Intini, the head of the Italian police forensic science unit, pointed out that unless contamination has been proved, it doesn’t exist.
Galati made the following common sense observation in his appeal:
“It is evident that the “non-exclusion†of the occurrence of a certain phenomenon is not equivalent to affirming its occurrence, nor even that the probability that it did occur.” (The Galati appeal, page 57).
It’s shocking that Professor Conti doesn’t seem to understand this self-evident truth, but instead claimed anything is possible.
“Professor Conti, questioned by the Public Prosecutor on the point, limited himself to claiming that “Anything is possible†(see pp 70 and following of the transcript of the hearing of 30 July 2011). (The Galati appeal, page 59).”
It should be noted that the Italian Scientific Police follow the guidelines of the ENFSI - European Network Forensic Science Institutes. Dr. Stefanoni observed that they followed these specific guidelines whereas Conti and Vecchiotti basically picked and mixed a random selection of international opinions:
“We followed the guidelines of the ENFSI, theirs is just a collage of different international opinionsâ€.
Incidentally, there is a great interview with the authors of Math on Trial which I found absolutely fascinating:
http://newbooksinlaw.com/2013/03/13/leila-schneps-and-coralie-colmez-math-on-trial-basic-books-2013/
Posted by The Machine on 03/23/13 at 03:07 PM | #
Thanks, Fly By Night, for this update. Impressive analysis.
In addition to the advances that have already been, and continue to be, made, I am also inclined to think that the administrators of justice systems see to it in future that judges of sufficient calibre and experience are always allotted to cases (certainly at Appeal Court level) involving disputed DNA evidence. Judges should have to rack up sufficient course and seminar hours on the subject so that they can belong to a panel.
This is not so that they replace the experts in any way but so that it can be known that they are at least capable of following the evidence, something which Hellmann and Zanetti (Appeal Court judges, no less) effectively and embarrassingly asserted they were unable to do as a reason for appointing the independent experts in the first place.
This would make for a powerful inquisitorial model in the field in places like Italy leaving our own adversarial models mired in amateurism at trial level because, of course, one can’t put lay jurors through a crash course before being selected.
Posted by James Raper on 03/23/13 at 03:32 PM | #
Statistics is not math; it is more of a black art.
I am unashamed that I regularly approach a profressional statistician even for simple problems. Better be safe than sorry!
The pictures, perhaps taken from a catalogue, was meant to impress on the importance of cleanliness. In my humble opinion, nothing can be further from the truth. The cleanliness is a matter of personal habit and good lab practices. I have seen many labs in Italy and they just look like, hmm, like labs everywhere.
One thing was missing from the C&V report: professionalism. Why so?
Posted by chami on 03/23/13 at 07:19 PM | #
“Statistics” can be effectively used for many purposes, however, in critical applications results and methods should always be subjected to expert peer review.
Anyone who seeks to use statistical evaluations to make a point, pass a law, or send someone to jail had best be prepared to discuss in depth how they arrived at their conclusions and explain the limitations.
Buyer beware - always ask questions!!
Posted by Fly By Night on 03/23/13 at 07:50 PM | #
Hi Chami,
Conti and Vecchiotti didn’t visit the laboratories of the Scientific Police or ask about their cleaning procedures. The defence experts were invited to observe the DNA tests carried out by Dr. Stefanoni. As far as I’m aware, none of the defence experts had any concerns about the cleanliness of these laboratories.
Posted by The Machine on 03/23/13 at 08:07 PM | #
@Fly by Night,
I said in half-jest. But you can see the link for some more fun. http://www.sciencenews.org/view/generic/id/60598/description/When_intuition_and_math_probably_look_wrong
Some semesters I teach biostatistics. I try to add some fun into learning.
@The Machine
I was just trying to allude to the contamination paradox.
AK’s DNA was found on a rather few samples, although the common belief is that she had not-so-clean habits.
Posted by chami on 03/23/13 at 08:58 PM | #
Comments on the article below show the after effects of the vile ‘Curt Knox’ PR campaign
Posted by Urbanist on 03/23/13 at 11:06 PM | #
Just goes to show how misinformed the public are of the real facts.
Posted by Urbanist on 03/23/13 at 11:07 PM | #
Hi Urbanist,
The majority of the people who have commented about the case on The Daily Mail website have embarrassed themselves with their ignorance and stupidity. I expect much more from the journalists who are paid to provide factually accurate and balanced articles and reports. Unfortunately, most of them are ignorant of the basic facts of the case and are happy to act as mouthpieces for Amanda Knox’s family.
Posted by The Machine on 03/24/13 at 12:53 AM | #
Excellent post, FBN.
As the Galati appeal notes (p57) contamination can’t just be a theoretical possibility, it has to be plausible. We have not heard a plausible contamination scenario for the knife, and certainly not for the bra clasp.
DNA can be challenging for lay judges, but that doesn’t stop courts from hearing DNA cases every day. Court-appointed DNA experts are virtually unheard of, and Conti-Vecchiotti’s analysis may have been done with an eye toward lucrative defense work in the future.
The C-V report comprises 25% of the Hellmann report by space, and in reality much more than that, since the supposed unreliability of the DNA evidence is used to shrug off other compelling evidence and testimony.
If the case goes back to trial, as it should, the C-V report should be considered as just another expert opinion. The fact that it does not address plausible routes of contamination should be examined closely.
Posted by brmull on 03/24/13 at 04:18 AM | #
fbn - delighted by the book Math on Trial, but I can’t find a poster called ‘Thoughtful’ . Any suggestions?
[added] ps fbn - I found Thoughtful - just joined TJMK, no posts here yet. Looking forward to Thoughtful posts.
Posted by Cardiol MD on 03/24/13 at 12:35 PM | #
Thoughtful is one of the veteran posters at PMF.org, Cardiol.
Posted by Vivianna on 03/24/13 at 04:03 PM | #
Hi brmull.
Great comment on Hellmann. However there is no possibility of the case going back to TRIAL as there already was one trial (Massei’s), on the whole a good one except for ascribing to Guede the role of prime motivator and giving the perps some breaks for “mitigating factors”.
Next time around (and for good reasons we think there will be one) the APPEAL will be run in whole or in part properly. It will much more resemble a UK or US appeal than the illegal repeat of a full trial attempted by Hellmann & Zanetti.
For the reasons Fly By Night and you give, among others, and the arguments by Galati, the DNA on the knife and bra clasp quite possibly wont be contended by the defenses. There just isnt enough “there” there and the strapped defenses know it.
If the DNA is revisited, it most lilkely will be because of a Cassation requirement or prosecution plea for Stefanoni’s remaining knife sample to be tested. If Meredith’s DNA appears, case REALLY over, and RS and AK head back to the Big House.
Posted by Peter Quennell on 03/24/13 at 05:06 PM | #
As has been mentioned elsewhere, and just to point it out, one of the Knox brigades greatest fears was/is that science will overtake them . What was difficult (DNA profiling etc:) four years ago is now commonplace. I hope they read this because it is bound to give them a whole lot of worry. That’s good in my book at least.
Posted by Grahame Rhodes on 03/24/13 at 08:55 PM | #
Thank you Fly By Night for this very timely article. For me, the DNA part of the case is one of the most confusing aspects to grasp and I am thankful to people who explain this rapidly changing science more in detail.
For anyone who would like to read Thoughful (“Leila”) in action, and BRILLIANTLY answering questions (and graciously fielding the abuse from the groupies) about her book, there is a rapidly growing, public thread, “Math on Trial” over at the innocent site, IIP. It’s quite remarkable and informative to read HER posts over there. BRAVO Leila / Thoughtful! You rock!
Thank you everyone. May tomorrow bring the light of justice for Meredith and her family.
Posted by Tara on 03/24/13 at 09:47 PM | #
Commenting is not available in this channel entry.
Where next:
Click here to return to The Top Of The Front PageOr to next entry Supreme Court Appeals: A Good Briefing On Tomorow’s Court Proceedings By Italy-Based Andrea Vogt
Or to previous entry One Week From Today The Rulings Of The Supreme Court of Cassation Should Be Announced