How useful is what we have? Limitations of Cochrane Reviews, the case of substance treatment in pregnancy

Aims: To describe the public health promise and methodological problems of
Cochrane reviews.
Cochrane reviews are central to evidence-based medicine. However by focusing
exclusively on randomized controlled trials (RCTs) a large swath of the published
literature is excluded from their purview. Our experience conducting three of the
four published reviews that examined interventions for substance use in pregnancy
has brought particular methodological issues to light. Pregnancy is timelimited
event and therefore amenable to a RCT intervention. However this study
design is employed mostly in academic centers serving poor inner-city women.
How useful then is this evidence? We begin with a brief overview of the
Cochrane Library with particular emphasis on the addiction and obstetric literature.
Then the following methodological topics are addressed: the tensions
between clinically useful, clinically expedient, and sociologically meaningful outcomes;
the difference between experimental and real world interventions; problems
in external validity due to RCT recruitment in pregnancy; and confounding
and bias in exposure assignment. We contrast our results from Cochrane
reviews with the observational studies that were excluded from our reviews.
Finally we argue for a more flexible design for systematic reviews that can allow
the inclusion of more diverse study designs. The current Cochrane software
(RevMan 5.0) appears to offer hope for this.

Conclusions: Pregnancy is a unique moment in a woman’s life course where individual
motivation for behavioral change is coupled with the availability of social
support (such as prenatal care). Furthermore, interventions for drug problems in
pregnancy have the possibility of not only improving birth outcomes, but affecting
maternal health and child development across the life course. The summary
of quality evidence is therefore necessary in the guidance of both clinical medicine
and current policy.

Support: Part of this study was funded by the Alcohol Education Research
Council, UK.
612