What Randomized Experiments of Alcoholics Anonymous Can’t Tell Us

Might Alcoholics Anonymous Not Work for Those Who Won’t Participate in Randomized Experiments of AA? A Study of Breast Feeding Promotion Could Help Answer the Question

Alcoholics Anonymous’ “faith-based 12-step program dominates treatment in the United States. But researchers have debunked central tenets of AA doctrine,” wrote Gabrielle Glaser in April’s Atlantic. She says that AA may not work for many with alcohol problems and that AA’s supporters unfairly dismiss effective drug treatments based in neuroscience.

Jesse Singal immediately fired back that Glaser had missed several randomized experiments of 12-step facilitation showing its effectiveness. And last week Austin Frakt explained how such randomized experiments can be analyzed to “tease apart a treatment effect (improvement due to AA itself) and a selection effect (driven by the type of people who seek [AA] help).” Keith Humphreys, Janet Blodgett and Todd Wagner did just that using combined data from five randomized experiments to show that AA really works—for those who use it.

Who is right?

Everyone.

Continue reading

More Than Big Data Needed for Estimates as Good as Randomized Clinical Trials: Prospective Observational Causal Studies

A research methods post…

Bill Gardner is cautiously—very cautiously—hopeful that big data from electronic health records (EHRs) will enable unbiased estimates of the effects of medical treatment, without any randomized controlled trials.  Gardner’s hope, his caution and his description of the data needed—“all the factors that determine who gets what treatment”—are all right on the mark. And EHRs will massively increase data on detailed clinical factors that drive clinical decisions.

But I fear researchers will focus only on the data laying around in EHRs. To cure confounding, you need to go out and measure all the confounders—everything related to both treatment and outcome. Continue reading