On reproducibility: the risks of the replication drive

ReplicationAn article called Reproducibility: the risks of the replication drive just came out in Nature. In it, Mina Bissell makes some great points.

The main idea: replicating studies is hard. It’s easy to tweak something (without even knowing that you did it) and end up with different results. Because of this, it’s important not to cast doubt on the results of someone else’s experiment too quickly. Communicating with the lab who did the original study is important if you find yourself running into problems. Failure to replicate can have serious consequences: good scientists can lose credibility, promising lines of research may not be pursued, etc. Thus, attempts at replication should be taken seriously, and everyone should try to remain civil during the process.

OK. I think everyone probably agrees with that!

But there were parts of the article that made me a little uncomfortable. Bissell gives compelling examples of how tiny changes–using the same cell line, but from different laboratories, for ex.–can torpedo replication attempts. I too believe that this happens frequently, so no arguments there. But unlike Bissell, I see this as a major problem. If you can’t replicate a study using virtually (but not totally) identical conditions, how generalizable are the original results likely to be? How useful is an experiment that yields such shaky findings? If we can’t replicate findings in the lab, what are the odds that they will describe what’s happening out in the messy real world?

Bissell describes a comforting example in which exploring a failure to replicate under slightly different conditions yielded valuable scientific data. I’m sure there are serendipitous situations like that one, but I also suspect that they are few and far between. My suspicion is that, in most cases, when other labs fail to replicate an experiment after credible attempts to do so, there is probably a real problem with the original study. Either (1) because the original results were faulty in some way or (2) because the original results, though valid, are not at all robust. Either way, the science community needs to know. So in my eyes, the drive for replication remains vital and the risks are well worth it.

A List of Things that Patients Should Question

I just learned about the Choosing Wisely campaign. It’s pretty amazing. The goal is to get each major medical specialty society in the US to make a list of 5 commonly done things that doctors and patients should question.

Being a patient is hard. People tell you that you are in large part responsible for your own care–but you are not a medical expert. When do you trust your doctor’s opinion? When do you need to do your own research? How many people are actually capable of doing this kind of research? The whole thing can be nerve-racking, especially if you are dealing with a serious health condition.

imgres-1The Lists for Choosing Wisely are still being written, but many are up on the website already. I wish they were being publicized more. I’d never heard of them until I happened on an article about the campaign in JAMA, and I doubt many other non-MDs are familiar with them either. I even asked a couple of MD acquaintances, and they hadn’t heard of these lists.

If you want to see them, you can check them out here. Unfortunately, they are not especially easy for patients to sort through. They may be helpful, though, especially if you know what you’re after. Take the American Academy of Pediatrics recommendations. Some are pretty well-known (no cough/cold meds for small children, no antibiotics for viral respiratory infections) but others might be less widely known (they all deal with common situations in which CT scans are not necessary–it was stuff I didn’t know).

Here’s a more user-friendly facet of the website: If you have questions about a particular condition/procedure, you may be able to find a fact sheet on it (look at the column on the left). Do you need a PAP smear? Maybe not! But you may have to convince your gynecologist of that. Are you thinking about scheduling an early delivery for your baby? Maybe not a great idea. But again, you might end up arguing with your OB about it. Need help controlling your migraines? Might want to avoid certain drugs. These fact sheets are being developed by Consumer Reports, so hopefully they will get disseminated widely. It’s a really nice idea!

One thing I like about this campaign is that it might give patients stronger footing when they decide to question a medical recommendation. Lots of times physicians recommend treatments that aren’t backed up by evidence, but patients are hesitant to speak up. Maybe this will give them (us) a little more confidence. It also may help people sort through the credible research and all the crazy stuff you find when you do a health-related internet search.

Can mice inherit learned fears from their fathers?

imgresVirginia Hughes wrote today about some exciting results presented on Tuesday at the Society for Neuroscience meeting in San Diego.

Here’s the setup: postdoc Brian Dias and colleagues used mild shocks to make male mice fear a certain scent. Then, they allowed the mice to mate. Their offspring, which had never been through the whole scent/shock thing, showed an increased startle response when they encountered the feared smell for the first time. And this was only true of the specific smell their fathers had been made to fear. Not only did the researchers report finding this inherited fear in the offspring of the shocked mice–they also found it in the grand-offspring! Neat, right? But also, really puzzling. How could information in the fathers’ brains be transmitted to their gonads and then to their offspring? The researchers admitted they have no idea what the mechanism is at this point, although I’m sure they have lots of plans to investigate just this.

With no mechanism in mind a priori, it kind of made me wonder how the team dreamed up this experiment in the first place. And with no biologically plausible mechanism to invoke, it is a little hard to believe that the results aren’t just some sort of fluke. There’s a great blog post by Kevin Mitchell, The trouble with epigenetics, that highlights some of the problems with similar existing epigenetic research (ie learned experience being passed on to offspring, with the mechanism being shaky/nonexistent).

Lots of things presented at meetings don’t make it past the hurdle of peer review. Or they are published in greatly changed form. There is nothing wrong with that–this is one of the most important functions of scientific meetings. Putting your stuff out there with your best interpretation of what it means, and hearing from other knowledgable people about their take on your data. I guess the fate of this fascinating dataset remains to be seen.

In the meantime, responses to the announcement of these findings are blowing up the twittersphere. Hughes set up a storify so you can check them out.

Using male circumcision to prevent HIV infection in Africa

01-circumcision paksA piece called AIDS prevention: Africa’s circumcision challenge just appeared in Nature. I thought it was a great update on where we stand as far as using circumcision to prevent HIV infections in Africa. At this point, four randomized controlled trials have shown that it reduces the odds of transmission for men. So big circumcision campaigns are now being rolled out. One of the big worries is that circumcision will change people’s sexual practices, lulling them into a false security so that they forgo condoms. Based on this article, it seems as though many of the men getting circumcised and their partners aren’t sure what protection is actually being offered. That’s not a good sign. Critics have also pointed out that if men were planning on using condoms, they probably wouldn’t sign up to be circumcised. If there is core population of guys who just aren’t going to use condoms no matter what, though, it seems like circumcision would be a good (though far from perfect) mitigation strategy.

Time will tell, but maybe this will be one more piece in the puzzle.

One thing I’ve heard a lot in debates about male circumcision in the US that kind of surprises me is that people don’t think the trials in Africa are relevant to men in the United States. That is, they don’t believe that circumcision will protect against HIV transmission here. I’m still not sure what the reasoning is. Yes, HIV is less prevalent here (well, depending on who your partners are). But for a given risky heterosexual interaction, is there a good reason to believe that the African RCT results wouldn’t transfer? I’d be curious to hear from someone who takes this position! The decision to circumcise obviously depends on lots of different factors, and I don’t think that protection from HIV is at the forefront of most new parents’ minds. That makes sense. But it does seem probable that circumcision would offer a little protection against HIV transmission.

The bogus HPV vaccine article that just won’t die

vaccinationI came across this article on Facebook today: Lead Developer of HPV Vaccines Comes Clean, Warns Parents & Young Girls It’s All A Giant Scam. It was published on some entertainment website called feelguide.com back in July, but it just won’t die. It’s got 198,000 Facebook likes, and it’s been tweeted 631 times. It claims that Diane Harper, a scientist involved in the clinical trials for Gardasil, one of the HPV vaccines, did a 180 and decided that the vaccine is no good. According to the article, she announced this abrupt change of face at the 4th International “Converence” on Vaccination in Reston, Virginia. She came clean to the audience so she “could sleep at night.”

The article conveniently makes it very difficult to distinguish between the (supposed) paraphrasing of what Harper actually said at that meeting and the interpolations of other people. It says scary things like “44 girls are officially known to have died from [HPV] vaccines.” Uh, really? Wouldn’t that death toll be all over the newspapers? Well, maybe not, since it’s not at all true.

You might wonder how this website can get away with printing things that are demonstrably false. Yesterday somebody pointed out to me the feelguide.com website’s disclaimer: “Feelguide.com contains published articles, speculation, assumptions, opinions as well as factual information. Information on this site may or may not be true and is not meant to be taken as fact.” And the author? Is he a vaccine expert? Nope, his name is Brent Lambert. As it happens, he is also Editor-In-Chief of this fabulous website, and you can reach him at his gmail address. Super professional.

Where did this article come from, you ask? Almost word for word, it was taken from an article that appeared on the website LifeWise in June. This article, in turn, seems to have drawn on a 2009 article in the Sunday Express by Lucy Johnston. (Note: The Sunday Express may sound respectable, but it’s actually a British tabloid.)  Their claims that Diane Harper said all this stuff were debunked back in 2009, the very week that they came out. Ben Goldacre of the Guardian talked to Diane Harper himself. In Harper’s words:

“I did not say that Cervarix was as deadly as cervical cancer. I did not say that Cervarix could be riskier or more deadly than cervical cancer. I did not say that Cervarix was controversial, I stated that Cervarix is not a ‘controversial drug’. I did not ‘hit out’ – I was contacted by the press for facts. And this was not an exclusive interview.”

The original article was promptly taken off the Sunday Express website, and Harper complained to the Press Complaints Commission.

How did this whole brouhaha start? For whatever reason, Harper decided to speak at the 4th International Public Conference on Vaccination, held by the National Vaccine Information Center in Reston Virginia. Sounds bland enough, right? But as it happens, the NVIC is one of the largest, most vocal anti-vaccine groups out there. Why would she attend such an event? I guess it’s possible that she was tricked, that she didn’t realize what she was getting into. Working in the vaccine field, it seems she would have to be familiar with the NVIC, though. Maybe she was trying to engage vaccine critics, hoping that a little education would bring them around. Perhaps we’ll never know. But not surprisingly, it appears that attendees twisted her words in the press.

So how did all the same 2009 tabloid junk get recycled in a 2013 article? And why do people take it at face value? Lord only knows.

I frequently see people post articles like this in places like Facebook after adding something like, “C’mon, people. Do your research. Vaccines are dangerous.” I am all for people doing research about vaccines. There is so much great vaccine research available that if most vaccine skeptics really delved into it, I think they would rapidly change their minds. But does anyone really consider reading an article like this research? Even if the lack of any citations didn’t clue you in, and you didn’t know about the backstory for this chunk of lies, wouldn’t the misspelled words, the disclaimer that says the website contains  information that “may or may not be true,” and the Editor-in-Chief’s gmail address give you reason to pause? Is this really where you want to get the information you use to make medical decisions? Feelguide.com? C’mon people. Do your research. For real.

If you’re interested, more information about this particular zombie anti-vaccine meme can be found on the Respectful Insolence and Skeptical Raptor blogs.

Another great article on the drawbacks of mammograms

1663_e449b9317dad920c0dd5ad0a2a2d5e49I posted a link to an article a few days ago that was critical of the Susan G. Komen Foundation’s emphasis on mammograms as a cure-all for breast cancer.

Here is a fantastic article on why news anchor Amy Robach’s post-mammogram double mastectomy might be sending the wrong message. It’s called An Inspiring, Misleading Tale About Breast-Cancer Screening and it appeared today in Nautilus.  Definitely worth a read!

How the media interprets studies of home vs hospital births: Do mothers matter?

mother-and-baby-201x300-1I thought the news coverage of a recent study on planned home births vs hospital births was really interesting. The article, Selected perinatal outcomes associated with planned home births in the United States, appeared in the October issue of AJOG.

I’ll come clean here–I have two kids, and they were both born in hospitals. My goal both times was to minimize intervention, but I never seriously considered a home birth. I know I don’t have a very high pain tolerance, and though I tried to get through labor without meds both times, I wanted to know that they would be an option. In the end, I got an epidural for each birth (and was extremely grateful for it). But the climbing C-section rates, the prison-like environment of the hospital… I understand the appeal of the home birth for many women. So I’ve been following home birth vs hospital birth safety studies with a lot of interest.

I’ll summarize the study’s findings and then review some of the media coverage, which I found a little surprising.

What the researchers did

Basically, the authors looked at two different types of outcomes: (1) neonatal outcomes, such as the 5 minute APGAR score for babies and whether babies had seizures and (2) maternal outcomes, such as operative vaginal delivery (i.e. forceps or vacuum used) and labor induction/augmentation. They analyzed over 2 million singleton births that occurred in 27 states in 2008, roughly 12,000 of which were home births. I think it’s fantastic they were able to look at only singleton births and that they were able to identify planned home births. That way, they aren’t counting emergency situations where women can’t make it to the hospital as “home births,” and riskier multiple births don’t enter the analysis, complicating things. In case you’re interested, here are some other birth types that were excluded from the dataset: breech deliveries, deliveries that were < 37 or > 43 weeks, and births at freestanding birthing centers. In other words, they were trying to focus on relatively low risk births.

Women who plan home births are a unique subset of mothers

The authors found that by virtually any measure, women who plan home births are different than those who do not. They are more likely to have given birth before. They are older. They are way more likely to be white. And married. They are more educated. They initiate prenatal care later. And their babies are born at a later gestational age. Obviously, when you are comparing an outcome in two groups of people that differ in so many ways, epidemiological studies are very, very tough to interpret. The authors tried to control for these differences as best they could by using multivariate models that adjusted for parity, maternal age, race/ethnicity, educational level, marital status, gestational age at delivery, smoking during pregnancy, prenatal visits, and medical conditions such as gestational diabetes and preeclampsia. However, residual confounding is always a problem. Sometimes a big one.

Planned home births are more likely to result in babies with an Apgar score of < 4 and babies who had seizures

Planned home births were roughly twice as likely to result in a baby with an Apgar score of < 4. A score of < 4 is pretty serious stuff–it is a very good indicator of neonatal death. However, the absolute number of such births was small in both home and hospital groups (0.37% for home births, 0.24% for hospital births). And babies from planned home births were roughly three times more likely to have seizures (although again absolute numbers were small: 0.06% vs 0.02%).

Babies born in planned home births are less likely to end up in the NICU

The authors found that babies born in the hospital were roughly five times more likely to end up in the NICU (remember that these are adjusted odds ratios, so gestational age and maternal complications are included in these calculations).

Planned home births are associated with significantly lower levels of intervention

This is probably no surprise to most people. Women who gave birth at home in a planned delivery had about a tenth the odds of operative vaginal delivery, a fifth the odds of labor induction, a third the odds of labor augmentation, and less than half the odds of antibiotic use.

So what’s the take home?

I think any epidemiologist will tell you that studies like this are really, really tough to interpret. There will  probably never be a randomized controlled trial of home vs hospital births. And the women who choose home births are different in many ways than women who do not, so residual confounding is always a problem. Plus, we don’t have a way at present to identify women who wanted to give birth at home but ended up in the hospital with complications… so this may actually result in underestimation of the risks associated with planned home birth.

This study had a lot of strengths, though. Relatively large sample size, relatively low-risk births being compared, adjustment for many potential confounders. Given the findings, what’s a pregnant woman to do? How do you weigh the greater (but still very unlikely) odds of neonatal complications with the lower (but much more common) odds of maternal interventions, which carry their own risks? There are some tough tradeoffs here. I was curious how the media would interpret this study.

The media spin: Ignore the mothers

Here’s how different news organizations interpreted the study.

The headline in Science News was Home births more risky than hospital deliveries. The reporter here decided to focus on neonatal outcomes without mentioning the other half of the paper: maternal outcomes. He did point out the low absolute occurrence rate of neonatal problems, though, which is great. I find that often absolute risk isn’t discussed in news articles, and it’s important information for readers trying to interpret health studies.

Same thing in coverage of the story on the New York Times blog, which was entitled Home births pose special risks. No discussion of the maternal outcome findings, and here they didn’t even mention the low absolute occurrence of neonatal complications.

The coverage in Medical Daily was even more alarmist: Home births linked to increased neonatal complications; Mothers should plan for emergency hospitalization. While planning for possible transport to the hospital certainly seems wise, including “plan for emergency hospitalization” in the title didn’t really seem to follow from this study’s findings.

So why did the media ignore the maternal half of the paper? Is it just the fact that home births involve a lower level of intervention is old news? Or do maternal risks not matter very much to the general public when babies are involved? I’d be curious to hear what other people think about this!