Tuesday, July 22, 2008

ADHD treatment: Placebo versus….placebo?

JAMA just published a study of St. John’s wort for ADHD. Pubmed18544723.

This made a lot of headlines. For example,
“Herbal Remedy No Better Than Placebo in Treating ADHD” [Psychiatric News]
“St. John's Wort No Help for ADHD” [TIME magazine]
“ATTENTION-DEFICIT HYPERACTIVITY DISORDER: St. John's wort may not affect symptoms” [Washington Post]

I spotted this article as I was looking at the recent publishing activities of Dr. Joseph Biederman, who has been investigated lately for either poor memory, or not really knowing how to fill out forms for his employer, Harvard.


This poor performance in filling out forms has drawn the attention of a senate investigating committee, and press in the New York Times, and none-too-soon. We can’t have Harvard researchers out there failing to properly fill out routine forms. It has been quite noble of Dr. Biederman to acknowledge that there were some problems filling out forms, and note that this should all be straightened out tout-suite. His employer has been supportive.

I can understand all of this. Dr. Biederman publishes well over ten peer-reviewed articles per year, and has maintained this steady rate for over a decade. Much of this has focused on ADHD, AKA “hyperactivity” and ADD.

It seems that some people have suspected that an herbal remedy, St. John’s wort, may be useful in treating ADHD. So, Dr. Biederman being a leading authority in ADHD diagnosis and treatment, along with his colleagues, ought to evaluate whether this herbal remedy actually does provide some relief.

Not to mentioned Dr. Biederman’s other interest in ADHD: he profits greatly from his involvement with the pharmaceutical companies who sell the FDA-approved drugs for ADHD. Specifically, ALZA corporation makes Concerta, a sustained-release Ritalin that is under patent for a few more years. ALZA supports Dr. Biederman, per the disclosures at the end of this article on St. John’s wort and ADHD.

OK – segue back to the article: Well, I had not heard of St. John’s wort as a possible treatment for ADHD. So I decided to look at this article.

A bunch of kids were screened for ADHD – not difficult to find nowadays since everyone has ADHD – you just need to exclude the kids that also have bipolar and autism. Then, randomized to receive either “treatment” or “placebo.” “Treatment” was hypericum, a component of St. John’s wort that is believed to be the active ingredient, or at least one of the active ingredients, in St. John’s wort.

So, the kids receive ADHD measurements at “baseline,” then every couple of weeks up to eight weeks. Ideally, if the herb has some effect, average ADHD scores will be lower for the hypericum group compared to the placebo group.

Well, Dr. Biederman, your financial status is safe. Hypericum did no better than placebo. Actually, the placebo group did a bit better than the hypericum group. The herbalists will not be replacing the pharmacists anytime soon.

OK, so I am done looking at the picture that tells me the main result, and I am on to the Discussion. Almost done gathering this bit of knowledge.

Wait. What is that in the discussion?

I have to quote this directly. This literary passage is too good to not share.

“Hyperforin is a very unstable constituent that quickly oxidizes and then becomes inactive, which is likely what happened to the product used in this clinical trial.”


Haven’t I heard that before?

Oh, yes: in the definition of “placebo.” A placebo is an inactive formulation, per Wikipedia and as per the rest of the knowledgeable world.

So, JAMA publishes a study that declares that St. John’s wort is not effective for ADHD, but the active arm of the randomized study received a formulation that was inactive, just as the inactive arm did?

What would the result be if the study was repeated with hypericum which had not gone stale?

We don’t know. The hypothesis is still totally untested.

However, the drug reps that are out there for ALZA are now able to declare, when it is worth mentioning, that “St. John’s wort was not any better than placebo, in a recent JAMA study.”

I am gonna wrap up this part with a couple comments: I believe there is such a thing as ADHD. I belive it exists at a much lower prevalence that Dr. Biederman might believe. But I believe it exists. I also believe that there are medications that can be helpful. I also believe that any child who has ADHD will need top-notch parenting, and will need skilled teachers. ADHD may make a kid distractible, leading to poor school performance, etc. A pill can help some of this. But a kid with ADHD still needs to learn all of the things that any other kid has to learn. To follow through. To figure out how to understand and complete assignments. To cooperate with others. Et cetera. ADHD interferes with the acquisition of these qualities. These learned qualities cannot be delivered by pill. They can only be delivered by people and the right circumstances. Usually, family and school teachers. School teachers get training and experience dealing with this sort of thing. But any given parent does not have some special place to learn how to be a top-notch parent for a kid with a demanding challenge. Whether an herb or some other pill works, it will be the social dimensions of response that will help get the kids with ADHD ready for the real world. The parents will need some training - perhaps from a therapist.

OK: You can return to your regular web-browsing.

If you want to read a bit further, I am gonna throw in a couple more thoughts and observations.

Would it have been that much more difficult to recruit another 60 kids, and run it all again with ACTIVE ingredient?

At least the authors note that the hypericum can easily become inactive. It gets stale, basically. Moisture from the air leads to processes that result in the breakdown of this molecule. They get credit for noting this. Nonetheless, it is sad that a leading medical journal chose to run this story as is. That means that the peer reviewers were OK with this (or that some editor over-rode the objections of peer reviewers).

Another acknowledged weakness is the sample size. Each group had 27 participants. Considering the heterogeneity of ADHD, the variability in ADHD scores generally, and the use of multiple tests (two subscales of RDS and two of CBCL = 4 tests), this study probably was under-powered. If you don’t understand all of that, don’t worry. Bottom line is they probably did not include enough study subjects to detect a clinically significant effect for St. John’s wort if, hypothetically, it actually works (which would apparently require using fresh, not stale, St. John’s wort).

One odd thing I noticed that was not acknowledged was the odd pattern of results on CBCL. The Child Behavior Checklist is a checklist of a bunch of problem behaviors that kids might perform. This is a widely recognized instrument for measuring child behavior, and a decent general measure of ADHD problems, although it is not specific to ADHD.

While the study may have been underpowered to have a real-world meaningful difference also show up as a statistically meaningful difference, the pattern of results on the CBCL show that the placebo worked better than the St. John’s wort. How could that be?

The CBCL scores are “standard scores:” a score is translated into a standard scale that is more easily interpreted and understood: this is the same as saying that the Farenheit scale can be translated in to a standard scale of Celcius, with freezing at zero and boiling at 100. It is just more interpretable. For the CBCL, the standard deviation is 15. The placebo group had their scores, on average, drop about 15 points, or one standard deviation, on both the “internalizing” and the “externalizing” scales of the CBCL, while the scores of the St. John’s wort group dropped only 4 points on each of these two parts of the CBCL.

Why would the placebo group have a consistently decent, even admirable, therapeutic response to placebo, but the herb group does not? The herb group should be benefitting from any placebo effect to about the same degree as the placebo group. The placebo group scores on the ADHD symptom scale also followed this pattern, with the placebo group experiencing lower scores, thus more improvement, than the herb group, although the actual differences are probably far from being at clinically meaningful difference.

Frankly, that pattern makes it plausible that something else was going on.

Here is my guess: as the authors note, the kids were randomized, but the randomization did not succeed, as it typically does, in spreading various factors out equally between the two groups. The authors go no further. However, my eye gets caught on the assignment of gender: for the placebo group, half were males, while for the herb group, three-fourths were males. So, maybe females happened to have more response just to the burst of clinical attention compared to guys. Or maybe females have more sensitivity to hypericum, possibly through some sex-based difference in metabolism. The authors note that they controlled for sex statistically, but this is really a difference that can only be controlled by a greater sample size: first, the study certainly was not powered to determine if my pet hypothesis is supported: that the treatment, whether placebo or herb, had a greater effect on females than males, and second, with a greater sample size, the assignment would increasingly get closer to an even match of boys and girls in each of the two groups.

Again, as I have said already: JAMA published this? The leading medical journal?

Hmmm. Pharmedout reports that a third of JAMA pages are advertisements.

JAMA has been heavy on publishing studies noting the influence of direct-to-consumer drug advertising upon prescribing habits. Maybe it is time that JAMA evaluate the influence of drug advertising in JAMA upon editorial decision-making. Cuz a ‘conflict-of-interest’ type explanation would sure fit this pattern of results.

No comments: