Things that used to be on a blog.

Burzynski's "research" – Part 1

July 21, 2012

Cancer quack Stanislaw Burzynski has 61 clinical trials listed on the register at http://clinicaltrials.gov/ct2/results?term=burzynski. Here is an analysis of the progress so far:

His supporters love to point to evidence of his success and can become quite abusive when anybody questions them. One piece of evidence they continually offer is the fact that he has a Phase 3 trial listed, but unfortunately that's not really evidence of anything as it's the trial that hasn't even started recruiting subjects yet. My guess is that Burzynski has no intention of ever starting that trial and it is just there for advertising purposes. No results have been published for the single completed trial, although it was supposedly finished in 2006.

Over the last two weeks his supporters have been calling all his detractors liars for saying that he doesn't publish anything and therefore there is no evidence of him curing anybody of anything. Oh yes, there are countless anecdotes and many web sites begging for money to send terminally ill children to Texas for a miracle cure. As Burzynski has been using web sites begging for money to save sick kids as an advertising gimmick for years not a lot has changed. What we want to see are papers in reputable peer-reviewed journals reporting on the results of double-blind randomised trials. You know, the sort of stuff we expect everyone else to do.

Why we are liars is because he has actually published something and I have been given links to it on at least a dozen occasions so I assume it's the best they've got. (One surprising thing is that some of his supporters seem to think that PubMed is a medical journal. It's an index to publications, so saying "He's been published in PubMed" is like saying "He must be legitimate. He's in the phone book".) Here it is.

So let's summarise that.

• Publication in high-impact journals is always better. The journal Integrative Cancer Therapies can be expected to be sympathetic to outlandish alternative medicine claims. In 2011 it had an Impact Factor of 2.136 and was cited 687 times. By contrast, the Journal of Clinical Oncology had an Impact Factor of 18.970 and gets cited about 114,000 times each year. (Nature has an IF of 36.280. That's where the Nobel Prize stuff gets published.)
• It is not normal practice to divide the abstract up into the sections that are found in the main paper. I was taught that in my first year at university.
• The paper was published in 2006, which is a long time ago in "cutting edge" medicine.
• This is not a paper giving the results of a double-blind, randomised clinical trial.
• It includes subjects from four trials, although which trials they are is not stated. It is not normal practice to aggregate trial data unless explicitly doing a meta-analysis and if that was the case the presentation of results would be completely different to what is given here.
• There were only 18 patients in total across all four trials. I remember having sample size beaten into my head in my first year studying statistics and research methods.
• "Fourteen patients had diffuse intrinsic tumors". What did the other four have?
• "Twelve patients suffered from recurrence" so only six improved (and Burzynski counts not getting worse as an improvement).
• "6 patients did not have radiation therapy or chemotherapy" so twelve did, simultaneously with Burzynski's treatment. (Many patients have to have had this sort of treatment before. Burzynski only accepts patients when all other options seem closed off. Desperate people are more ready to hand over money.)
• There is nothing to say if the twelve patients who didn't get better were the ones having radiation or chemotherapy, but if they were then the maximum number of patients who could have "improved" with Burzynski's treatment alone is six. I suspect it was fewer.

More numbers:

But wait -

By remarkable coincidence, those numbers add up to 100%, despite the fact that the rows are not independent (each figure includes the one on the row below it). Also, as everyone in the 2-year survival group must have made it beyond six months this seems to indicate that nobody ever gets worse after six months. This seems highly unlikely. The numbers are also inconsistent with the numbers in the first table – surely "Complete response" should not be only half of "5-year survival".

If this is the best research that Burzynski can come up with then I would hate to see the worst. In summary, a cobbled-together paper published in a low-impact journal six years ago which aggregates the results of four non-blinded trials with a small total number of subjects and which includes suspicious statistics is not about to get anyone the trip to Sweden that a cure for cancer would guarantee. If I had to rewrite the conclusion section of the paper it would say something like this:

CONCLUSION: We tried to find evidence that the treatments we have been selling at very high price for many years actually have an effect on the progression of certain forms of cancer. None of the clinical trials (that we pretend to do to get around the FDA's ruling about unproven treatments) were good enough on their own to show efficacy so we cherry-picked some of them, put them together as if they were independent studies suitable for meta-analysis and then added some numbers to make things look good. We reported the numbers in two inconsistent ways, hoping that nobody would notice that we didn't actually find any evidence of a cure. We even included an outlier to make the numbers look better, but we know that most people citing this paper will not know what that means or why it might be a problem.