(Reuters Health)—A new study shows how fake news—specifically, information about scientific research that may be tainted by fraud—keeps getting spread through the medical literature, misleading doctors about the safety and effectiveness of the drugs they prescribe.
At issue is the failure of medical journals to flag research that’s been identified by the U.S. Food and Drug Administration (FDA) as potentially fraudulent, a task made difficult by the FDA itself.
The error is compounded when such studies are combined into so-called meta-analyses that perpetuate the error or fraud, according to a new analysis released March 4 online in JAMA Internal Medicine.1
It found that if studies with falsified data had been excluded, some of those meta-analyses would have painted a very different picture for one drug — the $1.6 billion-a-year clot-preventing medicine apixaban, which is sold by Pfizer and Bristol-Myers Squibb Co under the brand name Eliquis.
In nearly half the cases, if the flawed studies had been ignored, the conclusions of the meta-analyses would have changed and one third of them would have concluded that the drug didn’t work better than an older drug or a placebo.
“For the changes we saw, you would either see that the (benefit of the) product was no longer statistically significant or the odds went from showing a protective effect to a harmful effect,” chief author Craig Garmendia, PhD, of Florida International University tells Reuters Health in a telephone interview.
The analysis expands on a 2015 analysis by Charles Seife of New York University, also published in JAMA Internal Medicine, which concluded that FDA information about suspected fraud seldom makes its way into the medical journals that publish the results.
The new report “shows how a tainted study can affect the literature downstream, and that’s significant because these are things physicians base their guidelines on,” Mr. Seife says in a phone interview with Reuters Health. He was not involved in the new study.
“It demonstrates another consequence of the failure to alert the medical community and the public to research misconduct,” Robert Steinbrook, MD, an editor at large at JAMA Internal Medicine, tells Reuters Health by email.
Dr. Garmendia and colleagues focused on a series of clinical trials of Eliquis known as ARISTOTLE, which produced the largest number of studies with falsified data identified by Mr. Seife. The FDA found evidence that patient records at one treatment site in China were altered.
Nonetheless, the falsified data were included in seven apixaban studies, according to the 2015 report. The researchers doing those studies “published the same stuff over and over again without acknowledging there was a problem,” says Mr. Seife.
And despite the FDA action, one or more of those seven flawed studies was used in 22 meta-analyses evaluating Eliquis, Dr. Garmendia found.
In response to the new report, Pfizer and Bristol-Myers Squibb jointly told Reuters Health in an email that “using ARISTOTLE as an example in this analysis is misguided and may confuse prescribers and adversely impact treatment decisions for patients. The falsified data referenced within the JAMA article was isolated to one site, occurred after the completion of the study report and had no impact on the integrity of the results or conclusion of the study.”
(Mr. Seife says in his interview that the problem of falsified data was known before the ARISTOTLE studies were published.)
The Pfizer-BMS statement also says, “The FDA was made aware at the time and conducted their own sensitivity analysis which confirmed no impact from this isolated incident on the ARISTOTLE results. Moreover, a growing body of clinical and real world data have continued to reinforce the efficacy and safety profile of Apixaban.”
“Not all the data within the ARISTOTLE publications were falsified. However, because the researchers knowingly published falsified data, a form of research misconduct, we removed all ARISTOTLE data” when the re-analyses were done, says Dr. Garmendia.
“A lot of the literature that shows a benefit (from prescription drugs) might have to be taken with a grain of salt” based on these findings, says Mr. Seife.
Part of the problem is the FDA itself, as Seife noted in his 2015 paper. The FDA does not typically notify journals when an inspection uncovers problems at a medical center doing the research. Nor does the agency alert the public, and “the documents the agency discloses tend to be heavily redacted. As a result, it is usually very difficult, or even impossible, to determine which published clinical trials are implicated by the FDA’s allegations of research misconduct,” Mr. Seife wrote at the time.
Mr. Seife recommends that “the FDA should make unredacted information about its findings of research misconduct more readily available. The agency should make sure that any substantial evidence of misconduct is available to editors and readers of the scientific literature.”
“It is insufficient for the FDA or another regulatory agency to merely identify research misconduct,” says Dr. Steinbrook. “The study shows that a strategy to fully inform the research community and the public is also needed”
“There’s no system set up to allow medical journals to track this information,” says Dr. Garmendia. “So at the end of the day they’re still relying on the authors to disclose that they have been inspected and if data was thrown out. That just seems to not be taking place, whether it’s because they want a nice clean publication or they don’t want to admit to any mistakes were made.”
Mr. Seife also wrote in his 2015 paper that “there is no formal requirement for authors seeking to publish clinical trial data to disclose any adverse findings noted during FDA inspections. Journals should require that any such findings be disclosed.”
But it’s not clear that disclosing the information to the medical journals would solve the problem.
Four years after his paper named a host of potentially flawed papers, such as the ones that resulted from the ARISTOTLE study, Mr. Seife says he was unaware of any cases where the journals that published those studies have retracted them or alerted their readers to a potential problem.
Reference
- Garmendia CA, Nassar Gorra L, Rodriguez AL, et al. Evaluation of the inclusion of studies identified by the FDA as having falsified data in the results of meta-analyses: The example of the apixaban trials. JAMA Intern Med. 2019 Mar 4. [Epub ahead of print]