Communities of practice
The ongoing debate over R&D output
10 Jan 2011
“What happens after a paper is published is as important as what occurred before it was published.” That is one of the core concepts of the journal PLoS ONE, an on-line, open-access resource published by the non-profit Public Library of Science. And it was in evidence last July, when the authors of a pair of peer-reviewed articles on research and development (R&D) output over the past several decades engaged in a post-publication debate through comments posted on the PLoS ONE site.
The debate centres around the work of a team led by the noted pharmacist and former WHO-EURO Health Advisor Patrice Trouiller, whose landmark 2002 analysis of R&D output since 1975 was the first to attempt a comprehensive accounting of drug development for neglected diseases, in order to fully document – in dollars and in drugs approved – the failure of the market to address priority global health needs .
By conducting searches of MEDLINE and databases of the US Food and Drug Administration (FDA) and the European Agency for the Evaluation of Medicinal Products, Trouiller et al. found that “of the 1,393 new chemical entities marketed between 1975 and 1999, only 16 were for tropical diseases and tuberculosis.”
The authors went on to describe how that imbalance was reflected in the overall level of pharmaceutical investments for research and development. “Of the $35.3 billion invested in 1999, 10.1% was spent on infectious diseases,” they wrote. “By contrast, estimates suggest that the total investment (public and private sector) in drug research for malaria, tuberculosis, leishmaniasis and African trypanosomiasis was less than $70 million.”
That paper, “Drug development for neglected diseases: a deficient market and a public health policy failure”, reverberated throughout the public health community, serving as a call to action to ramp up investment in drug discovery. It also provided policymakers and advocates of change with a concrete figure – 16 new chemical entities – published in no less an authority than the Lancet. In the eight years since the paper first appeared, it has been cited 183 times.
One of the most recent articles to cite Trouiller et al. did so in the course of revisiting the group’s 2002 estimates, while also looking at trends in funding and R&D output in the decade since.
In their paper in PLoS ONE last May , Joshua Cohen and colleagues at the Tufts Center for the Study of Drug Development (CSDD) at Tufts University, Boston, USA argue that Trouiller et al. appear to have undercounted the total number of drugs approved for NTDs and tuberculosis between 1975 and 1999.
Using a “G-FINDER definition” of neglected diseases, a reference to the George Institute’s global R&D investment survey, Cohen et al. identified 46 new products approved between 1975 and 1999, with a total of 56 indications. And where Trouiller et al. counted 16 marketed new chemical entities (NCEs), Cohen et al. count 32, doubling the previous estimate, despite employing what they claim to be the same methodology.
“Moreover,” they add, “the Trouiller et al. list is inaccurate as five of the 16 drugs were not approved between 1975 and 1999.” These include pyrazinamide, atovaquone + proguanil, benznidazole, nifurtimox, and pentamidine.
In conclusion, write Cohen et al., “lopsided progress” – a 250% increase in new products for malaria from 2000–2009 compared with the 1975–1999 period versus zero new drugs approved for tuberculosis in the last nine years, despite similar funding – “suggests that the infusion of more money itself is insufficient.”
Rather, they say, what is needed is a “balanced comprehensive approach” combining drug development with improved access to new drugs through greater infrastructure and capacity building.
Anyone interested in the topic might well have assumed that the PLoS ONE paper by Cohen et al. was the last word in the debate. But in fact, weeks after it was published, Pierre Chirac, a public health pharmacist, and Piero Olliaro, leader of drug development and evaluation for helminths and other neglected tropical diseases at WHO/TDR, posted a rebuttal to Cohen et al. in the form of a “comment” on the PLoS ONE site. (Olliaro was one of the authors of the original Trouiller et al. article.)
In that comment, Chirac and Olliaro refute the claim by Cohen et al. that, by using the same methodology as Trouiller et al., the Tufts researchers counted twice as many NCEs in the 1975–1999 period. Instead, they attribute the difference to a discrepancy in definitions and a complete lack of search parameters.
“What exactly is a ‘new drug’?” they write. “In regulatory jargon, a new drug is either a new chemical entity that contains an active substance receiving regulatory approval for the first time, or a new indication for a product that has already been approved.”
In total, Cohen et al counted 21 NCEs that Trouiller et al. did not – eight of which, write Chirac and Olliaro, “are fixed-dose combinations of drugs already in use both individually and concomitantly.” As fixed-dose combinations of already-approved drugs previously used together as loose combinations, they add, these are not, by definition, NCEs.
Among the remaining 12 drugs that Cohen et al. included and Trouiller et al. excluded is the anti-malarial drug amodiaquine. The Tufts team considered amodiaquine to have been approved in Kenya in 1998. However, as Chirac and Olliaro point out, the drug “had already long been on the market as Flavoquine® and Camoquin® since the 1950s.”
Cohen acknowledged in an e-mail that with regard to amodiaquine, Chirac is correct. “We went through a painstaking process of trying to find precise dates and origins of initial registrations wherever we could,” he said. “However, with drugs targeting neglected diseases, this proved to be a very difficult process.”
But amodiaquine wasn’t the only inaccuracy on the Tufts list. Four drugs targeting intestinal helminths were similarly mistaken for having been approved more recently than is, in fact, the case: “Cohen et al. were probably unaware that [these] received market authorization before 1975,” write Chirac and Olliaro. “Levamisole as Solaskil® in 1974 in France; mebendazole as Vermox® in 1974 in the US; pyrantel as Combantrin® in 1973 in France; and niclosamide as Tredemine® in 1964 in France.”
There was also the antimalarial, artemisinin, which Cohen et al. considered to have been launched in France in 1996. In fact, write Chirac and Olliaro, “this moiety [a part of a molecule that may include functional groups of atoms] is not marketed as such but only as a derivative (e.g. artesunate, artemether etc. which are already counted).”
And lastly five drugs targeting tuberculosis: kanamycin, amikacin, ofloxacin, ciprofloxacin, and moxifloxacin. Some of these drugs may have indications against multi-drug resistant tuberculosis – and moxifloxacin, for one, is currently under development for newly-diagnosed TB. But as Chirac and Olliaro explain, Trouiller et al. excluded them because they were first launched for indications other than for anti-TB drugs.
Though Cohen et al. don’t dispute that this is so, they take issue, they say, with the rationale behind the drugs’ exclusion. “The point is not, and should never be, what a drug is initially approved for, or whom the sponsor originally had in mind when it developed the drug,” he says. “It should be about which approved uses (now) target neglected diseases, when this happened, where, and whether WHO or other agencies view the drug or new indication as part of the neglected diseases armamentarium.”
Asked if he considers it fair to equate the incidental discovery of a new drug with the targeted development of an NCE, Cohen says that, while not “incidental,” the discovery of a supplemental indication for a neglected disease can indeed be equated with the successful development of an NCE targeting a neglected disease. “A lot of work, i.e. research and development, goes into all supplemental indications,” he says. “Almost as much as the initial or original indication.”
Nonetheless, he adds, “It was and is troubling that so few drugs and indications have been developed targeting neglected diseases.” And despite a widespread perception of the paper as a critique of Trouiller et al., he says, “we did not make a conscious decision to revisit Trouiller’s numbers.
“We simply used his study as a benchmark. Yet when we found discrepancies, as Chirac also noted (drugs approved prior to 1975 that were included on the 1975–1999 list), we decided to do a recount. We did not intend to put the spotlight on an otherwise brilliant piece by Trouiller et al.”
The result, however, was just that. In a paper of less than 2,700 words in total length, the term “Trouiller et al.” is mentioned 18 times (not including the three mentions in the abstract). That may be one reason it ranks higher in Google search results than the Trouiller et al. paper itself – this despite having never been cited by another peer-reviewed publication, existing as it has for less than one year. Likewise, the Cohen et al. paper ranks second in a search of “Trouiller et al.” on PubMed. Intentional or not, few spotlights could be brighter.
Yet amid the back-and-forth over R&D output, the two groups did find a bit of common ground, even if it doesn’t have any direct bearing on the issue itself. Namely, that what happens after a paper is published is as important as what happened before – and that PLoS ONE, for one, can serve as a valuable forum for debate.
However, while the papers by Trouiller et al. and by Cohen et al. are indexed in the medical literature and may readily be found using PubMed, the ensuing debate is “lost” in the PLoS ONE comments system, which is not searchable. “I think the follow-up discussion could be indexed, indeed, with some filter to ensure the integrity of indexed comments,” says Cohen. PLoS ONE managing editor Peter Binfield agrees. “PLoS does promote the use of commenting,” he says. “But it is true that our functionality isn’t as good as it could be.” Comments like Chirac’s and Cohen’s are easily missed – hidden, as they are, under one of four tabs, available exclusively to the online reader and unsearchable on the PLoS ONE site by author, title or keyword.
“We are aware of the problems and we are exploring possible solutions,” says Binfield, who came to PLoS from SAGE Publications in California, where he ran the publisher’s successful US Journals Division, including some 220 medical and social sciences journals. “For example, we have solved a similar problem with our Formal Corrections, which are clearly highlighted on the article, transmitted to PubMed and PMC, and are searchable at those sites (if not at ours yet). We’re looking into similar solutions for important Comments of the type [by Cohen and Chirac].”
In a last comment for this article, TDR’s Olliaro reported that an updated assessment of R&D output through 2010 is currently in the works. “Progress has been slow due to competing priorities,” he says. “But we will adhere to very clear and precise methodologies.”
1. Trouiller P, Olliaro P, Torreele E, Orbinski J, Laing R, Ford N (2002). Drug development for neglected diseases: a deficient market and a public-health policy failure. Lancet; 359: 2188-2194. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12090998
2. Cohen J, Dibner MS, Wilson A (2010). Development of and access to products for neglected diseases. PLoS ONE; 5(5): e10610. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20485552
Is your organisation working against the infectious diseases of poverty?