Quality and Reporting of Diagnostic Accuracy Studies in TB, HIV and Malaria: Evaluation Using QUADAS and STARD Standards

24 Nov 2009

Paul Chinnock

Source: PLoS ONE (see original article or PDF)

Comments | Related Articles

Citation: Fontela PS, Pant Pai N, Schiller I, Dendukuri N, Ramsay A, et al. (2009) Quality and Reporting of Diagnostic Accuracy Studies in TB, HIV and Malaria: Evaluation Using QUADAS and STARD Standards. PLoS ONE 4(11): e7753.

One of the most encouraging recent developments in global efforts to control malaria has been the appearance on the market of rapid diagnostic tests (RDTs) that can be used on the front line of care. Such tests have already been available for HIV infection for some years and rapid tests for tuberculosis are also now under development. How confident can we be in the accuracy of these tests?

Policy makers and health professionals need to make choices as to which tests are best suited to their patient populations and healthcare delivery systems and, to this end, they often use published diagnostic accuracy studies as a way of gathering evidence about the options available to them. But this raises a further question – how confident can they be in the studies that have been conducted? A systematic review published in PLoS ONE raises doubts on this issue.

The quality of diagnostic studies may be assessed using two tools that came into use in 2003: STARD (STAndards for the Reporting of Diagnostic accuracy studies) and QUADAS (Quality Assessment of Studies of Diagnostic Accuracy). STARD was developed to improve the quality of reporting of diagnostic accuracy studies in general, and QUADAS was designed to be used in systematic reviews to evaluate the quality of primary diagnostic accuracy studies. The two tools provide templates that should be used to ensure good quality in the reporting of studies.

Researchers based in Canada and at TDR (Special Programme for Research and Training in Tropical Diseases) in Geneva conducted a systematic review – and made use STARD and QUADAS – to assess the quality of diagnostic accuracy studies of commercial tests for TB, malaria and HIV published in the period 2004–06.

They found 90 articles that met their inclusion criteria, all of which were found to have design deficiencies. The use of STARD was not explicitly mentioned in any study. (Only 22% of 46 journals that published the studies required authors to use STARD.) Few of the studies included adequate information on such issues as withdrawals and reference test execution, absence of index test review bias and reference test review bias, and uninterpretable results. Less than 25% of the studies reported on nine key items required for quality assessment in STARD.

Of particular concern was the failure in most published studies to report patient selection criteria, recruitment methods or blinded test interpretation. The authors conclude that recently published diagnostic accuracy studies on commercial tests for TB, malaria and HIV have moderate to low quality and are poorly reported. They urge the more frequent use of tools such as QUADAS and STARD to improve the methodological and reporting quality of future diagnostic accuracy studies in infectious diseases.

Dr Madhu Pai, the senior author and principal investigator of the study has said in a press release that: “Poorly designed studies can lead to premature or misguided adoption of tests that may have little or no clinical and public health relevance, resulting in incorrect diagnosis and adverse consequences for the patient ... Whether it is for cancer testing, TB or even the flu, we must report study results in a clear and transparent manner in order to validate the accuracy of the test and ensure it is properly used at the clinical level.”

One can only agree with Dr Pai’s statement. The findings of his team’s study are of great concern, given the considerable efforts now being expended in stepping up the use rapid diagnostic tests for malaria, particularly in Africa.

2009 Fontela et al

Source

Title:
Quality and reporting of diagnostic accuracy studies in TB, HIV and malaria: evaluation using QUADAS and STARD standards.

Authors:
Fontela PS, Pant Pai N, Schiller I, Dendukuri N, Ramsay A, Pai M

References:
PLoS ONE 2009: Vol. 4 no. 11, pp. e7753

Abstract:
Poor methodological quality and reporting are known concerns with diagnostic accuracy studies. In 2003, the QUADAS tool and the STARD standards were published for evaluating the quality and improving the reporting of diagnostic studies, respectively. However, it is unclear whether these tools have been applied to diagnostic studies of infectious diseases. We performed a systematic review on the methodological and reporting quality of diagnostic studies in TB, malaria and HIV.We identified diagnostic accuracy studies of commercial tests for TB, malaria and HIV through a systematic search of the literature using PubMed and EMBASE (2004-2006). Original studies that reported sensitivity and specificity data were included. Two reviewers independently extracted data on study characteristics and diagnostic accuracy, and used QUADAS and STARD to evaluate the quality of methods and reporting, respectively.Ninety (38%) of 238 articles met inclusion criteria. All studies had design deficiencies. Study quality indicators that were met in less than 25% of the studies included adequate description of withdrawals (6%) and reference test execution (10%), absence of index test review bias (19%) and reference test review bias (24%), and report of uninterpretable results (22%). In terms of quality of reporting, 9 STARD indicators were reported in less than 25% of the studies: methods for calculation and estimates of reproducibility (0%), adverse effects of the diagnostic tests (1%), estimates of diagnostic accuracy between subgroups (10%), distribution of severity of disease/other diagnoses (11%), number of eligible patients who did not participate in the study (14%), blinding of the test readers (16%), and description of the team executing the test and management of indeterminate/outlier results (both 17%). The use of STARD was not explicitly mentioned in any study. Only 22% of 46 journals that published the studies included in this review required authors to use STARD.Recently published diagnostic accuracy studies on commercial tests for TB, malaria and HIV have moderate to low quality and are poorly reported. The more frequent use of tools such as QUADAS and STARD may be necessary to improve the methodological and reporting quality of future diagnostic accuracy studies in infectious diseases.

PMID: 19915664
Full text: HTML

Comments | Related Articles