Are randomised controlled trials really the best evidence we have when making treatment recommendations for clinical practice?
The randomised controlled trial (RCT) has become synonymous with evidence-based medicine and the guideline movement which seeks to standardise and improve medical practice.
However, most of the trials that underpin evidence-based medicine are designed for registration of new drugs, and this brings with it some limitations.
At the end of the last century, Roland and Torgerson observed that “the highly controlled environment of randomised controlled trials designed to examine efficacy or safety does not reflect everyday practice”.1
For example, in asthma, it was estimated that very few people with asthma in the community, that is only about 5%, would be eligible for participation in the RCTs that formed the basis of the Global Initiative for Asthma guidelines. What this means is that while RCTs have high internal validity, they may not be generalisable to the next patient with asthma that you see in your clinic.
The late Canadian physician, Dr Dave Sackett, one of the gurus of evidence-based medicine, used to respond to this generalisability gap by saying: “Well that may be the case, but the RCT is still the best evidence you’ve got.” And that was true, at least at the time.
Since then, developments in trial design and electronic data capture mean that it is possible to both evaluate a therapy using a randomised design, and embed this in usual clinical care, using a pragmatic effectiveness trial (PET).
Recent examples of this are the Salford lung studies, one in asthma and one in COPD. The Salford Lung Study in Asthma found that a once daily ICS/LABA was more effective than usual care in achieving a clinically significant improvement in asthma control.2
In the UK town of Salford, the home of television soap opera Coronation Street,
74 primary care practices randomised 4233 asthma patients who were symptomatic and on maintenance asthma therapy to either usual care or once daily fluticasone furoate/vilanterol (FF/VI) via a dry powder inhaler. The study had minimal exclusion criteria and ran alongside, or inside actually, the primary care practices in Salford.
Study monitoring and safety data were collected using an integrated electronic medical record system that linked primary care, pharmacies and hospital records. The primary outcome was asthma control.
The results showed that, on average, patients randomised to FF/VI had twice the odds of achieving a clinically significant improvement in asthma control. There was no difference in asthma exacerbations between the groups, and safety outcomes were similar.
The study has many learnings for primary care and PETs, in general.
The diagnosis of asthma in primary care is controversial, and the goal of widespread use of spirometry for diagnosis has never taken off in primary care in Australia.
In the Salford study, patients were diagnosed using clinical criteria and the study validated this approach, at least for entry-level therapy. Similarly, key comorbidities and exposures, such as smoking, were permitted. These are usually excluded from RCTs. Given the relatively high prevalence of smoking among people with asthma, this makes the Salford study one of the largest trials of treatment of smokers with asthma.
Electronic data-capture systems were used which allowed for minimal intrusion into usual practice, and demonstrated the great value for primary care research of an integrated health record system.
Importantly, the range of outcomes in the trial allow for real-life estimates of what happens when asthma patients receive ICS/LABA. Since patients in both arms could receive an ICS/LABA combination therapy, it is the drivers of effectiveness that really are being evaluated.
These are the real-life issues that determine whether a patient will initiate and continue your prescribed therapy, and these are the very things that are “designed-out”, that is controlled for, when doing an efficacy RCT for drug registration purposes. They include things such as adherence, side-effects, patient preferences for treatment acceptability, drug efficacy, comorbidity, and persistence with treatment.
Clinicians and patients need PETs because they build on efficacy data and generate real-world information that is both valid and minimally biased.
We sort of know this, but tend to ignore it. For example, in asthma we know that many asthma patients fill only about a third of their prescribed controller-medication scripts. What is driving this behaviour, and how do we address it?
Studies such as Salford draw attention to the drivers of effectiveness and ask us to consider them in clinical practice. The study also raises an interesting question for guideline developers. When making a treatment recommendation for clinical practice, is an efficacy RCT really the best evidence, or should we be asking for data that is both internally valid and externally generalisable?
It’s a message that is relevant across many therapeutic areas.
Our treatment decisions depend upon more than just estimates of efficacy in highly selected populations. PETs will lead to better guideline recommendations.
This is necessary, not only to inform the doctor who is prescribing a medication, but also to give patients realistic estimates of likely outcomes, and allow the doctor and patient to focus on the issues that determine the success (or not) of a treatment program.
Maybe it’s time RCTs had more PETs.
Conjoint Professor Peter G Gibson is a respiratory physician and clinical scientist studying the disease mechanisms and treatment of severe asthma, COPD and other airway disorders. He works with the University of Newcastle, the Hunter Medical Research Institute and the John Hunter Hospital
References
1. Roland M, Torgerson DJ. What are pragmatic trials? BMJ. 1998 Jan 24;316(7127):285.
2. Woodcock A, Vestbo J, Bakerly ND, New J, Gibson JM, McCorkindale S, Jones R, Collier S, Lay-Flurrie J, Frith L, Jacques L, Fletcher JL, Harvey C, Svedsater H, Leather D; Salford Lung Study Investigators. Effectiveness of fluticasone furoate plus vilanterol on asthma control in clinical practice: an open-label, parallel group, randomised controlled trial. Lancet. 2017 Nov 18;390(10109):2247-2255. doi: 10.1016/S0140-6736(17)32397-8.
