Rare diseases: no reason for lower demands for studies

On behalf of the Federal Ministry of Health (BMG), the German Institute for Quality and Efficiency in Health Care (IQWiG) examined whether specific methodological aspects have to be considered in the conduct, analysis and assessment of the certainty of results of studies on rare diseases. Within the framework of the same commission, the Institute also analysed the underlying studies for the approval of so-called orphan drugs, i.e. drugs for rare diseases, in Europe.

The result: For a different approach than in more common diseases, there are neither scientific reasons nor specific designs and methods that would not also be relevant for more common diseases. This is the case for both drug and non-drug treatments. If compromises have to be made with regard to the reliability of the conclusions because of a particularly small number of participants, raising the significance level is preferable to a limitation of the external or even internal validity of the studies.

Focus on rare diseases

In the European Union diseases are classified as rare if they occur in not more than 5 per 10 000 inhabitants. Very rare diseases are diseases that affect fewer than 2 per 100 000 inhabitants. Of approximately 30 000 known diseases, 7000 to 8000 are considered rare, so that in Germany alone up to 4 million people are affected.

The German National Action League for People with Rare Diseases (NAMSE), founded in 2010, has developed a catalogue of policy proposals, which, among other things, includes the development of criteria for the assessment and analysis of studies with few participants. Against this background, the BMG awarded a two-part commission to IQWiG, which has now published the results in a so-called rapid report. Its contents follow the rapid report on the evidence in guidelines on rare diseases, which IQWiG produced in 2011, also on behalf of the BMG.

RCT as the gold standard

As explained in the IQWiG report, there are four key components that primarily determine the reliability of the conclusions of a study: the internal validity (i.e. the risk of bias), external validity (i.e. applicability), effect size, and precision of the results. The risk of bias can be minimized by randomization, double-blinding, and the so-called intention-to-treat principle, i.e. within the framework of randomized controlled trials (RCTs).

Study sponsors occasionally argue that the methodological standards that are usually applied in clinical studies should be lowered for rare diseases. They justify this by claiming that RCTs are difficult to conduct due to the low number of participants, and that they are also ethically doubtful if effective comparator therapies are lacking. However, IQWiG reached a different conclusion: A low number of participants is equally problematic in all study types, and controlled studies are only ethically doubtful if the benefit or added benefit of the intervention under investigation is more or less proven already. However, further studies to clarify the question on benefit are then unnecessary anyway – not only in rare diseases.

Raising the significance level, if essential

Just like reduced internal validity, which increases the risk of bias, concessions on external validity can also lead to systematic errors, which cannot be adjusted for even with the most sophisticated statistical method.

Small effect sizes can only be determined with certainty by sufficiently large sample sizes in studies. Supraregional and international networks as well as disease registries that are as complete as possible are useful to collect a sufficient number of evaluable cases. The use of efficient statistical methods, e.g. sequential study designs, may allow for smaller sample sizes without affecting internal or external validity.

If this is not sufficient in the case of very rare diseases, IQWiG recommends compromising on precision: raising the significance level, e.g. from the usual five to ten percent, to obtain statistically significant results despite a small sample size. This makes it possible to at least quantify the probability of error: The uncertainty of a decision is known.

Empirical data support methodological expertise

For the second goal of the commission, IQWiG examined all 85 drug approvals for rare and very rare diseases in Europe between 2001 and 2013. 82 out of the 125 studies submitted within the framework of these approvals were RCTs, which clearly shows that it is not necessary to compromise on methodology. Most studies were multicentre, multinational, and multicontinental. Specific statistical methods aiming for high efficiency were used in about two thirds of the studies.

All patients have a right to quality

“It has been shown that in studies on rare diseases, deviation from the methods usually applied in studies is neither necessary nor possible without compromising on quality”, concludes the Institute’s Director Jürgen Windeler. “In most cases, reservations concerning RCTs in rare diseases are unwarranted.”

The EU regulation No. 141/2000 already stipulated that patients suffering from rare conditions should be entitled to the same drug quality, safety and efficacy as other patients and that orphan drugs should therefore be subjected to the normal evaluation process. “The best way to do justice to these requirements is to conduct multicentre RCTs, in which the risk of bias is minimized and the data recorded are carefully analysed”, says Windeler. “This makes it possible to estimate with sufficient certainty the benefit and harm of the intervention for the people affected, despite the small numbers of participants.”

Process of report production

The BMG commissioned IQWiG in December 2013. The report was to be prepared in an accelerated procedure as a “rapid report”. In contrast to the usual procedure, no preliminary reports are published here. Although a draft version of the report is reviewed by external experts, no hearing at which all interested parties can comment takes place. The report was sent to the commissioning agency on 8 September 2014.

An overview of the background, methods and further results of the report is provided in the following executive summary.