Clinical relevance of patient-reported outcomes: new threshold proven feasible in practice

In order to show the clinical relevance of a difference between two treatment alternatives, in recent years, the manufacturer dossiers submitted in early benefit assessments of new drugs have increasingly contained responder analyses for patient-relevant outcomes. In such analyses, it is investigated whether the proportion of patients experiencing a noticeable change in the respective outcome differs between the two treatment groups in a study. This involves information on health-related quality of life or on individual symptoms such as pain or itching, which patients recorded with the help of scales in questionnaires. But what difference makes a change relevant for the individual? That is, at what threshold can a response to an intervention be derived for the patient, so that, for example, the difference in the response rates of two groups can be used as an effect measure for early benefit assessments?

When is a difference relevant?

To answer this question, the methodological discussion has long revolved around so-called minimally important differences (MID). This approach is based on the idea that thresholds can be identified for the respective questionnaires that represent the smallest relevant change for patients. However, the methodological problems of this approach have recently become apparent. For instance, it has been shown that an MID is not a fixed value for a questionnaire, but is variable. It depends, for example, on the type and severity of a disease, the direction of a change (improvement or deterioration) or the methods used to determine it. Furthermore, it has become clear that many scientific studies on the determination of an MID no longer meet today's methodological standards or that the methodology used is inadequately described in the scientific publications. In this situation, IQWiG has developed a new approach that makes it possible to easily determine thresholds delimiting a relevant range with sufficient certainty. The aim of this specification was also to create clarity for manufacturers and to make arbitrary responder analyses based on incomprehensible responder definitions unattractive.

Concerns were unfounded

From an evaluation of new scientific reviews on the topic, last year IQWiG therefore identified a value of 15% of the range of the respective scales as a plausible threshold for a relatively small, but sufficiently certain noticeable change. In its General Methods 6.0 published in November 2020, the Institute then specifically stated that in future assessments, it would use responder analyses from a threshold of at least 15% of the scale range of the measurement instrument used. The strict stipulation of the 15% threshold for the acceptance of responder analyses in IQWiG assessments triggered discussions in the run-up to the General Methods 6.0. Among other things, some manufacturers feared that this approach would not be applicable to all scales and that proving the added benefit of a new drug would thus be made more difficult. On the occasion of the addenda published today on early benefit assessments of the drugs secukinumab (for psoriatic arthritis), ivacaftor (for cystic fibrosis) and alpelisib (for breast cancer), Katrin Nink from IQWiG's Drug Assessment Department emphasizes: “The concerns were unfounded. We see that the new threshold is suitable for use in practice.” In the assessments in question, the manufacturers subsequently submitted the requested data so that the Institute could use the responder analyses for its assessments. This applies to several questionnaires on both the burden of disease symptoms and on health-related quality of life in quite different indications such as cystic fibrosis in children, psoriatic arthritis or breast cancer. Nink further explains: “Based on analyses with the new response thresholds, we were able to show the advantages and disadvantages of treatments, such as the advantage of ivacaftor for the quality of life of children with cystic fibrosis. So the new generic criterion works."