Determining the extent of added benefit of new drugs in dossier assessments: Have IQWiG's methods worked in practice?

The German Act on the Reform of the Market for Medicinal Products (Arzneimittelmarktneuordnungsgesetz, AMNOG) was introduced in 2011. Since then, all new drugs entering the German market are generally subject to an early benefit assessment to determine their added benefit compared to the standard of care. In this assessment, the extent of added benefit (EAB) must be quantified in accordance with further pharmaceutical legislation (Arzneimittel-Nutzenbewertungsverordnung, 2010). The Institute for Quality and Efficiency in Health Care (IQWiG) published the methods for determining the EAB for relative effect measures (EAB methods) with the first dossier assessment in 2011 and subsequently incorporated them into its General Methods. How have the EAB methods worked in practice?

In a working paper entitled "Methods for determining the extent of added benefit of new drugs - empirical evidence from dossier assessments", the Institute analysed how far effect sizes that should at least be present for a given EAB are actually reached when applying the EAB methods. The results showed that in the majority of outcome categories, the actual effect sizes at least reached the desired effects, but there were also deviations.

Tiered expectations

Relative effect measures such as the relative risk are particularly suitable for quantifying an added benefit. In 2011, IQWiG defined desired effect sizes and thresholds for each outcome category (mortality, serious/severe symptoms and adverse effects, health-related quality of life, and non-serious/non-severe symptoms and adverse effects) and each EAB (minor, considerable, major). The following applies: The higher the EAB, the greater the effect on patient-relevant outcomes in drug trials must be.

Evaluation based on a broad dataset

From the introduction of AMNOG in 2011 until the end of 2022, the Institute prepared almost 700 dossier assessments, using the thresholds for confidence intervals to determine the EAB in 1747 outcomes. More than three quarters of these outcomes were from the field of oncology.

The analysis showed that for mortality, for example, the median of the actual effect estimates of the highest EAB category "major" was 0.54, i.e. slightly above the expected value of 0.50. Here, the actual effect fell short of the expectation defined for a major effect in more than half of the cases. Similar situations were also found for the highest EAB categories in the other outcome categories of symptoms, adverse effects and health-related quality of life.

In each of the following lower EAB categories, the values of the effect estimates were mostly below the expected values. The actual effects therefore met or even exceeded expectations in the majority of cases.

Methodological implications now under review

Based on a broad dataset, the aim of this working paper was to describe the expectations and reality of determining the EAB of new drugs after more than 10 years of early benefit assessments. IQWiG will consider in the near future whether these results should lead to a revision of the EAB methods. However, it is clear from the available results that any adjustment should focus on the highest EAB categories.