Methods paper: New version 6.1 becomes effective – English translation now available

The Institute for Quality and Efficiency in Health Care (IQWiG) revised its General Methods and published Version 6.1 on its website www.iqwig.de on 24 January 2022. With immediate effect, the document is the basis for the scientific work of the Institute and its external experts, as well as for collaboration with contracting agencies. New features include additions to the evidence search for clinical practice guidelines and a method for determining the extent of added benefit in the case of continuous data. Moreover, the Institute now follows the new concept of conclusive effects.

The General Methods describe IQWiG's scientific standards. In order to reflect the further development of standards in the scientific disciplines and the Institute’s expanded statutory responsibilities, the methods paper is constantly updated – sometimes in smaller, sometimes in larger steps – resulting in new version numbers.

New: evidence searches for clinical practice guidelines

Since 2020, the Association of the Scientific Medical Societies (AWMF) has been able to propose clinical practice guidelines to the Federal Ministry of Health. On behalf of the Ministry, IQWiG then supports the development or update of these guidelines with evidence searches. The Institute has implemented this new legal mandate with its new product "evidence report". The resulting changes in the General Methods 6.1 affect some sections in Chapters 1, 2 and 5.

Thresholds for determining the extent of added benefit in the case of continuous data

To determine the extent of added benefit of new drugs in early benefit assessments, the Institute has extended its approach in order to consider continuous data (Section 3.3.3). The thresholds for standardized mean differences (SMD) are based on Cohen's usual classification into small effects (SMD between 0.2 and 0.5), medium effects (SMD between 0.5 and 0.8), and large effects (SMD greater than 0.8). For both severe and non-severe symptoms, the known irrelevance threshold of SMD = 0.2 is the threshold for a minor added benefit. The Institute has set new thresholds for the extent categories "major" for severe symptoms and "considerable" for non-severe symptoms.

Application of the new concept of conclusive effects.

For the qualitative summary of study results, the Institute specifies a new approach for the qualitative summary of study results using the concept of conclusive effects (Section 3.1.4). Conclusive effects exist when the data allow inferring an effect in the sense of the research question, even though a common effect estimate across all studies is not meaningfully possible. In heterogeneous data situations, this corresponds to the well-established concept of unidirectional effects. The new concept of conclusive effects is also applicable in homogeneous data situations.

Commenting procedure used extensively

IQWiG published the draft for the update of its General Methods in August 2021 and invited comments; 30 comments, some of them very detailed, were received. After the commenting deadline, an oral scientific debate with the persons submitting comments was held in November 2021.

As a result of the commenting procedure, IQWiG explained the further development of its methods in Version 6.1 in more detail. This applies in particular to the changes in Chapter 3 (benefit assessment) and Chapter 9 (assessment of information). The comments had indicated a need for explanation here.

Unlike in the past, the Institute did not publish the documentation and evaluation of the oral scientific debate at the same time as the updated methods paper, but only on 22 March 2022. This was due to the time required to address in detail the arguments of persons submitting comments.

IQWiG will provide information on the next stage of its work on the methods paper as early as possible.