IQWiG publishes new version of its General Methods – English translation now available

The Institute for Quality and Efficiency in Health Care (IQWiG) revised its methods paper and published the German original version “Allgemeine Methoden 6.0” (General Methods 6.0) on www.iqwig.de in November 2020. This document is the basis for the scientific work of the Institute and its external experts as well as for the collaboration with its contracting agencies, the Federal Joint Committee (G-BA) and Federal Ministry of Health (BMG). The English translation is now available.

New features include statements on the investigation of the relationship between volume of services and quality, a section on different treatment periods in studies, and a more concrete approach to the assessment of clinical relevance.

In contrast to the draft published for commenting in December 2019, the published Version 6.0 no longer contains a section on determining the extent of added benefit in the case of continuous data, as further clarification is needed here. In a future draft, IQWiG will therefore revise the derivation of thresholds and submit an updated proposal for renewed commenting.

The General Methods summarize the scientific standards used by the Institute. In order to reflect the expansion of the Institute's legal tasks and the further development of standards in scientific disciplines, this manual is updated regularly – partly in smaller steps, partly by fundamental revision, which is then reflected in a new version number.

Minimum volumes: How are volume of services and quality related?

In Germany, hospitals may only provide and charge for certain elective services if they have performed them frequently enough in previous years. This is based on findings on the relationship between the volume of services and the quality of treatment outcome. On behalf of the G-BA, for several interventions IQWiG already investigated whether a positive correlation is proven. Section 5.2 describes the Institute's approach to information retrieval and assessment in these cases: possible volume-outcome relationships are examined on the basis of observational studies or controlled intervention studies.

Different mean observation periods may reduce the certainty of conclusions

In oncology in particular, the dossiers submitted by manufacturers for the early benefit assessment of new drugs often contain data from studies in which the mean observation periods differ in the two groups to be compared. This makes it difficult to conduct fair comparisons in which the adverse effects of treatments should neither be over- nor underestimated. The manufacturers usually argue that treatment in one arm was discontinued or switched more often. However, IQWiG stresses that simple statistical methods based on relative frequencies or incidence densities cannot adequately compensate for the decrease in the certainty of conclusions that this causes. For this reason, in Version 6.0, the Institute emphasizes the necessity of complete data collection and in the new Section 9.3.12 calls for the use of adequate survival time methods, also in the case of treatment discontinuation or switching.

In the course of the commenting procedure, the Institute adopted a proposal of the biometric societies and in the above section no longer refers only to "different mean observation periods", but generally to "variable observation periods". The new section now also includes references to special methods of survival time analysis, such as how to deal with competing risks.

Shift of emphasis in the assessment of clinical relevance

In order to assess the clinical relevance of a difference between two treatment alternatives, in recent years responder analyses with a response criterion for patient-relevant outcomes such as "health-related quality of life" or "symptoms" have been increasingly performed. IQWiG has now specified when responder analyses will be used for the assessment, that is, the required minimum scale range of the score of a measurement instrument (Section 9.3.3). This is intended to provide clarity for manufacturers and to prevent arbitrary responder analyses based on incomprehensible responder definitions.

Further changes and updates

The Institute also supplemented or modified many other parts of the General Methods. For example, Section 3.1.3 on the presentation of aspects of harm in benefit assessments now contains more details than before. In Chapter 5, renamed "Assessments of health care", the Institute fundamentally revised its comments on evidence-based guidelines. And in Section 9.3.7 on meta-analyses, the methodological approach for applying the Knapp-Hartung method in meta-analyses with random effects, which was introduced in 2016 in the last revision of the methods paper, was substantiated.

An overview of the main changes is given in the methods paper, which now comprises almost 300 A4 pages, under the heading "What is new?”

Active participation in the commenting procedure

IQWiG published the draft for the new version of the General Methods at the end of 2019 and called for comments. The Institute then received 40 comments, some of them very detailed. After the deadline for the submission of comments expired, a debate with persons who had submitted comments was held in June 2020.

Helpful suggestions from the commenting procedure were incorporated into the present version. In addition to the major adjustments mentioned above, the Institute, among other things, adopted a further proposal by statistical societies and supplemented the methods for meta-analyses of diagnostic accuracy studies in Section 9.3.7. The removal of the section on the assessment of open-label studies with subjective outcomes, which was criticized in the commenting procedure, should also be mentioned here. Due to new literature on the topic published in the meantime, the final version of the General Methods no longer includes this section.

Together with the methods paper, IQWiG publishes a documentation and evaluation of the hearing in which, on the one hand, all written comments are documented in full. On the other hand, the document contains the Institute's response to the comments submitted, which address all main arguments of the comments.

IQWiG will provide timely information on the next processing step.