IQWiG in dialogue

For many new drugs, there are insufficient data available at the time of authorization as to whether they actually represent an improvement for patients compared to the current therapeutic standard. This is particularly the case with drugs for rare diseases (orphan drugs). In certain cases, the Federal Joint Committee (G-BA) has therefore been given the opportunity to require the pharmaceutical company to conduct a study of the new drug in comparison to an appropriate comparator therapy using the procedure of routine practice data (RPD) collection.

As the basis for this requirement, the G-BA commissioned the Institute for Quality and Efficiency in Health Care (IQWiG) to draw up a concept for RPD collection. Among other things, it will evaluate whether there is an existing data platform in which the RPD collection can be carried out. Disease registries are the first option for this. As part of the concept development process, published information and a questionnaire completed by the registry operators are used to assess whether the data collected in the registry are of sufficient scope and quality to conduct a comparative study. In all previous cases, there was still a need for adjustment, which differed in extent between the registries. The presentation will examine which adjustments were necessary in the previous procedures.

One legal requirement of RPD collection is that the required comparative study must be conducted without randomization. On the one hand, this places additional requirements on the data collected, in particular the identification and recording of possible confounders or the determination of the start of follow-up. On the other hand, such a non-randomized study design is associated with significantly higher demands on the analysis. These aspects must be addressed in the study documents prepared by the pharmaceutical companies. IQWiG examines these documents to determine whether the methodology described is suitable as the basis for a comparative non-randomized study. The presentation will describe and reflect on the experience gained from previous evaluations.

Registry data as a type of routine practice data (RPD) have recently become increasingly important. The G-BA was authorized to commission the collection of RPD for the first time and a separate department for medical databases and registries was introduced in the Federal Ministry of Health during the last grand coalition of the CDU and SPD under Health Minister Spahn. Building on the agreements of the “Ampel” ("traffic light") coalition agreement, this department then pushed for a general legal basis for medical registries that are not regulated by special legislation. The aims of this "Registry Act" included the increased utilization of the more than 400 medical registries across Germany, enabling the linking of medical registries with other data sources and the creation of suitable conditions for the implementation of registry-based randomized controlled trials (rRCTs).

As early as 2015, the multiple sclerosis (MS) registry initiated by the German Multiple Sclerosis Society (DMSG) began to position the technically and content-wise revised registry as a platform for (health services) research in MS. After a brief review of the history of MS registry documentation in Germany, the presentation will focus on the current possibilities and highlight the challenges and current limitations based on specifically implemented research projects.

Evidence generated from observational data is increasingly being taken into account in regulatory decision-making processes at the national and international level. Observational data is already regarded as an established source of knowledge, particularly for assessing drug safety. Just as diverse as the various data sources are their potential applications for regulatory decision-making processes along the entire product life cycle of a drug. The research department at the Federal Institute for Drugs and Medical Devices (BfArM) conducts regulatory research with observational data, among other things. Two current research projects are FQrisk and Real4Reg. FQrisk is a cohort study based on Germany-wide routine billing data from one of the largest statutory health insurance funds (AOK) and investigates issues relating to the therapeutic safety of fluoroquinolone antibiotics. Real4Reg is an EU-funded multi-stakeholder project based on different observational data sources from Denmark, Finland, Portugal and Germany, focussing on the heterogeneity of observational data and their (artificial intelligence/machine learning-supported) analysis methods. The presentation will describe current pharmacoepidemiological research projects at the BfArM and describe potential problems, solutions and possible use of observational data for regulatory research.

The concept of target trial emulation (TTE) aims to emulate a randomized controlled trial (RCT) using observational data. The basic idea is to define a hypothetical ideal-typical RCT, i.e. without taking feasibility and ethical aspects into account, and to derive the analysis for a non-randomized study based on this. Causal inference methods are used for this purpose. By comparing RCTs and non-randomized analyses, "self-made" methodological problems can be avoided. In addition, possible sources of bias are identified. If the TTE perfectly emulates the reference RCT, causal conclusions can sometimes be drawn from it. However, this places very high demands on the underlying data.

Existing or new health technologies are evaluated as part of a Health Technology Assessment (HTA). Various aspects, such as benefits, harms, costs, as well as legal, ethical and social aspects, play an important role here. As the evidence on these aspects cannot always collected in clinical randomized controlled trials (RCTs), observational studies are often carried out. Drawing causal conclusions (referred to as "causal inference" in epidemiology) from observational data presents known problems such as confounding, immortal time bias and selection errors. Depending on the type of confounding and selection bias, traditional statistical methods are not sufficient to draw causal conclusions. Approaches such as causal diagrams and target trial emulations, paired with g-methods, help to use the potential of observational data while controlling for systematic bias.

The presentation will briefly introduce and discuss the terms and concepts of causal inference using a case example. In the case report study, second-line chemotherapy will be compared with no second-line chemotherapy in women with advanced ovarian cancer and potential biases in the analysis of observational data will be assessed. For this purpose, a large validated retrospective database will be used and analysed using a stepwise analytical approach. This begins with a rough, purely associative analysis and becomes more complex with each step of the analysis, finally culminating in a complete causal analysis. The effect estimates of the individual analytical approaches will be compared with the effect estimates of an RCT with the same research question (reference result)

The deviation of the results from the reference result emphasizes the importance of well-planned and methodologically adequate study designs and analysis techniques. The presentation will describe the challenges and opportunities of causal methods.

The availability of large data sets promises enormous potential for medical research. Electronic health records, administrative billing data, registries or information from apps raise hopes for so-called "real-world" evidence that allows patterns, rare events and long-term outcomes to be investigated. These hopes are accompanied by concerns that the publication of low-quality retrospective studies will waste valuable resources and distract from more credible research. Publishing studies on observational data may seem easier and more economical than conducting interventional studies, but properly analysing and interpreting the results is much more complex. In an interventional study, it is possible to tailor the design to the objectives of the study. In retrospective studies, on the other hand, the data are not collected with a research question in mind, and consequently researchers have no control over how the data are collected, which variables are measured, or when and how they are measured. Treatments are not randomized and the reasons for treatment are often unknown, blinding is not possible, and many measurements, including outcome measurements, are not predefined but are the result of daily clinical decisions. Observational studies must meet high standards in order to provide meaningful findings and deliver real benefits. Researchers and readers need tools to critically evaluate observational studies and interpret their results. This presentation will discuss key challenges of observational studies, particularly retrospective studies, and present strategies that can improve the quality and credibility of observational studies.