Datenqualitätsrahmen für EU-Arzneimittelverordnung: Anwendung auf Real-World-Daten (EMA/503781/2024)
1. General comments
IQWiG appreciates the opportunity to provide comments on the draft guidance. The discussion of recommendations derived from the Data Quality Framework for data generated in clinical practice is important to support an appropriate use of this data for regulatory and HTA decision making.
In general, we do not think that the term Real-World Data (RWD) should be used for data from routine clinical practice. It suggests a quality (“real world”) that is not appropriate in many situations (e.g., often data from US clinical practice are not “real world” for European health care systems and vice versa due to differences in how care is organised, or the data may not reflect routine care due to poor data quality). The euphemism included in this term has also resulted in misconceptions about the robustness and relevance of this type of data for certain research questions.
An alternative term could be routine practice data.
As for the term RWE, this is not informative because it does not include information on the study design. However, clarification of the study design is crucial to understand the potential robustness of the evidence. In general, it is recommended to clarify the study design when describing study results (Pacheco RL, Martimbianco ALC, Riera R. Let's end "real-world evidence" terminology usage: A study should be identified by its design. J Clin Epidemiol 2022; 142: 249-251. https://doi.org/10.1016/j.jclinepi.2021.11.013.)
We will, nevertheless, use the terms RWD/RWE in our comments to avoid confusion with regard to the text of the draft guidance.
2. Specific comments on text
| Line number(s) of the relevant text | Comment and rationale; proposed changes |
| 89-953 | Comment: IQWiG fully supports the aim of clearly defining the terms RWD and RWE. The appropriate use of data from routine clinical practice is hampered by the lack of clear definitions. A major issue is the fact that the term RWD often is not only referring to a specific type of data (data from routine clinical practice), but is connected to specific study designs (non-interventional studies). This is not necessary but has a major impact on how RWD can and should be used for decision making. IQWiG is aware that there is a continuum of study designs with randomised interventional studies on one side of the spectrum and non-randomised non-interventional studies on the other side. However, for clarity, we will use these two sides of the spectrum when discussing the requirement and impact of including study design elements in the definition of RWD and RWE. What adds to the problem of unclear definitions of RWD and RWE is the fact that the definitions currently used by the EMA and by the FDA seem to consider different study designs appropriate to collect RWD. This problem is not solved by the current draft guidance. Therefore, the definition of RWD (and RWE) provided in the document does not seem to be suitable to support an appropriate use of data from routine clinical practice in the regulatory context and beyond. The definition of RWD provided in this EMA draft guidance seems to have 2 levels of granularity. These include: 1) “RWD are data that describe patient characteristics (including treatment utilisation and outcomes) in routine clinical practice” 2) “In broader terms RWD represent: - data captured in routine care - which are not collected in a clinical trial - and are relevant to the subject (e.g., age, sex, ethnicity etc.), - the disease, - the treatment, - interactions with the healthcare system, as well as - social and environmental factors influencing health status.” It is unclear why the second definition includes that RWD cannot be collected in clinical trials. If the main characteristic of RWD is data collection in routine care, it is not relevant if this routine care is provided within a clinical trial (which is very well possible) or not. The only requirement would be that routine care is not relevantly affected by the trial requirements. For example, this is avoided in pragmatic (randomised) clinical trials. In such studies, randomisation might be the only interventional part of the trial and after treatment assignment, treatment itself and data collection (e.g., in registries) are performed according to routine care. There is no reason not to consider the data from this type of studies RWD. Potentially, the exclusion of data collection in clinical trials is due to the fact that the guidance works with study definitions used in the regulatory context, specifically in Regulation (EU) 536/2014. In this regulation “clinical trial” is defined as a clinical study which fulfils any of the following conditions: (a) the assignment of the subject to a particular therapeutic strategy is decided in advance and does not fall within normal clinical practice of the Member State concerned; (b) the decision to prescribe the investigational medicinal products is taken together with the decision to include the subject in the clinical study; or (c) diagnostic or monitoring procedures in addition to normal clinical practice are applied to the subjects A study according to this definition from Regulation (EU) 536/2014 could indeed not be considered data collection in routine care, because the definition specifically includes that the therapeutic strategy does not fall within normal clinical practice. However, if this is the reason for excluding clinical trials from the definition, this does not become clear from the text. In addition, such exclusion is not necessary, if data collection in routine care is part of the definition. If necessary, “routine care” should be further specified rather than excluding a specific study design. Alternatively, if there is any need to exclude clinical trials according to Regulation (EU) 536/2014, the guidance should clarify that “(randomised) clinical studies” according to this regulation are included in the definition of RWD. However, such an approach to the definition would be overly complex. Overall, it is unclear from the text why certain study designs are excluded from the definition. Why is it so important to include (randomised interventional) clinical trials in the definition? Non-interventional RWD can be helpful for a number of research questions, e.g., for describing the epidemiology of a disease or the utilisation of certain treatments. However, if treatment effects should be investigated, non-randomised non-interventional study designs are limited because of the difficulties to derive causal effects from studies with such designs. This topic is further elaborated below. Proposed change (if any): The definition of RWD should not include restrictions with regard to study design. In particular, it should not exclude (randomised) interventional studies. |
| 89-95 | Comment: The use of the term “primary data collection (primary use of data)” seems to contradict the definition provided in the glossary (line 739) where primary use of health data is not linked to data collection in a study. Proposed change (if any): |
| 96-103 | Comment: IQWiG agrees that the analysis of RWD should consider the research questions which the data is meant to answer. IQWiG also agrees that use cases 1 and 2 can provide relevant information for study planning and understanding of clinical context. However, from IQWiG’s point of view, for use case 3, the specified approach (investigating associations and impact) does not seem to be sufficient to support all of the research questions which should be addressed under this use case. Specifically, the investigation of associations is insufficient to investigate the effectiveness of interventions (which is a highly relevant HTA research question). Decision making in health care systems requires the description of causal effects rather than associations when benefits and harms of treatments are investigated. Randomised clinical trials are best suited to characterise causal effects. Excluding this design from the definition of RWD limits the potential use of RWD for decision making. This is all the more difficult to understand as there are numerous examples of RCTs in routine care. It is also unclear why the definition of RWD and RWE in Europe seems to exclude (randomised) interventional studies when the definition of FDA explicitly includes these studies [Food and Drug Administration. Framework for FDA’s real-world evidence program. 2018. https://www.fda.gov/media/120060/download: “RWD sources (e.g., registries, collections of EHRs, administrative and medical claims databases) can be used for data collection and, in certain cases, to develop analysis infrastructure to support many types of study designs to develop RWE, including, but not limited to, randomized trials (e.g., large simple trials, pragmatic clinical trials) and observational studies (prospective or retrospective).] A more recent publication from the FDA specifically addresses the problems resulting from unclear definitions of RWD and RWE, as well as the inconsistent description of the use of this data for regulatory decision making [Rahman M, Dal Pan G, Stein P, et al. When can real-world data generate real-world evidence? Pharmacoepidemiol Drug Saf. 2024; 33(1):e5715. doi:10.1002/pds.5715]. This publication also provides recommendations on how to include information on study designs in the description of RWD and RWE. Specifically, it clarifies that many different types of study designs can generate RWD (including randomised interventional trials). A harmonised definition of RWD and RWE including all study designs that can be applied in routine care or can use data from routine care would support the discussion about this type of data and its appropriate use. Proposed change (if any): |
| 566-602 | We support the steps 1 and 2 of a detailed fitness-for-use assessment, especially that it is essential to first articulate the research question and define relevant design elements as well as their respective operationalisations. However, if the data source does not meet these requirements, the following text (lines 595-602) suggests to reshape the research question to fit the data – even if it is stated that generally this should not be the case. Concerning the statements from line 595 onwards, the possibility should be added that if the data source does not meet the requirements of the research question, the option of supplementing missing data should also be examined (see proposed changes). Shifting the focus to feasible ways of rendering a generally suitable data source “fit for purpose” supports the aim of achieving higher levels of DQ in RWD data sources. Proposed change (if any): after line 602 It may also be useful to supplement the data source, namely a) By data linkage to other data sources (reference in the DQF to the statement in section 1.4. (line 152) b) By additional prospective collection, e.g., of PROM, confounders or additional timepoints. |