3. What are the special challenges in the assessment of non-drug interventions?

Numerous factors can render the assessment of the benefits and harms of non-drug interventions particularly challenging:

The interventions under assessment vary widely

There is a very wide range of non-drug interventions (see Section 1) with manifold possible modes of action intended to solve various medical problems. The spectrum ranges from very simple to highly complex diagnostic and treatment methods.

Systemic therapy, for example, is used as a psychotherapy technique for a variety of conditions, such as anxiety and obsessive-compulsive disorders, eating disorders, depression, and physical illnesses. It is based on numerous theoretical concepts and involves numerous treatment techniques. In practice (and also in studies), systemic and non-systemic treatment techniques are often combined.

Despite the complexity and diversity of this type of therapy, an assessment of its benefits and harms is indeed possible. However, before an assessment by IQWiG, it was first necessary to clarify which treatments can be classified as systemic therapy and which cannot, and which criteria are suitable for determining benefits and harms in the respective setting. From 2014 to 2017, IQWiG assessed systemic therapy on behalf of the G-BA and analysed more than forty studies (read more: N14-02: Systemic therapy in adults as a psychotherapeutic approach). Since the data provided at least indications of benefit in patients with anxiety and obsessive-compulsive disorders as well as in patients with schizophrenia, this psychotherapy technique was introduced as a generally accepted treatment method in 2019.

In addition to the more difficult transferability of results between variants of an intervention, time-related or human factors may also play a role. For example, for the assessment of low-dose CT for screening for lung cancer (see the German press release), the question arose as to what extent older studies should still be taken into account when new CT devices allow better image resolution and evaluation. The evaluation of the images is also influenced by the experience of the medical staff, a factor that must always be taken into account when assessing the benefits and harms of non-drug interventions in areas such as surgery, psychotherapy or dentistry.

The examples underline that it is not only the very wide range of non-drug interventions that presents a challenge for an assessment of benefits and harms. The differences and variations within a group of interventions also complicate their assessment.

Assessing examination and treatment methods is complex, as interventions consist of multiple components.

Non-drug interventions are often composed of multiple components whose interaction is expected to result in a patient-relevant benefit. To assess their benefits and harms, it may be necessary to include all components in the assessment.

For example, IQWiG was asked how well positron emission tomography (PET) (alone or in combination with computed tomography, CT) can help patients with colorectal cancer (see the press release: Benefit of PET or PET/CT in recurrent bowel cancer is not proven). The technique is used to detect recurrent tumours (recurrences) and provide information on whether the tumour is benign or malignant. If recurrences are detected by PET/CT, it may mean that further surgery is needed, but it may also mean that palliative treatment should be started instead of attempting curative treatment.

In the case of curative treatment, the question arises whether a diagnostic test and subsequent treatment together actually increase the chances of living longer: What is the benefit of PET/CT if, although tumours may be detected earlier and classified better, this does not lead to patients living longer at all, but only to more surgical interventions? The overall benefit for the patient can only be measured meaningfully if all components are included in the assessment: i.e., not only the test accuracy of PET/CT but also the subsequent measures and treatments.

The following example illustrates how important and at the same time complex it is to assess all components of a method: In telecardiology, people with severe heart failure or arrhythmia receive an electronic device (see the press release: Advanced heart failure: telemonitoring reduces cardiovascular mortality) that is implanted to restore a normal heart rhythm ("pacemaker"). Via telemonitoring, physiological parameters can be monitored remotely, and a physician can initiate additional treatment measures in certain cases. The telecardiology technique thus consists not only of the implanted technical device, but also of monitoring by the physician and subsequent treatment. Ultimately, the totality of the measures should lead to the avoidance of heart attacks and deaths and to an increase in quality of life. Whether telecardiology works depends on numerous components, for example, which values are recorded, how often they are transmitted, how precisely they are evaluated, and how quickly action is taken in the event of problems.

A first assessment did not demonstrate a clear benefit of telecardiology. Only when the assessment was narrowed down to well-organized programmes with close monitoring did it become clear that this technique is beneficial and can reduce cardiovascular mortality.

Studies of non-drug interventions often show weaknesses.

Clinical studies are conducted to determine the benefits and harms of medical interventions. For medical devices, which often represent components of non-drug interventions, less stringent requirements for market entry apply than for drugs. As a result, clinical studies of medical devices are often weaker than drug studies in terms of methods and content, rendering their assessment more difficult.

There are few incentives for medical device manufacturers to produce high-quality studies because they are not mandatory for market entry. In most cases only a few studies are therefore available to assess benefits and harms. The sample sizes are often far too small, the follow-up period too short, or there is not even a control group. For a few non-drug interventions, numerous studies with many patients are available (e.g., telecardiology). However, closer examination by IQWiG sometimes reveals that some studies have been completed but not published or data on important aspects such as side effects or health-related quality of life are lacking.

In the area of diagnostic tests, randomized controlled studies (RCTs) are not usually available, as in most cases manufacturers only want to demonstrate that a test is highly accurate in detecting a disease.

Under certain circumstances, this may be sufficient to assess the benefit of the test. In most cases, however, it is questionable whether detecting a disease earlier or more accurately will actually provide the patient with better treatment and a better outcome. This is especially true for screening tests. Therefore, for well-founded conclusions on benefits and harms, it is important to investigate the tests and the subsequent treatment together in studies (see the example of PET/CT above).

Contrary to some claims by manufacturers or health care service providers, high-quality studies for the assessment of non-drug interventions can in principle be conducted in a similar way to those for drugs. Although it is often not possible to implement a double-blind design in studies on surgical, psychotherapeutic or dental interventions, other measures (e.g. blinded outcome assessment) can be used to reduce potential bias.

The examples presented illustrate that the diversity and complexity of non-drug interventions can complicate evidence-based assessments of benefits and harms, but do not generally prevent them.

image: © PantherMedia / Golden Sikorka