One of HARMONY’s aims is to increase consistency in future clinical trials therefore allowing for the results achieved to be compared and combined creating increased value to the hematological community. To do this, clinical trials need to use consistent core outcomes, but which outcomes should be chosen?
HARMONY researchers are at this time compiling a set of the most important outcomes for AML by consulting a panel of stakeholders. The first consultation round has already started.
At present, several innovative compounds are being tested in clinical trials in order to improve the treatment of hematologic malignancies (HM). However, these trials measure different outcomes, making it difficult to compare and combine the results. Lars Bullinger (Charité) makes the following observations: “For instance, researchers may use a variety of parameters to describe response to treatment: complete remission, complete remission with incomplete hematologic recovery; complete remission and MRD negative; partial remission; or stable disease. In addition, the majority of clinical trials do not assess long-term side effects and their influence on the patients’ quality of life. A core outcome set (COS) may offer a solution to these problems.”
Lars Bullinger (Charité), together with Renate Schulze-Rath (Bayer AG), is leading HARMONY Work Package 2, which in collaboration with HARMONY Work Package 6, focuses on Definition of Outcomes.
Bullinger continues: “A COS constitutes the minimum set of outcomes that should be collected and reported in future clinical trials. A COS will be defined by several stakeholder groups including patients, clinicians, health authorities and the pharmaceutical industry. Of course, researchers may include additional outcomes in a particular trial. A COS functions as an international standard to evaluate treatments in clinical trials and other research settings (e.g., registry and observational studies) to assess real world data. Implementing a COS will improve the comparability of clinical trials and observational studies, improve the consistency of reporting, and reduce selective reporting bias. In addition, the COS can be incorporated in clinical guidelines and be used to improve patient management.” Download the Delphi AML Study Protocol document to inspect the full outcome list in Annex 1
Bullinger: “As a pilot study, HARMONY Work Package 2, is coordinating an endeavor to define a COS for AML. AML is the most common Acute Leukemia in adults. Despite recent scientific advances, AML cure rates remain poor compared to other hematologic malignancies. In addition to accelerating clinical research, defining a COS may help to improve the clinical management of AML. In the pilot study, we will focus on outcomes for AML in patients 18 years and older.” Read more >
The researchers will use the so-called “Delphi method” to reach a consensus on the COS. According to this method, a representative group of different stakeholders, by filling out a survey in multiple iterative rounds, gradually converges towards reaching a consensus. Bullinger explains: “We have first compiled a preliminary list of about 70 outcomes based on literature research and interviews with key opinion leaders, industry and patient representatives. We have added clear definitions in lay language. The outcomes in the list range from traditional clinical parameters, such as ‘complete remission’ and ‘leukemia free survival’, to molecular parameters such as ‘MRD molecular’, to patient-reported variables such as ‘fatigue’, and to health economic variables, such as ‘the cost of leukemia treatment’.” Download the Delphi AML Study Protocol document to inspect the full outcome list in Annex 1
The preliminary AML outcome list was converted into a Delphi survey through the use of an online survey tool provided by the COMET Initiative. Renate Schulze-Rath clarifies the approach: “We will ask a panel of stakeholders to rate the importance of the outcomes based on their personal experiences. Each outcome may be ranked on a nine-point Likert scale, with scores 1-3 corresponding to ‘not important’; 4-6 to ‘important, but not critical’; and 7-9 to ‘critical’. After the first Delphi round, participants may provide feedback and suggest additional outcome parameters.”
The Delphi survey will be run in at least three iterative rounds. After each round, the participants will receive an anonymized summary of the survey results per stakeholder group. This allows them to rethink their answers and to vote differently in the next round, should they wish. It is expected that the group will converge towards a consensus COS after a number of iterations. Schulze-Rath explains as follows: “We find it particularly important that patients and patient advocates have a strong influence on the final COS. Therefore, outcomes may be flagged as ‘important to patients’ in the summaries.”
The Delphi process will be stopped after pre-defined consensus criteria that have been defined in a study protocol. After each Delphi round, each outcome will be assigned to one of three categories:
1. Consensus in | 70 % or more respondents over all the respondents (clinicians, EFPIA members, regulators/HTA, patients and patient advocates) scored the outcome as critically important (7-9) AND 15% or fewer rate the outcome as limited important (1-3)
2. Consensus out |70 % or more of all the respondents (clinicians, EFPIA members, regulators/HTA, patients and patient advocates) scored the outcome as limited important (1-3) AND 15 % or fewer rate the outcome as critically important (7-9)
3. No consensus
Schulze-Rath explains how the results will be analyzed: “Outcomes that do not achieve a consensus through the several rounds in the Delphi survey will be discussed at a consensus meeting to finally ratify the AML core outcome set. This applies especially for outcomes that are necessary for special stakeholder groups and have not reached consensus in accordance with the consensus criteria. After completing the last Delphi round, each participant will be asked about willingness to participate in a face-to-face consensus meeting. We will discuss outcomes in the “no consensus” category at a concluding consensus meeting, where we will decide on the final COS. Participants of this meeting will be randomly selected from those that completed the full Delphi process, while ensuring that all stakeholder groups are represented.”
HARMONY clinicians and researchers have already started the Delphi process. Schulze-Rath states: “For this AML pilot, the panel members will be recruited within the HARMONY Alliance. We will strive to recruit participants from each of HARMONY’s major stakeholder groups: clinicians, patients, representatives from the pharmaceutical industry, and representatives from health authorities.”
Bullinger concludes by explaining what will be achieved by this process: “The Delphi method will help to ensure that the COS represents the priorities of clinicians, industry, health authorities, and patients alike. After completion of this pilot, we will start similar projects for the other Hematologic Malignancies that are being studied by HARMONY. Finally, we want to define a COS which will apply to all seven hematological malignancies. In the long term, we hope that patient satisfaction during and after treatment may be improved with the COS.”
View the HARMONY’s Research Projects or read more about the HARMONY Alliance, the HARMONY Big Data Platform, and the HARMONY Anonymization Concept of Data.
Receive the latest news. Click here to subscribe!
Public-private partnership for Big Data in Hematology
HARMONY and HARMONY PLUS are funded through the Innovative Medicines Initiative (IMI), Europe's largest public-private initiative aiming to speed up the development of better and safer medicines for patients. Funding is received from the IMI 2 Joint Undertaking and is listed under grant agreement for HARMONY No. 116026 and grant agreement for HARMONY PLUS No. 945406. This Joint Undertaking receives support from the European Union’s Horizon 2020 Research and Innovation Programme and the European Federation of Pharmaceutical Industries and Associations (EFPIA).
HARMONY Coordination Office
Institute of Biomedical Research of
Salamanca (IBSAL)
Salamanca, Spain
HARMONY Communications
European Hematology Association (EHA)
The Hague, The Netherlands
