In 2015, ASCO and the European Society for Medical Oncology (ESMO) introduced value frameworks that utilize algorithmic scales to evaluate the clinical benefit of cancer therapies and provide an objective assessment of outcomes and treatment toxicities experienced by patients.^1,2 Although the two frameworks were developed for different purposes, they share the common goal of providing an unbiased and reliable approach to data analysis. Both frameworks assign lesser weight to outcome measures derived from survival surrogates and incorporate other outcome measures, including long-term survival, quality of life, and adverse events, into their benefit evaluation.

ASCO’s Value Framework, which was updated in 2016,³ uses a Net Health Benefit version 2 score to assign the relative benefit of specific cancer treatments based on clinical benefit, treatment side effects, and improvement in patient symptoms or quality of life in the context of the cost of that therapy. In 2017, ESMO revised its version 1.0 Magnitude of Clinical Benefit Scale to version 1.1, which allows for the scoring of single-arm studies.⁴

To better understand the performance characteristics of the two frameworks and to identify reasons for discordant scoring, ASCO and ESMO collaborated on a joint project to evaluate the concordance between the two frameworks when used to assess the clinical benefit of cancer therapies. For this analysis, ASCO and ESMO researchers calculated the Net Health Benefit score and Magnitude of Clinical Benefit Scale for patients with advanced cancers enrolled in 97 randomized clinical trials, which yielded 102 drug comparisons graded for each framework. Their evaluation found that the two frameworks produced comparable measures of clinical benefits in about two-thirds of the clinical trials assessed, which was higher than what had been observed in other comparison studies.⁵

To improve convergence of the two frameworks, ASCO and ESMO suggest addressing these four issues in their respective frameworks:

• Revisit the weights assigned to absolute and relative gains in survival;
• Better account for limitations of progression-free survival as a surrogate for overall survival in the ASCO Net Health Benefit version 2 Framework;
• Reassess the methodology and terminology used to reward long-term gains in survival;
• Consider refinements in toxicity scoring.

Richard L. Schilsky, MD, FACP, FSCT, FASCO

The ASCO Post talked with Richard L. Schilsky, MD, FACP, FSCT, FASCO, Senior Vice President and Chief Medical Officer of ASCO, about the differences in ASCO’s and ESMO’s value frameworks; the early challenges ASCO faced in initiating conversations regarding the cost vs value of cancer therapies; the evolution of ASCO’s Value Framework over the past 4 years; and the next steps to provide clinicians with a user-friendly format to incorporate the ASCO Value Framework into their practices.

Factors Contributing to Discordance Between Frameworks

Please talk about the findings of ASCO’s collaboration with ESMO to evaluate the concordance between the two value frameworks when assessing the clinical benefit of new cancer therapies. How might the frameworks be refined to bring their scores into closer alignment?

One of the reasons for doing this study was to understand to what extent the two frameworks differ and why they differ in their scoring. It’s important to remember that these frameworks were developed for different purposes, so there is no reason for the results they produce to be the same. The ASCO Value Framework Net Health Benefit score is designed primarily to be a guide in doctor/patient decision-making and is meant to be used on an individual level. The ESMO Magnitude of Clinical Benefit Scale is designed to be used on a population level to help European countries identify an objective way of grading the clinical benefit of therapies tested in clinical studies to develop public health policy and improve clinical decision-making, among other purposes.

The frameworks offer complementary information and include all of the same raw data elements. What is different is the way those data elements are utilized in the two frameworks.

— Richard L. Schilsky, MD, FACP, FSCT, FASCO

Tweet this quote

Another reason we were interested in having a comparative assessment of the two frameworks is because we had noticed that other author groups were publishing their own analysis of agents tested in clinical trials using scores generated from the ASCO and ESMO frameworks; in many cases, they were publishing high levels of discordance between the two systems. We were suspicious that some of the apparent discordance might be related to the fact that these frameworks are technically -complicated and nuanced, and our concern was that the expert doing the scoring might not be doing the calculations properly. We also worried that as different groups use the frameworks for their own purposes, they might calculate a study score using each framework and then promote the score that looked best, even though it might not be valid. We wanted to understand why the two systems might assign different scores to the same study.

Our comparative assessment of the two frameworks found that the concordance rate is pretty high and that the main drivers of differences are in the way the frameworks calculate treatment toxicities and how they evaluate the tail of the survival curve gains. The ESMO scoring system uses both absolute as well as relative gains for overall and progression-free survival in assessing the clinical benefit of a new treatment over its comparator. In contrast, ASCO’s Value Framework uses only relative gain for overall and progression-free survival in its assessment of the clinical benefit of a new therapy over its comparator.

Cost vs Value in Cancer Care: Opening a Dialogue

In 2015, when ASCO published details of its conceptional value framework, there was some resistance from oncologists and patients to discuss treatment costs. Is ASCO now in a better position to address the issue of cost and help move the conversation forward?

ASCO has been concerned about the cost of cancer care since 2007, when ASCO formed its Cost of Cancer Care Task Force—the precursor of the current Value in Cancer Care Task Force—which was its initial entry into this discussion. Over the ensuing 12 years, we’ve recognized that cancer costs are high and are going up, creating a real financial burden for patients, but cost is only one part of the whole picture of cancer care.

There can be very costly therapies that deliver an enormous benefit to patients and in so doing are delivering value and could deliver more value if they cost less or worked better. That is why we decided to refocus our efforts on treatment value, recognizing that we have to consider the benefits obtained, the side effects experienced, and the burdens created for patients from having to go through the treatment, as well as the financial cost to patients.

From its inception, the ASCO Value Framework was intended to help guide oncologists and patients to the best possible decisions when there are several potential treatment options available. Since then, ESMO and other organizations have developed their own approaches to assessing the value of cancer treatment.

Tracking the Evolution of ASCO’s Value Framework

How has ASCO’s Value Framework evolved since its inception in 2015?

When we published version 1 of the value framework, we received comments from many stakeholder groups, including oncologists, patients, and representatives from the pharmaceutical industry. One of the concerns from many patient groups was that not enough consideration was given to a drug’s low-frequency, low-grade toxicities; from a patient’s point of view, a low-grade side effect that is persistent can be much more troubling than a high-grade side effect that is short-lived. So, when we revised the framework in 2016, we incorporated a mechanism to calculate a treatment’s Net Health Benefit score based on all grades of toxicities.

Working with ESMO on this project has been a terrific exercise; we are learning a lot from each other.

— Richard L. Schilsky, MD, FACP, FSCT, FASCO

Tweet this quote

The fundamental challenge of determining a Net Health Benefit score on a specific drug is that the interpretation of that score is heavily dependent upon the comparator treatment in the randomized clinical trial that the value framework is assessing. One widely held misinterpretation of the value framework is that a Net Health Benefit score of 0 means that a new treatment has no benefit, but that’s not accurate. What a score of 0 means is that the drug has no incremental benefit and no substantially worse toxicities than its comparator. So, a new treatment with a Net Health Benefit score of 0 might be a perfectly acceptable treatment; it’s just not necessarily better than the agent already available.

Accessing the Value Framework

Once development of the ASCO Value Framework is finalized, how will oncologists access it?

Our initial vision was to create a software application that would incorporate a treatment’s clinical benefit and toxicity and then crank out a Net Health Benefit score. However, we are not sure physicians would use that technology, so we haven’t yet moved in that direction. One approach we are considering as an initial way for oncologists to become familiar with the Net Health Benefit score is incorporating the information into relevant ASCO clinical practice guidelines that examine different treatment options.

ESMO has started to incorporate its Magnitude of Clinical Benefit Scale into its clinical practice guidelines, and that may also be an effective strategy to help physicians become more conversant with interpreting the Net Health Benefit score. One issue that was raised from our comparative assessment of the two frameworks is that because ASCO’s scoring system is a continuous system, not a categorical one like ESMO’s, it may be difficult for people to understand what constitutes a good, bad, or great Net Health Benefit score. And that’s a legitimate question.

In ESMO’s and ASCO’s comparative assessment of 102 treatments, we were able for the first time to articulate, based on our analysis, what comprises a high or low Net Health Benefit score. We found that a mid-range score is about 45; so, treatments that fall at 45 or above are considered to be of higher relative benefit and scores that fall below 45 are considered to be of lower relative benefit. That information, by itself, gives people an anchor for assessing value, and these ranges will likely shift over time as we generate more data. For example, we are exploring producing a range of scores in a specific disease area, as opposed to looking at scores across all diseases, as we did in the ESMO collaboration.

Making the Value Framework More User-Friendly

What are the next steps in the development of ASCO’s Value Framework?

Our Value in Cancer Care Task Force is still active, and it has charged a working group of technical experts to address some of the problems with the ASCO Value Framework, including some of the issues with cross-trial comparisons.

Value is not an American or a European issue. It’s a global issue, and we all need to work on this together to improve cancer care for our patients.

— Richard L. Schilsky, MD, FACP, FSCT, FASCO

Tweet this quote

One of the positive results of working with ESMO on this project is that our staff became more experienced in how to calculate Net Health Benefit scores. We now have written an instruction guide on how to operate the ASCO Value Framework that we intend to make available to users. Although the ASCO Value Framework is published and anyone can use it, the way it is currently written, users have to intuit how to use the clinical trial data in calculation of the Net Health Benefit score.

The user guide we are developing will provide a step-by-step approach to accurately calculating the score. I’m not sure what the format will be, but we should be ready to publish it later this year.

Working with ESMO on this project has been a terrific exercise; we are learning a lot from each other and will continue to work collaboratively on these issues. Value is not an American or a European issue. It’s a global issue, and we all need to work on this together to improve cancer care for our patients. ■

DISCLOSURE: Dr. Schilsky has received institutional research funding from AstraZeneca, Bayer, Bristol-Myers Squibb, Genentech/Roche, Lilly, Merck, and Pfizer.

REFERENCES

1. Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology Statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol 33:2563-2577, 2015.

2. ESMO Magnitude of Clinical Benefit Scale: Evaluation Forms Version 1.0. Available at www.esmo.org/Guidelines/ESMO-MCBS/Scale-Evaluation-Forms-v1.0-v1.1/Scale-Evaluation-Forms-v1.0. Accessed March 6, 2019.

3. Schnipper LE, Davidson NE, Wollins DS, et al: Updating the American Society of Clinical Oncology Value Framework: Revisions and reflections in response to comments received. J Clin Oncol 34:2925-2934, 2016.

4. Cherny NI, Dafni U, Bogaerts J, et al: ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 28:2340-2366, 2017.

5. Cherny NI, de Vries EGE, Dafni U, et al: Comparative assessment of clinical benefit using the ESMO-Magnitude of Clinical Benefit Scale Version 1.1 and the ASCO Value Framework Net Health Benefit Score. J Clin Oncol 37:336-349, 2019.