TO BETTER UNDERSTAND the performance characteristics of ASCO’s Value Framework Net Health Benefit Score version 2 (ASCO-NHB v2)¹ and the European Society for Medical Oncology’s (ESMO’s) Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1),² ASCO and ESMO undertook a joint project to identify reasons for discordant scoring. The initial step in this process addressed the noncurative setting and was undertaken to evaluate the association between the frameworks to scaling clinical benefit, identify the factors that contribute to discordant evaluations, and identify ways to improve each framework.

The results from their analysis found that the frameworks produce comparable measures of the clinical benefits of new therapies in approximately two-thirds of the more than 100 treatment comparisons examined. It also identified factors contributing to discordant scores, suggesting potential approaches to improve convergence between the algorithmic scales. The joint analysis was published in the Journal of Clinical Oncology.³

Methods

THE ASCO-NHB V2 was published in 2015 and revised in 2016. The ESMO-MCBS v1.1 was published in 2015 and revised in 2017. Both ASCO’s Value Framework’s NHB score and the ESMO’s MCBS are intended to be objective measures of the magnitude of clinically meaningful benefit of anticancer treatments and their side effects.

For this analysis, ASCO and ESMO researchers calculated the NHB and MCBS scores for 97 trials involving patients with advanced cancers, encompassing a total of 102 treatment comparisons. The authors used statistical analysis to establish scoring ranges for “substantial and low benefit.”

Results

THE AUTHORS FOUND that the correlation of the 102 pairs of scores for studies in the noncurative setting is estimated to be 0.68 (Spearman’s rank correlation coefficient; overall survival = 0.71; progression-free survival = 0.67). Receiver operating characteristic curves identified thresholds for ASCO-NHB v2 for facilitating comparisons with ESMO-MCBS v1.1 categories.

After applying pragmatic threshold scores of 40 or less (ASCO-NHB v2) and 2 or less (ESMO-MCBS v1.1) for low benefit and 45 or greater (ASCO-NHB v2) and 4 to 5 (ESMO-MCBS v1.1) for substantial benefit, 37 discordant studies were identified. Major factors that contributed to discordance were using different approaches to the evaluation of relative and absolute gain for overall and progression-free survival, crediting tail-of-the-curve gains, and assessing toxicity.

“This suggests we’re both on the right track toward our shared goals of providing clear, unbiased, and reliable assessments of the clinical benefits of new treatments.”

— Monica M. Bertagnolli, MD, FACS, FASCO

Tweet this quote

Steps to Improve Convergence

TO IMPROVE convergence of the two frameworks, the authors suggest addressing these four issues:

1. Revisiting the weights given to absolute and relative gains in survival
2. Better accounting for limitations of progression-free survival as a surrogate for improved overall survival in the ASCO framework
3. Revisiting methodology and terminology used to reward long-term gains in survival
4. Considering refinements to the scoring of toxicity, given the advantages of each framework’s approach.

“Both ESMO and ASCO recognize the importance of developing robust, valid tools for assessment of clinical benefit that are anchored in standards for accountability for reasonableness. This joint initiative is an important first step by the two organizations to develop their respective frameworks to meet the shared standards to which they aspire,” concluded the study authors.

“It’s encouraging that our frameworks—envisioned with distinct goals and currently at different stages of development—demonstrated good agreement in the scores they produced in this analysis,” said ASCO President Monica M. Bertagnolli, MD, FACS, FASCO, in a statement. “This suggests we’re both on the right track toward our shared goals of providing clear, unbiased, and reliable assessments of the clinical benefits of new treatments.” ■

DISCLOSURE: For full disclosures of the study authors, visit jco.ascopubs.org.

REFERENCES

1. ASCO Value Framework. Available at https://www.asco.org/practice-guidelines/cancer-care-initiatives/value-cancer-care. Accessed December 17, 2018.

2. Cherny NI, Dafni U, Bogaerts J, et al: ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 28:2340-2366, 2017.

3. Cherny NI, de Vries EGE, Dafni U, et al: Comparative assessment of clinical benefit using the ESMO-Magnitude of Clinical Benefit Scale version 1.1 and the ASCO Value Framework Net Health Benefit Score. J Clin Oncol. December 17, 2018 (early release online).