Researchers have developed and validated an artificial intelligence (AI)‒assisted volumetric response criteria for assessing response in pleural mesothelioma. The AI-backed criteria outperformed both humans and standard international Response Evaluation Criteria in Solid Tumors (RECIST) criteria, according to findings published in The Lancet Oncology.
Background
Most physicians use RECIST criteria to measure tumor growth across solid tumors for greater repeatability and comparison across various health systems, scans, and countries. RECIST criteria relies on a diameter-based measurement of tumor growth to indicate if the tumor is growing or reducing in response to treatment.
However, this method has limited applicability to pleural mesothelioma due to its irregular crescent growth pattern in the lining of the lungs. Additionally, many have found that these criteria cannot accurately predict patient survival.
Study Methods
A group of AI experts, radiologists, and pulmonologists from the Netherlands Cancer Institute developed the ARTIMES criteria to measure pleural mesothelioma tumor growth with a more volume-based approach that is compared on a pixel level to prior scans with AI, which is too difficult and time-consuming for physicians to complete.
They conducted a retrospective, multicenter study to develop and validate ARTIMES, with an evaluation of 10,926 computed tomography (CT) scans from 2,080 patients with pleural mesothelioma collected from 14 cohorts, including at least six clinical trial cohorts.
A deep-learning segmentation AI model was trained on a subset cohort of 1,176 CT scans plus 100 negative CT scans. The CT scans were annotated by 12 radiologists and a pulmonologist.
The model was then tested on 98 CT scans from independent international hospitals and external testing was conducted on a cohort of 138 CT scans from three sources. Performance of the AI model for segmentation was evaluated with dice similarity coefficient as a measure of overlap and a normalized surface distance of 3 mm as a threshold for a tolerated distance from the reference boundary. Thresholds for progressive disease were then established based on patients with multiple CT scans before and after treatment.
The ARTIMES model was then validated on data from eight clinical trials, encompassing 4,674 CT scans from 943 patients, and compared with modified RECIST criteria as a measure of survival.
Key Findings
ARTIMES demonstrated superior prognostic performance (concordance index = 0.83; 95% confidence interval [CI] = 0.79–0.87) compared with modified RECIST criteria (concordance index = 0.73; 95% CI = 0.66–0.80; P = .023). The AI model detected progression a median of 5 weeks ahead of the RECIST criteria at 124 days (95% CI = 115–126) vs 162 days (95% CI = 138–167), respectively.
In clinical trials, progression-free survival based on ARTEMIS criteria was more strongly correlated with overall survival (coefficient of determination [R2] = 88%; 95% CI = 42%–100%) than progression-free survival based on modified RECIST criteria (R2 = 6%; 95% CI = 0%–97%).
Additionally, the model showed a surrogate threshold effect with a hazard ratio for progression-free survival under 0.82, whereas no threshold was found with the modified RECIST criteria.
The AI-derived tumor volume measurements also outperformed standard T stage and World Health Organization performance status measurements.
Impact
“We obviously want patients worldwide to benefit from this,” said lead study author Kevin B. W. Groot Lipman, PhD, a technical physician in the Department of Thoracic Oncology at the Netherlands Cancer Institute. “We are in the process of getting the model approved for use in other hospitals. We are also eagerly awaiting a proposal from the EU to simplify the approval process for this type of medical device.”
Additionally, the study authors shared their code so that researchers globally can begin to use the new model.
“I expect this model to come as a shock to physicians and researchers outside the mesothelioma field,” Dr. Lipman predicted.
“This is going to open up a whole new field of research. We expect that AI will also be able to help with many other types of tumors,” he added, as the institution is already exploring the use of AI models for lung cancer and brain metastases.
He also noted that the model could be used to help make measurements in clinical trials more reliable, pending validation. “This allows us to better assess the efficacy of new treatments in clinical trials,” he said.
DISCLOSURES: Funding for the study provided by Asbestos-Related Disease Section (SAGA) of the Dutch Society of Pulmonology and Tuberculosis (NVALT), Dutch Cancer Society, and Dutch Ministry of Health, Welfare and Sport. For full disclosures of the study authors, visit thelancet.com.
