Researchers have explored whether an artificial intelligence (AI)-based tool could help to detect immune-related adverse events in patients with cancer, according to a recent study published by Sun et al in the Journal of Clinical Oncology.
Background
Although immune checkpoint inhibitors can provide patients with life-saving treatment, they are also known to cause immune-related adverse events that may impact nearly every organ in the body to varying degrees. The frequency and severity of immune-related adverse events in real-world data sets are little understood, making it challenging to combine cases effectively across institutions and gain insights into the optimal management of these patients.
Study Methods and Results
In the recent study, the researchers incorporated the use of a prebuilt large language model to identify information relating to instances of immune-related adverse events in hospital settings. They analyzed 10 years of data in a gold standard, manually curated data set from patients who were hospitalized after receipt of immune checkpoint inhibitor therapy.
The language model uncovered the most common immune-related adverse events leading to hospitalization, including immune checkpoint inhibitor–induced colitis, hepatitis, pneumonitis, and immune checkpoint inhibitor–induced myocarditis—which can be fatal.
The researchers then compared the performance of the large language model with International Classification of Disease (ICD) codes. After retrospectively identifying immune-related adverse events, they found the large language model demonstrated greater accuracy.
“Not only did the [large language model] demonstrate higher accuracy in detecting [immune-related adverse events] compared to ICD codes, it identified additional cases of [immune-related adverse events] not picked up via manual adjudication with an excellent specificity/sensitivity and at only 9.53 seconds per chart,” detailed senior study author Kerry Reynolds, MD, Director of the Severe Immunotherapy Complications Program at Mass General Cancer Center. “The results demonstrated that the model consistently achieved sensitivities and specificities above 90% across the four [immune-related adverse events], which is excellent,” she added.
Conclusions
“As a free and open-source model, the [large language model] pipeline opens up this field, enabling other institutions to quickly recreate similar databases and has the potential to ignite collaboration in unprecedented ways,” Dr. Reynolds underscored. “Historically, collaboration in the field of [immune-related adverse events] has been concentrated among large academic centers, leaving smaller community hospitals with less opportunity to contribute. This study has the potential to change that. The [large language model] presented in this study requires minimal computational resources and can be run on a local machine. [W]e are eager to share it with the broader community,” she concluded.
Disclosure: For full disclosures of the study authors, visit ascopubs.org.
