SABCS 2016: IBM Watson for Oncology Platform Shows High Degree of Concordance With Physician Recommendations
The artificial-intelligence platform Watson for Oncology (WFO) achieved a high degree of concordance with the recommendations of a panel of oncologists in a double-blinded validation study, according to results presented at the 2016 San Antonio Breast Cancer Symposium, held December 6 to 10 (Abstract S6-07).
WFO, developed by IBM in collaboration with Memorial Sloan Kettering Cancer Center, is a computing system that can extract and assess large amounts of structured and unstructured data from medical records through natural language processing and machine-learning to provide cancer treatment options.
WFO provides treatment recommendations for breast, lung, and colorectal cancers, said the study’s lead author, S.P. Somashekhar, MBBS, MS, MCH, FRCS, Chairman of the Manipal Comprehensive Cancer Center, Manipal Hospitals, in Bengaluru, India.
“Manipal Hospitals recently adopted Watson for Oncology as a tool to support our oncologists in making quality, evidence-based decisions for their patients,” Dr. Somashekhar said. “We wanted to know more about how it would impact oncologists’ day-to-day practice and to assess how Watson’s recommendations compared to the decisions of our team of experts.”
WFO vs Tumor Board
In order to assess concordance between WFO and Manipal’s multidisciplinary tumor board—a group of 12 to 15 oncologists who meet weekly to review cases at the hospital—Dr. Somashekhar and colleagues studied the cases of 638 breast cancer patients who had been treated at Manipal Hospitals. They entered data on the cases into the WFO system, then analyzed the degree of concordance between WFO’s recommendations and those of the tumor board, as well as the time it took each group to generate their recommendations.
WFO’s recommendations came in three categories: recommended standard treatment (REC); for consideration (FC); and not recommended (NREC). Overall, 90% of WFO’s REC or FC recommendations were concordant with the recommendations of the tumor board.
The degree of concordance varied depending on the type of breast cancer. Dr. Somashekhar said WFO recommendations were concordant nearly 80% of the time in nonmetastatic disease, but only 45% of the time in metastatic cases. In cases of triple-negative breast cancer, WFO agreed with the physicians 68% of the time, but in HER2/neu–negative cases, WFO’s recommendations matched the physicians’ recommendations only 35% of the time.
Dr. Somashekhar said the difference in concordance was not surprising given the fact that triple-negative breast cancer has fewer treatment options than HER2/neu–negative breast cancer.
“Including HER2/neu cases opens up many more treatments and variables for consideration,” he explained. “This increases the demands on human thinking capacity. More complicated cases lead to more divergent opinions on the recommended treatment.”
The study also compared the amount of time taken to capture and analyze data to generate recommendations. Dr. Somashekhar said that it took an average of 20 minutes when done manually, but after gaining more familiarity with the cases, mean time decreased to about 12 minutes. By comparison, WFO took a median time of 40 seconds to capture and analyze data and give a treatment recommendation.
Artificial Intelligence Complements Physicians’ Work
Dr. Somashekhar cautioned that while artificial intelligence is a step toward personalized medicine, it should not be viewed as a replacement for a physician—only a complement.
“There is always an important distinction between what can be done and what should be done based on a variety of very personal factors for the patient,” he said. “We are dealing with human beings, and the context and preferences of each individual patient, the patient-physician relationship, and human touch and empathy are very important.
“It’s always going to be the decision of the treating oncologist and patient to determine what is truly the best option for the patient,” he concluded.
This study was investigative and received no external funding. Dr. Somashekhar declares no conflicts of interest and acknowledges his team of scientific and technical collaborators, Manipal Hospital Comprehensive Cancer Center oncology fellows and IBM Watson Health.
The content in this post has not been reviewed by the American Society of Clinical Oncology, Inc. (ASCO®) and does not necessarily reflect the ideas and opinions of ASCO®.
