Sean Khozin, MD, MPH, on Randomized Trials vs Real-World Evidence in Patients With Advanced Cancer

We all know clinical trials are the cornerstone of establishing the safety and efficacy of new therapies. [inaudible 00:00:14] authorities rely on clinical trials to make approval decisions, while oncologists depend on the results to guide their treatment decisions at the point of routine care. When we design clinical trials, we ideally aim for studies that have an appropriate balance between internal and external validity. Internal validity refers to the degree to which the results of this study can be correctly attributed to the therapy under investigation. External validity, on the other hand, is important for our ability to generalize the results of this study to patients at the point of care, that is, real-world patient populations. Unfortunately, most cancer clinical trials today have a deficit in external validity, which can make it hard for oncologists to use the results in support of their treatment decisions. Now, there are several reasons behind external validity deficits in cancer clinical trials. Some of the major contributing factors include differences in protocol-specified procedures and standards of care. The way we treat patients in the context of clinical trials can be very different than how we treat patients in the real world. Cancer clinical trials also have narrow eligibility criteria, for example, excluding patients with poor performance status, comorbidities, and other factors such as brain metastases. In addition, there's significant barriers to clinical trial participation, especially for minorities and patients from disadvantaged socioeconomic backgrounds, for the same reasons that we have uneven access to broader healthcare resources. The end result of external validity deficits is that the current body of clinical evidence derived from conventional clinical trials may not sufficiently support [personalization 00:02:11] of treatment decisions by oncologists. This is one of the reasons that clinical judgment and anecdotal evidence continue to drive many treatment decisions today, despite our best efforts to develop evidence-based guidelines and recommendations. This not only leads to uneven practice patterns, but can also be contributing factors to disparities in health outcomes. Now, the three main strategies for addressing external validity deficits that I outlined during my presentation at the San Antonio Breast Cancer Conference. One is the use of real-world data extracted from electronic health records that captured experience and outcomes of patients outside of conventional clinical trials. The second strategy involves decentralization of clinical trials to community health systems and even the patient's homes with, for example, remote monitoring tools and telemedicine technologies. The third strategy is having oncologists adjust the results of conventional clinical trials mathematically to make them more relevant to their patients by accounting for external validity deficits. This can support more precise treatment decisions for patients, especially for minorities and the socio-economically disadvantaged, who I mentioned earlier are typically not adequately represented in conventionally designed clinical studies. I presented one such mathematical method at the conference, which I developed with my colleague, Roger Stein, at NYU. Since most oncologists don't have training in data science and probability calculus, we translated complex mathematical formulas into visual tools and decision trees that oncologists can easily use to guide their treatment decisions, adjusting the results of conventional clinical trials to fit the profile of their individual patients. Our method can also be used by sponsors of clinical trials to design studies with a more appropriate balance between internal and external validity, which is not only important for addressing health disparities in underserved communities and tailoring treatment decisions at the point of care, but in fact, something that regulators such as the FDA are increasingly looking at in making approval decisions.