Friends of Cancer Research (Friends) has been a leader in the push for better and faster cancer drug development. Now it is tackling the use of real-world evidence in clinical trials. This is the report of a meeting on the subject that took place on June 16 in Washington, DC.1
Real-world evidence refers to clinical data generated from sources other than traditional trials, including electronic health records, pragmatic clinical trials (randomized or not that use an existing clinical infrastructure to test interventions in everyday situations), patient registries, expanded access, administrative claims, surveys, and data collected via social media, smart phones, and even wearable devices.
Real-world evidence and traditional clinical data are not completely different; rather, they are separate parts of one entity.— Robert M. Califf, MD
Tweet this quote
It is widely believed in the cancer research community that in many respects, trial participants are not like “real” people with cancer due to the strict inclusion and exclusion criteria applied; thus, trial results may or may not be applicable to the majority of patients with the same type of cancer. There is also a belief that real-world evidence can provide a better assessment of the efficacy and safety of an investigative treatment than can data derived from a narrow population that does not mirror the reality of patients with a particular cancer.
Robert M. Califf, MD, U.S. Food and Drug Administration (FDA) Commissioner, did not entirely agree. He acknowledged that “people who enter clinical trials are different, but that will change soon, as enrollment criteria more accurately reflect larger populations.” He said that the agency’s Oncology Center of Excellence recognizes that clinical trials need to be organized around the needs of patients, not around the needs of researchers, as has been the case for so long. “Real-world evidence and traditional clinical data are not completely different; rather, they are separate parts of one entity.”
How Real-World Evidence May Benefit Drug Development
Amy P. Abernethy, MD, PhD, Chief Medical Officer/Chief Scientific Officer, Flatiron Health, New York, noted that the goal of this meeting was threefold: to identify disease and drug candidates as potential case studies; to develop regulatory pathways for optimal use of real-world evidence in oncology, and to outline potential pilots that could be used to generate evidence that drug development using real-world data can be designed in a way that is sufficiently robust to be accepted by the FDA.
[Real-world evidence] can expand safety profiles and identify populations with increased benefit or risk for an approved therapy.— Amy P. Abernethy, MD, PhD
Tweet this quote
Dr. Abernethy described her vision for the future of real-world evidence. “It can expand safety profiles and identify populations with increased benefit or risk for an approved therapy. Pilot studies using real-world evidence can determine the potential correlation between real-world measures (eg, time to treatment switching) and more traditional clinical endpoints (eg, time to progression). Real-world evidence also can be used to build evidence for a supplemental package insert to expand indications, and it can support efficacy results in a traditional clinical trial, especially for breakthrough therapies.”
Maria Koehler, MD, PhD, Vice President for Oncology Strategy, Innovation, and Collaborations, Pfizer Oncology, highlighted the importance of acceptance of alternative sets of evidence for scientific, reimbursement, and regulatory purposes. “If evidence gathered from treatment in a local oncologist’s practice could be substituted for that gleaned from a traditional clinical trial and be accepted by both payers and regulators, it would transform the way we conduct clinical trials. In particular for certain drugs that have demonstrated breakthrough efficacy, we can accelerate our time to get these important therapies to patients by making the conduct of clinical trials more seamless and attractive to participants. Our goal is to ensure that patients have access to breakthrough treatments as quickly as possible.”
We are looking beyond safety and tolerability—although those will always be critical—to finding the most promising new drugs with as little disruption as possible to patients’ lives.— Maria Koehler, MD, PhD
Tweet this quote
She described crizotinib (Xalkori), which was given accelerated approval in August 2011 for treatment of locally advanced ALK-positive non–small cell lung cancer (NSCLC). The FDA required two additional phase III randomized trials, which were subsequently completed and confirmed the results in the original trial. Full approval was granted in November 2013, and 3 years later, crizotinib received another approval for use in ROS1-positive metastatic NSCLC.
She wondered if, in retrospect, a trial using real-world evidence, generated in oncologists’ existing practice without the need for burdensome clinical study requirements of additional tests and doctor visits by the patient, could have been appropriate as a confirmatory study and thereby potentially allowed more NSCLC patients who tested positive for the ALK mutation to have received crizotinib rather than being randomized to the standard-of-care arm in the confirmatory phase III trials. Following crizotinib approval, a retrospective cohort study using real-world evidence was conducted in the United States and Canada using medical record review of 212 patients with ALK-positive NSCLC who were given crizotinib as first- or later-line treatment in physicians’ offices based on the drug’s approval. The study provided further information on outcomes that supported the FDA-mandated phase III trials: Response and 1-year survival rates were similar.
“Whether real-world evidence could supplement or in specific circumstances, such as with drugs that have proven exceptional efficacy, even replace traditional requirements for post-market testing would depend on knowing whether real-world evidence is sufficiently robust to confirm clinical trial results. This research is ongoing and if it indeed does prove to be an accepted source of data, we might incorporate real-world evidence and pragmatic randomized trials into drug development earlier,” said Dr. Koehler.
Although research with cell lines has been valuable, until we see how real people respond, we cannot be confident that new drugs will be safe and effective.— Jane Perlmutter, PhD, MBA
Tweet this quote
Jane Perlmutter, PhD, MBA, a cancer survivor and Co-Founder of the Gemini Group, a consultant group in Ann Arbor, is interested in speeding clinical trials as well as identifying low probability and long-term side effects that are not usually evident in traditional trials. “In fact, they’re nearly impossible to find,” she said.
Drs. Koehler and Perlmutter agreed that patients’ perspective, well-being, and voice are vitally important to developing new treatments. Dr. Koehler said, “We are looking beyond safety and tolerability—although those will always be critical—to finding ways to develop the most promising new drugs with as little disruption as possible to patients’ lives.”
Dr. Perlmutter added, “Although research with cell lines has been valuable, until we see how real people respond, we cannot be confident that new drugs will be safe and effective.”
Data Quality and Utility
Janet Woodcock, MD, Director, FDA Center for Drug Evaluation and Research, spoke about real-world evidence in clinical trials.
Janet Woodcock, MD
“Retrospective data derived from clinical practice—for instance, registries, data from off-label use, and phase IV (postmarketing) trials—can eventually become prospective evidence. One potential problem, though, is the amount of data required; there has to be enough to create accurate and usable information, particularly in view of the ever-increasing number of cancer subsets.”
She and other panelists raised concerns about data sources, almost all of which have an effect on accuracy. Regarding quality, the following issues were identified:
What elements must be considered (and which details captured), and should these elements differ by data source, for example, types of electronic health records?
How should various databases (community vs academic) be considered with respect to the extractability of relevant fields?
How should quality be reported and presented for review, and which thresholds need to be considered?
What analyses need to be done to generate usable real-world evidence?
Then there are questions about how data are to be used. For example, what aspects of efficacy need to be captured, and what is the best way to consider appropriate endpoints and outcome measures? How are adverse events and other safety considerations to be collected and reported—by physicians in their daily activities, by patients, or by other means? Do data requirements change depending on regulatory need and use, and how will changing data characteristics be accommodated over time?
In addition, several electronic health record companies collect various types of health information, but they vary by disease and are often proprietary and not interchangeable. Now might be the time to determine whether any of the collected information can be tested to meet regulatory needs and to determine which data fields are needed to address a study question. In other words, what is actually necessary to conduct a real-world evidence trial and get approval?
Potential Problems
Using real-world evidence for clinical trials with an aim toward approval is not without potential problems. To begin with, data from a variety of sources are often not complete and accurate.
Moreover, necessary data (diagnosis and staging, histology, radiology, pathology, laboratory, treatment, demographics, biomarker status, adverse events) may not be available. Hospitals and other institutions use different electronic health record software systems, creating high levels of ambiguity and complexity, and therefore are problematic for trials. This has been somewhat ameliorated by programming and informatics, but there is still too much variability.
Furthermore, data quality is uneven. Some data may be missing, methods to extract data and audit trials vary, and information technology capabilities are highly heterogeneous, as are data captured by clinicians. Likewise, different clinical endpoints are associated with different methods of collection, reliability, and recording precision in electronic health records. For example, can tumor shrinkage be quantified using radiology reports, and if so, is this always a reliable endpoint?
Finally, informed consent will be needed because data sources will be identifiable. This will involve devising and implementing related policies.
Policy and Applications
Collecting real-world evidence can answer specific clinical questions as well as inform product labels in the following areas:
- Safety profiles
- Identification of populations with increased benefit or risk from a therapy
- Determination of potential correlation between real-world measures and more traditional clinical trial endpoints
- Evidence for a supplemental package insert to expand therapeutic indications
- Supporting efficacy results of a traditional trial, especially when a new drug shows substantial clinical benefit in an area of unmet need
Progression-free survival is a particularly difficult endpoint to assess. There is a 30% discrepancy, even in well-controlled trials. Overall survival is much more likely to reflect reality because simple endpoints are always the best.— Richard Pazdur, MD
Tweet this quote
Richard Pazdur, MD, Director, FDA Office of Hematology and Oncology Products (and named Acting Director of the FDA Oncology Center of Excellence after this meeting), spoke about real-world evidence and endpoints. “Progression-free survival is a particularly difficult endpoint to assess. There is a 30% discrepancy, even in well-controlled trials. Overall survival is much more likely to reflect reality because simple endpoints are always the best.”
Nevertheless, it is important to consider symptom benefit, patient-reported outcomes, and response rate, he added.
Dr. Abernethy added that real-world datasets can be used to generate summary endpoints based on existing information routinely captured from charts and electronic health records. “These endpoints can be tied to source evidence such as radiographic studies and pathology reports and are meaningful as long as they are based on a predefined validation framework.”
Nevertheless, she cautioned, there are complexities: interpretability of radiology reports, variable time points for disease assessment, multiplicity of evidence sources for a given clinical event, missing data, and potential for error. “Therefore, we need an approach that accounts for these complexities, that is replicable across abstractors, and is portable,” she said.
Cost of Care
Roy Beveridge, MD
Regardless of where data come from and how they are collected, they always have to be applied to actual people with cancer—and someone has to pay for it. Roy Beveridge, MD, Chief Medical Officer, Humana, said that payers are generally supportive of real- world evidence. “The majority of cancer patients are on Medicare; in general, these people have five or six comorbidities, and they take five or six different medications. They are not the same as younger, healthier people who tend to get different diseases. And since Medicare and other payers need to predict a year or two in advance which drugs will be available, it would be advantageous to know what kinds of data are being collected for trials.”
Jeff Helterbrand, PhD
Jeff Helterbrand, PhD, Senior Vice President, Global Head of Biometrics, Genentech/Roche, agreed in general but noted that the pharmaceutical industry is still tied to traditional clinical trials. “We know the process is too cumbersome and time-consuming and that change is needed. Often, by the time the first patient is enrolled in a trial, so much time has passed that the standard of care has moved on. We need new ways to find new agents. Real-world evidence might be part of the solution.”
Among other considerations for pharma, the utility of overall survival derived from real-world evidence is well established, as are other variables such as biomarker characteristics and treatment patterns. In addition, a progression endpoint can be defined using reproducible electronic health records data.
It is still unclear whether real-world progression-free survival is similar to Response Evaluation Criteria in Solid Tumors (RECIST)-defined progression-free survival obtained in a clinical trial. Real-world endpoints seemingly have the potential to support results obtained from a prospective trial. On the other hand, progression-free survival data obtained from nonblinded trials may be biased by physician beliefs and depend on frequency of imaging.
But how do we find the way forward? For example, in terms of progression, application of RECIST criteria requires a demonstrable change in tumor size based on radiologic evidence, whereas real-world progression could be based on a radiologist’s interpretation of scans or on a clinician’s interpretation of the entire patient chart—or a change in treatment that signals worsening disease.
Should disease burden always be assessed using RECIST? Most likely not, said Dr. Abernethy. Flatiron reviewed the charts of 24 patients with advanced NSCLC to determine whether RECIST can reliably define progression and tumor response in real-world electronic health records data.
“Using a strict definition of RECIST, none of the patients could be assessed by those means. But relaxing the definition (by allowing a trained abstractor to select the target lesion), 25% of the patients could. For the other 75%, the endpoint would have to be considered missing,” Dr. Abernethy explained.
Therefore, she continued, application of RECIST is not a practical solution. Several other approaches are possible—for example, clinician-confirmed progression and radiology-based events. “After conducting a series of experiments using real-world data, we found that we should focus on the former as the primary definition, with radiology and pathology data supplementary.”
Needs of the Patients
Joe V. Selby, MD, MPH
Joe V. Selby, MD, MPH, Executive Director, Patient-Centered Outcomes Research Institute (PCORI), Washington, DC, described PCORnet, a PCORI initiative designed to make clinical research faster, easier, and less expensive by using large amounts of stored data derived from patients—that is, real-world evidence. The goal, he said, was to “transform the culture of clinical research from one directed by researchers to one driven by the needs of patients.”
A PCORI initiative called the Clinical Data Research Networks comprises groups of health systems that conduct research. Dr. Selby explained that each will develop the ability to conduct randomized trials and observational comparative effectiveness studies using data from their own practices. The networks involve two or more health-care systems and include integrated delivery systems, academic medical centers, and safety-net clinics (such as Oregon Community Health Information Network, University of Kansas Medical Center, Vanderbilt University, Weill Medical College, and others). ■
Disclosure: Drs. Califf, Perlmutter, Pazdur, Selby, and Woodcock reported no potential conflicts of interest. Dr. Abernethy is Chief Medical Officer/Chief Scientific Officer of Flatiron Health; Dr. Koehler is Vice President of Oncology Strategy, Innovation, and Collaborations, with Pfizer Oncology. Dr. Beveridge is Chief Medical Officer of Humana, and Dr. Jeff Helterbrand is Senior Vice President, Global Head of Biometrics, Genentech/Roche.
Reference
