Sir Murray F. Brennan, MD
About 4 decades ago, as a young physician, I observed that most surgeons were numerator doctors; they remembered their successes and their failures, but they did not remember the frequency of either. There was no denominator. Worse, the approach to any specific surgical problem was always the same. It depended less on critical observation than on where the surgeon was trained. We believed what we were doing was right because that is “the way we always do it.” Was it true that failure to close the abdomen with interrupted number 2 silk would result in incisional hernia or even evisceration and possible death? I thought I knew that was not true, but would I risk my residency to challenge it? As a resident I awoke at night, fearful that any transgression from “the way we always do it” would cost me my residency and some patients their lives.
In fact, there are facts! We just didn’t have them then. The old approach sometimes worked and sometimes did not. Now we know that by recording in real time what we are doing, and the consequences of those actions, we can create databases and begin to ask what factors influence outcomes for better or for worse. Where the change is small, we can discuss what benefit is enough to justify surgery. Facts need to precede opinion, not the other way around.
Having convinced myself and others that we were making a difference—the ultimate confirmation bias—I had to prove that what I believed to be true was true. In most occasions, this meant a randomized controlled trial. My early attempts at single-institution randomized trials worked remarkably well, albeit most of them showed that what I believed to be true was not! Facts overwhelmed opinions.
A Denominator Problem Remains
Time moved on, and we no longer practice numerator medicine, but we still have a denominator problem. Now that we know the facts, how can we decide what constitutes a significant clinically relevant benefit? This is a great struggle.
For example, a 20% benefit in most trials of metastatic solid tumors would be of the order of 6 months of life; is that enough? How to make progress if I was to demand unrealistic improvement? As difficult as it was, if we were to improve some endpoint (local recurrence, disease-free survival, or even overall survival for patients who already had a 90% favorable outcome), was getting to 93% really important? If I had to enter my patients into a 2,000-patient trial to show a 2% or 3% benefit, was that justifiable? Even if the predicted result was to be statistically real, did I start the consent process by explaining to the patient, “If this trial succeeds you have 3 chances in 100 of being helped”? Or, “If this works, we would have to treat 100 patients like you for 3 to benefit?”
Scenarios in Melanoma and Pancreatic Cancer
I am ethnically at risk of melanoma, and my childhood environment compounded the risk, so with my melanoma, should I have a sentinel lymph node biopsy (SLN)? I read the trials for intermediate-risk melanoma: I have a 15% risk of the SLN biopsy being positive, and if I then had a completion lymph node dissection, a maximal 20% improvement in melanoma-specific survival, but no overall survival benefit. My optimistic (but not proven) benefit to initiate the SLN biopsy is 20% of 15%, or 3%. Should I proceed? I am saved the decision, only because my thin melanoma and my advanced age (is over 65 really advanced?) preclude me from being in the trial. The results of a clinical trial only apply to those entering the trial!
Facts need to precede opinion, not the other way around.— Sir Murray F. Brennan, MD
Tweet this quote
Do we really need a randomized trial in resected pancreatic cancer, with a 5-year mortality of 90%? What I need to see is easy—if you throw a brick out the window and it goes up, you do not need a trial. So far, for that disease in those circumstances, no brick has ever failed to fall. As a surgeon, can I justify a placebo operation? The incision would need to be minimal, although it has been done in other countries. We do trials against regimens that have previously been shown to be marginally effective (ie, a clinical placebo, with side effects)!
Statistical Manipulations
Of greater concern, when do we address statistical manipulations? Should I be upset to see survival, particularly progression-free survival, being measured to 2 decimal places of a month? If 0.1 of a month is 3 days, 0.01 of a month is 7 hours! How ludicrous that appears, but statistical improvement is declared—and published—and the adjuvant approved by the U.S. Food and Drug Administration.
Are we correct to declare a positive clinical trial as the new standard of care? How often must we remind our trainees that the benefits of a positive clinical trial accrue only to the patients who enter the trial? The results may or may not have applicability to the population. How carefully we must screen the entrance criteria.
How often must we remind our trainees that the benefits of a positive clinical trial accrue only to the patients who enter the trial?— Sir Murray F. Brennan, MD
Tweet this quote
But allowing math to kill innovation is not the answer. We can enrich populations most likely to benefit by intervention, using mathematical models, such as nomograms, prognostic indicators, molecular diagnosis, and, dare I suggest it, validated recorded clinician experience.
We need to maximize the positive numerator and minimize the denominator. We need to apply the math before, not after, the medicine. ■
Dr. Brennan was Chairman of Surgery at Memorial Sloan Kettering Cancer Center for 21 years and is now Senior Vice President of International Programs and Director of the Bobst International Center, Memorial Sloan Kettering Cancer Center, New York.
Disclaimer: This commentary represents the views of the author and may not necessarily reflect the views
of ASCO or The ASCO Post.
