Study Finds Wide Variation in Quality, Content of Clinical Cancer Guidelines

Get Permission

Sandra L. Wong, MD, MS

Randomized clinical trials are supposed to be the gold standard, but even then, we’re interpreting results differently. It suggests the data can actually be controversial.

—Sandra L. Wong, MD, MS

What’s the best way to treat rectal cancer? Consult any of five top clinical guidelines for rectal cancer and you will get a different answer, according to a new study by researchers at the University of Michigan Comprehensive Cancer Center. Their findings were published online in the journal Cancer.1

They looked at clinical practice guidelines for rectal cancer from five highly regarded organizations in the United States, Europe, and Canada. The guidelines, which were all published within the last 6 years, were assessed for overall quality based on how they were developed. The tool used gives a percentage score based on six quality domains. Average scores for the rectal cancer guidelines ranged from 27% to 90%, suggesting wide variation in quality.

21 Points of Care Compared

A good amount of published data and randomized clinical trials exist to help guide best practices for rectal cancer treatment. The researchers took a deep dive into the guideline content itself, looking at 21 common points of care. They found that the five guidelines all agreed on only eight processes of care and that six recommendations were in direct conflict.

 “In this day and age of practicing medicine, particularly with cancer, physicians rely on these guidelines heavily. Our study suggests we need to be careful about that. The guidelines are of varying quality and they have varying recommendations. It’s not as easy as just viewing a guideline and following it,” said senior study author Sandra L. Wong, MD, MS, Associate Professor of Surgery at the University of Michigan Medical School.

Dr. Wong, a cancer surgeon, also serves on several guideline panels. She and her colleagues have previously published a study that found cancer guidelines do not fully meet Institute of Medicine standards for how they are developed.

Same Data, Different Recommendations

In the current study, the researchers found instances where guidelines cited the same published research but offered different recommendations.

“Guideline panels should be reviewing and assimilating data to help physicians understand what to do. Randomized clinical trials are supposed to be the gold standard, but even then, we’re interpreting results differently. It suggests the data can actually be controversial,” Dr. Wong said.

She urges physicians as well as patients to be aware of this variation in guidelines and not follow them blindly. In addition, organizations that produce guidelines should make it clear when expert consensus or opinion enters into the recommendations. ■

Disclosure: The study was supported by Agency for Healthcare Research and Quality grants T32 HS000053-22 and 1K08 HS20937-01, and National Cancer Institute grant 1K07 CA163665-22. The authors reported no potential conflicts of interest.


1. Abdelsattar ZM, Reames BN, Regenbogen SE, et al: Critical evaluation of the scientific content in clinical practice guidelines. Cancer. November 6, 2014 (early release online).