If a randomized, controlled trial of therapy for breast cancer was submitted for publication in which
1. The drug being tested was old and ineffective, and
2. prior to randomization, the women underwent a clinical breast examination and the study coordinators knew who had the largest cancers, and
3. allocation was on open lists so that it was possible to place the larger cancers into the treatment arm, and
4. a “statistically significant” excess of women with large cancers was assigned to the treatment arm, and
5. there was no benefit from using the drug, and
6. the study concluded that all chemotherapy was ineffective in reducing breast cancer deaths, the “study” would be rejected without any second thoughts, except that an ethics investigation might be initiated. Why is this obvious for a therapy trial, but such errors and violations are completely ignored by analysts when they actually happen in a major mammography screening trial?
Results Contrary to Other Trials
The trial in question is the Canadian National Breast Screening Study (CNBSS) whose recent update—reviewed in The ASCO Post, early release online—claims that mammography screening did not reduce deaths for women aged 40 to 59 years.1 These results are contrary to the other randomized, controlled trials of screening that have shown as much as a 31% decline in deaths2 for women offered screening (ie, intent-to-treat population) and an even greater reduction in deaths among women who were actually screened. The benefit has been confirmed in numerous observational studies.3-15
The ability of radiologists to find small cancers is directly related to the image quality of the mammograms. Unfortunately, a trial that should have tested the value of state-of-the-art mammography did just the opposite. The CNBSS employed old, often outdated, and even second-hand mammography devices.16 Scatter radiation clouds the images, which makes looking at them like looking through a soot-covered window. Grids to clear up scatter were not employed for most of the Canadian trial, so that the scatter could hide small cancers.
Proper positioning is required to image as much breast tissue as possible, yet there was no training for the technologists. The mediolateral oblique position allows the maximum amount of breast tissue to be imaged, but this was not employed in the CNBSS. Large volumes of breast tissue were not imaged, leading to missed cancers. The technologists and radiologists had no special training in mammography and mammographic interpretation.
Dubious Imaging Quality
I was invited to review the quality of the mammograms from this study with two other radiologists, and we found that the images were poor to unacceptable for a large part of the investigation.16,17 Their own reference physicist wrote:
“[I] identified many concerns regarding the quality of mammography carried out in some of the NBSS screening centers. That quality [in the NBSS] was far below state of the art, even for that time (early 1980’s).”18
Not a single radiologist involved in the CNBSS screening trial has defended its results.
Because of the poor quality of the mammograms, only 32% of the cancers in the mammography arm were not palpable—this should be at least 60%—and there was only a 2-mm difference in the average size of the cancers in both arms. Ineffective chemotherapy disqualifies a treatment trial. Ineffective mammography should disqualify the CNBSS results.
Fundamentals of Random Allocation
As oncologists well know, random allocation is critical in randomized, controlled trials. The CNBSS violated the fundamentals of random allocation. Every woman had a clinical breast examination before allocation. Women with breast masses and axillary adenopathy were identified to the study coordinators.19 Allocation was on open lists. The coordinator need only skip a line to insure that a woman with a lump was assigned to the screening arm. This would be unacceptable in a treatment trial; why is it acceptable in a screening trial?
The coordinators were never interviewed20 to find out if there was actual subversion, but the excess of women with advanced cancers allocated to the screening arm21 was statistically significant.22 Support for the notion that there was almost certainly compromise of the allocations is seen in the amazing survival of women in the CNBSS control group who only received usual care after an initial clinical breast exam. The 5-year breast cancer survival rate in Canada at the time was 75%, but women in the control arm had better than 90% 5-year survival. There is no way that this incredible survival could occur with “usual care.” It is clear that women with advanced cancers at allocation were moved into the screening arm. Furthermore, if cancers were shifted from the control group to the screened group, this could also explain the supposed “overdiagnosis” in the screening arm.
The CNBSS was a large trial, but larger numbers do not compensate for major trial errors. The U.S. Preventive Services Task Force and the American Cancer Society are reviewing their screening guidelines. The results of the CNBSS should be dropped from the reviews or viewed, at best, with great skepticism.
Disclosure: Dr. Kopans reported no potential conflicts of interest.
Dr. Kopans is Professor of Radiology, Harvard Medical School, and Senior Radiologist, Breast Imaging Division, Massachusetts General Hospital, Boston.
For more discussion of the Canadian National Breast Screening Study results, read the accompanying commentaries by Therese B. Bevers, MD, and Sudhir Srivastava, PhD, MPH, and Barnett S. Kramer, MD, MPH.
References
1. Miller AB, Wall C, Baines CJ, et al: Twenty five year follow-up for breast cancer incidence and mortality of the Canadian National Breast Screening Study: Randomised screening trial. BMJ 348:g366, 2014.
2. Tabár L, Vitak B, Chen TH, et al: Swedish two-county trial: Impact of mammographic screening on breast cancer mortality during 3 decades. Radiology 260:658-663, 2011.
3. Tabar L, Vitak B, Tony HH, et al: Beyond randomized controlled trials: Organized mammographic screening substantially reduces breast carcinoma mortality. Cancer 91:1724-1731, 2001.
4. Duffy SW, Tabar L, Chen H, et al: The impact of organized mammography service screening on breast carcinoma mortality in seven Swedish counties. Cancer 95:458-469, 2002.
5. Otto SJ , Fracheboud J, Looman CWN, et al: Initiation of population-based mammography screening in Dutch municipalities and effect on breast-cancer mortality: A systematic review. Lancet 361:1411-1417, 2003.
6. van Schoor G, Moss SM, Otten JD, et al: Increasingly strong reduction in breast cancer mortality due to screening. Br J Cancer 104:910-914, 2011.
7. Otto SJ, Fracheboud J, Verbeek ALM, et al: Mammography screening and risk of breast cancer death: A population-based case-control study. Cancer Epidemiol Biomarkers Prev 21:66-73, 2012.
8. Swedish Organised Service Screening Evaluation Group: Reduction in breast cancer mortality from organized service screening with mammography: 1. Further confirmation with extended data. Cancer Epidemiol Biomarkers Prev 15:45-51, 2006.
9. Hellquist BN, Duffy SW, Abdsaleh S, et al: Effectiveness of population-based service screening with mammography for women ages 40 to 49 years: Evaluation of the Swedish Mammography Screening in Young Women (SCRY) cohort. Cancer 117:714-722, 2011.
10. Coldman A, Phillips N, Warren L, et al: Breast cancer mortality after screening mammography in British Columbia women. Int J Cancer 120:1076-1080, 2007.
11. Kopans DB: Beyond randomized, controlled trials: Organized mammographic screening substantially reduces breast cancer mortality. Cancer 94:580-581, 2002.
12. Hofvind S, Ursin G, Tretli S, et al: Breast cancer mortality in participants of the Norwegian Breast Cancer Screening Program. Cancer 119:3106-3112, 2013.
13. Paap E, Holland R, den Heeten GJ, et al: A remarkable reduction of breast cancer deaths in screened versus unscreened women: A case-referent study. Cancer Causes Control 21:1569-1573, 2010.
14. Mandelblatt JS, Cronin KA, Bailey S, et al: Effects of mammography screening under different screening schedules: Model estimates of potential benefits and harms. Ann Intern Med 151:738-747, 2009.
15. Broeders M, Moss S, Nyström L, et al: EUROSCREEN Working Group. The impact of mammographic screening on breast cancer mortality in Europe: A review of observational studies. J Med Screen 19(suppl 1):14-25, 2012.
16. Baines CJ, Miller AB, Kopans DB, et al: Canadian National Breast Screening Study: Assessment of technical quality by external review. AJR Am J Roentgenol 155:743-747, 1990.
17. Kopans DB: The Canadian Screening Program: A different perspective. AJR Am J Roentgenol 155:748-749, 1990.
18. Yaffe MJ: Correction: Canada Study. Letter to the editor, J National Cancer Inst 85:94, 1993.
19. Bailar JC, MacMahon B: Randomization in the Canadian National Breast Screening Study: A review for evidence of subversion. Can Med Assoc J 156:193-199, 1997.
20. Kopans DB: NBSS: Opportunity to compromise the process. Letter to the editor. Can Med Assoc J 157:247, 1997.
21. Kopans DB, Feig SA: The Canadian National Breast Screening Study: A critical review. AJR Am J Roentgenol 161:755-760, 1993.
22. Tarone RE: The excess of patients with advanced breast cancers in young women screened with mammography in the Canadian National Breast Screening Study. Cancer 75:997-1003, 1995.
This commentary represents the views of the author and may not necessarily reflect the views of ASCO®.
