The questions I am challenged to answer are twofold: (1) Should you attend the 2023 ASCO Annual Meeting? (2) If you attend, how is your time best spent?
Most people agree scientific conferences are important venues for cancer researchers and clinicians to share and discuss research findings, exchange ideas and insights, and network for collaboration and career development.
The first question is easily answered: Yes, you should attend the 2023 ASCO Annual Meeting! However, whether this should be in person or virtually is a complex question discussed in detail elsewhere.1,2
Assuming you’re in Chicago or online, how best should you spend your time? Attending plenary sessions, presidential symposia, and educational programs are almost always a good investment. The challenge is in apportioning your time in specialized scientific sessions. Veterans will recall running between sessions (no easy feat in the McCormick Place; skip the morning gym) to catch presentations of data from the latest phase I and II studies as well as preliminary results from phase III studies. But is this really the best use of your time at the meeting?
The answer depends in large part on the reproducibility and replicability of conclusions of the studies you listen to. As such, it’s reasonable to review the scorecard. How many phase I and especially phase II studies that claim feasibility or encouraging preliminary results make it to phase III? What proportion of phase III studies are published in a peer-reviewed journal? And, lastly, what proportion of published phase III studies are validated?
From Presentation to Publication
These questions are addressed in many articles. For example, about one-third of 768 presentations at meetings of British surgical societies were published within 2 years of presentation.3 Success rates of oral presentations were about twice that of poster presentations. The average impact factor—a measure of the frequency with which a journal’s articles are cited—of journals where successful abstracts were published was about 3, and 95% were in journals with an impact factor of less than 6.
I suggest selecting a few scientific presentations of high quality focused on your field of interest. Use your remaining time to think critically about the presentations you attend.— ROBERT PETER GALE, MD, PhD
Tweet this quote
Our Dutch surgery colleagues did a bit better, with about two-thirds of abstracts making it to publication.4 Interestingly, there was relatively little difference in the proportion of accepted vs rejected abstracts eventually published. So, take solace if your abstract is rejected; it’s not the end of the world. Everything can be published somewhere.
(Well, almost anything.)
Some readers may be thinking this spotty track record applies to surgery meetings alone. Not so. An analysis of 578 abstracts presented at the 2009 Society of General Internal Medicine Annual Meeting indicated a subsequent publication rate of less than 50%.5 Unsurprisingly, phase III studies had higher publication rates compared with other study designs, 67% vs 46%. In contrast, studies with positive results were not associated with higher publication rates compared with studies with negative results (odds ratio = 0.89; 95% confidence interval = 0.6–1.31). And there are many other examples.
There is little question that peer review is reasonably successful in identifying high-quality studies for publication.6,7 However, there are fewer data on the value of peer review of submitted meeting abstracts. One might rightly wonder what the impact of peer review of abstracts has on the probability of subsequent publication. A study of this question using a data set of 702 submitted abstracts reported little predictive validity of peer review and probability of subsequent publication.8 Others report similar data.9
What About for Oncologists and Hematologists?
The next caveat for many readers is the illusion discordances between meeting presentations and subsequent publication, which may operate for surgeons and internists but certainly not for oncologists and hematologists. Think again.
Recently, Cucchi et al published data on abstracts presented at the American Society of Hematology (ASH) Annual Meeting & Exposition in 2013, 2014, and 2015.10 They looked at the likelihood of abstracts reporting positive results of a phase II study advancing to a confirmatory phase III clinical trial. The authors’ analyzed 292 abstracts covering diverse hematologic cancers where the abstract characterized the results as “encouraging, promising, could represent a novel therapeutic option, or warrants investigation in a randomized trial.” (Sound familiar?) Although 82% of these abstracts were subsequently published, besting our surgery and internal medicine colleagues, most publications reflected the abstract content, often with the same foggy conclusion. The triumph of hope over reason perhaps. Of note, less than 50% of positive studies advanced to a phase III study.
In a literature review of new drugs for acute myeloid leukemia, the authors found very few entered clinical practice. (A prior study estimated the success rate at less 5%.11) I’m unaware of similar analyses of publication of abstracts presented at the ASCO Annual Meeting, but this should be done. Results are likely to be similar to those reported for the ASH Annual Meeting & Exposition.
Troubles with reproducibility and especially replicability are not limited to meeting abstracts and presentation; they are common in publications in peer-reviewed articles in high–impact factor journals. Ioannidis, Prasad, and colleagues and others have analyzed the reliability of conclusions of these types of articles extensively.12,13 For example, Ioannidis reviewed 49 highly cited (> 1,000 citations) clinical research studies published in NEJM, JAMA, The Lancet, JNCI, JCO, Blood, and others. A total of 45 claimed the intervention was effective; 7 were contradicted by subsequent studies; 7 others reported effects stronger than observed in subsequent studies; 20 conclusions were replicated; and 11 conclusions were unconfirmed. Five of six highly cited nonrandomized studies reported stronger effects or were contradicted in subsequent analyses vs 9 of 39 randomized controlled trials. Caveat emptor.
Recommended Plan of Action
My advice? First, you should attend the 2023 ASCO Annual Meeting. Whether you do so in person or virtually depends on many considerations. Either way, benefits will be substantial, but some depend on which option you chose. Schmoozing with colleagues in the lobby bar of your hotel is not an option for virtual attendees, but they may benefit from not madly racing around the conference center to catch every potentially interesting presentation, waiting 30 minutes in line to get a venti nonfat latte, or paying $15 for a small glass of pinot noir of uncertain provenance.
A word of caution, especially for junior colleagues and colleagues in training: If you dash around trying to catch every potentially interesting presentation at the meeting, much of what you will hear, especially the results of phase I and II studies, is unlikely ever to be published in a peer-reviewed journal, advance to a phase III study, and even less likely to enter clinical practice. That is the case even when the presenter describes the results as “encouraging, promising, could represent a novel therapeutic option, or warrants investigation in a randomized trial”—or words to that effect to quote lawyers. In fact, some of what you hear is likely to be wrong and set your thinking back rather than forward.
I suggest selecting a few scientific presentations of high quality focused on your field of interest. Use your remaining time to think critically about the presentations you attend. The best outcome, of course, is to grab the presenter after the session and torture him/her with questions. Remember, good scientists look for evidence that their hypotheses are wrong, not right.
What else to do if you go to the meeting? Enjoy chatting with mentors, colleagues, and friends. See Chicago, one of our most important cities architecturally, home of the first skyscrapers (see the Monadnock Building, those of Mies van der Rohe, the Willis Tower nee Sears Tower [how the mighty have fallen], and several Frank Lloyd Wright homes like the Robie House), a boat trip on the Chicago River, and dinner at Topolobampo, probably the best Mexican restaurant in the United States (why is it in Chicago?). See you soon at the 2023 ASCO Annual Meeting.
DISCLOSURE: Dr. Gale has received research support from the UK National Institute of Health Research funding scheme.
REFERENCES
1. Sarabipour S, Khan A, Seah YFS, et al: Changing scientific meetings for the better. Nat Hum Behav 5:296-300, 2021.
2. Wallon G, Bendiscioli, Asif A, et al: The values of scientific conferences in a virtual framework: Analysis and practical options. Available at chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.embo.org/documents/courses_and_workshops/reports/values_of_scientific_conferences_in_a_virtual_framework_report.pdf. Accessed July 5, 2022.
3. Weale AR, Edwards AG, Lear PA, et al: From meeting presentation to peer-review publication: A UK review. Ann R Coll Surg Engl 88:52-56, 2006.
4. de Meijer VE, Knops SP, van Dongen JA, et al: The fate of research abstracts submitted to a national surgical conference: A cross-sectional study to assess scientific impact. Am J Surg 211:166-171, 2016.
5. Egloff HM, West CP, Wang AT, et al: Publication rates of abstracts presented at the Society of General Internal Medicine Annual Meeting. J Gen Intern Med 32:673-678, 2017.
6. Bornmann L, Daniel HD: The usefulness of peer review for selecting manuscripts for publication: A utility analysis taking as an example a high-impact journal. PLoS One 5:e11344, 2010.
7. Jackson JL, Srinivasan M, Rea J, et al: The validity of peer review in a general medicine journal. PLoS One 6:e22475, 2011.
8. Scholcoff, C, Sanghani, P, Jackson W, et al: Peer review of abstracts submitted to an internal medicine national meeting: Is it a predictor of future publication? J Gen Intern Med 33:1002-1003, 2018.
9. Deveugele M, Silverman J: Peer-review for selection of oral presentations for conferences: Are we reliable? June 2017. Patient Education and Counseling 100:2147-2150, 2017.
10. Cucchi DGJ, Polak TB, Ossenkoppele GJ, et al: The predictive value of a positive phase 2 ASH abstract for peer-reviewed publication and progression to phase 3. Blood 139:1920-1923, 2022.
11. Walter RB, Appelbaum FR, Tallman MS, et al: Shortcomings in the clinical evaluation of new drugs: Acute myeloid leukemia as paradigm. Blood 116:2420-2428, 2010.
12. Prasad VK, Cifu AS: Ending Medical Reversals: Improving Outcomes, Saving Lives. Baltimore, MD: Johns Hopkins University Press; 2015.
13. Ioannidis JP: Contradicted and initially stronger effects in highly cited clinical research. JAMA 294:218-228, 2005.
Dr. Gale is Visiting Professor of Haematology at Imperial College London and a Foreign Member of the Russian Federation and China Academies of Science and Medical Science. Photo courtesy of Patricia Williams.
