Jemez Mountains, northwest New Mexico (oil), by Ralph C. Williams Jr.
In response to the COVID-19 pandemic, all rheumatology fellowship interviews conducted in the 2020 fellowship recruitment season were conducted in a virtual, video-based format to reduce the risks associated with travel and face-to-face meetings.
In June 2020, the Coalition for Physician Accountability released guidance suggesting all programs commit to online-only interviews and recruitment events, even for local applicants.1 Electronic Residency Application Service (ERAS) timelines were adjusted, increasing the time applicants had to complete applications and upload documents, and slightly decreasing the time programs were given to review applications and interview candidates prior to submitting a rank list. Programs and applicants had to rapidly adapt to a new recruitment process with relatively short notice.2
The Association of American Medical Colleges (AAMC) released a document with tips for setting up and conducting virtual interviews, and some institutions and graduate medical education offices created local resources for programs and faculty. Despite rapid efforts to adapt, many program directors had concerns about potential changes to the applicant pool, their effectiveness in communicating their programs’ strengths and unique culture in a purely online format, and their ability to assess applicants without an in-person meeting.
However, in addition to alleviating safety concerns associated with the pandemic, virtual recruitment had a number of potential benefits, especially for applicants—including reduced costs and time away from current training programs—that could enable candidates to interview with programs that might otherwise be out of reach.
Reflecting on the challenges and successes of the 2020 recruitment season could help us plan for future adaptations to fellowship recruitment.
Virtual Recruitment Survey
The ACR’s Committee on Rheumatology Training and Workforce Issues (COTW) made the decision to conduct the Rheumatology Program Director 2020 Virtual Recruitment Survey with the aim of better understanding how rheumatology fellowship recruitment was affected across the country from the perspective of program directors. The COTW plans to survey applicants in the future.
The survey was distributed via email to all 149 adult and 39 pediatric rheumatology program directors in February 2021, and responses were received from 73 adult (49%) and 16 pediatric (41%) program directors.
The survey included questions regarding the applicant pool, informational resources, interview activities, assessing applicants and future directions, and allowed for open-ended comments. The survey results were presented at the ACR Division & Program Directors Conference, which took place March 18–20. Comments gathered at the time of the presentation were also reviewed for additional qualitative input.
Survey Results
Applicant pool—Most adult programs received the same number of, or more, applications in 2020 compared with 2019, with 58% of adult program directors reporting similar volume and 33% reporting increased volume of applications received. Pediatric programs experienced more variability, with more even distribution among lower, similar and higher volume of applications received.
A similar pattern was reported regarding the number of applicants interviewed per position, with 56% of adult programs interviewing a similar number and 38% interviewing more applicants per position, while pediatric programs experienced variability, with near even distribution among reduced, similar and increased applicants interviewed per position.
Informational resources & interview activities—Programs committed substantial personnel, time and other resources to adapt to the recruitment changes necessitated by the pandemic: 75% of both adult and pediatric programs generated new online resources for applicants in 2020. These resources included virtual tours of program facilities, clinics and hospitals, and recorded introductions by program directors and faculty. Some institutions also provided video tours of their city or region to highlight local attractions for applicants who might not be familiar with the area. Social media presence was also emphasized by many programs and institutions.
Zoom was by far the most common video conferencing platform; more than 81% of adult and 88% of pediatric programs used Zoom to conduct virtual interviews.
Most programs (71% adult; 78% pediatric) used a one-on-one interview format to conduct applicant interviews with faculty.
The length of the interview day decreased for many programs, although otherwise there were no major changes for most programs with regard to the number of faculty involved in interviews, the use of multi-
faculty interviews or time per interview. The time commitment for faculty and fellows decreased in many pediatric and some adult programs, although the administrative time commitment increased in nearly half of adult and pediatric programs.
Nearly half of all adult and pediatric programs reported increased challenges coordinating interview day activities, although challenges coordinating scheduling interviews with applicants and general correspondence with applicants were otherwise unchanged.
Some programs described a positive experience using Thalamus GME interview scheduling software. Programs considered the ability to connect interviewees with additional faculty with shared interests on days other than the interview day, if desired, an added benefit of virtual recruitment.
Assessing applicants—A majority of adult and pediatric program directors reported increased challenges assessing applicants’ levels of interest, fit, interpersonal skills, and professionalism.
Hybrid systems, in which some applicants visit in person while others interview virtually, may also become an option, in which case additional strategies to balance optimal information sharing with bias reduction will become more salient.
Discussion
Based on the results of this survey, the virtual application season was relatively smooth, with fewer challenges and disruptions than might have been expected given the rapid and profound change to previously well-established workflows. Programs noted some increase in administrative time and in efforts to create new informational resources for applicants, although scheduling did not seem unmanageable, and faculty and fellow time for interviews was relatively unchanged.
One common issue noted by many program directors involves applicants who might apply during a virtual interview season but who might not otherwise apply or accept an interview in an in-person format. This has the benefit of potentially increasing a program’s diversity, if it enables qualified and interested candidates to find a program in which they will thrive that they might otherwise be unable to consider seriously due to travel costs or time away from residency training.
However, some program directors expressed concern about investing time and effort in reviewing applications and interviewing candidates who might not be seriously considering their program. To address this, some organizations have advocated for limiting the number of applications any individual candidate could submit or the number of interviews they could accept, or by implementation of preference-signaling mechanisms.3 Such limits depend on applicants accurately assessing information about programs prior to the interview, but may improve the ability of programs to do a thorough and holistic review of a more targeted and smaller group of applications.
ERAS data demonstrate that fellowship applicants to most internal medicine subspecialties have been gradually increasing over the past several years, a trend previously noted for rheumatology fellowships. Although that trend continued between the 2020 and 2021 cycles, the percentage increase in applicants to rheumatology fellowships overall and in applications per applicant was similar to those seen in previous cycles, suggesting the applicant pool in general did not change substantially.4
Survey respondents seemed to support this, noting little change in their match results.
Rheumatology educators clearly have an interest in improving the process for virtual recruitment in the future—virtual recruitment will likely remain an option in future years—and a number of innovative solutions have been proposed.
Many program directors expressed support for development and dissemination of best practices, such as the creation of standardized interview questions to optimize our ability to accurately assess candidates and reduce bias. Capping the number of interviews per applicant, creating preference-signaling mechanisms or coordination of interview release dates may be a way for applicants to reduce the number of interviews they attend; these strategies might apply in either a virtual or in-person recruitment environment.
Hybrid systems, in which some applicants visit in person while others interview virtually, may also become an option, in which case additional strategies to balance optimal information sharing with bias reduction will become more salient.
Although the current timeline for rank lists makes it unattractive, conducting interviews during ACR Convergence (i.e., the ACR’s annual meeting) could be a way to reduce costs for applicants and allow for in-person interaction. Delaying applicant rank lists to allow time for “second looks” at institutions after program rank lists have been submitted may be another way to address the challenges of a hybrid recruitment system. These conversations will undoubtedly continue, and new suggestions will arise as well. Hopefully, the efforts put forth in 2020 and subsequent adaptations will result in improvements to recruitment across all disciplines in the future.
Adam Kilian, MD (@KilianMD), is an assistant professor of medicine and a rheumatologist at the Saint Louis University School of Medicine, Missouri.
Bethany Marston, MD, is an associate professor of pediatrics and medicine, the Pediatric Rheumatology Division director and the director of the Rheumatology Fellowship Program, University of Rochester School of Medicine, New York.
References
- Recommendations for away rotations and interviews for graduate medical education fellowship applicants during the 2020–2021 academic year. 2020.
- Kilian A, Dua AB, Bolster MB, Annapureddy N, Seo P. Rheumatology fellowship recruitment in 2020: Benefits, challenges, and adaptations. Arthritis Care Res (Hoboken). 2021 Apr;73(4):459–461.
- Hammoud MM, Standiford T, Carmody JB. Potential implications of COVID-19 for the 2020–2021 residency application cycle. JAMA. 2020 Jul 7;324(1):29–30.
- Huppert LA, Santhosh L, Babik JM. Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years. JAMA Netw Open. 2021 Apr 1;4(4):e218199.
