For years, doctors around the country taking an exam to become board certified in radiology have cheated by memorizing test questions, creating sophisticated banks of what are known as “recalls,” a CNN investigation has found.
The recall exams are meticulously compiled by radiology residents, who write down the questions after taking the test, in radiology programs around the country, including some of the most prestigious programs in the U.S.
“It’s been going on a long time, I know, but I can’t give you a date,” said Dr. Gary Becker, executive director of the American Board of Radiology (ABR), which oversees the exam that certifies radiologists.
Asked if this were considered cheating, Becker told CNN, “We would call it cheating, and our exam security policy would call it cheating, yes.”
Radiology residents must sign a document agreeing not to share test material, but a CNN investigation shows the document is widely ignored. Dozens of radiology residents interviewed by CNN said that they promised before taking the written test to memorize certain questions and write them down immediately after the test along with fellow residents.
“Our real mission is to the public,” said Dr. James Borgstede, the ABR’s president-elect. “Our real mission is to say that your certified radiologist has demonstrated, acquired and maintained the requisite skills and knowledge to practice with skill and safety on the public.”
To become board certified, a virtually essential step to obtain hospital privileges, residents have been required to pass two written exams and an intensive oral test during five years of residency training.
The practice of sharing exam answers is so widespread and considered so serious in the medical community that the ABR has put out a strongly worded video warning residents that the use of recalls must stop.
“Questions and answers have been memorized, sometimes verbatim, and contributed to extensive archives of old ABR test material that become the prize possessions of many residency programs,” Becker said in the video, which appears on the board’s website.
He said “accumulating and studying from lists of questions on prior examinations constitutes unauthorized access, is inappropriate, unnecessary, intolerable and illegal.”
About half of the questions on the radiology test are the same each year.
“The half of the exam that is not new comprises questions from a variety of previous exams, not from the prior year,” Becker told CNN.
“As always, the assembly of the exam is governed by the blueprint created by the ABR for that particular examination. All questions are reviewed for currency before being reused.”
Read Becker’s statement to CNN about ABR exam security (PDF)
Outside of the world of radiology, it’s rare for residents to speak openly about the recalls. But one doctor at a military program revealed how the recalls worked to CNN. Dr. Matthew Webb, an Army captain, trained at the San Antonio Uniformed Services Health Education Consortium (SAUSHEC), an Army/Air Force program that includes the renowned Brooke Army Medical Center.
“Cheating is the ultimate betrayal of trust to patients, and it’s also the most egregious and flagrant violation in academia,” Webb told CNN. “I got to where I was based on my own personal achievements, learning and educating myself. To have to take an exam up against others who have been cheating is … unfathomable.”
The recalls obtained by CNN show at least 15 years’ worth of test questions and answers, some prepared as PowerPoint presentations. The tests were available on a radiology residents’ website as well as a shared military computer server.
See an example of part of a recall exam (PDF)
Webb complained about the recalls to the ABR, which conducted an investigation.
“We’re outraged by this, and we took this case to our professionalism committee,” Becker said. “The results of the deliberations there and the decision of the board was to go directly back to the training director, the dean of the institution, and we’ve had those discussions.”
Webb, 31, said he failed the first radiology written exam, which focuses on physics, in the fall of 2008. He said the program director at the time, Dr. Liem Mansfield, told him to use the recalls in order to pass.
“He told me that if you want to pass the ABR physics exam, you absolutely have to use the recalls,” Webb said. “And I told him, ‘Sir I believe that is cheating. I don’t believe in that. I can do it on my own.’ He then went on to tell me, ‘you have to use the recalls,’ almost as if it was a direct order from a superior officer in the military.”
Reached for comment, Mansfield referred CNN to the Brooke Army Medical Center’s public affairs office.
In a statement, the military admitted that some faculty members and program directors “were aware of the use of recall examination questions by residents. A smaller number of faculty and a past program leader encouraged the use of recall questions as one of several tools to improve medical knowledge and prepare for the exam. Faculty members did not organize or run the recall question activities.”
“All faculty are now aware that SAUSHEC leadership will not tolerate support or use of recalled questions” in light of the American Board of Radiology’s crackdown, the statement said.
Dr. Woodson Jones, dean of SAUSHEC, also said in a statement to CNN that “radiology residents’ past practice of using recalled examination questions to study reflected the practice within radiology programs across the country and did not represent an unfair advantage.”
The program had the recalls removed from its computers and required residents to sign a document that they would not use them.
“Our residents were concerned we were now putting them at a disadvantage compared to the remainder of the country,” Jones told CNN in a phone interview. “Our perspective on this is it doesn’t matter. We are going to exhibit the highest level of professionalism.”
Jones said that “by looking at what has happened, we took very aggressive action to ensure that we lived up to the policy released by the (American Board of Radiology) as far as the new culture they were trying to generate across the country by having all of our residents sign attestation statements they would not use recalled exam questions, and they would destroy any exam questions they were aware of.”
Graduates of the SAUCHEC program contacted by CNN would not discuss the recalls.
Webb said he doubted anything would have been done if he had not complained to the ABR and the military about the recalls.
“This is not a gray area. This is absolute, definitive cheating. If we were in middle school, high school, college or anywhere else in academia, they would call it cheating,” Webb said.
Jones, in his statement to CNN, said, “No adverse action has been taken against a radiology resident for revealing the use of the recalled questions. However, a resident was terminated from training by SAUSHEC in June 2011 for failure to meet professionalism standards after extensive due process procedures were followed. These termination procedures took place prior to our learning of complaints by a resident regarding the use of recalled examination questions by radiology residents.”
The actions the SAUSHEC program has taken “meet the expectations of the ABR,” according to a letter sent to Jones from Jennifer Bosma, the ABR’s associate executive director for administration. “Furthermore, they are fully aligned with the direction given by the ABR’s professionalism committee following review of the whistleblower report. While follow-through at every level of program leadership will naturally be key to successful cultural change at SAUSHEC and every residency training program, the ABR is satisfied that the progress reported to date conforms with the intent of the new Examination Security Policy.”
Meanwhile, Webb has had his own problems with the Army. He was reprimanded last year for making “sexual comments” to another doctor and for “other conduct unbecoming an officer.” That led to his firing from the radiology program. Webb said the issue was a personality dispute that escalated.
Webb said his superiors called him into a meeting in December to ask what he was telling CNN.
In a statement, the military said that Webb “was asked by the hospital commander if he had recently had any inquiries from and/or dialogue with CNN, and the commander stated that the soldier wasn’t prohibited from expressing his personal views and responses, but that he needed to alert command if he had been contacted.”
To investigate the use of the recalls, CNN attended the Radiological Society of North America’s annual conference in Chicago, which is the country’s largest medical convention, drawing about 60,000 participants each November. In interviews over several days, residents and practicing radiologists confirmed that recalls have been widely used in most, if not all, radiology programs for more than a decade.
“From my understanding, I would say nationwide from my friends across the country who are all in the same stages of training throughout the years, everyone gets a group,” said Dr. John Yoo, a practicing radiologist. “People decided beforehand what sections I will focus on, in terms of trying to recall those questions and answers. And then immediately after the examination, the residents get together and try to put these down onto paper or word processor to be able to share it with the classes coming behind you.”
Asked if he considered it cheating, Yoo said, “I think that is a difficult question to answer, to be honest, because it is a matter of survival, so to speak. For better or worse, if you fail to jump over that barrier, you get, I don’t know, like ‘The Scarlet Letter,’ branded as, ‘Hey, this person didn’t succeed in passing a certain section.’ So it is sort of out of necessity to pass these examinations that you have to rely on the recall.”
Yoo said the recalls are used primarily as study guides.
“I can only speak for myself in that I’ve used these to serve as a guideline,” Yoo said. “These are subjects I need to know cold in order to pass this examination. I’ve used them as a guideline. I think many people have used them as a guideline.”
Dr. Joseph Dieber, another radiologist, said he considered the use of the recalls a “gray area.”
“If the questions were somehow reproduced with a camera and it was the exact question, and you were trying to memorize A, B and C, you know A for this answer, B for this answer, then yeah, it is cheating. You’ve got to ask yourself what the goal of the exam is. If the goal of the exam is to have you learn the information in that question and to make sure that you’re producing radiologists that are competent to handle the kinds of issues that come up, then yeah, if people are studying recalls, they learned that question. They learned everything about it.”
Dieber said the recalls are necessary because of the test itself.
“If they had a test where you could study relevant radiology knowledge and they tested on it, that would be fine,” Dieber said. “Part of the problem is the test and the questions that they ask. Because some of the questions are so obscure, that unless you know that they like to ask questions about that topic, you’re not going to study it because some of them are completely irrelevant to the modern practice of radiology.”
Radiology programs around the country are now warning residents that the recalls cannot be used.
“We have made it clear to the residents in written form that recalls, not only are they not encouraged, but they are not permitted. As far as we’re concerned, they are not appropriate,” said Dr. Carolyn C. Meltzer, who oversees the radiology and imaging sciences department at Emory University’s School of Medicine, which is one of the largest programs in the U.S.
Radiology programs around the country are now warning residents that the recalls cannot be used.
Dr. Anne Silas, chairwoman of the diagnostic radiology department at Dartmouth-Hitchcock Medical Center, where residents told CNN that recalls were widely used, said they are not condoned.
“In general and in the past, recalls have provided a framework for studies, a study guide, but none of it is coming from a program level; it is not something provided for anyone,” she said.
Asked if using recalls is considered cheating, Silas said, “I don’t believe it would be fair to accuse our trainees of cheating on that basis.”
At Massachusetts General Hospital, which is affiliated with Harvard Medical School, residents also told CNN that they routinely used recalls for years.
Dr. James Thrall, the hospital’s chief of radiology, told CNN he was not personally aware of the use of recalls at the hospital, but said if they were used, it was not endorsed by the faculty.
“I have a problem with the recall system if the exam is not changed. But if the exam is substantially changed from one year to the other, then the recalled questions really become in my view more of a learning resource,” Thrall said.
Asked about the ABR’s efforts to crack down on the recalls, Thrall said, “I think it’s going to be very difficult to police it.”
Dr. King Li, chairman of the department of radiology at The Methodist Hospital in Houston, said he does not endorse the use of recalls, but acknowledges it is difficult to stop because residents have used them for so long.
“It’s not OK when everybody is doing it, but the people who are doing it will use this as an excuse to make it acceptable to themselves,” he said.
Asked about someone like Webb, who is speaking out about the recalls, Li said, “If you are someone who doesn’t want to use the recalls on your exams, you may been seen as someone who is aloof and too good for his practice, and also potentially a whistle-blower. So that can make this person uncomfortable in the peer group.”
Dr. Kevin Weiss, president and CEO of the American Board of Medical Specialties, said he had not heard about anything similar to the radiology testing issue in other medical specialties. The ABMS is the umbrella group for 24 boards and 152 specialties and sub-specialties.
“The concern is sharing questions is ever-present. Our board takes great pains to make sure exam securities are well managed,” he said.
The issue of test sharing arose in 2010 when the American Board of Internal Medicine suspended 139 doctors for sharing test questions with an exam review company.
In a statement, Dr. Christine Cassel, president and CEO of the American Board of Internal Medicine, said, “Sharing test questions from memory is a serious problem that threatens the integrity of all standardized testing. Test takers need to know that this kind of ‘brain dumping’ is grossly unethical and the American Board of Internal Medicine will not tolerate unethical behavior from physicians seeking board certification. ABIM will take appropriate action against anyone who seeks to compromise the integrity of our examinations.
Most of the cases against the suspended doctors have been resolved. Drew Wachler, an attorney who handled many of those cases, told CNN, “The cases were really resolved through settlement agreements with ABIM.”
“We articulated the impact on these physicians,” Wachler said. “We tried to present their body of work throughout their entire careers, looking at the extraordinary references they received, their conduct throughout medical school, residencies. It was resolved in that fashion.”
At least one doctor who was suspended said she did nothing wrong. The ABIM suspended and sued Dr. Sarah Von Muller, accusing her of sharing test questions with the test review company. Von Muller denies the charges.
“I don’t know what kind of settlements occurred with the other doctors and ABIM,” she told CNN.
“But under no circumstance am I willing to sign a paper stating that I cheated or that I copyrighted their material, because I didn’t.”
She added that “a lot of people would just roll over and do whatever they had to do to get out of this mess, but I’m not going to do that. I’m going to fight because this is more — this is principle, and this is wrong, and if I don’t stand up to the playground bully and fight this, no one else will.”
The ABIM declined to comment because of the pending litigation.
The ABR’s Becker said that despite the use of the recalls, the public is protected because of the overall training and an intensive oral exam that residents must undergo to become certified.
Next year, the board is rolling out a new exam for the first time in more than 10 years. Instead of two written tests and one oral exam, the first exam will be a “core exam” taken after three years of residency training, and the second certifying exam taken 15 months after graduation. The oral exam is being eliminated.
“ABR is replacing its proven and time-tested exam format, which includes an oral exam, with new, highly standardized computer-based exams that will be more objective and better able to assess the abilities of the physician examinees, without measuring extraneous factors not uncommon when candidates take the orals, such as nervousness,” Becker said.
As far as the recalls, the only formal complaint to the ABR has been about the military program in San Antonio, Becker said.
“Now I could say we don’t have any more information on other programs, we haven’t heard similar reports from other residents, but if and when we ever hear of any, we’re going to track them down,” he said.
It's Interesting 