Archive for the ‘scientific fraud’ Category

fraud-kit

The FDA in 2011 announced years’ worth of studies from a major drug research lab were potentially worthless, but it has not pulled any of the compounds from the market nor identified them

By Rob Garver, Charles Seife and ProPublica

On the morning of May 3, 2010, three agents of the Food and Drug Administration descended upon the Houston office of Cetero Research, a firm that conducted research for drug companies worldwide. Lead agent Patrick Stone, now retired from the FDA, had visited the Houston lab many times over the previous decade for routine inspections. This time was different. His team was there to investigate a former employee’s allegation that the company had tampered with records and manipulated test data. When Stone explained the gravity of the inquiry to Chinna Pamidi, the testing facility’s president, the Cetero executive made a brief phone call. Moments later, employees rolled in eight flatbed carts, each double-stacked with file boxes. The documents represented five years of data from some 1,400 drug trials.

Pamidi bluntly acknowledged that much of the lab’s work was fraudulent, Stone said. “You got us,” Stone recalled him saying.

Based partly on records in the file boxes, the FDA eventually concluded that the lab’s violations were so “egregious” and pervasive that studies conducted there between April 2005 and August 2009 might be worthless.

The health threat was potentially serious: About 100 drugs, including sophisticated chemotherapy compounds and addictive prescription painkillers, had been approved for sale in the United States at least in part on the strength of Cetero Houston’s tainted tests. The vast majority, 81, were generic versions of brand-name drugs on which Cetero scientists had often run critical tests to determine whether the copies did, in fact, act the same in the body as the originals. For example, one of these generic drugs was ibuprofen, sold as gelatin capsules by one of the nation’s largest grocery-store chains for months before the FDA received assurance they were safe.

The rest were new medications that required so much research to win approval that the FDA says Cetero’s tests were rarely crucial. Stone said he expected the FDA to move swiftly to compel new testing and to publicly warn patients and doctors.

Instead, the agency decided to handle the matter quietly, evaluating the medicines with virtually no public disclosure of what it had discovered. It pulled none of the drugs from the market, even temporarily, letting consumers take the ibuprofen and other medicines it no longer knew for sure were safe and effective. To this day, some drugs remain on the market despite the FDA having no additional scientific evidence to back up the safety and efficacy of these drugs.

By contrast, the FDA’s transatlantic counterpart, the European Medicines Agency, has pulled seven Cetero-tested medicines from the market.

The FDA also has moved slowly to shore up the science behind the drugs. Twice the FDA announced it was requiring drug makers to repeat, reanalyze or audit many of Cetero’s tests, and to submit their findings to the agency. Both times the agency set deadlines, yet it has allowed some companies to blow by them. Today, six months after the last of those deadlines expired and almost three years after Cetero’s misconduct was discovered, the FDA has received the required submissions for just 53 drugs. The agency says most companies met the deadlines but acknowledged that “a few have not yet submitted new studies.” Other companies, it said, have not submitted new research because they removed their drugs from the market altogether. For its part, the FDA has finished its review of just 21 of the 53 submissions it has received, raising the possibility that patients are taking medications today that the agency might pull off the market tomorrow.

To this day, the agency refuses to disclose the names of the drugs it is reassessing, on the grounds that doing so would expose “confidential commercial information.” ProPublica managed to identify five drugs (http://projects.propublica.org/graphics/cetero) that used Cetero tests to help win FDA approval.

FDA officials defended the agency’s handling of the Cetero case as prudent and scientifically sound, noting that the agency has found no discrepancies between any original drug and its generic copy and no sign that any patients have been harmed. “It is non-trivial to have to redo all this, to withdraw drugs, to alarm the public and the providers for a large range of drugs,” said Janet Woodcock, the director of the FDA’s Center for Drug Evaluation and Research. “There are consequences. To repeat the studies requires human experimentation, and that is not totally without risk.” Woodcock added that an agency risk assessment found the potential for harm from drugs tested by Cetero to be “quite low,” an assessment she said has been “confirmed” by the fact that no problems have been found in the drugs the agency has finished reviewing. She declined to release the risk assessment or detail its design. A subsequent statement from the agency described the assessment as “fluid” and “ongoing.” The FDA also has not released its 21 completed reviews, which ProPublica has requested. Some experts say that by withholding so much information in the Cetero case the FDA failed to meet its obligations to the public.

“If there are problems with the scientific studies, as there have been in this case, then the FDA’s review of those problems needs to be transparent,” said David Kessler, who headed the FDA from 1990 to 1997 and who is now a professor at the University of California at San Francisco. Putting its reviews in public view would let the medical community “understand the basis for the agency’s actions,” he said. “FDA may be right here, but if it wants public confidence, they should be transparent. Otherwise it’s just a black box.”

Another former senior FDA official, who spoke on condition of anonymity, also felt the FDA had moved too slowly and secretively. “They’re keeping it all in the dark. It’s not transparent at all,” he said.

By contrast, the European Medicines Agency has provided a public accounting of the science behind all the drugs it has reviewed. Its policy, the EMA said in response to questions, is to make public “all review procedures where the benefit-risk balance of a medicine is under scrutiny.”

Woodcock dismissed comparisons to the EMA. “Europe had a smaller handful of drugs,” she said, “and they may not have engaged in as extensive negotiation and investigations with the company as we did.” She said the FDA would have disclosed more, including the names of drugs, had it believed there was a risk to public health. “We believe that this did not rise to the level where the public should be notified,” she said. “We felt it would result in misunderstanding and inappropriate actions.”

In a written response to Kessler’s comments, the FDA said, “We’ve been as transparent as possible given the legal protections surrounding an FDA investigation of this or any type. The issue is not a lack of transparency but rather the difficulty of explaining why the problems we identified at Cetero, which on their face would appear to be highly significant in terms of patient risk, fortunately were not.” Still, the FDA’s secrecy has had other ramifications. Some of Cetero’s suspect research made its way unchallenged into the peer-reviewed scientific literature on which the medical community relies. In one case, a researcher and a journal editor told ProPublica they had no idea the Cetero tests had been called into doubt.

Cetero, in correspondence with the FDA, conceded misconduct. And in an interview, Cetero’s former attorney, Marc Scheineson, acknowledged that chemists at the Houston facility committed fraud but said the problem was limited to six people who had all been fired.

“There is still zero evidence that any of the test results…were wrong, inaccurate, or incorrect,” he said. Scheineson called the FDA’s actions “overkill” and said they led to the demise of Cetero and its successor company.

In 2012, the company filed for Chapter 11 bankruptcy and emerged with a new name, PRACS Institute. PRACS, in turn, filed for bankruptcy on March 22 of this year. A PRACS spokesperson said the company had closed the Houston facility in October 2012.

Pamidi, the Cetero executive who provided the carts of file boxes, declined to comment. As for Stone, the former FDA investigator, he said he was disturbed by the agency’s decisions.

“They could have done more,” he said. “They should have done more.”

Cross-checking U.S. and European public records, including regulatory filings, scientific studies and civil lawsuits, ProPublica was able to identify a few of the drugs that are on the U.S. market because of tests performed at Cetero’s Houston lab. There is no evidence that patients have suffered harm from these drugs; the FDA says it has detected no increase in reports of side effects or lack of efficacy among Cetero-tested medications.

To be sure, just because a crucial study is deemed potentially unreliable does not mean that a drug is unsafe or ineffective. What it does mean is that the FDA’s scientific basis for approving that drug has been undermined.

The risks are real, academic experts say, particularly for drugs such as blood thinners and anti-seizure medications that must be given at very specific doses. And generic versions of drugs have been known to act differently from name-brand products.

There is no indication the generic ibuprofen gelatin capsules hurt anyone, but their case shows how the FDA left a drug on the market for months without confirmation that the drug was equivalent to the name brand.

The capsules were manufactured by Banner Pharmacaps and carried by Supervalu, a grocery company that operates or licenses more than 2,400 stores across the United States, including Albertson’s, Jewel-Osco, Shop ‘n Save, Save-A-Lot, and Shoppers Food & Pharmacy.

Cetero had performed a key analysis to show that the capsules were equivalent to other forms of the drug. Banner, the drug’s maker, said the FDA first alerted it to the problems at Cetero in August 2011. The FDA required drug companies to redo many of Cetero’s tests, but, a spokesperson for Banner wrote in an email, “We received no directive from FDA to recall or otherwise interrupt manufacture of the product.”

Banner said it repeated the tainted Cetero tests at a different research firm, and the FDA said it received the new data in January 2012 — leaving a gap of at least five months when the FDA knew the drug was on the market without a rock-solid scientific basis.

An FDA spokesperson wrote in an email that the agency found the new studies Banner submitted “acceptable” and told Banner it had no further questions.

A spokesperson for Supervalu told ProPublica it purchased the ibuprofen from a supplier, which has assured the grocery company that “there are no issues with the product.”

According to U.S. and European records, another one of the drugs approved based on research at Cetero’s troubled Houston lab was a chemotherapy drug known as Temodar for Injection.

Temodar was originally approved in 1999 as a capsule to fight an aggressive brain cancer, glioblastoma multiforme. Some patients, however, can’t tolerate taking the medication orally, so drug maker Schering-Plough decided to make an intravenous form of the drug.

To get Temodar for Injection approved, the FDA required what it called a “pivotal” test comparing the well-established capsule form of Temodar to the form injected directly into the bloodstream.

Cetero Houston conducted that test, comparing blood samples of patients who received the capsule to samples of those who got the injection to determine if the same amount of the drug was reaching the bloodstream. This test is crucial, particularly in the case of Temodar, where there was a question about the right dosing regimen of the injectable version. If too little drug gets into the blood, the cancer could continue to grow unabated. If too much gets in, the drug’s debilitating side effects could be even worse.

Cetero performed the test between September 2006 and October 2007, according to documents from the European Medicines Agency, and FDA records indicate that same test was used to win approval in the U.S.

In 2011, the FDA notified Merck & Co., which had acquired Schering-Plough, about the problems with Cetero’s testing. In April 2012, the FDA publicly announced that analyses done by Cetero during the time when it performed the Temodar work would have to be redone. But according to Merck spokesman Ronald Rogers, the FDA has not asked Merck for any additional analyses to replace the questionable study.

The FDA declined to answer specific questions about the Temodar case, saying to do so would reveal confidential commercial information. But Woodcock said that in some cases, drug manufacturers had submitted alternative test results to the FDA that satisfied the agency that no retesting was necessary for specific drugs.

The FDA never removed Temodar for Injection from the market. The European Medicines Agency also kept the injection form of the drug on the market, but the two agencies handled their decision in sharply different ways.

The EMA has publicly laid out evidence — including studies not performed by Cetero — for why it believes the benefits of the injection drug outweigh its risks. But in the United States, the FDA has kept silent. To this day, Temodar’s label — the single most important way the FDA communicates the risks and benefits of medication — still displays data from the dubious Cetero study. (The label of at least one other drug, a powerful pain reliever marketed as Lazanda, also still displays questionable Cetero data.)

Woodcock said the agency hadn’t required manufacturers to alter their labels because, despite any question about precise numerical precision, the FDA’s overall recommendation had not changed.

In a written response to questions, Merck said it “stands behind the data in the TEMODAR (temozolomide) label.” The company said it learned about “misconduct at a contract research organization (CRO) facility in Houston” from the FDA and that it cooperated with investigations by the FDA and its European counterpart. It said that Cetero had performed no other studies for Merck.

Even one of the researchers involved in evaluating injectable Temodar didn’t know that the FDA had flagged Cetero’s analysis as potentially unreliable until contacted by a reporter for this story.

Dr. Max Schwarz, an oncologist and clinical professor at Monash University in Melbourne, Australia, treated some brain-cancer patients with the experimental injectable form of Temodar and others with the capsule formulation. Blood from his patients was sent to Cetero’s Houston lab for analysis.

Schwarz said he still has confidence in the injectable form of the drug, but said that he was “taken aback” when a reporter told him that the FDA had raised questions about the analysis. “I think we should have been told,” he said.

Suspect research conducted by Cetero Houston was not only used to win FDA approval but was also submitted to peer-reviewed scientific journals. Aided by the FDA’s silence, those articles remain in the scientific literature with no indication that they might, in fact, be compromised. For example, based on Cetero’s work, an article in the journal Cancer Chemotherapy and Pharmacology purports to show that Temodar for Injection is equivalent to Temodar capsules.

Edward Sausville, co-editor-in-chief of the journal, said in an email that the first he heard that something might be wrong with the Cetero research was when a reporter contacted him for this story. He also said the publisher of the journal would conduct a “review of relevant records pertinent to this case.”

During his years of inspecting the Houston lab, the FDA’s Stone said he often had the sense that something wasn’t right. When he went to other contract research firms and asked for data on a trial, they generally produced an overwhelming amount of paper: records of failed tests, meticulous explanations of how the chemists had made adjustments, and more.

Cetero’s records, by contrast, showed very clean, error-free procedures. As Stone and his colleagues dug through the data, though, they often found gaps. When pressed, Cetero officials would often produce additional data — data that ought to have been in the files originally handed over to the FDA.

Stone said, “We should have looked back and said, ‘Wait a minute, there’s always something missing from the studies from here. Why?'”

One reason, the FDA would determine, was that Cetero’s chemists were taking shortcuts and other actions prohibited by the FDA’s Good Laboratory Practice guidelines, which set out such matters as how records must be kept and how tests must be performed.

Stone and his FDA colleagues might never have realized Cetero was engaging in misconduct if a whistleblower hadn’t stepped forward.

Cashton J. Briscoe operated a liquid chromatography-tandem mass spectrometry device, or “mass spec,” a sensitive machine that measures the concentration of a drug in the blood.

He took blood samples prepared by Cetero chemists and used mass specs to perform “runs” — tests to see how much of a drug is in patients’ blood — that must always be performed with control samples. Often those controls show readings that are clearly wrong, and chemists have to abort runs, document the failure, recalibrate the machines, and redo the whole process.

But Cetero paid its Houston chemists based on how many runs they completed in a day. Some chemists doubled or even tripled their income by squeezing in extra tests, according to time sheets entered as evidence in a lawsuit filed in U.S. District Court in Houston by six chemists seeking overtime payments. Briscoe thought several chemists were cutting corners — by using the control-sample readings from one run in other runs, for example.

Attorney Scheineson, who represented Cetero during the FDA’s investigation, acknowledged that the Houston lab’s compensation system was “crappy” and that a handful of “dishonest” chemists at the Houston facility committed fraud.

In April 2009, Briscoe blew the whistle in a letter to the company written by his lawyer, reporting that “many of the chemists were manipulating and falsifying data.” Soon thereafter, Briscoe told the company that he had documented the misconduct. According to Stone and documents reviewed by ProPublica, Briscoe had photographic evidence that mass spec operators had switched the quality control samples between different runs; before-and-after copies of documents with the dates and other material changed; and information about a shadow computer filing system, where data from failed runs could be stored out of sight of FDA inspectors.

On June 5, apparently frustrated with Cetero’s response, Briscoe went a step further and called the FDA’s Dallas office. He agreed to meet Stone the following Monday, but never showed. Stone called him, as did other FDA officials, but Briscoe had changed his mind and clammed up.

Still, Stone’s brief phone conversation with Briscoe reminded the agent of all those suspiciously clean records he had seen at Cetero over the years. “Now that you have a bigger picture,” Stone recalled, “you’re like, ‘Oh, some of this stuff is cooked.'”

Two days after Stone’s aborted meeting with Briscoe, Cetero informed the FDA that an employee had made allegations of misconduct and that the company had hired an outside auditor to review five years’ worth of data. That led to months of back-and-forth between the agency and Cetero that culminated when Stone and his inspectors arrived in Houston in May 2010.

Two teams of FDA investigators eventually confirmed Briscoe’s main allegations and cited the company for falsifying records and other violations of Good Laboratory Practice. The net effect of the misconduct was far-reaching, agency officials wrote in a July 2011 letter:

“The pervasiveness and egregious nature of the violative practices by your firm has led FDA to have significant concerns that the bioequivalence and bioavailability data generated at the Cetero Houston facility from April 1, 2005, to June 15, 2010 … are unreliable.”

Bioequivalence studies measure whether a generic drug acts the same in the body as the name-brand drug; bioavailability studies measure how much drug gets into a patient’s system.

The FDA’s next step was to try to determine which drugs were implicated — information the agency couldn’t glean from its own records.

“We couldn’t really tell — because most of the applications we get are in paper — which studies were actually linked to the key studies in an application without asking the application holders,” the FDA’s Woodcock said. “So we asked the application holders,” meaning the drug manufacturers.

In the interim, the FDA continued to investigate processes and procedures at Cetero.

“We put their operations under a microscope,” said Woodcock. A team of clinical pharmacologists, statisticians and IT experts conducted a risk analysis of the problems at Cetero, she said, and they “concluded that the risk of a misleading result was very low given how the studies were done, how the data were captured and so forth.”

In April 2012, nearly three years after Briscoe first alerted the FDA to problems at Cetero, and nearly two years after Cetero handed over its documentation to inspectors, the FDA entered into a final agreement with the company. Drug makers would need to redo tests conducted at the company’s Houston facility between April 1, 2005 and Feb. 28, 2008, if those studies had been part of a drug application submitted to the FDA. If stored blood samples were still usable, they could be reanalyzed. If not, the entire study would need to be repeated, the FDA said. The agency set a deadline of six months.

Cetero tests done between March 1, 2008 and Aug. 31, 2009 would be accepted only if they were accompanied by an independent data integrity audit.

Analyses done after Sept. 1, 2009 would not require retesting. The FDA said that Cetero had issued a written directive on Sept. 1, 2009, ordering one kind of misconduct to stop, which was why it did not require any action on Cetero Houston studies after that date. According to public documents, however, the agency’s inspectors “found continued deficiencies” that persisted into December 2010.

In response to questions, the FDA said the problem period “was subsequently narrowed as more information regarding Cetero’s practices became available.”

A year after concluding its final agreement with Cetero, the FDA’s review is still not finished. “Without the process being public it’s hard to know, but it seems that this has been going on for too long,” said Kessler, the former FDA chief.

“The process has been long,” the FDA said, “because of the number of products involved and our wish to be thorough and accurate in both our requests for and our review of the data.”

Cetero’s attorney Scheineson said the FDA scaled back its requirements because it finally talked with company officials. He noted that Cetero had tried repeatedly to talk with the FDA before the agency issued its strongly worded July 2011 letter, and that more than 1,000 employees have since lost their jobs.

“If you would get an honest assessment from the leaders of the agency,” he said, “I think in retrospect they would have argued that this was overkill here and that they should have had input from the company before essentially going public with that death sentence.”

“I’m not sure what is meant by ‘death sentence,'” an FDA spokesperson wrote in response, “but our first priority was and is patient safety and we proceeded to conduct the investigation toward that objective.”

The FDA’s Stone draws little satisfaction from unraveling the problems at Cetero.

There are thousands of bioequivalence studies done every year, he pointed out, with each study generating thousands of pages of paper records. “Do you really think we’re going to look at 100 percent of them? We’re going to look at maybe 5 percent if we’re lucky,” he said. “Sometimes 1 percent.”

Still, given how often he and other FDA teams had inspected the Houston lab, he thinks regulators should have spotted Cetero’s misconduct sooner.

“In hindsight I look back and I’m like, ‘Wow, should I be proud of this?'” he said. “It’s cool that I was part of it, but it’s crap that we didn’t catch it five years ago. How could we let this go so long?”

Rob Garver can be reached at rob.garver@propublica.org, and Charles Seife can be reached at cgseife@nasw.org.

Research assistance for this story was contributed by Nick Stockton, Christine Kelly, Lily Newman, Joss Fong and Sarah Jacoby of the Science, Health, and Environmental Reporting Program at NYU.

http://www.scientificamerican.com/article.cfm?id=fda-let-drugs-approved-on-fraudulent-research-stay-on-market

Advertisements

JN2_2846a1360362374

Daniel Yuan, pictured at his home in Laurel, raised doubts for years about the work of his colleagues in a Johns Hopkins medical research lab. “The denial that I am hearing from almost everyone in the group as a consensus is troubling to me,” he wrote in one e-mail. In December 2011, after 10 years at the lab, he was fired.

By Peter Whoriskey
The Washington Post Published: March 11
The numbers didn’t add up.

Over and over, Daniel Yuan, a medical doctor and statistician, couldn’t understand the results coming out of the lab, a prestigious facility at Johns Hopkins Medical School funded by millions from the National Institutes of Health.

He raised questions with the lab’s director. He reran the calculations on his own. He looked askance at the articles arising from the research, which were published in distinguished journals. He told his colleagues: This doesn’t make sense.

“At first, it was like, ‘Okay — but I don’t really see it,’ ” Yuan recalled. “Then it started to smell bad.”

His suspicions arose as reports of scientific misconduct have become more frequent and critics have questioned the willingness of universities, academic journals and the federal government, which pays for much of the work, to confront the problem.

Eventually, the Hopkins research, which focused on detecting interactions between genes, would win wide acclaim and, in a coup for the researchers, space in the pages of Nature, arguably the field’s most prestigious journal. The medical school even issued a news release when the article appeared last year: “Studies Linked To Better Understanding of Cancer Drugs.”

What very few readers of the Nature paper could know, however, was that behind the scenes, Yuan’s doubts seemed to be having profound effects.

In August, Yu-yi Lin, the lead author of the paper, was found dead in his new lab in Taiwan, a puncture mark in his left arm and empty vials of sedatives and muscle relaxants around him, according to local news accounts — an apparent suicide.

And within hours of this discovery, a note was sent from Lin’s e-mail account to Yuan. The e-mail, which Yuan saved, essentially blamed him for driving Lin to suicide. Yuan had written to Nature’s editors, saying that the paper’s results were overstated and that he found no evidence that the analyses described had actually been conducted. On the day of his death, Lin, 38, the father of three young daughters, was supposed to have finished writing a response to Yuan’s criticisms.

The subject line of the e-mail to Yuan, sent by an unknown person, said “your happy ending.”

“Yu-yi passed away this morning. Now you must be very satisfied with your success,” the e-mail said.

Yuan said he was shocked by the note, so much so that he began to shake.

But in the seven months since, he has wondered why no one — not the other investigators on the project, not the esteemed journal, not the federal government — has responded publicly to the problems he raised about the research.

The passions of scientific debate are probably not much different from those that drive achievement in other fields, so a tragic, even deadly dispute might not be surprising.

But science, creeping ahead experiment by experiment, paper by paper, depends also on institutions investigating errors and correcting them if need be, especially if they are made in its most respected journals.

If the apparent suicide and Yuan’s detailed complaints provoked second thoughts about the Nature paper, though, there were scant signs of it.

The journal initially showed interest in publishing Yuan’s criticism and told him that a correction was “probably” going to be written, according to e-mail rec­ords. That was almost six months ago. The paper has not been corrected.

The university had already fired Yuan in December 2011, after 10 years at the lab. He had been raising questions about the research for years. He was escorted from his desk by two security guards.

More recently, a few weeks after a Washington Post reporter began asking questions, a university spokeswoman said that a correction had been submitted to Nature and that it was under review.

“Your questions will be addressed with that publication,” a spokeswoman for the Hopkins medical school, Kim Hoppe, wrote in an e-mail.

Neither the journal nor the university would disclose the nature of the correction.

Hoppe declined an opportunity to have university personnel sit for interviews.

In the meantime, the paper has been cited 11 times by other published papers building on the findings.

It may be impossible for anyone from outside to know the extent of the problems in the Nature paper. But the incident comes amid a phenomenon that some call a “retraction epidemic.”

Last year, research published in the Proceedings of the National Academy of Sciences found that the percentage of scientific articles retracted because of fraud had increased tenfold since 1975.

The same analysis reviewed more than 2,000 retracted biomedical papers and found that 67 percent of the retractions were attributable to misconduct, mainly fraud or suspected fraud.

“You have a lot of people who want to do the right thing, but they get in a position where their job is on the line or their funding will get cut, and they need to get a paper published,” said Ferric C. Fang, one of the authors of the analysis and a medical professor at the University of Washington. “Then they have this tempting thought: If only the data points would line up . . . ”

Fang said retractions may be rising because it is simply easier to cheat in an era of digital images, which can be easily manipulated. But he said the increase is caused at least in part by the growing competition for publication and for NIH grant money.

He noted that in the 1960s, about two out of three NIH grant requests were funded; today, the success rate for applicants for research funding is about one in five. At the same time, getting work published in the most esteemed journals, such as Nature, has become a “fetish” for some scientists, Fang said.

In one sense, the rise in retractions may mean that the scientific enterprise is working — bad work is being discovered and tossed out. But many observers note that universities and journals, while sometimes agreeable to admitting small mistakes, are at times loath to reveal that the essence of published work was simply wrong.

“The reader of scientific information is at the mercy of the scientific institution to investigate or not,” said Adam Marcus, who with Ivan Oransky founded the blog Retraction Watch in 2010. In this case, Marcus said, “if Hopkins doesn’t want to move, we may not find out what is happening for two or three years.”

The trouble is that a delayed response — or none at all — leaves other scientists to build upon shaky work. Fang said he has talked to researchers who have lost months by relying on results that proved impossible to reproduce.

Moreover, as Marcus and Oransky have noted, much of the research is funded by taxpayers. Yet when retractions are done, they are done quietly and “live in obscurity,” meaning taxpayers are unlikely to find out that their money may have been wasted.

Johns Hopkins University typically receives more than $600 million a year from NIH, according to NIH figures.

For someone who has taken on a battle with Johns Hopkins and Nature, Yuan is strikingly soft-spoken.

He grew up in Gainesville, Fla., and attended MIT and then medical school at Johns Hopkins. He worked briefly as a pediatrician and an assistant professor of pediatrics before deciding that he preferred pure research. He has a wife and two kids and is an accomplished violinist.

In 2001, he joined the lab of Jef Boeke, a Hopkins professor of molecular biology and genetics. Boeke’s work on the yeast genome is, as academics put it, “highly cited” — that is, other papers have used some of his articles numerous times for support. Last year, he was named a member of the prestigious American Academy of Arts and Sci­ences.

The lab’s research focused on developing a methodology for finding evidence of genes interacting, primarily in the yeast genome and then in the human genome. Genetic interactions are prized because they yield insights into the traits of the genes involved.

During Yuan’s time there, the lab received millions in NIH funding, and according to internal e-mails, the people in the lab were under pressure to show results. Yuan felt the pressure, too, he says, but as the point person for analyzing the statistical data emerging from the experiments, he felt compelled to raise his concerns.

As far back as 2007, as the group was developing the methodology that would eventually form the basis of the Nature paper, Yuan wrote an anguished e-mail to another senior member of the lab, Pamela Meluh.

“I continue to be in a state of chronic alarm,” he wrote in August 2007. “The denial that I am hearing from almost everyone in the group as a consensus is troubling to me.”

Meluh quickly wrote back: “I have the same level of concern as you in terms of data quality, but I have less basis to think it can be better. . . . I’m always torn between addressing your and my own concerns and being ‘productive.’ ”

Then Boeke weighed in, telling Yuan that if he could improve the data analysis, he should, but that “the clock is ticking.”

“NIH has already given us way more time than we thought we needed and at some point we’ve got to suck it up and run with what we have,” Boeke wrote to Meluh and Yuan.

A few years later, another deadline was looming, and Elise Feingold, an NIH administrator, wanted to know what the lab had accomplished.

“I do need some kind of progress report on what you have been doing the past two years . . . and what you think you can accomplish with these funds,” she wrote to Boeke.

Citing Feingold’s message, ­Meluh wrote to Yuan, asking for help in explaining what the lab had produced. Its members had worked diligently, Yuan says, but hadn’t arrived at the kind of significant findings that generally produce scientific papers.

“I want to make it look like we’ve been busy despite lack of publications,” Meluh wrote.

Meluh did not respond to a request for an interview. Boeke referred questions to the university’s public relations team, which declined to comment further. An NIH official declined to comment.

While Yuan was growing increasingly skeptical of the lab’s methodology, Yu-yi Lin, who was also working at the lab, was trying to extend it. In the past, it had been applied to the yeast genome; Lin would extend it to the human genome — and this would become the basis of the Nature paper.

Lin, who was from Taiwan, was an up-and-comer. As a graduate student at Johns Hopkins just a few years before, he’d won an award for his work in cell metabolism and aging. He was also arranging for a prestigious spot at National Taiwan University.

At one point, when he was still at the Boeke lab at Hopkins, Lin asked Yuan to help analyze the data that would become the basis for the Nature paper, Yuan says. Yuan said he declined to get involved because he thought the methodology still had deep flaws.

Interactions between Lin and Yuan at the lab were few, Yuan said, and at any rate, Yuan had other things to worry about. He was slowly being forced out. He was demoted in 2011 from research associate to an entry-level position. A disagreement over whether Yuan should have asked Boeke if he wanted a byline on a paper erupted into further trouble, e-mail and other records show.

The Johns Hopkins spokeswoman, Hoppe, declined to discuss Yuan’s job termination.

On Dec. 15, 2011, Yuan was forced to leave the lab. He wasn’t allowed to make copies of his cell collection. He spent the next month trying to keep his mind busy. He read books about JavaScript and Photoshop, which he thought would enrich his research abilities. As he looked for other research jobs, he sensed that he had been blackballed.

Then, in February 2012, the Nature paper was published.

The research was a “profound achievement” that would “definitely be a great help to solve and to treat many severe diseases,” according to a news release from National Taiwan University, where Lin was now working.

Upon reading it, Yuan said, he was astonished that Lin had used what he considered a flawed method for finding genetic interactions. It had proved troublesome in the yeast genome, he thought. Could it have possibly been more reliable as it was extended to the human genome?

Lin, Boeke and their co-authors reported discovering 878 genetic interactions, or “hits.”

But Yuan, who was familiar with the data and the statistics, reanalyzed the data in the paper and concluded that there was essentially no evidence for any more than a handful of the 878 genetic interactions.

One of the key problems, Yuan wrote to the Nature editors, was that the numerical threshold the investigators used for determining when a hit had arisen was too low. This meant they would report far more hits than there actually were.

Yuan also calculated that, given the wide variability in the data and the relative precision required to find a true hit, it would have been impossible to arrive at any conclusions at all. By analogy, it would be like a pollster declaring a winner in an election when the margin of error was larger than the difference in the polling results.

“The overwhelming noise in the . . . data and the overstated strength of the genetic interactions together make it difficult to reconstruct any scientific process by which the authors could have inferred valid results from these data,” Yuan wrote to the editors of Nature in July.

His analysis attacks only the first portion of the paper; even if he is correct, the second part of the paper could be true.

Nevertheless, Yuan wanted Nature to publish his criticism, and following instructions from the journal, he forwarded his letter to Boeke and Lin, giving them two weeks to respond.

Just as the two weeks were to elapse, Boeke wrote to Nature asking for an extension of time — “a couple weeks or more” — to address Yuan’s criticism. Boeke explained that end-of-summer schedules and the multiple co-authors made it difficult to respond on time.

A day later, Lin was discovered dead in his office at National Taiwan University.

“Renowned scientist found dead, next to drug bottles,” the headline in the Taipei Times said.

Even in his death, the Nature paper was a kind of shorthand for Lin’s scientific success.

“A research team [Lin] led was featured in the scientific journal Nature in February for their discovery of the key mechanism for maintaining cell energy balance — believed to be linked to cellular aging and cancer,” the newspaper said.

If there was a suicide note, it has not been made public, and it is difficult to know what went through Lin’s mind at the end of his life. The apparent suicide and the e-mail to Yuan suggest only that Lin may have been distraught over the dispute; they do not prove that he acted improperly.

Shortly after the Nature paper appeared, Yuan hired lawyer Lynne Bernabei to challenge the way he was terminated at Hopkins.

In late August, Yuan asked the Nature editors again whether they would publish his criticism. Lin was dead, but Boeke and the others had had a month to respond, and Yuan hadn’t heard a thing.

On Sept. 28, a Nature editor informed Yuan by e-mail that the journal was still waiting on a fuller response from Boeke and that “experiments are being done and probably a Correction written.”

Such a correction has not appeared.

So as a last attempt, he figured he’d try the federal government, which paid for much of the research. But the government suggested that the threat to the federal research, if there was any, ended with Lin’s death.

“It is our understanding that these allegations are being investigated by Johns Hopkins University,” said the letter from the Office of Research Integrity.

Besides, it noted, the person responsible for the paper was Lin.

“Deceased respondents no longer pose a risk,” the letter said.

http://www.washingtonpost.com/business/economy/doubts-about-johns-hopkins-research-have-gone-unanswered-scientist-says/2013/03/11/52822cba-7c84-11e2-82e8-61a46c2cde3d_story_4.html

Thanks to Kebmodee for bringing this to the attention of the It’s Interesting community.