Tag: NIH

Male researchers’ ‘vague’ language more likely to win grants

On By A.P.In UncategorizedLeave a comment

by Holly Else

Grant reviewers award lower scores to proposals from women than to those from men, even when they don’t know the gender of the applicant, an analysis of thousands of submissions to the Bill and Melinda Gates Foundation has found (1).

That’s because male and female scientists use different types of word on grant applications, according to the study, published by the US National Bureau of Economic Research.

The study finds that women are more likely to choose words specific to their field to describe their science, whereas men tend to use less precise terms. These broader terms seem to be preferred by the reviewers who decide how to distribute the cash, says the analysis — even though proposals containing those words don’t lead to better research outcomes.

The findings aren’t surprising, says Kuheli Dutt, who works in academic affairs and diversity at Columbia University in New York City. Dutt sees parallels with research showing that men are more likely to boast and overstate their performance in tests, whereas women are more likely to be cautious in their statements (2). Using broad words might lead to sweeping claims, but narrow words might imply more cautious claims, she says.

Loaded language

Previous research had highlighted how differences in the way men and women use language can drive bias. For example, some studies show that the words in some job adverts can put women off applying, and women in the geosciences are less likely than their male counterparts to receive a recommendation letter whose tone suggests that they are outstanding candidates (3).

A meeting between four executives in a large, open space.
Reviewers give higher scores to grant applications from men than those from women.Credit: TommL/Getty

Grant reviewers award lower scores to proposals from women than to those from men, even when they don’t know the gender of the applicant, an analysis of thousands of submissions to the Bill and Melinda Gates Foundation has found1.

That’s because male and female scientists use different types of word on grant applications, according to the study, published by the US National Bureau of Economic Research.

The study finds that women are more likely to choose words specific to their field to describe their science, whereas men tend to use less precise terms. These broader terms seem to be preferred by the reviewers who decide how to distribute the cash, says the analysis — even though proposals containing those words don’t lead to better research outcomes.

The findings aren’t surprising, says Kuheli Dutt, who works in academic affairs and diversity at Columbia University in New York City. Dutt sees parallels with research showing that men are more likely to boast and overstate their performance in tests, whereas women are more likely to be cautious in their statements2. Using broad words might lead to sweeping claims, but narrow words might imply more cautious claims, she says.

Loaded language
Previous research had highlighted how differences in the way men and women use language can drive bias. For example, some studies show that the words in some job adverts can put women off applying, and women in the geosciences are less likely than their male counterparts to receive a recommendation letter whose tone suggests that they are outstanding candidates3.

But this is the first time that ‘gendered’ language has been explored in grant applications, says Julian Kolev, who studies entrepreneurship at the Southern Methodist University in Texas and led the work.

Kolev’s analysis looked at almost 7,000 proposals submitted to the Grand Challenges Explorations programme of the Bill & Melinda Gates Foundation between 2008 and 2017. The fund awards grants of between $100,000 and $1 million to address challenges in global health and is open to anyone through a two-page online application. Reviewers are blind to the gender of the applicants.

The researchers singled out the applications from US researchers and sought information from the Gates Foundation on applicants’ gender, discipline and where they work. The group also looked at each scientist’s publication record and grant history before and after the application.

The team found that women received significantly lower scores from reviewers than men did. This couldn’t be explained by the applicants’ experience, publications record or the gender of the reviewers. Instead, it seemed to be down to their communication style in the proposal.

The researchers found that men tended to use ‘broad’ words, such as “control”, “detection” and “bacteria”, more often. These were defined as words that appeared at the same rate in proposals regardless of the topic. By contrast, women favoured ‘narrower’ or more topic-specific terms, such as “community”, “oral” and “brain” (see ‘Broad language’). The authors linked broad words to higher review scores, and narrow ones with lower scores.

But funded applications that contained many broad words didn’t result in work that led to more publications and future grants, the researchers found. And when women secured funding, they generally outperformed men on these measures.

Closing the gap

The Gates Foundation says that it is committed to ensuring gender equality and that its grand-challenges programme uses blind reviews in an attempt to eliminate reviewer bias. It is also reviewing the results of this study.

Kolev suggests that grant reviewers could be trained to limit their sensitivity to communication styles. The make-up of the review panel also seems important. “We consistently show that female reviewers’ scores do not favour proposals from male applicants in the way that male reviewers’ scores do,” he notes. “So increasing the number of female reviewers is one potential way to mitigate the effects we find.”

doi: 10.1038/d41586-019-01402-4

References
1.Kolev, J., Fuentes-Medel, Y. and Murray, F. Natl Bureau Econ. Res. Working Paper No. 25759 https://www.nber.org/papers/w25759 (2019)

2.Reuben, E., Sapienza, P. and Zingales, L. Proc. Natl Acad. Sci. USA 111, 4403–4408 (2014).

3.Dutt, K. et al. Nature Geosci. 9, 805–808 (2016).

https://www.nature.com/articles/d41586-019-01402-4?utm_source=Nature+Briefing&utm_campaign=96860aed6e-briefing-dy-20190502&utm_medium=email&utm_term=0_c9dfd39373-96860aed6e-44039353

Peer-reviewed scientific grants are more likely to be funded when applicants are allowed to suggest reviewers

On By A.P.In UncategorizedLeave a comment

Peer reviewers are four times more likely to give a grant application an “excellent” or “outstanding” score rather than a “poor” or “good” one when they are chosen by the grant’s applicants, an analysis of Swiss funding applications has found.

The study, at the Swiss National Science Foundation (SNSF), was completed in 2016, and the SNSF acted quickly on its findings by banning grant applicants from being able to recommend referees.

The authors, who are affiliated with the SNSF, posted their results online at PeerJ Preprints1 on 19 March, and in their paper call on other funders to reconsider their funding processes.

“I think this practice should be abolished altogether,” says study co-author Anna Severin, a sociologist who studies peer review at the University of Bern. Other experts are also wary of the problems that author-picked peer reviewers might cause, but some question whether banning them altogether is the right step.

The study examined more than 38,000 reviews from nearly 13,000 SNSF grant applications by about 27,000 peer reviewers from all disciplines between 2006 and 2016. The authors found that reviewers nominated by applicants were more likely to give these applicants higher evaluation scores than referees chosen by the SNSF.

Higher scores
The study found that reviewers affiliated with non-Swiss institutions gave higher evaluation scores, on average, than those based in the country. Male reviewers gave higher scores than female reviewers did, and male applicants received higher scores than female applicants, although the difference was small. Academics aged over 60 received the best feedback, regardless of their gender.

The findings echo those of previous studies of manuscript peer review2,3, which have found that author-nominated reviewers rate papers more favourably than do referees picked by journal editors.

Liz Allen, who is the director of strategic initiatives at the open-access publisher F1000, says that the latest study is robust, but notes that making a policy change based solely on its data is questionable. “This almost automatically assumes that the scores must be ‘too high’ and therefore biased instead of perhaps testing out who the reviewers were and whether there were reasons why the scores might have been higher,” says Allen, who is also the former head of evaluation at the UK biomedical funder Wellcome Trust.

Johan Bollen, who studies complex computer systems and networks at Indiana University Bloomington, says he sees benefits to both sides of the argument. Grant applicants or study authors “have important information with respect to the experts that are most suited to provide an in-depth and knowledgeable review of their proposal”. But it might create an opportunity for authors to bias the reviewing process, he adds.

A new system
Bollen has previously argued for a system in which all researchers are guaranteed some money, provided they anonymously allocate a fraction of their funding to researchers of their own choice. The goal would be to shift the focus from funding projects to funding people.

Funding agencies around the world have different approaches to choosing grant reviewers. The US National Science Foundation does consider nominated reviewers, as well as those who applicants say are not fit to evaluate their work. Applicants to the US National Institutes of Health, however, are not allowed to suggest potential reviewers.

A spokesperson of UK Research and Innovation, Britain’s central research funder, told Nature that the organization’s individual, topic-based research councils invite applicants to nominate prospective peer reviewers, but suggested reviewers are not always used. When they are, the process also includes at least one additional referee, the spokesperson says.

Finding reviewers who want to referee papers or grant applications can also be a struggle, notes study co-author João Martins, a data scientist at the European Research Council Executive Agency in Brussels. A 2018 survey of more than 11,000 researchers worldwide found a growing “reviewer fatigue”. As a result, journal editors must now invite, on average, a greater number of peer reviewers to referee manuscripts to get each review completed.

https://www.nature.com/articles/d41586-019-01198-3?utm_source=Nature+Briefing&utm_campaign=b5d0e19ae2-briefing-dy-20190418&utm_medium=email&utm_term=0_c9dfd39373-b5d0e19ae2-44039353

Why the Medical Research Grant System Could Be Costing Us Great Ideas

On By A.P.In UncategorizedLeave a comment

By Aaron E. Carroll

The medical research grant system in the United States, run through the National Institutes of Health, is intended to fund work that spurs innovation and fosters research careers. In many ways, it may be failing.

It has been getting harder for researchers to obtain grant support. A study published in 2015 in JAMA showed that from 2004 to 2012, research funding in the United States increased only 0.8 percent year to year. It hasn’t kept up with the rate of inflation; officials say the N.I.H. has lost about 23 percent of its purchasing power in a recent 12-year span.

Because the money available for research doesn’t go as far as it used to, it now takes longer for scientists to get funding. The average researcher with an M.D. is 45 years old (for a Ph.D. it’s 42 years old) before she or he obtains that first R01 (think “big” grant).

Given that R01-level funding is necessary to obtain promotion and tenure (not to mention its role in the science itself), this means that more promising researchers are washing out than ever before. Only about 20 percent of postdoctoral candidates who aim to earn a tenured position in a university achieve that goal.

This new reality can be justified only if those who are weeded out really aren’t as good as those who remain. Are we sure that those who make it are better than those who don’t?

A recent study suggests the grant-making system may be unreliable in distinguishing between grants that are funded versus those that get nothing — its very purpose.

When a health researcher believes she or he has a good idea for a research study, they most often submit a proposal to the N.I.H. It’s not easy to do so. Grants are hard to write, take a lot of time, and require a lot of experience to obtain.

After they are submitted, applications are sorted by topic areas and then sent to a group of experts called a study section. If any experts have a conflict of interest, they recuse themselves. Applications are usually first reviewed by three members of the study section and then scored on a number of domains from 1 (best) to 9 (worst).

The scores are averaged. Although the bottom half of applications will receive written comments and scores from reviewers, the applications are not discussed in the study section meetings. The top half are presented in the meeting by the reviewers, then the entire study section votes using the same nine-point scale. The grants are then ranked by scores, and the best are funded based on how much money is available. Grants have to have a percentile better than the “payline,” which is, today, usually between 10 and 15 percent.

Given that there are far more applications than can be funded, and that only the best ones are even discussed, we hope that the study sections can agree on the grades they receive, especially at the top end of the spectrum.

In this study of the system, researchers obtained 25 funded proposals from the National Cancer Institute. Sixteen of them were considered “excellent,” as they were funded the first time they were submitted. The other nine were funded on resubmission — grant applications can be submitted twice — and so can still be considered “very good.”

They then set up mock study sections. They recruited researchers to serve on them just as they do on actual study sections. They assigned those researchers to grant applications, which were reviewed as they would be for the N.I.H. They brought those researchers together in groups of eight to 10 and had them discuss and then score the proposals as they would were this for actual funding.

The intraclass correlation — a statistic that refers to how much groups agree — was 0 for the scores assigned. This meant that there was no agreement at all on the quality of any application. Because they were concerned about the reliability of this result, the researchers also computed a Krippendorff’s alpha, another statistic of agreement. A score above 0.7 (range 0 to 1) is considered “acceptable.” None were; the values were all very close to zero. A final statistic measured overall similarity scores and found that scores for the same application were no more similar than scores for different applications.

There wasn’t even any difference between the scores for those funded immediately and those requiring resubmission.

Neuroscientist Says NIH Funding Squeeze Causing ‘Crisis’ in Biomedical Enterprise

On By A.P.In Jeannie Baumann, Mark Tessier-Lavigne, National Institutes of Health, Neuroscience, neuroscientist, NIH, NIH funding, Pete Cuomo, ScienceLeave a comment

By Jeannie Baumann

Many scientists now spend more time scrambling to raise money for their work than actually doing the research because of the erosion of NIH funding over the last decade, the president of a biomedical research university said during a June 18 congressional briefing.

Mark Tessier-Lavigne said the 25 percent decline in the National Institutes of Health’s purchasing power has led to grants being funded at historically low rates, causing promising young scientists to leave the field altogether and threatening the future of the biomedical research workforce.

“The financial squeeze has triggered a crisis in the biomedical research enterprise,” according to Tessier-Lavigne, who is president of the Rockefeller University in New York and investigates how neural circuits in the brain form during embryonic development. “Renewing NIH funding is an essential investment, not just for our health, but also for our economy.”

Tessier-Lavigne was the main speaker at the Capitol Hill briefing, “Paying Dividends: How Federally Funded Biomedical Research Fuels the Pharmaceutical Industry in the U.S.,” which was organized by the Coalition for the Life Sciences and theCongressional Biomedical Research Caucus as part of the 2014 caucus briefing series.

The key point of Tessier-Lavigne’s presentation—that scientific opportunity has never been greater while federal funding for basic research is at a low—has been echoed, especially by NIH Director Francis S. Collins when testifying before lawmakers in both the House and the Senate.

“We live in a golden age of biological research, of disease research, and of drug discovery that’s been enabled by a revolution in the biosciences that’s occurred over the past 40 years, thanks to the development of very powerful technologies,” said Tessier-Lavigne, citing as examples recombinant DNA, gene sequencing, human genetics and imaging. “We can now tackle disease systematically and that is enabling systematic drug discovery.”

The research ecosystem requires early investment through NIH funding to academia to yield the treatments and cures from the pharmaceutical industry, Tessier-Lavigne said.

“There’s a division of labor,” he said. “Most of the scientific discovery that leads to the insights that are built upon are made in academia, in research labs, in research institutes, in universities supported by the NIH. At the other end of the spectrum, industry—mostly large pharmaceutical companies and large biotech companies—are responsible for making the drugs and taking them through human clinical trials.”

Tessier-Lavigne has worked at both ends of the spectrum, serving as chief scientific officer at biotechnology company Genentech before taking over at Rockefeller. He rejected the idea that drug companies could take on funding the basic research. The cost and time lines of drug discovery and development are already too great, he said.

“To make a drug, to get a drug approved there’s huge attritions,” he said. The process starts with targeting 24 projects, and scientists try to make drugs to fight them that yields on average about nine drug candidates that make it into clinical trials.

“But of those nine, only a single one will make it over the finish line as an approved drug,” he said.

That drug-making process takes an average of 13 years, including five years to make the drug candidates and eight years to get to clinical approval. Including failures, he estimated those costs at anywhere between $2 billion to $4 billion per drug.

“So companies that do this are already struggling to succeed just at this. There are no more resources to fund the ferment back here that leads to the identification of new knowledge. The companies can’t do it and they won’t do it,” he said.

“Couldn’t we just rely on other nations to generate the basic knowledge and then industry here could continue to do the translational work?” Tessier-Lavigne asked rhetorically.

“Well, that’s not how it works. Industry wants its R&D [research and development] sites to be located next to the sites of innovation. It’s as simple as that,” he said.

Over the past 30 years, Tessier-Lavigne said, there has been a “massive” transfer of industry from Europe to the U.S. because of the prominence of the U.S. biomedical enterprise.

“If we don’t maintain, sustain our investment in our basic biomedical enterprise, industry will pick up and move to the other sites,” he said, adding that countries like China are where these companies will move, taking jobs with them.
Rep. Jackie Speier (D-Calif.), co-chairman of the Congressional Biomedical Research Caucus, also mentioned that the U.S. may lose its position as the leader in R&D.

“We still lead in terms of patents and overall research, but China is about to eat our lunch,” said Speier, whose district includes the Bay Area and Genentech’s headquarters. “In fact, China has just about eclipsed Japan now in terms of research and within the next 10 years, it is anticipated that they will indeed overcome us in terms of research and development. And that would indeed be a tragic set of circumstances.”
Action Plan

Tessier-Lavigne proposed an action plan that primarily involves gradually restoring NIH funding in absolute dollars to its 2003 level—the final year of a five-year doubling. Since the 2003 doubling, the NIH’s budget has remained flat at about $30 billion. Collins has said that his agency would have about a $40 billion annual budget if the NIH had continued to receive the steady, 3 percent increases it received from the 1970s onward.

Restoring funding to the 2003 levels would relieve the squeeze on existing programs so scientists can focus on their work as well as stimulate new initiatives to accelerate progress and open new areas of discovery, Tessier-Lavigne said.

At the same time, the academic sector has a responsibility to make sure it spends these dollars effectively while developing a pipeline of new talent. And all stakeholders—academia, the NIH, disease foundations and the private sector—must ensure research discoveries are effectively translated into new therapies and cures.

The next congressional briefing is scheduled for July 16 on the advances and potential of embryonic stem cell research, withLawrence Goldstein, director of the University of California, San Diego, Stem Cell Program.

Thanks to Pete Cuomo for bringing this to the attention of the It’s Interesting community.