Scientists can draw very different meanings from the same data, study shows

Giving the same information to multiple scientific teams can lead to very different conclusions, a report published today in Nature shows.

And that’s exactly why two researchers think scientists should share their data with others — well before they publish.

In this experiment, 29 scientific teams were given the same information about soccer games. They were asked to answer the question “Are dark-skinned players more likely to be given red cards than light-skinned ones?” Some scientists found that there was no significant difference between light-skinned and dark-skinned players, whereas others found a very strong trend toward giving more red cards to dark-skinned players. So, even though a pooled result showed that dark-skinned players were 30 percent more likely than light-skinned players to receive red cards, the final conclusion drawn from this exercise — that a bias exists — was a lot more nuanced than it likely would have been if only one team had conducted the analysis.

Part of that attitude has to do with the desire to prevent other research teams from publishing similar results before they have the opportunity.