The slow road of science

Science is an objective means of seeking truth, but it also needs to add human judgment.

suppose you are a psychologist, and you propose a research hypothesis: people know that they may be unconsciously biased against stigmatized groups. If you ask them directly, they may admit it. This seems to be a very simple research method, directly asking "yes or no". However, the best research method is obviously not that simple.

what do you mean by negative stereotypes?

which stigmatized groups are you talking about?

how will you measure people's understanding of their implicit attitudes?

how do you measure their willingness to disclose these attitudes?

these questions can be answered in many different ways. Different research designs may lead to very different findings.

this is demonstrated by a new crowdsourcing study, (crowdsourced study), involving more than 15000 participants and 200 researchers in 24 countries. When different research teams design their own methods to test the same research problem, they get inconsistent and sometimes opposite results. Whether through p-hack (researchers keep trying to calculate until p & lt;.05), or picking among a variety of data, researchers may intentionally or unintentionally push their results to a particular conclusion.

the study was initiated by Eric Ulmann (Eric Uhlmann), a psychologist at (INSEAD) at Insead in Singapore, who first completed a study that provided a data set for 29 research groups and asked them to answer a simple question: "will football referees give more red cards to players with dark skin?" Although they analyzed the same data, no team came up with exactly the same answer. In that case, however, the findings of these groups do point in the same direction.

the red card study shows that the method of analyzing data affects the results , but Ulmann wants to know more about whether the design of the study affects the results of the study. As a result, he launched the latest crowdsourcing study, which is larger and more ambitious, and the results will be published in the Psychological Bulletin (Psychological Bulletin). The URL for data sharing is https://osf.io/9jzy4/.

faster speed and more pressure

HAPPY2020

this project starts with five hypotheses that have been tested in advance, but the results have not yet been published. In addition to the above-mentioned assumptions about implicit connections, there are other issues such as "how people deal with aggressive negotiation strategies" or "what factors make people more willing to accept athletes' use of stimulants". Ulmann and his colleagues asked more than a dozen research teams the same questions without telling them anything about the original study or results. These teams then created their own experiments independently and maintained some design consistency. These studies must be conducted online, and each participant is randomly selected from a common library. Each study design was run twice: one was to select participants from Amazon's Mechanical Turk, and the other was to find participants through a research company called Pure Profile.

the results show that the design of the study will affect the results of the study . For example, in testing the first hypothesis (that people are aware of their unconscious biases), a team simply asked participants to rate the following statement: "regardless of my explicit attitude towards social equality (consciously), I think I will have a negative bias against stigmatized groups (unconsciously)." The researchers concluded that the hypothesis was wrong because people did not report their understanding of implicit negative stereotypes.

to test the same hypothesis, another group asked participants to identify with a political party and then rank a hypothetical member of the opposition. As a result, they found that people were very willing to report their negative stereotypes.

at the same time, the third group showed participants pictures of white, black or overweight men and women (as well as puppies or kittens) and asked participants to provide their instantaneous evaluation of the person (or animal). The results show that people do report more negative connections to stigmatized groups.

at the end of the study, 7 groups found evidence to support the original hypothesis, while 6 groups found evidence that did not support it. Generally speaking, the results of the study did not support people knowing their implicit connections. However, if you look at only one group of studies, it is easy to draw different conclusions. The study found similar patterns for four of the five hypotheses: different research teams produced statistically significant results in the opposite direction. Even when a research question is answered in the same direction, the reported impact is spread throughout the confidence interval. For example, the data generated by 11 of the 13 research groups explicitly support the hypothesis that excessive discounts reduce people's trust in negotiations, while the findings of two groups only suggest that this may happen. Moreover, some groups found that excessive discounts had a big impact on trust, while others found that the impact was small.

Anna Draper (Anna Dreber), an economist at the Stockholm School of Economics and author of the project, said: "these results show that a particular study does not make much sense.Our researchers need to be more careful to say,'I have tested this hypothesis.' What you need to say is,'I've tested it in some very specific way.' More research is needed to prove whether it can be extended to other environments. "

this problem and this method of proof are not unique to social psychology. A recent similar project asked 70 teams to test nine hypotheses using the same data set of fMRI images. As a result, no two teams use exactly the same approach, and the results are very different. If judged solely on the basis of the results of these projects, it is reasonable to guess that the scientific literature will be the epitome of opposing discoveries. But the opposite is true. The journal is full of studies that confirm the effects of hypotheses, while invalid results are discarded in document drawers. Consider the results of the above implicit bias hypothesis: half of the teams found evidence of support and half found evidence of opposition. If this work is carried out with the rapid development of scientific publishing, the former will be rooted in officially published papers, while the rest will be buried and ignored.

the research of Ulmann and his colleagues suggests that hypotheses should be tested in a variety of transparent ways. Dorothy Bishop (Dorothy Bishop), a psychologist at the University of Oxford, said: "We need to do more research to try to look at the same ideas in different ways." In this way, you can "find out the true firmness of the steps before jumping up and down and making a big jump."

Ulmann says psychologists should always be modest: "We must be careful about what is said in articles, what universities say in press releases, and what we say in media interviews." We all hope that scientific research can make great strides forward, but good science may mean slowing down and being more cautious . "

There is no other place you are guaranteed of versatility, only by second dress for wedding reception. Let them shower you with a display of exquisite taste.

more observation and more thinking

happynewyear

psychologist Utafriss (Uta Frith), a psychologist at University College London, published a recent article on Trends in Cognitive Science (Trends in Cognitive Sciences), advocating slowing down. Frith wrote: "the current\ & # 39; publish or perish\ & # 39; culture has a destructive influence on scientists as well as on science itself." The publication of many papers, rather than focusing on the publication of high-quality papers, puts researchers under great pressure and piecemeal in their research. "Rapid science leads to Jerry-building, which is bound to lead to a repeatable crisis," she wrote. And "slow Science" focuses on "larger scientific goals" as a way to find truth . One way to promote slow science is for researchers to look for inspiration from the practice of "craftsmen" winemakers who try their best to limit their wine production in order to maintain the highest quality of wine.

Bishop made a similar suggestion, advising scientists to limit their own output. "in order to develop a theory, you need a lot of observations, but I find that we often have very few observations," she said. It would be better if we explored the scope of these observations, which would avoid coming up with a permanent theory before it could have been better. "

if a lesson is drawn from these studies, it is that science is a process, a process that takes time.

A Buddhist monk asked the Zen master, "Master, how long can I be enlightened with my qualifications?"

the Zen master said, "Ten years."

the monk asked again, "will it take ten years?" If I double my penance, how long will it take me to be enlightened? "

the Zen master said, "it will take twenty years."

the monk was very confused, so he asked, "if I stay awake day and night, just for meditation, how long will it take me to be enlightened?"

the Zen master said, "then you will never be enlightened."

the monk was surprised and said, "Why?"

the Zen master said, "if you only care about the results of meditation, how can you have time to focus on yourself?" When you only see the result and can't calm down, you will never get that result. "

when we pay too much attention to results, the pace of walking will be out of order. We only stare at the result, lose ourselves, and lose all the happiness that we should feel on the road of life.