Where do researchers get the money to replicate others' work?

In my field (cognitive science), your supposition that replication is primarily pursued by investigators prior to extension is true. For example, I have an idea while reading a paper on learning targeted movements, I first implement the paradigm as described (either an exact replication or a conceptual replication) and then I extend it to test my idea. Thus, the money for this comes from the same places that the money for all research comes from: largely from government allocations and grants, partly from NGOs and industry.

However, there has a been a recent trend--again, in my field, but as part of a larger movement in the social sciences--to improve replicability of results. This has included fostering availability of code for software projects and pursuing independent replications purely for their own merit. Last semester I attended a talk on a set of these replications, which, along with similar attempts from other labs, were pushing to overturn the original result. In these cases, the money to pursue the replication is probably somewhat independent from other projects of the researcher. This means funding agencies would be unlikely to provide a grant entirely for this work, so it is probably going to be supported as a passenger on previously funded work or on University funds (e.g. start up and salaries).

I can only tell you from the field of chemistry (and for germany). Normaly you only get funding from institutions to produce/research new stuff. Additional you get money (resp. Lab space, chemicals, technicians, positions for phds/post docs) from the university/institute to work on your field.

So how to check others work. Thats easy. Lets say professor A finds a great new reagent to produce something. If you work in a similiar field or you (resp. a lot of times the phds get the ideas and the work) wants to get something different but you want to try his reagent/method you will redo his work to produce somthing new with it.

That means you use his work (and you will hoefull cite him for that) to try some of your ideas. And then you will also see, if what he published was correct.

For the down side (in my expreience),especially if the results were posted in a top tier journal it is very hard to reproduce others work at first. This may be because of the lack of your knowledge, but oftentimes authors "forget" to put in some details/knacks which will make the whole reproducing very hard. This will give Prof. A some more time to use his published method for his own research befor a colleague can copy his approach. Oftentimes something published in a mid or lower tier journal is better to reproduce since the experimental section is oftentimes more detailled or more "realistic". Please note, that this paragraph is only from expreience and I don't want to blame someone special. It`s just what I experienced.

As an addition, I think there a (few) journals/book series where the reviewer should/must reproduce the result first befor he can accept your paper. but that's really rare and I right now can't list you one from my memory.

In human (and non-human animal) behavioral fields such as Psychology and areas of Medicine, which often involve many paid participants and quite a lot of expense, replication is often NOT done "first" or separately from a new experiment! There just isn't the money and time to be so inefficient - so you need to dual-purpose and build upon existing literature instead.

Story Time

I'll use the example that was used to explain it to me, because I think it's a great example of the concept. In the study Effects of Marijuana on Memory (Weil, Zinberg, & Nelson, 1968), the researchers effectively wanted to know if pot screws with your memory - does it make you forgetful, etc? So they designed a simple experiment where they gave some people real pot, and other people fake pot - then they tested their memory. In that experiment, it turned out that the answer was "pot does screw up your memory" - people have a decreased ability to recall things while high on the ganja.

But that was just one small experiment - what if it was just wrong? As a part of the idea of building upon past work to answer more questions, they designed a new study that asked if the effect of pot on memory depended on prior experience with the drug. This is a new question, but note that it assumes that pot has an effect on memory in the first place! In the new experiment they split people into the previous two groups (pot and no pot), but then split those groups as well so they were testing experienced pot heads and people who hadn't smoke it before.

Part of analyzing the results includes comparing the memory to pot heads and non-drug users, as well as comparing the memory of people who did and did not smoke marijuana. This is, effectively, a replication of the previous study! However, the new study made it possible to look at interaction effects - and it happened to turn out that pot only made pot-naive people forget things they encountered while high. True pot heads, it would seem, have the same memory stoned as sober - and stoners sober had no notable difference in memory to non-stoners. Go figure!

If the first time was interesting...

The key in such experiment designs is that not only is a design testing something new (and thus making it worthy of funding and publication in most venues!), but it also serves to re-test (and replicate) previous studies. And this happens pretty darn often!

If the effect is interesting, it naturally raises a lot of other questions - questions that are closely linked to the original effect. In testing these other questions, it is often difficult to even avoid testing the original thing too, and so replication is a natural part of the scientific process in these fields. It also produces a lot of the "controversy" that some fields experience.

My favorite example of this is probably gender/sex differences in intelligence factors. Do men have better visual-spacial reasoning? Do women have better verbal abilities? For years it was thought that these were real, significant effects and gender differences - and more recently they appear to be very close to pure fantasy, with many studies indicating no difference in men and women whatsoever in such areas. At most, it is estimated that both of these effects have the respectively-weaker sex at average outperforming 30-40% of the opposite sex - rather than the 50% expected if the effect were nonexistent, with many studies indeed showing less, no, or reversed differences.

And none of these studies had to be pure replications - any study that looked at IQ needed only to collect basic demographic data from the participants to allow such replications, which they probably were going to get anyway. No extra funding, yet loads of replication!