The Null-Result Portal

Motivation

The Null-Result Portal originates from an initiative of psycholinguistics and computational linguistics/computer science. Especially in the field of empirical linguistics and psycholinguistics, data are collected with human subjects and the results are ideally published as journal papers (including data sets and analysis scripts). The high pressure on researchers to publish large numbers of papers leads to the fact that scientific exchange takes place ONLY through publications, as they are the currency through which one can establish, position, or consolidate one’s standing. On the other hand, the pressure to publish leads to a flood of papers, so that (rightly) calls are being made for more quality than quantity. The peer review system is reaching its limits, because it can hardly keep up with the large quantity of often only average or incremental papers that must undergo quality control.

One solution could be to increase the scientific exchange via different channels: not every result has to be published – instead, only the really interesting, surprising or novel projects and results are (again) published as paper – and the smaller, sometimes not so clearly interpretable results are shared with colleagues in other ways. It is obvious that they should be shared: even studies that are not suitable for publication can usually provide information about the subject or the method used, but this is primarily relevant for those who are dealing with very similar topics.

So how could one share results with colleagues that do not necessarily have to be included in a publication? This must be as easy as possible, because there is never enough time anyway, and as many (potential) colleagues as possible should be reached.

Our suggestion is to use and further develop the “Null-Result” platform on http://null-result.uni-leipzig.de/. This platform offers exactly the desired functionality: studies that do not produce revolutionary results but nevertheless provide useful empirical data can be described there via an input mask and put online. Conversely, one can of course also search for studies on one’s own topic or method using a category or keyword search while planning a project.

The platform is thus different from all previous scientific media. It is intended to serve the informal, easy exchange of information and, if there is interest among colleagues, to encourage further enquiries. This means that the quality-assuring but costly peer review process and even writing manuscripts, is no longer necessary. At the same time, a contribution CANNOT directly extend the own publication or citation list. The listed studies would not be citable without further ado and only serve the purpose of informal exchange.

In order to facilitate this informal exchange, the platform offers the possibility to comment on and discuss entries. To some extent, this can even be seen as a reversal of the previous procedure: instead of first carrying out a peer review and then publishing it, first publishing and then peer reviewing is carried out. In contrast to the traditional peer review, this process is open to those who have a concrete interest in the study and have already dealt with the topic. Because of the reversal of publication and peer review, entries should and can be updated, expanded or deleted. A lively discussion and many changes can then even be an indicator of relevance. In turn, it is up to the user to critically review entries and assess their quality, instead of blindly trusting the peer review system and the good reputation of the journal in which the article was published.

There are of course alternative media and channels of exchange, such as preprints.org, the “Journal of Articles in Support of the Null Hypothesis”, or the arXiv. So why another platform? Many results remain unpublished because it is not worth writing a whole article. Nevertheless, they are valuable and should be shared, but not completely unstructured. The Null-Result Portal offers the possibility to publish null results or results that are difficult to interpret and to provide them with additional documentation. Similar to an abstract, for each result the underlying hypothesis, a short description and the most important findings must be formulated. This offers two advantages: on the one hand, this allows a much more efficient search, which also runs across the hypothesis fields that have been formulated; on the other hand, one is forced to clearly highlight the essence of the study and (for the first time) also the errors or disadvantages. This makes it easier for other scientists to understand what the result reflects and what should be avoided or considered in future projects of this kind.

Objectives

For such a platform to work, it must be guaranteed to exist and be maintained over the long term. Also, a possibility to reference entries (even if not to cite them in the classical sense) will be essential for the success of the platform. This means, for example, that the DOI system would have to be introduced. Only consistency, maintenance and referencing can convince scientists that it is worthwhile to contribute their research results here.

Review by community

Individual researchers from the psycholinguistic community in Germany and the USA have already been asked to provide feedback on the existing portal design. Suggestions have been incorporated into the motivation and goals (see above).