After handing in my NWO VIDI application last October, I had to wait for the referee reports to come in. Once NWO forwarded them to me, I had 5 days to write a reply to their comments and objections. Based on their peer review and the applicants reponses, all project proposals would be ranked and the top 20% or so would be invited for interviews. I will be hearing soon if I will have made it to the interview stage this year.

Before addressing some of the comments I received, I’d like to mention some of the questions that the referees are requested to answer. Among the first questions is the very straightforward one: “Does the applicant belong to the top 10-20% of his/her international peer group?” Which makes me wonder what my “international peer group” would be. Scholars in philosophy at large or only in history of philosophy? Researchers with permanent contracts or postdocs that drift from temporary contract to temporary contract every few years? I know that there are applicants that are already full professors that apply for this grant, while I am a temporary part-time lecturer at the moment. Perhaps they are just thinking of my age cohort, i.e. applicants who obtained their PhD around 2009?

This invites precisely the kind of criticism I got from “Reviewer 2”: “His profile would have been even stronger if there were more evidence of citations and influence within the field, although there is some evidence of this.” and “Although publication record is strong, greater evidence of applicant’s influence through citations would have strengthened their profile.” Clearly the criterion used to assess whether I belong to the top 20% of my international peer group is citations. This is highly problematic in my opinion, and I said so also in my reply. Many journals in philosophy (e.g. Brentano Studies, Meinong Studies, The New Yearbook for Phenomenology) are not tracked in databases like SCOPUS, which only registered 5 of my nearly 40 publications and reports only 2 citations. Google Scholar in some way finds 97 citations, but among other things misses the five from the Stanford Encyclopedia of Philosophy. This situation makes it really hard to assess the impact and reception of philosophy scholars through citations. If one is not part of the niche itself, however, what would be good evidence? Maybe alt-metrics, that track social media shares of scientific works should be part of the picture? Maybe invitations to speak at international conferences? Public recognition?

In the current situation, the grant agency NWO delegates a significant part of the assessment to the referees, which apparantly then delegate a significant part of their assessment to the reaction of the scholarly community. If we had perfect tracking of every citation, would that mean that we could assign funding simply directly to the top-cited 10-15%, sight unseen? That would mean turning the Matthew effect into a policy. Yet, if the referees themselves were really part of the peer group, they should just be able to assess the value of a colleague based on having read some of their publications. Shouldn’t actual acquaintance with the results of the research have some weight in the ranking? Maybe this is unfeasible given the current numbers of academic philosophers and applicants, i.e. the peer group is too broad, maybe this would lead to favoritism and in-crowds, i.e. the peer group is too narrow. Yet the current situation definitely favours candidates who play it safe, write things that are not controversial, not too innovative, not too risky, because they need the external recognition to get grant money. Clearly, I should have listed my most-cited works as my most important ones, not those that I actually consider the best; older work that has had time to accumulate citations, not recent breakthroughs that reflect my current thinking and goals.

Referee 1, who gave me the highest ranking, noted that my hypothesis sounded almost “paradoxical” and the research risky, but added “no pain no gain”. Indeed, a grant application scheme that wants to support true innovation should value unusual and daring, not established and safe “me-too” research. I look forward to hearing what the commission thought of my reply.