Reviewer tracking systems: from pool management to turnaround time

What most journals call their “reviewer system” is really this: the editor’s memory, a spreadsheet, and hundreds of emails in an inbox with the subject line “Invitation to review.” The setup holds while the journal gets five submissions a month. What happens when submissions grow, or when the editor changes, everyone knows: the pool retires along with the editor.

What does a journal or press that wants to put reviewer tracking on a systematic footing actually need? Let’s take it piece by piece.

The pool: profiles, not a list

The first step is turning the list of names into profiles. Alongside the name, a reviewer record should hold at least:

• Areas of expertise — searchable subject tags, not free text.
• Language — which languages they can review in; in institutions publishing multilingually, pools can split by language altogether.
• Availability — “not taking assignments this term” needs somewhere to live; otherwise the same person gets three invitations while on holiday.
• History — how many invitations received, how many accepted, average days to return, how many assignments currently open.

That last item is the critical threshold: the pool being able to answer the question “who is available?” In the ideal flow, the system puts a well-aimed shortlist in front of the editor; the editor picks. Nasirus is arranged in that spirit — the suggestion comes from the system, the decision from the editor.

The invitation: not send-and-forget, send-and-watch

An invitation has a lifecycle of its own: sent → seen/not seen → accepted/declined → expired. In a manual process only the two ends of this cycle are known; the middle is darkness. In a systematic process every invitation carries a status and is backed by two automations: a polite reminder on the appointed day, and automatic expiry plus a substitute suggestion after a reasonable period.

The data explains why this matters: acceptance rates per invitation have been falling for years — some journals need more than 20 invitations to secure two reviewers. Once invitation traffic reaches that volume, an unwatched invitation process becomes unmanageable.

External reviewers: keep the door open

However rich your pool, for some works the right name is outside it — at another university, perhaps in another country. Putting a “first, register for our system” barrier in front of that name is the politest possible way to get your invitation declined.

The right structure lets an external reviewer work without opening an account: a secure link of their own, with the files, the form, and the due date at the end of it. Frictionless for the reviewer, fully recorded for the institution.

Anonymity and separation of duties

Reviewer tracking is identity-data management, and it carries two sensitivities at once. The first is the double-blind arrangement — the reviewer not seeing the author, the author not seeing the reviewer — a subject we took up in a separate post. The second is separation of duties: a reviewer sees only the work they are assigned; the pool records, the performance data, and the other works are closed to them. And reviewer data is itself personal data; its retention and access rules fall under data-protection discipline.

Performance: for thanks and for balance

The point of turnaround and completion records is not to keep score on reviewers but two concrete jobs: balance — preventing the load from always landing on the same three volunteers; and thanks — being able to pull the year-end list of the journal’s workhorses straight from a report. Reviewing is invisible labor; the record is also the only way to make that labor visible.

A side note: indexes such as TR Dizin (Türkiye’s national journal index, run by TÜBİTAK ULAKBİM) require the peer-review process to be documented — invitations, reports, decisions. A pleasant by-product of systematic reviewer tracking is that this file assembles itself.

How the pool grows: three feeder channels

Building the system is not enough; keeping the pool alive takes regular feeding. Three channels work in practice:

Authors. Every researcher who submits to your journal is also a potential reviewer — they have already shown you their field, their style, and their rigor. An invitation to accepted authors as their process closes — “would you join our reviewer pool?” — is the most natural growth channel there is.

Citation and reference-list mining. The reference list of the manuscript under review is a ready-made roster of the topic’s active names. The editor never losing a new name found there — that is, processing the invitation as a pool record rather than a one-off email — enriches the pool a little with every manuscript.

Reviewer referrals. The “but my colleague might be a fit” note from a reviewer who declines is the most valuable sentence in any decline email. A referral field on the decline form makes that sentence systematic; even a declined invitation feeds the pool.

There is a maintenance side too: as the pool grows, it also silts up. Records of the retired, the field-changers, and the serially unresponsive should be reviewed periodically. A big dead pool is worth less than a small live one — what raises your invitation hit rate is not the number of records but their freshness.

Five questions for your own pool

To test your current setup:

1. How many reviewers in your pool have up-to-date expertise tags?
2. How many invitations are open right now, and how many days have they gone unanswered? (Can you answer that within a minute?)
3. How many assignments is your most-loaded reviewer carrying?
4. If your editor changed tomorrow, what percentage of the pool’s knowledge would stay with the institution?
5. How many hours would it take to produce a thank-you list for last year’s reviewers?

If three of the five made you pause, your reviewer tracking is entrusted to individuals. To see the systematic version, book a demo — one of the first things we set up is the profiled version of your own pool.