The author–reviewer game

At the peer review stage, a natural metric to measure the performance of the author is the quality of the revised manuscript, while a natural metric to measure the performance of the reviewer is the mismatch cost between the manuscript quality and the journal standard. This mismatch refers to incorrectly or unsuitably matching the manuscript quality and the journal standard. The matching between a submission and the journal standard seeks to ensure that the manuscript quality acceptably matches the journal standard. However, we show that journals will not be able to align the interests of the author and reviewer with its own interests because these metrics create a distortion in the reviewer’s incentives. The motivational aspects of traditional peer review mechanisms are not sufficient in relation to the reviewer’s effort required to reach the best level for the journal, because this method of compensation for the reviewer side is overly mismatch cost focused. In this paper, we also show that if this compensation is extended to reward the reviewer based on the manuscript quality achieved, the journal will be able to align the interests of all parties.

This research was sponsored by the Spanish Board for Science, Technology, and Innovation under Grant TIN2017-85542-P, and co-financed with European FEDER funds. We would like to thank the reviewers for their thoughtful comments and efforts towards improving our manuscript.

Authors and Affiliations

1. Departamento de Ciencias de la Computación e I. A., CITIC-UGR, Universidad de Granada, 18071, Granada, Spain

   J. A. Garcia, Rosa Rodriguez-Sánchez & J. Fdez-Valdivia

Corresponding author

Correspondence to J. A. Garcia.

Appendix 1: Benchmark solution for the author–editor game

Following the same approach used in Jerath et al. (2007) to find a benchmark outcome, here we assume that the journal can dictate the quality enhancing effort by the author and the reviewer. Therefore, the journal will dictate to them the optimal decisions and accordingly compensate the author and the reviewer for their respective efforts. We assume that the quality uncertainty \(\epsilon\) is distributed as Uniform[\(-\sigma\), \(\sigma\)], and thus the quality of the revised manuscript \(D(A) = A + \epsilon\) is distributed as Uniform[\(A-\sigma\), A + \(\sigma\)]. The expected effort disutilities for both the author and the reviewer are \(E V_\mathrm{a} (A) = V_\mathrm{a} (A) = \frac{A^2}{2}\) and \(E V_\mathrm{r} (Q) = \frac{k_0 Q^2}{2}\) with \(k_0 = 1 + w\), where \(w = \chi ^2 E [ \zeta ^2 ]\) and represents the uncertainty in the manuscript quality.

Recall that the total compensation for an author is given by \(S_\mathrm{a} = w_\mathrm{a} + \alpha _\mathrm{a} P_\mathrm{a}\) comprising a fixed part \(w_\mathrm{a}\) and a variable part \(\alpha _\mathrm{a} P_\mathrm{a}\), and the compensation for a reviewer is \(S_\mathrm{r} = w_\mathrm{r} + \alpha _\mathrm{r} P_\mathrm{r}\) comprising a fixed part \(w_\mathrm{r}\) and a variable part \(\alpha _\mathrm{r} P_\mathrm{r}\). Also, bear in mind that the author’s utility is \(\varPi _\mathrm{a} = S_\mathrm{a} - V_\mathrm{a} (A)\), and the reviewer’s utility is \(\varPi _\mathrm{r} = S_\mathrm{r} - V_\mathrm{r} (Q ).\)

We suppose that both the author and the reviewer have outside options, i.e., if they choose not to work with the journal each one has a fall-back option that gives them a certain level of utility. We normalize this to zero without any loss of generality. Then, the journal must ensure that the expected utilities of the author and the reviewer are equal to their utilities from the outside options. In this case the conditions \(\max _{A} E[ S_\mathrm{a} | Q , A ] - V_\mathrm{a} (A) \ge 0, \text{ and, } \max _{Q} E[ S_\mathrm{r}| Q , A ] - V_\mathrm{r} (Q ) \ge 0\) are binding. Hence, the journal sets the fixed components of the compensations as

and

To simplify the notation, we will denote \(E [ X | \varOmega _1, \ldots , \varOmega _k ]\) by EX from now on.

Using all this in the expression for profit, the journal solves for optimal values of Q and A to maximize

with \(Y = \min \{ Q, D(A) \}\) and \(C =c_u U + c_b B\). Now, considering that \(U = Q - Y\) and \(B = D(A) - Y\), it follows that

which is independent of \(\alpha _\mathrm{a}\) and \(\alpha _\mathrm{r}\).

Under these assumptions, we have to prove that the solution for this problem is unique for any arbitrary distribution of manuscript quality uncertainty \(\epsilon\). To this aim, we must show that \(E \varPi _j\) is concave, or equivalently, the Hessian matrix is negative semidefinite.

First, we will show that diagonal elements of the Hessian are negative. The first derivatives are

and

where \(D(A) \sim\) Uniform[\(A-\sigma\), A + \(\sigma\)]. Therefore, the second derivatives are

and

and

Then, to prove that both diagonal elements of the Hessian are negative \(\frac{\partial ^2 E \varPi _j}{\partial Q^2} <0\) and \(\frac{\partial ^2 E \varPi _j}{\partial A^2} < 0\), we only have to see that \(\frac{{\text {d}}^2 E V_\mathrm{r}(Q)}{{\text {d}} Q^2} >0\) and \(\frac{{\text {d}}^2 V_\mathrm{a}(A)}{{\text {d}} A^2} >0\) which is implied by the fact that disutilities functions \(V_\mathrm{a}\) and \(V_\mathrm{r}\) are convex by definition. Besides, the determinant is positive when

or equivalently

which is clearly true.

Therefore, we get that the Hessian matrix is negative semidefinite and the benchmark solution for the author–reviewer game is unique. Moreover, the optimal solution for this maximization can be obtained by the first order conditions of the optimization problem (a mathematical condition for optimization stating that the first derivatives are zero). Hence, we get this unique solution by substituting in the first-order conditions \(\frac{\partial E \varPi _j}{\partial Q} = 0\) and \(\frac{\partial E \varPi _j}{\partial A} = 0\) and solving simultaneously for the equations to get

and

with \(k_0 = 1 + w\) as stated above.

Appendix 2: Misalignment between author and reviewer

Given their compensations, the author and the reviewer solve for \(A^*\) and \(Q^*\) respectively:

where \(S_\mathrm{a} = w_\mathrm{a} + \alpha _\mathrm{a} Y\), with \(Y = \min \{ Q, D(A) \}\), and

with \(S_\mathrm{r} = w_\mathrm{r} - \alpha _\mathrm{r} C\), where \(C =c_u U + c_b B\). Hence, the author and the reviewer solve:

or equivalently (using the substitutions \(C =c_u U + c_b B\), \(U = Q - Y\), \(B = D(A) - Y\), and \(Y = \min \{ Q, D(A) \}\))

For these problems, the first derivatives are

where \(D(A) \sim\) Uniform[\(A-\sigma\), A + \(\sigma\)]. Furthermore, the second derivatives are

which should be positive to ensure unique utility maximizing solutions for the author and the reviewer, and thus we get the conditions

Under these constraints, the first-order conditions \(\frac{{\text {d}} E \varPi _\mathrm{a}}{{\text {d}} A} = 0\) and \(\frac{{\text {d}} E \varPi _\mathrm{r} }{{\text {d}} Q} = 0\) give the unique optimal values of \(A^*\) and \(Q^*\)

and

Then, given these optimal values of \(A^*\) and \(Q^*\), the scholarly journal solves for optimal values of \(w_\mathrm{a}, \alpha _\mathrm{a}, w_\mathrm{r}, \alpha _\mathrm{r}\) to maximize the journal’s net profit. Again, the journal must ensure that the expected utilities of the author and the reviewer are equal to their utilities from the outside options. Therefore, the conditions \(\max _{A} E[ S_\mathrm{a} | Q , A ] - V_\mathrm{a} (A) \ge 0, \text{ and, } \max _{Q} E[ S_\mathrm{r}| Q , A ] - V_\mathrm{r} (Q ) \ge 0\) are binding. Hence, the journal sets the fixed components of the compensations as

and the journal’s optimization problem \(\max _{\{ \alpha _\mathrm{a}, \alpha _\mathrm{r}\}} E \varPi _j\) is

To find the optimal values of \(\alpha _\mathrm{a}\) and \(\alpha _\mathrm{r}\) we follow the same approach used in Jerath et al. (2007). Given that the benchmark solutions \(Q^\mathrm{benchmark}\) and \(A^\mathrm{benchmark}\) are unique, Eqs. (2) and (5) will be identical if the journal can set compensation parameters such that the benchmark solutions are achieved, i.e., \(Q^* = Q^\mathrm{benchmark}\) and \(A^* = A^\mathrm{benchmark}\). Hence, we get the optimal values \(\alpha _\mathrm{a}^*\) and \(\alpha _\mathrm{r}^*\) by equating the solution to Eq. (2) in “Appendix 1” and Eq. (5). This gives us

and

subject to the conditions \(\alpha _\mathrm{a}^* + 2 \sigma > 0\) and \(\alpha _\mathrm{r}^* (c_u + c_b) + 2\sigma k_0 >0\) which ensure that the first-order conditions maximize the utilities of the author and the reviewer. Note that the condition for the reviewer, \(\alpha _\mathrm{r}^* (c_u + c_b) + 2\sigma k_0 >0\), is satisfied only if

In conclusion, the journal can achieve the benchmark solution for the author–reviewer game using the compensations \(S_\mathrm{a} = w_\mathrm{a} + \alpha _\mathrm{a} Y\) and \(S_\mathrm{r} = w_\mathrm{r} - \alpha _\mathrm{r} C\) only if this condition holds.

Appendix 3: A solution to coordinate the author and reviewer’s actions

The compensation for the author is the same as before \(S_\mathrm{a} = w_\mathrm{a} + \alpha _\mathrm{a} Y\), while the compensation for the reviewer is now

Therefore, the author solves the same problem as in “Appendix 2”. However, the reviewer solves a new problem

or equivalently (using the substitutions \(C =c_u U + c_b B\), \(U = Q - Y\), \(B = D(A) - Y\), and \(Y = \min \{ Q, D(A) \}\))

For this problem, the first derivative is

Likewise, the second derivative is

To find the optimal parameters of the coordinating compensations we follow the same approach used in “Appendix 2”. Thus, we solve the first-order condition \(\frac{{\text {d}} E \varPi _\mathrm{r} }{{\text {d}} Q} = 0\) for \(Q^*\) and equate it to the benchmark solution \(Q^\mathrm{benchmark}\) in “Appendix 1”, i.e., \(Q^* = Q^\mathrm{benchmark}\). This shows us that, in order to achieve the benchmark solution, the optimal pair \(\alpha _\mathrm{r}^*, \beta _\mathrm{r}^*\) for the reviewer’s compensation must satisfy

subject to the condition \(\alpha _\mathrm{r}^* (c_u + c_b) + \beta _\mathrm{r}^* + 2 \sigma k_0 > 0\) which ensures that the first-order condition maximizes the utility of the reviewer.

Reprints and permissions