Referee Report
Pretend that you are a peer reviewer for a top general-interest journal, say the American Economic Review. Your job is to advise the editor on whether or not to accept the paper. I expect reports with no less than 3 pages and no more than 4 pages. You will also have to give a 10 min presentation (5-6 slides) summarizing the paper.
A good referee report takes the following format:
1) An introduction that provides a very short overview of the main thrust of the paper (1 paragraph).
2) A summary of the paper, focusing on the main points of the paper and the points that will be important to your critique. A summary that is typically one paragraph and contains a brief statement of the question addressed by the author, an outline of how the author answers the question at hand, and a brief synopsis of the results. (2-3 paragraphs)
3) Referees outline both the strengths and weaknesses of the methodology used by the authors. Many referees make the mistake of only noting what they do not like about the paper. Spend some time discussing what you like and found innovative as well. (3-4 paragraphs)
4) Finally, most referees make some suggestions about how to improve the paper. This can include such details as catching typos, to suggestions about how to motivate or structure the paper, and most importantly, other models or estimation results that can be run that help bolster the hypothesis the author is testing. (2-3 paragraphs)
5) A concluding section (1-2 paragraphs)
The answers required for a good referee report of experimental economics papers:
1. What is the purpose of this paper?
2. Does the paper accomplish what it set out to do?
3. Is the purpose best served by the design? Is there any confound of the design?
4. Is there a relevant and important literature that the author does not cite and/or use when it should be cited and/or used?
5. If the paper contains theory (explicit or implicit), does it hold up upon closer scrutiny? Is there an alternative theory that is better suited that the author has ignored?
6. Are the hypotheses best tested by the statistics tests the authors used? Do the tests or regressions well address possible issues like: the censored data, session fixed effect, individual dependence, and highly correlated dependent variables?
7. Are the tables and figures self-contained and easy to comprehend, and could you, if needed, replicate them?
8. Is this paper well structured?
9. Are the conclusions overstated?
