The instructions given to the program committee members for GROUP 2014 are unusually detailed and well-organized. I think they provide a great jumping-off point for scholars to reflect on how they review as well as write papers. Kudos to Program Co-Chairs David McDonald and Pernelle Bjorn for writing these up — reviewers and authors everywhere should take notice and step up their game!

1: Briefly SUMMARIZE the main points of the submission:

2: Describe the IMPORTANCE OF THE WORK:
-Is it new, original and/or innovative?
-Do they make a case in the paper itself for its originality (relationship to previous work)?
-Is it a useful, relevant and/or significant problem?
-Do they sufficiently motivate the problem?
-Is the work done on the problem useful, relevant and/or significant?
-Did you learn anything?
-Is this submission appropriate for the GROUP conference?

3: Describe the CLARITY OF GOALS, METHODS AND CREDIBILITY OF RESULTS:
-Are the goal(s) clear?
-Do they clearly describe what was done and/or how it was studied?
-Are the method(s) and/or analysis used to achieve the goal appropriate and used correctly?
-Do they provide sufficient data and/or well-supported arguments?
-Do the results and discussion follow from the method and/or argumentation? Are they believable?
-Does the paper cover all the important issues at the appropriate level of detail?
-Do they cite relevant work?
-Is there sufficient detail so that another researcher can replicate (more or less) the work?

4: Describe the QUALITY OF THE WRITING, addressing the following:
-Is the writing clear and concise?
-Is the paper appropriately focused?
-Is the paper well-organized
-Do they provide the right level of detail?
-Will the paper be understandable to the GROUP audience, including international readers?
-Are figures clear?
-Any figures needed/not needed?

5: Provide any OTHER COMMENTS you believe would be useful to the author (including pointers to missing relevant work).

6: SUMMARIZE your ASSESSMENT of the paper, pointing out which aspects described above you weighted most heavily in your rating.