The COLING 2027 review form, explained

How COLING reviewers read your paper

COLING 2027 does not run its own reviewing: papers are reviewed in ACL Rolling Review and then committed to COLING, whose program chairs decide from the ARR reviews and meta-review. COLING 2027 takes papers from the ARR October 2026 cycle, shared with NAACL 2027; at commitment authors pick one primary venue and may opt in to be considered by the other. ARR reviews are about half a page. The form separates objective Soundness (must be justified in the text) from subjective Excitement, and the Overall Assessment is an outcome recommendation (Award / Conference / Findings / Resubmit), all on 1–5 with half points. Weaknesses are a numbered list specific enough for an AC to follow; suggested extra work goes under Comments. The guidelines list lazy-reviewing heuristics to avoid, e.g. "not novel" without a reference, "not SOTA", "simple method", "only tested on English", "I would have done X", or listing limitations the authors already acknowledge.

• Number the weaknesses; each must be concrete enough for an AC to follow without reading the paper.
• Avoid ARR lazy-reviewing heuristics: no "not novel" without citing the prior work, no "not SOTA" as a reason to reject, no "the method is too simple", no "only tested on English" unless the claims are general, no "I would have done X", no listing limitations the authors already acknowledge as weaknesses.
• Put suggested additional experiments under Comments, not Weaknesses.
• Justify the Soundness score in the text; Excitement is explicitly personal taste.

Where the bar sits. COLING has not historically published a Findings volume; check the COLING 2027 call. In the Overall Assessment, 4 ("Conference") argues for the main conference and 3 ("Findings") for the Findings of the ACL volume, which the program chairs may offer to committed papers that are sound but less exciting. ARR publishes no cutoff; venue senior ACs decide. Rule of thumb: average Overall Assessment ≥ 3.5 → main conference plausible, 3.0–3.5 → Findings, < 3.0 → likely rejected. Main conference acceptance is about 20–25%, plus ~13–15% Findings.

The reviewer form, field by field

In the order reviewers see it, with every score and its official wording.

1. 01

   Paper Summary

   Free text

   Describe what the paper is about so the AC and action editor can catch any misunderstanding.

2. 02

   Summary Of Strengths

   List of points

   The major reasons to publish this paper at a selective *ACL venue.

3. 03

   Summary Of Weaknesses

   List of points

   Numbered list of the concerns that would make you prioritize other high-quality papers. Each must be specific enough for an AC to follow without reading the paper. Do NOT include suggested extra work here (that goes in Comments), and do not list limitations the authors already acknowledge.

4. 04

   Comments Suggestions And Typos

   List of points

   Other improvements that are not part of the concerns above, including suggested additional experiments, presentation fixes and typos.

5. 05

   Confidence

   Score · 1–5

   How confident you are in your evaluation.

   Score	Meaning
   5	Positive that my evaluation is correct. I read the paper very carefully and am familiar with related work.
   4	Quite sure. I tried to check the important points carefully. It's unlikely, though conceivable, that I missed something that should affect my ratings.
   3	Pretty sure, but there's a chance I missed something. Although I have a good feel for this area in general, I did not carefully check the paper's details, e.g., the math or experimental design.
   2	Willing to defend my evaluation, but it is fairly likely that I missed some details, didn't understand some central points, or can't be sure about the novelty of the work.
   1	Not my area, or paper is very hard to understand. My evaluation is just an educated guess.

6. 06

   Soundness

   Score · 1–5 (step 0.5)

   Objective: does the study support its claims? Must be justified in the review text.

   Score	Meaning
   5	Excellent This study is one of the most thorough I have seen, given its type.
   4	Strong This study provides sufficient support for all of its claims. Some extra experiments could be nice, but not essential.
   3	Acceptable This study provides sufficient support for its main claims. Some minor points may need extra support or details.
   2	Poor Some of the main claims are not sufficiently supported. There are major technical/methodological problems.
   1	Major Issues This study is not yet sufficiently thorough to warrant publication or is not relevant to ACL.

7. 07

   Excitement

   Score · 1–5 (step 0.5)

   Explicitly subjective: how exciting you personally find the work.

   Score	Meaning
   5	Highly Exciting I would recommend this paper to others and/or attend its presentation in a conference.
   4	Exciting I would mention this paper to others and/or make an effort to attend its presentation in a conference.
   3	Interesting I might mention some points of this paper to others and/or attend its presentation in a conference if there's time.
   2	Potentially Interesting this paper does not resonate with me, but it might with others in the *ACL community.
   1	Not Exciting this paper does not resonate with me, and I don't think it would with others in the *ACL community.

8. 08

   Overall Assessment

   Score · 1–5 (step 0.5)

   An outcome recommendation (Award / Conference / Findings / Resubmit), not a quality score.

   Score	Meaning
   5	Consider for Award I think this paper could be considered for an outstanding paper award at an *ACL conference (up to top 2.5% papers).
   4.5	Borderline Award
   4	Conference I think this paper could be accepted to an *ACL conference.
   3.5	Borderline Conference
   3	Findings I think this paper could be accepted to the Findings of the ACL.
   2.5	Borderline Findings
   2	Resubmit next cycle I think this paper needs substantial revisions that can be completed by the next ARR cycle.
   1.5	Resubmit after next cycle I think this paper needs substantial revisions that cannot be completed by the next ARR cycle.
   1	Do not resubmit This paper has to be fully redone, or it is not relevant to the *ACL community (e.g., it is in no way related to computational processing of language).

9. 09

   Best Paper Justification

   Free textoptional

   Only if Overall Assessment is 4.5 or 5; otherwise write "N/A".

10. 10

    Limitations And Societal Impact

    Free text

    Have the authors adequately discussed the limitations and potential positive and negative societal impacts of their work?

11. 11

    Ethical Concerns

    Free text

    Concerns with respect to the ACL Code of Ethics and the Responsible NLP checklist. "None" if there are none.

12. 12

    Needs Ethics Review

    Single choice

    Should the paper go to the ethics committee? Answer No or Yes (with a reason).

    • No
    • Yes
13. 13

    Reproducibility

    Score · 1–5

    Could others reproduce the results?

    Score	Meaning
    5	They could easily reproduce the results.
    4	They could mostly reproduce the results, but there may be some variation because of sample variance or minor variations in their interpretation of the protocol or method.
    3	They could reproduce the results with some difficulty. The settings of parameters are underspecified or subjectively determined, and/or the training/evaluation data are not widely available.
    2	They would be hard pressed to reproduce the results. The contribution depends on data that are simply not available outside the author's institution or consortium and/or not enough details are provided.
    1	They would not be able to reproduce the results here no matter how hard they tried.

14. 14

    Datasets

    Score · 1–5

    Value of any newly released datasets (1 if none).

    Score	Meaning
    5	Enabling The newly released datasets should affect other people's choice of research or development projects to undertake.
    4	Useful I would recommend the new datasets to other researchers or developers for their ongoing work.
    3	Potentially useful Someone might find the new datasets useful for their work.
    2	Documentary The new datasets will be useful to study or replicate the reported research, although for other purposes they may have limited interest or limited usability. (Still a positive rating)
    1	No usable datasets submitted.

15. 15

    Software

    Score · 1–5

    Value of any newly released software (1 if none).

    Score	Meaning
    5	Enabling The newly released software should affect other people's choice of research or development projects to undertake.
    4	Useful I would recommend the new software to other researchers or developers for their ongoing work.
    3	Potentially useful Someone might find the new software useful for their work.
    2	Documentary The new software will be useful to study or replicate the reported research, although for other purposes they may have limited interest or limited usability. (Still a positive rating)
    1	No usable software released.

Area chair meta-review

ARR meta-reviews use the reviewers' Overall Assessment scale. Weigh arguments, not averages.

• Conference
• Findings
• Resubmit next cycle
• Resubmit after next cycle
• Do not resubmit

Responsible NLP Research Checklist

Reviewers check these. Gaps are easy points to lose.

1. A1. Limitations section describing the work's limitations (required; missing it is grounds for desk rejection)
2. A2. Potential risks of the work discussed
3. B1. Creators of artifacts used are cited
4. B2. License and terms of use of artifacts discussed
5. B3. Use of existing artifacts consistent with their intended use
6. B4. Data checked for personally identifying or offensive content and anonymized
7. B5. Documentation of artifacts (domains, languages, linguistic phenomena, demographics)
8. B6. Relevant statistics (number of examples, train/test/dev splits) reported
9. C1. Number of parameters, total computational budget and computing infrastructure reported
10. C2. Experimental setup, hyperparameter search and best-found values discussed
11. C3. Descriptive statistics (e.g. error bars) reported, and whether results are single runs or aggregates
12. C4. Implementation, model and parameter settings of existing packages reported
13. D1. Full text of instructions given to human participants/annotators
14. D2. Recruitment and payment of participants, and whether payment was adequate
15. D3. Consent obtained from participants whose data is used
16. D4. Data collection protocol approved by an ethics review board
17. D5. Basic demographic and geographic characteristics of annotators reported
18. E1. Use of AI assistants (in research or writing) disclosed

Policy on LLMs in reviewing