Summary of Changes Since Original Submission (TMLR)

A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis · TMLR

Baseline: original submission of 13 June 2026. This note summarizes the substantive additions and revisions made across the review period. No headline claim from the original submission was retracted; the revisions add evidence, a second real benchmark, and presentation improvements.

The revision keeps the original contribution (a controlled synthetic ABSA corpus plus an audit–filter–validate methodology) and strengthens it on three fronts: the label-faithfulness audit is now validated against humans and independent model families, the synthetic-to-real story is completed on a second real benchmark, and every reported control is expanded to multiple seeds and a second architecture.

Paste-ready cover note to the Action Editor

Post as an Official Comment on the Action Editor thread (OpenReview renders Markdown). Per-reviewer point-by-point responses are the three boxes below.

# Response to the Action Editor

*A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis (TMLR)*

Dear Action Editor,

Thank you, and please pass our thanks to the three reviewers, for a careful and constructive review. We have revised the paper thoroughly and believe every requested change is now in place. Detailed, point-by-point responses to each reviewer accompany this note; below we summarize the significant changes and confirm how the revision aligns with each reviewer's requests.

## Significant changes in this revision

- **Direct human validation of the audit on the synthetic corpus.** Three annotators labeled a stratified sample of the actual synthetic reviews; human confirmation rises monotonically with the audit score, and inter-annotator reliability, sentiment agreement, confidence intervals, and a review-level clustered bootstrap are reported (Table 15).
- **A second real benchmark, completed end to end.** EduRABSA is added as a full multi-seed synthetic-to-real transfer with a real-only reference and a synthetic-pretrain-plus-fine-tune result, reported with uncertainty (Table 10), so the transfer story now rests on two independent annotated corpora.
- **Audit independence and provider-agnostic pipeline.** The faithfulness audit is reproduced by two independent open-weights auditors, and both the generation pipeline and the prompted baseline are reproduced across four generator families spanning four providers.
- **A stronger sample-efficiency experiment.** Figure 6 now shows a real-only-from-scratch learning curve alongside the synthetic-pretrained curve at every data size across five seeds with confidence intervals, and the accompanying claim is stated conservatively.
- **Faithfulness-aware filtering isolated.** A covariate-matched comparison removes composition and prediction-mask confounds, and a quartile dose-response confirms the effect end to end.
- **Framing, transparency, and reproducibility.** The resource is described as quantified noisy synthetic supervision rather than a gold benchmark; a Data and Code Availability section points to an anonymized archive; the real-data mapping procedure and both mappings are documented; and the generation-cost claim is quantified.

We also corrected two factual issues we found while revising (a mislabeled figure panel and a transcription error in an appendix table), tightened the writing, and reconciled the internal numbers and citations.

## Alignment with each reviewer

**Reviewer nfat** asked for direct human annotation of the synthetic corpus, filtering subsets matched on covariates, resolution of the inconsistent numbers and the incomplete abstract, a clearer noisy-supervision framing, and a broader impact discussion. All are addressed: the human study validates the audit on synthetic text; a covariate-matched filtering comparison isolates faithfulness; the abstract is complete and the Table 8 versus Table 9 and aspect-count issues are reconciled through consistent multi-seed reporting; the corpus is framed as measured noisy supervision; and the ethics discussion now covers instructor evaluation, comment privacy, stylistic bias, and the safeguards for institutional use.

**Reviewer h7LN** asked about the token-capped rows, the low-value bottom quartile, the strict 0.42 match and the moderate audit-human agreement, statistical realism and full-schema re-annotation, broader baselines beyond one provider, figure and table formatting, and the narrow evaluation scope. All are addressed: the truncated rows are regenerated and shown to be unbiased; the bottom quartile is exactly the signal the filter exploits; the audit is positioned as a validated selection instrument, not an oracle, and is validated directly on synthetic text; the provider axis is covered across four families on the full test split; the named figures and table were reformatted; and the external evaluation now spans four real corpora with the scope stated explicitly.

**Reviewer dWED** (who assessed the evidence positively) asked for a dedicated discussion of generator-auditor circularity, a bias analysis of the incomplete rows, two additional citations, a stronger statement of transfer limits, and a concrete practitioner roadmap. All are addressed: circularity is settled empirically with independent open-weights and cross-family auditors; the incomplete rows are shown unbiased in aspect, polarity, and faithfulness; both suggested works are cited; the limits of the nine-aspect, partial-recovery transfer are stated prominently; and a fine-tuning-size roadmap with monitoring guidance is provided.

We are grateful for the feedback, which materially improved the paper, and we hope the revision now meets the bar for TMLR.

Sincerely,
The Authors

Paste-ready summary for the OpenReview revision field

Plain text for the “List of Changes” box. Click Copy (or select all inside and copy).

## Summary of Changes Since the Original Submission

We thank the reviewers and the Action Editor for a careful and constructive review. The revision keeps the original contribution (a controlled synthetic ABSA corpus with an audit-filter-validate methodology) and strengthens it: the label-faithfulness audit is now validated against humans and against independent model families, the synthetic-to-real evaluation is completed on a second real benchmark, and every reported control is expanded to multiple seeds and a second architecture. No result or headline claim was withdrawn; the revision adds evidence, broadens the evaluation, and tightens the presentation.

**New empirical results**

1. Direct human validation of the audit on the synthetic corpus: a three-annotator study (Fleiss kappa 0.70 on aspect presence) in which human confirmation of a declared aspect rises monotonically with the audit score and reproduces the presence-faithful, sentiment-noisier pattern (Table 15, Section 5.7).

2. Audit independence: two independent open-weights auditors (Llama-3.3-70B and GLM-4.6) reproduce the audit at cross-family Cohen kappa 0.56 to 0.65 (Appendix A.4.4).

3. Provider-agnostic pipeline: the generation pipeline and the prompted baseline both reproduce across four generator families spanning four providers (Appendix A.3.2, A.6.2).

4. A second real benchmark, EduRABSA, evaluated end to end: a multi-seed synthetic-to-real transfer table, a real-only reference, and a synthetic-pretrain-plus-fine-tune result (Table 10).

5. Cross-architecture and multi-seed controls: the faithfulness-filtering contrasts are replicated on DistilBERT alongside BERT and extended to eight seeds, with a covariate-matched comparison isolating the filtering effect (Table 14, Table 13).

6. Signal-validity controls: label permutation, training-size scaling, and a clean-label ceiling establish learnable signal rather than label priors.

7. Truncated-row robustness: the 841 output-token-capped rows are regenerated at a higher cap and shown to leave the benchmark unchanged in aspect, polarity, and faithfulness (Appendix A.1.5, A.1.6).

**Presentation and structure**

8. The abstract is complete and tightened, Section 1.1 is de-duplicated, and the paper-structure paragraph now closes the Introduction.

9. All figures use white backgrounds with enlarged, non-overlapping labels; Figure 2 is redesigned with labeled panels, and Figures 4 and 5 are re-laid for legibility.

10. Table 1 formatting and inter-table spacing are standardized, and every table and figure is introduced in the immediately preceding paragraph.

11. The reference list is alphabetized by author surname (TMLR convention) with citations renumbered, and recent (2024 to 2026) citations are added.

12. The appendices are organized into seven thematic sections with subsections.

13. The corpus, datasets, human study, and model checkpoints are released as a versioned archive.

**Further strengthening in this round**

14. Figure 3's left panel now plots the Herath nine-aspect overlap, consistent with Table A3.

15. The EduRABSA transfer is reported with full uncertainty (Table 10 gives mean and standard deviation across eight seeds on all metrics, with the seeds and the train and test sizes), and the appendix screening table now reports the eight-seed EduRABSA value (0.448) consistent with Table 10.

16. Figure 6 now shows a real-only-from-scratch learning curve alongside the synthetic-pretrained curve at every sample size across five seeds with confidence intervals; the accompanying statement describes a consistent per-budget pre-training boost and roughly double data efficiency in the low-data regime, with the two curves converging near the full real set.

17. The human study now reports inter-annotator agreement for sentiment, Wilson intervals on the quartile confirmation rates, and a review-level clustered bootstrap, and states the annotation protocol (each candidate aspect shown, its declared presence and sentiment hidden) and the Table 15 column definition (declared aspect decisions).

18. A Data and Code Availability section (Section 6.4) gives the anonymized Zenodo DOI (10.5281/zenodo.21717252) and lists the archive contents (corpus, fixed splits, prompts, schema, both mappings, audit scores, scripts), and states that the released corpus ships the regenerated higher-cap rows together with the original rows.

19. Both real-data mappings are documented in a new appendix (A.8) with parallel Herath and EduRABSA source-label tables and the EduRABSA support and polarity distribution.

20. The generation cost is quantified (about US$3 for the full pipeline), roughly three orders of magnitude below fair-pay human annotation per label (Section 6.2).

21. The pre-training comparison is stated as matching or slightly exceeding real-only training, and the transfer metric is described as external real-data evaluation; the generation description reflects the inter-cycle prompt-revision loop; and the M-ABSA dataset is cited.

Counts: main tables 12 to 15, appendix tables 15 to 30, main figures 5 to 6, references 41 to 42. Point-by-point replies to each reviewer accompany the revision.

Paste-ready answer for the Human Subjects Reporting field

Enter this in the submission's “Human Subjects Reporting” field. The label study used research-team annotators labeling synthetic, non-personal text, so it is not human-subjects research; all real-data benchmarks are pre-existing public datasets.

N/A. The label-validation study used three annotators from the research team to label synthetic, LLM-generated course reviews for aspect presence and sentiment. No personal or identifying information about the annotators or any real individual was collected or analyzed, and all real-data benchmarks (Herath, EduRABSA, OMSCS) are pre-existing public datasets not collected by the authors. This activity does not constitute human-subjects research and did not require Institutional Review Board (IRB) approval.

Official responses to reviewers

Post each block as an OpenReview Official Comment in that reviewer's thread (OpenReview renders Markdown).

Reviewer nfat — official response

# Response to Reviewer nfat

*A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis (TMLR)*

We thank Reviewer nfat for a careful and constructive report. Both the validation of the audit and the internal consistency of the reporting are now stronger. We answer each point with the change and its location.

### 1. Human validation of the audit on the synthetic corpus

**Requested.** Human-annotate a representative sample of the actual synthetic corpus and report aspect and sentiment agreement.

Done (new Table 15). Three annotators labeled 300 synthetic reviews (610 declared aspect decisions), shown each candidate aspect but blind to its declared presence and sentiment. Human confirmation of the declared aspect rises monotonically with the audit score (0.55 to 0.79 across quartiles, with Wilson intervals and a review-level clustered bootstrap), and the human labels reproduce the audit's presence-faithful, sentiment-noisier split (human sentiment agreement 0.38, matching the audit's strict 0.42). Inter-annotator reliability is substantial for presence (Fleiss 0.70) and moderate for sentiment (Fleiss 0.49). The audit score is therefore a human-validated selection signal on the synthetic corpus.

*Location:* Section 5.7 (Table 15).

### 2. Match filtering subsets to isolate faithfulness

**Requested.** Match retained and control subsets by aspect, polarity, aspect count, length, and style.

Done (new Table 14). A covariate-matched comparison pairs the retained and control subsets one-to-one on aspect set, aspect count, polarity, length band, and formality (3,441 pairs, audit-score means 0.90 versus 0.26) and scores both on the same gold-present cells, removing the prediction-mask confound. The faithfulness-retained subset has lower transferred sentiment error in every seed (0.412 versus 0.519) and higher detection micro-F1 (0.400 versus 0.338), so the gain is attributable to faithfulness alone.

*Location:* Section 5.7 (Table 14).

### 3. Reconcile inconsistent numbers

**Requested.** Resolve the Tables 8 and 9 BERT scores, the aspect-count totals, and the incomplete abstract.

Done. The aspect-count totals sum to 10,000 (3,032 + 3,917 + 3,051) and the abstract is complete. The Table 8 versus Table 9 discrepancy came from two single-seed runs; the transfer is now a multi-seed table, so every score traces to one consistent set of runs.

*Location:* Abstract; Section 5.1; Section 5.4 (Tables 8 and 9).

### 4. Shorten repeated discussion

**Requested.** Shorten repeated discussion.

Done. The generation process is stated once, the shared-split contract is cross-referenced, and the Section 5.8 recap is collapsed into a short bridge, while every experiment and caveat is retained.

*Location:* Sections 3, 5.8, 6.

### 5. Frame the resource as noisy synthetic supervision

**Requested.** Describe the resource as noisy synthetic supervision rather than a gold benchmark.

Done. The abstract, Section 6.2, and conclusion describe a controlled synthetic-supervision resource whose label faithfulness is measured (strict 0.42 per-aspect, 0.58 per-row) and controlled by the audit-and-filter pipeline, not a gold-labeled corpus.

*Location:* Abstract; Section 6.2; Section 7.

### 6. Broaden the impact discussion

**Requested.** Address instructor evaluation, comment privacy, bias against non-native writing, and fictional negative reviews, and require human review, uncertainty reporting, data protection, and an appeals process.

Done. The ethics statement adds stylistic-bias monitoring for non-native and non-standard writing, a prohibition on attaching model-inferred negatives to identifiable courses or instructors, and per-aspect uncertainty with low-confidence routing to human review and an appeals process, on top of the existing no-identifiable-data, licensing, and re-consent provisions.

*Location:* Section 6.3.

### 7. Rule out generator-specific split effects

**Requested.** Random splits from the same generator and prompt may reward generator-specific patterns.

Clarified, with a new control. Regenerating and auditing with three independent model families reproduces the label fidelity, and a held-out-generator check shows a detector trained on one generator's data transfers to other generators' data without collapse.

*Location:* Appendix A.3.2, A.5.4, A.3.3.

---

Locations reference the revised manuscript; every requested change is in place at the cited location.

Reviewer dWED — official response

# Response to Reviewer dWED

*A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis (TMLR)*

We thank Reviewer dWED for a generous and constructive review. We are pleased the experimental design, filtering pipeline, and transfer evidence came through clearly, and we have strengthened each noted point below.

### 1. Generator-auditor circularity

**Requested.** The same provider family generates the data and performs the audit; a dedicated discussion of circularity is needed.

Done, with a new experiment. We re-ran the audit with two open-weights auditors from independent families (Llama-3.3-70B and GLM-4.6). All three families converge on the same per-aspect judgments (support-rate 0.77, 0.74, 0.73; cross-architecture Cohen's kappa 0.56 to 0.65), and the two open-weights auditors agree with each other at the same level, so the GPT auditor is not privileged. A same-family artifact would make out-of-family auditors diverge; instead they reproduce the audit, which shows it measures textual faithfulness.

*Location:* Section 6.1; Appendix A.3.2, A.4.4.

### 2. Bias analysis of the incomplete rows

**Requested.** Analyze whether the 841 incomplete rows are biased across aspects or polarities, and whether excluding them changes results.

Done. The truncated rows are not systematically biased across aspects (Cramer's V=0.034) or polarities (V=0.020), and their audit faithfulness matches the complete corpus (0.573 versus 0.577, p=0.76); the only differences are mechanical (fewer aspects, shorter text), so excluding them leaves the benchmark unchanged.

*Location:* Appendix A.1.5, A.1.6.

### 3. Broader related-work discussion

**Requested.** Discuss emerging methods, including multimodal sarcasm perception in vision-language models and set-matching for generalized category discovery.

Done, and we thank the reviewer for both pointers. The related-work section now cites probing how vision-language models perceive multimodal sarcasm [5] and exploiting relational structure among unlabelled samples for generalized category discovery [24], and states that our data-quality-control contribution is orthogonal to these modeling advances. Both are cited in the running text and the bibliography.

*Location:* Section 2; References [5] and [24].

### 4. Strengthen the transfer-limits statement

**Requested.** State prominently what practitioners should not conclude: the full 20-aspect schema lacks real validation and high-stakes decisions require human-in-the-loop review.

Done. Section 6.1 states that only 9 of 20 aspects are externally validated, that synthetic-only training recovers about half of a real-trained model (micro-F1 0.402 versus 0.767 across five seeds), that the full schema is not yet externally validated, and that high-stakes use requires human-in-the-loop review.

*Location:* Section 6.1.

### 5. Practitioner adoption roadmap

**Requested.** Provide concrete guidance: minimum fine-tuning data size, expected degradation, and monitoring.

Done. Section 6.2 gives a fine-tuning-size curve: roughly 250 to 500 local reviews capture most of the benefit, synthetic pretraining stays above from-scratch training at every budget and roughly doubles data efficiency in the low-data regime (Figure 6, five seeds), practitioners should expect the Figure 6 curve rather than internal-benchmark numbers, and deployments should be monitored against a held-out locally-adjudicated slice with re-checks on distribution shift.

*Location:* Section 6.2; Figure 6.

### 6. Moderate scores and qualitative error analysis

**Requested.** Add a qualitative error analysis of common failure modes.

Done. The moderate scores reflect the intrinsic difficulty of 20-aspect ABSA under conservative overlap. A new qualitative error analysis shows four systematic patterns: high-prevalence diffuse aspects are over-predicted while specific aspects are under-detected; missed specific aspects are substituted by generic ones; polarity compresses toward neutral; and a positive skew appears under real-review transfer. Practitioners can expect reliable detection and polarity on frequent, lexically distinctive aspects, and should treat fine-grained aspects and non-positive polarities on out-of-domain reviews as the weak regime.

*Location:* Section 6.1; Appendix A.7.1.

### 7. Broader impact

**Requested.** No broader-impact concerns were raised.

Thank you for the positive assessment. Prompted by the other reviews, we extended the ethics statement with stylistic-bias monitoring for non-native writing, a prohibition on attaching model-inferred negatives to identifiable courses or instructors, and per-aspect uncertainty with low-confidence routing to human review and an appeals process.

*Location:* Section 6.3.

---

Locations reference the revised manuscript; every requested change is in place at the cited location.

Reviewer h7LN — official response

# Response to Reviewer h7LN

*A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis (TMLR)*

We thank Reviewer h7LN for a thorough and technically precise report, and for the concrete pointers on baselines, figures, and the truncated rows, which we acted on directly. Each point is answered with its evidence and location.

### 1. Statistical realism and full-schema re-annotation

**Requested.** LLM-as-judge realism does not statistically establish indistinguishability from real feedback; the full label set lacks comprehensive human re-annotation.

Reframed, with analysis and a new human study. The realism that matters for a labeled training benchmark is functional: a model trained on the synthetic corpus recovers real aspect and sentiment signal on independent external corpora. Judged sentence by sentence, synthetic sentences are near-indistinguishable from real ones (the judge is 60% accurate, near the 50% floor, versus 93% on whole reviews). On the label side, three annotators re-annotated a stratified sample (Table 15): inter-annotator reliability is substantial (Fleiss 0.70), human confirmation rises monotonically with the audit score, and the human labels reproduce the audit's presence-faithful, sentiment-noisier split, so label validation now rests on direct human annotation of the synthetic text.

*Location:* Section 5.7 (Table 15); Section 6.1; Appendix A.2.7.

### 2. Broaden baselines beyond one provider and prompt

**Requested.** Baselines are confined to a single provider's GPT family and one prompting method.

Done. The prompting axis is covered by four zero-, fixed-, diverse-, and retrieval-few-shot variants (Table 7). The provider axis is now covered on the full 1,000-review test split: the identical zero-shot contract run across four families spanning four providers (GPT-5.4, Gemini-2.5-Flash, GLM-4.6, Llama-3.3-70B) lands in a single narrow band (micro-F1 0.239 to 0.267), all below the trained encoders.

*Location:* Appendix A.3.2; Appendix A.6.2.

### 3. Figure and table formatting

**Requested.** The formatting of Figure 1, Table 5, Figure A2, and Figure A3 could benefit from refinement.

Done, and we thank the reviewer for the specific pointers. Figure 1 is tightened with even margins and a clean sans-serif axis, Table 5 was reset for consistent alignment, and the two appendix panels flagged (Figures A2 and A3 in the reviewed version, now Figures A3 and A6 after the appendix reorganization) were regenerated with matched fonts and spacing.

*Location:* Figure 1; Table 5; Figures A3 and A6.

### 4. The 841 token-capped rows

**Requested.** 841 of 10,000 samples hit the token cap; full-corpus length-band adherence is 0.6819.

Done. Regenerating those rows at a higher cap restores full-corpus length adherence. The truncated rows are not systematically biased: aspect distribution not skewed (chi-square p=0.23, V=0.034), polarity negligibly different (V=0.020), and audit faithfulness statistically identical (0.573 versus 0.577, p=0.76). They differ only mechanically, so excluding them does not change the benchmark.

*Location:* Appendix A.1.5, A.1.6.

### 5. Bottom-quartile rows and training value

**Requested.** The bottom 25% markedly inflate sentiment error and hold little training value.

Clarified. This is exactly the signal the filter exploits: the negative-control bottom quartile collapses on sentiment error across both architectures and both transfer targets (0 of 8 seeds recovering), which is why retaining the top 50% by audit score reduces error at half the training cost. A quartile dose-response confirms the effect is monotone across the whole distribution.

*Location:* Section 5.7.

### 6. Audit-human agreement (kappa 0.56)

**Requested.** Cohen's kappa 0.56 is moderate, so the audit cannot be a fully reliable proxy for human judgement.

Clarified. The 0.42 match is a strict per-aspect lower bound (per-row mean 0.58), and the audit's value is downstream: filtering by it reduces sentiment error at half the data, and it agrees with humans at kappa 0.56 and 0.62 across independent families. It is a validated selection instrument, not a ground-truth oracle; the synthetic-corpus annotation study measures its agreement on synthetic text directly.

*Location:* Section 5.7; Section 6.2.

### 7. Scope of the external validation

**Requested.** The OMSCS reviews number only 32 and Herath 2,829, so conclusions apply to a narrow scope.

Clarified. External validation now spans four real corpora (Herath, EduRABSA, M-ABSA, OATS) across institutional and MOOC settings. We state the current scope (English STEM and graduate) and identify cross-domain and cross-language extension as future work.

*Location:* Appendix A.5.4; Section 6.1.

---

Locations reference the revised manuscript; every requested change is in place at the cited location.

New empirical results

Presentation and structure

Point-by-point replies to each reviewer are in the author responses; the submission rationale is in the cover letter. This summary is a reading aid and is not part of the manuscript.