A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis (TMLR)
We thank the reviewer for the careful and constructive report. We have addressed all five requested changes; each is answered below with the specific evidence added and its location in the revised manuscript. New measurements were run for the first, second, and fifth points; the third and fourth are additions to the text.
Requested: the generator and auditors share a provider family; more explicitly discuss whether the audit detects same-family latent patterns rather than true textual faithfulness.
Done, with a new experiment. We repeated the human-validation of the audit with a fully independent model family, Google gemini-2.5-flash, scoring the identical 1,200 perturbation-controlled variants (faithful / polarity-flipped / aspect-injected) over the two human-annotated corpora. Gemini agrees with the human labels at Cohen's kappa 0.62 (precision 0.89), matching and slightly exceeding the same-family GPT auditor (kappa 0.56). A model with no shared provenance with the gpt-5-nano generator therefore reproduces the human faithfulness judgments at least as well, so the signal is human-grounded rather than an artifact of the generator's own family. We also added an explicit statement of this concern and its resolution as a limitation.
Requested: analyze whether the output-token-capped rows are distributed differently across aspects/sentiment, and whether excluding them changes benchmark results.
Done, with a new analysis and re-run. We recovered the exact 841 rows by joining the generation batch output's per-row API status to the corpus. They are modestly different from complete rows: shorter (mean 100 vs 119 words) and carrying fewer aspects (a 0.07 vs 0.33 share of three-aspect reviews), with a comparable sentiment mix. Retraining the detection benchmark with these rows excluded (three seeds) leaves the held-out result essentially unchanged, micro-F1 0.264 vs 0.275 and macro balanced accuracy 0.613 vs 0.619, a small difference consistent with the roughly 8% smaller training set rather than a systematic bias. The benchmark is robust to their inclusion.
Section 3.6; Appendix A.18 and Table A16.Requested: acknowledge recent LLM-prompting and related methods, including the two named papers.
Done. We added a sentence to Related Work situating the work against recent modeling advances, citing the reviewer's two references, multimodal sarcasm perception in vision-language models [39] and set-matching for generalized category discovery [40], and noting that our contribution is orthogonal to these, targeting the data-quality control of LLM-supervised labels rather than a new model or task.
Section 2 (Related Work); references [39], [40].Requested: state more prominently what practitioners should not conclude, that the full 20-aspect schema lacks real validation and high-stakes decisions require human-in-the-loop review.
Done. We strengthened the first limitation to say explicitly that the transfer results do not validate the full schema: only 9 of the 20 aspects are externally checked, synthetic-only training recovers about 60% of a real-trained model, and any high-stakes or personnel decision requires human-in-the-loop review rather than reliance on the eleven aspects that have no real-data validation.
Section 6.1, limitation 1.Requested: concrete guidance on minimum fine-tuning data size, expected performance degradation, and monitoring.
Done, with a new experiment. We ran a fine-tuning-size sweep: a synthetic-pretrained detector fine-tuned on subsampled real reviews reaches micro-F1 0.58 at 100 reviews, 0.70 at 500, 0.74 at 1,000, and 0.77 at the full real set (matching real-only training, 0.767). From this we added concrete guidance: roughly 250 to 500 locally labeled reviews capture most of the benefit and are a realistic departmental budget; the synthetic pretrain reaches real-only quality with about half the real data; a deployment should expect the curve's micro-F1 for its own N and be monitored against a held-out locally adjudicated slice, re-checked when the feedback distribution shifts.
Section 6.2 (new roadmap paragraph and Figure 6).We believe these revisions address the report in full, and we are grateful for feedback that measurably strengthened the paper. We are happy to provide any further detail.
Sincerely,
The authors