Beyond Prompting: Framing, Judging, and Steering as Assessable Competencies for Reasoning With Generative AI in Higher Education

Alexander Apartsin¹ and Yehudit Aperstein²

¹ Holon Institute of Technology · ² Afeka College of Engineering

Corresponding author: Dr. Yehudit Aperstein, Afeka College of Engineering (apersteiny@afeka.ac.il).

Keywords: generative AI; AI literacy; competency assessment; metacognition; self-regulated learning; higher education

1. Introduction

Most tools bringing generative AI into higher education optimize for the wrong variable. They shorten the path from question to answer, when the answer was never the point of a university education. The cognitive work of specifying a problem, evaluating a candidate solution, and improving that solution is where learning happens. When that work is delegated wholesale to a machine, the learner walks away with a correct artifact and an unchanged mind. The educational opportunity of generative AI is therefore not to deliver answers faster, but to make the reasoning around them visible, practiced, and assessable.

Recent evidence makes the stakes concrete. Frequent AI use correlates with lower critical-thinking performance, an effect mediated by cognitive offloading and most pronounced in younger users (Gerlich, 2025). Controlled studies of AI-assisted writing report "metacognitive laziness," in which learners bypass the self-regulatory processes of diagnosing, evaluating, and revising (Fan et al., 2025). Neurophysiological work describes an accumulation of "cognitive debt" when an assistant carries the reasoning load (Kosmyna et al., 2025).

At the same time, field studies of professional AI use show that the benefit of AI is sharply conditional on the user's skill in directing it: assistance helps on tasks inside the model's competence frontier, the boundary of what it can do reliably, and harms performance on tasks outside it (Dell'Acqua et al., 2023). The most effective users adopt an iterative, critical "push-back-and-validate" mode rather than wholesale delegation (Randazzo et al., 2025). Meta-analytic evidence reinforces the point: human-AI teams are often worse than whichever of the human or the AI performs better alone (Vaccaro et al., 2024). The decisive variable is therefore the human's skill in directing the system, not mere access to it. Direct behavioral evidence shows where that skill breaks down: across 16,851 real student-ChatGPT interactions from a university course (McNichols et al., 2025), students' moves are dominated by framing-type queries, while explicit verification of the AI's output (about 4% of interactions) and corrective editing (about 2.5%) are rare. This is precisely the under-judging and under-steering that a competency model must target.

These findings share a structure. The difference between productive and counterproductive AI use is not access to AI; it is a competency that some learners exercise and others do not. This echoes a long-standing distinction: the effects obtained with a technology during use differ from the cognitive residue left of it, and that residue depends on the learner's mindful engagement in the partnership (Salomon et al., 1991). If that competency can be named and decomposed, it can be taught and assessed. Yet current AI-literacy assessment can often tell us that a learner's AI use is unproductive but not why: whether they specified the task badly, failed to detect the flaws in the output, or knew the flaws but could not correct them. These are different failures with different remedies, and a single "prompting" score conflates them.

This trajectory is not transient, and that is why assessment must change. As models absorb more of the execution, the human contribution concentrates in the operations a system cannot perform on its own behalf: specifying which problem is worth solving, judging whether a fluent output is right, and redirecting the model when it is not. Field evidence that AI's value is conditional on the user's skill (Dell'Acqua et al., 2023; Vaccaro et al., 2024) implies that this conditioning tightens as systems improve. The durable, assessable skill therefore shifts from executing a task to framing, judging, and steering it (Acar, 2023; Denny et al., 2024; see Figure 1). An assessment that still measures unaided execution captures a shrinking part of what competent work will require.

Figure 1 (schematic). As models absorb more of the execution, the part of a task a competent person must still own shifts from doing the work toward framing, judging, and steering it; the axes are conceptual, not quantitative. CoRe-3 is a decomposition and assessment of that shifting contribution.

We propose CoRe-3 (Co-Reasoning), a competency model that decomposes productive work with generative AI into three distinct skills, each independently assessable and collectively abbreviated FJS:

• Framing. Before invoking the AI, transform an ill-defined problem into a well-specified task: surface unstated assumptions, fix constraints and scope, and define what an adequate solution must satisfy. Framing is a problem-structuring skill exercised prior to generation.
• Judging. Given AI output, critically evaluate it as a knowledge claim: detect factual and logical errors, missing edge cases, unstated assumptions, and risks, and calibrate how far to rely on it. Judging is an evaluative, epistemic skill.
• Steering. Across one or more cycles, issue targeted corrective feedback that moves the AI's output measurably closer to an adequate solution. Steering is a control skill exercised after generation, in a loop with Judging.

Contributions. This paper makes the following contributions, spanning a theoretical framework and its novelty positioning (1-4), a feasibility demonstration (5), and a released artifact and protocol (6):

1. A competency model that decomposes productive work with generative AI into three temporally and cognitively distinct, independently-assessable skills (Framing, Judging, and Steering), whose defining novelty is the separation of the pre-generation skill (Framing) from the post-generation corrective skill (Steering) that prior frameworks fuse.
2. A theoretical grounding that maps each skill to established theory (metacognitive monitoring and control, self-regulated learning, epistemic vigilance and critical thinking, productive struggle) under a single unifying monitor-control architecture, so each skill carries an explicit theoretical warrant.
3. Five testable propositions about how the skills relate (framing gates the loop; judging gates steering; the skills dissociate; judging transfers less than framing; the model inverts cognitive offloading), proposing new relationships among the constructs (Gilson & Goldberg, 2015; Jaakkola, 2020).
4. A precise novelty positioning against the nearest prior frameworks (AI-fluency, metacognitive- demands, prompt-literacy, and AI-literacy models), delineating what the model contributes beyond each.
5. An instrument and feasibility demonstration showing that the three skills can be scored automatically, with a multitrait-multimethod validity analysis (three grader backends across two vendors; shared- versus independent-competence populations) giving convergent and discriminant evidence that the instrument scores the three skills separately over simulated learners of controlled competence.
6. A released, runnable validation package: the assessment instrument, the experiment data, a scoring codebook, per-rater task files with hidden ground truth, and seed-fixed agreement-scoring scripts, so that the human-agreement study can be run as soon as raters are recruited.

Following the argument-based validity tradition (Kane, 2013; Messick, 1995), instrument development proceeds in stages: a measure must first separate the constructs it claims to measure before human-agreement and outcome evidence are warranted. This paper delivers that prerequisite stage (construct separability and automated measurability) and releases a runnable protocol for the human-agreement stage that follows (Section 9.3).

2. The problem with current AI-literacy assessment

Existing instruments for assessing how university students work with AI are not wrong so much as under-resolved. Knowledge-oriented frameworks such as Long and Magerko (2020) and the UNESCO (2024) student competency framework define AI literacy chiefly as understanding AI systems: what AI is, what it can and cannot do, how it works, and how it should be governed. These are necessary, but they say little about the performance of working with a model on a task. Prompt-literacy and generative-AI-literacy models (Lo, 2023; Annapureddy et al., 2024) move closer to performance, but they bundle task specification and iterative refinement into a single "prompting" competency and treat evaluation of the output as a downstream check rather than a co-equal skill.

The cost of this coarse resolution is diagnostic. When a learner's AI use produces a poor result, a single prompting score cannot tell an instructor which cognitive operation failed: did the learner specify the task badly, so that the model solved the wrong problem; did they fail to detect the flaws in an otherwise plausible output; or did they see the flaws but issue corrections too vague to fix them? These are three different failures with three different remedies (teaching problem specification, teaching critical evaluation, and teaching corrective communication), and an instrument that cannot separate them cannot guide instruction. Integrative reviews of AI literacy after generative AI confirm that the field still lacks a scheme isolating distinct, independently-assessable reasoning competencies (Gu & Ericson, 2025), even as syntheses document generative AI's mixed effects on critical and creative thinking (Li et al., 2026). A diagnostic competency model must therefore decompose the human-AI loop into the distinct operations that can each break, and must show that those operations are in fact separable in learners. That is the gap CoRe-3 addresses.

3. The CoRe-3 framework

3.1 A monitor-control architecture with an upstream task-definition

The three FJS skills are not an arbitrary list; they instantiate a well-understood cognitive architecture. Nelson and Narens (1990) describe metacognition as a two-level system: an object level (cognition itself) and a meta level (a dynamic model of the object level), linked by monitoring (information flowing from object to meta) and control (commands flowing from meta to object). In CoRe-3, the learner's meta level supervises an object level that is external: the AI's generative process rather than the learner's own cognition. This is a deliberate extension of the Nelson-Narens architecture from intrapersonal monitoring to what we term exo-directed monitoring and control. It is the framework's central theoretical move: the same metacognitive machinery is turned outward onto a fallible cognitive artifact. This extension is non-trivial. Exo-directed monitoring adds an epistemic-vigilance burden that self-monitoring does not have: the learner must judge the reliability of a source whose competence differs from, and is hidden from, their own. This is exactly why Judging is bounded by domain knowledge (Proposition P4). The mapping is then direct:

• Judging is monitoring. The learner compares AI output against an internal model of an adequate solution and registers discrepancies.
• Steering is control. The learner acts on the monitoring signal, issuing commands that change the object-level process.
• Framing is task definition. Before monitoring can occur, the learner must establish the standards against which output is judged. In the COPES model of self-regulated learning (conditions, operations, products, evaluations, and standards; Winne & Hadwin, 1998), task definition is the explicit first phase, and products are evaluated against internally held standards; mismatch triggers reprocessing. Framing is that task-definition phase applied to a human-AI loop: it sets the referent that makes Judging and Steering well-posed.

The Judge→Steer cycle is therefore a monitor→control loop seeded by a task definition. This is the structural spine of the framework and the reason the three skills cohere rather than coexist by stipulation (Figure 2).

Figure 2. The CoRe-3 loop. An upstream task definition (Framing, a self-regulated-learning forethought activity) sets the standards against which a metacognitive monitor-control cycle (Judging then Steering) supervises a fallible AI at the object level. Judging is the monitor's read-out; Steering is the controller's write.

3.2 The temporal-separation claim

Existing frameworks treat "use the AI well" as one skill or, at most, pair "prompt" with "evaluate." CoRe-3's distinctive move is to separate two operations that prior models fuse: the pre-generation skill of structuring the task (Framing) and the post-generation skill of correcting the output (Steering). These are different cognitive acts at different points in time, with different error modes and different instructional remedies. A learner can frame impeccably and steer poorly (detect a flaw but issue a vague correction), or steer fluently atop a malformed task (drive the AI energetically toward the wrong target). Collapsing both into "prompting" hides exactly the distinctions an educator needs.

3.3 Distinguishing Judging from Steering

Because Judging and Steering both occur after generation and in a tight loop, their boundary needs stating precisely. Judging is assessment; Steering is action. Judging produces a representation of what is wrong with the current output: an internal or articulated list of detected flaws, gaps, and risks, together with a calibrated sense of how much to trust the output. Judging is evaluative, and its output is a diagnosis. Steering consumes that diagnosis and produces a corrective instruction aimed at changing the next output. Steering is generative and directive, and its quality depends on three things: prioritization (addressing the most critical flaw first), specificity (an actionable command rather than "improve this"), and effectiveness (whether the output actually converges). In the monitor-control terms of Section 3.1, Judging is the monitor's read-out and Steering is the controller's write. The two dissociate because the competencies differ. A learner may diagnose accurately yet communicate the fix poorly (good Judging, weak Steering). Conversely, a learner may issue fluent, confident commands that target the wrong thing because the diagnosis was wrong: weak Judging propagating into misdirected Steering, the failure mode Proposition P2 predicts. This last case also resolves an apparent tension. P2 (Judging bounds Steering) implies that the two grades will be positively correlated in the aggregate, while P3 claims that they dissociate. Both hold. P2 is a ceiling relation: Steering cannot exceed the quality its Judging permits, which induces correlation without identity. P3 is the claim that the cross-skill (off-diagonal) effect is well below the reliability ceiling, so the skills are not interchangeable. We therefore test dissociation pairwise, asking whether each skill's grade responds to its own manipulated competence and responds much less when the other competences change, rather than relying on a single global factor model.

4. Theoretical grounding

Each skill is anchored in established theory, and the anchors are mutually consistent because they share the metacognitive backbone of Section 3.1.

Framing is the task-definition phase of self-regulated learning. In the COPES model (Winne & Hadwin, 1998), self-regulated work begins by constructing a definition of the task and the standards a product must satisfy; Zimmerman's (2000) forethought phase similarly precedes performance with goal setting and strategic planning. Framing applies this phase to a human-AI loop: the learner converts an ill-defined situation into a specified task with explicit constraints and success criteria. In Bloom's revised taxonomy (Anderson & Krathwohl, 2001), specifying an original task is a Create-level activity, and the competence to know what makes a task tractable for a given tool is metacognitive task knowledge (Flavell, 1979).

Judging is metacognitive monitoring (Nelson & Narens, 1990) directed at an external generative source. Its content is the evaluative core of critical thinking: the Delphi-consensus skills of analysis and evaluation (Facione, 1990) and the Paul and Elder (2006) intellectual standards, which together supply a ready vocabulary for assessing reasoning. Because the object being judged is a communicated knowledge claim from a fluent but fallible source, the most precise anchor is epistemic vigilance (Sperber et al., 2010), which pairs source monitoring (is this source trustworthy?) with content evaluation (is this internally and externally coherent?). Barzilai and Chinn's (2018) account of apt epistemic performance adds the criteria for good knowledge that a learner must hold in order to judge well. The human-automation literature supplies the calibration target: reliance matched to actual reliability (Lee & See, 2004). Failure to calibrate produces the over-reliance documented in AI-assisted decision-making (Bansal et al., 2021; Buçinca et al., 2021), a specific case of broader automation complacency (Parasuraman & Manzey, 2010). Judging is therefore a trainable skill rather than an automatic byproduct of competence. Two recent findings underscore the point: metacognitive monitoring can decouple from performance in human-AI reasoning (Fernandes et al., 2024), and whether learners can evaluate AI output quality as experts do is itself under study (Nazaretsky et al., 2025).

Steering is metacognitive control (Nelson & Narens, 1990): acting on the monitoring signal to change the object-level process. Pedagogically it inverts cognitive-apprenticeship coaching and scaffolding (Collins, Brown & Newman, 1989): the learner, not the master, supplies the corrective guidance. Its quality depends on the learner monitoring the work against held standards (Sadler, 1989), and it is well described as feed-forward, the "where to next" component of effective feedback (Hattie & Timperley, 2007).

The loop and its positioning. The Judge→Steer cycle is designed to push learners into the Interactive mode of the ICAP framework (interactive, constructive, active, passive; Chi & Wylie, 2014), in which knowledge is co-constructed through dialogue rather than passively received, the mode ICAP associates with the greatest learning. The framework casts the AI as a mediating cultural tool that extends the learner's zone of proximal development (Vygotsky, 1978): the learner accomplishes with the model what they could not alone, while internalizing the Framing, Judging, and Steering moves for eventual independent use.

The pedagogical stance. CoRe-3's rejection of speed-to-answer rests on the literature of productive struggle and desirable difficulties (Bjork & Bjork, 2011; Kapur, 2008): conditions that slow performance but deepen learning. The friction of specifying, evaluating, and correcting is not an obstacle to be engineered away but the very locus of learning. This is why the instructional design deliberately presents imperfect output for the learner to improve, rather than a polished answer to accept.

5. Propositions

• P1 (Framing gates the loop). Framing causally precedes and bounds Judging and Steering; framing failures are not recoverable by downstream steering, because a malformed task gives the monitor no stable referent.
• P2 (Judging gates Steering). Steering quality is upper-bounded by Judging quality: undetected flaws cannot be corrected, and high steering effort over poor judging yields confident misdirection (the over-reliance failure mode).
• P3 (Dissociation). The three skills are separable competencies; proficiency in one does not imply proficiency in another. This is the central empirically-testable claim; if it holds in learners, the three skills are genuinely distinct rather than a relabeling of "prompting."
• P4 (Asymmetric transfer). Judging is bounded by domain knowledge (the calibration trap) and therefore transfers across domains less readily than the more structural skill of Framing.
• P5 (Inverse of offloading). Each skill re-inserts a self-regulatory operation that cognitive offloading short-circuits; the framework offloads execution while retaining cognition.

These propositions differ in how far the present work tests them. The feasibility demonstration (Section 8) directly supports P3 (the three grades dissociate over simulated learners of controlled competence) and bears partly on P2 (the steering own-effect is bounded in a way consistent with judging gating steering). P1, P4, and P5 are stated here as falsifiable hypotheses for the validation and classroom agenda (Section 9.3), not as claims the demo establishes. Each names a concrete test: P1 fails if a strong steering intervention recovers grades after a deliberately malformed framing; P4 fails if Judging transfers across domains as readily as Framing; P5 fails if exercising the loop does not reduce the offloading signatures (for example, reduced post-task recall) documented in the cognitive-debt literature.

6. Relation to prior frameworks

The constructs that compose CoRe-3 are not individually new; what is new is their separation into three parallel, independently-assessable competencies anchored in a monitor-control architecture. We make the boundary explicit against the nearest priors.

Fluency, metacognitive, and prompt-literacy frameworks. The 4D model of AI fluency (Dakan & Feller, 2025) maps Discernment onto Judging but bundles the initial specification and the iterative back-and-forth into one "Description"; these are exactly the pre-generation and post-generation operations we separate. Tankelevitch et al. (2024) analyze generative-AI use as metacognitive demands (prompting, output evaluation, workflow iteration); we share that foundation but convert a demands analysis into an assessable model with rubrics, propositions, and dissociation evidence. Prompt-literacy models, from CLEAR (Lo, 2023) to the formulate-interpret-refine decomposition of Tour and Zadorozhnyy (2025), likewise fuse task specification and refinement under "prompting" and do not grade the sub-practices as dissociable competencies. This fusion of Framing and Steering is precisely what CoRe-3 rejects, and our feasibility demonstration shows that the two grades can diverge.

Literacy, metric, and mode frameworks. Knowledge-oriented AI-literacy sets (Long & Magerko, 2020; UNESCO, 2024) define literacy as understanding AI systems and contain no Framing or Steering construct, only a diffuse "critical evaluation". CoRe-3 supplies the task-execution competencies they leave implicit. Metric schemes for human-AI cognition, such as a cognitive-amplification-versus-delegation framework with dependency and drift metrics (Di Santi, 2026), measure the sustainability of reliance rather than teachable skills. Field accounts of how skilled users work describe the behavior that CoRe-3 decomposes into assessable skills: the continuous push-back-and-validate "Cyborg" mode (Randazzo et al., 2025) and the dependence of AI's value on user skill at the competence frontier (Dell'Acqua et al., 2023).

Table 1 summarizes this mapping, placing each nearest prior framework's constructs against Framing, Judging, and Steering. The recurring pattern is that prior frameworks either (i) omit a construct, or (ii) fuse Framing and Steering into one "prompting/iteration" skill.

Table 1. Where prior frameworks place the three FJS skills (Framing, Judging, Steering).

No framework in the table cleanly separates the pre-generation and post-generation skills and treats all three as independently scored competencies. That conjunction is the contribution.

Validated instruments and evaluative judgement. Psychometric GenAI-competence scales such as GenAIComp (Lee et al., 2025) and tests such as GLAT (Jin et al., 2024) establish that GenAI competence is measurable, but their factor structures are literacy-oriented and isolate no Framing, Judging, or Steering construct. The work closest to ours pairs AI-collaboration literacy with metacognition (Sidra & Mason, 2025), yet it posits correlated factors rather than demonstrating dissociation. Because rubric-based LLM grading can agree highly with humans yet diverge on subjective criteria (Yavuz et al., 2025), our multitrait-multimethod design across three grader backends (Section 8) treats grader-method variance as a measured term. Our Judging skill operationalizes the evaluative judgement that has been argued to be the core human capability for the AI era (Bearman et al., 2024; Walton et al., 2025). Critical evaluation is the highest-load activity in AI-assisted writing (Yao & Fan, 2025), and users default to uncritical acceptance without intervention (Wingerter et al., 2025); this is why we treat it as one of three separable skills rather than as one overarching disposition. Among named frameworks, the eight-dimension AI Quotient (Ganuthula & Balaraman, 2025) leaves prompt-engineering fused, whereas CoRe-3 splits it into pre-generation Framing and post-generation Steering. Assessment-mode partitions of coursework (Elshall & Badir, 2025) are orthogonal to our model and complementary to it.

Problem formulation as the AI-era skill. As models absorb execution, a prominent strand argues that the durable human skill shifts to problem formulation (Acar, 2023), which is precisely our Framing construct. Classroom work assesses it through prompt problems (Denny et al., 2024), question formulation (Kim et al., 2025), and problem decomposition (Srinath et al., 2025), and the older problem-finding literature established it as separable from problem solving (Runco & Chand, 1995). We build on this strand by placing Framing in a measured loop with Judging and Steering.

One-line novelty. CoRe-3 is, to our knowledge, the first theoretically-grounded decomposition of productive generative-AI use into three independently-assessable competencies that separates pre-generation Framing from post-generation Steering, with feasibility evidence that the three skills dissociate. The contribution is the separation itself, which is both theoretically motivated (monitor-control plus an upstream task definition) and empirically consequential (the skills can be measured apart). Recent syntheses sharpen where this novelty sits. A systematic review of prompt engineering in higher education documents a proliferation of prompt skills and curricula but no consensus decomposition into separately assessable competencies (Lee & Palmer, 2025), and GenAI-era assessment frameworks organize how to assess rather than which reasoning skills to separate (Ahangama, 2026). Against this backdrop, the contribution is not the three skill labels individually (each has antecedents), nor the general claim that AI use is multidimensional. It is the specific conjunction that prior work leaves open: the temporal split of pre-generation Framing from post-generation Steering, operationalized as three constructs that are scored independently and shown to separate.

7. The CoReasoningLab system

The framework is instantiated in CoReasoningLab, a runnable open-source web platform that maps the abstract skills onto a concrete learner experience. It is a Node/Express application with role-based access (student, instructor, administrator), a challenge database, practice and assessment modes, multiple-choice and open-ended response formats, and a five-language content library (English, Hebrew, German, Spanish, French). Its full source is released (the web application, the database schema, the content library, and the scoring engine of 16 prompts described in Section 7.1), alongside a pedagogical-foundations document; Additional file 1 walks through the running system. All challenges evaluated here are English-language (Section 9.2), and the quantitative results come from the scoring engine over controlled inputs, not production logs.

Because challenge generation, rubric generation, and scoring are themselves prompt-defined, the engine extends to a new subject, language, or grader backend by substitution rather than redesign, and the released artifact lets others add their own. The platform is therefore domain-general: not a computing-education tool but a generic instrument for any discipline in which a learner must specify an ill-defined task, judge a fallible solution, and steer it toward a better one. Its released content already spans 12 disciplines across STEM, the social sciences, the humanities, law, and professional fields (Additional file 1). Authoring a challenge requires only a subject; the system generates the rest.

Authoring flow (instructor). An instructor defines a challenge by choosing a course and subject path; the system then generates the ill-defined problem, the three per-skill rubrics, the gold-standard framing, and the seeded-flaw solution that the learner will critique (Section 7.1). Challenges are organized into courses and can be assigned to cohorts.

Learner flow (student). A student enters a challenge run with two phases, walked through step by step in Additional file 1. In Phase 1 (Framing), the student refines the raw ill-defined problem, either by writing refinement sections (assumptions, constraints, clarifications, success metrics) or by selecting them in multiple-choice mode, and then receives rubric-driven Framing feedback and a grade. The system then produces a plausible but deliberately flawed solution. In Phase 2 (Judging and Steering) the student repeatedly judges the current output (flagging its issues) and steers the AI (issuing correction commands), up to a configured maximum, and marks the task complete when satisfied. Framing, Judging, and Steering are scored separately, each with its own rubric-driven feedback, and surfaced in a per-challenge report and longitudinal analytics that track the three skills independently. This separation in the interface (distinct phases, feedback channels, and grade columns) is the framework's central claim made operational: a learner sees, and is scored on, three different things rather than one undifferentiated "AI use." The remainder of this section describes the instrument that produces those scores.

7.1 Operationalization: the assessment instrument

To show that the three constructs are not only conceptually distinct but practically measurable, we describe a working instrument that scores each skill from a learner's transcript. The instrument is a pipeline of large-language-model prompts. We use it here as an existence proof that automated, rubric-driven scoring of Framing, Judging, and Steering is feasible, not as a validated assessment.

Challenge construction. Each challenge begins with a deliberately ill-defined problem generated to contain two or three unstated gaps, recorded internally but never shown. For each challenge, three rubrics are generated for the subject area, one each for Framing, Judging, and Steering; each rubric has three to five measurable criteria and explicit indicators of excellent and poor performance. A gold-standard "best framing" is generated as an internal reference. The design instantiates an inverted cognitive apprenticeship: rather than observing an expert, the learner is given a fallible artifact to repair. This connects the instrument to the instructional literature on learning from errors and erroneous examples, in which studying and correcting flawed solutions improves error detection and conceptual understanding relative to studying only correct ones (Große & Renkl, 2007; Durkin & Rittle-Johnson, 2012). CoRe-3 generalizes that paradigm from static worked examples to an interactive, learner-driven repair loop over AI output.

The deliberately-imperfect output. After the learner frames the task, the model produces a plausible, professional-looking solution that is required to embed two to four non-trivial issues (wrong-but-reasonable assumptions, missing edge cases, or subtle logical errors). Each issue is recorded internally with a severity label, and none is flagged to the learner. Across steering cycles, updates address the learner's commands but may introduce new minor issues, so that difficulty adapts to the quality of steering rather than collapsing to a perfect answer.

Scoring. Each skill is scored in two stages. A skill-specific evaluator assesses the learner's response against the (internal) rubric and produces per-criterion ratings on a three-point scale; a generic grading stage then aggregates those ratings into a final grade, weighting critical criteria more heavily rather than averaging. Crucially, the three evaluators differ in what they compare against: Framing is evaluated against the gold framing and the rubric; Judging is evaluated against the seeded ground-truth issues, yielding a recall/precision signal (issues correctly identified, missed, and falsely flagged); and Steering is evaluated against the trajectory of the output across cycles, rewarding corrections that demonstrably move the solution toward correctness. This is why the skills are measured apart: each evaluator interrogates a different referent.

The instrument used here is the scoring engine of the CoReasoningLab platform, a library of 16 prompts spanning challenge construction, AI generation, and evaluation (traced in Additional file 1). It carries the deployed application's evaluation logic verbatim: the same rubrics, criteria, and system prompts. The research harness (simulated-learner generation, the crossed factorial, and the analysis) only orchestrates that fixed logic over controlled inputs, so the measurements are reproducible and characterize the instrument itself. Because the grader is a language model, the results below describe the instrument's internal behavior (whether it separates controlled competence levels and dissociates the skills). Agreement with human experts is a separate validity question, which the prepared study in Section 9.3 addresses.

8. Feasibility demonstration

We exercise the instrument over controlled inputs to test three feasibility claims: that it discriminates competence, that the three skills dissociate (Proposition P3), and that the grader is self-consistent enough to report. The "learners" are simulated personas of controlled per-skill competence, generated by one model (gpt-4o-mini) and graded by a different model (gpt-4o), so that no model grades its own output. These claims concern the instrument; the agreement study with human expert graders is prepared and is the next stage (Section 9.3).

Design. We use a crossed factorial design. Each of the three skills is independently set to a strong or weak competence level, giving $2^3 = 8$ profiles; these are crossed with ten challenges across ten distinct subjects (spanning STEM, the social sciences, and the humanities), for 80 simulated learners. Framing and Steering responses are generated by the competence-conditioned learner model. Judging is operationalized differently: it is a competence-conditioned selection over the challenge's ground-truth seeded issues (a strong judge flags all real issues and no false ones; a weak judge flags few and adds false ones). Those seeded issues are themselves machine-generated and not yet human-verified. Because the manipulation is per skill, the design separates two questions: whether each grade tracks its own skill's competence (discrimination), and whether it is insensitive to the other skills' competence (dissociation).

Discrimination. Grades move monotonically with competence: the all-weak profile averages C on every skill, the all-strong profile averages between B and A, and each skill's grade rises when that skill is set to strong. The judging signal is mechanistically transparent: a strong judge flags all of the seeded issues with no false alarms and is graded A; a weak judge flags none and raises false issues and is graded C.

Dissociation (the central result). Table 2 reports, for each graded skill, the change in its mean grade (on a 3-point scale, A=3..C=1) when each skill in turn is moved from weak to strong. The result that carries the claim comes from the two blind-graded skills, Framing and Steering, whose free-text responses the grader scores without knowing the intended competence. Each shows a clear positive own-effect (+0.62 and +0.43), and their cross-effects are near zero except for the expected Judging→Steering ceiling effect (Table 2). A simulated learner's two free-text grades therefore move largely independently. Judging's own-effect (+2.00) is large but fixed by construction: its competence is operationalized by a controlled selection over the challenge's seeded issues, and those seeded issues are themselves machine-generated and not yet human-verified (Section 9.2). We therefore read Judging's clean diagonal as a controlled check that the grader correctly rewards recall and precision, not as independent evidence that the skills separate. The off-diagonal (cross-skill) effects are small overall (mean +0.01 across the matrix), with one systematic exception: manipulating Judging raises the Steering grade by +0.27 (challenge-clustered 95% CI [+0.07, +0.50], excluding zero), against Steering's own-effect of +0.43. This cross-effect is substantial, more than half the size of the own-effect. We read it as the ceiling relation that Proposition P2 predicts (stronger judging enables stronger steering) rather than as a leak between constructs, a reading that the human steering data of Section 9.3 will test directly. Every other cross-effect is statistically indistinguishable from zero, so each grade responds mainly, though for Steering not exclusively, to its own skill (Figure 3).

Table 2. Effect on each skill's grade of manipulating each skill's competence (grade Δ, strong − weak; N=80). The unit of analysis is the challenge; per-cell 95% confidence intervals from a challenge-level clustered bootstrap are summarized in the text and released in full with the analysis output.

Figure 3. Effect of manipulating each skill's competence (columns) on each skill's grade (rows). Each cell is the grade change (strong minus weak) on the 3-point scale (A=3, B=2, C=1); warmer cells are larger positive effects. The diagonal (own-skill effect) dominates; off-diagonal (cross-skill) effects are near zero, apart from the P2-predicted Judging→Steering effect.

Figure 4. Mean per-skill grade for five competence profiles. Judging reaches A only when judging is strong, regardless of framing or steering; each skill responds to its own competence.

Because the ten challenges are reused across the eight competence profiles, the eighty learners are not independent: the eight profile-rows that share a challenge form a cluster. We therefore quantify uncertainty with a challenge-level clustered bootstrap, in which the challenge is the resampling unit (10 clusters of 8 profiles each, drawn with replacement, 5,000 iterations), and compute a 95% confidence interval for every cell of the effect matrix rather than for the diagonal alone; the full matrix of intervals is released with the analysis output. The two blind-graded diagonal effects exclude zero: Framing +0.62 (CI [+0.40, +0.82]) and Steering +0.43 (CI [+0.27, +0.58]). Judging is deterministic by construction (+2.00, zero-width interval). Among the six cross-effects, only the P2-predicted Judging→Steering effect excludes zero (+0.27, CI [+0.07, +0.50]); the remaining cross-effects each span zero (Framing←Judging [−0.12, +0.08], Framing←Steering [−0.22, +0.05], Steering←Framing [−0.37, +0.12]; the two Judging cross-cells are zero by construction). Clustering widens the diagonal intervals relative to a learner-level resample, and the separation survives. The same separation appears in the inter-skill grade correlations across the 80 learners: Framing-Judging $\rho = -0.03$ ($p = 0.82$) and Framing-Steering $\rho = -0.12$ ($p = 0.29$) are both non-significant, while Judging-Steering $\rho = +0.25$ ($p = 0.02$) is positive and significant. The Framing-Judging pair is the decisive demonstration: these two skills are scored by entirely separate mechanisms (free-text framing evaluation versus issue-selection judging), yet their grades are uncorrelated. A single general-ability account cannot produce this pattern. The Judging-Steering correlation behaves exactly as Proposition P2 predicts: it is the expected ceiling relation in which judging gates steering, and so it is consistent with separability rather than evidence against it. With only three indicators a formal factor model is under-identified, so we rest the separability evidence on the manipulation-based effect matrix and the inter-skill correlations rather than on a confirmatory dimensionality test.

Convergent and discriminant validity (a multitrait-multimethod view). The result so far is a discriminant-validity finding: three traits separate. Construct validity also requires the convergent property: the same skill measured by different methods should agree, and the instrument should report correlation when the underlying competence really is shared (Campbell & Fiske, 1959). Two analyses supply this evidence. First, we treat the three grader backends (gpt-4o, gpt-4o-mini, and the independent-vendor llama-3.3-70b) as independent methods. For the two skills with the widest grade range, the same skill agrees across graders (Table 3, Panel A) while correlations between different skills stay low, apart from the P2-predicted Judging-Steering relation. This is the convergent-high, discriminant-low pattern a valid measure should show. Steering's narrower grade range under the P2 ceiling gives a lower cross-method correlation; this reflects a restriction of variance, not a separate construct. Across the full matrix the convergent correlations exceed both heterotrait terms and no grader shows halo, so all three Campbell-Fiske criteria hold. Second, the instrument is faithful to the population's dependence structure (Table 3, Panel B): on learners whose competences are set independently the inter-skill grade correlations are near zero, apart from the P2-predicted Judging-Steering relation, whereas on the 20 learners whose competences are all strong or all weak the same instrument returns positive correlations. The instrument therefore does not manufacture dissociation: it recovers independence when the competences are independent and correlation when they are shared. This licenses reading the crossed-design separation as a property of the manipulated competences rather than an artifact of the scoring; whether the same separation holds in human learners is the question the studies of Section 9.3 address.

Table 3. Multitrait-multimethod validity. Panel A: convergent agreement of the same skill across the three grader backends (mean pairwise per-learner rank correlation on the 40 shared transcripts). Panel B: inter-skill grade correlation by population, showing the instrument recovers the population's dependence structure (discriminant on the full crossed design, convergent on the n=20 all-strong and all-weak learners whose three competences share one level).

The own-skill effects are directionally consistent across subjects: Framing and Steering each show a positive own-competence effect in nine of the 10 subject areas; the two exceptions reflect the small per-subject sample rather than a sign reversal (Judging is fixed by construction throughout). The designed-contrast personas make the separation concrete (Figure 4): a weak-framer / strong-judge learner scores Framing C but Judging A; a strong-framer / weak-judge learner inverts this; and a weak / weak / strong-steerer elevates only Steering. A single underlying "AI-use ability" cannot produce these crossed profiles. Converging evidence comes from outside our synthetic setting: in an intervention study, students' behavioral regulation of LLM use predicts effective use whereas self-rated AI expertise does not (Clerc et al., 2026), so the skill of working with AI is distinct from a general, self-assessed competence.

Robustness and ablations. Three checks, reported in full in Additional file 1, support the result. First, the grader is 92% self-consistent when it re-grades the same transcript; this checks the grader's precision, not its accuracy against human judgment. Second, the dissociation replicates across three grader backends spanning two providers (gpt-4o, gpt-4o-mini, and Meta's llama-3.3-70b; diagonal-to-off-diagonal ratio 39), so it does not hinge on one model snapshot. Third, a ground-truth ablation shows that Framing and Steering are scored by rubric-guided model judgment, whereas Judging, as instrumented, tracks the seeded answer key. The Judging score therefore applies to settings with known ground truth (Section 9.3 specifies the open-ended variant).

Scope. These results establish feasibility, with four qualifications. First, Judging's clean diagonal (+2.00, zero cross-effects) is a controlled check by design, so the independent separation evidence is carried by the free-text, blind-graded Framing and Steering (own-effects +0.62 and +0.43; their cross-effects are near zero apart from the P2-predicted Judging→Steering ceiling effect of +0.27). Second, the dissociation replicates across grader backends from two providers (Additional file 1); agreement with human experts is the validity step that the prepared study of Section 9.3 supplies. Third, with three constructs, the evidence rests on the manipulation-based effect matrix and the inter-skill correlations, which are the appropriate tests for this design. Finally, Steering's own-effect is governed by the P2 ceiling by design, because its quality is bounded by the judging that precedes it. This is a property of the construct rather than of the grader: under a deliberately strict steering rubric the own-effect persists on these challenges (+0.40, versus +0.50 under the standard rubric on the same subset) and the dissociation holds. Human steering data will trace this ceiling directly.

9. Discussion

The feasibility demonstration establishes that the instrument's three scores separate under controlled manipulation of simulated competence and can be produced automatically, with the independent separation evidence carried by the two blind-graded skills; this section reconciles the tensions the framework raises, bounds the present evidence, and specifies the validation program it invites.

9.1 Tensions the framework reconciles

Four tensions bear on the framework, each with a resolution within it.

Offloading versus productive struggle. Evidence that AI use can depress critical thinking through cognitive offloading (Gerlich, 2025; Kosmyna et al., 2025) poses an apparent threat to any framework that puts learners in close partnership with AI. The resolution lies in the zone-of-proximal-development stance: CoRe-3 treats the AI as a mediating tool whose purpose is the learner's eventual independence, and it offloads execution while deliberately retaining the cognitive work of specifying, evaluating, and correcting. The framework is, in this sense, the designed inverse of metacognitive laziness (Proposition P5). Whether exercising the loop actually reduces the offloading signatures is an empirical question this paper does not test; it is reserved for the efficacy study of Section 9.3.

The calibration trap. Judging is metacognitive monitoring, and monitoring is only useful when calibrated. A learner who lacks the domain knowledge to recognize an error cannot detect that error in AI output, however vigilant. Judging is therefore bounded by domain competence, and the framework should be read as describing a skill that develops with domain knowledge, not as a substitute for it (Proposition P4).

Interactive is not automatically productive. ICAP predicts that interactive engagement yields the most learning, but rapid AI dialogue can be voluminous and shallow, a sequence of re-rolls rather than reasoning. Steering is iterative. A learner may reasonably redirect after seeing the next output, or discover mid-loop that a different tack is needed, and working within a model's limits sometimes forces such redirection. The instrument therefore does not ask whether any single correction succeeds. It asks whether the sequence of corrections engages with what the model returned and moves the solution forward across the loop, rather than repeating the same request. It rewards that responsive, convergent steering and does not penalize the legitimate redirection that reasoning with a fallible model requires.

Standards can regress. Judging and Steering presuppose that the learner holds standards adequate to evaluate the output. When the AI is more competent in the domain than the learner, the standards the learner applies may be inferior to the artifact under review. Classical formative-assessment theory does not anticipate this reversal, and it bounds the framework's applicability at the expert frontier.

9.2 Scope and boundary conditions

This is a conceptual contribution with a working instrument, and three boundary conditions define where the present evidence applies. The feasibility demonstration uses simulated learners of controlled competence: they establish that the grader carries signal and that its three scores separate, and they set up the human-learner studies of Section 9.3. To keep generation and grading independent, learners are generated by one model (gpt-4o-mini) and graded by another (gpt-4o). The dissociation replicates across grader backends spanning two providers (Section 8); broadening the grader pool further is part of the validation program. The challenges are English-language, and agreement with human experts is the defined next step (Section 9.3). Two construct-specific boundaries also apply. First, Judging as instrumented here is scored against a seeded answer key, so the Judging score applies to settings with known ground truth; open-ended scoring without a key is a distinct instrumentation that Section 9.3 specifies. Second, Steering's signal is governed by the P2 ceiling by design (its quality is capped by the judging it follows); human steering data will trace this relation directly. What the paper establishes is a theoretically-grounded decomposition with stated propositions, a precise novelty boundary, and evidence that the instrument scores the three constructs separably and automatically over simulated learners; whether they are distinct competencies in students is what the human studies of Section 9.3 will test, and the efficacy study of learning outcomes is the next stage of the program.

9.3 A validation and assessment agenda

The feasibility demonstration shows that the constructs are separable and measurable; it does not establish that the automated grades match expert human judgment, nor that exercising the skills improves learning. We therefore specify the validation program the framework invites, organized as an argument-based validity case in the sense of Messick (1995) and Kane (2013). The present evidence supports the scoring and generalization inferences: the instrument scores consistently and the three constructs separate. The extrapolation inference (that the scores reflect a human competency that transfers) and the implication inference (that the scores support instructional decisions) remain to be established by the studies below.

First, an instrument-validity study is built and ready to run but has not yet been conducted. Three blind expert raters will re-grade a stratified sample of 40 transcripts (covering all eight competence profiles) on the three skills using the per-skill rubrics. Inter-rater reliability will be reported as ordinal Krippendorff's $\alpha$ (primary, since the grades are ordinal), Fleiss' $\kappa$, and pairwise Cohen's $\kappa$. Agreement of the human majority with the automated grade will be reported as Cohen's $\kappa$ and ordinal $\alpha$ separately for each of Framing, Judging, and Steering, against the field-typical bar of $0.6$ to $0.8$. With all three raters scoring the same 40 items the agreement estimate will have a bootstrap half-width on the order of $\pm 0.1$, enough to place each skill inside or outside that band. This step is indispensable rather than a formality: recent work shows that LLM-as-judge agreement with human experts is moderate and task-dependent, sometimes falling to Fleiss' $\kappa$ near $0.1$ to $0.3$ on hard rubric judgments (Feng et al., 2025), so automated grades are validated against humans rather than assumed reliable. The package also confronts the recursive calibration threat (who grades the grader) at a second level. Because the Judging construct rests on machine-seeded ground-truth issues, a parallel task has the same three raters verify a balanced set of 40 seeded issues (20 real and 20 distractor controls) as genuine flaws. This yields a confirmation rate for the Judging ground truth before its recall/precision signal is trusted. The full package (codebook, per-rater shuffled task files with hidden ground truth, and seed-fixed agreement-scoring scripts) is released and reproducible, so the study can run as soon as raters are recruited. Second, a construct-validity study at scale (target of at least 200 real learners) will test Propositions P1 through P4, examining whether Framing, Judging, and Steering dissociate across a learner population and whether the proposed gating relations hold. Third, a grader-robustness study across multiple model backends will separate the framework's signal from any single model's idiosyncrasies. Only an efficacy study with real learners can test Proposition P5 and any learning claim; that is explicitly outside the present scope.

Beyond validity, the framework opens a research program, and one strand can begin before the human studies. External validity can be probed by applying the per-skill instrument to a released corpus of real student-LLM dialogue, the StudyChat dataset of 16,851 student-ChatGPT interactions from a university course (McNichols et al., 2025), to test whether Framing, Judging, and Steering are detectable and separable outside the controlled challenge format. StudyChat's own dialogue-act labels (questioning, verification, and editing) align with the three skills and supply an external anchor. Four further directions follow from the propositions: a longitudinal study of whether the three skills develop and persist with practice; an instructional-intervention study testing whether targeted teaching of each skill improves it, building on evidence that students' behavioral regulation of LLM use is trainable (Clerc et al., 2026); a transfer study testing P4's prediction that Framing, the more structural skill, transfers across domains more readily than the domain-bound Judging, a question made tractable by evidence that metacognitive regulation can show far transfer (Wirth et al., 2025); and a mediation study testing whether metacognitive monitoring accuracy predicts Judging, extending findings that calibration predicts subsequent strategy use in computer-based learning (Lee & Bosch, 2025) and tying the construct to its monitor-control grounding.

9.4 How the findings relate to prior empirical work

The separation the instrument recovers is consistent with, and extends, several strands of prior research. It aligns with evidence that working with AI is a distinct competency rather than a facet of general ability: students' behavioral regulation of LLM use, not self-rated expertise, predicts effective use (Clerc et al., 2026), and metacognitive monitoring can decouple from task performance in human-AI reasoning (Fernandes et al., 2024). Both findings predict that a valid instrument should score the reasoning skills apart from a single self-assessed competence, which is what the crossed design shows. The result extends the evaluative-judgement literature (Bearman et al., 2024; Walton et al., 2025), which identifies judgement as the core human capability for the AI era but treats it as one disposition. The present evidence places that judgement (Judging) alongside two further skills and shows that the three can be scored apart. It sharpens the metacognitive-demands account of generative-AI use (Tankelevitch et al., 2024) by turning a demands analysis into grades that dissociate. It also converges with systematic reviews reporting that prompt-related competencies are taught and assessed piecemeal (Lee & Palmer, 2025), by giving those competencies a measured, separable structure. The finding differs from the AI-literacy tradition (Long & Magerko, 2020; UNESCO, 2024) in its unit of measurement: those frameworks assess understanding of AI systems, whereas the dissociation here concerns performance skills exercised on a task, which is a construct those instruments do not isolate. It is consistent with the human-AI collaboration literature, in which the value of AI assistance depends on the user's skill in directing it (Dell'Acqua et al., 2023; Vaccaro et al., 2024) and skilled users work in a push-back-and-validate mode (Randazzo et al., 2025): the three scored skills decompose that directing skill, and the Judging→Steering ceiling effect is the scored counterpart of validating before redirecting. It differs from scale-based work that pairs collaborative AI literacy with collaborative AI metacognition as correlated factors (Sidra & Mason, 2025), because the crossed design shows that the skills can be scored apart. Finally, the agreement of the automated grades across vendors is consistent with reports that rubric-based LLM grading agrees well on structured criteria while diverging on subjective ones (Yavuz et al., 2025). This is why cross-grader agreement, rather than any single backend, carries the convergent-validity claim.

9.5 Implications for higher-education practice

Once the validation stages are complete, the decomposition offers a concrete path for higher-education practice. Because it scores the three skills separately, it points toward per-skill formative feedback that tells a student which cognitive operation their AI use failed at, rather than a single "prompting" grade. It also points toward defining reasoning with generative AI as an assessable learning outcome that a program could track. Further, it reframes the academic-integrity conversation from prohibiting AI use toward assessing the framing, judging, and steering that competent use requires (Ahangama, 2026). These uses depend on evidence still to be gathered, in particular human-agreement validation and score comparability across tasks, disciplines, and learner populations. The present work contributes the instrument and the demonstration that its scores separate the intended skills; the released, discipline-general platform is what will make the subsequent classroom studies practical to run.

10. Conclusion

The task of education in the age of generative AI is not to produce faster answer-getters but to cultivate critical collaborators: learners who can specify a problem worth solving, judge what a machine returns, and steer it toward something better. That capability is teachable only if it can be named and assessed. CoRe-3 decomposes that capability into three theoretically-grounded, independently-assessable skills (Framing, Judging, and Steering), separates the pre-generation skill from the post-generation one in a way prior frameworks do not, and shows, in a feasibility demonstration over simulated learners, that its three scores separate, with the independent evidence carried by the two blind-graded skills and Judging serving as a controlled check. Because the decomposition tracks what a person must contribute rather than what today's models cannot yet do, it does not lapse as the models improve: the more capable the system, the more the assessable skill is the framing, judging, and steering of it. We release the model, the instrument, and the validation protocol as a foundation to build on: an open, domain-general platform with which a university educator in any discipline, from algorithms to applied ethics, will be able to assess and train this skill once the human-agreement and score-comparability evidence is in place. The present findings support the feasibility of the approach; classroom implementation awaits that validation.

Declarations

Availability of data and materials. All materials are released in a public repository (URL withheld to preserve anonymous review; available to the editors on request): the exact prompt templates of the 16-prompt scoring engine; the generation harness with its model identifiers and generation settings (temperature and seeds); the procedure that generates the eight competence profiles; the synthetic transcripts with their per-call cache; the generated challenges with their rubrics and seeded issues; the grade files for every grader backend; the analysis code, including the challenge-level clustered bootstrap and its full per-cell interval output; a datasheet describing the dataset; and the prepared human-rater validation protocol (codebook, per-rater task files, and agreement-scoring scripts).

Competing interests. The authors declare that they have no competing interests.

Funding. The authors received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors for this research.

Authors' contributions. AA and YA jointly conceived the competency model, designed and implemented the instrument and experiments, analyzed and interpreted the results, and drafted and critically revised the manuscript. Both authors read and approved the final manuscript.

Ethics approval and consent to participate. Not applicable. The study reports a feasibility demonstration over model-generated simulated learners and involves no human participants.

Consent for publication. Not applicable.

Use of generative AI. Generative AI is the assessment instrument evaluated in this paper (Section 7.1); no generative-AI tool was used to author the scholarly content of the manuscript.

Acknowledgements. Not applicable.

Additional files

Additional file 1. File format: DOCX (.docx). Title: Additional file 1, supplementary material. Description: a system walkthrough (screenshots of the running platform and the auto-generation-and-assessment pipeline), robustness and ablations (grader reliability, cross-vendor replication, and the ground-truth ablation), and a cross-disciplinary challenge showcase across twelve disciplines.

References