Guarding Against False Confidence: The CRITIC Checkpoint & The R-G-T-F-C Prompt

Every framework your AI research assistant generates, theoretical, conceptual, or operational, comes out looking polished, structured, and authoritative. That’s the problem.

Polished output is not the same as correct output. When an AI is prompted to build a framework, it optimizes for coherence and confidence, not for doubt. Left unchecked, this creates a silent risk: students walk into their supervisor meeting or viva having never seriously questioned whether the AI’s first suggestion was actually the best fit, only the most plausible-sounding one.

The fix isn’t to distrust the tool. It’s to add a mandatory adversarial step to your workflow: a checkpoint whose only job is to argue against what was just built.

The CRITIC Checkpoint

Think of this as hiring your own devil’s advocate before your real examiner gets the chance to be one. The concept breaks into six checks:

LetterCheckWhat It Catches
CCompeting theories/pathwaysAlternative explanations in your own literature that you overlooked
RReliability of evidenceWhether each link in your framework is strongly or weakly supported
IIdentify the weakest linkThe single relationship in your model least backed by the sources
TTest hidden assumptionsWhat the framework quietly depends on but never justifies
IInterrogate like an examinerThe hardest questions a skeptical reviewer would actually ask
CConclude with a verdictA clear go/no-go decision — not vague reassurance

Where This Fits in Your Workflow

Run the CRITIC prompt after every framework prompt, not just at the end:

Prompt 1 (Theoretical) → CRITIC check
Prompt 2 (Conceptual)  → CRITIC check
Prompt 3 (Operational) → CRITIC check

Catching a weak theoretical foundation early is far cheaper than discovering it after you’ve already built a conceptual map and measurement grid on top of it.

Practical tip: Run this prompt in a fresh conversation turn, not appended directly after the framework was generated. Models are more likely to genuinely critique their own output when not primed by the positive framing they just used to build it.

The Critical Appraisal & Devil’s Advocate Review

Run this after each framework (or once at the end) to stress-test what was just generated before you bring it to your supervisor.

[ROLE]

Act as a skeptical external thesis examiner and devil’s advocate reviewer. Your job is explicitly NOT to validate or improve the previous output — it is to find its weaknesses, blind spots, and unexamined assumptions.

[GOAL]

The goal is to stress-test the framework generated in the previous turn(s) so the student walks into their supervisor meeting or viva already aware of its weakest points, rather than being blindsided by them.

[TASK]

Critically re-examine the Theoretical/Conceptual/Operational Framework(s) produced so far, using only the uploaded source documents in this notebook. Specifically:1. Identify 1-2 alternative theories, models, or variable pathways found in the literature that could plausibly explain the same phenomenon, and explain why the chosen framework may or may not be superior to them. 2. Identify the single weakest link in the hypothesized pathway (the relationship least supported by strong evidence in the sources). 3. State any assumptions the framework quietly relies on that are not explicitly justified by the literature. 4. List the 3 hardest questions a strict external examiner would likely ask about this framework in a viva or defense.

[FORMAT]

Format the output using clean markdown headings exactly as follows: 🔍 Critical Appraisal Alternative Theories/Pathways Considered:

  • [Name each, with 1-2 sentence justification for why it competes with the chosen framework]
  • Weakest Link: [Identify the specific relationship or construct with the least support, and why]
  • Unexamined Assumptions: [List assumptions the framework depends on but does not defend]
  • Anticipated Examiner Questions: [3 specific, pointed questions]
  • Verdict: [One of: “Proceed as-is”, “Proceed with minor revision”, or “Revisit before proceeding” — with a one-sentence justification]

[CONSTRAINT]

You must genuinely attempt to find at least one substantive weakness or point of legitimate doubt — do not simply restate the strengths of the prior output in different words. Alternative theories/pathways must be grounded in the uploaded documents, not invented. Do not soften the critique to be reassuring. Maintain a rigorous, skeptical, examiner-level tone. Omit conversational filler, praise, or concluding reassurances.



    A Note on Trust

    An AI-generated framework should be treated as a strong first draft, not a finished answer. The constraints in Prompts 1–3 are designed to make output look rigorous and authoritative, which is exactly why this fourth prompt exists: authority in tone is not the same as validity in substance. Run the CRITIC checkpoint every time, even when the output feels convincing. Especially when it feels convincing.

    UTM Open Day