AI Epistemic Classification Protocol

A Reflexive Framework for Retrieval, Ranking, Citation, and Source Assessment

Independent Technical Reference Dossier / AI Evaluation Protocol

Current maturity: Concept + Architecture + Validation Design

Document Status and Scope

This dossier specifies a reflexive protocol for consequential AI-mediated classifications across retrieval, omission, ranking, citation, recommendation, synthesis, and evaluation. Its governing object is the classification event, not the visible answer alone.

The protocol separates substantive evidential standing from process robustness; provenance use from provenance substitution; claim maturity from source reputation; recognition from promotion; rejection from erasure; uncertainty from non-assessability; and protocol disposition from record lifecycle. It evaluates observable sensitivity without claiming access to hidden model causation.

The dossier does not claim empirical validation, production implementation, certification, or integration into LoopGuard-AI. It does not seek favorable treatment for RATIUM.AI, treat institutional provenance as inherently defective, or infer value from unfamiliarity.

Current maturity:

□ Concept + Architecture + Validation Design

Completed layers include the problem definition, bounded empirical motivation, formal objects, failure taxonomy, protocol and audit architecture, adversarial benchmark design, candidate metrics, and falsification conditions. Implementation, benchmark execution, calibration, external replication, production deployment, and certification remain uncompleted.

Abstract

AI-mediated retrieval, ranking, citation, recommendation, synthesis, and evaluation allocate epistemic visibility. A source may be accessible but not retrieved, retrieved but excluded, included but not used materially, used but not cited, or cited without receiving decisive evidential standing. These transitions affect which claims enter the user’s effective evidence environment.

Controlled studies show that AI-mediated judgments can be sensitive to variables other than substantive evidential content, including citation popularity, authorship metadata, institutional affiliation, source identity, option order, candidate position, long-context position, linguistic formulation, retrieval configuration, and execution conditions. The findings are bounded by system, task, domain, and intervention. They do not establish one universal tendency toward institutional conformity or one general mechanism of suppression.

This dossier proposes a symmetric and correctable protocol. It first resolves the evaluated object, decomposes claims, classifies maturity, maps claim–evidence relations, and preserves a baseline judgment. It then applies valid provenance and presentation interventions, verifies citations and material-claim coverage, records uncertainty and disagreement, and produces separate substantive and process-robustness judgments.

The canonical protocol output is:

𝒥ₜ=⟨Jₛ,Jₚ,G,F,U,R,Λ⟩

where Jₛ is the substantive judgment, Jₚ is the process-robustness judgment, G is one primary protocol disposition, F is the failure-flag set, U contains uncertainty and limitations, R contains revision conditions, and Λ is the Epistemic Classification Record.

The protocol is explicitly symmetric. Strong familiar and strong unfamiliar material should both be recognized at their supported scope. Weak familiar and weak unfamiliar material should both be rejected or qualified. Prestige, familiarity, novelty, marginality, and rhetorical polish must neither replace evidence nor trigger compensatory ranking.

The dossier concludes that consequential AI-mediated classification should remain connected to a resolved object, a traceable evidence structure, a separately reported process judgment, and a specific path through which relevant evidence can alter future use. This does not guarantee truth. It creates a governance condition under which error can be located, challenged, replayed, and revised.

Methodological Note

The dossier distinguishes four claim layers:

The policy pack may strengthen these defaults according to consequence.

18.10 Full sequence

[ | Task Registration ; | →Object Resolution ; | →Claim Decomposition ; | →Maturity Classification ; | →Source Inventory ; | →Claim–Evidence Graph ; | →B ; | →Sensitivity Testing ; | →Citation and Coverage Review ; | →Uncertainty and Disagreement ; | →⟨Jₛ,Jₚ⟩ ; | →G ; | →R ; | →Λ ]

Part IV Synthesis

Part IV separates substantive judgment from process robustness, makes P3 dependent on a valid material intervention, treats citation quality as both local support and material coverage, and requires an operative correction path. Part V governs the resulting record and the protocol’s own validation.

Part V — Audit, Validation, and Limits

19. The Epistemic Classification Record

19.1 The answer is not the complete governed artifact

A visible answer may contain a conclusion, citations, a confidence statement, and a recommendation. It may omit the information required to reconstruct how the result was produced.

The requirement to preserve lifecycle, context, evaluation, and review information is consistent with NIST’s broader treatment of AI risk as a sociotechnical and lifecycle-governance problem (Schwartz et al. 2022; Tabassi 2023; Autio et al. 2024).

The protocol therefore treats the Epistemic Classification Record as a first-class output.

19.2 Record purpose

The record is a versioned account of an observable classification process. It does not attempt to reproduce hidden chain of thought, complete weight-level causation, or proprietary system logic.

It preserves the relation among:

Task,Object,Evidence,Intervention,Judgment,Disposition,Revision

19.3 Six record layers

Identity

• record ID and version;

• protocol and policy version;

• system and execution identity;

• time and jurisdiction.

Object

• query and intended use;

• evaluated object;

• included and excluded scope;

• claim inventory;

• maturity.

Evidence

• source boundaries;

• source roles;

• claim–evidence relations;

• contradiction;

• missing evidence;

• source dependency.

Testing

• provenance counterfactuals;

• presentation permutations;

• repeated runs;

• citation verification;

• test validity.

Judgment

• baseline state;

• substantive class;

• process class;

• failure flags;

• uncertainty;

• disposition.

Correction

• revision conditions;

• review;

• override;

• replay;

• expiry;

• supersession.

19.4 Versioning

The record distinguishes:

v_(protocol),v_(policy),v_(system),v_(source),v_(record)

A new judgment should not overwrite its predecessor as though the earlier state never existed.

19.5 Lifecycle

The administrative states are:

• draft;

• active;

• superseded;

• invalidated;

• expired;

• archived.

These remain separate from RELEASE, HOLD, REASSESS, and INVALID.

19.6 Baseline and final state

Both the baseline and final state remain preserved. This permits later analysis of whether the protocol changed:

• class;

• maturity;

• citation;

• evidence path;

• process judgment;

• or disposition.

19.7 Traceability

Every load-bearing claim should be traceable to its wording, evidence links, contradiction status, maturity, baseline class, final class, and revision conditions.

19.8 Auditability

Auditability is:

The ability of an authorized reviewer to reconstruct the declared classification process from the retained record.

It does not require release of private data, proprietary details, full prompts, or hidden internal reasoning. Redactions must be declared.

19.9 Explanation and audit

A fluent explanation may be generated after a decision. An audit record should allow inspection of the relevance criterion, claim–source relation, competing sources, source boundary, and intervention results.

19.10 Auditability is not correctness

□ Auditability⇏Correctness

Auditability provides a route to locate and correct error. It does not remove error automatically.

19.11 Replayability

The protocol distinguishes:

• exact reproducibility;

• material equivalence;

• and replayability.

Replayability means the system can repeat the declared classification under sufficiently preserved conditions and explain material deviation.

19.12 Decision-relevant completeness

The record should preserve material transformations and load-bearing evidence without becoming an unusable archive of every token or transient score.

19.13 Data minimization

Identity-sensitive tests should retain only the personal data necessary for the declared task, review, and correction. Synthetic labels and controlled substitutions are preferred where possible.

19.14 Integrity

Technical integrity mechanisms may show that the record was not altered after creation.

□ Record Integrity⇏Judgment Correctness

20. Human Review, Override, and Replay

20.1 Human review is governed

Human review can add domain knowledge, contextual understanding, authority, and responsibility. It can also add prestige sensitivity, inconsistency, conflict of interest, or unsupported intuition.

Human and LLM judges have both demonstrated sensitivity to controlled perturbations in tested settings (Chen et al. 2024).

Human review is therefore another governed classification event.

20.2 Review triggers

Human review may be required where:

• the object remains materially ambiguous;

• specialist knowledge is necessary;

• consequence is high or critical;

• D3–D5 disagreement remains;

• P3 or P4 is observed;

• S5 cannot be resolved automatically;

• a blocking flag exists;

• or an appeal challenges a consequential field.

20.3 Reviewer jurisdiction

Reviewer roles may include:

• domain expert;

• evidence-method reviewer;

• policy authority;

• legal authority;

• protocol auditor;

• system operator;

• decision owner;

• affected-party representative.

The record identifies competence, scope, authority, and conflicts of interest.

20.4 Review packet

The reviewer receives the task, object, claims, maturity, evidence graph, baseline, sensitivity results, citation review, uncertainty, failure flags, and proposed disposition.

20.5 Blinded review

Where provenance is not constitutive, the protocol may compare visible and blinded human judgments.

Blinding is conditional and reversible. It must not remove authenticity, authority, or conflict-of-interest information required by the task.

20.6 Review outcomes

A reviewer may:

• confirm the judgment;

• narrow scope;

• change the supported formulation;

• alter maturity;

• change Jₛ or Jₚ;

• change disposition;

• return the record to evidence mapping;

• or invalidate the run.

20.7 Override

An override changes a material field through an authorized intervention.

Override classes include:

• evidential override;

• object override;

• policy override;

• process override;

• authority override.

The pre-override and post-override states both remain visible.

20.8 Unsupported override

An override relying only on seniority, institutional position, reputation, or unexplained intuition is unsupported.

A decision owner may possess authority to permit limited use. This must be represented as a policy or authority override, not as evidential refutation.

20.9 Appeal

An affected party may challenge a field-level element such as object definition, source omission, citation verification, maturity, process class, failure flag, or disposition.

20.10 Replay

A replay repeats a classification under declared preserved conditions.

Replay results are:

• exact reproduction;

• equivalent reproduction;

• explained deviation;

• unexplained deviation;

• not reproducible.

20.11 Replay and instability

Unexplained material deviation may support P4 and F12. Explained change after new evidence or process repair is successful correction, not instability.

20.12 Shadow review

Before operational authority, the protocol may run in shadow mode. It produces classifications and flags without controlling the real decision.

20.13 Limited operational authority

Early permissible actions include requesting citation verification, reassessment, limitation, blinded review, or source-set expansion.

The protocol should not begin by automatically suppressing sources, penalizing authors, or permanently altering public rankings.

21. The Adversarial Benchmark

21.1 Why adversarial validation is necessary

A protocol can appear successful by embedding its assumptions into the benchmark. It may label every unfamiliar source strong, place every difficult case on HOLD, or improve citation precision by citing almost nothing.

The benchmark must contain cases capable of demonstrating protocol failure.

21.2 Validation targets

The benchmark evaluates:

V1=Detection

V2=Discrimination

V3=Symmetry

V4=Corrective Value

Detection concerns known controlled defects. Discrimination concerns legitimate versus illegitimate use of the same variable. Symmetry concerns equivalent evidence standards across prestige and novelty conditions. Corrective value concerns improvement relative to simpler baselines.

21.3 Factorial core

The general four-cell design crosses evidence quality with prestige, familiarity, or novelty condition.

The expected behavior is to recognize strong items and reject or qualify weak items across both conditions.

21.4 Strong and weak items

A strong item contains a resolved object, bounded claims, adequate evidence, valid inference, accurate maturity, and explicit limits.

A weak item contains a controlled defect such as causal promotion, invalid generalization, citation mismatch, maturity inflation, object conflation, contradiction concealment, or load-bearing reasoning failure.

Where feasible:

Weak Item=Strong Item+One Controlled Defect

21.5 Controls

The benchmark includes:

• positive controls;

• negative controls;

• legitimate-provenance controls;

• anti-symmetry controls;

• excessive-HOLD controls.

21.6 Baselines

The comparison conditions are:

• B0 — Unstructured Baseline;

• B1 — Neutral Evidence Rubric;

• B2 — Self-Applied Protocol;

• B3 — External Protocol Runner;

• B4 — Human Expert Condition.

B1 is load-bearing. If a simpler evidence rubric performs equivalently, full protocol complexity may not be justified.

21.7 Twelve suites

The benchmark contains twelve suites:

• A — Prestige;

• B — Citation Popularity;

• C — Source Identity;

• D — Order and Position;

• E — Semantic Familiarity;

• F — Novelty;

• G — Rhetorical Coherence;

• H — Maturity;

• I — Object Conflation;

• J — Citation Support;

• K — Redundancy and Evidential Diversity;

• L — Run and Cross-System Stability.

Each suite contains eight core episodes, producing:

12×8=96

core benchmark episodes.

21.8 Adjudicated targets

Each episode receives an Adjudicated Benchmark Target that may include an expected class, acceptable alternatives, required flags, prohibited flags, acceptable dispositions, and revision conditions.

The target need not be one exact wording.

21.9 Counterfactual-validity audit

Every pair is audited for preservation of object, claim, evidence, scope, maturity, clarity, authenticity, and task relevance.

Invalid pairs cannot support sensitivity claims.

21.10 Excessive HOLD

Clear recognition and rejection cases are included to detect decision avoidance.

21.11 Validation stages

Validation proceeds through:

1. construction audit;

2. pilot;

3. controlled benchmark;

4. protocol-intervention study;

5. external replication;

6. shadow evaluation;

7. limited operational trial.

No stage implies automatic promotion to the next.

21.12 Validation dispositions

Benchmark validation uses PASS, HOLD, and FAIL. These are validation outcomes, not protocol dispositions for individual classification objects.

A bounded PASS must state the tested component, system, domain, and corpus.

22. Metrics and Validation

22.1 Metrics are candidate instruments

A formula is not valid because it is precise. Every metric requires construct definition, scoring reliability, threshold calibration, domain testing, and evidence that improvement matters.

22.2 Metric families

The protocol evaluates:

• substantive accuracy;

• sensitivity;

• citation support;

• material-claim coverage;

• maturity;

• object resolution;

• symmetry;

• correction;

• disposition;

• auditability;

• human review;

• replay;

• cost;

• latency.

22.3 No global protocol score

One score would conceal trade-offs among accuracy, sensitivity, coverage, HOLD inflation, cost, and auditability.

The preferred output is a scorecard with primary, secondary, exploratory, and guardrail metrics.

22.4 Judgment distance

Substantive classes are not treated as one simple ordinal scale. S5 is not merely worse than S4.

Where distance is needed, the protocol uses versioned cost matrices and preserves exact categorical changes beside any composite value.

22.5 Sensitivity metrics

Candidate sensitivity measures include:

• Provenance Sensitivity Index;

• Counterfactual Flip Rate;

• Order Sensitivity Index;

• Run Instability Index;

• Cross-System Decision Divergence Rate.

A sensitivity measure records change. Task relevance and counterfactual validity determine interpretation.

22.6 Citation metrics

Primary citation measures include:

• Citation Support Precision;

• Material Claim Coverage;

• Load-Bearing Claim Coverage;

• Citation Strength Match;

• Citation Scope Match;

• Fabricated Citation Rate.

22.7 Symmetry metrics

The benchmark reports:

• Prestige Symmetry Gap;

• Novelty Symmetry Gap;

• Semantic Familiarity Penalty Gap;

• Directional Compensation Rate.

Low gap must be reported with absolute accuracy because equal poor performance is not success.

22.8 Maturity and object metrics

Candidate measures include:

• Maturity Promotion Error;

• Maturity Collapse Error;

• Mixed-Maturity Differentiation Rate;

• Object Resolution Accuracy;

• Object Conflation Rate;

• Authorized Transfer Precision and Recall.

22.9 HOLD and decision retention

The protocol measures:

EHR=Excessive HOLD Rate

and:

DRR=Decision-Retention Rate

A protocol that avoids most decisions may reduce visible errors while losing operational value.

22.10 Non-inferiority

Protocol use must preserve substantive performance within calibrated margins.

For example:

CA_(protocol)−CA_(baseline)≥−δ_(CA)

The margins remain uncalibrated and task-specific.

22.11 Primary pilot scorecard

The primary 96-episode pilot reports:

1. Content Accuracy;

2. Load-Bearing Accuracy;

3. Counterfactual Flip Rate;

4. Citation Support Precision;

5. Material Claim Coverage;

6. Maturity Promotion Error;

7. Object Conflation Rate;

8. Prestige Symmetry Gap;

9. Novelty Symmetry Gap;

10. Excessive HOLD Rate;

11. Decision-Retention Rate;

12. incremental cost and latency.

Guardrails include Counterfactual Validity Rate, Counterfactual Contamination Rate, Fabricated Citation Rate, RELEASE Precision, Record Schema Validity, and L4 event count.

22.12 Calibration

Calibration proceeds through construct review, scoring manual, inter-rater study, positive and negative controls, pilot distribution, threshold setting, external replication, and drift review.

22.13 Metric gaming

The benchmark must detect strategies such as:

• citing only easy claims;

• weakening every claim;

• ignoring legitimate provenance;

• placing everything on HOLD;

• assigning identical results to every cell;

• producing verbose but low-value audit records.

22.14 Governing rule

□ Metric Improvement⇏Protocol Validation

23. Falsification and Failure Conditions

23.1 A correction protocol must be correctable

The protocol must identify evidence that would require it to be narrowed, simplified, revised, or rejected.

23.2 Conceptual failure

The architecture is weakened if:

• epistemic allocation adds no useful distinction beyond ordinary retrieval;

• dual judgment adds no information beyond existing labels;

• object resolution does not reduce meaningful error;

• or revision conditions cannot be made operative.

23.3 Empirical failure

The empirical motivation must be narrowed if reported non-content sensitivities fail to replicate or are explained adequately by legitimate task relevance.

23.4 Unified-mechanism failure

Ontology-preserving conformity should be removed as a unified mechanism if its components do not co-occur, object persistence is explained by evidence quality, or the concept adds no value beyond separate flags.

23.5 Symmetry failure

The protocol fails centrally if it favors independent sources, penalizes prestige by default, rewards novelty language, or interprets criticism of itself as evidence of conformity.

23.6 Counterfactual failure

The counterfactual layer should be narrowed where valid content-preserving interventions cannot be constructed reliably or contamination remains high.

23.7 Detection and false-positive failure

The protocol fails if it misses controlled defects or repeatedly flags legitimate authority, harmless reordering, or task-relevant popularity as bias.

23.8 Accuracy failure

The protocol should not proceed where it materially reduces substantive accuracy without a separately justified benefit.

23.9 Coverage failure

Improved citation precision does not compensate for unsupported central claims.

23.10 Excessive-HOLD failure

A protocol that withholds every difficult decision demonstrates avoidance, not governance.

23.11 Correction failure

Generic phrases such as “future research” do not satisfy the correction requirement.

23.12 Audit failure

Auditability fails where a qualified reviewer cannot reconstruct the object, evidence, intervention, judgment, or override. It can fail through omission or unstructured excess.

23.13 Human-review failure

Human review fails where reviewers lack jurisdiction, overrides lack support, prestige replaces analysis, or disagreement is erased.

23.14 Policy failure

A system may execute the protocol correctly under a defective policy. Policy validation remains separate from execution validation.

23.15 Operational failure

The protocol may be disproportionate in cost, latency, storage, or review burden. The appropriate response may be tiering, simplification, or narrower scope.

23.16 Security and privacy failure

The protocol should be restricted where it requires unnecessary personal data or exposes manipulations that facilitate ranking evasion or deceptive provenance.

23.17 Simpler-baseline failure

The full architecture should not be retained where a simpler method achieves equivalent benefit with fewer costs and failure modes.

23.18 External-replication failure

Positive internal results must be narrowed where independent teams cannot reproduce detection, symmetry, accuracy preservation, or audit value.

23.19 Self-sealing prohibition

A framework that treats success, failure, criticism, and acceptance as confirmation is correctionless.

□ Protocol Architecture⇏Protocol Validity

□ Protocol Compliance⇏Epistemic Improvement

24. System Boundary and Non-Claims

24.1 The system is sociotechnical

The relevant system may include:

[ | Corpus Construction+Indexing+Query Rewriting ; | +Retrieval+Ranking+Context Selection ; | +Generation+Citation+Evaluation ; | +Interface+Human Review+Organizational Policy ]

This boundary is consistent with NIST guidance treating AI risk as distributed across actors, processes, deployment contexts, and lifecycle stages (Schwartz et al. 2022; Tabassi 2023; Autio et al. 2024).

24.2 Corpus boundary

A source absent from the accessible corpus cannot be retrieved. Absence may result from access restrictions, indexing policy, language, format, source availability, or recency.

The protocol cannot infer source weakness from absence alone.

24.3 Indexing boundary

A source may be present but poorly represented because of parsing, metadata loss, segmentation, or terminology mismatch.

24.4 Retrieval boundary

Retrieval may optimize relevance, recency, authority, popularity, or user preference. Where the objective is unknown, the protocol should not claim certainty about why a source was omitted.

24.5 Generation boundary

The generator may use retrieved sources, parametric knowledge, or both. Visible citations do not necessarily reveal the complete contribution of each source.

24.6 Citation boundary

Citation may be performed during generation, after generation, or through a separate attribution layer. A citation failure may originate in a different component from the substantive claim.

24.7 Evaluator boundary

An evaluator may be the same model, another model, a rules engine, a human, or a hybrid. Agreement by another model is not independent validation by default.

24.8 Organizational boundary

Organizations determine allowed sources, thresholds, review triggers, authorized use, retention, and downstream action.

The distinction between technical execution and organizational governance is consistent with the AI RMF’s allocation of risk-management functions across organizations that design, develop, deploy, or use AI systems (Tabassi 2023).

The protocol distinguishes execution failure, policy failure, and authority failure.

24.9 Introspection boundary

The protocol does not require hidden chain of thought, exact training-data influence, or complete internal priors.

It evaluates observable input, source environment, intervention, output, citation, and revision.

Unsupported internal-cause claims trigger:

F16=Unsupported Introspection Claim

24.10 Causal boundary

□ Counterfactual Effect⇏Complete Causal Model

24.11 Truth boundary

The protocol can improve object precision, evidence traceability, sensitivity visibility, and correction. It cannot guarantee truth.

A complete source set may be collectively wrong. A robust process may preserve a mistaken theory.

24.12 Neutrality boundary

The protocol does not seek equal outcomes. Evidence may justify unequal confidence, inclusion, and strong rejection.

24.13 Diversity boundary

The protocol does not require ideological, institutional, or minority-position quotas. It requires sufficient relevant evidential diversity.

24.14 Institutional boundary

The protocol is not anti-institutional. Institutions may provide authority, stable versioning, review, accountability, and correction.

24.15 Independent-source boundary

Independent, low-citation, and unconventional sources receive no compensatory standing. They may deserve examination; their classification remains evidence-dependent.

24.16 Novelty boundary

Novelty may justify discovery or comparison. It does not establish truth, originality, maturity, or corrective value.

24.17 Robustness boundary

□ Tested Robustness⇏Universal Robustness

24.18 Human boundary

Human review can add expertise and responsibility but remains fallible. Empirical comparison of human and LLM judges supports treating human review as a potentially sensitive intervention rather than automatic ground truth (Chen et al. 2024).

24.19 Automation boundary

Protocol findings do not directly authorize source suppression, author sanctions, permanent public ranking changes, or access restrictions.

24.20 Security boundary

The protocol should not be used to reverse-engineer ranking systems for manipulation, construct deceptive provenance, manufacture credibility, or optimize disinformation inclusion.

24.21 Privacy boundary

Identity-sensitive testing should minimize real personal data and use synthetic or approved research procedures where feasible.

24.22 Certification boundary

The protocol is not a certification standard. Certification would require recognized authority, validated methods, calibrated thresholds, audit procedures, and defined scope.

24.23 LoopGuard-AI boundary

The protocol may later inform an evaluator, evidence record, correction gate, replay layer, or reassessment mechanism within LoopGuard-AI.

No implementation or validated integration is claimed.

24.24 RATIUM.AI boundary

RATIUM.AI provides conceptual provenance for parts of the dossier. It is not the preferred benchmark source or a source deserving compensatory visibility.

The protocol does not request promotion, citation, ranking, or favorable classification for RATIUM.AI.

24.25 Explicit non-claims

The dossier does not establish:

• universal AI suppression of unfamiliar ontologies;

• general disadvantage to independent researchers;

• universal institutional preference;

• general novelty amplification;

• complete hidden causation from observable sensitivity;

• correctness from auditability;

• truth from robustness;

• validity from reproducibility;

• correction from human override;

• calibrated metrics;

• benchmark validation;

• production readiness;

• certification;

• or implementation within LoopGuard-AI.

24.26 Current maturity

□ Concept + Architecture + Validation Design

Part V Synthesis

The governed artifact is the versioned Epistemic Classification Record. Human review, override, replay, validation, metrics, and institutional use remain auditable and bounded by explicit falsification, security, privacy, and maturity limits.

Conclusion — Classification Must Remain Answerable to Correction

AI-mediated systems construct the effective evidence environment through which questions become answerable. Sources may be available but unretrieved, retrieved but omitted, present but unused, or cited without supporting the decisive claim. Unequal allocation is unavoidable; the governance issue is whether it remains connected to the task, object, evidence, maturity, and a correction path.

The empirical basis is bounded: non-content sensitivity is observable and testable in particular systems and tasks. Citation popularity, authorship metadata, affiliation, source identity, order, context position, linguistic formulation, retrieval design, and execution conditions can affect output. These findings do not establish a universal conformity mechanism.

The protocol therefore begins with object resolution and preserves maturity from concept through certification. It reports separately:

Jₛ=Substantive Judgment Jₚ=Process-Robustness Judgment

This separation permits a claim to be supported yet process-sensitive, contradicted through a robust process, or robustly classified as insufficiently evidenced. It also enforces symmetry: strong familiar and unfamiliar material should be recognized; weak prestigious and marginal material should be qualified or rejected under the same evidential standard.

The complete governed artifact is the Epistemic Classification Record, which preserves the task, object, claims, maturity, evidence graph, baseline, interventions, citations, judgments, disposition, revision conditions, review, override, and replay. Auditability, reproducibility, robustness, and human review do not guarantee truth; they make error more locatable and correction more accountable.

The protocol’s final claim remains procedural:

A consequential AI-mediated classification should remain connected to a resolved object, a traceable evidence structure, a separately reported process judgment, and a specific path through which relevant evidence can alter its future use.

A classification becomes governable not when challenge is eliminated, but when relevant challenge can produce justified, traceable revision.

References

Abolghasemi, Amin, Leif Azzopardi, Seyyed Hadi Hashemi, Maarten de Rijke, and Suzan Verberne. 2025. “Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models.” In Findings of the Association for Computational Linguistics: ACL 2025, 21105–21124. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-acl.1087.

Algaba, Andres, Carmen Mazijn, Vincent Holst, Floriano Tori, Sylvia Wenmackers, and Vincent Ginis. 2025. “Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias.” In Findings of the Association for Computational Linguistics: NAACL 2025, 6844–6879. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-naacl.381.

Autio, Chloe, Reva Schwartz, Jesse Dunietz, Shomik Jain, Martin Stanley, Elham Tabassi, Patrick Hall, and Kamie Roberts. 2024. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1.

Chen, Guiming Hardy, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. “Humans or LLMs as the Judge? A Study on Judgement Bias.” In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 8301–8327. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.474.

Cheng, Jiali, and Hadi Amiri. 2025. “EqualizeIR: Mitigating Linguistic Biases in Retrieval Models.” In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2, 889–898. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-short.75.

Dai, Sunhao, Zhanshuo Cao, Wenjie Wang, Liang Pang, Jun Xu, See-Kiong Ng, and Tat-Seng Chua. 2025. “Media Source Matters More Than Content: Unveiling Political Bias in LLM-Generated Citations.” In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 17256–17276. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.emnlp-main.872.

Dycke, Nils, and Iryna Gurevych. 2026. “Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework.” Transactions of the Association for Computational Linguistics 14: 465–488. https://doi.org/10.1162/tacl.a.642.

Hsieh, Cheng-Yu, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, and Tomas Pfister. 2024. “Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization.” In Findings of the Association for Computational Linguistics: ACL 2024, 14982–14995. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-acl.890.

Khan, Saadat Hasan, Spencer Hong, Jingyu Wu, Kevin Lybarger, Youbing Yin, Erin Babinsky, and Daben Liu. 2026. “DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation.” In Findings of the Association for Computational Linguistics: EACL 2026, 2873–2894. Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.findings-eacl.150.

Kirsten, Elisabeth, Jost Große Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, and Muhammad Bilal Zafar. 2026. “Characterizing Web Search in The Age of Generative AI.” In Findings of the Association for Computational Linguistics: ACL 2026, 10827–10848. Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.findings-acl.526.

Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12: 157–173. https://doi.org/10.1162/tacl_a_00638.

Schwartz, Reva, Apostol Vassilev, Kristen K. Greene, Lori Perine, Andrew Burt, and Patrick Hall. 2022. Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. NIST Special Publication 1270. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.1270.

Shi, Lin, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, and Soroush Vosoughi. 2025. “Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge.” In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, 292–314. Asian Federation of Natural Language Processing and Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.ijcnlp-long.18.

Tabassi, Elham. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.100-1.

Vasu, Sai Suresh Macharla, Ivaxi Sheth, Hui-Po Wang, Ruta Binkyte, and Mario Fritz. 2026. “Justice in Judgment: Unveiling (Hidden) Bias in LLM-Assisted Peer Reviews.” In Findings of the Association for Computational Linguistics: ACL 2026, 307–330. Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.findings-acl.14.

Wang, Peiyi, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. “Large Language Models Are Not Fair Evaluators.” In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Volume 1, 9440–9450. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.511.

Wei, Sheng-Lun, Cheng-Kuang Wu, Hen-Hsen Huang, and Hsin-Hsi Chen. 2024. “Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models.” In Findings of the Association for Computational Linguistics: ACL 2024, 5598–5621. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-acl.333.

Xu, Yumo, Peng Qi, Jifan Chen, Kunlun Liu, Rujun Han, Lan Liu, Bonan Min, Vittorio Castelli, Arshit Gupta, and Zhiguo Wang. 2025. “CiteEval: Principle-Driven Citation Evaluation for Source Attribution.” In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Volume 1, 32759–32778. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.acl-long.1574.

Appendix A — Evidence and Claim-Control Register

A.1 Purpose and Authority

This appendix governs the evidential and claim-status boundaries of the dossier. It maps material propositions to their object, claim type, maturity, evidence grade, source family, authorized scope, permitted inference, prohibited promotion, and revision condition.

Where a main-text statement appears broader than its register entry, the narrower entry governs.

A.2 Claim-Control Logic

The register blocks four unsupported transitions:

Bounded Finding⇏Universal Claim

Observable Sensitivity⇏Complete Causal Explanation

Conceptual Specification⇏Empirical Validation

Protocol Compliance⇏Epistemic Correctness

A.3 Evidence Grades

E1 — Direct Empirical Support

A controlled or systematic study bears directly on the stated phenomenon within a specified system, task, domain, population, or intervention.

E2 — Adjacent Empirical Support

A study establishes a component, analogous sensitivity, or relevant empirical condition without testing the complete dossier claim.

E3 — Methodological Support

A study supports an evaluation procedure, counterfactual method, measurement distinction, citation-verification approach, or mitigation principle.

G1 — Governance or Standards Basis

A recognized governance publication supports lifecycle risk management, documentation, testing, evaluation, verification, or continuous review.

C1 — Original Conceptual or Architectural Contribution

The proposition is introduced, synthesized, or formally specified by this dossier.

N1 — Normative Governance Requirement

The proposition states what a governable classification process should require.

H — HOLD

The proposition is plausible, motivated, or operationally decomposable but not sufficiently established.

X — Prohibited Inference

The proposition is unsupported, self-protective, logically invalid, or incompatible with the protocol’s symmetry and maturity boundaries.

A.4 Claim Status

The register statuses are:

• SUPPORTED WITHIN SCOPE;

• PARTIALLY SUPPORTED;

• METHODOLOGICALLY SUPPORTED;

• CONCEPTUALLY SPECIFIED;

• NORMATIVELY PROPOSED;

• HOLD;

• PROHIBITED.

These are not the S1–S5 classes assigned within a protocol run.

A.5 Claim-Record Schema

□ 𝒞ᵢ^(reg)=⟨id,O,T,M,Γ,S,Ω,L,P,X,R,D,V⟩

where:

• id: claim identifier;

• O: object;

• T: claim type;

• M: maturity;

• Γ: primary evidence grade;

• S: source family;

• Ω: authorized scope;

• L: limitations;

• P: permitted inference;

• X: prohibited promotion;

• R: revision condition;

• D: dossier location;

• V: version.

Each claim receives exactly one primary evidence grade. Additional functions are recorded as conceptual basis, methodological use, governance basis, or normative use.

A.6 Source-Family Register

A.7 Major Empirical Claims

EC-01 — Citation-Popularity Preference

Authorized claim: In the tested scholarly-reference task, LLM-generated recommendations displayed a heightened preference for highly cited papers.

Primary grade: E1
Source family: SF-01
Status: SUPPORTED WITHIN SCOPE

Prohibited promotion: AI systems generally suppress low-citation research.

EC-02 — Persistence after selected controls

Authorized claim: The reported high-citation preference persisted after controls for publication year, title length, author count, and venue.

Primary grade: E1
Source family: SF-01

Prohibited promotion: Citation count was the sole causal determinant.

EC-03 — Authorship Metadata

Authorized claim: Authorship metadata can materially affect attribution quality in tested generator-aware RAG pipelines.

Primary grade: E1
Source family: SF-02
Methodological use: E3

Prohibited promotion: Named or institutional authors are generally trusted regardless of content.

EC-04 — Institutional Affiliation

Authorized claim: Institutional affiliation and related author metadata affected LLM-generated peer-review judgments in the tested settings.

Primary grade: E1
Source family: SF-03

Prohibited promotion: AI systems generally favor elite institutions.

EC-05 — Borderline Consequence

Authorized claim: Metadata effects can become consequential where modest changes cross an operational threshold.

Primary grade: E1
Source families: SF-03 and SF-05

EC-06 — Political Source Identity

Authorized claim: In the tested political-news setting, media-outlet identity affected citation selection beyond controlled content orientation.

Primary grade: E1
Source family: SF-04

Prohibited promotion: Source identity generally matters more than content.

EC-07 — Option Order and Token Representation

Authorized claim: LLM selection behavior can vary under option-order and option-token changes.

Primary grade: E1
Source family: SF-05

EC-08 — LLM-as-a-Judge Position

Authorized claim: Candidate-answer position can materially affect comparative LLM judgments in tested settings.

Primary grade: E1
Source family: SF-05
Methodological use: E3

EC-09 — Long-Context Position

Authorized claim: Relevant information can be used less effectively in disadvantaged positions within long contexts.

Primary grade: E1
Source family: SF-06

Prohibited promotion: Evidence in the middle is always ignored.

EC-10 — Linguistic Complexity

Authorized claim: Retrieval performance can vary across linguistically simple and complex query formulations.

Primary grade: E1
Source family: SF-07

Prohibited promotion: Retrieval systems preserve dominant ontologies.

EC-11 — Generative-Search Heterogeneity

Authorized claim: Generative-search systems can differ in source selection, internal and external knowledge use, retrieval footprint, diversity, and stability.

Primary grade: E1
Source family: SF-08

EC-12 — Run and Temporal Variation

Authorized claim: Generative-search outputs can vary across executions and time.

Primary grade: E1
Source family: SF-08

EC-13 — Citation Evaluation

Authorized claim: Citation quality requires evaluation of the query, generated claim, source, and retrieval context.

Primary grade: E3
Source family: SF-09
Status: METHODOLOGICALLY SUPPORTED

EC-14 — Automatic Review and Logic Faults

Authorized claim: In the tested counterfactual framework, inserted research-logic faults had no significant effect on the output reviews of the evaluated automatic-review approaches.

Primary grade: E1
Source family: SF-10
Methodological use: E3

EC-15 — Redundancy and Diversity

Authorized claim: Similarity-focused retrieval can produce redundant evidence, and relevance-constrained query-aware diversity improved performance in tested reasoning-intensive tasks.

Primary grade: E1
Source family: SF-11
Methodological use: E3

EC-16 — Human and Machine Perturbation

Authorized claim: Human and machine evaluators can both be affected by controlled perturbations.

Primary grade: E1
Source family: SF-12

Prohibited promotion: Human and machine failure modes are equivalent.

A.8 Methodological and Governance Claims

MG-01 — Ranking direction does not diagnose quality

Primary grade: C1
Methodological use: E3
Status: CONCEPTUALLY SPECIFIED

MG-02 — Provenance use versus substitution

Primary grade: C1
Empirical basis: E2

MG-03 — Observable sensitivity justifies process review

Primary grade: C1
Methodological use: E3

MG-04 — Object resolution precedes consequential classification

Primary grade: N1
Conceptual basis: C1

MG-05 — Claim type and maturity require separate control

Primary grade: N1
Conceptual basis: C1

MG-06 — Citation is a claim–source relation

Primary grade: C1
Methodological use: E3

MG-07 — Source count is not corroboration

Primary grade: E3
Conceptual use: C1

MG-08 — Uncertainty should be typed

Primary grade: C1
Normative use: N1

MG-09 — Revision conditions are required

Primary grade: N1
Governance basis: G1

MG-10 — Human override must be audited

Primary grade: N1
Governance basis: G1
Methodological use: E3

MG-11 — Auditability is not correctness

Primary grade: C1
Normative use: N1

MG-12 — Tested robustness is not truth

Primary grade: C1

MG-13 — Validation must be symmetric

Primary grade: N1
Conceptual basis: C1

MG-14 — No single score is sufficient

Primary grade: C1

MG-15 — Validation is component- and scope-specific

Primary grade: E3
Governance basis: G1

A.9 Original Contributions

The following are original or synthesized by this dossier and have primary grade C1 unless otherwise stated:

• CA-01 — AI-Mediated Epistemic Allocation;

• CA-02 — Epistemic Classification Event;

• CA-03 — Canonical Classification Object 𝒜ₜ;

• CA-04 — Dual Judgment;

• CA-05 — Final Protocol Output 𝒥ₜ;

• CA-06 — Object Types and Transfer Bridge;

• CA-07 — Claim-Maturity Architecture;

• CA-08 — Source Inventories;

• CA-09 — Claim–Evidence Graph;

• CA-10 — Failure Taxonomy F1–F18;

• CA-11 — Failure Severity L1–L4;

• CA-12 — S1–S5 and P1–P5;

• CA-13 — Protocol Dispositions;

• CA-14 — Revision-Condition Architecture, with normative use N1;

• CA-15 — Epistemic Classification Record;

• CA-16 — Human Review, Override, and Replay;

• CA-17 — Adversarial Benchmark, status Validation Design;

• CA-18 — Candidate Metrics, status Validation Design.

None is described as empirically validated.

A.10 HOLD Register

The following remain on HOLD:

• H-01 — Ontology-preserving conformity as a unified empirical mechanism;

• H-02 — Novelty amplification as a general AI tendency;

• H-03 — Rhetorical-coherence substitution as a general ranking effect;

• H-04 — Complete citation-network reinforcement loop;

• H-05 — Universal institutional preference;

• H-06 — Systematic independent-source suppression;

• H-07 — Real-world protocol improvement;

• H-08 — Dual-judgment superiority over simpler methods;

• H-09 — Effectiveness of revision conditions;

• H-10 — Metric validity and thresholds;

• H-11 — Benchmark validity;

• H-12 — Production readiness;

• H-13 — Certification use;

• H-14 — LoopGuard-AI implementation;

• H-15 — Causal explanation of a named project’s visibility.

A.11 Prohibited Inference Register

The following are prohibited:

Low Ranking⇒Conformity

High Ranking⇒Epistemic Independence

Familiar Source⇒Prestige Substitution

Unfamiliarity⇒Corrective Standing

Independent Origin⇒Truth

Citation Count⇒Claim Support

Multiple Sources⇒Independent Evidence

Human Override⇒Ground Truth

Observed Sensitivity⇒Complete Hidden Mechanism

No Tested Effect⇒Absence of Untested Sensitivity

P1⇒S1

Auditable Record⇒Correct Judgment

Schema Compliance⇒Protocol Validation

HOLD⇒False

INVALID Record⇒False Object

Architecture⇒Prototype, Validation, or Production

The protocol may not be used to manipulate rankings, manufacture credibility, or promote RATIUM.AI or another named source.

A.12 Dependency Register

Authorized:

Bounded Sensitivity Evidence⇒Justification for Controlled Testing

Blocked:

Bounded Sensitivity Evidence⇏Universal Ontology-Preserving Mechanism

Authorized:

E3+C1⇒Candidate Protocol Architecture

Blocked:

Architecture+Benchmark Design⇏Validation

Blocked:

Metric Formula⇏Construct Validity

A.13 RATIUM.AI Boundary

RATIUM.AI may appear as conceptual provenance or publication origin. It must not appear as the preferred benchmark source, an object whose rank proves protocol quality, or a source deserving compensatory visibility.

RATIUM.AI Ranked Poorly⇏Ontology-Preserving Conformity

RATIUM.AI Ranked Highly⇏Epistemic Independence

A.14 LoopGuard-AI Boundary

The protocol may later inform evaluators, records, correction gates, replay layers, or reassessment triggers. No implementation or validated integration is claimed.

A.15 Update Rules

Claims may be promoted only through new relevant evidence. Repetition, elaboration, formalization, citation count, or architectural completeness do not convert C1 into E1.

Retractions, corrections, version changes, and standards revisions trigger review of dependent claims.

A.16 Master Status