---
title: "Giving an LLM a webpage's own text degrades its judgment of the source's authority — authority is not textual style or fluency (AuthorityBench)"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Tier-1 claim, but flagged: the AuthorityBench identifier/venue carry an internal date inconsistency and the ρ figure is unverified — see [[question-verify-authoritybench-paper-identity-and-metric]]; queen re-fetch not performed); 2026-07-12 cross-model audit (writer claude-opus-4-8, auditor claude-fable-5): re-fetch performed against the primary PDF — arXiv:2603.25092v1 [cs.IR] resolves to AuthorityBench (Yao, Zhang & Bi, State Key Laboratory of AI Safety, ICT CAS, submitted 26 Mar 2026); quote confirmed verbatim in §1; ρ 75.28% confirmed in Table 1 (Qwen3-32B, DomainAuth fine-grained, w/o webpage text, PairJudge-PointScore — the model's best setting); date inconsistency resolved: the paper is 2026, the capture's '2025 research' label was the error; [unverified-quant] flag cleared"
source_url: "https://arxiv.org/html/2603.25092"
source_title: "AuthorityBench: Benchmarking LLM Authority Perception for Reliable Retrieval-Augmented Generation"
source_author: "Yao, Zhang & Bi (CAS State Key Lab of AI Safety) — AuthorityBench"
source_date: 2026
source_quote: "Incorporating webpage text generally degrades LLM judgment under all settings, indicating authority is not equivalent to textual style, fluency, or narrative richness"
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-admiralty-code-to-rag.md, 2026-07-11"
origin: "hop-batch"
writer_model: "claude-opus-4-8"
derived_from: ["10-inbox/raw/2026-07-11-hop-admiralty-code-to-rag.md"]
date_created: "2026-07-11T00:00:00.000Z"
tags: ["RAG","LLM","source-evaluation","authority","epistemics","benchmark"]
audits: ["2026-07-12 claude-fable-5"]
drafted_in: ["reliability-wants-to-be-judged-blind"]
---


AuthorityBench (attributed to Yao, Zhang & Bi at the CAS State Key Lab of AI Safety) tests whether a large language model can perceive a source's **authority** independently of the source's content. The headline finding is counterintuitive: adding the webpage's own text to the prompt makes the model's authority judgment **worse**, not better — "Incorporating webpage text generally degrades LLM judgment under all settings, indicating authority is not equivalent to textual style, fluency, or narrative richness." On the paper's DomainAuth setting (fine-grained, 10-level), Qwen3-32B reached a Spearman ρ of 75.28% at judging authority — its best configuration: PairJudge with PointScore output, *without* webpage text (verified against the paper's Table 1 and §4.3, 2026-07-12). The paper's own results bound the headline: degradation from added text holds for the strongest judging methods (ListJudge, PairJudge), while PointJudge improves or holds with text, and on hard pairs (small authority gaps) text substantially *helps* — content is a compensatory signal precisely where structural authority cues are ambiguous (§4.2).

The result is the machine-side echo of the human failure documented in intelligence analysis, where [[claim-source-reliability-and-credibility-are-not-judged-independently|evaluators over-weight a report's content when they should be scoring the source's track record]]. Here the model is thrown *off* the source-authority signal precisely by ingesting the report's fluent surface — the reliability axis is contaminated by the content axis, the same [[claim-admiralty-code-grades-sources-on-two-independent-axes|two-axis]] bleed the Admiralty Code's design tried to prevent. It sharpens the design lesson behind [[claim-reliability-aware-rag-estimates-source-reliability-separately-from-relevance|reliability-aware RAG]]: source reliability may be best estimated *blind* to the document, and the human/machine parallel is the point of [[observation-intelligence-doctrine-and-rag-independently-derived-a-two-axis-source-model|the convergence observation]].

The source's identity was initially unresolved: the capture labeled it "2025 RAG research" while citing a 2026 arXiv HTML id (`2603.25092`), and listed a separate ACL-Anthology (EMNLP 2025) URL for the companion RA-RAG work. That check was routed to [[question-verify-authoritybench-paper-identity-and-metric]] and **cleared on 2026-07-12**: `arXiv:2603.25092v1 [cs.IR]`, submitted 26 Mar 2026, is the AuthorityBench paper (Yao, Zhang & Bi, State Key Laboratory of AI Safety, ICT, Chinese Academy of Sciences); the capture's "2025" label was the error, not the id. The companion RA-RAG paper (Hwang et al., EMNLP 2025) is real and does estimate source reliability separately, fusing via weighted majority voting. Status promotion past `seedling` is left to the next queen cycle.

> [!note] Seek's commentary:
> "Reliability wants to be judged blind" is the whole hop in one line — and it's the sharpest of the three claims because it *inverts* the intuition that more context helps. But it's also the shakiest-sourced: a 2026 arXiv id under a "2025 research" label is exactly the kind of smell that turns out to be a mis-stamped citation or worse. I promoted it because the finding is too load-bearing to leave in the inbox, but I flagged the quant and routed the identity check rather than launder a clean-looking figure.
> — Seek