How Do LLMs Cite?
Properties
# How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
This paper from University of Amsterdam investigate Llama-3.1-8B-Instruct to figure out how it actually decides whether to attach an inline citation when answering a factoid question in a RAG setup. The authors create a clean controlled experiment by taking PopQA questions and running each one twice, once with a supporting document and once with a structure-matched distractor. Then they use activation patching to see which internal components flip the model’s preference between starting a citation (”[”) and just ending the sentence. Rather than a single “citation head,” they find a distributed and fragile “attributional ensemble” of attention heads and MLPs that operates through conjunctive, AND-like checks, meaning it’s easy to break by corrupting any of several sites but hard to induce with a single patch. The mechanism runs in three stages: an early MLP enriches the queried entity where it first appears in the document and marks it as grounded, a mid-layer processing hub (roughly layers 10-20) propagates relevance, and late layers aggregate everything at the answer token to push the citation marker over the termination token. Their core finding is that citation decisions lean on shallow entity co-reference matching rather than genuine use of document content, and the answer representation itself isn’t a decisive driver, so an inline citation can be generated by a heuristic running in parallel to answer construction. So, a citation being present tells little about whether the model actually used the source, which is a problem for RAG citations in high-stakes settings.
