Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

A paraphrase pair is a small piece of evidence that two activations should agree. We treat that as a sheaf condition. A cellular sheaf over a paraphrase graph defines a coboundary δ⁰ whose kernel holds activations that paste consistently across the graph, and the rest of the Laplacian spectrum reads off how badly they fail to. On an acyclic matching graph the construction collapses to projected within-pair covariance. Cycles change that. Stacking L layers into a grid yields b₁ = N(L−1) algebraic 1-cycles, and a multi-paraphrase ring per fact yields b₁ = N.

The multi-layer grid produces a model-derived Hodge harmonic mass varying four orders of magnitude across nine architectures and ranking the same models as their steering fragility (Spearman ρ = 0.94, n = 6, exact permutation p = 0.008). The ring produces a cycle-aware H⁰ that beats pair-only H⁰ on every one of eight architectures by a mean of +6.5 pp, a gain within-pair covariance cannot reach. Across nine models from 124M to 13B parameters, the spectral complement of L_F carries 5.6 to 26.5× the causal influence of variance-matched random subspaces (p < 10⁻¹⁵). On held-out CounterFact, sheaf H⁰ at 20 dimensions beats LEACE's preserved subspace at full hidden dimension by a mean +17.1 pp, and swapping the joint-PCA restriction map for random or identity projections barely moves the number. The kernel selection drives the result, not the projection.

On the steering-fragile Llama-2-7B, contrastive sheaf steering preserves 31.0% of facts against 4.2% for random perturbations (n = 1000, McNemar p < 10⁻⁵⁰). The same operators recover collapse prevention: tr(L_F) − β log det Cov is VICReg in coordinates, holding encoder rank at 57–60/64 where naive consistency collapses to 14–42/64.