Research Archive

Deep learning models learn internal representations of the world by progressively compressing nuisance variation while preserving task-relevant structure. We study this process through the lens of statistical sufficiency and group actions. Given a compact group acting on the sample space, we show that when the likelihood or conditional target law is invariant along group orbits, the orbit quotient X/G is not only an invariant representation but a sufficient statistic for the underlying parameters; under a likelihood-ratio separation condition it is minimally sufficient. This connects classical sufficiency theory to geometric factorization of nuisance symmetries. We then specialize to compact Lie groups and homogeneous spaces, where Peter-Weyl theory provides harmonic coordinates. For finite-band harmonic exponential families, we prove that empirical generalized Fourier coefficients form minimally sufficient statistics under standard full-rank conditions. Using Clebsch-Gordan decomposition, we express the normalization constant of these families as the trivial-irreducible projection of an exponential tensor algebra, yielding an algebraic view of the partition function. Finally, we extend minimal sufficiency from single tasks to families of tasks by characterizing a task-complete representation as the quotient by the intersection of task-specific invariance subgroups. This framework links invariant and equivariant deep learning, harmonic analysis, and statistical sufficiency, and offers a structural perspective on representation minimality, AI safety, and interpretability.

In this work, we introduce a localized, MDL-based divergence measure that quantifies the structural complexity of induced subgraphs relative to a global reference model. The measure compares the compressibility of local neighborhoods under globally fitted and locally optimized comparator models while penalizing model flexibility, yielding a statistically grounded notion of local structural surprise. The results show that the measure converges under controlled ablations, is robust to sampling choices, and detects meaningful structural irregularities that are invariant to geometric embedding. This framework generalizes MDL-based network analysis to arbitrary information graphs and provides a principled bridge between global structure and agent-level experience.

Continuing our Foundations series, we explore sigma-algebras—the mathematical scaffolding that makes probability rigorous and connects abstract theory to real-world randomness in our models.