Sigma-Algebras: a Principled Approach to Asking Good Questions

Kennon Stewart

Reading

Sigma-Algebras: a Principled Approach to Asking Good Questions

Published: 9/18/2025•Authors: Kennon Stewart

Cite this work

Show citation formats

APA

Kennon Stewart (2025). Sigma-Algebras: a Principled Approach to Asking Good Questions.

BibTeX

@inproceedings{sigma_algebras_2025,
  title={Sigma-Algebras: a Principled Approach to Asking Good Questions},
  author={Kennon Stewart},
  year={2025},
}

📥 Download BibTeX 📥 Download APA

Why This Matters to the Lab

In our work, we constantly deal with uncertainty. We build models to predict customer behavior, forecast demand, and find patterns in noisy data. But have you ever wondered what makes the math of probability actually work? How can we be sure that our calculations are logically sound, especially when dealing with complex, real-world events?

The answer lies in a concept called the $\sigma$ -algebra (sigma-algebra). Think of it as the official rulebook for probability. It defines the set of “reasonable questions” we’re allowed to ask about our data. Without this framework, we could easily fall into mathematical paradoxes, and the probabilistic foundations of our most advanced models would be built on shaky ground. It’s the hidden scaffolding that ensures consistency, whether we’re modeling traffic flow across Detroit or the flow of information into a neural network.

What Is a Sigma-Algebra? A Principled Approach to Asking Questions

Let’s start with a simple experiment: flipping a coin twice. The set of all possible outcomes, called the sample space ( $\Omega$ ), is {HH, HT, TH, TT}. We might want to ask about the probability of an “event,” like “getting at least one head,” which corresponds to the subset {HH, HT, TH}.

A $\sigma$ -algebra is simply the collection of all such events that we want to be able to measure. For this collection to be useful and consistent, it must follow three common-sense rules:

Certainty is measurable: The event “one of the outcomes happens” (i.e., the entire sample space {HH, HT, TH, TT}) must be in our collection. Its probability is 1.
Opposites are measurable: If we can ask for the probability of an event, we must be able to ask for the probability of it not happening. If “at least one head” is a valid event, its opposite, “no heads” ({TT}), must also be a valid event. This is called being closed under complements.
Combinations are measurable: If we have a list of measurable events, the event that “at least one of them occurs” must also be measurable. If “getting exactly one head” ({HT, TH}) and “getting two heads” ({HH}) are measurable, their combination, “getting at least one head,” must also be. This is called being closed under countable unions.

That’s it. A collection of events satisfying these three rules is a $\sigma$ -algebra. It’s the “well-behaved” set of questions we can ask, for which probability theory guarantees a consistent answer.

A Very Special Case: Data Over Time (Stochastic Processes)

This framework truly shines when we analyze data that arrives sequentially, like stock prices, sensor readings, or user interactions on a website. This is the domain of stochastic processes.

Imagine you are monitoring a system in real-time. As time passes, you gain more information. We can model this accumulation of knowledge using a filtration, which is nothing more than a sequence of nested $\sigma$ -algebras.

Let’s call the $\sigma$ -algebra representing our knowledge at time $t$ as $\mathcal{F}_t$ .

$\mathcal{F}_0$ represents our knowledge at the start (maybe just the initial state).
$\mathcal{F}_1$ represents all the events we can determine after one minute. It includes everything in $\mathcal{F}_0$ plus new information.
$\mathcal{F}_2$ represents our knowledge after two minutes, containing everything in $\mathcal{F}_1$ and so on.

This sequence, $\mathcal{F}_0 \subseteq \mathcal{F}_1 \subseteq \mathcal{F}_2 \subseteq \dots$ , is the filtration. When we say a process is adapted to a filtration, it’s a mathematically precise way of saying that at any point in time, the process’s value is knowable based only on the information gathered so far. This concept is fundamental for:

Streaming algorithms: Ensuring our algorithm doesn’t “peek into the future.”
Reinforcement learning: Modeling the information an agent has at each step before it makes a decision.
Financial modeling: Defining fair games and pricing derivatives based on the flow of market information.

Why Machine Learning Cares

The $\sigma$ -algebra isn’t just an abstract curiosity; it’s deeply embedded in the principles of machine learning.

Defining Random Variables: In ML, a feature (like ‘age’ or ‘income’) is a random variable. Formally, a random variable is a function that maps outcomes to numbers. The $\sigma$ -algebra guarantees that we can ask probabilistic questions about these features, like “What is the probability that income is over $100k?”
Probabilistic Models: Models like Bayesian networks, Gaussian processes, and VAEs are built on the laws of probability. The $\sigma$ -algebra is the underlying structure that ensures the probability distributions these models learn are mathematically sound.
Controlling Information: In fields like fairness and privacy, we consciously decide what information a model should or should not have access to. Defining the events we choose to measure is equivalent to constructing a specific $\sigma$ -algebra, giving us a powerful tool to control information flow.

Summary

A $\sigma$ -algebra defines the collection of events we can meaningfully measure and assign a probability to.

It ensures logical consistency by being closed under complements (if we can measure A, we can measure not-A) and countable unions (if we can measure several events, we can measure their combination).

In machine learning, it provides the foundation for defining features, building probabilistic models, and

crucially for modern applications—modeling the flow of information over time using filtrations.