Natural Transformations

Overview & Motivation

In Categories & Functors we built the language of categories (objects and morphisms) and functors (structure-preserving maps between categories). We can now ask: what is a morphism between functors?

The answer — a natural transformation — is the concept that Eilenberg and Mac Lane originally invented category theory to define. The term “natural” in mathematics (natural isomorphism, natural map, canonical construction) had been used informally for decades before 1945, when Eilenberg and Mac Lane gave it a precise meaning. A natural transformation is a family of morphisms, one for each object of the source category, that commutes with every morphism in the source. The commutativity condition — the naturality square — is what distinguishes canonical constructions from arbitrary choices.

The central example: every finite-dimensional vector space $V$ is isomorphic to its double dual $V^{**}$, and this isomorphism is natural — it requires no choice of basis. The embedding $v \mapsto (\varphi \mapsto \varphi(v))$ is defined uniformly for all $V$, and it commutes with linear maps. By contrast, $V$ is also isomorphic to its dual $V^*$ when $\dim V < \infty$, but this isomorphism requires choosing a basis — it is not natural.

Why this matters for ML:

• Equivariance is naturality. A CNN’s translation equivariance, a GNN’s permutation equivariance, and a spherical CNN’s rotation equivariance are all instances of the naturality condition. Weight sharing and symmetric aggregation are the mechanisms that enforce naturality.
• The Yoneda lemma — the deepest result we develop here — says that an object is completely determined by its morphisms to all other objects. This is the categorical version of the idea behind distributional semantics, word embeddings, and attention mechanisms: “you shall know a word by the company it keeps.”
• Entropy is natural. Shannon entropy defines a natural transformation from probability distributions to the reals. The data processing inequality — entropy cannot increase under deterministic transformations — is a direct consequence of naturality.

What we cover:

1. Natural transformations — the definition, the naturality square, and first examples.
2. A gallery of natural transformations — determinant, double dual, abelianization, trace, entropy.
3. Composition — vertical, horizontal, whiskering, and the interchange law.
4. Functor categories — the category $[\mathcal{C}, \mathcal{D}]$ whose objects are functors and whose morphisms are natural transformations.
5. The Yoneda lemma — $\mathrm{Nat}(\mathrm{Hom}(A, -), F) \cong F(A)$, the deepest result in basic category theory.
6. The Yoneda embedding and presheaves — “an object is determined by its relationships.”
7. Equivariance as naturality — the categorical perspective on symmetric neural networks.
8. Computational notes — verification in Python.

---

Natural Transformations: Morphisms Between Functors

Given two functors $F, G: \mathcal{C} \to \mathcal{D}$ between the same pair of categories, a natural transformation $\alpha: F \Rightarrow G$ is a family of morphisms in $\mathcal{D}$ — one for each object of $\mathcal{C}$ — that is “compatible” with the structure of $\mathcal{C}$. Compatibility means that for every morphism $f: A \to B$ in $\mathcal{C}$, the following square commutes:

$F(A) \xrightarrow{\alpha_A} G(A) \xrightarrow{G(f)} G(B)$

$F(A) \xrightarrow{F(f)} F(B) \xrightarrow{\alpha_B} G(B)$

The commutativity means $G(f) \circ \alpha_A = \alpha_B \circ F(f)$. There are two paths from $F(A)$ to $G(B)$: go right then down ($G(f) \circ \alpha_A$), or go down then right ($\alpha_B \circ F(f)$). Naturality says both paths give the same morphism.

The naturality condition is the key. It says that the transformation $\alpha$ is “uniform” across all objects — the component at $B$ is determined by the component at $A$ in a way that respects the morphism $f$. We can think of this as a consistency condition: if we have two ways of transforming $F(A)$ into $G(B)$ (via two different paths around the naturality square), they must agree.

Preset:Double Dual: Id ⇒ (-)** in VecDeterminant: GL_n ⇒ (-)×Trace: End(-) ⇒ kCustomMorphism:TAnimate

Functor FFunctor GComponents αPath: G(f) ∘ α_APath: α_B ∘ F(f)

Naturality Verification

Double dual: Id ⇒ (-)** in Vec

Top-then-right

G(f) ∘ α_A = T**∘η_V

Down-then-across

α_B ∘ F(f) = T**∘η_V

✓Square commutes — naturality holds

Full naturality check: ✓ Valid for all morphisms

---

A Gallery of Natural Transformations

Natural transformations appear everywhere once we know what to look for. Here are the most important examples, starting with the one that motivated the entire theory.

The double dual embedding $\eta: \mathrm{Id}_{\mathbf{Vec}} \Rightarrow (-)^{**}$. For each vector space $V$, the component $\eta_V: V \to V^{**}$ sends $v$ to the evaluation functional $\hat{v}: V^* \to k$ defined by $\hat{v}(\varphi) = \varphi(v)$. This construction is basis-free — we never chose a basis for $V$. For any linear map $T: V \to W$, the naturality condition $T^{**} \circ \eta_V = \eta_W \circ T$ holds: both sides send $v \in V$ to the functional on $W^*$ that evaluates at $T(v)$.

The determinant $\det: GL_n \Rightarrow (-)^\times$. The functor $GL_n$ sends a ring $R$ to the group $GL_n(R)$ of invertible $n \times n$ matrices over $R$, and the functor $(-)^\times$ sends $R$ to its group of units $R^\times$. The determinant is a natural transformation: for any ring homomorphism $\phi: R \to S$, we have $\phi^\times \circ \det_R = \det_S \circ GL_n(\phi)$. In words: applying the ring homomorphism entrywise and then taking the determinant gives the same result as taking the determinant first and then applying $\phi$.

The trace $\mathrm{tr}: \mathrm{End}(-) \Rightarrow k$. The functor $\mathrm{End}$ sends a vector space $V$ to its endomorphism ring $\mathrm{End}(V) = \mathrm{Hom}(V, V)$, and $k$ is the constant functor to the ground field. The trace map is natural: $\mathrm{tr}(TMT^{-1}) = \mathrm{tr}(M)$ for any invertible $T$. Naturality here is the statement that the trace is invariant under change of basis — it depends only on the endomorphism, not on how we represent it.

Abelianization $\pi: \mathrm{Id}_{\mathbf{Grp}} \Rightarrow (-)^{ab}$. The functor $(-)^{ab}$ sends a group $G$ to its abelianization $G/[G, G]$. The component $\pi_G: G \to G^{ab}$ is the quotient map. For any group homomorphism $\phi: G \to H$, naturality says that abelianizing then mapping is the same as mapping then abelianizing — because $\phi$ sends commutators to commutators.

Entropy as a natural transformation. Shannon entropy $H: \Delta \Rightarrow \mathbb{R}$ defines a natural transformation from the probability distribution functor $\Delta$ (which sends a finite set $X$ to the set $\Delta(X)$ of probability distributions on $X$) to the constant functor $\mathbb{R}$. Naturality says: for any function $f: X \to Y$, $H(f_*p) \leq H(p)$, where $f_*p$ is the pushforward distribution. This is the data processing inequality — a consequence of naturality.

---

Composition of Natural Transformations

Natural transformations compose in two fundamentally different ways: vertically (stacking transformations end-to-end between functors in a chain $F \Rightarrow G \Rightarrow H$) and horizontally (composing transformations side by side between functor compositions). These two operations interact via the interchange law.

The interchange law has a deep structural consequence: it gives the 2-category $\mathbf{Cat}$ a well-defined notion of “composition of 2-cells” (natural transformations) that is consistent in both directions. This is the starting point of higher category theory.

---

Functor Categories

Vertical composition with identity natural transformations gives us everything we need to form a category whose objects are functors and whose morphisms are natural transformations.

Propositions 1 and 2 guarantee that this is indeed a category: vertical composition is associative and identity natural transformations are neutral.

Source category:Two objectsThree objects (triangle)Functors:23

Source category C

Functor category [C, D]

Click a natural transformation edge to see component details

---

The Yoneda Lemma

The Yoneda lemma is the deepest result in basic category theory. It says that a natural transformation from a representable functor $\mathrm{Hom}(A, -)$ to any functor $F$ is completely determined by a single element of $F(A)$ — the image of the identity morphism $\mathrm{id}_A$.

The intuition is this: if we know what $\alpha$ does to $\mathrm{id}_A \in \mathrm{Hom}(A, A)$, then naturality forces the value of $\alpha$ on every other morphism. For any $f: A \to B$, the naturality condition $F(f) \circ \alpha_A = \alpha_B \circ \mathrm{Hom}(A, f)$ applied to $\mathrm{id}_A$ gives:

$\alpha_B(f) = \alpha_B(\mathrm{Hom}(A, f)(\mathrm{id}_A)) = F(f)(\alpha_A(\mathrm{id}_A))$

So $\alpha_B(f) = F(f)(x)$ where $x = \alpha_A(\mathrm{id}_A) \in F(A)$. One element determines everything.

Yoneda Lemma Explorer

Reverse direction →

Category C

Hom(A, –)

Hom(A, A)

id_A

Hom(A, B)

f

Hom(A, C)

g∘f

F(–)

F(A)

a1a2

F(B)

b1b2b3

F(C)

c1c2

Pick x ∈ F(A):

a1a2

Induced natural transformation

Pick an element x ∈ F(A) below to see the Yoneda bijection in action.

---

The Yoneda Embedding and Presheaves

The Yoneda lemma has an immediate corollary that is one of the most powerful tools in category theory.

The Yoneda embedding says: an object is completely determined by its relationships to all other objects. Two objects $A$ and $B$ are isomorphic if and only if $\mathrm{Hom}(-, A) \cong \mathrm{Hom}(-, B)$ as functors — if and only if they “look the same from the outside.”

---

Equivariance as Naturality

Here is the payoff for readers who have been following both the Category Theory and Graph Theory tracks. The property of equivariance — the requirement that a function commute with a group action — is precisely the naturality condition.

A group $G$ defines a one-object category $BG$ whose single object we call $*$ and whose morphisms are the elements of $G$, with composition given by the group operation. A group action of $G$ on a set $X$ is a functor $\rho: BG \to \mathbf{Set}$ with $\rho(*) = X$ and $\rho(g): X \to X$ for each $g \in G$.

A function $f: X \to Y$ between two $G$-sets is $G$-equivariant if

$f(\rho_X(g)(x)) = \rho_Y(g)(f(x)) \qquad \text{for all } g \in G, \; x \in X$

This is exactly the naturality condition for a natural transformation between the functors $\rho_X, \rho_Y: BG \to \mathbf{Set}$! The naturality square at the morphism $g$ is:

$f \circ \rho_X(g) = \rho_Y(g) \circ f$

Architecture:CNN (translation)GNN (permutation)Shift:0 px1 px2 px3 px4 pxVerify

Naturality Square

X ──ρ_X(g)──▶ X │ │ f f │ │ ▼ ▼ Y ──ρ_Y(g)──▶ Y

ρ_X(g) = shift input · ρ_Y(g) = shift output · f = convolution

f(ρ_X(g)(x)) = ρ_Y(g)(f(x))

---

Computational Notes

Here we verify the key examples from this topic in Python, making the abstract mathematics concrete.

Naturality of the double dual embedding. For a linear map $T: \mathbb{R}^2 \to \mathbb{R}^3$, we verify $T^{**} \circ \eta_V = \eta_W \circ T$. In finite dimensions with a chosen basis, the double dual embedding is the identity (since $V \cong V^{**}$ canonically), so both paths reduce to $T$:

import numpy as np

T = np.array([[1, 2], [3, 4], [5, 6]])  # T: R^2 -> R^3
v = np.array([1.0, 0.0])

left_path  = T @ v   # T**(eta_V(v)) = T(v) in coordinates
right_path = T @ v   # eta_W(T(v))   = T(v) in coordinates

print(f"Left path  (T** ∘ η_V)(v) = {left_path}")
print(f"Right path (η_W ∘ T)(v)   = {right_path}")
print(f"Naturality holds: {np.allclose(left_path, right_path)}")

The trace as a natural transformation. Naturality means $\mathrm{tr}(TMT^{-1}) = \mathrm{tr}(M)$ — the trace is invariant under conjugation:

M = np.array([[1.0, 2.0], [3.0, 4.0]])
T_inv = np.array([[0.5, -0.5], [1.0, 0.5]])
T_mat = np.linalg.inv(T_inv)

conjugated = T_mat @ M @ T_inv
print(f"tr(M)          = {np.trace(M):.4f}")
print(f"tr(T M T^(-1)) = {np.trace(conjugated):.4f}")
print(f"Equal: {np.isclose(np.trace(M), np.trace(conjugated))}")

Entropy and the data processing inequality. Shannon entropy defines a natural transformation, and naturality gives us $H(f_*p) \leq H(p)$:

from scipy.stats import entropy as scipy_entropy

def entropy_bits(p):
    return scipy_entropy(p, base=2)

def pushforward(p, f_map, target_labels):
    q = {label: 0.0 for label in target_labels}
    for i, pi in enumerate(p):
        q[f_map[i]] += pi
    return np.array([q[label] for label in target_labels])

p = np.array([0.5, 0.3, 0.2])
f_map = {0: "a", 1: "b", 2: "a"}  # f merges elements 0 and 2
f_star_p = pushforward(p, f_map, ["a", "b"])

print(f"H(p)     = {entropy_bits(p):.4f} bits")
print(f"H(f_*p)  = {entropy_bits(f_star_p):.4f} bits")
print(f"H(f_*p) ≤ H(p): {entropy_bits(f_star_p) <= entropy_bits(p) + 1e-10}")

Vertical and horizontal composition. The interchange law $(δ ∘ γ) * (β ∘ α) = (δ * β) ∘ (γ * α)$ can be verified componentwise on small examples. See the companion notebook for full implementations.

---

Connections & Further Reading

Where this fits

Natural transformations are the second topic in the Category Theory track and the conceptual bridge between the static structure of categories/functors and the dynamic structure of adjunctions and monads:

• Categories & Functors — the direct prerequisite. All definitions (categories, functors, morphisms, composition, Hom sets, opposite categories) are assumed.

• Adjunctions — formalizes the unit-counit pairs as natural transformations satisfying the triangle identities, with the free-forgetful paradigm as the primary example. The Hom-set definition requires naturality in both variables, and the Yoneda lemma underlies the uniqueness-of-adjoints proof.

• Monads & Comonads — uses natural transformations as the defining data: the unit $\eta: \mathrm{Id} \Rightarrow T$ and multiplication $\mu: T^2 \Rightarrow T$ are natural transformations whose commutative diagrams encode the monad laws. A monad is a monoid in the functor category $[\mathcal{C}, \mathcal{C}]$ — whose morphisms are natural transformations.

Cross-track connections

• Shannon Entropy & Mutual Information — entropy as a natural transformation from the probability distribution functor to the reals; the data processing inequality as a consequence of naturality.

• Message Passing & GNNs — permutation equivariance of message passing layers is precisely the naturality condition for the symmetric group action on node features.

• The Spectral Theorem — the double dual embedding and the trace are natural transformations in Vec, the category where the spectral theorem lives.

• Measure-Theoretic Probability — the Dirac delta embedding $\delta: X \to P(X)$ is a natural transformation from the identity functor to the probability measure functor, forming the unit of the Giry monad.

• Smooth Manifolds — the de Rham theorem establishes a natural isomorphism between de Rham cohomology and singular cohomology.

Notation summary

Symbol	Meaning
$\alpha: F \Rightarrow G$	Natural transformation from $F$ to $G$
$\alpha_A: F(A) \to G(A)$	Component of $\alpha$ at object $A$
$\beta \circ \alpha$	Vertical composition
$\beta * \alpha$	Horizontal composition
$\alpha H$	Right whiskering (pre-compose with $H$)
$K \alpha$	Left whiskering (post-compose with $K$)
$\mathrm{Nat}(F, G)$	Set of natural transformations from $F$ to $G$
$[\mathcal{C}, \mathcal{D}]$	Functor category
$F \cong G$	Natural isomorphism
$\mathsf{y}$	Yoneda embedding
$\mathrm{Hom}(A, -)$	Covariant representable functor
$\mathrm{Hom}(-, A)$	Contravariant representable functor
$\widehat{\mathcal{C}}$	Presheaf category $[\mathcal{C}^{\mathrm{op}}, \mathbf{Set}]$
$V^{**}$	Double dual of $V$
$\rho_X(g)$	Group action of $g$ on $X$
$BG$	One-object category associated to group $G$
$\eta: \mathrm{Id} \Rightarrow T$	Unit of a monad (preview)