Infoblock · public evidence graph · transformer

Softmax self-attention (the Transformer's core operation) is exactly ONE update step of a continuous modern Hopfield network with an exponential energy — i.e. associative-memory attractor retrieval — with pattern-storage capacity scaling ~2^(d/2) and the softmax inverse-temperature beta as the retrieval-sharpness knob.

✓ VERIFIED

Grade E2 · state: verified · id: IB-attention-hopfield

Quantities: attention = 1 modern-Hopfield update; pattern-storage capacity ~2^(d/2); knob = softmax inverse-temperature beta

Primary source

Ramsauer et al. 2020, ICLR — 'Hopfield Networks is All You Need'

Edges (outgoing)

• relates → Iterated serial reproduction behaves as a Markov chain whose STATIONAR — the two domains where the attractor is mathematically exact (Hopfield fixed point ~ Markov stationary / shared prior)

Incoming edges (what points here)

• Iterated serial reproduction behaves as a Markov chain whose STATIONAR — relates → this (the two domains where 'attractor' is mathematically EXACT: Markov stationary here, Hopfield fixed point there)