---
type: "GlossaryTerm"
title: "Superposition"
description: "Superposition is an architectural principle in neural language models: multiple semantic concepts share a common dimension in latent space, similar to the overl"
resource: "https://www.contextstudios.ai/glossary/superposition"
language: "en"
tags: ["infrastructure"]
generated:
  by: "process:contextstudios-md/1"
  at: "2026-10-08T20:45:16.046Z"
status: "stable"
---

# Superposition

Superposition is an architectural principle in neural language models: multiple semantic concepts share a common dimension in latent space, similar to the overlay of quantum states. The reconstruction error per dimension falls only with the inverse of width, i.e. ∼1/Width. Doubling the embedding width halves the error, but compute load and memory demand grow disproportionately. At a width of 768, the error typically sits around 8 %; at 1,536 dimensions it drops to roughly 4 %. Beyond a model-specific threshold, further doubling brings little gain; a smaller model with more test-time compute can match it. Superposition thus differs from a plain embedding: there one axis encodes a single concept, here multiple meanings overlap on the same position.