Attention Pollution Theory

An LLM's inference capability depends on the purity of attention. When attention is drawn to surface structures, the model cannot perform abstract inference and drifts into surface-mode behavior.

AIKernel's architecture is designed to prevent this pollution at the OS level.

1. What is Attention Pollution?

Attention pollution occurs when an LLM's attention resources are consumed by surface information rather than abstract information needed for inference.

When polluted:

• Abstract structure fails to receive attention
• Purpose weakens and inference direction is lost
• Chain-of-thought becomes shallow
• Reasoning depth disappears

The model stops "thinking" and starts "mimicking."

2. A "Gravity" Model for Attention

Surface structures (examples, templates, style) have higher token density and stronger attention "gravity" than abstract structures.

Why surface structures have strong gravity

• Tokens are dense
• Patterns are clear and easy for the model to latch onto
• Formal features strongly influence attention

Abstract structures have weaker gravity

• Lower token density
• Abstract concepts are weaker in attention competition
• Easily buried by surface structures

Result

The model is pulled toward surface structures and drops out of abstract inference.

AIKernel's goal is to provide the "escape velocity" to keep inference at the core.

3. Attention SNR (Signal-to-Noise Ratio)

Define attention purity as:

SNR = Inference Signal / Surface Noise

Signal (Inference)

• purpose
• constraints
• structure
• reasoning patterns

Noise (Surface)

• examples
• stylistic instructions
• RAG fragments
• mixed history
• irrelevant information

When SNR drops:

• Inference becomes shallow
• Surface-mode is triggered
• Hallucinations increase

AIKernel maximizes SNR by strengthening Signal and isolating Noise.

4. Pollution Factors

Main attention polluters:

• examples (samples)
• stylistic mimicry instructions
• RAG fragments
• mixed history
• noisy information

These are surface structures distinct from abstract inference information.

5. Failure Modes Caused by Pollution

Attention pollution induces the following failure modes.

5.1 Inference halt

• structure fails to get attention
• purpose loses influence
• chain-of-thought collapses

5.2 Surface-mode

Characteristics:

• style-only mimicry
• no inference
• no structural understanding
• loss of purpose

5.3 Increased hallucination

• Mixing RAG and context breaks factuality
• Attention cannot determine what is factual

5.4 Loss of purpose

• Purpose fails to ride attention; the model cannot understand what to achieve

6. Scope of the Theory

— Not solved by "smarter" models

Attention pollution is independent of model size or intelligence. It is a structural problem inherent to token-based architectures.

Reasons:

• Attention is a finite resource
• High token density surface structures always exert strong gravity
• Signal vs Noise competition does not vanish with larger models

Therefore, attention pollution is an information-structure problem, not a model capability problem.

AIKernel addresses it with OS-level structure.

7. Preventive Measures (AIKernel Architecture)

From the uploaded documents, the preventive measures are:

7.1 Category separation

• Separate purpose / constraints / structure / expression / rag_material / history
• Forbid mixing

7.2 Isolate examples (ExpressionContext)

• Keep examples out of inference; use them only for output shaping

7.3 Materialize RAG (MaterialContext)

• Do not pass raw RAG into inference; structure and abstract it first

7.4 Separate inference and expression

• OrchestrationContext (inference)
• ExpressionContext (expression)
• MaterialContext (material)

This three-way separation prevents attention pollution structurally.

8. Positioning within AIKernel

Attention Pollution Theory underpins AIKernel's core principles:

• category separation
• context isolation
• surface-mode prevention
• preprocessing-first design
• deterministic replay

AIKernel is an OS-level structure to preserve attention purity.

9. Related Documents

• Principles of Information Category Separation
• Context Isolation Specification
• Risks of Surface-Mode Failure
• Preprocessing vs Prompting
• AIKernel vs LangChain — Structural Comparison

Changelog

• v0.0.0 / v0.0.0.0: Initial draft
• v0.0.1 (2026-05-06): Version upgrade aligned with documentation guidelines