Distributed output templates, not single positions, drive LLM in-context learning

Researchers have demonstrated that in-context learning in large language models is driven by distributed output templates rather than single-position activations. Through multi-position intervention, they achieved up to 96% task transfer, pinpointing layer 8 as a causal locus for in-context learning task identity. This finding holds across multiple model architectures, suggesting a universal intervention window around 30% network depth. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Reveals that in-context learning relies on distributed output templates, not single positions, potentially impacting how models are trained and prompted.

RANK_REASON Academic paper detailing new findings on in-context learning mechanisms in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Llama-3.2-3B
• Qwen
• Gemma

• paper
• other