Researchers have developed a new method using sparse crosscoders to track the emergence and consolidation of linguistic features within large language models during pretraining. This technique, which includes a novel metric called Relative Indirect Effects (RelIE), helps identify when specific capabilities become causally important for task performance. The approach is architecture-agnostic and scalable, offering a more interpretable way to analyze representation learning in LLMs. Separately, another study explores the use of LLMs to detect language ideologies in Luxembourgish news comments, a small language with limited representation in training data. The research investigates whether machine translation to high-resource languages improves LLM performance on this task, suggesting LLMs can be practical tools for identifying ideological content despite current optimization limitations. AI
Summary written by gemini-2.5-flash-lite from 4 sources. How we write summaries →
IMPACT Provides new methods for understanding LLM internal representations and explores LLM utility for sociolinguistic analysis.
RANK_REASON This cluster contains two academic papers published on arXiv, one detailing a new method for analyzing LLM pretraining and another exploring LLM applications in sociolinguistics.