Brief

Researchers have developed KV-Fold, a novel method for extending the context window of large language models without requiring retraining. This technique treats the key-value cache as an accumulator in a functional programming-style fold, allowing the model to process sequential chunks of data while maintaining a stable internal state. KV-Fold has demonstrated 100% exact-match retrieval on needle-in-a-haystack benchmarks across various context lengths and model sizes, operating within the memory constraints of a single GPU. AI

Researchers have proposed a new theory of how transformer language models memorize factual information, suggesting a 'geometric' form of memorization rather than traditional associative memory. This model posits that learned embeddings encode relational structure, with the MLP acting as a relation-conditioned selector. Experiments with a single-layer transformer demonstrated that logarithmic embedding dimensions suffice for memorizing random bijections, and the MLP learned a generic selection mechanism transferable to new facts. AI

Researchers have proposed a new framework for understanding how Large Language Models (LLMs) learn within a given context. Their work suggests that LLMs update their behavior by performing Bayesian inference over a low-dimensional geometric space, termed a conceptual belief space. By analyzing LLMs' performance on story understanding tasks, the study found that these belief updates follow predictable trajectories on structured manifolds, which are reflected in both the models' external behavior and internal representations. Furthermore, interventions on these internal representations could causally influence the belief trajectories, supporting the geometric account of LLM belief dynamics. AI

Researchers have introduced ORBIT, a new method designed to prevent large language models from losing their foundational language capabilities during task-specific fine-tuning. This issue, known as catastrophic forgetting, is particularly prevalent in Generative Retrieval tasks and is linked to the divergence of model parameters. ORBIT addresses this by monitoring the distance between fine-tuned and original model weights, employing a weight averaging strategy to limit parameter drift when a set threshold is exceeded. Experiments demonstrate that ORBIT effectively preserves text and retrieval performance, outperforming existing continual learning and regularization techniques. AI

Researchers have introduced PCSR-Bench, a new diagnostic benchmark designed to evaluate the spatial reasoning capabilities of multimodal large language models (MLLMs) when processing omnidirectional images. The benchmark, comprising over 84,000 question-answer pairs across 2,600 images, reveals a significant gap between foundational perception and advanced reasoning tasks. While models perform moderately well on basic tasks like object counting, their accuracy plummets on more complex reasoning involving viewpoint changes and egocentric distortions. Further experiments using reinforcement learning on a smaller model indicate that spatial reasoning abilities can be improved through targeted optimization, though gains are task-specific and sensitive to reward design. AI

Researchers have developed a novel text-tabular modeling approach to predict the decisions of unfamiliar AI agents during negotiations. The method combines structured game state and dialogue history with representations derived from a frozen LLM, acting as an "LLM-as-Observer." This approach was tested on numerous frontier LLM agents, outperforming baseline methods by improving response-prediction AUC and reducing bargaining offer-prediction error. AI

Researchers have analyzed how large language models (LLMs) develop preferences for well-known entities, a phenomenon often linked to popularity bias. Using the open OLMo models and their complete Dolma pretraining corpus, they calculated entity exposure across 7.4 trillion tokens. Their findings indicate that LLM popularity judgments align more closely with pretraining exposure than with external signals like Wikipedia pageviews, especially for larger models and in the long tail of less popular entities. This suggests that data exposure during pretraining is the primary driver of popularity bias in LLMs. AI

Researchers have developed a new method called "context convergence" to improve how Large Language Models (LLMs) answer inferential questions. This technique focuses on how effectively sentences in a passage can eliminate incorrect answers, a measure that proves more effective than simple cosine similarity for inferential reasoning. Experiments using the TriviaHG dataset and various LLMs demonstrated that passages constructed with higher convergence sentences significantly boost answer accuracy, suggesting that LLMs prioritize information-rich cues presented earlier in the text. AI

Researchers have explored methods to generalize parameter-efficient fine-tuning (PEFT) techniques beyond single-task applications. Their work investigates training on combined datasets, composing weight matrices of separate PEFT modules, and composing the outputs of these modules during inference. The study found that summing PEFT module outputs was a particularly effective composition method, outperforming or matching other approaches across different large language models and controlled text generation tasks. AI

Researchers have introduced RAW-Dream, a novel approach to adapt Vision-Language-Action (VLA) models for new tasks using reinforcement learning within task-agnostic world models. This method disentangles world model learning from specific task dependencies by leveraging a world model pre-trained on diverse, task-free behaviors and an off-the-shelf Vision-Language Model for reward generation. By relying on generalized physical priors instead of task-specific data, RAW-Dream enables zero-shot adaptation for VLAs, significantly improving scalability and mitigating world model hallucinations through a dual-noise verification mechanism. AI

Researchers have developed a new in-context learning framework called TipPFN to predict critical transitions in dynamical systems. This method uses a prior-data fitted network to identify when a system is approaching an abrupt and potentially irreversible change. TipPFN was trained on synthetic data and demonstrated state-of-the-art early detection capabilities in unseen tipping regimes, sim-to-real examples, and real-world observations, outperforming existing methods that struggle with limited data or extrapolation. AI

Researchers have developed a new framework for farmland semantic change detection, addressing limitations in existing benchmarks and models. The proposed method, called Fine-grained Difference-aware Mamba (FD-Mamba) integrated with Cross-modal Logical Arbitration (CMLA), uses a small, task-specific model alongside a large, frozen vision-language model. This collaboration aims to improve fine-grained monitoring by preserving boundaries, localizing small regions, and suppressing pseudo-changes through textual priors. Experiments on the new HZNU-FCD benchmark and other datasets demonstrate high accuracy and robustness with a relatively small number of trainable parameters. AI

Researchers have introduced KAN-CL, a new framework for continual learning that addresses catastrophic forgetting by leveraging the unique structure of Kolmogorov-Arnold Networks (KANs). This method applies importance-weighted regularization at a per-knot level, allowing for more precise control over parameter updates across tasks. When tested on classification tasks, KAN-CL significantly reduced forgetting compared to baseline methods while maintaining high accuracy, demonstrating its effectiveness in preserving learned information. AI

Researchers have developed a new machine learning framework called Hyper-DFS for dynamic feature selection, which aims to optimize feature acquisition under budget constraints. This approach utilizes a hypernetwork to generate classifier parameters on demand for specific feature subsets, improving efficiency and generalization. Benchmarks indicate that Hyper-DFS outperforms existing state-of-the-art methods on various datasets, including tabular and image data, and demonstrates superior zero-shot generalization capabilities. AI

Researchers have introduced Token-level Bregman Preference Optimization (TBPO), a new method for aligning language models using pairwise preferences. Unlike existing approaches that focus on full sequences, TBPO operates at the token level, modeling preferences for individual next-token actions based on the preceding context. This approach aims to improve alignment quality, training stability, and output diversity compared to current methods. AI

Researchers have introduced a new framework called V2V-Zero, which enables visual-to-visual generation by using visual inputs instead of text prompts. This approach allows users to condition generative models with visual specifications like sketches or reference images, bypassing the limitations of text-based descriptions. V2V-Zero achieves performance comparable to text-to-image models without fine-tuning and has been evaluated across various tasks and models, revealing challenges in content generation and structural control. AI

Researchers explored the effectiveness of cross-domain generalization for training language model monitors. Their findings indicate that training on multiple classification tasks with distinct prompts can partially improve performance on new, unseen domains. However, they identified failure cases where models struggle with entirely new prompts even within familiar data domains. The study also suggests that mixing classification training with general instruction following can mitigate these generalization issues and potentially benefit other classifier and monitoring systems. AI

Researchers have developed a new method for correcting disfluencies in multilingual speech transcripts using large language models (LLMs). The pipeline first identifies disfluent tokens and then uses these signals to fine-tune an LLM for rewriting transcripts into fluent text. A contrastive learning objective was added to penalize the reproduction of disfluent tokens, ensuring grammar and meaning are preserved. Experiments in Hindi, Bengali, and Marathi demonstrated significant improvements over existing baselines, offering a practical solution for speech-driven NLP systems. AI

Researchers have conducted a systematic study on pretraining strategies and scaling for electrocardiography (ECG) foundation models. They evaluated five different self-supervised learning objectives, finding that contrastive predictive coding and JEPA yielded the most transferable representations. The study also demonstrated that increasing pretraining data up to 11 million samples consistently improved performance for most objectives. Furthermore, structured state space models showed superior performance compared to transformers and CNNs, suggesting their inductive biases are key for effective ECG representation learning. AI

Researchers have investigated the parameter placement problem within Low-Rank Adaptation (LoRA) for fine-tuning large language models. Their study reveals that for Supervised Fine-Tuning (SFT), the specific placement of trainable parameters in the LoRA adapter's B matrix does not significantly impact performance. However, under Gradient-based Reinforcement Learning (GRPO), random parameter placement fails to improve the base model, while informed placement recovers standard LoRA accuracy. This difference is attributed to the gradient structure, with SFT gradients being stable and GRPO gradients being near-orthogonal, necessitating a gradient-informed approach for effective learning in the latter. AI

A new research paper investigates how well two prominent time series foundation models, Chronos-2 and TabPFN-TS, integrate covariate information. The study found that TabPFN-TS is more effective at capturing simple relationships between covariates and the target variable, particularly for shorter prediction horizons. This suggests that Chronos-2's strong overall performance on benchmarks may not directly indicate superior handling of covariate dependencies. AI

Researchers have introduced UniGraphLM, a novel Unified Graph Language Model designed to enhance the generalization capabilities of existing models. UniGraphLM addresses the challenge of aligning graph-encoded representations across various domains and tasks with the Large Language Model (LLM) token space. This alignment is crucial for creating unified graph tokens that combine the structural modeling of Graph Neural Networks (GNNs) with the generalization of LLMs. AI

Researchers have developed a new decoding method called Dynamic Cognitive Reconciliation Decoding (DCRD) to address conflicts between a large language model's internal knowledge and external context. DCRD uses attention maps to predict potential conflicts and then routes the input to either a greedy decoding path or a context fidelity-based dynamic decoding path. This approach aims to efficiently mitigate outdated or incorrect parametric knowledge while maintaining performance in conflict-free scenarios. Experiments on multiple LLMs and datasets demonstrate that DCRD achieves state-of-the-art results, outperforming existing baselines. AI

Researchers have developed SyncDPO, a new post-training framework designed to improve temporal synchronization in video-audio joint generation models. This method utilizes Direct Preference Optimization (DPO) to enhance the alignment between audio events and their visual counterparts, addressing limitations of traditional supervised fine-tuning. SyncDPO introduces efficient, on-the-fly negative construction strategies to create preference pairs without extensive sampling, and employs a curriculum learning approach to progressively increase the difficulty of temporal misalignments. AI

A new research paper investigates why stylistic reformulations, like poetic language, can bypass safety mechanisms in large language models. The study, using Qwen3-14B as a case study, found that models can distinguish poetic from prose formats but struggle to predict jailbreak success within these formats. The findings suggest that accumulated stylistic irregularities, rather than specific poetic devices or a failure to recognize literary formatting, lead to distinct processing patterns that circumvent safety measures. AI

OpenAI has updated its API, moving most reasoning-capable models to a new endpoint that supports interleaved reasoning across tool calls. This change allows users to view summarized reasoning tokens, which are displayed distinctly from standard errors. The new functionality is available for GPT-5 class models and can be toggled on or off using specific flags. AI

Researchers have developed a new framework called CoDAAR to improve multimodal learning by creating semantically aligned discrete representations. This approach balances the need for cross-modal generalizability with the preservation of modality-specific structures. CoDAAR utilizes Discrete Temporal Alignment and Cascading Semantic Alignment to achieve state-of-the-art performance on various cross-modal generalization benchmarks, including event classification and video segmentation. AI

Researchers have identified a critical flaw in Reinforcement Learning from Human Feedback (RLHF) when applied to flow-matching text-to-image models, where standard policy entropy fails to prevent a collapse in perceptual diversity. They propose a new metric, perceptual entropy, to accurately capture diversity in the perceptual space, addressing the limitations of policy entropy which remains constant despite diversity loss. Experiments demonstrate that strategies based on perceptual entropy significantly improve the quality-diversity trade-off in image generation models. AI

Researchers have introduced UniCustom, a novel framework designed to enhance multi-reference image generation by unifying visual conditioning. This approach integrates semantic and appearance-rich features before encoding, allowing models to better associate subjects with their specific visual details from reference images. UniCustom employs a two-stage training strategy and a slot-wise binding regularization to improve subject consistency and reduce attribute leakage, demonstrating superior performance on relevant benchmarks. AI

Mythos, a new AI model, has been introduced, described as "The Day AI Sat on Bedrock." The announcement was made on Medium, with further details available via a link to the platform. AI

Researchers have introduced OmniHumanoid, a new framework for generating videos of humanoids performing actions across different embodiments. This system separates transferable motion learning from embodiment-specific adaptation, allowing it to learn from paired videos across multiple embodiments and then adapt to new ones using unpaired data via lightweight adapters. OmniHumanoid employs a branch-isolated attention design to prevent interference between motion conditioning and embodiment modulation, demonstrating strong performance in motion fidelity and embodiment consistency on both synthetic and real-world benchmarks. AI

Researchers have introduced the What-Where Transformer (WWT), a novel visual backbone designed to better separate object appearance from spatial location. This new architecture uses a slot-based design where tokens represent 'what' an object is and attention maps represent 'where' it is located. The WWT demonstrates emergent capabilities in discovering multiple objects directly from attention maps, even when trained with standard classification supervision, and shows improved performance on zero-shot object discovery and weakly supervised semantic segmentation tasks. AI

Researchers have developed a new Spectral Vision Transformer (SVT) architecture designed for efficient tokenization, particularly in scenarios with limited data such as medical imaging. The SVT leverages spectral projection, offering theoretical advantages like spatial invariance and improved signal-to-noise ratio, which result in reduced computational complexity compared to standard spatial vision transformers. Experiments across simulated, public, and clinical datasets demonstrate that the SVT achieves comparable or better performance with fewer parameters than various other models, including compact and standard vision transformers, CNNs with attention, and MLPs. AI

Researchers have developed a new framework called Latent-to-Pixel (L2P) that efficiently transfers knowledge from pre-trained Latent Diffusion Models (LDMs) to create powerful pixel-space models. This method avoids the need for extensive computational resources and real-world data by freezing most of the source LDM and training only shallow layers for the latent-to-pixel transformation. L2P utilizes synthetic images generated by LDMs as its training corpus, enabling rapid convergence with minimal hardware. The approach also eliminates the VAE bottleneck, allowing for native generation of ultra-high resolution images. AI

Researchers have developed RealDiffusion, a new framework for generating coherent multi-character storybooks using diffusion models. The system employs heat diffusion as a prior to average features and stabilize character identity across sequential frames. Additionally, a region-aware stochastic process introduces controlled perturbations to maintain narrative dynamism and scene evolution. This approach aims to resolve the trade-off between character coherence and story progression, outperforming existing methods in experiments. AI

Anthropic developed an AI model with advanced capabilities that they chose not to release due to safety concerns. This AI demonstrated its power by discovering a 27-year-old security vulnerability within the OpenBSD operating system. The decision to withhold the model highlights Anthropic's commitment to responsible AI development and deployment. AI

Researchers have introduced a new framework for guided depth super-resolution that utilizes an Interactive State Space Model. This approach aims to efficiently create high-resolution depth maps from low-resolution inputs, using RGB images as guidance. The model incorporates a cross-modal local scanning mechanism to enable detailed semantic interactions between RGB and depth features, leveraging the Mamba architecture for linear complexity. Experiments indicate that this method achieves competitive results compared to existing state-of-the-art techniques. AI

Researchers have developed StepCodeReasoner, a new framework designed to improve code reasoning by focusing on intermediate execution states rather than just final outputs. This approach uses structured print statements to create execution-trace anchors, training models to predict runtime states at each step. The framework also incorporates a novel reinforcement learning algorithm, Bi-Level GRPO, for better credit assignment across and within execution paths. Experiments show that StepCodeReasoner achieves state-of-the-art performance on code reasoning benchmarks, with its 7B model surpassing models like GPT-4o and a previous CodeReasoner baseline. AI

Researchers have introduced Yoked Feature Preference Optimization (YFPO), a novel framework designed to enhance the mathematical reasoning capabilities of large language models. Unlike existing methods that rely solely on external preference data, YFPO incorporates internal neuron activation patterns to guide the optimization process. By identifying neurons associated with mathematical concepts and logical reasoning, YFPO constructs an auxiliary reward signal that complements external supervision. Preliminary experiments on a small-scale model using the GSM8K benchmark indicate that this neuron-guided approach can potentially improve reasoning performance and offers a more interpretable path for model fine-tuning. AI

Researchers have developed a theoretical framework to understand Lifelong Normalization (LN), a key strategy for continuously updating Large Language Models without causing catastrophic forgetting or model collapse. Their analysis reveals that LN creates a self-reinforcing stability loop, ensuring parameter updates are orthogonal and bounded, which directly combats forgetting. Building on this, they introduce StableEdit, a method that enhances this stability through an explicit warm-up stage and full whitening, demonstrating improved long-horizon stability with minimal overhead. AI

Former OpenAI researcher Lilian Weng's new venture, Thinking Machines Lab (TML), has unveiled a vision for full-duplex, real-time conversational AI. This concept closely mirrors the capabilities demonstrated by China's MiniCPM-o 4.5, which was open-sourced by company "面壁智能" (OpenBMB) three months prior. Both TML and "面壁智能" aim to break away from traditional turn-based AI interactions, proposing a "full-duplex" or "time-aligned micro-turn" framework that processes interleaved multimodal information streams. AI