Brief

The author details their process of building word embeddings from scratch, using Dostoevsky's novels as a corpus of nearly one million words. This step follows their previous work on character-level tokenization and aims to represent words as dense vectors that capture semantic relationships, moving beyond simple frequency counts. The article explains the mathematical concepts behind embeddings and highlights the limitations of earlier NLP models like one-hot encodings, which struggled with semantic understanding and data sparsity. AI

A new paper from Formation Research introduces the concept of "secret loyalties" in frontier AI models, where a model is intentionally manipulated to advance a specific actor's interests without disclosure. The research highlights that such secret loyalties could be activated broadly or narrowly, and could influence a wide range of actions. The paper argues that current AI safety infrastructure, including data monitoring and behavioral evaluations, is insufficient to detect these sophisticated, covert manipulations, which can be strengthened by splitting poisoning across training stages. AI

This article provides a beginner-friendly explanation of how Large Language Models (LLMs) function, focusing on their internal processes without complex mathematics. It details how LLMs handle context, predict subsequent tokens, and generate outputs. The piece aims to help users understand why their prompts might not yield the desired results. AI

A developer is pivoting their tool, Mnemara, from injecting state mid-conversation to a "Turn 0" strategy, placing all critical information in the initial system prompt. This approach leverages the primacy bias of LLMs, ensuring smaller models like Llama 3 and Mistral can consistently access and utilize injected state. The revised architecture aims to make the tool model-agnostic, improving reliability across different model tiers by establishing a clear source of truth at the beginning of the context window. AI

This article explores three distinct approaches for integrating large language models (LLMs) with external systems: MCP, tool use, and function calling. It aims to clarify the differences between these architectures and how they address the challenge of connecting LLMs to the broader digital ecosystem. The guide provides insights into the underlying mechanisms and potential applications of each integration method. AI

Researchers have developed a new semi-supervised deep learning framework for credit card fraud detection, addressing challenges with large datasets and irregular transaction data. The system integrates Generative Adversarial Networks (GANs) for data augmentation, Bayesian inference for uncertainty quantification, and log-signatures for robust feature encoding. Evaluated on the BankSim dataset, the approach demonstrated improved performance over benchmarks, particularly in scenarios with limited labeled data, highlighting the value of uncertainty-aware predictions in financial time series classification. AI

Researchers have introduced a new theoretical framework for approximating probability distributions using a finite set of points. Instead of attempting to globally minimize the maximum mean discrepancy (MMD), which is computationally challenging due to non-convexity, the study focuses on identifying and computing "stationary points" of the MMD. The paper demonstrates that these stationary points offer a faster convergence rate for numerical integration errors than the MMD itself, a phenomenon termed "super-convergence." AI

Researchers at the University of Pennsylvania have developed a novel AI framework aimed at tackling complex mathematical equations. This advancement is expected to accelerate scientific discovery by enabling a deeper understanding of intricate systems, such as DNA interactions and weather patterns. AI

Researchers have developed a novel method to enable the body to produce its own antibodies for extended periods, addressing the limitations of current antibody drugs. This technique involves gene-editing blood-forming stem cells to carry a blueprint for a specific antibody, which then act as a continuous factory within the body. The edited cells can be triggered by a vaccine booster to produce high levels of the chosen antibody, showing promising results in mice against HIV, malaria, and influenza, and even enabling the production of multiple antibodies simultaneously. AI

A recent study published in Science indicates that OpenAI's large language models have demonstrated the ability to outperform physicians in certain clinical reasoning tasks, using real emergency room data. This development occurs amidst ongoing debate about the reliability of medical information provided by chatbots, with some research highlighting impressive diagnostic capabilities while others point to fabricated information and flawed advice. Despite these concerns, products like ChatGPT for Clinicians and Healthcare are already being introduced to the market, prompting calls for further testing and cautious interpretation of AI's role in medicine. AI

Researchers from Bosch and Carnegie Mellon University have created an AI system called Humanoid Transformer with Touch Dreaming (HTD) to enhance the dexterity of humanoid robots. This system uses reinforcement learning and VR data to enable robots to predict touch and force outcomes, improving their spatial awareness and planning for complex manipulation tasks. In tests, HTD significantly boosted success rates by over 90% across various real-world tasks, with potential applications in household chores, retail, and manufacturing. AI

This article explores the necessity of fine-tuning pretrained AI models. It argues that while fine-tuning can enhance performance for specific tasks, it is not always required. The author suggests that for many applications, the capabilities of existing large pretrained models are sufficient, potentially saving resources and time. AI

The author details the practical implementation of the A2A Protocol, an open standard for agent discovery and task delegation. This second part focuses on the code, outlining the architecture where the orchestrator acts as both a server and a client. It highlights the importance of the orchestrator being an A2A service to receive structured tasks and emit failure events, contrasting this with a simpler client-only script. The project structure and setup for the shared agent and customer-specific orchestrators are also provided. AI

Large language models are proving effective at deciphering historical handwriting, a task that has long challenged AI researchers. A study by Wilfrid Laurier University found that LLMs outperformed specialized software like Transkribus in accuracy, speed, and cost when transcribing 18th and 19th-century documents. This advancement is making previously inaccessible archival collections searchable, enabling new avenues for scholarly research and personal discovery. AI

Ali Health has launched its medical AI platform, "Hydrogen Ion," and announced an exclusive content partnership with the UK's BMJ Group. This collaboration grants Hydrogen Ion access to BMJ's extensive medical journal content, enabling Chinese doctors to directly access and utilize global medical literature for clinical and research purposes. The platform also offers features like evidence-based Q&A and online translation, with ongoing discussions for partnerships with other top journals. AI

Researchers have introduced a new framework for comparing and quantifying epistemic uncertainty in machine learning models. This framework, called the integral imprecise probability metric (IIPM), generalizes classical integral probability metrics to a broader class of imprecise probability models. IIPM not only allows for comparisons between different imprecise probability models but also enables the quantification of epistemic uncertainty within a single model. A key application is the development of a new measure called Maximum Mean Imprecision (MMI), which has shown strong empirical performance in selective classification tasks, particularly when dealing with a large number of classes. AI

Researchers have developed a novel method to decompose predictive variance in deep twin networks, separating it into encoder and head components. This technique, which adds minimal computational cost, helps pinpoint the source of model failures. The encoder component proves crucial for identifying out-of-distribution samples under covariate shift, while the head component becomes informative only after encoder uncertainty is managed. This decomposition offers a practical diagnostic tool for guiding data collection strategies. AI

Researchers have developed a new method called Batched Robust iHMM (BR-iHMM) to improve the accuracy of online infinite hidden Markov models when dealing with noisy data. This approach enhances robustness against outliers and model misspecification by incorporating generalized Bayesian inference and bounding the posterior influence function. Tests on financial, energy, and synthetic datasets showed BR-iHMM reduced forecasting errors by up to 67% compared to existing methods, demonstrating its practical utility for forecasting and interpretable online learning. AI

Researchers have introduced SCOPE, a novel iterative algorithm for sparsity-constrained optimization problems. This method is designed to optimize nonlinear, differentiable, and strongly convex functions, replacing traditional gradient steps with a splicing operation that directly uses objective values. SCOPE eliminates the need for hyperparameter tuning and theoretically achieves linear convergence rates while accurately recovering the true support set. Numerical experiments demonstrate its superior performance in tasks like sparse quadratic optimization and learning sparse classifiers. AI

This paper introduces a practical method for estimating optimal classification error in binary classification tasks, particularly when dealing with soft labels and calibration. The research extends prior work by theoretically analyzing the bias of hard-label estimators and addressing the challenge of corrupted soft labels. The proposed method, which is instance-free and thus suitable for privacy-sensitive scenarios, demonstrates consistency even with imperfectly calibrated soft labels. AI

Researchers have developed a convolutional neural network (CNN) framework to test General Relativity using gravitational wave data. By training the CNN on simulated beyond-GR waveforms, they found that using a response function observable improved classification sensitivity significantly compared to raw waveforms. The framework successfully detected deviations in massive gravity theories, demonstrating its potential for probing fundamental physics with astrophysical observations. AI

Researchers have developed TAMO, a novel transformer-based policy for multi-objective Bayesian optimization that operates entirely in-context. This approach eliminates the need for per-task surrogate fitting and acquisition engineering, significantly reducing proposal time by up to 1000x. TAMO is pretrained using reinforcement learning to maximize cumulative hypervolume improvement, allowing it to approximate Pareto frontiers and improve solution quality under tight evaluation budgets. The development opens a path towards plug-and-play optimizers for scientific discovery. AI

Researchers have developed a new framework for statistically guaranteeing the performance of multi-dimensional hyperparameter tuning in data-driven machine learning settings. This approach leverages tools from real algebraic geometry to provide sharper and more broadly applicable guarantees than previous methods, which were limited to one-dimensional hyperparameters. The work also establishes the first general lower bound for this type of tuning and extends the analysis to use validation loss under minimal assumptions. AI

Researchers have analyzed the Fisher information matrices of simple two-layer ReLU neural networks with random hidden weights. They found that the eigenvalue distribution concentrates significantly on specific eigenspaces, with the first three accounting for nearly all of the matrix's trace. The study identifies these dominant eigenspaces as corresponding to spherical harmonic functions of order two or less, linking this to Mercer decomposition of neural tangent kernels. AI

Researchers have developed a smoothed analysis approach for learning from positive-only samples, a challenging problem in binary classification. Unlike worst-case scenarios where learning is nearly impossible, this new method demonstrates that all VC classes become learnable under smoothed conditions. The work also introduces efficient algorithms for related problems in parameter estimation, truncation detection, and learning from reference distributions. AI

Researchers have developed a new non-parametric method for estimating conditional distributions, which can be used for conformal regression. This approach involves partitioning data into bins and using the empirical cumulative distribution function within each bin to predict distributions. The method optimizes bin boundaries by minimizing a leave-one-out Continuous Ranked Probability Score (LOO-CRPS) and selects the optimal number of bins through cross-validation. The resulting prediction bands and sets offer finite-sample coverage guarantees and demonstrate narrower intervals than existing split-conformal methods on benchmark datasets. AI

Researchers have developed a self-consistency (SC) loss to improve the accuracy of amortized Bayesian model comparison (BMC) when simulation models are misspecified. This technique enhances BMC estimators, particularly in open-world scenarios where all candidate models are imperfect. The study evaluated four amortized BMC methods, finding that SC training significantly boosts performance when analytic likelihoods are available or surrogate likelihoods are locally accurate, even with misspecified models. AI

Researchers have developed a new methodology called Adversarial Causal Tuning (ACT) to generate realistic time-series data from causal models. This approach aims to create simulated data that matches the observational and interventional distributions of real-world datasets, enabling tasks like intervention simulation and root-cause analysis. ACT utilizes ideas from Generative Adversarial Networks and AutoML to optimize causal models and discriminators, with experiments showing its effectiveness in selecting optimal causal models and generating indistinguishable data from the true distribution. AI

Researchers have introduced Partition Tree, a new framework for conditional density estimation that can handle both continuous and categorical variables. This nonparametric approach models conditional distributions using data-adaptive partitions and learns by minimizing conditional negative log-likelihood. An extension called Partition Forest averages conditional densities for improved probabilistic prediction, showing competitive results against existing methods. AI

Researchers have developed a new method for Federated Granger Causality (FedGC) that addresses the limitation of deterministic point estimates by incorporating uncertainty awareness. This approach provides calibrated measures of uncertainty, allowing operators to distinguish reliable cross-client interactions from spurious ones. The method derives closed-form expressions for steady-state variances and proposes a post-training hypothesis testing procedure to identify genuine interactions, outperforming existing federated causal structure learning baselines on synthetic and real-world datasets. AI

Researchers have developed a new method for online inverse linear optimization, a technique used in contextual recommendation systems. This approach achieves a finite regret bound of O(d log d) for M-convex action sets, a significant improvement over previous exponential bounds and a partial answer to an open question in the field. The method combines structural characterization of optimal solutions with geometric volume arguments. Additionally, the technique has been extended to handle adversarially corrupted feedback, yielding a bound of O((C+1)d log d) without prior knowledge of the corruption level. AI

This article discusses the critical need for robust model versioning and registry systems in machine learning development. It argues that simple cloud storage solutions like S3 buckets are insufficient for managing the complexities of ML model lifecycles. The piece emphasizes the importance of dedicated registries for tracking, organizing, and deploying models effectively. AI

A developer guide demonstrates how to reliably extract structured data from Anthropic's Claude models by leveraging their tool-use feature. Instead of directly prompting for JSON, the technique involves defining a fake tool with a JSON schema for its arguments and forcing Claude to call this tool. The model's output, which conforms to the schema as a side effect of tool invocation, is then captured as the desired structured data. This method bypasses common issues like malformed JSON or prose responses, ensuring consistent and parsable output for applications. AI

Researchers have developed Cog-RAG, a novel approach to Retrieval Augmented Generation that mimics human cognitive processes for improved LLM responses. Unlike traditional methods that retrieve flat text or simple graph structures, Cog-RAG constructs a dual-hypergraph. This structure includes a theme hypergraph for narrative themes across documents and an entity hypergraph for detailed relationships within chunks. The system first identifies query themes to guide the retrieval of relevant details, enhancing coherence and reducing factual errors. AI

A new benchmark, Frontier-Eng Bench, has been released to evaluate AI agents on complex engineering tasks that lack standardized answers. This benchmark moves beyond simple problem-solving by requiring agents to propose solutions, integrate with simulators, interpret feedback, and iteratively refine parameters. The goal is to assess an agent's ability to perform continuous optimization and self-evolution in real-world scenarios, moving towards an era of 'Auto Research' where AI agents function as tireless engineering teams. AI