Pulse

Nous Research has developed Token Superposition Training (TST), a new method designed to significantly accelerate the pre-training of large language models. This technique can reduce pre-training time by up to 2.5 times for models ranging from 270 million to 10 billion parameters, without altering the model's architecture or how it performs inference. TST achieves this by modifying the training loop in two phases: an initial 'superposition' phase where token embeddings are averaged and processed in larger bags, followed by a 'recovery' phase that reverts to standard training. Experiments showed TST achieving lower final training loss with substantially less compute time compared to traditional methods. AI

Cisco Systems is laying off 4,000 employees while simultaneously investing $1 billion to shift its focus towards AI infrastructure hardware. This strategic pivot is driven by the surging demand from hyperscale cloud providers. AI

Microsoft's proposed $1 billion AI data center in Kenya is facing significant hurdles due to immense power demands. The project's electricity needs are so substantial that Kenyan officials estimate it could require shutting off power to half the country. Disagreements over power capacity and insufficient infrastructure have stalled the development. AI

A Nevada electric company plans to divert power from 50,000 residents to supply data centers. This decision affects multiple towns, which have been told to find alternative power sources. The move highlights the growing demand for energy to support data center infrastructure, particularly for AI. AI

Fervo Energy's initial public offering experienced a significant 33% increase on its first day of trading. This surge in valuation, which propelled the company's worth beyond $10 billion, is largely attributed to the escalating demand for AI data centers. The company's focus on geothermal energy solutions is seen as a key factor in meeting this growing need for sustainable power in the AI infrastructure sector. AI

Cerebras Systems has announced a new wafer-scale engine designed to accelerate AI model training and inference. The company claims this new hardware significantly reduces the time required for processing tokens, a key metric in large language model performance. This advancement aims to address the growing computational demands of complex AI workloads. AI

A massive data center planned for Utah is projected to consume an enormous amount of energy, potentially releasing the equivalent of 23 atomic bombs' worth of heat daily. This facility is intended to support artificial intelligence operations, raising concerns about its environmental impact and energy demands. The project highlights the growing tension between the rapid expansion of AI infrastructure and the need for sustainable energy solutions. AI

A massive data center, projected to be twice the size of Manhattan, is planned for Utah. This facility will be powered by liquefied natural gas (LNG), a decision that has drawn criticism regarding its environmental impact and emissions. The project is moving forward despite concerns about its energy source and scale. AI

A proposed Microsoft AI data center in Kenya faces significant power challenges, with government officials stating it would necessitate shutting down power to half the country. The scale of the energy demand for the AI facility has raised concerns about its impact on the national grid and existing power supply. AI

OpenAI has launched a new entity, the OpenAI Deployment Company, backed by over $4 billion in initial investment. This new venture aims to help organizations integrate and deploy AI systems by embedding specialized engineers. The move follows a trend of tech companies, including Google and Anthropic, establishing dedicated teams and partnerships to facilitate enterprise AI adoption. AI

The Czech AI Factory (CZAI) has officially launched in Ostrava, marking Czechia's entry into Europe's AI infrastructure development. This initiative is part of the broader EuroHPC AI Factories project, aiming to bolster AI services and supercomputing capabilities across the continent. The launch signifies a strategic move to integrate Czechia into the growing European AI ecosystem. AI

Major tech companies like Microsoft and Google are investing heavily in chip manufacturers to secure essential AI resources. This strategic move aims to guarantee access to critical components amidst an intense competition for AI hardware. The financial backing for these factories is expected to impact consumers through increased costs or limited availability. AI

Robo.ai has secured $180 million in financing from ATW Partners to advance its AI innovation in areas like smart logistics and eVTOL technologies. In parallel, Rivian has announced its own AI autonomy strategy, which includes the development of custom silicon chips to enhance its electric vehicle technology. AI

A new, lightweight AI model named Needle has been developed by distilling Gemini's tool-calling capabilities into a 26 million parameter model. This smaller model is designed to run on smartphones, making it easier for developers to build AI agents for mobile devices. The project aims to bring advanced AI functionalities to edge devices. AI

Persian Gulf nations are rapidly developing advanced AI infrastructure, but this digital expansion comes with a significant environmental cost. The region faces a critical shortage of water and energy, resources essential for powering these AI systems. A report from QIASS highlights the potential for infrastructural gridlock if water policies are not integrated into technological planning. AI

Samsung's AI chip production faces potential disruption due to a planned two-week strike by its workers, set to begin on May 21. This strike could impact the supply of high-bandwidth memory (HBM), a critical component for AI computing, as Samsung and SK Hynix are among the few global manufacturers. The company recently reported a significant increase in chip income and achieved a valuation of over one trillion USD. AI

A potential strike by South Korean workers at Samsung Electronics could significantly disrupt the global supply of AI chips. The union is demanding better wages and working conditions, and a strike could halt production of crucial components. This disruption could impact the development and deployment of AI technologies worldwide. AI

Meta is constructing a massive $27 billion AI data center in rural Louisiana, a project that is significantly impacting the local economy and job market. The facility is expected to bring numerous construction and operational jobs to Richland Parish. This development highlights the growing infrastructure demands for advanced AI technologies and Meta's substantial investment in this area. AI

Copper prices have reached new all-time highs, driven significantly by the escalating demand from artificial intelligence data centers. Analysts at Citi predict further price increases, projecting copper could reach approximately $15,000 per metric ton. This surge is attributed to both AI-related demand and strategic inventory stockpiling, indicating a strong macro backdrop supporting higher commodity prices. AI

A call is being made for a pause on the construction of new AI data centers in Canada until strong federal regulations are established. The argument is that these massive corporate data centers are being built without democratic debate and raise concerns about data sovereignty, especially with Canadian data being transferred to U.S. tech giants like Google Cloud. The author emphasizes the need for oversight and robust regulatory frameworks before proceeding with AI development. AI

Nvidia's B200 GPUs are being deployed in large clusters, utilizing RoCEv2 Ethernet and Tomahawk switches for efficient inference. This setup allows for significant cost savings as more machines are added, indicating a trend towards scaled-out AI infrastructure. AI

Investors are channeling significant capital into private nuclear energy ventures to power the UK's growing AI data centers. Over $370 million has been invested, with a notable surge of $170 million in 2024 alone. This pivot towards nuclear innovation is driven by the need for sustainable and sovereign energy solutions to meet the immense power demands of AI infrastructure. AI

Residents in Vineland, New Jersey, are experiencing significant disruptions, including an inability to do laundry, due to the proliferation of data centers in their area. The increased demand for water by these facilities is straining local resources, leading to water pressure issues and impacting daily life for the community. This situation highlights the growing environmental and resource-management challenges posed by the expansion of data center infrastructure. AI

Florida has enacted a new law, SB 484, that specifically targets AI data centers. The legislation mandates that these facilities must cover their own utility costs, preventing them from passing these expenses onto customers. Governor Ron DeSantis expressed gratitude to the legislature for passing the bill, noting it was a step forward, though less comprehensive than his initial proposals. AI

A new internet cable proposal, Polar Connect, aims to bypass critical choke points in the Middle East by running fiber optic lines through the Arctic. This route would connect Asia and Europe via North America, offering a more resilient and potentially faster data transfer path. The project seeks to mitigate risks associated with the current undersea cable infrastructure, which is vulnerable to disruptions. AI

Holmes, a startup based in Ghent, has secured 1.1 million EUR in pre-seed funding to develop an autonomous quality assurance platform specifically for AI development. This platform is designed to learn product workflows and automatically generate tests for essential user journeys, aiming to streamline the AI testing process. AI

Pillar, an Italian construction technology startup, has secured 12 million EUR in seed funding. The company plans to use this capital to develop an AI-powered operating system specifically designed for the construction industry. This platform aims to automate key processes such as quote generation, margin tracking, and workforce management. AI

A technical article explores how optimizing prompts for LLM agents can inadvertently break the prefix cache, leading to higher costs than expected. The author explains that while fewer tokens in a prompt might seem cheaper, the underlying mechanism of prefix caching in agent cycles can cause inefficiencies. This issue arises because local optimizations can disrupt the cache's effectiveness across the entire agent's workflow. AI

OpenAI has established a new subsidiary, the OpenAI Deployment Company, to assist organizations in building and operating AI systems. This new entity aims to accelerate AI adoption by operating with increased speed and a customer-centric focus, distinct from OpenAI's core research and product development. The company is launching with an initial investment of $4 billion from 19 partners, including Bain Capital and SoftBank, and will integrate approximately 150 Forward Deployed Engineers from AI consulting firm Tomoro, which OpenAI is acquiring. AI

Exeo Group and Internet Initiative Japan (IIJ) have partnered to offer edge data centers specifically designed for AI workloads. This collaboration aims to provide the necessary infrastructure to support the growing demand for AI processing power. The service will leverage Exeo Group's expertise in data center construction and IIJ's network and operational capabilities. AI

Hugging Face and AWS have collaborated to detail the infrastructure required for training and running large foundation models. The blog post outlines a layered architecture, emphasizing the interplay between AWS's compute, networking, and storage services with open-source software frameworks. It highlights the importance of efficient resource management and observability for large-scale AI operations. AI

Foxconn has launched its second generation of PEARL satellites, the PEARL-1A and PEARL-1B, as part of its "3+3" expansion strategy. This initiative focuses on electric vehicles, digital health, and robotics, leveraging AI, semiconductors, and next-generation communications. By controlling both satellite hardware and communication infrastructure, Foxconn intends to provide integrated services for the automotive and industrial sectors, including real-time data for its EV platform. AI

A proposed data center championed by Kevin O'Leary has received approval in Utah, despite concerns about its immense power consumption. The facility is projected to require 9GW of electricity annually, a figure that dwarfs the current power usage of the entire state of Utah. This approval comes amidst broader discussions about the energy demands of large-scale data infrastructure. AI

Shanghai Xiyu Jizhi, the parent company of AI startup MiniMax, has significantly boosted its registered capital. The capital injection represents a 300% increase, raising it from 1 billion to 4 billion RMB. This substantial financial move indicates a strong commitment to scaling up AI infrastructure and operations. AI

Super Micro Computer (SMCI), a key AI infrastructure provider, has reported a significant increase in profitability with revenues reaching $10.2 billion. Despite strong sales, the company consumed $6.6 billion in cash last quarter due to aggressive component procurement and production line expansion. This rapid growth and cash burn highlight the intense demand and investment required in the AI hardware sector. AI

OpenAI is establishing a new enterprise unit, backed by over $4 billion, to focus on integrating AI into production systems. This unit will also absorb Tomoro, a company with approximately 150 deployment specialists. The move aims to bridge the gap between AI demonstrations and actual customer implementation by embedding engineers within client organizations. AI

Tesla's AI hardware development is creating significant demand for ABF substrates, a key component in advanced semiconductor manufacturing. This demand has sparked a competitive race between LG Innotek and Samsung Electro-Mechanics to supply these critical materials. The focus on AI hardware by Tesla, particularly for its autonomous driving and robotics initiatives, is driving innovation and investment in the semiconductor supply chain. AI

The Matter smart home connectivity standard is partnering with the OpenADR protocol to enable seamless communication between smart home appliances and the energy grid. This collaboration aims to simplify demand response programs, allowing devices like EV chargers and HVAC systems to automatically adjust energy consumption based on grid needs. The integration could eliminate the need for separate demand response hardware, embedding this functionality directly into appliances for more efficient energy management and potential cost savings for consumers. AI

OpenAI has launched a new entity, the 'OpenAI Deployment Company,' aimed at assisting businesses in building and deploying AI solutions. This new venture will be majority-owned and controlled by OpenAI, and it will collaborate with 19 initial partners, including investment firms, consulting companies, and system integrators. The company's focus is on facilitating the production-level deployment of frontier AI technologies. AI

Developers are increasingly targeting unincorporated rural land for AI data center construction to bypass city council approvals and reduce regulatory friction. This strategy allows projects to proceed more quickly, avoiding delays caused by public scrutiny and land-use reviews common in urban areas. Meta, for example, is investing $800 million in a new AI data center in Idaho, a state often associated with more favorable regulatory environments for such developments. AI

Anthropic has secured a significant compute deal with SpaceXAI, a newly merged entity combining SpaceX and xAI, to address Claude's token usage limits. This partnership is notable given Elon Musk's prior vocal criticism of Anthropic. The agreement grants Anthropic access to compute capacity at Musk's Colossus 1 data center, with future discussions about placing data centers in space. AI

Cerebras Systems is significantly increasing its IPO price and share count due to high demand driven by the AI industry's need for compute power. While GPUs, particularly from Nvidia, have dominated AI workloads like training, the future of AI compute is expected to be more heterogeneous. This shift acknowledges that specialized hardware beyond GPUs will be crucial for both training and inference, especially as AI agents require substantial computational resources. AI

Researchers from Sakana AI and NVIDIA have developed TwELL, a novel method that significantly speeds up large language model (LLM) operations. By targeting the feedforward layers, which are computationally intensive, TwELL induces high sparsity and translates this into practical performance gains on GPUs. This approach achieves up to a 21.9% speedup in training and a 20.5% speedup in inference without compromising model accuracy. AI

A developer is exploring how to train a Large Language Model (LLM) using Swift on Apple Silicon, focusing on optimizing matrix multiplication performance. The initial article details a AI

Anthropic announced new Managed Agents features at its Code with Claude developer conference, aiming to allow users to achieve goals by simply providing an outcome and budget. The company is focusing on building the infrastructure to support agents running continuously and at scale. This development, alongside OpenAI's reported GPT-5.5 launch, suggests a bifurcation in AI development between real-time collaborative tools and long-running, delegated agents. AI

NVIDIA AI researchers have introduced cuda-oxide, an experimental compiler that enables developers to write GPU kernels in Rust and compile them directly to PTX, NVIDIA's intermediate representation for GPUs. This new tool aims to bring the CUDA programming model directly into safe Rust, bypassing the need for C++ or other intermediate languages. The project utilizes a custom rustc codegen backend and a Rust-native MLIR-like framework called Pliron, allowing host and device code to coexist in a single source file. AI

NVIDIA researchers have introduced Star Elastic, a novel post-training method that embeds multiple reasoning models of varying parameter sizes within a single checkpoint. This approach allows for the extraction of smaller, nested submodels from a larger parent model without requiring additional fine-tuning. Star Elastic utilizes a trainable router and knowledge distillation to optimize the selection of model components, enabling efficient resource utilization and tailored model performance for different reasoning tasks. AI

Maryland is challenging a $2 billion charge for power grid upgrades, arguing that the costs should not be borne by state citizens. The state's Office of People’s Counsel contends that these upgrades are primarily to benefit out-of-state AI data centers, which are disproportionately consuming power. Maryland is appealing to federal energy regulators to reallocate these costs, citing a broken cost allocation system that unfairly burdens its ratepayers. AI

This tutorial demonstrates how to implement Memori, an agent-native memory infrastructure, to enable persistent, context-aware LLM applications across multiple users and sessions. It guides users through setting up Memori in a Google Colab environment, integrating it with OpenAI clients for automatic memory interception, and storing/retrieving user data across different identities and sessions. The implementation showcases how Memori allows AI agents to retain context across interactions, moving beyond stateless conversations. AI

Researchers have developed the Fast Byte Latent Transformer (BLT) to address the slow generation speeds of byte-level language models. The new BLT Diffusion (BLT-D) method uses a block-wise diffusion objective during training, allowing for parallel byte generation during inference and reducing memory bandwidth usage by over 50%. Additional techniques like BLT Self-speculation (BLT-S) and BLT Diffusion+Verification (BLT-DV) offer further trade-offs between speed and generation quality, making byte-level LMs more practical. AI