prose

New architectures combat catastrophic forgetting in LLMs

Researchers have developed new architectural approaches to address catastrophic forgetting in large language models during continual pre-training and fine-tuning. One method, TFGN, introduces an overlay that allows for …