wikitext

New methods like SMF and SAM reduce catastrophic forgetting in LLMs

Two new research papers explore methods to mitigate catastrophic forgetting in language models during fine-tuning. One paper introduces Sparse Memory Finetuning (SMF), which adds memory layers and updates only heavily a…