TinyLoRA

BoostLoRA method grows adapter rank to surpass full fine-tuning

Researchers have introduced BoostLoRA, a novel parameter-efficient fine-tuning method designed to enhance model expressivity without increasing inference overhead. This technique iteratively trains and merges small adap…