Hugging Face PEFT Library Adds Model Merging Capabilities

The Hugging Face team has expanded its Parameter-Efficient Fine-Tuning (PEFT) library with the addition of new model merging methods. This update allows developers to combine multiple fine-tuned models into a single, more capable model without requiring full retraining. The merging techniques enable efficient knowledge integration, reducing memory and computational costs while improving performance on downstream tasks. The PEFT library, already widely used for low-cost fine-tuning of large language models, now supports merging through techniques like model interpolation and task arithmetic. These additions are expected to accelerate research and deployment in transfer learning and multi-task learning.