Hugging Face has announced a significant upgrade to its Hub, moving from chunk-based to block-based uploads and downloads. This change is designed to accelerate data transfers, especially for large files and datasets.
The new system replaces the previous chunk-based approach with a block-based method that allows for parallel transfers and more efficient handling of large files. Users can expect faster uploads and downloads, as well as improved reliability during network interruptions.
"With block-based transfers, we can better utilize bandwidth and reduce latency, making the Hub more responsive for users worldwide," said a Hugging Face engineer.
The update is now available to all users. Hugging Face encourages developers to update their clients to take advantage of the new protocol, which is backwards-compatible with the older chunk-based system.