In a detailed technical blog post, developer Stas Bekman documents the process of porting Facebook's fairseq WMT19 translation system to the popular Hugging Face Transformers library. The project, suggested by Sam Shleifer, involved converting pre-trained models for English-Russian and Russian-English translation pairs, chosen because Bekman speaks both languages, enabling him to evaluate translation quality directly.
Bekman started by creating a simple notebook that emulated the Transformers API using a proxy to fairseq, providing a quick proof of concept. The real work, however, involved creating several new files: a configuration class, a conversion script, a model architecture implementation, a tokenizer, and accompanying tests.
The conversion process required careful handling of fairseq's tokenizer, which used a BPE (Byte-Pair Encoding) approach via fastBPE and Moses tokenizer. Bekman explains how he ported the tokenizer's encoder and decoder to work seamlessly with Transformers, including support for separate vocabularies used by the Russian-English models. Once that was done, supporting merged vocabularies for German-English models was straightforward.
A key challenge was porting the core translation functionality, including weight conversion and architecture code. Bekman also implemented sinusoidal positional embeddings from scratch to match fairseq's implementation. After conversion, models were uploaded to S3, and extensive testing—both manual and automated—ensured correctness. The final PR included documentation and model cards.
Bekman emphasizes the collaborative nature of the project, crediting Sam Shleifer for breaking down the task and providing guidance. The result is an integrated translation system that brings high-quality fairseq models to the broader Transformers ecosystem.