Hugging Face offers a Hub for sharing and deploying models, but real-world production deployment remains complex. BentoML, an open-source platform for ML model serving, bridges this gap. This guide demonstrates deploying DeepFloyd IF, a state-of-the-art text-to-image model, using BentoML's workflow.
DeepFloyd IF Overview
DeepFloyd IF is an open-source text-to-image model that excels in photorealism and language understanding. Unlike latent diffusion models like Stable Diffusion, it operates directly in pixel space with a modular architecture: a frozen text encoder (T5-XXL-1.1) and three cascaded diffusion stages. Stage 1 generates a 64x64 px image, which Stage 2 and Stage 3 upscale to 1024x1024 px. BentoML allows independent scaling of each stage for optimal resource allocation.
Prerequisites
- Python 3.8+
- At least 2x16GB VRAM GPU or 1x40GB VRAM GPU (e.g., 2 NVIDIA T4 GPUs)
- Clone the demo repository and navigate to the directory:
git clone https://github.com/bentoml/IF-multi-GPUs-demo.git
cd IF-multi-GPUs-demo
Key Files
import_models.py: Downloads models for each IF stage to local storage.requirements.txt: Lists dependencies.service.py: Defines a BentoML Service with three Runners and an API for image generation.start-server.py: Starts an HTTP server with a Gradio interface.bentofile.yaml: Configuration for building a Bento.
Workflow
- Save model: Run
python import_models.pyto download models into BentoML's Model Store. - Create Service: Use
service.pyto wrap models with Runners and expose an API. - Build Bento: Execute
bentoml buildto package everything into a deployable Bento. - Deploy: Containerize the Bento into a Docker image for Kubernetes or deploy via Yatai.
Testing
Start the server locally with python start-server.py. Use the Gradio interface to input prompts and generate images. For production, build and serve the Bento with bentoml serve.
Conclusion
BentoML streamlines deploying complex models like DeepFloyd IF, enabling independent scaling of stages and seamless cloud-native deployment.