Stable Diffusion XL, the latest high-resolution image generation model from Stability AI, has been released and is now accessible on Apple Silicon Macs thanks to collaborative work from Apple and Hugging Face. The model produces 1024x1024 images with improved prompt adherence and lighting, but its larger size previously limited it to high-end hardware.
Using the Hugging Face diffusers library, the model can run on 16GB CUDA GPUs. However, for local inference on Apple devices, the teams have ported the base model to Core ML and introduced mixed-bit palettization—a compression technique that reduces the model size from 4.8 GB to 1.4 GB (a 71% reduction) with minimal quality loss. This method assigns different bit depths (1, 2, 4, or 8 bits) per layer based on each layer's impact on output quality, achieving an effective 4.5 bits per parameter.
The Core ML models are available on Hugging Face Hub, with full pipelines for both unquantized and mixed-bit versions. Performance benchmarks on various Macs show end-to-end latencies ranging from 20 seconds on a Mac Studio (M2 Ultra) to 46 seconds on an M1 Max MacBook Pro. The refiner stage has not yet been ported, and the models require macOS 14 beta.
The teams have also updated their open-source tools, including Apple's conversion and inference repo and Hugging Face's Swift demo app, enabling developers to convert their own fine-tuned models.