Hugging Face has introduced a new offering called Inference for PRO users, providing subscribers with dedicated API endpoints for a curated selection of powerful models. This initiative aims to offer faster inference and improved rate limits compared to the free Inference API available to all users.
PRO subscribers now get exclusive access to optimized endpoints for models like Meta Llama 3, Mixtral, Code Llama, Stable Diffusion XL, and more, all powered by Hugging Face's text-generation-inference technology. The idea is to simplify experimentation and prototyping by eliminating the need to deploy models on personal infrastructure.
Getting started is straightforward: users authenticate with a PRO token and send POST requests to the model's API endpoint. For instance, generating text with Meta Llama 3 8B Instruct can be done using a simple curl command or via the huggingface_hub Python library's InferenceClient.
The service supports various generation parameters, including temperature and max tokens, as well as advanced features like streaming and caching. PRO users can also integrate these endpoints into applications such as VS Code extensions for code completion or custom chat interfaces.
For heavy production workloads, Hugging Face recommends its Inference Endpoints product. The new PRO offering is seen as an intermediate step for developers needing more power than the free tier but without committing to full dedicated endpoints.