Climate change is one of the greatest challenges we face, and reducing greenhouse gas emissions such as carbon dioxide (CO₂) is critical to addressing it. Training and deploying machine learning models emit CO₂ due to the energy consumed by computing infrastructure—from GPUs to storage. The amount of CO₂ emitted depends on runtime, hardware, and the carbon intensity of the energy source.
The Hugging Face Hub now provides tools to help users track and report their own emissions, improving transparency, and to choose models based on their carbon footprint.
How to Calculate Your Own CO₂ Emissions Automatically with Transformers
To get started, ensure you have the latest version of the huggingface_hub library:
pip install huggingface_hub -U
How to Find Low-Emission Models Using the Hugging Face Hub
Once a model is uploaded to the Hub, you can search for eco-friendly models using the new emissions_threshold parameter. Specify a minimum or maximum number of grams of CO₂ emitted during training.
For example, to find models that emitted at most 100 grams:
from huggingface_hub import HfApi
api = HfApi()
models = api.list_models(emissions_thresholds=(None, 100), cardData=True)
len(models)
# >>> 191
This also helps identify smaller models, as they typically have lower emissions. Inspect one:
model = models[0]
print(f'Model Name: {model.modelId}\nCO₂ Emitted during training: {model.cardData["co2_eq_emissions"]}')
# >>> Model Name: esiebomajeremiah/autonlp-email-classification-657119381
# CO₂ Emitted during training: 3.516233232503715
Alternatively, search for models with high emissions (e.g., at least 500 grams):
models = api.list_models(emissions_thresholds=(500, None), cardData=True)
len(models)
# >>> 10
Check one:
model = models[0]
print(f'Model Name: {model.modelId}\nCO₂ Emitted during training: {model.cardData["co2_eq_emissions"]}')
# >>> Model Name: Maltehb/aelaectra-danish-electra-small-cased
# CO₂ Emitted during training: 4009.5
That's a lot of CO₂! With just a few lines of code, you can quickly vet models to ensure you're being environmentally conscious.
How to Report Your Carbon Emissions with Transformers
If you use transformers, you can automatically track and report carbon emissions thanks to the codecarbon integration. After installing codecarbon, the Trainer object will automatically add the CodeCarbonCallback during training, storing emissions data.
Example training script:
from datasets import load_dataset
from transformers import AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments
ds = load_dataset("imdb")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased", num_labels=2)
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
def tokenize_function(examples):
return tokenizer(examples["text"], padding="max_length", truncation=True)
small_train_dataset = ds["train"].shuffle(seed=42).select(range(1000)).map(tokenize_function, batched=True)
small_eval_dataset = ds["test"].shuffle(seed=42).select(range(1000)).map(tokenize_function, batched=True)
training_args = TrainingArguments(
"codecarbon-text-classification",
num_train_epochs=4,
push_to_hub=True
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=small_train_dataset,
eval_dataset=small_eval_dataset,
)
trainer.train()
After training, an emissions.csv file appears in the codecarbon-text-classification directory. You can then include the CO₂ emissions in your model card.
Further Reading
- Rolnick et al. (2019) - Tackling Climate Change with Machine Learning
- Strubell et al. (2019) - Energy and Policy Considerations for Deep Learning in NLP
- Schwartz et al. (2020) - Green AI