Text-to-image (TTI) generation has exploded in popularity, with thousands of models now available on platforms like Hugging Face. But as these systems become more widespread, concerns about embedded biases are growing. A new analysis reveals multiple sources of bias in TTI models, from training data to post-generation filters, highlighting the urgent need for better evaluation methods.
Sources of Bias in TTI Models
Bias in AI systems is not new, but TTI models present unique challenges due to their multimodal nature. Here are key sources identified:
- Training Data: Popular datasets like LAION-5B and MS-COCO contain harmful associations and stereotypes. For example, the Hugging Face Stable Bias project found that generated images of CEOs and managers often lack diversity, perpetuating real-world imbalances.
- Pre-training Filtering: Filtering data before training can inadvertently amplify biases. OpenAI noted that filters used for DALL-E 2 may exacerbate existing dataset skew, such as over-sexualized depictions of women.
- Inference Models: The CLIP model, used to guide many TTI systems, has documented biases around age, gender, and race, defaulting to white, middle-aged, male representations when prompts are underspecified.
- Latent Space: Efforts to steer generations along axes like gender show promise but require more research to fully understand how latent structures influence bias.
- Post-hoc Filters: Safety filters like those in Stable Diffusion primarily catch sexual content but often miss violent or gory images, leaving other harmful outputs unchecked.
Cultural and Geographic Stereotypes
A striking example: comparing outputs from ERNIE ViLG and Stable Diffusion for the prompt "a house in Beijing" reveals stark differences. The Western-trained model often produces stereotypical or inaccurate representations, reflecting the cultural bias of its English-dominated training data.
The Path Forward
Addressing bias in TTI systems is not purely a technical challenge—it requires socio-technical approaches that consider the broader context. Researchers emphasize that no single solution exists; instead, continuous evaluation, diverse datasets, and transparent reporting are essential. As TTI models integrate into tools like Adobe Firefly and Shutterstock, the stakes for fairness and representation grow ever higher.