Skip to content
Get packs

DALL-E 2

A PyTorch re-implementation of OpenAI's DALL-E 2 text-to-image model, built for researchers to train.lucidrains/DALLE2-pytorch

GitHub Download 1.15.6
Stars
11.3k
Forks
1.1k
Licence
MIT
Language
Python
Last update
2.5 years ago
Type
Library

No commits in over a year — check for forks or alternatives before adopting.

What it is

This is a Python library that reimplements the components described in OpenAI's DALL-E 2 paper, which generates images from text descriptions. It is aimed at people who want to train or study the model rather than just use a finished product. The README says it focuses on the diffusion prior network, which maps text embeddings from CLIP to image embeddings, and it also includes a decoder that turns those embeddings into images.

  • Train a CLIP model on your own text-image pairs using the included examples
  • Train a diffusion prior that predicts image embeddings from text embeddings
  • Train a decoder that generates images from CLIP image embeddings
  • Experiment with the prior and decoder architectures using the provided Unet and Decoder classes

Great for

  • Machine learning researchers replicating or studying the DALL-E 2 architecture
  • Developers with PyTorch experience who want to train their own text-to-image components
  • Teams with access to GPUs who plan to run the included training pipeline
  • Students learning how CLIP-based diffusion text-to-image systems fit together

Not ideal for

  • People who want to type a prompt and get images without training anything, since the README describes a multi-step training process rather than a ready-to-use model
  • Anyone expecting official OpenAI DALL-E 2 weights, because this is an independent implementation and the README mentions that pre-trained checkpoints are from LAION and are still in progress
  • Non-programmers, because the usage is all Python code and assumes familiarity with PyTorch
Step-by-step guide: how to use DALL-E 2Install, first run, examples, tips and FAQs

How to get it

  1. 1

    Install with a package manager

    You'll need: Python · PyTorch · CUDA-capable GPU (the examples call .cuda())

    Install the package from PyPI

    pip install dalle2-pytorch
  2. 2

    Run it

    • Install the package with pip install dalle2-pytorch.
    • Train a CLIP model first, either with the x-clip package or using the LAION community resources the README links to.
    • Run the decoder training example with a Unet and Decoder, feeding it images in a loop.
    • Train the diffusion prior using a trained CLIP model and DiffusionPrior.
  3. 3

    Get the most out of it

    • Use the x-clip package, which the README names as the starting point for CLIP training
    • Check the LAION Hugging Face and Weights & Biases links for prior checkpoints and training statistics
    • Look at the sparse attention decoder test run linked in the README for an example of a variant
    • Use the distributed training code credited in the README as a reference for scaling to multiple GPUs

Downloads

Source code (.zip)main branch · straight from GitHubRelease 1.15.6Release notes and source

Strengths

  • Follows the published DALL-E 2 design closely enough that researchers have used it to train working components, according to the README
  • Uses well-known libraries such as PyTorch, einops, and Hugging Face Accelerate, per the appreciation section
  • MIT license allows broad reuse

Trade-offs

  • The README is partly outdated: it says the model is state of the art and notes it is no longer SOTA as of 5/23/22, and the last push was in May 2024
  • Training requires substantial compute and a separate CLIP training step; the README points to other packages and communities for that
  • Pre-trained checkpoints are not provided by this repo; the README links to LAION's separate in-progress checkpoints and test runs

How it compares

ProjectStarsTypeLevelUpdatedBest for
DALLE2-pytorch (this)11.3kLibraryadvanced2.5 years agoMachine learning researchers replicating or studying the DALL-E 2 architecture
denoising-diffusion-pytorch10.7kLibraryintermediate1 months agoResearchers and engineers who want a readable PyTorch reference for diffusion models
AI-For-Beginners70.5kLearning resourcebeginner13 days agoBeginners who want a structured, self-paced introduction to AI
awesome-ai-engineering22.5kCurated listbeginner15 days agoDevelopers who want an overview of the AI engineering topics covered in the description
machine-learning-for-trading21.7kLearning resourceintermediate7 days agoReaders of the Machine Learning for Trading book who want to run its code
awesome-artificial-intelligence16.8kCurated listbeginner3 months agoSoftware developers who want to learn the foundations behind language models and agents

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.