SDS 570: DALL-E 2: Stunning Photorealism from Any Text Prompt — with Jon Krohn

Model Comparison

The DALL-E 2 model from OpenAI represents a significant leap from its predecessor, DALL-E, and differs notably from GPT-3. While GPT-3 is a pure language model with 175 billion parameters, DALL-E is a smaller, multimodal model trained on text that describes images, enabling it to generate stunning visuals from text prompts 1. This multimodal capability allows DALL-E to create images like a baby shark in a tutu or a teapot shaped like a Rubik's cube, showcasing its versatility and creativity.

Want an illustration of a baby shark in a tutu serving ice cream? Provide that as an input to DALL-E, and it returns countless examples of exactly that bizarre illustration.

--- Jon Krohn

DALL-E 2 further enhances this functionality with four times greater resolution, producing more realistic images and new capabilities like image editing 1.

   

Training Data

DALL-E's training data set is pivotal to its capabilities, as it was trained on text that describes images, making it both a natural language processing and machine vision model. This unique training allows DALL-E to generate visuals that are not only creative but also contextually accurate, such as depicting cameras from different decades of the 20th century 1. The model's ability to understand and visualize temporal information highlights its advanced comprehension of the input data.

This multimodal functionality enables DALL-E to churn out staggering visual examples of whatever your mind can dream up.

--- Jon Krohn

DALL-E 2 builds on this foundation by incorporating image inputs, allowing for complex tasks like image edits, where it can seamlessly integrate new elements into existing images in a specified style 1.