How AI Image Generators Work
Meta description: Have you ever wondered how AI turns text prompts into images? This article explains how AI image generators work in simple terms.
All you have to do is describe “a cat in a miniature wizard hat riding a skateboard in the rain” and boom! The AI creates the image in a matter of seconds. It’s like a magic trick, but it’s really just math, art history, and a very well trained algorithm doing all of the work. And it’s all happening at speeds our brains can’t even comprehend. Let’s see how it works, shall we?
Step 1: AI Learns About “Fur” by Looking at Millions of Photos
Before the AI can start generating images, it has to learn about what different things look like. It does this by being shown millions of images with captions explaining what’s in the picture.
Over time, the AI learns what different textures look like and how to associate those textures with words. It doesn’t remember individual pictures like you would a photo album. Instead, it stores information about the statistical likelihood of different characteristics appearing in an image.
Think of it like learning about birthdays. You might not remember any specific birthday, but you know that there’s usually a cake, some candles, frosting, and that the cake is round.
Step 2: The Text is Turned Into “Meanings”
Now that the AI knows what different things look like, it has to understand the text you type in. Instead of trying to understand the text, the AI turns the text into a mathematical representation of what you’re asking for. This is how it knows that puppies are dogs, but not umbrellas.
Step 3: AI “Paints” the Picture, Starting With Static
Most AI generators don’t work like painters, building a picture brushstroke by brushstroke. Instead, the AI begins with a “blank canvas” of static (like channel surfing until you find a snowstorm) and gradually removes the static to reveal an image. Every time the AI removes some static, it makes minor changes to the picture so the whole thing gradually looks more and more like “what you asked for.”
It’s almost like sculpting. If you don’t know how to sculpt, maybe you’ve seen sculpting tutorials online. A sculptor takes a block of marble and gradually chips away at it until it resembles something. The AI is doing a similar, if less impressive, task. The process usually takes about 20-50 iterations (removing static) and the AI checks the image after each iteration to see if it looks like what you asked for.
Step 4: AI Learns What “Good” Looks Like
During training, the AI was shown these same iterations of “starting with static and gradually unveiling an image.” It knows what each step looks like because it’s seen them before. By training on millions of images, the AI has learned how to create images that look nice in the same way that we do.
In fact, it’s sort of like seeing time-lapse videos of artists drawing portraits or landscapes and then seeing how they go about their work in real time.
Why the Hands Sometimes Look “Wrong”
If you’ve ever seen an AI image generator create images of a hand, you might have noticed that they don’t always look right. Sometimes hands will have an extra digit or the body part just doesn’t look like how we expect it to. This is because hands (and limbs in general) tend to be one of the hardest body parts for the AI to replicate.
Hands appear in so many poses and can often be seen only partially (behind a person’s back, for example) that the AI isn’t able to replicate them perfectly. The same issue occurs with text. Since text appears most commonly in signs or on objects, AI generators tend to create odd looking text that looks more like what you’d find on a street sign. These quirks are gradually being ironed out as newer AI generators arrive.
Where the Training Pictures Come From
This is an important and hotly debated issue at the moment. AI image generators are trained on images collected from across the internet. This has raised some serious concerns about whether or not these AI companies are exploiting artists by using their work without permission or compensation.
Most companies have responded by implementing opt-out programs, using only images with given licenses, or limiting their training data to images they already own the rights to. If you’re concerned about this issue, it’s worth looking into how the particular service you’re using handles this dilemma.
Is It “Creating” or “Stealing”?
This is a matter of serious debate among artists, scientists, and even lawyers. Most of these debates rage on in the comments sections of articles, but it’s worth touching on here. In short: nobody knows. These tools aren’t creating images by pulling from a database of existing images, but are instead generating entirely new sets of pixels based on mathematical relationships between those pixels. They’re a form of remixing, but remixing is also a form of creation. There’s no easy answer.
Other Observations
Better prompts yield better results. Since the AI works off of text, it can only do so much. Being specific about the look you want will always yield better results. Specificity might seem trite, but it really does help.
It’s not always factually accurate. Don’t ask the AI to generate historical figures or objects and expect it to be accurate. It’s great at creating images, but not so much at generating facts.
Copyright laws are complicated everywhere they’re legal. Copyright laws relating to AI are still pretty fluid everywhere they’re legal. If you’re planning on using AI imagery in any commercial capacity, you might want to look into your options in your country or region.
Real photos are still useful. Don’t use AI generated images for photographs of real things. If someone asks for a picture of your product, show them a real picture of your product.
Final Thoughts
Next time an AI image generator creates a perfect picture for you, remember that it wasn’t magic. Underneath the hood, it’s using math and art history to create images brushstroke by brushstroke. But it’s creating those images one static pixel at a time, starting from a chaotic mess and gradually revealing a coherent whole. It really is amazing, when you think about it. But maybe that’s an article for another day.
Your turn: What’s the weirdest thing an AI image generator has created for you? Let us know in the comments below!
Before you publish:
Fact check any technical information with official documentation from your AI image company before publishing, as these details change rapidly as newer models are introduced.
Include some visual aids. A few generated images (with your permission!) would liven up this article and be great content for social media, too. You can even create some shareable infographics with your takeaways!
This draft was created with the assistance of an AI image generator. Edit this draft in your own words and disclose that on your site. No need to thank the AI, but it did help!
