Stable Diffusion Prompts, Tokens, Weights, and CFG

Three mechanical facts separate stable diffusion prompts from typing a description into a hosted image tool, and none of the pages ranking for this phrase lead with any of them.

There's a hard ceiling on how much text the model reads, and it's small. There's a weighting syntax that lets you turn individual words up and down, which no chat-style interface offers. And there's a guidance setting that, below a certain value, causes your negative prompt to be ignored entirely, which is documented plainly and almost never mentioned.

Everything else people argue about, the tag ordering, the quality words, the enormous negative blocks that get copied between users like folklore, sits on top of those three. So that's what this covers, plus the awkward bit at the end where the habits you build here stop transferring to newer models.

The Ceiling Is 77 Tokens

The text encoder in the original Stable Diffusion setup is a frozen CLIP model, and its tokenizer configuration sets a maximum length of 77.

Not 77 words. Tokens, which are smaller and less predictable than words, since common English words tend to map to roughly one each while unusual ones, invented compounds, and technical terms split into several. I don't have an exact token count for any given phrase to give you here, and I'd rather say that than make one up.

What I can tell you is the shape of the problem. A prompt of maybe fifty to sixty ordinary words is comfortably inside the ceiling. A prompt of a hundred and fifty is not, and the excess doesn't error, it just stops being read.

Now consider what the copy-paste culture around this puts in that budget. A typical circulated prompt opens with a subject clause, then a style clause, then eight to twelve quality words, then a camera clause, then another handful of quality words at the end for good measure. The quality words are eating a third of a ceiling you didn't know you had, in order to do a job I've never been able to demonstrate they do.

The negative prompt runs through the same encoder, so the same ceiling applies to it separately. Which is worth sitting with, given that the widely shared negative blocks run to thirty or forty terms.

That's my allocation rather than anyone's standard. The row I'd defend is lighting, and the row I'd delete first is the second from the bottom.

Weighting Is The Feature Nobody Else Has

This is the genuine advantage of the Stable Diffusion family over prompting a hosted model through a chat box. You can say how much you mean each word.

Hugging Face's prompting guide describes weighting as scaling attention scores so you control how much influence each concept has, and publishes the multipliers for the parenthesis syntax.

The last row is written exactly as they have it and it reads oddly next to the others, since 0.5 as a multiplier would be a halving rather than a quartering. I'd take it as describing perceived strength rather than a literal arithmetic scale, but I'm reading their table rather than their code, so treat that as my interpretation and not a fact.

The practical point stands regardless. Weighting is the fix for the single most common complaint in image prompting, which is that one term keeps overwhelming another. Rather than deleting the loud word or piling more words behind the quiet one, you turn one down.

Their documented implementation runs through a helper library that generates weighted embeddings, and they note it supports Stable Diffusion, SDXL, SD3, Stable Cascade, and Flux. The same note mentions the library also handles longer prompts, which is the escape hatch from the ceiling above, at the cost of adding a dependency.

Guidance Scale, And The Setting That Kills Negatives

The pipeline reference documents guidance_scale with a default of 7.5, and describes it this way. A higher value "encourages the model to generate images closely linked to the text prompt at the expense of lower image quality", and guidance is enabled when the value is above 1.

Then, in the entry for negative_prompt, one clause that I think should be in every guide on this subject. It is "Ignored when not using guidance", meaning below that threshold your negative prompt is not applied at all.

The reason this matters more now than it used to is that a lot of the fast, distilled, few-step models people run today are designed to work at very low guidance. If you moved to one of those and kept your old negative block, it's decoration.

Two other defaults worth knowing from the same reference, since people fiddle with them without knowing where they started. Denoising steps default to 50, with the note that more steps usually improve quality at the cost of speed. Guidance rescale defaults to 0.

The broader argument about what negative prompts can and cannot do, including the research on the mechanism, is in negative prompts.

Two Dials That Aren't Words At All

Some of the most useful controls in this ecosystem aren't part of the prompt, and if you came from a chat-box tool you may not know they exist.

The first is the seed. The pipeline takes a generator argument documented as making generation deterministic, which means the same prompt with the same seed gives you the same picture. That sounds like a convenience feature. It's actually the thing that makes learning possible, because it lets you change one word and see only the effect of that word rather than the effect of that word plus a completely different roll of the dice.

Lock it while you're testing wording. Let it float again once the wording has stopped changing, and what you get then is variations within a look rather than variations of the look.

I wasted a genuinely stupid amount of time before I understood this, celebrating prompt edits that turned out to be different seeds, and the fix costs nothing.

The second is clip skip, documented as the number of layers to skip from the text encoder when computing embeddings, where a value of 1 means the output of the pre-final layer gets used instead of the final one. In plain terms, you're reading the prompt from slightly earlier in the encoder, which changes how literally the text is interpreted, and different community checkpoints were tuned expecting different settings.

I'm describing what the reference says rather than making a recommendation, because the right value depends on the checkpoint you're running and I've not tested enough of them to have an opinion worth publishing. What I would say is that if a checkpoint's own notes specify a setting, use theirs, since somebody tuned it and you didn't.

Neither dial is a prompt. Both change what your prompt does.

Why Old Composition Advice Breaks At Large Sizes

One historical detail explains a lot of the weirdness people hit.

The original model was trained on 512 by 512 images from a subset of a very large image-text dataset. That's the resolution its sense of composition was formed at, and it's why generating substantially larger than that with an early checkpoint produces the duplicated-subject effect, where you ask for one person and get two heads, or a landscape that repeats itself horizontally.

None of that is a prompt problem and no wording fixes it. The fix is generating near the training resolution and upscaling, or using a model trained at a larger size.

I mention it because a lot of prompt advice from that era is really compensation for that behaviour, and people still repeat it as though it were about words.

The Habits That Stop Transferring

Here's the uncomfortable part if you learned image prompting on this family of models.

Hugging Face's current prompting guidance says that prompts should be a structured narrative rather than a keyword list, on the grounds that modern models understand language better than keyword matching. Their three core elements are subject, style, and context, written as sentences.

That is close to the opposite of how Stable Diffusion prompting was practised for years, where the house style was comma-separated tags, weight parentheses everywhere, and a long trailing block of quality terms.

The same page goes further, noting that weighting "doesn't necessarily help for newer models like Flux which already has very good prompt adherence". So the flagship technique of this ecosystem is documented as being less necessary on newer architectures.

That bottom row is the durable one. Everything mechanical about a model has a shelf life, and everything about photography and drawing does not, which is why I'd spend learning time on vocabulary rather than on syntax.

Questions

What's the ideal prompt length here? Well inside the ceiling, and shorter than the circulating examples. Six to nine strong clauses is where my own hit rate peaks across image models generally, and that's comfortably under 77 tokens for ordinary English.

Do quality words like "masterpiece" do anything? I ran pairs with and without, maybe a dozen of them, and couldn't see a consistent difference. Twelve pairs is not a study, so take that as my experience. What I'm confident about is that they occupy budget in a system that has a hard limit on budget.

Does token order matter? Yes, and more than in a chat prompt. Terms near the front carry more weight and terms at the tail lose arguments with earlier ones. Put whatever you'd be angriest about getting wrong first.

Should I use weighting or just rewrite? Rewrite first, weight second. Weighting is precise and it's also a way to avoid admitting your prompt has two terms fighting, which is usually the real problem.

How does this compare to Midjourney? Different control surface entirely, since a lot of Midjourney's behaviour lives in flags rather than in the prose. I went through that in midjourney prompts.

Is any of this worth learning if I use hosted tools? The mechanics, not really. The vocabulary, absolutely, and that half is in ai image prompt.

What I'd Do First

Count the words in your current prompt, then delete every one that describes quality rather than content.

You'll usually free up a quarter of a ceiling you didn't know you were pressed against, and you can spend it on lighting and composition, which are the two things that reliably change a picture. The wider map of how image prompts differ from chat prompts is in chatgpt prompts.