ChatGPT Prompt Engineering, Minus the 2023 Advice

Almost every chatgpt prompt engineering guide on page one teaches the same list, and a good chunk of that list was written for models that no longer exist. Zero-shot, few-shot, chain of thought, assign a role, use delimiters, ask it to think step by step. Those were the moves in 2023, when the model needed you to scaffold its reasoning because it wasn't doing any on its own.

Direct answer, since that's what you came for. ChatGPT prompt engineering today is mostly two things, writing a clear specification of what you want, and having some way to tell whether an edit made the output better. The phrasing tricks matter less than they did, and at least one of them is now actively counterproductive on reasoning models, which OpenAI say themselves. The measuring part is what almost nobody teaches, and it's the part that separates people who improve from people who accumulate superstitions.

I'll take the technique list first, then the part nobody covers.

Half The Technique List Quietly Changed Meaning

OpenAI's own prompt engineering guide draws a distinction that the roundups mostly ignore. GPT models "benefit from more explicit instructions", they write, while reasoning models produce "better results on tasks with only high-level guidance."

Read that twice, because it inverts a lot of received advice. If the model is doing its own decomposition, handing it your decomposition is handing it a worse one. You're constraining a process that was already working.

Here's how I'd sort the classic list now. The right-hand column is my read rather than anybody's benchmark, and I'd expect disagreement on at least two rows.

The rows that survived the transition are the ones that are specification rather than persuasion. Format, examples, boundaries, constraints. The rows that faded are the ones that were trying to make the model reason harder, because it now reasons by default and you can't make it try more by asking nicely.

Persona is the one I'd argue about. It clearly changes register, so if you want a different voice it does something. What I've never seen it do is change correctness, and the guides that promise "you are a world-class expert" improves accuracy have never shown me the comparison.

The Message Order OpenAI Publishes, And Why It Isn't Arbitrary

Their documentation lays out a recommended structure for a developer message, and the order is Identity, Instructions, Examples, Context.

They also describe the role system with an explicit priority. Developer instructions are "prioritized ahead of user messages", user instructions sit behind them, and the guide suggests thinking of the roles "like a function and its arguments in a programming language." Which is a nicer mental model than the assistant-and-user framing most people carry, because it tells you where to put things. Stable stuff goes in the function. Variable stuff goes in the arguments.

There's a second reason for that order and it's about money rather than quality. Prompt caching works by matching a prefix, so the guide says to position reusable content "at the beginning of your prompt." Identity and instructions rarely change. Context changes every call. Putting the fixed material first means the cache can do its job, and putting the variable material first means it never can.

If you're only using the chat window none of that bills you anything, but the ordering discipline is still worth having, because the same layout is what makes a prompt reusable as a template. There's a whole treatment of parameterised templates and where the seams should go in chat gpt prompts.

One more line from that guide that I think is the most useful sentence in it. "Add representative fixtures, tests, and evaluation checks before changing production prompts." Which brings me to the uncomfortable part.

Engineering Means Measuring, And The Measuring Is Uglier Than People Expect

Nobody teaches this because it's boring and it makes the discipline look less magical.

You changed a word. The output looks better. Did the word do it? These models are stochastic, so the same prompt sent twice gives different text, and any single comparison between two prompts is comparing one sample against one sample.

I worked out the arithmetic on this because I kept catching myself believing my own edits. Take two prompts that are genuinely identical in quality, run each of them n times, and score every output pass or fail at a fifty percent rate. Then look at how often pure noise hands you a difference big enough to feel real.

Two things fall out of that. With one run each, you'll see a difference half the time between two prompts that are the same, which means a single A/B test carries no information whatsoever. And the five-runs row looks worse than the one-run row on the twenty-point column, which isn't a mistake, it's granularity, since with five runs the smallest possible gap is already twenty points.

The number I'd actually hold onto is twenty. Twenty runs a side gets the chance of a spurious thirty-point gap down to about eight percent, which is roughly where I stop worrying. Below ten runs you're reading tea leaves.

Now, nobody is running twenty samples on a prompt they use to draft emails, and I don't either. The honest conclusion is milder than the math suggests. For casual work, accept that you don't know whether your edits helped and don't build a philosophy on top of them. For anything you're going to reuse a hundred times, or ship inside a product, run the fixtures, because the alternative is shipping a change that was a coin flip.

What A Test Harness Looks Like When You're One Person

Five fixtures. That's it, that's the whole apparatus, and it takes about twenty minutes to build once.

Pick five real inputs that represent the spread of what you actually send. Not five easy ones. One should be the messy input that always breaks things. Write down, before you run anything, what a correct output looks like for each, in one line, specific enough that you could hand it to somebody else and they'd agree on pass or fail. That pre-writing step is the part people skip and it's the part that stops you grading on vibes after you've seen the answer.

Then run the old prompt and the new prompt against all five, and count. If it's three-two you learned nothing. If it's five-nil you probably learned something.

I keep mine in a spreadsheet with a column per prompt version, which is unglamorous and works. There's a whole market of software for doing this properly, with real logging and real scoring, and I went through what it costs in prompt engineering tools. Short version, the free tiers are generous and the tools are all built for people shipping applications rather than people writing prompts by hand.

The Generation Half Is Engineered Completely Differently

Here's where I think the guides fail hardest, and it's not a subtle failure. Search volume for image prompting phrases is enormous. Coverage of image prompting inside prompt engineering material is close to zero.

The reason it matters is that the discipline changes shape. Almost none of the text-side machinery applies. There's no role to assign, no chain of thought to invoke, no examples you can paste in, no system message. What you have is a caption, a seed, and a set of parameters, and the engineering happens in a completely different place, which is controlled comparison.

Google's Veo documentation gives a decent picture of the spec surface for video. They name five prompt components, subject, action, style, camera positioning, and composition, with focus and lens effects and ambiance as optional extras, plus audio cues where dialogue goes in quotes. Then the hard constraints sit around it. Clips run 4, 6, or 8 seconds, and 8 is required if you want 1080p or 4K or want to supply a reference image. Aspect is 16:9 by default with 9:16 available.

That's a much smaller design space than a text prompt and a much harder one to search, because every attempt costs a render rather than a second. The video specifics are worked through in ai video prompts.

The engineering method that works on the generation side is the one-variable render. Write the prompt. Change exactly one term. Render both twice, because a single render is that one-sample problem again with an even worse noise floor. Keep a note of what the term did. Do that thirty or forty times over a few evenings and you end up with a personal vocabulary of terms whose behaviour you'd bet on, which is worth more than any list of copyable prompts.

I run generation locally on an M4 Pro, so an extra render costs me electricity and nothing else, and that changes how much testing I'm willing to do. Worth saying out loud, because if every attempt bills you, you'll reason from memory instead of testing, and reasoning from memory is exactly how prompt folklore forms.

Where The Old Advice Is Still Right

I've been dismissive, so let me give the list its due.

Delimiters still matter and they matter more as context windows grow. When you paste four pages of somebody else's writing into a prompt, the model needs to know where your instructions stop and their material starts, and OpenAI's guide recommends markdown headers and lists or XML tags for exactly this, to mark "logical boundaries of your prompt and context data." The tag-based version of this is a whole style in itself and it's covered in claude prompts.

Few-shot survived everything. Showing two or three examples of the output shape you want is still the fastest way to stop format drift, and it works on every model I've used.

And saying what to avoid still works, in text at least. It's on the image side that negatives get strange, since some systems have a dedicated field for them, some want them written into the prompt, and some largely ignore them.

Asked Often

Is prompt engineering still a real skill? The specification half is, and it's mostly just writing clearly about a task, which was always a rare skill. The phrasing-trick half has been eroding since the models started reasoning on their own.

Does telling ChatGPT it's an expert improve the answer? It changes the voice. I've never seen anyone demonstrate that it changes correctness, and I've looked. If someone has a clean comparison with enough samples to mean anything, I'd genuinely like to read it.

How long should a prompt be? Long enough to specify the task and no longer. The failure mode of a short prompt is that it's underspecified and you can see that immediately. The failure mode of a long one is that instructions start competing and you can't tell which one lost.

Should I take a course for this? Maybe, if you need the sequencing. The syllabi converge hard and the free ones teach the same material as the paid ones, which I went through course by course in prompt engineer course.

What's the difference between this and just writing well? Less than the field would like to admit. The genuinely new parts are the evaluation habit and the generation vocabulary. Everything else is specification, which is a skill borrowed from writing requirements documents.

The One Habit Worth Copying

Write the fixtures before you write the prompt.

Five inputs, five one-line descriptions of what a good answer looks like, written down before you see any output. Everything else in this piece is optional and that one isn't, because without it you have no way of knowing whether any of your engineering did anything. The underlying anatomy of a working prompt is in chatgpt prompts if you want the layer beneath this one.