Two halves, ageing at different speeds
Everything filed under this heading belongs to one of two groups, and knowing which one you are practising is most of the skill. The first group is phrasing: the orderings, framings and forms of words that happen to work well on whichever model you are using this month. That knowledge is real and worth having, and it goes off quickly, because each provider trains its next release partly to remove the need for it. A trick that reliably unlocked better output last year is now either built into the default behaviour or broken, and the article that taught it to you has not been revised. The second group is specification, which is the same discipline behind a good bug report or a clear brief to a contractor. State the task. State what the output must contain and what it must never do. Supply the material the answer depends on instead of hoping it was memorised. Show one worked instance of an answer you would accept. None of that is tied to a particular model, it survives version changes, and it marks the line between a system that works and one that only demonstrates well. The word engineering is doing heavy lifting it has not earned, though, because it leaves out the step that would justify it. Writing a prompt is drafting. Holding a fixed set of inputs with their expected answers, re-running them after every edit and noticing when something got worse is what turns drafting into a controlled process. Without that, a team is not engineering anything. It is redecorating, one clever sentence at a time, with no way to tell an improvement from a lucky sample.
- Write the acceptance criteria before the prompt. If you cannot say what would make an answer wrong, no amount of rewording will reliably produce a right one.
- Supply the material rather than summoning it. Anything the answer depends on belongs in the request, which is why retrieval-augmented generation exists at all once that material grows past a paste.
- One demonstrated answer does more than three paragraphs of description. Models imitate a shape you show far more dependably than they follow a shape you describe.
- Pin the output format explicitly. Anything that downstream code has to parse should be specified as a structure, never requested politely.
- Keep a regression file. A saved set of inputs with the answers you expect, re-run after each change, is the cheapest instrument available here and almost nobody builds one.
