A prompt that changes how your product behaves is code, whatever file it lives in. Treat it that way and a surprising number of problems disappear.
Put it in the repo#
Keep prompts next to the code that uses them, in plain text files. Now every change has a diff, an author and a reason. "Who changed the tone last week?" becomes a one-line git command.
Review the diff, then the outputs#
A prompt diff on its own tells you very little. Ask for the diff and the outputs on a fixed set of examples, before and after. Reviewers should read the outputs first.
- Does it still do the main job on the examples we care about?
- Did anything that used to work get worse?
- Is the new wording something a person on the team could explain?
Keep a small, honest test set#
Ten to twenty real examples beat a thousand synthetic ones. Add a new one every time something goes wrong in production. After a few months you own a test set that reflects what your users actually do.
Heads up
Do not tune a prompt against the same examples you use to judge it. Keep a few examples hidden until the end.
Small changes, one at a time#
Change one thing per commit. If two things changed and the results moved, you have learned nothing.