What belongs in the system prompt versus the user message?
#A product team puts user data, rules and examples all into one long user message.
Show answer
The system prompt holds stable instructions: the assistant's role, rules, output format, tone and how to use tools. The user message holds the request and the data for this turn.
Keeping them apart makes prompts easier to version and test, helps caching because the stable part stays identical, and makes it clearer to the model which text is instruction and which is data. Mark untrusted content, such as retrieved documents or user files, as data with clear delimiters, and never treat it as instructions.
Trade-offs
A long system prompt is convenient but costs tokens on every request; keep it focused.
Common mistake
Inserting retrieved documents into the system prompt as if they were trusted rules.
Follow-up
How would you handle per-customer rules that change weekly?