supraj.dev

Interview questions & answers

Prompt & Context Engineering

System prompts, structured output, caching and what goes into the window.

  • 7 questions
  • 1 easy
  • 5 medium
  • 1 hard
  • 2 related articles

7 questions

Question 1EasyPrompt structure

What belongs in the system prompt versus the user message?

#

A product team puts user data, rules and examples all into one long user message.

Show answer

The system prompt holds stable instructions: the assistant's role, rules, output format, tone and how to use tools. The user message holds the request and the data for this turn.

Keeping them apart makes prompts easier to version and test, helps caching because the stable part stays identical, and makes it clearer to the model which text is instruction and which is data. Mark untrusted content, such as retrieved documents or user files, as data with clear delimiters, and never treat it as instructions.

Trade-offs

A long system prompt is convenient but costs tokens on every request; keep it focused.

Common mistake

Inserting retrieved documents into the system prompt as if they were trusted rules.

Follow-up

How would you handle per-customer rules that change weekly?

Question 2MediumRetrieval for agents

A coding agent works in a 2-million-line repository. How should it decide what goes into its context?

#

Loading "relevant files" up front fills the window, and the agent still misses the file that matters.

Show answer

Pre-loading large amounts of code wastes context and goes stale as the agent edits. A more effective pattern is just-in-time context: give the agent tools to list directories, search by name and content, and read specific ranges, and let it explore the way an engineer would, keeping file paths as lightweight references.

Add a short, maintained guide file with project layout, commands and conventions that loads at the start. Use semantic search where plain text search struggles, and keep tool output trimmed so one big file does not crowd out everything else.

Trade-offs

Exploration costs extra steps and tokens, but each piece of context it gathers is fresh and relevant.

Common mistake

Embedding the whole repository and pasting the top 50 chunks into every request.

Follow-up

What would you put in the project guide file for a new codebase?

Question 3MediumCost and latency

How does prompt caching work, and how do you structure prompts to benefit?

#

Each request sends the same 20,000-token policy manual followed by a short question.

Show answer

Prompt caching reuses work for a prompt prefix that was seen recently, which cuts latency and often cost for that part. Providers and engines like vLLM match on an identical prefix, so the order of content matters.

Put stable content first: tool definitions, system instructions, long reference documents, and examples. Put per-request content such as the user question, timestamps and IDs last. Any change early in the prompt invalidates everything after it. Check the provider's rules for cache lifetime and minimum size, and monitor cache hit rates.

Trade-offs

Fixing the prompt order for caching can make prompts slightly less natural to edit.

Common mistake

Putting the current timestamp at the top of the system prompt, so nothing ever caches.

Follow-up

How would you measure the cache hit rate in production?

Question 4HardContext management

An agent gets worse as a session grows long. What is happening, and what do you do?

#

After two hours of coding, the agent forgets earlier decisions and repeats fixed bugs.

Show answer

As the context fills with old tool output, logs and abandoned attempts, the important information becomes a small part of what the model reads, and quality drops even before the window is full. Context is a limited budget.

Manage it actively: summarise or compact older history while keeping decisions and open tasks, clear bulky tool results once used, keep durable notes outside the context and load them when needed, and give subtasks to sub-agents that start with a clean context and return a short result.

Trade-offs

Compaction saves space but can drop details; keep raw history available for retrieval if needed.

Common mistake

Switching to a bigger context window and keeping everything.

Follow-up

What would you always keep when compacting a coding session?

Question 5MediumDecomposition

When should you split one large prompt into a chain of smaller calls?

#

One prompt extracts fields, checks them against policy and writes a customer reply. Each part fails in a different way.

Show answer

Split when a task has distinct stages that can be checked separately, when one stage needs a different model or settings, or when you want code between stages: validate extracted fields before the policy check, and stop early if they are wrong. Each call gets a focused prompt and its own evaluation.

Keep a single call when the stages are tightly linked and latency matters most. Pass structured data between steps, not free text, and trace each step so you can see which one failed.

Trade-offs

Chains improve control and debuggability but add latency and calls for every request.

Common mistake

Splitting into many steps that each paraphrase the previous step's output and lose details.

Follow-up

Which step in this example would you assign to a cheaper model?

Question 6MediumInstruction following

The model keeps ignoring one instruction in a long system prompt. What do you check?

#

Rule 14 of 30 says "never quote prices". The assistant still quotes prices about one time in ten.

Show answer

Look for conflicts first: an example, a retrieved document or another rule may encourage the behaviour, such as a sample answer that includes a price. Long prompts also dilute individual rules, so shorten and group them, state the rule once clearly with the reason and what to do instead ("say a specialist will confirm pricing").

Test it: build cases that tempt the model to break the rule and measure the rate before and after changes. For rules that must never break, do not rely on the prompt alone; check the output in code and block or rewrite responses that violate it.

Trade-offs

Shorter prompts are followed more reliably, but you may need to move details into retrieval or tools.

Common mistake

Adding "IMPORTANT" in capitals instead of finding the conflicting instruction.

Follow-up

Which rules in your product would you enforce in code rather than in the prompt?

Question 7MediumTool output

A tool returns 40,000 tokens of JSON. How do you stop it from flooding the context?

#

A list_orders tool returns every field of every order for the customer.

Show answer

Design the tool to return less: filter and paginate on the server, return only the fields that matter for the task, and offer a concise and a detailed response mode. When results are truncated, say so and explain how to get more, such as a next-page parameter or a narrower filter.

For very large data, return a summary plus an ID the agent can use to fetch specifics. Prefer readable fields over opaque internal IDs, and clear tool results from the context after they are used.

Trade-offs

Smaller results save tokens but can force extra calls; tune defaults to common tasks.

Common mistake

Returning everything and hoping the model finds the right row.

Follow-up

How would you decide the default page size for this tool?