LLM Prompt Failure Modes: A Diagnostic Cheatsheet

For: A product engineer at a 20–80-person B2B SaaS company who shipped an LLM-powered feature (summarization, extraction, or classification) that worked reliably in demos but now produces wrong, inconsistent, or refused outputs in production — and cannot tell whether the problem is the prompt, the model, the input data, or the temperature setting

Most production LLM failures are not model failures. They are prompt-contract violations: the input technically fit the token window but broke an implicit assumption the prompt was written against — clean formatting, one language, bounded length, no adversarial content, predictable structure. The symptom that shows up in production maps predictably back to which clause broke. Debugging becomes a lookup, not a tuning exercise.

This cheatsheet maps the failure modes we see most often in shipped summarization, extraction, and classification features, the root cause to check first, and the fix that actually holds.

The prompt contract: what your demo assumed

Every prompt that worked in a demo carries unwritten clauses. List them before you debug:

• Input shape: length distribution, language(s), encoding, formatting (markdown, HTML, plaintext)
• Input quality: typos, OCR noise, mixed casing, truncation
• Content boundaries: no PII, no jailbreak strings, no nested instructions, no empty inputs
• Output shape: JSON validity, field set, enum values, length cap
• Model state: same model version, same temperature, same system prompt

Production breaks at least one of these. The failure mode tells you which.

Symptom → root cause lookup

Failure mode taxonomy

1. Contract violations (most common)

The prompt assumed something the input didn't deliver. Fix at the input boundary, not the prompt.

• Empty string / whitespace-only input → reject before the call
• Encoding issues (smart quotes, BOM, mojibake) → normalize at ingest
• Mixed languages → detect and route, or pin output language
• Inputs above your tested length p95 → chunk or summarize-then-process

2. Output-shape failures

The model produced semantically correct content in the wrong structure. Always cheaper to constrain than to parse defensively.

• Use JSON mode, function calling, or a response schema where the provider supports it
• Validate with a schema (Zod, Pydantic) and retry once with the validator error appended
• Don't ask the model to "return only JSON" in prose — providers ignore this under load

3. Capability ceiling

The model genuinely can't do the task at the quality you need. Symptoms: failure is consistent across phrasings, persists at temperature 0, and a stronger model fixes it.

• Test the same prompt against the strongest model in the family. If that fails too, it's a task decomposition problem, not a prompt problem.
• Decompose: extract → validate → transform → format, each as its own call

4. Non-determinism and drift

• Pin model snapshots (e.g., gpt-4o-2024-08-06, claude-sonnet-4-20250514) — not aliases
• Temperature 0 reduces but does not eliminate variance; budget for it in evals
• Track output diffs on a fixed eval set every deploy; alert on regression

5. Prompt injection

Any field a user can write into is hostile. Treat retrieved documents the same way.

• Wrap untrusted content in clearly delimited blocks (<user_input>...</user_input>)
• Place trust boundary instructions after the untrusted content
• Never give the model tools it can call with user-supplied arguments without an allowlist

A debugging order that actually works

1. Reproduce. Capture the exact failing input, model id, temperature, full message array, and output. If you can't reproduce, you can't fix.
2. Diff against a known-good input. What changed — length, language, structure, special characters?
3. Strip and rebuild. Run the failing input through a minimal prompt. If it still fails, it's an input or model problem. If it passes, it's a prompt problem.
4. Constrain the output. Move from prose instructions to schemas before tuning wording.
5. Escalate the model. If a stronger model also fails, decompose the task.
6. Add a regression test. Every production bug becomes a fixed input/expected pair in your eval set.

What to log on every call

• model (full snapshot id), temperature, seed, max_tokens
• Input token count, output token count, finish reason
• Hash of the system prompt (to catch silent edits)
• Schema validation result and retry count
• Latency and provider request id (for support tickets)

Without these, every incident is a guess. With them, most incidents are a SQL query.

Where this falls apart

This cheatsheet assumes you have a labeled eval set and a way to replay production inputs. If you shipped without either, fix that first — no diagnostic framework helps when you can't measure whether a change made things better or worse. Teams building production LLM features from scratch can see how we structure evals and observability in the AI Studio work.

Frequently Asked Questions

Why does my LLM output change between calls with the same input?

Three causes, in order of likelihood: temperature is above 0, the provider silently rolled the model alias forward, or you're hitting residual non-determinism that exists even at temperature 0 on some providers. Pin a dated model snapshot, set temperature to 0, set a seed where supported, and log the model id on every call so drift is visible.

Should I fine-tune or improve my prompt?

Improve the prompt first, then constrain outputs with schemas, then escalate to a stronger model, then decompose the task. Fine-tuning is the right answer when you've done all four and the failures are consistent, narrow, and you have a few hundred high-quality labeled examples. Most teams reach for it too early.

How do I stop the model from hallucinating facts in summaries?

Check input length first — facts most often go missing from the middle of long contexts. Add an explicit instruction to use only facts present in the input and to say "not stated" when unknown. Then validate by extracting claims from the output and checking each against the source; treat unverifiable claims as failures in your eval set.

How do I prevent prompt injection from user content?

Treat any user-supplied or retrieved text as hostile. Wrap it in delimited blocks, never concatenate it into the system prompt, place your trust-boundary instructions after the untrusted content, and allowlist any tool arguments the model can produce. Assume some injection will succeed and design downstream actions to be reversible.

Can you help us audit an LLM feature that's misbehaving in production?

Yes. Audits typically cover prompt structure, eval coverage, observability, model selection, and failure-mode regression testing. Contact CodeNicely for a personalized assessment based on your stack and the specific failures you're seeing.