Cache LLM Embeddings in Redis Without Stale Vector Drift

For: A backend engineer at a seed-to-Series-A SaaS startup who owns the RAG pipeline and is watching OpenAI embedding API costs climb as their document corpus grows — they've naively cached the raw text responses but keep getting semantically wrong search results whenever source documents are edited

The fix is one line: key your Redis embedding cache on sha256(chunk_text + model_version), not on document ID or URL. When a user edits a paragraph, the chunk's hash changes, the new key misses, and you re-embed only what actually changed. The old key ages out via TTL. No manual flush, no drift, no per-document bookkeeping. The rest of this post is a runnable walkthrough — with the failure modes I've watched teams hit in production.

Why naive caching drifts

Most teams start here: cache the embedding under the document ID or a URL. It works until someone edits the source. The document ID hasn't changed, so the cache returns the old vector. Your retriever now surfaces a chunk whose text no longer matches the vector — a confidently wrong result, which is the worst kind. You won't see it in metrics. You'll see it in a Slack message from a customer.

The other common pattern — flushing all cache entries for a document on save — trashes your hit rate every time someone fixes a typo. On a 400-chunk PDF, editing one sentence should re-embed one chunk, not 400.

Content-addressed keys solve both. The key is the content. Identical text under two different document IDs shares a cache entry. Edited text automatically misses. Model upgrades don't collide with old vectors because the model version is part of the key.

Prerequisites

• Python 3.10+
• A running Redis 7+ instance (docker run -p 6379:6379 redis:7 is fine for local)
• An OpenAI API key exported as OPENAI_API_KEY
• pip install openai redis tiktoken
• A vector store of your choice — this tutorial focuses on the embedding cache layer, not retrieval. Examples work standalone.

Step 1: Define the cache key

The key is deterministic. Same text + same model = same key, forever.

import hashlib

EMBEDDING_MODEL = "text-embedding-3-small"
MODEL_VERSION = "v1"  # bump when you change model or preprocessing

def cache_key(chunk_text: str, model: str = EMBEDDING_MODEL, version: str = MODEL_VERSION) -> str:
    normalized = chunk_text.strip()
    payload = f"{model}:{version}:{normalized}".encode("utf-8")
    digest = hashlib.sha256(payload).hexdigest()
    return f"emb:{model}:{version}:{digest}"

Three things to notice:

• Normalize before hashing. Strip trailing whitespace. Do not lowercase — embeddings are case-sensitive and "Apple" ≠ "apple" in vector space. If your chunker produces trailing newlines, you'll silently double your cache size otherwise.
• Model name in the key. Upgrading from text-embedding-ada-002 to text-embedding-3-small should not return old vectors. Different key namespace, clean separation.
• Explicit version tag. If you change your chunker, tokenizer, or any preprocessing, bump MODEL_VERSION. This is your escape hatch for logic changes that don't change the model name.

Expected: cache_key("hello world") returns something like emb:text-embedding-3-small:v1:b94d27b9934d3e08.... Deterministic across processes and machines.

Step 2: Wire up the Redis client with binary storage

Store embeddings as raw float32 bytes, not JSON. A 1536-dim vector is 6 KB as bytes vs. ~30 KB as a JSON array of floats. On a corpus of a million chunks that's the difference between fitting in RAM and not.

import numpy as np
import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=False)

DEFAULT_TTL_SECONDS = 60 * 60 * 24 * 30  # 30 days

def _serialize(vec: list[float]) -> bytes:
    return np.asarray(vec, dtype=np.float32).tobytes()

def _deserialize(blob: bytes) -> list[float]:
    return np.frombuffer(blob, dtype=np.float32).tolist()

Set decode_responses=False. If you leave it True, redis-py will try to decode bytes as UTF-8 and you'll get garbage or a UnicodeDecodeError.

Step 3: The cached embedding function

from openai import OpenAI

client = OpenAI()

def embed_cached(chunk_text: str) -> list[float]:
    key = cache_key(chunk_text)
    cached = r.get(key)
    if cached is not None:
        r.expire(key, DEFAULT_TTL_SECONDS)  # sliding TTL on hit
        return _deserialize(cached)

    resp = client.embeddings.create(
        model=EMBEDDING_MODEL,
        input=chunk_text,
    )
    vec = resp.data[0].embedding
    r.set(key, _serialize(vec), ex=DEFAULT_TTL_SECONDS)
    return vec

The sliding TTL is a small but important detail. Chunks that are read frequently stay warm. Chunks that stop being referenced — because their parent document was edited or deleted — age out on their own. You never have to write cleanup code.

Quick sanity check:

>>> import time
>>> t=time.time(); v1=embed_cached("The quick brown fox"); print(f"{time.time()-t:.3f}s")
0.412s
>>> t=time.time(); v2=embed_cached("The quick brown fox"); print(f"{time.time()-t:.3f}s")
0.001s
>>> v1 == v2
True

Step 4: Batch embedding without breaking the cache contract

Single-item embedding calls are wasteful. OpenAI accepts up to 2048 inputs per request. The batching function has to preserve the partial-hit case: some chunks in the batch are cached, some aren't.

def embed_batch_cached(chunks: list[str]) -> list[list[float]]:
    keys = [cache_key(c) for c in chunks]
    cached_blobs = r.mget(keys)

    results: list[list[float] | None] = [None] * len(chunks)
    misses: list[tuple[int, str, str]] = []  # (index, chunk, key)

    for i, blob in enumerate(cached_blobs):
        if blob is not None:
            results[i] = _deserialize(blob)
            r.expire(keys[i], DEFAULT_TTL_SECONDS)
        else:
            misses.append((i, chunks[i], keys[i]))

    if misses:
        miss_texts = [m[1] for m in misses]
        resp = client.embeddings.create(model=EMBEDDING_MODEL, input=miss_texts)
        pipe = r.pipeline()
        for (idx, _text, key), item in zip(misses, resp.data):
            results[idx] = item.embedding
            pipe.set(key, _serialize(item.embedding), ex=DEFAULT_TTL_SECONDS)
        pipe.execute()

    return results  # type: ignore

One MGET, one API call for the misses, one pipelined write. On a re-index of an unchanged corpus this makes zero API calls. On an ingest of a partially-edited document, it makes one API call for exactly the changed chunks.

Step 5: Handle the document-edit flow correctly

Here's where teams get this wrong. When a document is edited, do not delete cache entries. Do this instead:

1. Re-chunk the new version of the document.
2. Call embed_batch_cached() on the new chunks. Unchanged chunks hit the cache. Changed chunks miss and get re-embedded.
3. Upsert the new vectors into your vector store, keyed by the same sha256 hash. Old vectors in the vector store whose hashes no longer appear in the document get deleted.

def reindex_document(doc_id: str, new_text: str, vector_store):
    new_chunks = chunk_text(new_text)  # your chunker
    new_hashes = [cache_key(c).split(":")[-1] for c in new_chunks]

    old_hashes = set(vector_store.list_chunk_hashes(doc_id))
    new_hash_set = set(new_hashes)

    to_delete = old_hashes - new_hash_set
    to_add = [(h, c) for h, c in zip(new_hashes, new_chunks) if h not in old_hashes]

    if to_add:
        vectors = embed_batch_cached([c for _, c in to_add])
        vector_store.upsert(doc_id, list(zip([h for h, _ in to_add], vectors)))

    if to_delete:
        vector_store.delete(doc_id, list(to_delete))

The set-difference logic is the whole trick. You've turned document editing into a diff operation. Fixing a typo touches one chunk. Rewriting the intro touches two. Deleting the document deletes all its hashes from the vector store but leaves the Redis cache alone — those entries will TTL out, and if the same text reappears in another document tomorrow, it's a cache hit.

Step 6: Verify no stale drift

Write this test. Run it in CI.

def test_edit_invalidates_only_changed_chunks():
    original = ["Paragraph one.", "Paragraph two.", "Paragraph three."]
    edited   = ["Paragraph one.", "Paragraph TWO edited.", "Paragraph three."]

    v1 = embed_batch_cached(original)
    api_calls_before = openai_call_counter.value
    v2 = embed_batch_cached(edited)
    api_calls_after = openai_call_counter.value

    assert v1[0] == v2[0]           # unchanged
    assert v1[2] == v2[2]           # unchanged
    assert v1[1] != v2[1]           # changed
    assert api_calls_after - api_calls_before == 1  # exactly one miss

If that last assertion ever fails you have drift. Either your normalization is off, your chunker is non-deterministic, or someone added lowercasing to the key function.

Step 7: Observability worth adding on day one

Two counters and one histogram is enough:

• embedding_cache_hits_total and embedding_cache_misses_total — Prometheus counters. Hit rate should climb past 90% within a week of production traffic on a stable corpus.
• embedding_cache_key_collisions_total — increment whenever a cache hit's stored text length differs from the requested text length. This should always be zero. If it ever fires, your hash space has a bug.
• embedding_latency_seconds — histogram of end-to-end latency for embed_batch_cached. The p50 should be ~1 ms on hits; the p99 tracks OpenAI's latency on misses.

To do the collision check cheaply, store len(chunk_text) as a small Redis hash field alongside the vector, or just accept SHA-256's collision resistance and skip it. On a corpus under a billion chunks, SHA-256 collisions are not something you need to plan for.

Common errors

"My cache hit rate is 0% even on identical inputs"

Almost always a normalization issue. Print the raw bytes of the chunk before and after your chunker. Common culprits: trailing \n, BOM markers, non-breaking spaces from Word docs, or an inconsistent tokenizer that produces different chunk boundaries on re-ingest.

"Redis is returning None for keys I just set"

Check decode_responses. If it's True, redis-py silently mangles binary payloads. Also confirm you're pointing at the same Redis instance (dev vs. staging Redis is a classic).

"Embeddings look right but retrieval quality dropped after a model upgrade"

You upgraded the model but didn't bump the model name in the key, or you're mixing vectors from two models in one vector store index. Content-addressed keys prevent the cache from mixing, but your vector store index doesn't know that — reindex from scratch into a new index when you change embedding models.

"Redis memory usage is growing linearly forever"

Your TTL isn't being applied, or you set it once and forgot to refresh on read. Run redis-cli --scan --pattern 'emb:*' | head and then TTL <key> on a few — if you see -1, they were written without expiry. Also confirm your maxmemory-policy is allkeys-lru or similar as a backstop.

"OpenAI is rate-limiting me during large reindexes"

Batching helps but doesn't eliminate this. Wrap client.embeddings.create with tenacity retries on RateLimitError with exponential backoff, and cap concurrency with a semaphore. The cache will absorb the retry pressure on subsequent runs.

When this pattern is the wrong choice

Content-addressed caching is bad at a few things, and you should know them:

• Personalized or user-scoped embeddings. If your embedding input includes user context, every user gets their own cache entry. Hit rate collapses. Either drop the user context from the embed input, or accept the cost.
• Very short chunks. If your chunks are single sentences that repeat across many documents ("Thanks for reading."), you'll get high hit rates but low retrieval value. That's a chunking problem, not a caching problem.
• Streaming or near-real-time corpora. If documents change every few seconds, TTL-based aging isn't tight enough — you'll want explicit invalidation on write, keyed by the hash of the removed chunks.
• Compliance requirements around data residency. Hashed text keys are fine, but the vector itself may still be regulated. Redis Enterprise or a self-hosted cluster in the right region matters here.

Teams building production RAG systems at scale — especially in regulated verticals like healthcare or lending — hit these edge cases fast. If you're stuck between rebuilding your retrieval layer and shipping the next feature, CodeNicely's AI studio team works on exactly this class of problem.

Frequently Asked Questions

Should I use Redis or a dedicated vector database for the embedding cache?

Use Redis for the cache (chunk-text-hash → vector) and a dedicated vector database (pgvector, Qdrant, Pinecone, Weaviate) for the index that ANN search runs against. They serve different access patterns: the cache is a point lookup by exact key, the vector store is approximate nearest neighbor across millions of vectors. Redis Stack does support vector search, but for most RAG workloads splitting the two responsibilities keeps each layer simple.

How do I handle embedding model deprecations without losing all cached vectors?

Because the model name is baked into the cache key, the old vectors stay under their old keys and TTL out naturally while new requests populate the new namespace. You do need to reindex your vector store from scratch though — you cannot compare distances between vectors from different models.

Is SHA-256 overkill for a cache key? Would MD5 or xxHash be faster?

SHA-256 on a typical chunk (~500 tokens, ~2 KB) takes microseconds. The embedding API call takes hundreds of milliseconds. The hash is not your bottleneck. xxHash is faster but has a much higher collision rate at scale, and MD5 has enough weaknesses that some compliance frameworks disallow it. Stick with SHA-256.

What TTL should I set for cached embeddings?

Long enough that active chunks stay warm across ingestion cycles, short enough that abandoned content ages out. 30 days with sliding TTL on read is a reasonable default. If your corpus is very stable, extend it. If your Redis memory pressure is high, shorten it and rely on the fact that misses only cost one API call each.

How much can this actually reduce embedding API spend?

It depends entirely on your read/write ratio and how often your corpus changes. Teams doing frequent reindexes of mostly-static content see the largest reduction; teams with rapidly-churning corpora see less. For a personalized assessment of your RAG pipeline's cost profile and architecture, talk to CodeNicely.