Same Cache Key, Zero Cached Tokens: What to Check

Two requests to an OpenAI model send the same long instructions and the same prompt_cache_key, yet the second response reports zero cached tokens. On GPT-5.6 and later models the key does not decide reuse. A later request can reuse only the part of an earlier request that ended at a cache breakpoint, the point where OpenAI saved it, and by default OpenAI places that point at the end of the latest message. If the current time or another changing value sits in the same message as the stable instructions, every request saves a copy that the next request cannot match.

That pattern costs money rather than merely saving none. Saving a prompt's beginning is a cache write; reusing it on a later request is a cache read, reported as cached tokens. OpenAI's prompt caching guide prices writes on GPT-5.6 and later at 1.25 times the uncached input rate and reads at 0.1 times (0.05 times on GPT-6.1 Sol). Tokens that are written on every request and never read therefore cost a quarter more than sending them without caching.

We met this pattern in an AI assistant that calls GPT-6 Sol through OpenAI's Responses API. This article follows one changing time value through two turns, shows the change we made to the outgoing request, and separates what our two tests proved from what they did not. We did not prove the cause of the original misses, and the sections below say where the evidence stops.

Why is a matching cache key not enough?

A matching key does not make two requests share a cached prefix, because reuse depends on the request content up to a breakpoint. OpenAI's guide defines the prefix as "the unchanged tokens at the beginning of a prompt" and a cache breakpoint as the point that "marks the end of a prompt prefix that OpenAI can save to the cache and reuse in later requests." The first request writes the prefix up to its breakpoint; a later request can read it only if its own content matches all the way to that breakpoint.

The key has a narrower job. For GPT-5.6 and later, the guide's section on prompt cache keys says routing is automatic and "the key is not needed to optimize caching"; it separates cache accounting between customers or users. On earlier models a stable key helps route related requests to the same cache, but it does not "guarantee a cache hit."

Where the breakpoint lands matters most. GPT-5.6 and later offer two modes. In implicit mode, the default, OpenAI "places a breakpoint at the end of the latest eligible message." The guide lists the eligible messages: user messages, the last tool response in a consecutive group of tool responses, and the last developer message in the opening group of developer messages. In explicit mode, only the breakpoints your application marks count. Neither mode places a breakpoint inside a message on its own. The guide's gotcha "A shared prefix is not always a cached prefix" describes the result: when requests share a long beginning but end differently, caching each complete request does not make the shorter shared beginning reusable.

Models before GPT-5.6 behave differently: they support only implicit breakpoints, placed at fixed token intervals (every 2,048 tokens for GPT-5.5, per the guide's summary of model differences), so this article's mechanism and fix apply to GPT-5.6 and later.

Follow one changing detail through two turns

Consider an illustrative assistant that needs a long set of operating instructions and the current time. Its application prepares two turns:

1. First turn: a developer message with the shared instructions followed by "Current time: 09:00", then the user's question.
2. Second turn: the same developer message with "Current time: 09:01", then the next question.

A developer message carries the application's instructions to the model; many client libraries send their system prompt this way. In implicit mode, the first turn writes a cache entry that ends after the user's question, so it includes "09:00". The second turn differs at "09:01", before that breakpoint, so it cannot match the entry, and it writes a new one. The point where the shared instructions end would be the useful boundary, but it lies inside the developer message, where no breakpoint exists. OpenAI's gotcha "Extending a message can prevent reuse of its cached prefix" names the remedy: keep the reusable text in a separate content block and put an explicit breakpoint after it in both requests.

The time still has to reach the model. Deleting it to improve cache numbers could make the answers worse. The fix moves the boundary, not the information: the breakpoint goes after the shared instructions and before the time, and the time and the question follow it.

What did the zero-read calls show?

The assistant's recorded run matched this example in shape. Two consecutive calls used the same model, the same 12 tools in the same order, the same cache key and the same 63,157-character stable instruction segment. Between them, the date and time section of the developer message changed, along with short run-status details next to it, such as the turn number. Both calls reported no cache reads. Each wrote a new entry: 18,030 tokens on the first call and 18,551 on the second.

Those writes cover almost the whole request: its instructions alone ran to about 74,000 characters, which at roughly four characters per token of English text is about 18,000 tokens. A breakpoint at the end of the latest message, after the changing time, would produce that pattern: each call saves nearly everything, and the next call differs inside the saved part. The calls were seconds apart, well inside the 30-minute lifetime the guide gives entries on these models, so expiry does not explain the misses. Every request in this pattern paid the write rate on about 18,000 tokens and read none.

The numbers fit, but they do not prove the cause. Our logs held hashes and lengths, not the full requests sent to OpenAI or a cache diagnostic, so we cannot exclude another difference. What we could inspect was the code that built the outgoing request. The application kept its stable instructions and its changing turn data apart internally, but the adapter that formatted the request for OpenAI joined them into one developer text block, separated by a blank line. The model received every word, and the request had no boundary where the stable part ended.

What we changed in the outgoing request

We kept the words and their order and changed only how the developer message is split:

1. Before: one developer text block held the shared instructions followed by the changing turn data. No breakpoint marked the end of the instructions.
2. After: the first content block holds the shared instructions and carries an explicit breakpoint. The second block holds the blank line and the changing turn data, including the time. The request sets prompt_cache_options.mode to explicit.

In OpenAI's request format, the change looks like this (illustrative text). The prompt_cache_breakpoint object with "mode": "explicit" is how the guide marks a breakpoint on a content block:

{
    "prompt_cache_options": { "mode": "explicit" },
    "input": [
        {
            "role": "developer",
            "content": [
                {
                    "type": "input_text",
                    "text": "Shared operating instructions...",
                    "prompt_cache_breakpoint": { "mode": "explicit" }
                },
                { "type": "input_text", "text": "\n\nCurrent time: 09:01" }
            ]
        },
        { "role": "user", "content": "The user's next question..." }
    ]
}

The guide says "Explicit-only mode lets you choose where cache writes end": content after the last marked breakpoint is processed at the uncached rate with no cache-write charge, so the changing time and question are never written. We applied the change only to models that support explicit breakpoints (GPT-5.6 and later, per the guide); requests to other models keep their earlier shape. A separate developer message after the breakpoint would also work, and OpenAI's own example in Preserve conversation history does that; we split content blocks because it kept the message layout our client library already produced.

Explicit-only mode is a trade-off, not a rule for every application. It stops the growing conversation from being written to cache, and the guide notes in Preserve conversation history that in multi-turn applications reusing that history "can save more input tokens than caching only the initial instructions." An application with long conversations that only append can keep implicit mode and add the same explicit breakpoint after its stable instructions: the guide says "You can add explicit breakpoints without turning off the implicit breakpoint." In that setup the implicit breakpoint still writes the latest message on every turn at the write rate, so the changing tail keeps costing 1.25 times; that pays off only when later turns read it back. We chose explicit-only because our changing data should not be written at all.

What did the two tests establish?

The first test checked the request we send. Wire tests captured outgoing requests through a mock network layer, a stand-in that records the request instead of sending it, over two turns with different timestamps, for GPT-6 Sol, Luna and Astra. They confirmed the two content blocks, the breakpoint on the first, explicit mode, unchanged instruction order and tool list, and the changing timestamp in the second block. That proves what the adapter sends. A mock cannot show that OpenAI accepts or reuses a cache entry.

The second test ran the two-turn example against OpenAI. We made one synthetic pair of live calls to GPT-6 Sol with the same long instructions (about 14,000 characters of filler notes) and a current-time line that differed by one second. The first call reported 2,742 cache-write tokens and no cache reads. The second reported 2,742 cache-read tokens and no cache writes. Both returned the requested one-word answer. The stable part was reused while the new time stayed in the request.

The pair has limits. It is synthetic, with no tools and a one-word reply, and we did not run the old single-block shape alongside it as a control. It does not prove that the missing boundary caused the original zero-read calls, and it measures no production savings or response time. It shows that the new request shape works in a bounded case.

Check your own zero-read pair

1. Capture two consecutive requests as sent, with their usage. The prompt template and the cache key are not enough. Compare the model, tool definitions and their order, every developer message and content block, cache options, service tier and response settings, after redacting private content. The diagnostics guide says reuse needs "an exact prefix match and compatible request settings, including the model, service tier, and tools." If you do not have code access, ask whoever owns the integration for this comparison; steps 1, 2 and 4 need only the logged requests and responses.
2. Find the first difference and the breakpoint. Start with tool definitions and request settings, which the guide counts as part of the cached context, then compare the messages in order and locate the first character that differs. Then find the first request's breakpoint: an explicit marker, or in implicit mode the end of its latest eligible message, which is usually the user's question. If the difference comes before the breakpoint, the second request cannot reuse that entry. Move the changing text after the breakpoint instead of deleting it. Check that the stable part reaches the minimum length too: the guide says "The minimum cacheable prompt length is 1,024 tokens" for GPT-5.6 and later.
3. Ask OpenAI's diagnostics, if you can send requests. On GPT-5.6 and later, prompt cache diagnostics compare a request with an earlier response: set prompt_cache_options.comparison_response_id to a recent response's ID and read prompt_cache_diagnostics on the new one. The reason input_changed covers this article's case; the diagnostics table gives the example where "instructions contain a timestamp or request ID." Other reasons name a changed model, tools, service tier, output format, reasoning effort, verbosity, cache key or compacted context. The guide's limitations note that "Diagnostics report the first classified reason", so fix it and compare again.
4. Read the usage fields, not the answer. A correct answer says nothing about the cache. The guide's Monitor cache performance section lists usage.input_tokens_details.cached_tokens and usage.input_tokens_details.cache_write_tokens. A write on every request with no reads is this pattern. After a fix, expect a write on the first request and a read of the same size on the next.
5. Expect one more write when you switch modes. The gotcha "Switching to explicit-only mode can miss an implicit cache write" explains that an explicit-only request checks only its own explicit breakpoints, so entries written in implicit mode are not reused. The first request after the change writes again.

Once one pair works, test representative turns from your application and compare total input cost, response time and answer quality before calling the change an improvement. Model choice, caching and spend limits together set the monthly bill of an AI integration in existing software, so check all three before launch.

The OpenAI documentation quoted here was checked on 3 October 2026.