Reduce proxy bandwidth with HTTP caching and conditional requests

The diagram shows a generic caching proxy; it does not imply that the MooProxy entry provides caching. Client-side conditional requests must be implemented separately.

Measure a comparable baseline

Use the same target, region, session and time window. Record request counts, response codes, body bytes, correct-result rate and your proxy bill. Separate first visits from repeat visits; a 200 response is not proof that the content is correct.

For illustration, downloading a 2 MB response 1,000 times transfers about 2,000 MB in bodies. If 900 validations return 304 without a response body, roughly 1,800 MB of body transfers are avoided. Headers, TLS, retries and billable traffic are additional.

Use ETag and 304 correctly

When a server supplies an ETag, send it in If-None-Match on the next request. A server may respond with 304 Not Modified; reuse the previously stored representation. A 304 is not a replacement for a missing cached body. Last-Modified and If-Modified-Since provide another validation mechanism. Follow the response’s Cache-Control and other directives.

curl -sS -D headers.txt -o response.json https://example.com/data
# Read the returned ETag from headers.txt, preserving its quotes
curl -sS -D recheck-headers.txt -o recheck-body.txt \
  -H 'If-None-Match: "ETAG_FROM_RESPONSE"' https://example.com/data

Use this only for resources you are authorized to access. The example URL is a placeholder. On 304, a real client must load its cached copy; if none exists, fetch a complete response.

Separate accounts, regions and variants

private, no-store, Vary and authentication affect whether a response may be reused. Never share personal account data across users. Keep language, authorization, location, cookies and other content-affecting inputs in the cache boundary. Identical URLs can return different prices in different regions.

For balances, inventory and prices, agree freshness requirements with the data owner. Do not store a response that prohibits storage, and do not use caching to bypass access controls.

Roll out with a measured comparison

1. Choose a small set of permitted public resources and record the baseline.
2. Enable validation and verify 200/304 content and update delays.
3. Compare the proxy bill and track failures, retries and incorrect results.
4. Define expiry, invalidation and rollback before expanding.

See also reducing Playwright proxy traffic for browser resource choices.