Skip to main content
Agents deployed on Agent Server often call external services, such as config stores, auth providers, and feature-flag systems. The data these services return rarely needs to be re-fetched on every invocation. The built-in cache lets you store those results server-side so subsequent runs read from memory instead of making redundant network calls. The cache exposes two layers:
  • swr: stale-while-revalidate. Pass a key and an async loader; the server caches the result, serves it on future reads, and revalidates in the background when the entry goes stale. Use this for most caching needs.
  • cache_get / cache_set: low-level key-value API for when you need manual control over reads and writes.
All cache APIs are server-side only and require the Agent Server runtime. Values must be JSON-serializable.
swr requires Agent Server runtime v0.7.79 or later and is currently in beta. cache_get and cache_set require v0.7.29 or later.

Quick start

Pass a key and an async loader function. swr returns the cached value if available, or calls your loader to fetch it:
On the first call, swr awaits fetch_config() and caches the result. On subsequent calls, it returns the cached value instantly and revalidates in the background.

Control freshness and expiry

Control how long cached values are considered fresh and when they expire:

How revalidation works

Inspect and update cached values

swr returns an SWRResult object with the value and cache status:
.status tells you whether the value came from cache or was freshly loaded. Call .mutate(new_value) to replace the cached entry, or .mutate() with no arguments to force the loader to run again.

Cache auth credentials

Cache credential validation in a custom auth handler to avoid hitting your identity provider on every request:
With this setup, the server returns the cached user for 5 minutes without contacting the identity provider. After that, it revalidates in the background for up to 1 hour. After 1 hour, the next request blocks until validate_token completes.

Use with Pydantic models

Pass a model parameter to automatically serialize and deserialize Pydantic models:
swr calls model_dump(mode="json") before storing and model.model_validate() when reading back.

Low-level cache API

For simple get/set caching without revalidation, use cache_get and cache_set directly:

cache_get

Return the deserialized value, or None if the key does not exist or has expired.

cache_set

Next steps