Capstone: a client that survives a slow backend
Puts the track together in one client: an LRU cache, a timeout with AbortController and retries with growing waits.
Combines the 6 exercises of the Reliability track. It unlocks when you finish them, but you can try it now.
Write createClient({ fetchImpl, retries, baseMs, timeoutMs, cacheSize }), which returns an object with get(key). It is the wrapper you would put in front of any slow service, a local model included.
1. If key is in the cache, return its value without calling fetchImpl and mark it as used now.
2. Otherwise call fetchImpl(key, { signal }) with an AbortController that aborts after timeoutMs. A timeout becomes an Error whose name is 'TimeoutError'.
3. On any failure (a timeout too), wait sleep(baseMs * 2 ** attempt) and try again, at most retries times; then throw the last error.
4. Store the value. If the cache goes over cacheSize keys, drop the least recently used one. Errors are not stored.
fakeBackend({ failures, ms, slowKeys }) imitates the service and counts calls in window.__calls and cut-offs in window.__aborted; sleep(ms) records every wait.
Challenges 0/6
- Returns the backend's value
- The second request for the same key comes from the cache
- Retries failures waiting 100 then 200 ms
- Cuts off what takes too long with a TimeoutError, on every retry too
- With cacheSize 2 it drops the least recently used key
- Does not cache errors
function createClient({ fetchImpl, retries = 2, baseMs = 100, timeoutMs = 1000, cacheSize = 50 }) {
const cache = new Map(); // a Map keeps insertion order: its first key is the least recently used
return {
async get(key) {
// 1. cache hit: move the key to the end and return its value, without calling fetchImpl
// 2. call fetchImpl(key, { signal }) with an AbortController that aborts after timeoutMs (an Error named 'TimeoutError')
// 3. retry any failure with sleep(baseMs * 2 ** attempt), at most `retries` times, then throw the last error
// 4. store the value and drop the oldest key when there are more than cacheSize
return null;
},
};
}
const client = createClient({ fetchImpl: fakeBackend({ failures: 1 }), baseMs: 10 });
client.get('llama3.1:8b').then(console.log, (e) => console.error(e.message));Go deeper: the AWS reference →
This in production, with your data? Let's talk for 15 minutes →