An LRU cache for model answers
When two users ask the same question there is no need to generate again. Keep the latest answers and drop the least used.
Write the class LRUCache with new LRUCache(capacity), get(key), set(key, value) and a size property.
get returns undefined when the key is missing and, when present, marks it as recently used. set adds or updates a value and also marks it as recent. When capacity is exceeded, remove the least recently used entry.
Design hint: a Map remembers insertion order.
Challenges 0/4
- Stores and returns values
- Evicts the oldest entry when over capacity
- get marks the entry as recent
- Updating a key does not grow the size and refreshes it
class LRUCache {
constructor(capacity) {
this.capacity = capacity;
this.map = new Map();
}
get(key) {
return this.map.get(key);
}
set(key, value) {
// refresh recency and evict the oldest entry when over capacity
this.map.set(key, value);
}
get size() {
return this.map.size;
}
}
const cache = new LRUCache(2);
cache.set('¿Qué es GGUF?', 'Un formato de modelo');
cache.set('¿Qué es Ollama?', 'Un servidor local');
cache.set('¿Qué es vLLM?', 'Otro servidor');
console.log(cache.size, cache.get('¿Qué es GGUF?'));Console output appears here (console.log).
Go deeper: the MDN reference →
This in production, with your data? Let's talk for 15 minutes →