Caching Part 1: Fundamentals — Why, When, and When Not to Cache

You know that feeling when you open a frequently-used app and it loads instantly? That's caching. When it takes 5 seconds? That's what happens without it.

Caching is one of the most powerful tools in a developer's arsenal — and one of the most misused. Let's build a solid mental model before we dive into the fancy stuff.

What Is a Cache, Really?

A cache is a temporary storage layer that sits between your application and a slower data source. Its job is simple: serve data faster by keeping a copy closer to where it's needed.

The performance gap between storage layers is massive:

Storage	Latency
CPU L1 Cache	~1 ns
Main Memory (RAM)	~100 ns
SSD Disk	~100 μs
Network (same region)	~500 μs
Database query (simple)	~5 ms
Database query (complex)	~50-500 ms

A Redis lookup (~1 ms) is50x faster than a simple database query (~50 ms). At scale, that difference is the line between a responsive app and a sluggish one.

Cache Hits, Misses, and Hit Ratio

Every time your app asks the cache for data, one of two things happens:

Cache Hit — The data is there. Served instantly.
Cache Miss — The data isn't there (or expired). Go fetch from the source.

The hit ratio tells you how effective your cache is:

Hit Ratio = Cache Hits / (Cache Hits + Cache Misses)

Target Hit Ratio

85-95%

Below 70% means your cache isn't earning its keep. Above 95% means you might be over-caching.

A simple Kotlin implementation to track this:

class CacheMetrics {
    private var hits = 0L
    private var misses = 0L

    fun recordHit() { hits++ }
    fun recordMiss() { misses++ }

    fun hitRatio(): Double {
        val total = hits + misses
        return if (total == 0L) 0.0 else hits.toDouble() / total
    }
}

When to Cache

Cache when your data has these characteristics:

Read-heavy access patterns. If you read the same data 100x more than you write it, caching is a no-brainer. Product catalogs, user profiles, configuration data.

Data that doesn't change often. The less frequently data changes, the more a cache helps. A blog post that's edited once a day but read 10,000 times? Perfect candidate.

Expensive computations. If a calculation takes 500ms and the result is valid for an hour, cache it. Aggregation queries, report generation, recommendation scores.

Network round trips. Every hop adds latency. Caching data locally avoids the network entirely.

When NOT to Cache

This is the part most tutorials skip. Caching adds complexity — don't use it when you don't need it.

Small datasets that fit in memory. If your entire database is 100MB and your app has 2GB RAM, just load it all into memory. A cache adds overhead for no benefit.

Data that changes every request. Real-time stock prices, live sensor data, chat messages — if the data is stale the moment you cache it, don't bother.

Unique per-request data. Search results for a specific query, personalized recommendations, one-time tokens. The cache hit ratio will be near zero.

When consistency is critical. Financial balances, inventory counts, medical records. If serving stale data causes real harm, cache only with extreme care (or not at all).

The Golden Rule: Never cache as a fix for a slow query you haven't optimized. Fix the query first, cache second. A cache masks problems; it doesn't solve them.

Cold Start vs Warm Cache

A cold cache is empty — every request is a miss. This happens after deployments, restarts, or cache flushes.

A warm cache has been populated with frequently-accessed data and is serving hits at a good ratio.

The transition from cold to warm can be painful:

Cache warming is the practice of pre-loading data before traffic hits your service. Common approaches:

Load top-N items on startup
Run a background job to populate the cache
Use a write-through cache that's always populated

class CacheWarmer(private val cache: Cache, private val repository: ProductRepository) {
    
    suspend fun warmUp() {
        val topProducts = repository.findTopProducts(1000)
        topProducts.forEach { product ->
            cache.put("product:${product.id}", product, ttl = Duration.ofHours(1))
        }
        log.info("Warmed cache with ${topProducts.size} products")
    }
}

Key Takeaways

A cache is a temporary, fast storage layer between your app and a slower source
Hit ratio is the #1 metric — aim for 85-95%
Cache when: read-heavy, infrequently changing, expensive to compute
Don't cache when: small datasets, rapidly changing, per-request uniqueness, or consistency is critical
Fix slow queries before caching — a cache is an optimization, not a fix
Plan for cold starts — warming your cache avoids post-deploy latency spikes

You know that feeling when you open a frequently-used app and it loads instantly? That's caching. When it takes 5 seconds? That's what happens without it.

Caching is one of the most powerful tools in a developer's arsenal — and one of the most misused. Let's build a solid mental model before we dive into the fancy stuff.

What Is a Cache, Really?

A cache is a temporary storage layer that sits between your application and a slower data source. Its job is simple: serve data faster by keeping a copy closer to where it's needed.

The performance gap between storage layers is massive:

Storage	Latency
CPU L1 Cache	~1 ns
Main Memory (RAM)	~100 ns
SSD Disk	~100 μs
Network (same region)	~500 μs
Database query (simple)	~5 ms
Database query (complex)	~50-500 ms

A Redis lookup (~1 ms) is50x faster than a simple database query (~50 ms). At scale, that difference is the line between a responsive app and a sluggish one.

Cache Hits, Misses, and Hit Ratio

Every time your app asks the cache for data, one of two things happens:

Cache Hit — The data is there. Served instantly.
Cache Miss — The data isn't there (or expired). Go fetch from the source.

The hit ratio tells you how effective your cache is:

Hit Ratio = Cache Hits / (Cache Hits + Cache Misses)

Target Hit Ratio

85-95%

Below 70% means your cache isn't earning its keep. Above 95% means you might be over-caching.

A simple Kotlin implementation to track this:

class CacheMetrics {
    private var hits = 0L
    private var misses = 0L

    fun recordHit() { hits++ }
    fun recordMiss() { misses++ }

    fun hitRatio(): Double {
        val total = hits + misses
        return if (total == 0L) 0.0 else hits.toDouble() / total
    }
}

When to Cache

Cache when your data has these characteristics:

Read-heavy access patterns. If you read the same data 100x more than you write it, caching is a no-brainer. Product catalogs, user profiles, configuration data.

Data that doesn't change often. The less frequently data changes, the more a cache helps. A blog post that's edited once a day but read 10,000 times? Perfect candidate.

Expensive computations. If a calculation takes 500ms and the result is valid for an hour, cache it. Aggregation queries, report generation, recommendation scores.

Network round trips. Every hop adds latency. Caching data locally avoids the network entirely.

When NOT to Cache

This is the part most tutorials skip. Caching adds complexity — don't use it when you don't need it.

Small datasets that fit in memory. If your entire database is 100MB and your app has 2GB RAM, just load it all into memory. A cache adds overhead for no benefit.

Data that changes every request. Real-time stock prices, live sensor data, chat messages — if the data is stale the moment you cache it, don't bother.

Unique per-request data. Search results for a specific query, personalized recommendations, one-time tokens. The cache hit ratio will be near zero.

When consistency is critical. Financial balances, inventory counts, medical records. If serving stale data causes real harm, cache only with extreme care (or not at all).

The Golden Rule: Never cache as a fix for a slow query you haven't optimized. Fix the query first, cache second. A cache masks problems; it doesn't solve them.

Cold Start vs Warm Cache

A cold cache is empty — every request is a miss. This happens after deployments, restarts, or cache flushes.

A warm cache has been populated with frequently-accessed data and is serving hits at a good ratio.

The transition from cold to warm can be painful:

Cache warming is the practice of pre-loading data before traffic hits your service. Common approaches:

Load top-N items on startup
Run a background job to populate the cache
Use a write-through cache that's always populated

class CacheWarmer(private val cache: Cache, private val repository: ProductRepository) {
    
    suspend fun warmUp() {
        val topProducts = repository.findTopProducts(1000)
        topProducts.forEach { product ->
            cache.put("product:${product.id}", product, ttl = Duration.ofHours(1))
        }
        log.info("Warmed cache with ${topProducts.size} products")
    }
}

Key Takeaways

A cache is a temporary, fast storage layer between your app and a slower source
Hit ratio is the #1 metric — aim for 85-95%
Cache when: read-heavy, infrequently changing, expensive to compute
Don't cache when: small datasets, rapidly changing, per-request uniqueness, or consistency is critical
Fix slow queries before caching — a cache is an optimization, not a fix
Plan for cold starts — warming your cache avoids post-deploy latency spikes

Caching Part 1: Fundamentals — Why, When, and When Not to Cache

Caching Deep Dive(9-part series)

What Is a Cache, Really?

Cache Hits, Misses, and Hit Ratio

When to Cache

When NOT to Cache

Cold Start vs Warm Cache

Key Takeaways

Tags

Related Posts

Caching Part 2: The Landscape — Where to Put Your Cache

Caching Part 6: When It Goes Wrong — Thundering Herds, Hot Keys, and Nightmares

Caching Part 7: In Production — Monitoring, Sizing, and Staying Alive

Caching Part 1: Fundamentals — Why, When, and When Not to Cache

Caching Deep Dive(9-part series)

What Is a Cache, Really?

Cache Hits, Misses, and Hit Ratio

When to Cache

When NOT to Cache

Cold Start vs Warm Cache

Key Takeaways

Tags

Related Posts

Caching Part 2: The Landscape — Where to Put Your Cache

Caching Part 6: When It Goes Wrong — Thundering Herds, Hot Keys, and Nightmares

Caching Part 7: In Production — Monitoring, Sizing, and Staying Alive