What is caching?
Caching is a process of storing files in a temporary storage location for easy retrieval whenever the same data request is made in the future. Web browsers cache static files such as javascript, images in order to load websites faster, DNS cache DNS records for faster lookups, CDN cache content to reduce latency, APIs cached records for faster retrieval.
How to implement caching in a microservice environment.
In a microservice environment, it’s best to handle caching by using distributed caching. Distributed cache is a cache shared by multiple app servers and maintained as an external service to the app servers that access it.
This improves the performance of microservices by reducing the number of requests that need to be made to the underlying data store.
Steps to follow while implementing distributed caching in a microservice environment.
- Choose a caching technology, such as redis, Memcached, Hazelcast or Apache ignite
- Determine caching strategy. Decide what data should be cached, how long it should be cached, and how cached data should be invalidated.
- Implement caching in each microservice. Each microservice should have a caching layer that interacts with the distributed caching
Caching strategies
Caching strategies depend on the data and data accessing patterns i.e read and write functionalities such as:
- Is data returned always unique?(e.g search queries)
- Is the system write heavy and reads less frequently?(e.g time based logs)
- Is data written once and read multiple times?(e.g user profiles)
Cache-Aside
This strategy involves storing frequently accessed data in a cache separate from the main memory and retrieving it whenever it’s requested.
The application checks for the data requested, if the data is found in the cache then it is returned to the user (cache hit) and if it is not found then the application retrieves the data from the database then adds it to the cache and returns to the user (cache miss).
Pros
Reduces the number of expensive operations made to the database
Cons
Cache needs to be synchronized with the database every time
Write-Through Cache
Data is written simultaneously to both cache and main memory. Whenever a read operation occurs, the data is both written to cache and memory in real time. Here the cache sits in the middle of the application and the database.
Pros
Data consistency is achieved.
Cons
With time performance issues tend to occur due to overhead of writing data to the cache and database for every write operation that occurs.
Read-Through Cache
When there is a cache miss, it loads missing data from the database, populates the cache and returns it to the application. This ensures subsequent read requests from the same data can be achieved directly from the cache, resulting in faster read access times.
Pros
It improve read performance by reducing the time needed to retrieve data from the memory
Cons
Not suitable for write-heavy workloads since it doesn’t automatically update memory when data is modified in the cache.
Write-Around Cache
Data is written directly into the main memory, bypassing the cache. Whenever a write operation occurs the data is written directly into the memory and the cache is updated only when there is subsequent data read.
Cache is not used for write operations and only read benefits from the cache. When a read operation is requested and the data is not present in the cache, the cache fetches the data from the main memory and stores it in the cache for future reference. This ensures that subsequent read requests for the same data can be satisfied directly from the cache, resulting in faster read access times
Pros
Reduce the workload on the cache by not requiring it to handle write operations, which can be particularly useful in write-heavy workloads.
Data in the cache is always consistent with the data in the main memory since the cache is updated only when data is read from the main memory.
Cons
May not be suitable for read-heavy workloads since the data may not be present in the cache, resulting in slower read access times.
Cache Eviction
Cache eviction is a process of removing stale data from the cache after its usage exceeds the set time quota. How you do cache eviction depends on the policy set out. You can set automatic cache eviction by setting the time line, when the timeline is reached the file is removed from the cache instantly.
Whilst we can use automatic eviction policy, there are algorithms which do the same and the most used one is Least Recently Used Algorithm (LRU).
The LRU keeps track of the usage history of cached items and evicts the least accessed data from the cache when it reaches capacity. Every item in the cache has its own timestamp about when it was last accessed. Every time an item which is already in the cache is accessed, its timestamp is updated and the old ones are evicted to pave way for the new data when cache capacity is reached.
Example of LRU Code
Rules of caching
- Cache functions that are frequently called, its output doesn’t change more often and it takes time to execute.
- Accessing data from the introduced caching mechanism should be faster than getting data without caching and if it’s vice versa then don’t introduce caching.
- Choose appropriate data structures for caching because it impacts memory performance, for example querying databases, performing data intensive operations, for this reason only cache the attributes of the data such as names.
What are the benefits of caching
- It helps speed up the application performance and increase efficiency
- It improves user experience and encourages people to visit the site more often
- It helps avoid making too many requests or reprocessing data every time, this helps to avoid overhead and also reduce CPU usage resources. Example is when dealing with a cloud environment, you are billed according to the network communication between the services.
- It reduces the load on the backend services by redirecting significant parts of the read load from the backend database to the in-memory layer such as redis, caching can reduce the load on database, and protect it from slower performance under load, or even from crashing at peak hours.
Possible problems with caching implementation
- Any data can be cached, therefore choosing what to cache and not to cache is open to debate. If data to be cached changes too often then we don’t need to cache for too long on the other hand it depends on how long you need to have stale data.
- Cache should always be required to store frequently accessed data, identifying this data is not an easy task, you might end up filling the cache storage with unnecessary data.
- Dealing with cache misses poses another challenge because it introduces unintended latency cost. Cache misses should always be low but achieving this state is often not easy.
Conclusion
You have learned various caching strategies, caching is very important thing to know even though its a very complicated topic. Refer to Redis documentation for more.
