Java: Caching without Crashing

When building larger Java applications you sooner or later stumble over the decision "Quite intensive computation  - should I recompute this every time?". Most of the time the alternative is to take a Map<K,V> and cache the computed value. This works for short lived objects where the total number of cached values predicted. But sometimes you just don't know how many values would be cached. So you'd say, we better take Apache Collection's LRUMap and limit the size of each cache, so things don't go out of hands. This is of course far better than the first solution. However, those caches still grow and grow until they reached their max size - and they'll never shrink!

We've had such a solution running on our servers. After some days of operation, the JVM had always maxed out it's heap. Well it didn't crash, it was acutally very stable, it just consumed a lot of expensive resources, since all caches where full, no matter if they were currently used or not.

So what we needed and built is a CacheManager. Whenever you need a cache, you ask the CacheManager to provide one for you. Internally it will still use Apache Collection's robust LRUMap, along with some statistics and bookkeeping. It will therefore let you specify a maximal "Time To Live (TTL)" for each cache. The CacheManager then checks all caches regularly and cleans out unused entries. Using this, you can easily use caches here and there, knowing that once the utilization goes down, the entries will be evicted and no longer block expensive resources.

Here's how you'd use this class - see ExampleCache.java for a full example:

    Cache<String, Integer> test = CacheManager
            .createCache("Test", // Name of the cache

                         10,     // Max number of entires

                         1,      // TTL

                         TimeUnit.HOUR //Unit of TTL (hours) 

                         );

The Cache can then be used like a Map:

     value = test.get("key");

     test.put("key", value);

All you need to do is set up a Timer or another service which regulary invokes: "CacheManager.runEviction()" - to clean up all caches. 

A complete example and the implementation can be found here (open source - MIT-License):

https://github.com/andyHa/scireumOpen/blob/master/src/examples/ExampleCache.java 

https://github.com/andyHa/scireumOpen/tree/master/src/com/scireum/open/cache

Each Cache can also provide statistics like size, hit rate, number of uses, etc:

Visualization of the provided usage statistics in our systems - captions are in German, sorry ;-)

This post is the second part of the my series "Enterprisy Java" - We share our hints and tricks how to overcome the obstacles when trying to build several multi tenant web applications out of a set of common modules.