//log//

In one of our modules we make use of unirest client library for Java in order to access remote REST endpoints. As, right now, this involves quite some communication, we were looking into response caching which seems both an obvious idea and yet something that is defined pretty well in HTTP and, too, implemented pretty well in apache httpclient which lives at the core of unirest. And actually, even though undocumented in unirest, getting this done has proven to be mostly straightforward. Yet I decided to do a quick write-up, maybe it will help others get started faster.

So what to do: If not already done, add the httpclient-cache module to your project. Assuming you’re using mvn, it would look like this:

 
   org.apache.httpcomponents
   httpclient-cache
   4.3.6
 

Then, in one of your modules that makes use of unirest, you should add code like this:

CacheConfig cacheConfig = CacheConfig.custom()
        .setMaxCacheEntries(1000)
        .setMaxObjectSize(8192)
        .setSharedCache(false)
        .build();
RequestConfig requestConfig = RequestConfig.custom()
        .setConnectTimeout(30000)
        .setSocketTimeout(30000)
        .build();
CloseableHttpClient cachingClient = CachingHttpClients.custom()
        .setCacheConfig(cacheConfig)
        .setDefaultRequestConfig(requestConfig)
        .build();
Unirest.setHttpClient(cachingClient);

Most of this code has simply been copied out of the official httpclient caching tutorial; chances are you want to modify some of the values (especially timeouts and amount of entries to be cached) according to your needs. The only change we needed here was the setSharedCache(false) configuration option in CacheConfig. Default behaviour is to have a shared (public) cache which results in the httpclient-cache infrastructure not caching any HTTP responses for requests that contained authentication headers.

By then you’re mostly done, and your unirest client should transparently do caching without any further ado. We, however, had some issues with this transparency as, this way, we failed to see whether it actually does anything. So, for debugging purposes: First off, it’s a good idea to set log level for org.apache.http.impl.client.cache to DEBUG or even more verbose to make the caching infrastructure produce debugging output, which will at least leave you with “cache hit / cache miss” statements in your logfile. If that doesn’t work, you should use your IDE, run your client in debug mode and try to set a breakpoint at org.apache.http.impl.client.cache.ResponseCachingPolicy#isResponseCacheable(...); this is where the caching logic checks whether a response should (or should not) be cached. Stepping through this, we quickly found out that (a) we need to disable the shared cache (see above) and (b) caching still doesn’t work because most of the endpoints we use explicitely disable caching (Cache-Control: No-Cache), but that’s another story. 🙂

One thing to add, however: The configuration outlined above is in-memory, which possibly is not what you want. httpclient-cache comes with good support for other caching providers, and especially ehcache can be enabled pretty easily. For this, please see this stackoverflow post – I just would copy/paste code over anyway.

Filed under: