Retrofit ETAG and Caching

Question

Is there a proper explanation on how to add caching and ETAG/If-None-Match support to Retrofit+OkHttp? I'm struggling to add Etag support on 2 projects, and at first I suspected that there might be an issue with HTTP headers, another project has everything set correctly and caching still doesn't work as expected.

Following are my attempts to make it work. Results show that caching seems to be working within the same instance of the application, but as soon as I restart - everything loads long again. Also, in my logs I didn't see If-None-Match being added to a request, so I assume that server isn't aware of ETag and still recalculates the response completely.

Here are some code samples:

public class RetrofitHttpClient extends UrlConnectionClient
{

    private OkUrlFactory generateDefaultOkUrlFactory()
    {
        OkHttpClient client = new com.squareup.okhttp.OkHttpClient();

        try
        {
            Cache responseCache = new Cache(baseContext.getCacheDir(), SIZE_OF_CACHE);
            client.setCache(responseCache);
        }
        catch (Exception e)
        {
            Logger.log(this, e, "Unable to set http cache");
        }

        client.setConnectTimeout(READ_TIMEOUT, TimeUnit.MILLISECONDS);
        client.setReadTimeout(CONNECT_TIMEOUT, TimeUnit.MILLISECONDS);
        return new OkUrlFactory(client);
    }

    private final OkUrlFactory factory;

    public RetrofitHttpClient()
    {
        factory = generateDefaultOkUrlFactory();
    }

    @Override
    protected HttpURLConnection openConnection(retrofit.client.Request request) throws IOException
    {
        return factory.open(new URL(request.getUrl()));
    }
}

Rest adapter is then created with FULL log level and a custom tag:

restAdapter = new RestAdapter.Builder()
        .setClient(new RetrofitHttpClient())
        .setEndpoint(Config.BASE_URL)
        .setRequestInterceptor(new SignatureSetter())
        .setConverter(new JacksonConverter(JsonHelper.getObjectMapper()))
        .setLogLevel(RestAdapter.LogLevel.FULL)
        .setLog(new AndroidLog("=NETWORK="))
        .build();

I have a long request on the first screen of the app for testing. When I open the app - it takes 7 seconds to complete the request. If I pause and resume the app - same request takes 250ms, clearly hitting the cache. If I close the app completely and restart - it again takes 7 seconds.

UPDATE: As was suggested, I have used a custom Retrofit build and attached a LoggingInterceptor. Here's what I'm getting.

Received response for *** in 449,3ms
Date: Wed, 07 Jan 2015 09:02:23 GMT
Server: Apache
X-Powered-By: PHP/5.4.31
Access-Control-Allow-Credentials: true
Pragma:
Cache-Control: public, max-age=3600
X-Frame-Options: SAMEORIGIN
Etag: "hLxLRYztkinJAB453nRV7ncBSuU=-gzip"
Last-Modified: Wed, 24 Dec 2014 13:09:04 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8
OkHttp-Selected-Protocol: http/1.1
OkHttp-Sent-Millis: 1420621288104
OkHttp-Received-Millis: 1420621288554


Sending request **** on Connection{****:80, proxy=DIRECT@ hostAddress=**** cipherSuite=none protocol=http/1.1}
Accept: application/json;
Host: ****
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/2.2.0

Response is equal to described above

As you can see, no If-None-Match header is present in the next request.

Answer 1

I see this question keeps getting attention and as soon as there is no real answer I can pick from - I'm am providing my investigation on the topic and closing the thread for now.

The end result of investigation and some discussions in the retrofit and okhttp threads on GitHub was that there was supposedly an issue in OkHttp that could prevent If-None-Match tag being set for the outgoing requests.

The issue was supposed to be fixed in OkHttp 2.3, and I'm using 'supposed' here because I didn't yet test if it really works. The testing was difficult because I was using Retrofit, and Retrofit itself had to be updated to use the new version of OkHttp and add some new Interceptors support to be able to debug all headers that are set by OkHttp. Related thread is here: https://github.com/square/okhttp/issues/831

I'm not sure if Retrofit was updated after that. Hopefully it was, so there is a good chance that issue is already fixed and Etag should properly work - just make sure you have latest versions of Retrofit and OkHttp.

I will try to test everything myself once I have time.

Answer 2

Using OkHttp interceptors will help you to diagnose the headers coming in & out of your application. The interceptors doc gives a code example of an interceptor that logs request & response headers on the network. You can use this as-is.

class LoggingInterceptor implements Interceptor {
  @Override public Response intercept(Chain chain) throws IOException {
    Request request = chain.request();

    long t1 = System.nanoTime();
    logger.info(String.format("Sending request %s on %s%n%s",
        request.url(), chain.connection(), request.headers()));

    Response response = chain.proceed(request);

    long t2 = System.nanoTime();
    logger.info(String.format("Received response for %s in %.1fms%n%s",
        response.request().url(), (t2 - t1) / 1e6d, response.headers()));

    return response;
  }
}

To hook it up to Retrofit, you'll need to get a pre-release snapshot of Retrofit. As of January, 2015, the currently-shipping versions of Retrofit don't participate in OkHttp's interceptors. There will be a release shortly that does, but it's not ready yet.

Answer 3

I was having a similar problem: OkHttp not hitting cache ever, even when the server was sending same ETAG.

My issue was SIZE_OF_CACHE. I was defining a very small size.

Try to increase it (something like 10 * 1024 * 1024 works for me) Also you can explore /data/data//files/cache to see if there is actually something stored there