HTTPClient Caching in .Net

If you are thinking this is just another of those post that talks about Caching in .Net, then you may be in for a surprise. What this post highlights is something that is already build into .Net and can be leveraged at this very moment. Still I have not seen developers using it.

Consider a scenario where you are creating a client component to consume a HTTP base service and you want to cache data retrieve to save expensive network calls. The solution would be cache data using either

  • Static variable or class.
  • In memory caching solutions such as .Net MemoryCache.
  • Any third party cache storage such as SQL Server, AppFabric Cache.

    While these storage alternatives would fix the caching part, the solution is still incomplete. As soon as we puts any data into the cache, the very next task is to determine when to invalidate the cache. You don’t want to cache forever and neither want a very short caching interval. So why not ask server component for help.

HTTP Protocol

HTTP protocol has a great caching story. With correct HTTP headers (Cache-Control, Expires) enough information can be made available to the .Net client to help it cache effectively. In-fact all browsers support caching based on these HTTP headers. Remember

Caching is support only for GET requests (over simplification).

The little know RequestCachePolicy

In .Net we use WebRequest HttpWebRequest class (and higher abstractions such as WebClient) to request for resource over HTTP. What you may not realize is that these classes have been configured to support a caching policy. This caching policy dictates whether the response would be cached or not. A static property DefaultCachePolicy once set on WebRequest(or HttpWebRequest) can affect the caching behavior for all HttpClient instance.

The RequestCachePolicy class has a constructor that takes RequestCacheLevel to decide when to cache or not.

By default this value is set to BypassCache, which means response is not cached!

To cache data set the RequestCacheLevel property to Default, CacheOnly or CacheIfAvailable. How exactly do these level affect the cache would be be detailed in section ‘Understanding Cache Behaviour’.

Cache Data Storage

You may be wondering where is the data being saved? Is it in memory, in some other process, on disk. Well the answer is on disk. As it turns out .Net uses the WinINet caching which is the underlying caching mechanism for IE. All files are stored in Temporary Internet Files folder which the same location where IE stores temp content

Understanding Caching Behavior

If you look at the documentation around RequestCachePolicy and RequestCacheLevel it provides multiple option to affect cache behavior.

To better understand the RequestCacheLevel behavior lets look at some examples. Consider a basic HTTP GET scenario. The typical request code looks like

The typical response header as seen in Fiddler is (Assuming that the server allows client caching)

The above cache header have max-age set to 604800 seconds (7 days) which allows content caching for 7 days.Both Expires and Cache-Control header help caching data but Cache-Control is the preferred one. We would not dwell into these headers and there is a lot of documentation available already. Instead lets look at how to make it work with the above code.

|

As you can see we changed the DefaultCachePolicy for HttpWebRequest to Default. What Default means is that that Cache headers would be honored and request would be fulfilled either by making a request to server or from local cache. If we make another request to this resource with Default the request would be fulfilled from local cache. This can be verified using Fiddler, where no request would be registered.

Other levels such as BypassCache, CacheOnly, Reload, NoCacheNoStore are well documented and work as advertise.

The difference between Default and CacheIfAvailable is that, if Default is used, the request is still subject to the underlying cache policy of the system, and that can prevent the use of intermediate caches.

If Revalidate is used each request is first validated from server to freshness irrespective of the cache header. So a conditional GET (HTTP 304) is made every time. See the fiddler trace below

|

Abstractions

One limitation with the default WinINet provider is that cache is stored on disk and there is no option to configure storage. CacheCow is one .Net solution that tries to implement the caching behavior similar to WinINet but provides the ability to select the storage mechanism. With CacheCow we can cache data in memory, AppFabric, SQL and as a matter of fact on any storage medium.

Once caveat with CacheCow is that is has been created for Web API, but it should not be hard to remove this dependency.

Conclusion

As you can see caching HTTP is extremely easy and is supported out of the box by .Net. Whenever building connected systems using HTTP, this is one option that cannot be ignored.

Related posts