Http Client Factory
January 9, 2012 Uncategorized Comment off Darrel Miller

HttpClient, it lives, and it is glorious.

Along with the latest release of WCF Web API there was a updated version of HTTPClient .  With it came a bunch of breaking changes, most notably, there are no more Sync methods for doing HTTP requests.  This is a change that brings consistency with Microsoft’s new policy that all APIs that take more than 30ms (or is it 50ms?) should be async requests.  Yes, it’s a bit annoying to get used to, but I believe in the long run it will be worth it.

Http Client Factory

For the purposes of these examples that I have written, I cheated.  I simply tacked a .Result onto the end of the request to force the test to wait for response to come back.  It looks straightforward but as I was warned by Brad Wilson, there are threading perils hidden behind .Result, so be warned.

I’ve been sitting on these examples for a couple of months now, waiting for time to write a proper blog post on them, but I can see that’s not likely to happen soon, so this is what you are going to get instead.  Below are a bunch of client examples, along with the WCF Web API operation that responds to the request.  Hope they help someone.

Basic Stuff

Retrieving a representation.

[Fact]
public void GetResource() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  var responseMessage = httpClient.GetAsync("").Result;

  Assert.Equal("Hello World",responseMessage.Content.ReadAsStringAsync().Result);
}

[WebGet(UriTemplate="")]
public HttpResponseMessage GetResource(HttpRequestMessage requestMessage) {
  var response = new HttpResponseMessage(HttpStatusCode.OK);
  response.Content = new StringContent("Hello World", Encoding.UTF8, "text/plain");
  return response;
}

Echoing a representation,

[Fact]
public void RoundTripRepresentation() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);

  var responseMessage = httpClient.PostAsync("Echo",new StringContent("Echo...Echo")).Result;
            
  Assert.Equal("Echo...Echo", responseMessage.Content.ReadAsStringAsync().Result);
}

[WebInvoke(UriTemplate = "Echo", Method = "POST")]
public HttpResponseMessage PostResource(HttpRequestMessage requestMessage) {
  var response = new HttpResponseMessage(HttpStatusCode.OK);
  response.Content = requestMessage.Content;
  return response;
}

Authentication

Manually providing authentication credentials to access a protected resource,

[Fact]
public void GetProtectedResource() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("WRAP","bigAccessToken");
  var responseMessage = httpClient.GetAsync("").Result;

  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
}
 

if your client code is running on a platform that supports Windows Credentials , then you use them by providing the HttpClient with preconfigured request handler .  e.g.

[Fact]
public void ConfigureAuthenticationOptions() {
  var clientHandler = new HttpClientHandler();
  clientHandler.UseDefaultCredentials = true;
  clientHandler.PreAuthenticate = true;
  clientHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;

  var httpClient = new HttpClient(clientHandler);

  var responseMessage = httpClient.GetAsync("http://www.bing.com").Result;

}

Extensibility

The ability to pass in a derived HttpMessageHandler into the constructor is a very useful extensibility point for adding cross cutting concerns and to enable unit testing.

Check this out,

[Fact]
public void UseCustomEchoHandler() {
  var clientHandler = new EchoClientHandler();
  var httpClient = new HttpClient(clientHandler);

  var responseMessage = httpClient.PostAsync(new Uri("http://dev.null/blah"), new StringContent("payload")).Result;

  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal("payload", responseMessage.Content.ReadAsStringAsync().Result);
}

public class EchoClientHandler : HttpMessageHandler {

  protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) {

    return Task<HttpResponseMessage>.Factory.StartNew(() => {
      var response = new HttpResponseMessage(HttpStatusCode.OK);
      var stream = request.Content.ReadAsStreamAsync().Result;
      stream.Position = 0;
      var memoryStream = new MemoryStream();
      stream.CopyTo(memoryStream);
      response.Content = new StreamContent(memoryStream);
      return response;
    });
  }

}

I used the HttpClient just as I would if I were going across the wire, but my EchoClientHandler completely replaced all the real HTTP stuff and no request was actually made.

Exceptions

One major difference between the behaviour of HttpClient and HttpWebRequest is that HttpClient does not throw exceptions when you get Http status codes like 4xx and 5xx.

[Fact]
public void GetUnknownResource() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  var responseMessage = httpClient.GetAsync("itdoesnotexist").Result;

  Assert.Equal(false, responseMessage.IsSuccessStatusCode);
  Assert.Equal(HttpStatusCode.NotFound, responseMessage.StatusCode);
}

[Fact]
public void GetResourceThatReturnsA500() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  var responseMessage = httpClient.GetAsync("badserver").Result;

  Assert.Equal(false, responseMessage.IsSuccessStatusCode);
  Assert.Equal(HttpStatusCode.InternalServerError, responseMessage.StatusCode);
}

Redirects

As you would expect, redirection happens automatically by default,

[Fact]
public void GetSomethingThatRedirects() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  var responseMessage = httpClient.GetAsync("RedirectSource").Result;

  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.True(responseMessage.RequestMessage.RequestUri.OriginalString.Contains("RedirectTarget"));
}

[WebGet(UriTemplate = "RedirectSource")]
public HttpResponseMessage GetRedirectSource(HttpRequestMessage requestMessage) {
  var response = new HttpResponseMessage(HttpStatusCode.Redirect);
  response.Headers.Location = new Uri(requestMessage.RequestUri,"/RedirectTarget");
  return response;
}

[WebGet(UriTemplate = "RedirectTarget")]
public HttpResponseMessage GetRedirectTarget(HttpRequestMessage requestMessage) {
  return new HttpResponseMessage(HttpStatusCode.OK);;
}

if you want to disable the automatic redirects then you will need to configure the HttpClientHandler.  You do this by creating your own instance and setting the appropriate properties.

public void UseManualRedirects() {
  var clientHandler = new HttpClientHandler();
  clientHandler.AllowAutoRedirect = false;
  var httpClient = new HttpClient(clientHandler);
  var responseMessage = httpClient.GetAsync("https://www.bing.com").Result;
}

Caching

If you want to turn on some of the more advanced features that depend on the Windows OS like caching, ClientCertificates  and Pipelining, you need to use the WebRequestHandler.

public void UseDesktopHandlerToEnableCaching() {
  var clientHandler = new WebRequestHandler();
  clientHandler.CachePolicy = new RequestCachePolicy(RequestCacheLevel.CacheIfAvailable);  // Enable private caching
            
  var httpClient = new HttpClient(clientHandler);

  var responseMessage = httpClient.GetAsync("http://www.bing.com").Result;

}

Large Files

One area that seems to trip up lots of people is sending and receiving big files.  By default HttpClient uses an internal 64K buffer to hold the request and response bodies.  If you are expecting more data than that, you can increase that buffer size.  However, if you are going to be receiving a wide range of sizes and don’t want to preset the buffer size you can stream the responses directly.   To do that you need to use the SendAsync method.

// download a file bigger than the 64k buffer in HttpClient
[Fact]
public void DownloadAFile() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);

  var request = new HttpRequestMessage(HttpMethod.Get, "bigfile");
            
  var responseMessage = httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;
  var memoryStream = new MemoryStream();
  var stream = responseMessage.Content.ReadAsStreamAsync().Result;
  stream.CopyTo(memoryStream);
  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal(102061, memoryStream.Length);
}

Sending is easy, on the client side at least.

// upload a file bigger than the 64k buffer in HttpClient
[Fact]
public void UploadAFile() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  httpClient.DefaultRequestHeaders.TransferEncodingChunked = true;

  var fileContent = new StreamContent(typeof(SimpleApi).Assembly.GetManifestResourceStream(typeof(SimpleApi),"bigfile.pdf"));
  fileContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream"); 
  var responseMessage = httpClient.PostAsync("sendbigfile", fileContent).Result;
            
  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal("102061", responseMessage.Content.ReadAsStringAsync().Result);
}

on the server side though, you need to enable streaming mode and boost up the size of the MaxReceivedMessageSize which is just a denial of service protection value.

var httpConfiguration = new HttpConfiguration() {
       TransferMode = TransferMode.Streamed,
       // Bypass 64K buffer in request body handler
       MaxReceivedMessageSize = 1024*200
}; // Increase DOS protection limit
_ServiceHost = new HttpServiceHost(typeof (SimpleApi), httpConfiguration, _HostUrl);


_ServiceHost.Open();

Http Headers

I regularly see questions about sending custom headers to APIs.  You can do this either by setting the header on the DefaultRequestHeaders collection that hangs off HttpClient, or need to create the request message manually.

// Send custom header
[Fact]
public void GetUsingCustomHeader() {
            
  var httpClient = new HttpClient();
            
  httpClient.BaseAddress = new Uri(_HostUrl);
  var request = new HttpRequestMessage(HttpMethod.Get, "customheader");
  request.Headers.Add("foo","bar");
  var responseMessage = httpClient.SendAsync(request).Result;

  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal("bar", responseMessage.Content.ReadAsStringAsync().Result);
}

Reading that custom header is even easier.

// Read custom header
[Fact]
public void ReadCustomHeader() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
            
  var responseMessage = httpClient.GetAsync("receivecustomheader").Result;

  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal("bar", responseMessage.Headers.GetValues("foo").First());
}

[WebGet(UriTemplate = "receivecustomheader")]
public HttpResponseMessage ReturnCustomHeader(HttpRequestMessage requestMessage) {
  var response = new HttpResponseMessage(HttpStatusCode.OK); ;
  response.Headers.Add("foo","bar");
  return response;
}

Sending standard headers is more fun because of the strongly typed nature.  You don’t have to care what RFC2616 (HTTP Date  Good luck with this!) has to say about formatting dates in headers because HttpClient will take care of that for you.

[Fact]
public void ReadIfModifiedSinceHeader() {

  var httpClient = new HttpClient();
  httpClient.BaseAddress = new Uri(_HostUrl);
  var request = new HttpRequestMessage(HttpMethod.Get,new Uri("SendIfModifiedSinceHeader",UriKind.Relative));
 request.Headers.IfModifiedSince = new DateTimeOffset(DateTime.Parse("2010-10-01 09:00am +00"));

  var responseMessage = httpClient.SendAsync(request).Result;
  HttpContent content;
  Assert.Equal(HttpStatusCode.OK, responseMessage.StatusCode);
  Assert.Equal("2010-10-01 09:00AM", responseMessage.Content.ReadAsStringAsync().Result);
}
[WebInvoke(UriTemplate = "SendIfModifiedSinceHeader", Method = "GET")]
public HttpResponseMessage SendIfModifiedSinceHeader(HttpRequestMessage requestMessage) {

  var response = new HttpResponseMessage(HttpStatusCode.OK); ;
  response.Content = new StringContent(requestMessage.Headers.IfModifiedSince.Value.ToString("yyyy-MM-dd hh:mmtt"));
  return response;

}
 

More?

So, this covers most of the common questions I see from people who are trying to learn how to use a new HTTPClient, if there are other use cases I have missed, ping me on Twitter @darrel_miller and I’ll try and add them.

One last note is regarding disposing of HttpClient.  Yes, HTTPClient does implement IDisposable, however, I do not recommend creating a HttpClient inside a Using block to make a single request.  When HttpClient is disposed it causes the underlying connection to be closed also.  This means the next request has to re-open that connection.  You should try and re-use your HttpClient instances.  If the server really does not want you holding open it’s connection then it will send a header to request the connection be closed.

HttpClient is an extremely powerful client library that looks pretty straightforward on the surface.  However, there a number of gems hidden that are easy to miss, hopefully, this article will bridge the gap until we get some real documentation on how it works.

Related Blog

The Strategic Advantage of Nearshore Staff Augmentation for US and UK Businesses

The Strategic Advantage of Nearshore Staff

June 4, 2024
In The Mood For HTTP – Open Q&A

In The Mood For HTTP –

February 20, 2022
Optimizing for the Speed of Light

Optimizing for the Speed of Light

February 18, 2019