REST Agent – Lessons learned in building generic hypermedia clients

Several years ago when I was beginning to work on a large hyper media driven client, I recognized frequently recurring patterns and decided that I could raise the level of abstraction from an HTTP client to a hypermedia client.  It took about a year to realize that it was not as good an idea as I had first thought.

Introducing RESTAgent

Four years ago I developed the RESTAgent library, created some samples, wrote some blog posts (an intro, basic usage, and more advanced) to try and make it easier to deal with hypermedia on the client by providing generic support for any hypermedia type.  It was designed to provide two methods for navigating links.  Navigate(Link link) and Embed(Link link).  The returned representation is stored as state in the RESTAgent class.

If the media type was recognized then RESTAgent would parse the content looking for additional links contained in the response.  In order for RESTAgent to know how to construct an HTTP request when following a link, a Link class was developed that would encapsulate all the link attributes defined in RFC5988.  This Link class could then be used to construct a complete HTTP request.

The combination of the Link class, the RESTAgent request generator and pluggable media type parsers allowed RESTAgent to navigate all over a hypermedia API using just a few simple methods.

Abstractions need to make things simpler

The challenge, as I discovered, is that HTTP is an extremely flexible protocol.  There are many different valid ways of dealing with the wide variety of HTTP status codes.  Dealing with client errors, redirects and server errors are all situations where the right solution is, “it depends”.  Trying to encapsulate that behaviour inside RESTAgent left me with two choices.  Be opinionated about the behaviour, or build a sophisticated interface to allow users of RESTAgent, to configure or replace the default RESTAgent behaviour. 

Being opinionated about the way HTTP is used is fine for server frameworks, but for a client library you need to be able to accommodate the chosen approaches of all of the APIs that you want to interact with.  Flexibility is a critical feature for clients.

Trying to build a lot of flexibility into an abstraction ends up with a solution just as complex as the thing that you are trying to abstract.  At this point the value that you are adding is negated by requiring people to learn the new abstraction.

Generic content is generic

Another significant limitation of a generic hypermedia client is that the only semantics that it understands are links.  Links can add a huge amount of value to a distributed systems, but it is quite difficult to do useful tasks with only links.  Client applications need some domain semantics to work with.  Having RESTAgent only able to partially handle content ended up creating content being managed in multiple places.  Which is less than ideal.

The Link is where the magic is

Further experimentation showed me that the real value of the RESTAgent library was in the Link abstraction.  It’s ability to provide a place to encapsulate the interaction semantics related to a particular link relation type was critical to making dealing with hypermedia easier.

By creating various media type parsers that expose their links using this standardized link class, we can obtain a large amount of the value that was provided by RESTAgent without needing to abstract away the HTTP interface.  My recent efforts in this area have been on building out a more useful Link class.

Managing Client State

The other work that RESTAgent was trying to help with was managing client state.  However, I realized that with a full blown client application there tends to be lots of “mini” client state machines that have different lifetimes and work together to form the complete client application.  I initially described these mini state machines as missions.  They used a single RESTAgent to achieve a goal through a set of client server interactions.  Once I actually formalized the notion of this encapsulating mission into an actual implementation , I decided that the mission itself was a better place to manage state than within the RESTAgent where all state had to be handled generically.  I will write more on the concept of missions in a future post.

My advice

I’ve seen a few people recently talking about building generic hypermedia clients.  Hopefully, if they read about my experiences, they may avoid making the same mistakes I did, and get a better result.  I would definitely be interested to hear if anyone else is experimenting with concepts similar to Links and Missions.

Photo credit: Chain https://flic.kr/p/3fdHLE
Photo credit: Package https://flic.kr/p/4A15jm
Photo credit: Mountain top https://flic.kr/p/883WpP
Photo credit: Light switch https://flic.kr/p/6mZCc4