Solving Dropbox’s URL Problems

A recent post on the Dropbox developer’s blog post talked about the challenges of constructing URLs due to the challenges of encoding parameters.  They proposed the idea of using encoded JSON to embed parameters in URLs. I believe URI Templates offer a much easier and cleaner way to address this issue.  This blog posts shows how.

I’ve talked about using a URI Template library to construct URLs before, but in this post I’m going to consider the specific examples highlighted in the Dropbox post.

Picky About Punctuation

The first example introduced the problem of spaces in URLs,

/1/search/auto/My+Documents?query=draft+2013

It is true that spaces are not allowed in URLs, however interestingly, the plus sign used in this example is not the correct way to deal with spaces according to RFC 3986, the URI specification.  Web browsers allow you to use the plus sign as a replacement for space in the address bar, however, technically spaces should be encoded as %20.

Using a URI Template library you are able to clearly distinguish which parts of the final URL are literals and syntax and which parts are parameters.  The parameters are considered “data” and therefore any characters that have special meaning in an URI will automatically be escaped.

[Fact]
public void EncodingTest1()
{
 
    var url = new UriTemplate("/1/search/auto/{folder}{?query}")
        .AddParameter("folder","My Documents")
        .AddParameter("query", "draft 2013")
        .Resolve();
 
    Assert.Equal("/1/search/auto/My%20Documents?query=draft%202013", url);
}

In the above example, the space in the folder parameters and the query parameters are automatically escaped.

Dastardly Delimiters

The next example given in the Dropbox post highlights the challenges of using parameter data that contains characters that are considered URL delimiters and are therefore considered reserved.

“/hello-world” isequivalent to “/hello%2Dworld”
“/hello/world” isnot equivalent to “/hello%2Fworld“

The example is a bit misleading because the hypen character is not a reserved character therefore doesn’t need to be escaped.   However, the point remains that a forward slash in a parameter value should be escaped to prevent it from being considered a delimiter.

This ambiguity is easily avoided in URI Templates because parameter values are specified explicitly.

[Fact]
public void EncodingTest2()
{
 
    // Parameter values get encoded but hyphen doesn't need to be encoded because it
    // is an "unreserved" character according to RFC 3986
    var url = new UriTemplate("{/greeting}")
        .AddParameter("greeting", "hello-world")
        .Resolve();
 
    Assert.Equal("/hello-world", url);
 
    // A slash does need to be encoded
    var url2 = new UriTemplate("{/greeting}")
        .AddParameter("greeting", "hello/world")
        .Resolve();
 
    Assert.Equal("/hello%2Fworld", url2);
 
    // If you truly want to make multiple path segments then do this
    var url3 = new UriTemplate("{/greeting*}")
        .AddParameter("greeting", new List<string> {"hello","world"})
        .Resolve();
 
    Assert.Equal("/hello/world", url3);
 
}
</string>

Parameter Preferences

The next Dropbox example demonstrates that there is some flexibility in the way you can represent lists of values in URLs.

/docs/salary.csv?columns=1,2
/docs/salary.csv?column=1&column=2

The problem with flexibility that is given to API producers, is that API consumers have to deal with additional complexity.  Fortunately URI Templates allows these different approaches to be handled by adding one additional character to the URI Template.

[Fact]
public void EncodingTest3()
{
 
    // There are different ways that lists can be included in query params
    // Just as a comma delimited list
    var url = new UriTemplate("/docs/salary.csv{?columns}")
        .AddParameter("columns", new List<int> {1,2})
        .Resolve();
 
    Assert.Equal("/docs/salary.csv?columns=1,2", url);
 
    // or as a multiple parameter instances
    var url2 = new UriTemplate("/docs/salary.csv{?columns*}")
        .AddParameter("columns", new List<int> { 1, 2 })
        .Resolve();
 
    Assert.Equal("/docs/salary.csv?columns=1&columns=2", url2);
}
</int></int>

The only difference between the two templates is the asterisk at the end of the parameter token.  The is called the “explode modifier“.  The additional bonus for hypermedia driven APIs that provide the templates to the client, is the client code can be completely ignorant of which approach is being used and the server can change its mind at some point in the future and nothing breaks.

Nested Nomenclature

The next example shows a technique developers use for including nested data as part of query string parameters

/emails?from[name]=Don&from[date]=1998-03-24&to[name]=Norm

Because of the clear separation between parameters and URI Templates, it makes this scenario fairly trivial.  Also, considering the potentially dynamic nature of the this type of query string parameters, another feature of URI Templates can be used to make this type of URL even easier to construct.

[Fact]
public void EncodingTest4()
{
    var url = new UriTemplate("/emails{?params*}")
        .AddParameter("params", new Dictionary<string,string>
        {
            {"from[name]","Don"},
            {"from[date]","1998-03-24"},
            {"to[name]","Norm"}
        })
        .Resolve();
 
    Assert.Equal("/emails?from[name]=Don&from[date]=1998-03-24&to[name]=Norm", url);
}

Query string parameter names are passed through the URI Template to the URL, untouched, which is why the square brackets are not escaped.  According to RFC 3986 they should be escaped.  However, it is fairly common to see them in URLs unescaped.  Although it is a violation of the rules, the impact is minimal because square brackets are currently only used in the host name for specifying IPV6 addresses.

Separating Syntax

The key to URI Templates being able help in URL encoding is that it is obvious which pieces are data and which pieces are URL syntax.  This allows the encoding to only be performed where it is needed, on the data.

Personally, I  do not think we need to resort to such extreme measures as JSON encoding parameters to make it easy for developers to safely construct URLs.  Hopefully, this post will convince a few other people.

Image Credit: Three body problem https://flic.kr/p/pNHgi5