API design is one of the more important skills possessed by web developers today. The reason behind this fact is that web applications are increasingly unable to live in isolation while at the same time providing the scope of features and web awareness users have come to expect. Developing for the web means, to a greater or lesser degree, being a productive citizen in a kingdom of online data exchange; and whether by formal engineering or organic growth, this means building an API.
But API design isn’t limited to pursuing the affable integration promises of the open web. Understanding the power of an API in relation to application design is just as valuable to business needs as sharing data across services is to user desires. The power, in fact, stems from the same source as every other factor in systems design: managing complexity.
Unless you’re writing a Turing Complete API then your guiding objectives will be to simplify the intricacies of your application down to a suitable input/output abstraction, and describe the application’s behavior with as natural and predictable language as possible. The API is a white box with many different black boxes inside. Its design should generally represent a boundary point after which any further encapsulation of logic would require unproven assumptions, or else seriously violate the separation of concerns, and a boundary point before which any more flexibility and power requires special operating knowledge of the system because the API is too dense to be self-descriptive.
This is a razor-fine line and easy to slip from when moving within the system, but an API that’s clear and approachable to strangers serves in an equally valuable capacity at documenting behavior and intent. In the same way that TDD promotes the ancillary yet no less important benefit of forcing the developer to conceive of their code’s structure, methods, organization, etc. in terms of real-world usage, an API can also provide a sort of blueprint allowing for a theoretical rehearsal of the design impossible with strict bottom-up development.
None of these notions are particularly controversial or open to much debate; the tension is drawn between the gap of theory and practice, and for that I want to look at some examples from some of today’s prevailing web services. Reflecting on the conclusions reached by web leaders should provide us with insight into our own design efforts, and read the collective pulse of the open web.
Twitter
Positive: Twitter has become the gold standard in API adoption among the developer community, and much of this owes to a lucid API design. The organization of methods under loose “resource” groupings is easy to understand, and the individual method documentation is clean, forthright, and contains exactly the information you would expect including usage examples, authentication details, parameter constraints, response data, and more. What’s more, you can usually expect the resources to answer in XML, JSON, RSS, and Atom formats, another area Twitter stands out.
The documents do an excellent job of providing implementation these details, alongside top-level summaries of the general API design and conventions.
Twitter’s greatest accomplishment, though, is managing information overload from the potential expressiveness of the system. To point: you can almost immediately write code to begin talking with the API after reading only a few pages of documentation, and that’s as good a litmus test as any.
Negative: There’s not much to dislike about Twitter’s API and documentation, but one uncomfortable area is the quasi-RESTful model it implements. While the resource structure is clear, Twitter’s naming conventions are arbitrary and depend on the developer’s learning custom vocabulary for utility methods. This is a challenge failed by most APIs, though, and while Twitter is exceptional in many positive ways, here the API is no exception.
Basecamp
Positive: The Basecamp API and documentation matches the 37 Signals adherence to simplicity, and you can familiarize yourself with the whole system in short order. The introduction covers implementation details like status codes, content types, and file uploads. If there’s anything to like about the API, it’s that you can learn it over a cup coffee.
Negative: For all its virtues of simplicity, the API lacks the cohesion found, say, in the only slightly more verbose Twitter API. The resources are well-named but the URLs are subject to unguessable nesting and inconsistent patterns. Searching and filtering data is not easily done, and the usage notes for those few methods which permit manipulation are hastily explained. As far as formats, only XML is supported.
Facebook
Positive: Facebook’s API sits at the opposite end of the design spectrum, with a rich, intricate framework behind it. The power of that framework and data stores gives the API its strength and more than justifies its complexity.
Facebook deals with this by progressive disclosure in the documentation, and a small ecosystem of sub-APIs for permissions, application authorization, database queries, a markup and widget language, and many things in between. If the learning curve is steep, the documentation does a respectable job of guiding the developer down through specific methods and use cases as needed, and everything is well-organized, indexed, and linked.
Negative: The downside of the breadth, depth, and power is obvious. To use the API you must essentially learn a new framework, along with its own pitfalls and nuances. The entire nature of the design revolves around specialist knowledge, with heavily parameterized methods and enough usage exceptions to make casual experimentation and rapid development less inviting than alternative APIs.
Flickr
Positive: Flickr supports a tremendously flexible API with an impressive number of both response (REST,XML-RPC,SOAP, PHP, JSON) *and* request (REST,XML-RPC,SOAP) formats. The documentation is descriptive, consistent, and concise, and the brief overviews on dates, encoding, tags, urls, and suchlike make further explanation in the method unnecessary.
Like many APIs, Flickr’s authentication is the most complex area of the API, which Flickr eases with instructive guides to authenticating from the web, desktop, and mobile platforms. The method naming is sensible, if not intuitive, while the method grouping is generally both.
Negative: Between support for SOAP and RPC calls, and a prescriptive approach to method naming conventions, Flickr demands slightly more special understanding than we would prefer, but with this as its greatest deficiency I find it to be on the whole well designed.
Digg
Positive: Unlike other APIs here, Digg is essentially an endpoint to a single resource: stories. The heavy parts of the API revolve around filtering the stories on multiple criteria, as can be seen from the route index. The API’s basic concepts are standard enough, and while Digg expects its consumers to cache and chain data requests to prevent system overload, the documentation clearly explains the request patterns intended for talking to the API with these constraints.
Data can be requested in a commendable variety of formats, from XML, JSON, Javscript, or serialized PHP. Each documented method also contains plenty of illustrative example response data.
Because of the focus on stories, there are fewer API methods and each method call can be customzied and purposed with a variety of optional arguments. The documentation systematically explains the exceptions and conditions under which parameters are used, and the design gives far more power to arguments than to routes, in other words more RPC than REST.
Negative: On the other hand, the argument-laden nature of the API methods require careful review and implementation, overhead that could have been avoided by consolidating the filtiering logic either in the documentation or the API itself. This also makes it difficult to grasp the full range of the API without serious review of all of the its corner cases.
The flood controls on API requests are explained well, but irritating to accommodate in contrast to something like Google Data which has a more passive approach to imposing usage restrictions on its consumers. Further requirements like, for example, a User Agent header, make Digg’s API feel less inviting and fault-tolerant than alternative designs.
eBay
Positive: eBay is firmly in the RPC design camp, and frankly, aside from desirable auction data, there’s not much to recommend the API. If the documentation seems exhaustive – and it most certainly is, see the guide to REST, for example – then it’s an honest emmisary of the beast of a system which sits behind an ostensibly friendly API frontend. Although there are detailed code samples and a development sandbox, these are more necessary complications than any kind of extra, facilitating support. The API does support XML, SOAP, Name Value, and JSON output, though, which makes sense beside the seeming design ambition of leaving no possibility unimplemented.
Negative: Challenges for the developer abound in the API. Issues in the documentation from PDF-only guides, broken hyperlinks, poor IA for a system large enough to clearly deserve it, inconsistent method implementation and naming schemes, duplication of functionality, you name it! If there’s a potential roadblock to eBay integration, you’re sure to find it somewhere in this API.
The documentation is a mix of structured, heavily formatted schema descriptions with confusing conventions on when to bold/italic, change header size, change fonts, etc., and writing that ranges from conversational to clinical in enough voices to give the impression of design by committee. The jargon required to follow the nine-plus API products (searching, shopping, matching, feedback, trading, etc, etc) is everything from dependency and monkey-patching, and nothing from an open, agnostic web. Individual method writeups are an embarrassment of riches with ample reference but no progressive disclosure and little indication as to why one thing is more important than another, or suddenly described at a different level of abstraction than the rest of the page, even digressing into legal or business discussion in the middle of a technical section.
Perhaps the most damning complaint is that there is no suitable top-down entry point for acquainting yourself with design of the API and the extent of its power, no way to understand the goals and constraints of the system or even a clear idea of what exactly it’s modeling.
Technorati
Positive: Technorati answers the same needs as Digg, but does so with a considerably more elegant (if a bit less flexible) API. In fact, it’s elegant enough that no introductory document is needed, and there are only a handful of methods you need to access its full functionality. Responses are formatted as XML or RSS, and further documentation is provided for setting up custom pings to the service.
Negative: Although the superficial depth of the API makes for swift development, API influence is largely relegated to reading, not manipulating, data. And while a simple system might otherwise lend itself to descriptive structure and resource names, Technorati is not much better at its taxonomy that much more complex APIs.
Google Data
Positive: The Google Data API is difficulty to appraise simply because it’s so diverse. Google Data imposes the governing design for dozens of Google services that each depart from, reduce, or extend the Base API to varying degress. While there is a solid conceptual overview, introductory video, and plenty of examples and framework-specific libraries, the price of admission to Google’s power is study and specialization.
In addition to the resources mentioned above, Google’s documentation acquits itself well in presenting information on the daunting scope of functionality and flexibility found in its APIs. Progressive disclosure, intuitive taxonomy of services, methods, and overall IA, and concise, consistently formatted documentation are the winning conditions that keep the API accessible and friendly.
Negative: It’s hard to fault google for complexity while understanding the design challenge, and especially in consideration of the fact that large but comparatively smaller APIs than Google (e.g. eBay) fail much more spectaularly at intelligble design and documentation.
Netflix
Positive: The Netflix API is very RESTful with a convincing explanation (and diagrams) behind how resources are organized and manipulated, and some clever tricks like an “expand” parameter for retrieving associated model data.
Documentation takes an instructional approach, walking through the various aspects of the API with perhaps too much depth at places, a distraction otherwise compensated for by the outstanding layout, formatting, example code, and succinct writing style. The extensive annotated screenshots are well chosen and never seem to clutter your search for information. A more formal reference for the API is available as well, where you’ll find the Netflix design to be quite respectful of RESTful principles. There’s even a lightweight Javascript API for basic Netflix actions.
In fact, the documentation is so excellent, if a little verbose, only because it reflects a tight, descriptive API design. In this case,I don’t mind the API writers erring on the side of explanation because it’s always clear how what you’re reading relates to the API as a whole.
Negative: Netflix stands out among rival APIs, both in thoughtful documentation and technical design. Perhaps its weakest point is the use of OAuth for authentication, considering the added complexity. XML is also the only supported format outside of their Javascript API.
