Kin Lane asks Where Do Developers Get Idea That APIs Should Never Go Away?,

Why do developers feel so burned when a popular API like Netflix, ESPN, or Google Translate is deprecated? The main reason is lack of communication. Developers need a long runway that an API is going to be deprecated.

This isn’t why.

Developers hate it when APIs go away because it means they have to rebuild something they already built once, and because the discontinuance will cause apparent bugs in their product for their end users.

For reasons including process and toolset churn and environmental changes, developers often find themselves writing the same functionality over and over. Things that tend to encourage rewriting something already written, rather than allowing the developer to write something new, will cause frustration. It’s hard to get ahead on a treadmill.

Things the developer already sold will break, and the end users are uninterested in the decisions of some third party to deprecate an API.

The analogy with physical products

Kin again:

I cannot think of a single business, where you select a supplier of products or services you depend on for your business, without any chance of that company going bankrupt, shutting down, being sold, or possibly just discontinuing providing a particular products or services you depend on.

I’ve never sourced parts from vendors for a physical product (that shipped), but I can certainly imagine some irritation at a supplier which chose to discontinue a line of parts without some lead time. But what would we use that lead time for? In the case of a physical product, we’d look for replacements, think about redesigning our product to use other parts, and, crucially, we’d buy a stockpile of these parts in whatever amount we felt appropriate. Modulo difficulties like decay or cash on hand, in this situation we as the manufacturer can build in our own buffer for when we have to do something drastic like redesign our product. When we use a third-party API like Netflix or ESPN, there is usually no way to stockpile API requests for when we need them later.

We could cache, but there are a bunch of other circumstances that prevent us from caching as much as we’d like: the client might not have a lot of storage, terms of service might prevent us, the request and/or response might be time-sensitive, etc. Nevertheless, I think caching is a good fit for a lot of API uses, and API client developers don’t use it enough.

However, there’s a separate problem: programs which shipped weeks, months, or years ago will now break as the API they depend on goes away. Even if the product is caching, it’s unlikely to need only the same data over and over (else we would have just built that data into the product, no?). In the best case, the product functionality will degrade over a period of days or weeks, and in the more common worst case, the product turns into a brick or a useless waste of space on the devices of people who may well have paid good money for it.

It does happen in the physical product world, of course, that some products depend on third-party services or products after shipping. My appliances don’t work correctly without constant supplies of electricity and/or water. The furnace needs filters at regular intervals, and certainly didn’t come with a lifetime supply of them. My car needs oil and gasoline. There’s an interesting difference in the situation between an app that uses Google Translate and all these physical product scenarios, though. In the physical product world, the application user is expected to arrange supply of the third-party parts. In some cases, this is easy because there are multiple providers, and in some cases it’s easy because the product is essentially the same no matter what single supplier you have (water, for example). In the world of apps, this isn’t as easy, since APIs aren’t often compatible with each other, and even if they were, this kind of solution would need some place for the user to enter the API endpoint. Most API clients probably do their work without the end user ever being aware of the source API at all.

Sometimes there is a very specialized third-party product required for the end use product. This often results in the very specialized product being made available directly from the manufacturer of the end use product. I have a Nissan with a CVT, which requires a particular sort of fluid for the transmission, not widely available. Nissan sells this fluid itself, in effect caching the supply from the manufacturer and insulating me from any period where the manufacturer might discontinue the product, requiring Nissan to find another supplier. It might be that Nissan developed and manufactures this fluid in-house, but the point is that I don’t have to care, because my interface to it is through Nissan: they are hosting the supply needed by the product they sold me.

When a widely used third-party product is phased out, we often get a lead time greater than the mean lifetime of the products that use the third-party product! For example, automobile engines used leaded gasoline (at least in the US) to prevent knock, and when that was discontinued, automobile manufacturers and users were given about twenty years, or over twice the typical lifetime of a car in the 1970s.

Do we need to provide every API for twenty years? No. But we do need to provide an API (which is in use) for enough time that end users will usually not notice it has gone away, which means at least the typical lifetime of the product you encouraged use by. In general, people use application and software-based devices for times a lot shorter than the lifetime of a car (even a 1970s car), due to upgrade pressure from other sources. For a lot of applications, two to five years is probably enough time to provide an API after deprecation announcements.

Committing up front to supporting a (probably free) API for many years after you’ve decided to discontinue it might seem like quite a burden, and might even discourage an API provider from exposing it at all, so here’s some alternatives.

Deprecate before launch

Announce the deprecation information in the launch documentation. If, when it launches in 2015, the initial documentation for AwesomeAPI v1.0.0 says that v2.x.x will be preferred past May 2017, and that v1.0.0 will be discontinued in June 2019, client developers will feel secure in using your API if and only if they ought to feel secure for their use case. As Kin points out,

[client API developers] should carefully research your vendors, make sensible choices about why you use one over another

Providing full information about when an API will be deprecated before anyone is using it facilitates that research.

Use flexible architectures

Often we want to deprecate an API not because we’re not providing the information any more, but because there’s a newer API which has advantages for us or the clients, or both. The best deprecation is no deprecation at all, so try to design your API such that it can be extended and changed. Hypermedia can assist with this.

Encourage caching

Try to design your app to separate cachable information from time or location sensitive information, so that app functionality isn’t impacted where it need not be.

Use your terms of service to promote caching as much as possible.

Enforce lifetimes

Providing one to two lifetimes of product use before discontinuance of an API is only possible if the lifetime in question starts about the same time you announce deprecation. New cars sold in the US after the mid 1970s couldn’t use leaded gasoline, which meant that the deprecation clock had a definite start. If cars that needed lead had still been sold, as in the UK, the applicable lifetime would have started later, requiring some sort of shim replacement.

One way to avoid this is to have a term of service that prohibits developer distribution or sales of products using the API at some point after deprecation is announced. In this way, you can ensure that client developers are thinking about providing a new version of their product or some other workaround early. Or, if not, at least you have done what you could.

Client developers

Client developers should mirror many of the suggestions above. If it’s convenient to cache long term on the client, do so. Ideally, cache as much as possible for the expected lifetime of the app or device. Prefer using providers who support hypermedia APIs if you have a choice so that providers can make minor changes without disruption.

Ideally, if your app or device depends strongly on some non-unique API and the terms of service allow it, host an API that uses the source API in its backend. Controlling both sides of an API is the best way to ensure longevity, and if you’re caching aggressively on your server, it can be possible to switch to another backend provider without any change to your app or device at all.

Take away

So, learning some lessons from physical manufacturers, let’s sum up:

As an API provider:

• be clear about lifetime of the API
• ideally give notice of deprecation early enough for two product lifetimes (US leaded gasoline strategy), but one product lifetime is probably good enough
• announce limited lifetimes for your API at launch to assist client developers in planning
• allow caching when upstream licensing permits and it makes sense
• use API architecture that can be changed without breaking conforming clients (hypermedia, for example)
• consider terms of service requiring downstream deprecation

As an API client:

• cache on the client whenever convenient and permitted
• prefer hypermedia APIs which can (hypothetically) change without breaking your app
• host your own server and cache more aggressively on it if possible and permitted (Nissan’s CVT fluid strategy)