Caching data | AccountRight API

"There are only two hard things in Computer Science: cache invalidation and naming things."
Phil Karlton

In this article we'll explore how you can use HTTP responses and error codes to build API integrations that strike a balance between caching data for performance and building error resilient code.

When writing data to the API you'll almost always have to reference another resource. Invoices require reference to customers and tax codes, journal entries require accounts etc. It's logical then to first retrieve dependant data before using it. Repeating this process for every POST or PUT to the API adds a noticeable overhead so to improve performance it also makes sense to cache resources within your application.

However sometimes r,esources can change. Your client might update an important field through the AccountRight Live product interface or another API connection might make an update you're not aware of.

Harnessing HTTP headers

In the simplest case, you might want to check if a resource has been modified. The best way to achieve this is to store the Etag header of the resource and then use it with the If-None-Match header.

This means that, if there is no change to the data you're requesting, then you don't need to download the same data a second time.

Resource versions and updating data

When you retrieve data from our API we'll also include a RowVersion field. This is a sort of checksum of the data. When you modify a resource the RowVersion changes.

If you are attempting to update data via a PUT request to the API then you need to make sure your RowVersion is up to date. If it isn't (perhaps someone has modified the resource since you last retreived it) then you'll encounter an HTTP 409 Conflict error. That error looks a little like this:

{
  "Errors": [
    {
      "Name": "IncorrectRowVersionSupplied",
      "Message": "An Update operation requires the latest RowVersion for each of the existing entities being modified.",
      "AdditionalDetails": null,
      "ErrorCode": 111,
      "Severity": "Error"
    }
  ]
}

To resolve this conflict you first need to GET that resource and then determine the best method to apply your changes. This might be to prompt your client for input, or to re-apply your changes to the updated fields. how best to accurately apply your changes to the resource. Once you've resolved the conflict you can re-attempt the PUT a second time with the updated RowVersion.

Reusing data

If you are creating a resource via the API its highly likely that you’ll first need to reference other resources in the API. For example creating an invoice requires you to also provide the customer, tax code and accounts for the sale.

As a simple example, if you’re creating a new invoice for a customer its likely you’ll need to make a number of initial calls to the API:

GET Company/Preferences
GET GeneralLedger/TaxCode?$filter=Code eq ‘GST’
GET GeneralLedger/Account?$filter=Name eq ‘Sales - Other’
GET Contact/Customer?$filter=DisplayId eq ‘CUS000006’
POST Sale/Invoice/Service

If you’re creating multiple invoices, its highly likely that the company preferences and other data hasn’t changed between POST requests so you can re-use that data without retrieving it a second time. If you're creating many resources then your calls may look like:

// First get the preliminary data
GET Company/Preferences
GET GeneralLedger/TaxCode?$filter=Code eq ‘GST’
GET GeneralLedger/Account?$filter=Name eq ‘Sales - Other’
GET Contact/Customer?$filter=DisplayId eq ‘CUS000006’

// Create many invoices.
POST Sale/Invoice/Service
POST Sale/Invoice/Item
POST Sale/Invoice/Service
POST Sale/Invoice/Service

Unique identifiers in AccountRight Live

If you’re storing this data between API calls we’d recommend using the UID field as the unique identifier. The UID field is a uuid formatted identifier to uniquely reference a given resource. For example "38a37ae8-565c-46f8-ad4a-2a87069607f8" will uniquely identify a taxcode but may also, however unlikely, reference an account. Your caching/storage solution will need to account for composite keys, or store references to different resources separately.

Storing and refreshing data

If you are caching resources from the API its also important that you have the ability to refresh that data as it changes in the API. This means you need to both detect and then refresh changes in the API.

In the simplest case, you might cache data for a single session. A session could be the period of time a user is logged in, or it could simply be a collection of API calls such as a batch import. In more complicated integration such as a realtime ecommerce site then you may need to think about where the balance is for your solution. The end goal is to cache accurate data, and handle failures gracefully.

Handling out of date data

If you're caching data then there's a chance you might find that someone deletes a cached resource. This is tricky because you cannot retrieve it again and merge like you can with a 409 conflict.

In these cases, you'll need to use your best judgment on how to respond. In some cses you may be able to automatically recover by refreshing your cache and applying some safe defaults.  In other cases you might need user input to update their configuration and mapping.

For example if a client deletes or merges a tax code that you have cached you'll receive a validation error that the tax code no longer exists:

{
  "Errors": [
    {
      "Name": "ValidationError",
      "Message": "TaxCodeNotFound",
      "AdditionalDetails": "UID",
      "ErrorCode": 6000,
      "Severity": "Error"
    }
  ]
}

Because this relates to your clients configuration and mapping you may need to prompt them to configure a new tax code to use (e.g. through your admin or configuration screens).

By implementing graceful error handling and applying well thought out caching strategies, your API integration will be faster for customers to use and resilient to changes in the API.