Commerce Layer’s CDN caching strategy at the edge.

Seth:

How does Commerce Layer cache data?

Massimo:

We cache transactional data that is used to show things like the prices of a SKU and stuff like that. When a frontend client (web, app, smart device, whatever) makes a request for Commerce Layer transactional data, the request is managed by the Fastly and the transacitonal data is returned.

Our setup is tied directly to the design of our authentication pattern and the fact that we are a client-side API

• We authenticate a “sales channel” role with a set of restricted access permissions tailored for public use. The sales channel is treated as if it were a visitor with typical guest permissions.
• Based on the guest permissions, the Sales Channel can now access a set of resources we cache in a dynamic way.

We cache a few key resources like SKUs, prices, and availability. Other stuff, we obviously cannot cache, the big one being the order. Again, we are focused on edge caching, not on the caching of our application. (That is content for a future interview.)

Seth:

It sounds like Fastly also does a lot of the heavy lifting here. What does Fastly do during this process?

Massimo:

We use Fastly’s service, which has migrated away from Varnish in order to expand their optimization opportunity. They provide very dynamic caching, and we continually work on optimizing our implementation with them. We strike a balance between maintaining fresh data and delivering the fastest response possible. Both aspects are very typical considerations. However, in the case of our transactional engine, transactional content like prices and availability is always changing.

Fastly invalidates the cache and updates data based on our configuration. Our aim is for optimal performance. And this proves to be pretty challenging when compared to other services that cache “normal website content”, which may not have as frequent changes as commerce data can have. There's so much dynamic transactional data, particularly for large product catalogs with thousands of SKUs, bundles, business rules, inventory, etc. We have used the tools that Fastly provides to configure our settings to reliably update with this ever-changing data. And Fastly is really responsible for the magic here.

Seth:

Anything specific you can mention about that Fastly configuration?

Massimo:

To do this effectively, we used Fastly’s tools to create rules-based events and actions taken when those specific events take place. Fastly provides surrogate keys and we use them to organize cache. They describe surrogate keys as “a system that makes it possible to quickly purge related content.” We use them to maintain a hierarchy of objects that relate directly to Commerce Layer’s data resources. The hierarchy starts with the environment, goes down to a set of resources, and finally down to a single resource. In total, we have four levels to our keys:

• the organization
• the environment
• the resources
• the single resource

Seth:

Any examples you can share that would help bring this to life with a real example?

Massimo:

Let's say a web client requests a set of SKUs. Because Commerce Layer creates a hierarchy using Fastly’s surrogate keys, a SKUs included in that response is a single resource at the lowest level of our hierarchy. Whenever one of the SKUs changes, Fastly invalidates the corresponding key, which in turn invalidates all the requests held within that key.

In the past, we were really “over-optimized” toward performance. But this started to create some support issues. So we recently changed our setup to always provide live data. We do this also at the first request after cache gets cleared. We have seen that this setup creates less support issues, while not affecting performances too much.

For Fastly to update the cache, their proxy server sends a request to Commerce Layer’s core API at our origin server, and we deliver a response from our backend database. Fastly updates the cache and revalidates with updated surrogate keys.

Seth:

What if the cache changes exactly at the same time as an order is taking place? Isn’t there a chance for errors in the order?

Massimo:

I have been asked whether a loophole might exist where old data gets passed through to a client via API request that might result in an incorrect order…and a soon-to-be unhappy customer. The simple answer is no.

Let's say that there's a very popular SKU in a product catalog that’s coming to market. A product manager decides to update the SKU availability and price due to supply issues. When the SKU resource gets updated, Fastly starts the process of invalidating cache (described above).

There is no chance that an order with the former price could slip through because we do not use the cached data when creating the order object that shows up in a cart! At the point of creating the order object, the frontend makes a direct request to Commerce Layer’s core API and we add the most updated data from the database, in this case, the updated price and availability of the popular item. We always use real data when the order object is created by going directly to Commerce Layer’s servers and not to the data stored in the cache on the CDN. The order data is never cached for obvious reasons.

Seth:

What are the specific performance gains that our APIs can deliver to a customer’s ecommerce application?

Massimo:

Our core API is fast. We often observe a response time of 40 milliseconds at the application level. But let’s stay focused on the edge. Response times can vary depending on network speeds and the physical location of a device. Our global average response time is 89ms, but since many requests come from Europe and North America, the responses are actually much faster.

And while our core API is fast, those are not the fastest response times we observe. Because our client-side API runs from the caching scheme described above, the response times for cached data can be very fast. Cached resources on Fastly’s CDN can be up to 3ms to 5ms…imperceptible to a human being.

Seth:

Are there other things that add to this speed?

Massimo:

Our ability to serve data this quickly and reliably comes down to Commerce Layer's singular focus on building the world’s best transactional engine. We invest in fine tuning our APIs. Another aspect is that we follow the JSON API standard, which is the starting point for delivering data as fast as possible. We also focus on optimizing our database. There's a very long list of optimizations that we've done to minimize our API response time. Some tweaks might only result in a 1% advantage, but because we can focus on performance, they add up over time. Recently, we made a database optimization that resulted in a 10% improvement in speed.

Seth:

What about other platforms in the ecosystem? Specifically monoliths. Can they do these optimizations?

Massimo:

When it comes to monolithic platforms, their data model is more rigid and opinionated with a lot of structured relationships between resources. They can’t cleanly segment content data from transactional data, and the result is that the caching of transactional data is nearly impossible. Every API call has to include certain things just to make everything work, which is why their speed is slow and hard to optimize.

Seth:

Any customer stories you can share about all this?

Massimo:

We interviewed a developer from a customer who was beginning a migration from Adobe Commerce (aka Magento) to Commerce Layer. They told us they had optimized Magento extensively, even building their own Varnish cache layer over everything… except their transactional data. Their content and product information could be stored in cache, but the transactional data required direct API requests to their Adobe database. Among the many considerations they weighed to go composable, the speed of our client-side API and caching of data helped them choose Commerce Layer as their transactional engine.

On another occasion, I met a customer at the Headless Commerce Summit in London in May. The first thing he mentioned to me was how happy he was with the performance of Commerce Layer’s API. He was struck by the performance that was, in his opinion, really very different from the other platforms. We discussed Fastly, surrogate keys, and how we cache based on the above-mentioned hierarchy. His quote, not mine: “I don't know how you do that, but it’s amazing.”

Seth:

Is there an upper bound to all of this? What’s next when it comes to performance?

Massimo:

Sure, some things can’t go much faster, other aspect depend on services like Fastly.

Commerce Layer always looks at our technical infrastructure setup to make sure we can deliver. We think about optimizing metrics like number of transactions per second and scaling the absolute number of transactions that we can manage in a specific time window.

We have our sights set on other frontiers like accessing data in a more localized setup. We can look at ways to create instances of data in specific geographic areas to minimize round-trip requests times to our servers. That will allow customers to work on those specific instances based on their geographic location.

Seth:

Thanks, Massimo. That’s a wrap.

---

We hope you’ve enjoyed reading the overview of our caching strategy and how we build fast APIs. Our goal was to peek into our high-level approach that uses client-side APIs, Fastly cache, and micro frontends to achieve maximum performance. 

Commerce Layer’s goal is to build the best transactional engine in the world, making transactions as fast, efficient, and accurate as possible. That way, you can rely on us to power terrific content-led ecommerce.