Improve security and user management with our Authentication API.

Product

Commerce Layer’s CDN caching strategy at the edge.

November 8, 2022 Seth Bindernagel

Our customers often wonder how Commerce Layer delivers such fast API response times with accurate transactional data across multiple clients, whether it’s a website, an app or any digital channel.

To answer this question, I decided to sit down with the Commerce Layer CTO Massimo Scardellato to discuss a lot of the core elements that make this possible. We discussed just the part of our caching at the edge servers powered by our CDN. This article does not dive into our application caching, which is the other critical part for how we deliver the fastest service possible.

Here is the Q&A we did, edited slightly for this blog post.


Seth:

To kick it off, how has Commerce Layer created fast responses to API requests made to the edge servers in our CDN?

Massimo:

It comes down to two things:

  • We have a sophisticated caching strategy that uses Fastly’s caching service on their CDN.
  • We are always working on improving the performance of our Core API. (Note, this also includes optimizing our application caching to gain performance improvements, but that is not the focus of this conversation.)

The result is that we can use our micro frontends and APIs, which are consumed on the client-side, to bring data directly to the user experience. This removes the need for complex orchestration layers. The fast response times directly correlate to improved sales conversions. This results in our customers focusing more on creating great content, rather than spending time on optimizations we manage.

Seth:

What are some specific technical decisions we have made that help us deliver those step-function improvements in speed?

Massimo:

A big one for us is that Commerce Layer offers an API on the client-side to manage nearly all of the transactional data stored by our customers. Client-side requests can be made to our APIs because we optimize them for a speedy response to deliver essential transactional data required to sell products. In practice, we cache resources on the Fastly CDN so that any request for data from a frontend client has very speedy turnaround times.

Seth:

OK, I’m technical enough, and understand cache at a high level, but maybe we should give a quick primer on that. What is cache and why is it important to the performance of Commerce Layer?

Massimo:

Commerce Layer uses Fastly as our CDN. Fastly defines cache as “a location that temporarily stores data for faster retrieval by the things that need to access it.” They manage a broad network of proxy servers where they store customer data “on the edge”, a term that describes a strategically-placed set of servers that provide users with cached versions of content from origin servers like Commerce Layer’s. We cache our customers’ transactional data on Fastly’s edge network

Seth:

How does Commerce Layer cache data?

Massimo:

We cache transactional data that is used to show things like the prices of a SKU and stuff like that. When a frontend client (web, app, smart device, whatever) makes a request for Commerce Layer transactional data, the request is managed by the Fastly and the transacitonal data is returned.

Our setup is tied directly to the design of our authentication pattern and the fact that we are a client-side API

  • We authenticate a “sales channel” role with a set of restricted access permissions tailored for public use. The sales channel is treated as if it were a visitor with typical guest permissions.
  • Based on the guest permissions, the Sales Channel can now access a set of resources we cache in a dynamic way.

We cache a few key resources like SKUs, prices, and availability. Other stuff, we obviously cannot cache, the big one being the order. Again, we are focused on edge caching, not on the caching of our application. (That is content for a future interview.)

Seth:

It sounds like Fastly also does a lot of the heavy lifting here. What does Fastly do during this process?

Massimo:

We use Fastly’s service, which has migrated away from Varnish in order to expand their optimization opportunity. They provide very dynamic caching, and we continually work on optimizing our implementation with them. We strike a balance between maintaining fresh data and delivering the fastest response possible. Both aspects are very typical considerations. However, in the case of our transactional engine, transactional content like prices and availability is always changing.

Fastly invalidates the cache and updates data based on our configuration. Our aim is for optimal performance. And this proves to be pretty challenging when compared to other services that cache “normal website content”, which may not have as frequent changes as commerce data can have. There's so much dynamic transactional data, particularly for large product catalogs with thousands of SKUs, bundles, business rules, inventory, etc. We have used the tools that Fastly provides to configure our settings to reliably update with this ever-changing data. And Fastly is really responsible for the magic here.

Seth:

Anything specific you can mention about that Fastly configuration?

Massimo:

To do this effectively, we used Fastly’s tools to create rules-based events and actions taken when those specific events take place. Fastly provides surrogate keys and we use them to organize cache. They describe surrogate keys as “a system that makes it possible to quickly purge related content.” We use them to maintain a hierarchy of objects that relate directly to Commerce Layer’s data resources. The hierarchy starts with the environment, goes down to a set of resources, and finally down to a single resource. In total, we have four levels to our keys:

  • the organization
  • the environment
  • the resources
  • the single resource

Seth:

Any examples you can share that would help bring this to life with a real example?

Massimo:

Let's say a web client requests a set of SKUs. Because Commerce Layer creates a hierarchy using Fastly’s surrogate keys, a SKUs included in that response is a single resource at the lowest level of our hierarchy. Whenever one of the SKUs changes, Fastly invalidates the corresponding key, which in turn invalidates all the requests held within that key.

In the past, we were really “over-optimized” toward performance. But this started to create some support issues. So we recently changed our setup to always provide live data. We do this also at the first request after cache gets cleared. We have seen that this setup creates less support issues, while not affecting performances too much.

For Fastly to update the cache, their proxy server sends a request to Commerce Layer’s core API at our origin server, and we deliver a response from our backend database. Fastly updates the cache and revalidates with updated surrogate keys.

Seth:

What if the cache changes exactly at the same time as an order is taking place? Isn’t there a chance for errors in the order?

Massimo:

I have been asked whether a loophole might exist where old data gets passed through to a client via API request that might result in an incorrect order…and a soon-to-be unhappy customer. The simple answer is no.

Let's say that there's a very popular SKU in a product catalog that’s coming to market. A product manager decides to update the SKU availability and price due to supply issues. When the SKU resource gets updated, Fastly starts the process of invalidating cache (described above).

There is no chance that an order with the former price could slip through because we do not use the cached data when creating the order object that shows up in a cart! At the point of creating the order object, the frontend makes a direct request to Commerce Layer’s core API and we add the most updated data from the database, in this case, the updated price and availability of the popular item. We always use real data when the order object is created by going directly to Commerce Layer’s servers and not to the data stored in the cache on the CDN. The order data is never cached for obvious reasons.

Seth:

What are the specific performance gains that our APIs can deliver to a customer’s ecommerce application?

Massimo:

Our core API is fast. We often observe a response time of 40 milliseconds at the application level. But let’s stay focused on the edge. Response times can vary depending on network speeds and the physical location of a device. Our global average response time is 89ms, but since many requests come from Europe and North America, the responses are actually much faster.

And while our core API is fast, those are not the fastest response times we observe. Because our client-side API runs from the caching scheme described above, the response times for cached data can be very fast. Cached resources on Fastly’s CDN can be up to 3ms to 5ms…imperceptible to a human being.

Seth:

Are there other things that add to this speed?

Massimo:

Our ability to serve data this quickly and reliably comes down to Commerce Layer's singular focus on building the world’s best transactional engine. We invest in fine tuning our APIs. Another aspect is that we follow the JSON API standard, which is the starting point for delivering data as fast as possible. We also focus on optimizing our database. There's a very long list of optimizations that we've done to minimize our API response time. Some tweaks might only result in a 1% advantage, but because we can focus on performance, they add up over time. Recently, we made a database optimization that resulted in a 10% improvement in speed.

Seth:

What about other platforms in the ecosystem? Specifically monoliths. Can they do these optimizations?

Massimo:

When it comes to monolithic platforms, their data model is more rigid and opinionated with a lot of structured relationships between resources. They can’t cleanly segment content data from transactional data, and the result is that the caching of transactional data is nearly impossible. Every API call has to include certain things just to make everything work, which is why their speed is slow and hard to optimize.

The impact of slow site performance.

Seth:

Any customer stories you can share about all this?

Massimo:

We interviewed a developer from a customer who was beginning a migration from Adobe Commerce (aka Magento) to Commerce Layer. They told us they had optimized Magento extensively, even building their own Varnish cache layer over everything… except their transactional data. Their content and product information could be stored in cache, but the transactional data required direct API requests to their Adobe database. Among the many considerations they weighed to go composable, the speed of our client-side API and caching of data helped them choose Commerce Layer as their transactional engine.

On another occasion, I met a customer at the Headless Commerce Summit in London in May. The first thing he mentioned to me was how happy he was with the performance of Commerce Layer’s API. He was struck by the performance that was, in his opinion, really very different from the other platforms. We discussed Fastly, surrogate keys, and how we cache based on the above-mentioned hierarchy. His quote, not mine: “I don't know how you do that, but it’s amazing.”

Seth:

Is there an upper bound to all of this? What’s next when it comes to performance?

Massimo:

Sure, some things can’t go much faster, other aspect depend on services like Fastly.

Commerce Layer always looks at our technical infrastructure setup to make sure we can deliver. We think about optimizing metrics like number of transactions per second and scaling the absolute number of transactions that we can manage in a specific time window.

We have our sights set on other frontiers like accessing data in a more localized setup. We can look at ways to create instances of data in specific geographic areas to minimize round-trip requests times to our servers. That will allow customers to work on those specific instances based on their geographic location.

Seth:

Thanks, Massimo. That’s a wrap.


We hope you’ve enjoyed reading the overview of our caching strategy and how we build fast APIs. Our goal was to peek into our high-level approach that uses client-side APIs, Fastly cache, and micro frontends to achieve maximum performance. 

Commerce Layer’s goal is to build the best transactional engine in the world, making transactions as fast, efficient, and accurate as possible. That way, you can rely on us to power terrific content-led ecommerce.