The team relied on microservices to build applications, AWS Step Functions to distribute them, and invoked Lambda functions to automate routine events with logic. But the challenge came when developers realized that closely tracking their collection of many, smaller microservices was complex and time-consuming. If something went wrong, it was difficult to find the cause or know where to start looking. Even though they had finished developing their serverless system, the Sonos team determined that working its way backward from a problem was not going to be sustainable after launch, when consumers had items waiting in their carts.
In order to gain visibility into the stack, the team integrated Lumigo, a solution that is complementary to AWS serverless.
To start, Sonos added Lumigo to its Lambda library and Amazon API Gateway to get the monitoring system up and running quickly. Sonos connected its notifications channels to Lumigo to notify Sonos in real time whenever a problem arose. In addition to catching bugs or performance issues, Lumigo flagged items that helped the team plan ahead during early development.
“We could see when our Lambda services were nearly out of memory or over their memory limit and could quickly fix it to make applications more performant,” said Jonathan Lemon, a software engineering manager at Sonos.
Sonos has a highly distributed system that employs an array of third-party tools, and because the development team was so focused on building applications, monitoring across all of them in-house was challenging. With Lumigo, the team can observe Sonos’ entire technical stack across its serverless ecosystem—the Lambda services that interact with Sonos’ existing backend systems; AWS AppSync logs; Salesforce Commerce Cloud; and Sanity CMS, which unifies the website’s content. When there’s an incident, the Sonos team knows exactly where to go in the stack grid—and Lumigo provides guidance on what to do next.
For example, Sonos creates typescript Lambda functions in its internal database, both behind its Amazon Virtual Private Cloud (VPC) and outside of it. Lumigo can interface with the new website Lambda functions inside or outside the VPC. Since Sonos receives the necessary information from Lumigo upfront, the developer team doesn’t need deep technical insight to solve the problem. This centralized, supportive process and improved time to resolution is what helped Sonos launch its new site four months after engaging Lumigo.
The monitoring service has been invaluable in identifying rogue processes, something the Sonos team needed as it built out the website architecture with iterative changes. In one instance, the development team saw an irregular spike in their invocations and Lumigo was able to help them quickly identify a test script that had been left running. “With Lumigo, we were able to catch issues early and resolve them quickly,” Jonathan said.
Lumigo’s ability to connect the dots and quickly identify root causes gives Sonos full control over its technical stack, which inspired Sonos to broaden Lambda usage by eight times. Today, Sonos executes up to 30 million invocations a month with an error rate of 0.01%. Because the company can use Lumigo’s tooling immediately after any deployment, Sonos has reduced its Mean Time to Repair (MTTR) by a staggering 80%. “We were able to take the new Sonos.com to production with the utmost confidence, knowing it would scale with minimal errors,” Jonathan said. “I don’t think our website would be where it is today without Lumigo.”
Since Lumigo only charges based on resource usage, Sonos is able to save on serverless architecture management costs. “That’s one of the key benefits of serverless,” said Timi Petrov, lead solution engineer at Lumigo. “During those times when accounts see low traffic, our customers don’t have to pay for services.”
The success that Lumigo and AWS have helped Sonos achieve in a serverless environment is why the sound experience company plans to expand serverless beyond its software development team—and it’s taking both partners along.