About NearSt
NearSt is a retail technology solution that attracts online shoppers to physical store locations. The goal is to make physical shopping as easy for customers as online shopping at Amazon. The NearSt platform acts as a big data pipeline, taking data from over 200 different point-of-sale systems in thousands of physical stores and transforming billions of daily data points, including barcodes, quantities, prices, etc., into rich product profiles that can be shown online. By doing this, when someone searches Google for a product, they will see local stores in their area that sell that specific product alongside the typical online results for their search term.
Embracing microservices for faster development
NearSt comprises a small yet potent team of approximately 20 individuals, with six dedicated to engineering. Operating in a landscape where data retrieval from 200 distinct point-of-sale systems is crucial and dissemination to platforms like Google, Facebook, and Instagram is imperative, NearSt has prioritized agility and nimbleness. The key to this operational dexterity lies in adopting a microservices architecture, predominantly relying on Lambda functions.
The foundation of NearSt’s system is constructed upon an event-driven architecture, predominantly utilizing Lambda functions, with an extensive reliance on SNS, SQS, and EventBridge, and recently extending the use of step functions into their workflow. The strategic decision to embrace microservices capabilities within the AWS ecosystem has freed the team from some of the burdens of infrastructure management. Consequently, the focus has shifted towards rapid iteration, launching initiatives swiftly, and dedicating time to feature development rather than infrastructure upkeep.
This microservices approach has accelerated the software delivery process and resulted in significant time savings, equivalent to months of effort. The team highlights the ability to spin up testing resources effortlessly at negligible costs, enabling comprehensive integration tests in a real-world environment and ensuring a high level of confidence that successful test outcomes will translate to seamless performance in live scenarios.
Navigating the complexity of numerous Lambdas
NearSt faces a formidable challenge with a multitude of purpose-built Lambda functions, numbering in the hundreds. The primary hurdle lies in efficiently tracking and diagnosing errors occurring in real-time. Thomas Schoffelen, Co-Founder and Head of Engineering, sheds light on this challenge, stating, “The distributed nature can make it quite time-consuming to debug issues. In a distributed environment, when an issue arises, one must inspect various elements such as Lambda configuration, lambda logs, or DynamoDB activity. This process becomes increasingly time-consuming.”
Marton Veto, Senior Software Engineer at NearSt, shares insights into their evolving perspective, noting, “Observing and troubleshooting this growing serverless system became increasingly challenging for us. As our platform has evolved, we have been looking for more robust monitoring.” NearSt began looking for a solution so that if there was an error, they could quickly see the request that initiated the flow. NearSt conducted tests with several observability tools to find the right solution, ultimately selecting Lumigo as the optimal solution to meet their specific needs. Or, as Marton put it, “Looking at the other platforms we tested, they didn’t feel as mature as Lumigo did. Lumigo seemed like the only platform that matched our needs regarding maturity, stability, and feature set.”
Cutting debugging time to 30 seconds
With Lumigo, NearSt has streamlined the process of identifying errors and tracing lambda functions with remarkable efficiency. Joe Leggett, Software Engineer at NearSt, underscores the significance of Lumigo by stating, “Seeing the full request and response payload was quite important because everything we do is event-driven. In the event of an error, we can pinpoint the exact request that initiated the entire flow. Having a comprehensive view of the transaction from start to finish is paramount to us.” With Lumigo facilitating issue debugging, NearSt has dramatically slashed troubleshooting time from over half an hour to a mere 30 seconds, leaving developers with more time to iterate on feature development rather than debugging rapidly.
Leveraging Lumigo’s Explorer tab, NearSt’s engineers capitalize on the search functionality, enabling swift searches for user IDs or barcodes to visualize the entire pipeline instantly. According to NearSt, the Explorer tab has significantly enhanced the visibility and accessibility of their serverless environment for monitoring purposes. Marton further emphasizes the utility, stating, “I keep rediscovering how easily I can search for anything. It’s just a great feature.” NearSt has maximized Lumigo’s Execution Tags to augment their search capabilities, simplifying the process of locating specific elements of interest.
Beyond the Explore Tab, Lumigo’s issue pages have proven invaluable to NearSt’s on-call team. Serving as a focal point during on-call rotations, the Issue pages offer early signals for potential issues even before formal alerts are triggered. As NearSt refines its alerting setups, Lumigo’s Issues pages continue to serve as a proactive resource for identifying issues that might not immediately trigger alarms.
About Lumigo
Lumigo is a troubleshooting and observability platform that autonomously deploys OpenTelemetry in under 5 minutes with a single click, automatically capturing and contextualizing all of the data developers need to troubleshoot microservice issues in production. Lumigo is the only distributed tracing platform that enriches traces with complete in-context request and response payloads and correlates them to the relevant logs and metrics, enabling developers to resolve issues up to 80% faster.
