BLOG
< Back Aviad Mor | Nov 14 2019

EventBridge vs CloudWatch Events, Kinesis and SNS


AWS recently launched the Serverless Event Bus, AWS EventBridge. Some have described it as an extension of CloudWatch Events, while others say it is provides the same features as AWS SNS.

So, what exactly is EventBridge? And how does it fit into the existing AWS serverless ecosystem?

In this article we’ll explore the reasoning behind its introduction, its functionality, and its advantages (and limitations) compared to existing services you might be using, such as CloudWatch Events, SNS and Kinesis.

Finally, we’ll walk through some simple use cases, demonstrating how utilizing EventBridge compares to using those more familiar AWS services. 

What is AWS EventBridge?

EventBridge brings together your own (legacy) applications, SaaS (Software-as-a-Service), and AWS Services. It can stream real-time data from various event sources like PagerDuty, Symantec and routes to various targets (AWS services) like SQS, Lambda, and others. 

At launch EventBridge arrived offering support for 10 SaaS application partners, more than 90 AWS services as event sources, and 17 AWS services as targets. And this list will undoubtedly grow over time

In simple terms, AWS EventBridge is an event bus that supports the publish/subscribe model. Applications can publish events to it and it can fan-out to multiple target services. 

Now, let’s answer the  basic question: what’s new in it? The event bus is far from a new concept, and AWS itself already provides that functionality through CloudWatch Events. 

The Genesis of EventBridge

EventBridge has been introduced mainly to address the problems of SaaS platform integration with AWS services. 

This is critical, because in the current cloud world, SaaS platforms like CRMs, identity providers, and so on, have become key partners. They generate a large amount of data and need to pass it to AWS platforms for business processing. Before EventBridge, there were two main solutions for sending event data to AWS:

Polling

With this solution, we generally set up a Cron job or CloudWatch Scheduler to go out and call the SaaS APIs. It will check if there has been any change in data and then pull the data. Polling frequency can be minutes to hours depending on the use case and the capacity of the particular SaaS platform to bear the load. A typical flow might look like this:

This solution seems simple but it raises two major issues:

  • Data freshness issue – The scheduler will be calling the API at set intervals, maybe every few minutes or perhaps only once per hour. So, it will never get real-time data. There will always be a gap, which might not be suitable for some business scenarios. 
  • Costing and Performance issues – To alleviate the data freshness issue we could choose to reduce the interval of polling, but that will increase costs as call frequency will be increased. Additionally, more resources will be consumed on the SaaS platform side. This may cause throttling and slow down performance.

So, the overall recommendation is to avoid the polling mechanism if you can.

SaaS Webhooks

This technique eliminates the data freshness issue. Here, we find out an HTTP endpoint of the AWS-hosted application which the SaaS platform can call to send event data. The SaaS platform will set up the webhooks and send real-time data when records change.

Here’s an example of a typical use case:

A schematic of a typical use case for AWS EventBridge.

In this flow, we still need to manage the public endpoint of the application for handling security/DDoS attacks. It will also require Authentication handling. In AWS, this is usually done through API Gateway or WAF/ALB. We would also need to write the code to handle the events.

So, in light of the shortcomings inherent in the existing techniques, AWS introduced EventBridge, which enables SaaS platforms to create a Native Event source on the AWS side and establish a secure connection just by sharing Account ID and region with platforms. 

It not only solves the issue of real-time data processing but also takes care of event ingestion and delivery, security, authorization, and error handling for you.

EventBridge handles event ingestion and delivery.
Image source

Now, let’s take a look at the other options available in AWS for Event routing and see how they compare with EventBridge. There are four main AWS-native options for event routing:

  • CloudWatch Events
  • SNS
  • EventBridge
  • Kinesis

CloudWatch Events vs EventBridge

CloudWatch Events can support only AWS services as event sources. It uses only the default event bus, which accepts events from AWS services, PutEvents API calls, and other authorized accounts. You can manage permissions on the default event bus to authorize other accounts.

EventBridge provides an option to create custom event buses and SaaS event bus on top of the default bus. The custom event bus is used to handle custom events raised by using PutEvents APIs. SaaS event bus is used to channel through events triggered by SaaS platforms. 

For default bus, EventBridge leverages the CloudWatch Events API, so CloudWatch Events users can access their existing default bus, rules, and events in the new EventBridge console, as well as in the CloudWatch Events console.

SNS vs EventBridge

SNS is a well-known event-sourcing service. It really shines when the throughput is very high, up into the millions of tps. EventBridge, meanwhile, supports 400 requests per second only

However, the number of targets supported by SNS is limited compared to EventBridge.

For example, if an event needs to trigger Step Functions, it cannot do it directly as it is not available as a target. It needs to call a Lambda function, and that can trigger the Step Functions. On the other hand, EventBridge supports 17 targets as of now. But, each Rule in EventBridge can configure a maximum of 5 targets.

SNS scales practically infinitely, but filtering is limited to attributes, not event content. SNS doesn’t give any guarantee as to the ordering of the messages.

Kinesis vs EventBridge

Kinesis can be used for event-routing as well as event-storing. This is an ideal solution for processing real-time data at large scales. It can fan-out to multiple consumers, however, there is a limit on the number of consumers that can connect to a single stream. Each individual consumer would carry some responsibility for filtering out of any messages that they weren’t potentially interested in. 

Kinesis also provides ordering guarantees. However, it doesn’t have an entirely usage-based pricing model. It doesn’t automatically scale to demand. 

On the other hand, EventBridge cannot buffer the events. It needs SQS or Kinesis integration for event storing. 

EventBridge Use Cases

Let’s take a couple of use cases and compare how they would be implemented using SNS and EventBridge.

1.  Say I want to build a system where if an EC2 instance is down, it should reboot and then trigger a Lambda function to store the incident to the DynamoDB table.

If I build it using SNS as an event-routing service, it would need to use SQS as well, as it cannot be subscribed by EC2 directly. Here is the design for this solution:

Using SNS as an event-routing service requires the addition of SQS.

If we implement the same use case using EventBridge, the design will look like this:

Using EventBridge entails a much simpler design.

With EventBridge the design is much simpler. We can implement our solution using fewer services.

2.  Let’s take another use case where an employee resigns from an organization and his record is updated in the CRM tool. It needs to trigger different workflows for all approvals as part of an exit checklist.

If we implement this use case using SNS, the design will look something like this:

Using SNS to trigger various workflows in the event that an employee leaves an organization.

If we use EventBridge, the design will be much simpler. It doesn’t need polling, CloudWatch Scheduler, or Lambda functions. The design will look more like this:

Using EventBridge for the employee-resignation again entails a much simpler design.

Things to Remember

Now, we have a better understanding of the EventBridge service and how it can be used to simplify design in AWS. Let’s keep a few things in mind while using this service:

  • Pricing for EventBridge is the same as for CloudWatch Events, at $1 per 1 million events published to the event bus.
  • EventBridge will ensure a successful delivery to targets. If failure happens, it will retry for 24 hours only before marking it as failed. In the case of Lambda, what successful delivery means from the EventBridge perspective is that it was able to asynchronously invoke your function. As long as EventBridge is getting a 200 response, from its perspective it is a successful transaction. If there is any failure in Lambda execution, it has to reply on Lambda retry policy. EventBridge cannot apply it’s retry policies.
  • EventBridge makes connection seamless for an AWS service from an AWS account to another AWS service in a different account. It has a target option as “event bus from another account”.
  • EventBridge needs SQS to bring resiliency but Kinesis has that feature built-in.

Summary

The introduction of EventBridge helps to solve some of the big issues around SaaS platform integration with AWS services. Also, for existing AWS services, integration has become much simpler and smoother. When comparing it to similar AWS services, it’s clear that while CloudWatch Events, SNS and Kinesis each offer certain advantages, EventBridge has the potential to be a key ingredient in modern, highly-distributed microservices architecture.

How are you using EventBridge in your serverless environment? Continue the conversation on Twitter. We’d love to hear your thoughts!

Facebook LinkedIn Email
 
Registration request was sent successfully
GET ALL THE LATEST
NEWS & OPINION