Amazon’s Elastic Container Service (ECS) is an efficient and scalable service for managing containers, designed to simplify the deployment and scaling of containerized applications on AWS. This service reduces the burden of setting up and maintaining your own container orchestration software, managing a virtual machine cluster, and scheduling containers on these VMs. You can think of ECS as a simpler, managed, commercial alternative to Kubernetes.
With AWS ECS, you can control Docker-supported applications via API calls, starting and stopping them as needed. It also enables you to keep track of your application’s complete status and utilize a range of management features, including security groups, EBS volumes, IAM roles, and Elastic Load Balancing. ECS assists in strategically placing containers across your cluster in accordance with your resources and availability needs.
In this article
A cluster is a logical grouping of services or tasks. If you have multiple tasks or services that are related to each other, you can group them in a cluster for better management and isolation. You can view your clusters in the AWS Management Console, AWS CLI, or using the AWS SDKs.
Clusters in ECS deployments are region-specific, meaning they are limited to one particular geographic area. However, you do have the flexibility to ensure high availability by replicating your clusters across different regions. Furthermore, within a single AWS account, it’s possible to establish multiple clusters to maintain resource isolation.
Tasks and task definitions are key concepts in AWS ECS:
In AWS ECS, a service is designed to execute and sustain a designated count of task definition instances concurrently within an ECS cluster. If any task encounters a failure or stops unexpectedly, the ECS service scheduler steps in, initiating another instance of your task definition to maintain the predetermined number of tasks.
Services primarily function to guarantee that a specific number of tasks are continuously operational and in good health. They provide a higher level of abstraction than tasks, allowing you to manage your applications at the service level rather than the individual task level.
A container instance is an EC2 instance that runs in your ECS cluster. It’s the virtual machine that hosts your containers. When you launch a new container instance, the ECS agent software is automatically installed and runs on the instance, enabling it to connect to your cluster.
Container instances support multiple types of tasks and services. You can customize them in terms of CPU, memory, and networking to match your application requirements.
The ECS agent is a critical component of AWS ECS that allows the ECS service to interact with your container instances. It’s a software package that comes pre-installed on the Amazon ECS-optimized AMIs, but you can also install it manually on any EC2 instance that supports ECS.
The ECS agent is responsible for starting, stopping, and monitoring your Docker containers on behalf of the Amazon ECS service. It communicates with the ECS APIs and reports back the state of the container instances. It also collects and sends telemetry data for AWS CloudWatch, enabling you to monitor your applications and troubleshoot any issues.
Learn more in our detailed guide to ECS monitoring
When it comes to deploying your containerized applications in AWS ECS, you have two primary options: Amazon EC2 and AWS Fargate.
Amazon EC2 allows you greater control over your server environment. You manage the underlying EC2 instances, meaning you can optimize them for CPU, memory, storage, and networking to fit your application requirements. This approach offers more granular control but comes with the added responsibility of managing the VMs, which can become complex and time-consuming.
AWS Fargate is a serverless option that eliminates the need to manage the underlying EC2 instances. You only worry about defining your tasks and services, and Fargate takes care of the rest. Fargate is quicker to set up and may reduce your operational overhead.
In terms of cost, EC2 instances might be more cost-effective for long-running and predictable workloads, where the resource utilization is well-understood. Fargate can be more cost-effective for sporadic, short-lived, or variable workloads that do not require a full-time EC2 instance.
Here are the primary steps involved in setting up your first ECS cluster. The examples are based on the official quick start tutorial.
Before following the tutorial:
Even though you are granted a default cluster when you sign up for an AWS account, it’s important to create a unique, custom cluster. You can do this by using the command:
aws ecs create-cluster --cluster-name fargate-cluster
Before you can run tasks on your newly formed ECS cluster, you need to register a task definition. Here’s a basic task definition that sets up a Python web application using the python:3.8-slim container image:
{
"family": "sample-fargate",
"networkMode": "awsvpc",
"taskRoleArn": "arn:aws:iam::aws_account_id:role/execCommandRole",
"containerDefinitions": [
{
"name": "fargate-app",
"image": "python:3.8-slim",
"portMappings": [
{
"containerPort": 80,
"hostPort": 80,
"protocol": "tcp"
}
],
"essential": true,
"entryPoint": [
"python"
],
"command": [
"-m", "http.server", "80"
]
}
],
"requiresCompatibilities": [
"FARGATE"
],
"cpu": "256",
"memory": "512"
}
The taskRoleArn parameter is only important if you are deploying your task in a private subnet and want to test the deployment. Replace it with the IAM task role created for ECS Exec as per the prerequisites (here are instructions).
To register this task definition with AWS, store it as a JSON file and then register it using the –cli-input-json file://path_to_file.json option. To get the current folder location on Unix based systems, use the command pwd.
aws ecs register-task-definition --cli-input-json file://$HOME/tasks/fargate-task.json
The register-task-definition command will provide a detailed description of the task definition post-registration.
Once you’ve registered a task with your account, you can set up a service around this registered task in your cluster. In this example, we’ll establish a service running one instance of the sample-fargate:1 task definition in your cluster.
This task demands internet access. You can fulfill this requirement in two ways:
Here is an example with public subnet:
aws ecs create-service --cluster fargate-cluster --service-name fargate-service --task-definition sample-fargate:1 --desired-count 1 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[subnet-abcd1234],securityGroups=[sg-abcd1234],assignPublicIp=ENABLED}"
After you run the create-service command, it returns an overview of the task definition upon successful registration.
Inspect the services of your cluster. You should see the service that you initialized in the earlier step. The service name or the ARN returned by this command can be utilized to detail the service later.
aws ecs list-services --cluster fargate-cluster
You can investigate the service further by drawing on the service name. You can get a more detailed insight into your task with the following command:
aws ecs describe-services --cluster fargate-cluster --services fargate-service
This command provides a breakdown of the service, including any failures and the specifics of the services. The services section might provide information like the status of deployments (either running or pending tasks), details about the task definition, the setup of the network, and a log of events with timestamps. The failures section is useful for identifying any issues related to the command.
The example output might look something like this:
{
"services": [
{
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"subnet-abcd1234"
],
"securityGroups": [
"sg-abcd1234"
],
"assignPublicIp": "ENABLED"
}
},
...
"deployments": [
{
"status": "PRIMARY",
...
"taskDefinition": "arn:aws:ecs:region:aws_account_id:task-definition/sample-fargate:1",
...
"runningCount": 0
}
],
"serviceName": "fargate-service",
"events": [
{
"message": "(service fargate-service) has started 2 tasks: (task 53c0de40-ea3b-489f-a352-623bf1235f08) (task d0aec985-901b-488f-9fb4-61b991b332a3).",
"id": "92b8443e-67fb-4886-880c-07e73383ea83",
"createdAt": 1510811841.408
},
...
],
...
"taskDefinition": "arn:aws:ecs:region:aws_account_id:task-definition/sample-fargate:1"
}
],
"failures": []
}
Here is how to test the task you deployed via a public subnet. First, describe the task in the service using this command:
aws ecs list-tasks --cluster fargate-cluster --service fargate-service
In the response, you can find the task ARN, which looks like this:
{
"taskArns": [
"arn:aws:ecs:us-east-1:123456789012:task/fargate-service/EXAMPLE
]
}
Run the following command to get the Elastic Network Interface (ENI):
aws ecs describe-tasks --cluster fargate-cluster --tasks arn:aws:ecs:us-east-1:123456789012:task/service/EXAMPLE
The output provides the ENI, which looks like this:
{
"name": "networkInterfaceId",
"value": "eni-0fa40520aeEXAMPLE"
},
Use this command, providing the ENI, to get the public IP address:
aws ec2 describe-network-interfaces --network-interface-id eni-0fa40520aeEXAMPLE
The output will contain a section like this, which provides the IP address:
{
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-34-229-42-222.compute-1.amazonaws.com",
"PublicIp": "198.51.100.2"
},
…
}
Now you can enter the public IP address in your web browser and see the Amazon ECS sample application. This confirms your ECS task is running correctly.
For all the benefits that AWS ECS brings to developing and running containers, these distributed applications still need observability to ensure they run at the highest performance, with the greatest reliability to deliver seamless customer experiences.
Lumigo is a cloud native observability platform purpose-built for microservice applications that provides deep visibility into applications and infrastructure, enabling users to easily monitor and troubleshoot their applications running on Amazon ECS.