AWS ECS Tutorial: Creating Your First ECS Cluster

  • Topics
AWS ECS: Understanding Launch Types, Service Options, and Pricing
ECS vs. EKS: 5 Key Differences and How to Choose
ECS Monitoring
AWS ECS Tutorial: Creating Your First ECS Cluster

What Is AWS ECS? 

Amazon’s Elastic Container Service (ECS) is an efficient and scalable service for managing containers, designed to simplify the deployment and scaling of containerized applications on AWS. This service reduces the burden of setting up and maintaining your own container orchestration software, managing a virtual machine cluster, and scheduling containers on these VMs. You can think of ECS as a simpler, managed, commercial alternative to Kubernetes.

With AWS ECS, you can control Docker-supported applications via API calls, starting and stopping them as needed. It also enables you to keep track of your application’s complete status and utilize a range of management features, including security groups, EBS volumes, IAM roles, and Elastic Load Balancing. ECS assists in strategically placing containers across your cluster in accordance with your resources and availability needs.

Understanding ECS Components 

Clusters

A cluster is a logical grouping of services or tasks. If you have multiple tasks or services that are related to each other, you can group them in a cluster for better management and isolation. You can view your clusters in the AWS Management Console, AWS CLI, or using the AWS SDKs.

Clusters in ECS deployments are region-specific, meaning they are limited to one particular geographic area. However, you do have the flexibility to ensure high availability by replicating your clusters across different regions. Furthermore, within a single AWS account, it’s possible to establish multiple clusters to maintain resource isolation.

Tasks and Task Definitions

Tasks and task definitions are key concepts in AWS ECS: 

  • A task definition is a JSON formatted text document that lays out the structure of one or more containers that together constitute your application. Task definitions specify the containers to run, the Docker repository that houses the containers, the memory and CPU requirements, the port mappings, and the data volumes.
  • A task is an instance of a task definition running within a cluster. It’s a container in operation, governed by the parameters outlined in the task definition.

Services

In AWS ECS, a service is designed to execute and sustain a designated count of task definition instances concurrently within an ECS cluster. If any task encounters a failure or stops unexpectedly, the ECS service scheduler steps in, initiating another instance of your task definition to maintain the predetermined number of tasks.

Services primarily function to guarantee that a specific number of tasks are continuously operational and in good health. They provide a higher level of abstraction than tasks, allowing you to manage your applications at the service level rather than the individual task level.

Container Instances

A container instance is an EC2 instance that runs in your ECS cluster. It’s the virtual machine that hosts your containers. When you launch a new container instance, the ECS agent software is automatically installed and runs on the instance, enabling it to connect to your cluster.

Container instances support multiple types of tasks and services. You can customize them in terms of CPU, memory, and networking to match your application requirements.

ECS Agent

The ECS agent is a critical component of AWS ECS that allows the ECS service to interact with your container instances. It’s a software package that comes pre-installed on the Amazon ECS-optimized AMIs, but you can also install it manually on any EC2 instance that supports ECS.

The ECS agent is responsible for starting, stopping, and monitoring your Docker containers on behalf of the Amazon ECS service. It communicates with the ECS APIs and reports back the state of the container instances. It also collects and sends telemetry data for AWS CloudWatch, enabling you to monitor your applications and troubleshoot any issues.

Learn more in our detailed guide to ECS monitoring 

Deployment on EC2 vs. Fargate

When it comes to deploying your containerized applications in AWS ECS, you have two primary options: Amazon EC2 and AWS Fargate. 

Amazon EC2 allows you greater control over your server environment. You manage the underlying EC2 instances, meaning you can optimize them for CPU, memory, storage, and networking to fit your application requirements. This approach offers more granular control but comes with the added responsibility of managing the VMs, which can become complex and time-consuming.

AWS Fargate is a serverless option that eliminates the need to manage the underlying EC2 instances. You only worry about defining your tasks and services, and Fargate takes care of the rest. Fargate is quicker to set up and may reduce your operational overhead.

In terms of cost, EC2 instances might be more cost-effective for long-running and predictable workloads, where the resource utilization is well-understood. Fargate can be more cost-effective for sporadic, short-lived, or variable workloads that do not require a full-time EC2 instance.

AWS ECS Tutorial: Creating a Cluster With a Fargate 

Here are the primary steps involved in setting up your first ECS cluster. The examples are based on the official quick start tutorial.

Prerequisites

Before following the tutorial:

Step 1: Create a Cluster

Even though you are granted a default cluster when you sign up for an AWS account, it’s important to create a unique, custom cluster. You can do this by using the command:

Copy Code 


aws ecs create-cluster --cluster-name fargate-cluster

Step 2: Register a Task Definition

Before you can run tasks on your newly formed ECS cluster, you need to register a task definition. Here’s a basic task definition that sets up a Python web application using the python:3.8-slim container image:

Copy Code 


{
    "family": "sample-fargate",
    "networkMode": "awsvpc",
    "taskRoleArn": "arn:aws:iam::aws_account_id:role/execCommandRole", 
    "containerDefinitions": [
        {
            "name": "fargate-app",
            "image": "python:3.8-slim",
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "entryPoint": [
                "python"
            ],
            "command": [
                "-m", "http.server", "80"
            ]
        }
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "256",
    "memory": "512"
}

The taskRoleArn parameter is only important if you are deploying your task in a private subnet and want to test the deployment. Replace it with the IAM task role created for ECS Exec as per the prerequisites (here are instructions).

To register this task definition with AWS, store it as a JSON file and then register it using the –cli-input-json file://path_to_file.json option. To get the current folder location on Unix based systems, use the command pwd.

Copy Code 


aws ecs register-task-definition --cli-input-json file://$HOME/tasks/fargate-task.json

The register-task-definition command will provide a detailed description of the task definition post-registration.

Step 3: Create a Service

Once you’ve registered a task with your account, you can set up a service around this registered task in your cluster. In this example, we’ll establish a service running one instance of the sample-fargate:1 task definition in your cluster. 

This task demands internet access. You can fulfill this requirement in two ways:

  1. Employ a private subnet configured with a NAT gateway that includes an elastic IP address in a public subnet.
  2. Use a public subnet and assign a public IP address directly to your task.

Here is an example with public subnet:

Copy Code 


aws ecs create-service --cluster fargate-cluster --service-name fargate-service --task-definition sample-fargate:1 --desired-count 1 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[subnet-abcd1234],securityGroups=[sg-abcd1234],assignPublicIp=ENABLED}"

After you run the create-service command, it returns an overview of the task definition upon successful registration.

Step 4: Explore Your Services

Inspect the services of your cluster. You should see the service that you initialized in the earlier step. The service name or the ARN returned by this command can be utilized to detail the service later.

Copy Code 


aws ecs list-services --cluster fargate-cluster

You can investigate the service further by drawing on the service name. You can get a more detailed insight into your task with the following command:

Copy Code 


aws ecs describe-services --cluster fargate-cluster --services fargate-service

This command provides a breakdown of the service, including any failures and the specifics of the services. The services section might provide information like the status of deployments (either running or pending tasks), details about the task definition, the setup of the network, and a log of events with timestamps. The failures section is useful for identifying any issues related to the command.

The example output might look something like this:

Copy Code 


{
    "services": [
        {
            "networkConfiguration": {
                "awsvpcConfiguration": {
                    "subnets": [
                        "subnet-abcd1234"
                    ], 
                    "securityGroups": [
                        "sg-abcd1234"
                    ], 
                    "assignPublicIp": "ENABLED"
                }
            }, 
            ...
            "deployments": [
                {
                    "status": "PRIMARY", 
                    ...
                    
                    "taskDefinition": "arn:aws:ecs:region:aws_account_id:task-definition/sample-fargate:1", 
                    ...
                    "runningCount": 0
                }
            ], 
            "serviceName": "fargate-service", 
            "events": [
                {
                    "message": "(service fargate-service) has started 2 tasks: (task 53c0de40-ea3b-489f-a352-623bf1235f08) (task d0aec985-901b-488f-9fb4-61b991b332a3).", 
                    "id": "92b8443e-67fb-4886-880c-07e73383ea83", 
                    "createdAt": 1510811841.408
                }, 
                ...
            ], 
            ...
            "taskDefinition": "arn:aws:ecs:region:aws_account_id:task-definition/sample-fargate:1"
        }
    ], 
    "failures": []
}

Step 7: Test Your Services

Here is how to test the task you deployed via a public subnet. First, describe the task in the service using this command:

Copy Code 


aws ecs list-tasks --cluster fargate-cluster --service fargate-service

In the response, you can find the task ARN, which looks like this:

Copy Code 


{
    "taskArns": [
        "arn:aws:ecs:us-east-1:123456789012:task/fargate-service/EXAMPLE
    ]
}

Run the following command to get the Elastic Network Interface (ENI):

Copy Code 


aws ecs describe-tasks --cluster fargate-cluster --tasks arn:aws:ecs:us-east-1:123456789012:task/service/EXAMPLE

The output provides the ENI, which looks like this:

Copy Code 


{
    "name": "networkInterfaceId",
    "value": "eni-0fa40520aeEXAMPLE"
},

Use this command, providing the ENI, to get the public IP address:

Copy Code 


aws ec2 describe-network-interfaces --network-interface-id  eni-0fa40520aeEXAMPLE

The output will contain a section like this, which provides the IP address:

Copy Code 


{
    "NetworkInterfaces": [
        {
            "Association": {
                "IpOwnerId": "amazon",
                "PublicDnsName": "ec2-34-229-42-222.compute-1.amazonaws.com",
                "PublicIp": "198.51.100.2"
            },
…
}

Now you can enter the public IP address in your web browser and see the Amazon ECS sample application. This confirms your ECS task is running correctly.

AWS ECS with Lumigo

For all the benefits that AWS ECS brings to developing and running containers, these distributed applications still need observability to ensure they run at the highest performance, with the greatest reliability to deliver seamless customer experiences.

Lumigo is a cloud native observability platform purpose-built for microservice applications that provides deep visibility into applications and infrastructure, enabling users to easily monitor and troubleshoot their applications running on Amazon ECS.

  • Trace end-to-end applications running on Amazon ECS, AWS Lambda and consuming AWS services and 3rd party APIs
  • Easily monitor and debug ECS clusters and underlying services and tasks in real-time
  • Setup automatic alerts to notify you in Slack, Pagerduty and other workflow tools

Get started with Lumigo today!

Debug fast and move on.

  • Resolve issues 3x faster
  • Reduce error rate
  • Speed up development
No code, 5-minute set up
Start debugging free

Product

Company

Support