Introduction
Managing containerized workloads at scale demands precise coordination between platform engineers and application teams. With the introduction of managed daemon support for Amazon Elastic Container Service (ECS) Managed Instances, you can now decouple the lifecycle of operational agents—such as monitoring, logging, and tracing tools—from your application deployments. This guide walks you through setting up a managed daemon using the Amazon CloudWatch Agent as an example, giving you independent control, improved reliability, and centralized resource management.
Follow these steps to get started, from prerequisites to verification.
What You Need
- An AWS account with appropriate permissions to create and manage ECS resources.
- An existing Amazon ECS cluster with at least one Managed Instance capacity provider. If you don't have one, refer to the ECS documentation to set it up.
- An IAM role for ECS task execution (e.g., ecsTaskExecutionRole) that grants permissions to pull container images and write logs to CloudWatch.
- Basic familiarity with the AWS Management Console and ECS concepts.
Step-by-Step Guide
Step 1: Access the ECS Daemon Task Definitions Section
Open the Amazon Elastic Container Service console. In the left navigation pane, locate the new Daemon task definitions option (introduced with the managed daemon feature). Click on it to access the dedicated daemon management area.
Step 2: Create a New Daemon Task Definition
Click the Create new daemon task definition button. This opens a form where you’ll define the operational agent you want to run on every instance. For this example, we’ll configure the CloudWatch Agent.
- Daemon task definition family: Enter a recognizable name, such as
cloudwatch-agent-daemon. This helps you identify it later. - Task execution role: Select the ecsTaskExecutionRole from the dropdown. If you haven’t created one yet, do so in IAM with the necessary permissions (
ecr:GetDownloadUrlForLayer,logs:PutLogEvents, etc.). - Container definitions: Add a container. For the CloudWatch Agent, use the official image
amazon/cloudwatch-agent:latest. Define the Container name (e.g.,cwagent).
Step 3: Configure Resources and Environment
In the container definition, set the CPU and Memory limits. The original example uses 1 vCPU and 0.5 GB of memory. These resources are dedicated to your daemon and separate from your application tasks. You can also specify environment variables if needed (e.g., CW_CONFIG_CONTENT for the agent configuration).
Optionally, configure Log collection for the container by enabling Auto-configure CloudWatch Logs and specifying a log group name.
Step 4: Review and Create the Daemon Task Definition
Check all settings, then click Create. Your daemon task definition is now saved. It will appear in the list under Daemon task definitions with a status of INACTIVE until you deploy it.
Step 5: Deploy the Daemon to Your Capacity Provider
To activate the daemon, you need to associate it with one or more capacity providers. From the daemon task definition details page, choose Actions > Deploy. You’ll be prompted to:
- Select capacity provider(s): Choose the Managed Instance capacity provider(s) that should run this daemon. You can deploy across multiple providers for broad coverage, or target specific ones for staged rollouts.
- Deployment options: Define a deployment name (e.g.,
cwagent-deploy-1) and optionally set a rollout strategy (e.g., rolling update with a certain percentage of instances at a time).
Click Deploy. The daemon will be scheduled to start on all instances in the selected capacity providers. Daemons are guaranteed to start before any application tasks and will drain last when instances terminate, ensuring logging and monitoring are always available.
Step 6: Verify the Daemon is Running
After deployment, check the status:
- Navigate to your ECS cluster, then go to the Managed Instances tab.
- Select an instance and view the Daemons section. You should see your CloudWatch Agent listed with a status of RUNNING.
- Alternatively, use the Daemon task definitions page to see overall deployment health and number of instances with the daemon running.
- Verify the agent is sending metrics by opening the CloudWatch console and checking for custom namespaces or logs.
Step 7: Update a Managed Daemon Independently
One of the key benefits is decoupled updates. To update the CloudWatch Agent (e.g., to a new version or configuration):
- Go to Daemon task definitions, select your daemon family, and create a new revision by editing the container image tag or environment variables.
- Deploy the new revision to the same capacity providers. The update rolls out without touching your application tasks. Use rolling updates to minimize impact.
Conclusion and Tips
Managed daemons for ECS Managed Instances simplify the operational burden of hosting agents at scale. By separating daemon lifecycles from application deployments, you gain flexibility, consistency, and resource efficiency.
Tips for Success
- Start small: Test with a single daemon like CloudWatch Agent on a non-production cluster before rolling out to production.
- Optimize resource allocation: Each instance runs exactly one copy of the daemon shared among all application tasks, so keep CPU and memory requests minimal to maximize available instance capacity.
- Use infrastructure as code: Define your daemon task definitions in CloudFormation or Terraform for repeatable deployments.
- Monitor daemon health: Set up CloudWatch alarms for daemon status to detect when an instance fails to run the required agent.
- Plan for multi-agent scenarios: You can deploy multiple daemons (logging, tracing, security) – just be mindful of total resource consumption per instance.