ECS rolling deployments: what "done" actually means

Rolling is the ECS deployment strategy most people start with, and the one everyone assumes they already understand. Old tasks out, new tasks in, two percentages to control the pace. The percentages are documented properly. What the documentation never actually defines is when a rolling deployment is done. Three separate layers each declare completion at their own moment (the deployment record, the service’s own events, and the CloudFormation resource), and on an update they disagree by well over a minute. We measured where each one lands, on two very different rolling services, and this post is what came out of it.

The test stack

Everything here was measured on the small CloudFormation stack described in the first part of this series: three Fargate services on one tiny test container, deployed by a stack update that flips an image tag parameter. This post is about the two rolling ones, worker (two tasks, no load balancer) and singleton (exactly one task at all times, the configuration a hard singleton forces, celery beat being the classic example). Neither had a container health check when these runs were measured, which matters for the counting rules below.

What Min and Max actually control

The whole of rolling is these two properties:

 1WorkerService:
 2  Type: AWS::ECS::Service
 3  Properties:
 4    ServiceName: worker
 5    Cluster: !Ref Cluster
 6    LaunchType: FARGATE
 7    DesiredCount: 2
 8    TaskDefinition: !Ref WorkerTaskDef
 9    DeploymentController:
10      Type: ECS
11    DeploymentConfiguration:
12      Strategy: ROLLING
13      MaximumPercent: 200
14      MinimumHealthyPercent: 100

The arithmetic is documented on the rolling update page. MaximumPercent is the upper limit on tasks in the RUNNING or PENDING state as a percentage of the desired count, rounded down. MinimumHealthyPercent is the lower limit on tasks that must stay running, as a percentage of the desired count, rounded up. At 200 and 100 on a desired count of two, the scheduler is allowed four tasks and must keep two, so it starts both replacements before it stops anything.

The part that matters more is what “healthy” means for that counting, because it sets how fast a roll can move. The CloudFormation reference is the page that spells it out, not the developer guide:

• No load balancer and no container health check: the scheduler “will wait for 40 seconds after a task reaches a RUNNING state before the task is counted towards the minimum healthy percent total”.
• No load balancer, with a container health check: it waits for the task to reach a healthy status instead.
• With a load balancer: it waits for the target group health check to return healthy, and for the container health check too if there is one.

So on a service with no load balancer and no health check, every task carries a flat 40-second wait before it counts for anything. Here is a clean roll of the worker service, elapsed from the moment the stack update was issued (July 2026, eu-west-1):

Two up, then two down, exactly as the percentages promise. Both replacements were running before any old task was stopped, the peak was four tasks, and capacity never dropped below the desired two. The whole roll took 3m36s, and the actual task replacement is the first two minutes of it.

The 40-second rule is visible in the gaps. The first old task was stopped 42 seconds after the first replacement reached RUNNING, and the second 46 seconds after the second (45 and 49 seconds in a later run of the same service). The scheduler isn’t waiting for anything clever. With no load balancer and no health check, there is nothing else it can look at.

Three layers of “done”, and they disagree

Look at the last four rows of that table again. Between 02:04 and 03:36 nothing happens to any task, and yet three different layers announce completion in that window.

The deployment record (what describe-service-deployments returns) was stamped finishedAt at 02:04, with status SUCCESSFUL and no reason. The service’s own events said “deployment completed” and “has reached a steady state” at 03:12, one minute and eight seconds later. The CloudFormation resource went UPDATE_COMPLETE at 03:32, twenty seconds after that. The gap between the record and the events was around a minute in every rolling update we timed, and CloudFormation trailed the steady-state event by 9 to 26 seconds across five service completions in four stack updates.

Then we created services rather than updating them, and the order flipped. On a service create, the record’s finishedAt came 14 seconds after the steady-state event instead of a minute before it. Same API, same field, opposite ordering, depending only on whether the service already existed.

So a record status of SUCCESSFUL doesn’t mean the service is settled, and the record-versus-events ordering isn’t stable enough to build on. Anything in your pipeline that needs a dependable “everything is finished” moment should wait on the CloudFormation waiter, which came last in every run.

None of this contradicts the documentation, because the documentation doesn’t say it at all. The EventBridge reference states that SERVICE_DEPLOYMENT_COMPLETED “is sent once a service reaches a steady state after a deployment”, which matches what we saw for the events, and says nothing about the record’s own timestamp. And “steady state” is only ever described in a circle: the service event message reference says the event is sent “when the service is healthy and at the desired number of tasks, thus reaching a steady state”. Healthy according to the counting rules above, presumably, but that link is never made explicitly.

The sharpest completion criterion AWS has published for rolling deployments isn’t in the developer guide at all. It is in a containers blog post about alarm rollbacks: “the deployment process is deemed complete when the primary deployment is healthy and has reached the desired count and the active deployment has been scaled down to 0”. That is the definition to work from, and it is worth knowing where it lives.

Worth adding that a rolling deployment has no bake time by default (the 15-minute default applies to the shifting strategies only), so completion follows the last old task going away with nothing in between. If you do set one, the CloudFormation handler timeout rises to its 36-hour maximum.

Reading the record while it is running

Two small details worth knowing before you read a rolling deployment record.

lifecycleStage is populated on a rolling deployment record while the deployment is in flight, where it reads SCALE_UP. On the terminal record it is null, and it also clears if the deployment starts rolling back. Your tooling can read the stage while polling, but never after the fact.

And when the old tasks are stopped, the stop reason names the old deployment:

Scaling activity initiated by (deployment ecs-svc/<OLD deployment id>)

Scale-down of the outgoing tasks is attributed to the deployment those tasks belonged to. Logical once you see it, and thoroughly confusing while you are grepping for the id of the deployment you just started.

The singleton configuration

The second rolling service is the interesting one:

1    DeploymentConfiguration:
2      Strategy: ROLLING
3      MaximumPercent: 100
4      MinimumHealthyPercent: 0

With DesiredCount: 1, MaximumPercent: 100 allows one task at a time, so a replacement can’t start while the old task is alive. And the minimum healthy count rounds up, so any MinimumHealthyPercent above 0 would round to one whole task and leave the scheduler with nothing it is permitted to move. For a hard singleton this pair isn’t a choice, it is the only combination that can deploy at all.

Measured, the behaviour is exactly stop-then-start. This is the singleton’s part of a stack update that rolled both services at once, elapsed from the update:

Zero overlap at any level. The new task was created 6 seconds after the old one was fully STOPPED, and no second task ever existed. The functional outage, from the stop being ordered to the replacement running, was 55 seconds, and the two task records themselves were 20 seconds apart.

The number that surprised us isn’t the length of the gap, it is where the gap sits. The stop was ordered 10 seconds after the deployment record started, which is 24 seconds into a stack update that ran for four and a half minutes. Your singleton is down near the beginning of the deployment, not at the end. If anything in your system notices a missing scheduler, it will notice while CloudFormation still says UPDATE_IN_PROGRESS.

That is the whole story of this configuration when it works, and around a minute of downtime per release is a price most teams will take for a component that genuinely cannot run twice. What happens when the replacement task doesn’t come good is a much longer story, and it is the subject of a later part in this series, “The deployment that lied: ECS singletons and false success”.

One update, several services

Because both services take their image from the same parameter, one flip rolled both of them. CloudFormation started the two service updates within 65 milliseconds of each other and then waited for both. The worker resource took 4m23s, the singleton resource 3m23s, and the stack completed in 4m37s. A combined roll costs the slowest service, not the sum of the services.

Two related behaviours from the same runs, both worth knowing before you plan a pipeline around them:

• An unchanged service gets nothing at all. The blue/green web service was not just left alone, it was absent from the stack events entirely. No no-op touch, no ECS events, no task replacement, and its request loop answered every one of its 281 requests with a 200. This is consistent with CloudFormation’s normal behaviour: “didn’t change, don’t touch it.”
• Changes confined to DeploymentConfiguration never create a deployment. Adding or removing a lifecycle hook, attaching or detaching a circuit breaker or deployment alarms: each was applied as a quiet in-place configuration update, with no deployment record and no task replaced. We watched this four times over the rolling runs, and each took between 37 and 47 seconds. As of July 2026 in eu-west-1 that floor looks like CloudFormation’s minimum for any UpdateService call rather than anything to do with what changed, so treat it as an observation rather than a promise.

Where that leaves rolling

Rolling’s real contract is narrower than the word “done” suggests. When a rolling deployment completes, what you know is that the new tasks reached RUNNING, that they counted as healthy by whichever rule applies to your service, and that the old ones were stopped afterwards. On a service with no load balancer and no container health check, “counted as healthy” means a task stayed up for 40 seconds. That is the entire promise.

So: wait on the CloudFormation waiter, not on the deployment record. Know which counting rule applies to each of your services, because it sets both the pace of the roll and the strength of the check. And for a single-task service, plan for the outage to start at the front of the deployment.

What none of this covers is a release that starts and then doesn’t work, which is where the rest of the series lives. Part one covered the native blue/green machinery and part two pause hooks and where they do nothing. Still to come: who owns the rollback when ECS and CloudFormation are both involved (“ECS circuit breaker and CloudFormation: who owns the rollback”), what deployment alarms really promise (“ECS deployment alarms: the fine print”), the crash-looping release that every layer marked successful (“The deployment that lied: ECS singletons and false success”), and the check that catches a service that is alive but sick (“ECS container health checks: catching alive-but-sick services”).

Getting deployment pipelines to behave predictably is part of our DevOps and infrastructure as code work - if your ECS deployments finish at a moment nobody can quite pin down, we can help.