ECS Fargate Scheduled Task not running
ECS Fargate Scheduled Task not running
I'm trying to setup a scheduled task with ECS Fargate but I cannot figure out why it is not running. I can confirm the task works correctly using RunTask
but when I try to trigger it on a schedule all I get is a bunch of 'FailedInvocations' with no explanation.
RunTask
I do know though that the rule is being triggered so this is a good sign. See the screenshot below:
But everytime it is triggered there is just a 'FailedInvocation'. Here's the scheduling rule:
And the default permissions on the ecsEventRole
with just ecs:runTask
:
ecsEventRole
ecs:runTask
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Action": [
"ecs:RunTask"
],
"Resource": [
"*"
]
]
My hunch says that this ecsEventsRole
doesn't have enough permissions. Should I try to give it the ones that ecsTaskExecutionRole
has?
ecsEventsRole
ecsTaskExecutionRole
Thanks
EDIT: This is now supported in us-east-1
region. See comments.
us-east-1
Nope. I'm still stuck :( Given up on all options.
– coolboyjules
Jan 9 '18 at 17:25
I talked to AWS support, its not supported yet. No ETA
– Marc Young
Jan 9 '18 at 20:28
Brutal. Thanks Marc
– coolboyjules
Jan 15 '18 at 2:39
Fargate tasks are now supported: aws.amazon.com/about-aws/whats-new/2018/08/…
– FBryant87
Sep 4 '18 at 16:30
3 Answers
3
I ran into a similar issue where regular ECS Scheduled Tasks were not running.
I finally resolved it by adding an additional policy to ecsEventsRole
which allows CloudWatch Events to pass IAM roles to ECS Tasks:
ecsEventsRole
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Action": [
"iam:ListInstanceProfiles",
"iam:ListRoles",
"iam:PassRole"
],
"Resource": "*"
]
I think this is hitting a few people because the first bit of documentation (docs.aws.amazon.com/AmazonECS/latest/developerguide/…) says 'You must add iam:PassRole permissions for any task role overrides to the CloudWatch IAM role', but it turns out that the task needs it to use a role already applied (explained in a different area of the documentation docs.aws.amazon.com/AmazonECS/latest/developerguide/… - 'If your scheduled tasks require the use of the task execution role or a task role override, then you must add iam:PassRole permissions').
– Chris
Mar 14 '18 at 16:27
In addition to ecs:RunTask, I only needed iam:PassRole on both the Task Role and Task Execution role resources.
– gileri
Jan 18 at 21:35
Here is a possible workaround: use a lambda function as target for the cloudwatch rule and create the task in the lambda function code.
Here is an example code for the lambda function:
https://lobster1234.github.io/2017/12/03/run-tasks-with-aws-fargate-and-lambda/
The links describes how to pack the new boto version with the lambda function but this is not necessary anymore since AWS already updated the lambda boto version to 1.4.8
I've tested and it works.
Thats what I was trying to avoid but ended up with
– Marc Young
Aug 7 '18 at 21:43
I ended up using this. Probably the most reasonable solution presently.
– coolboyjules
Aug 8 '18 at 19:08
Have you tried using the aws cli and running aws events put-rule
followed by aws events put-targets --rule <value> --targets <value>
instead? I was having a similar problem, and using the (recent version of) the aws cli worked for me.
aws events put-rule
aws events put-targets --rule <value> --targets <value>
Here's a sample:
aws events put-rule --name "DailyLambdaFunction" --schedule-expression "cron(0 9 * * ? *)"
aws events put-rule --name "DailyLambdaFunction" --schedule-expression "cron(0 9 * * ? *)"
Followed by the below command all in one line:
aws events put-targets --rule cli-RS-rule --targets '"Arn": "arn:aws:ecs:1234/cluster/clustername","EcsParameters": "LaunchType": "FARGATE","NetworkConfiguration": "awsvpcConfiguration": "AssignPublicIp": "ENABLED", "SecurityGroups": [ "sg-id1233" ], "Subnets": [ "subnet-1234" ] ,"TaskCount": 1,"TaskDefinitionArn": "arn:aws:ecs:1234:task-definition/taskdef","Id": "sampleID111","RoleArn": "arn:aws:iam:1234:role/eventrole"'
aws events put-targets --rule cli-RS-rule --targets '"Arn": "arn:aws:ecs:1234/cluster/clustername","EcsParameters": "LaunchType": "FARGATE","NetworkConfiguration": "awsvpcConfiguration": "AssignPublicIp": "ENABLED", "SecurityGroups": [ "sg-id1233" ], "Subnets": [ "subnet-1234" ] ,"TaskCount": 1,"TaskDefinitionArn": "arn:aws:ecs:1234:task-definition/taskdef","Id": "sampleID111","RoleArn": "arn:aws:iam:1234:role/eventrole"'
make sure to edit the security group, subnet and account ID strings before running it - they're just placeholders in the command i used above.
– Tannavee
Oct 1 '18 at 18:05
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Any updates? I'm at exactly the same spot and stuck
– Marc Young
Jan 9 '18 at 3:09