Regarding local orchestration and Multi-Factor Authentication. With my current aws profile I need to authenticate every now and then which becomes a problem with long running workflows. I imagine this problem will be gone if I move the orchestration to Step Functions, but currently I am not ready for that. So I guess I could create an aws user which does not have MFA and which has exactly the permissions needed in order to run the orchestration. But what are these permissions exactly? Would this be the recommended way forward? (Thanks for all the help, very much appreciated! 🙏)

here you go - https://github.com/outerbounds/metaflow-tools/blob/master/aws/cloudformation/metaflow-cfn-template.yml#L756

Thanks @square-wire-39606, but it seems like the names of these roles does not map well to the roles created with Terraform. What would be the role I should assume in the Terraform setup? I got these roles available:

metaflow-batch-execution-role-<suffix>
metaflow-batch_s3_task_role-<suffix>
metaflow-ecs-execution-role-<suffix>
metaflow-ecs-iam-role-<suffix>
metaflow-lambda_ecs_execute-<suffix>
metaflow-metadata-ecs-task-<suffix>

you'd likely want to create the

MetaflowUserRole

using terraform with similar permissions as those listed in the cloudformation example – note that if you're assuming the role from another, you'd still be limited to a max 12hr TTL, so you would need to explicitly use that non-MFA role. another option that's preferable IMO, is to use an EC2 instance in AWS as your development workstation – you can use existing auth/SSO when connecting to the instance, but any long-running processes can continue when you disconnect. there's lots of other benefits to cloud-native development too – as your "laptop in the cloud" you get much more flexibility on compute resources, faster networking, less traffic sent over public internet/home wifi, stronger security, easier to reproduce environments, the list goes on for awhile 😛

Thanks @fresh-laptop-72652. Great idea with the EC2 approach as well. Now I feel I have a much clearer view of my options.