Hi I've used Metaflow pretty heavily on a previous team and been pitching my new organization to try it out. Was trying to set up some basic infra using the "minimal" terraform example and think I'm mostly there, but my Batch compute environment is

INVALID

with this stated reason:

CLIENT_ERROR - Your compute environment has been INVALIDATED and scaled down because none of the instances joined the underlying ECS Cluster. Common issues preventing instances joining are the following: VPC/Subnet configuration preventing communication to ECS, incorrect Instance Profile policy preventing authorization to ECS, or customized AMI or LaunchTemplate configurations affecting ECS agent.

Anyone else run into anything like this? Thanks!

meow wave Hey @bright-author-73055 great to see you again! are there any internal policies/agents that are not allowing for the template to execute as expected?

Great to see you again too! 🙂 I'm taking a look at our EC2 launch templates and see a couple in there. So I go into them and click into

Advanced details

and scroll down and see that it sets the

ECS_CLUSTER

env variable. When I go to ECS in the console, I'm seeing multiple ECS clusters. One of them is running the metadata service. Is that the one main cluster for all of Metaflow to run on? And if so, should the launch template be referencing that one with

ECS_CLUSTER

? Right now it looks like it's pointing to a different cluster. Not sure if that made any sense...

cool cool. we should catch up sometime! plenty of progress since we last spoke

Would love to! Was able to solve this btw. For posterity, the issue here was that my Batch Compute Environment was associated with public subnets that had "Auto assign public IP" turned off. So all I had to do was turn that on and it works now

glad it worked out! let me follow up on a dm for finding time to chat