The human-centric platform for production ML & AI

Outerbounds

Hi, we have a deployment for metaflow utilizing batch ec2, and trying to figure out recommended approach re: compute environment minimum/desired vcpu settings. I’ve found that if I set the min/desired to 0 in order to not keep an instance around, if all flows have run successfully and we scale back down to zero, when another run is kicked off it goes to a new autoscaling group. Is this the expected action when we have scaled down the compute environment to 0? I would assume it would have utilized the original autoscaling group?

oh interesting, i just realized the 2 autoscaling groups are using different instance types. to just help my understanding, is metaflow expected to spin up separate autoscaling groups for different instance types based on resources requested in the decorator?

<@U04255WR2E9> do you have more than one compute environment ?

no only 1 compute environment. but i have 2 autoscaling groups, one with instance type `c5.4xlarge` and one with `m5.large` , which both are in the instance types as we allow m5, c5, and r5