Hello, While deploying

metaflow

with terraform module , the AWS batch compute environment is creating a ec2 instance. What is the reason behind it? And is there a way to use stop instances?

You could look further into the details .. I would guess its the metadata service. P.S I am new to MF too,~ I think this should be because of the default minvcpu set to 8. Apparently AWS batch always tries to keep an instance up to reduce latency but you can change that to 0 so you can spin up instances only when needed. Little longer for your job to start but you pay only when needed.

Thank you

@fancy-mouse-14245 I think you should wait for someone who knows better to confirm 🙂. But you can change the minvcpus and see

Yeah, I’m going to try that out

+1 to @jolly-afternoon-77975. If you would like to reduce costs, you can set the min vCPUs to 0 so that an instance is always brought up on demand.

Can we use spot instances while creating compute env through terraform module? If yes, please help. Cause I only found

ec2 & fargate

as option.

would love to hear an easy option as well but heres an old related thread meanwhile, seems like its not a neat idea...but i thought we can use more retries and some way to track failures due to spot vs the task itself. You could give this a try https://outerbounds-community.slack.com/archives/C02116BBNTU/p1657582989853079

@glamorous-queen-9531 @square-wire-39606 will changing this in

modules/computation/batch.tf

work to use the

SPOT

instances rather than

EC2 on-demand

?

@fancy-mouse-14245 I think this thread is marked done, so maybe copy paste this in anew one else savin might miss

Ok thank you