Hi all, Was wondering how to incorporate both

on-demand

and

spot

instances in our Metaflow infra, and let the user decide which one they want to use at run-time. Perhaps a way to spin up spot instances through a flag in the

@batch

decorator? Would love to hear any recommendations?

different batch queues perhaps? you can switch between them with env variable at runtime if Im not mistaken

Thanks Jacopo. Yeah that probably makes the most sense.

+1 to @worried-mechanic-36312! You can also specify the queue using

@batch(queue='foo')

Do you recommend using separate queues or using a mixture of both

SPOT

and

ON-DEMAND

in 1 queue and then setting the allocation strategy as appropriate?

You could mix the two together in a single job queue, in your case I am guessing you are using this as you want to optimize for costs while maintaining a base level of throughput for time-sensitive applications.

Thanks so much @curved-island-17262 This is really helpful. Which paradigm do you recommend? Having a separate queue or 2 mixed together in 1 queue with priority? Yes - one of our stakeholders noted the need to use SPOT instances for cost savings. They indicated that Ray Train has this ability and is the cheaper option, so I want to add this functionality as well to keep the cost consistent. But also I want to make this optional in a sense so that users can turn it on/off at their leisure.

If you are able to know ahead of time which pipelines need to be run as cheaply as possible then stick to giving the user the ability to do so, the mixed CE option is a sort of middle ground between running it as cheaply as possible and having the max amount of throughput.