Hi, I had a workflow question on how people typically use Metaflow My Flow has a basic setup like -

Flow:

self.start()

@batch 
self.run_train()

self.end()

I start the flow locally from my machine, with the

start

running locally,

run_train

running on Batch, and

end

running locally But if my laptop currently disconnects from the Internet, the run will fail (I think?) Is there a way to avoid local dependency altogether? Especially for long-running trainings?

One option is to create a VM like an EC2 instance, log in and configure Metaflow, and then run the flow with that computer as the "local" env. A general pattern is this + running the flow in the background using something like tmux (so you don't have to babysit the ec2 instance terminal). Does this work in your case?

yeah, I ended up doing it like this, but thought there might be better alternative ways

@salmon-furniture-31733 you can deploy the flow on step functions

yeah, I can try the step function way ideally I was looking for something like "detach"..so let the step run, but get detached from it, and connect again later on