Hi Team, i would like to launch multple tasks of a...
# ask-metaflow
m
Hi Team, i would like to launch multple tasks of a same step in metaflow something similar to what foreach does based on the inputs, but i would like to run them sequentially and not parallely launch all the runs, is it possible to achieve that in metaflow?
1
d
use --max-workers = 1 but it will apply to all foreach.
m
@dry-beach-38304 This is where i have declared the foreach , where do i have to pass the --max-workers arguement?
do i need to pass it while invoking the metaflow run python flow.py run --max-workers 1?
d
yes. As I mentioned, it’s a global thing (so your entire flow will run sequentially)
m
Great, say for example if i have 20 tasks to be run on the whole and i configure the --max-workers to 1 and if one of the tasks fails, does it automatically start the next one or would it fail the complete flow. I would like to ideally let the next one to process even if current one fails
d
it wouldn’t start it unless you used retry or catch to prevent the task from failing the flow. In general (with our without maxworkers), one task failure will fail the flow (unless you use catch or retry)
👍 1
m
is there a way to kill a running metaflow job ?
when i killed the aws batch jobs , i still see the job running in the ui