handsome-ocean-79057
12/14/2022, 10:20 PMresume from foreach steps?
Let's assume we have a flow A -> B(foreach) -> C(join) -> D where B is split across 20 countries, where 1 country fi fails
I understand python flow.py resume would only run the B tasks that failed on last run, so fi. But python flow.py resume B would instead run all 20 B tasks. Also if fi didn't explicitly error but we want to resume it manually with some tweaks without redoing expensive setup steps, this wouldn't be possible with existing resume command.
I've read some discussions about the most obvious workaround in case we want to resume on specific countries without failing, which would be to create a new temporary flow with only the steps we want (or modify the flow with some if/else skip logic) and some client API magic to load in the necessary artifacts from another run (seems a bit complex within nested foreach blocks, but doable).
Is this a common use-case or are there any simpler approaches others have found?