Is it possible to just run a specific later step - similar to: python GreatFlow.py --environment=conda --package-suffixes .sql resume --with batch basically my last step before end failed and the above worked great but now I would like to run this step again as the previous steps take 2 hours or so … of course could create a dedicated flow but maybe there is an easier way?

do i understand correctly that you want to resume a resumed run at the last step?

thanks for the reply - maybe better expressed - sorry about that: I would like to “restart”(/resume) at a specifie step benefiting from the artefacts of previous steps … example. I have (latest) run: start -> S1 -> S2 -> S3 -> end I would like to start again at step S3 (just coincident that last step) Hope that makes sense? Thanks!

makes sense! did you try

resume S3

i.e give it a step name to resume from?

ah I see resume also works if the step did not fail and already ran? Apologies I missed this - love it. Thanks for this great framework!

yep!

resume --origin-run-id RUNID mystep

- it's actually a super powerful feature! 💪

I just tried: python Bla.py --environment=conda --package-suffixes .sql --origin-run-id 306 mystep but get: Error: no such option: --origin-run-id any ideas?

it seems you forgot the

resume

command above

so this should work?

python Bla.py --environment=conda --package-suffixes .sql resume --origin-run-id 306 mystep

thanks

yep! Let me know if it doesn’t

it works thanks!

right, no need to pass parameters again. In fact, you can’t change them with resume

Thanks I ran it yesterday and it worked great.