Hey Team 👋 In our setup, we use Metaflow with Argo Workflows to run production workflows. When a workflow fails, I try using

argo resubmit --memoized

, expecting it to skip the successful steps and rerun only the failed ones. However, this doesn’t seem to work as expected. I encounter a `DataStore Error`:

Setting up task environment.
Downloading code package...
Code package downloaded.
Task is starting.
    Data store error:
    No completed attempts of the task was found for task 'HelloWorld/argo-dummyproject.user.j.kollipara.helloworld-xgx57/print_country/t-f22f4ed2'

Is there a limitation from using

argo resubmit --memoized

on metaflow triggered flows?

hi! at the moment we don't support the

--memoized

option in

argo

. you can use the resume functionality instead.

Hey thanks for the response, Yes

resume

with

--origin-run-id <argo-workflow-id>

works ✅ the problem with this approach is, you can only run the “resume” command from your local machine and the work can be done locally or in kubernetes (

--with kubernetes

), This requires the user’s local machine to remain ON throughout the process. For workflows with hundreds of tasks still pending, this can take a significant amount of time, which users may find inconvenient. any workaround/insights for this would be super helpful?

That’s correct - one work around although not super clean would be to use the Runner API to run the resume process inside another flow deployed on Argo