Hey Hey, We are trying to use metaflow client API to figure out if a metaflow run is completed successfully or not. Client API offers run object, which has 2 useful flags and they are

successful

and

finished

. For a successful metaflow run both flags work correctly. However, we are observing a non obvious behaviour for

finished

flag for failed runs that is finished flag is never set to

True

for failed runs. because of this we cannot programatically know if a metaflow run is finished if a run has failed. This seems like an obvious bug to notice, it is quite possible that we are missing something very obvious. Does anyone has come across this behaviour? or anyone tried to used client api for programatically figuring metaflow run completed successfully or not?

a challenge is that the current logic doesn’t allow us to distinguish between running and failed states easily, so

finished

is true only when the task has completed in a manner that allows the run to continue executing past the task in question In other words,

finished

being false indicates either a currently running task or a crashed task

Thanks ville for the response! this helps. we were able to get the state by command line query and filtering it with run id. ex:

python flow.py --production step-functions  list-runs | grep $(run-id)

This solves our issue. However, it would be nice to expose this functionality with Client API

good to hear that you found a solution! It is definitely on the todo list to expose more accurate lifecycle/state information through the Client API. You can upvote this issue: https://github.com/Netflix/metaflow/issues/1512

Any updates on this feature? It would be really helpful 🙂 I am trying to monitor a run triggered using

python run.py argo-workflows trigger

and then had to take inspiration from this thread with:

python metaflow_test_run.py argo-workflows status argo-testflow-xjzqf 2>&1 | tail -n 1

and then decide the status based on

Succeeded | Failed | Running

.