Hi all, Is there a way to monitor/look for status of metaflow runs either by command line or with client api, when they are triggered by argo events with ‘ArgoEvent.publish’

Hey! I believe, that you might want to use the

--run-id-file

option when triggering the workflow. When you list help, this is the description:

--run-id-file TEXT            Write the ID of this run to the file
                                specified.

Then you can use the id from the file to monitor the run using the metaflow client.

Thanks Ales! However, we are not using step functions. We are using ARGO EVENTS. With metaflow supported argo events trigger. All it says after triggering is ‘triggered successfully’ and I could not find a way to look for status using command line or client api.

I undrestand. What I had suggested works even with Argo, but you have to trigger using command line:

python run.py argo-workflows trigger --run-id-file xyz

. I am not sure, whether something like this is possible with

ArgoEvent.publish

.

I am using ArgoEvent.publish

There is some kind of id returned, when

ArgoEvent.publish

finishes. Maybe somebody here knows how to query the runs using that id.

Thanks ales!

I am interested in the question. From my current understanding, it is not possible to know which flows have been triggered by an event; events are unidirectional communications and don't get a response. It is not even possible to know that a flow started, as stated in the docstring from

ArgoEvent.publish()

. The simplest solution, as of now and to the best of my knowledge, to get lineage from the parent flow is to trigger the child flow on the commandline. Otherwise, you may be able to use the Metaflow Client, to go through all runs and find those that have been triggered by your event. This did not sound great for my use case, but maybe it makes sense for yours. It may help if you make sure your flow would be tagged with some ID from the triggering event, as you may filter runs on this tag.

Thanks Fabien, this really helps.. 🙂