Team, I've been playing around with the integration of metaflow with argo-workflows and I was hoping to better understand to what extent metaflow infrastructure usurps the need for a full setup on argo-workflows as documented here: https://argoproj.github.io/argo-workflows/workflow-archive/. On leveraging the execution of a workflow using

argo-workflows create

... which generates a workflow-template ... and a subsequent

trigger

as a workflow, does the metaflow-service capture the metadata around that flow/workflow ... and are the artifacts also captured by metaflow? Is metaflow in effect performing archiving for you, supplanting the need to set this up in argo-workflows directly? Thanks

yes, if you use our Terraform templates to set up everything, you don't have to do anything with Argo Workflows directly. In fact, you wouldn't even have to know that it uses Argo Workflows behind the scenes when you are using Metaflow

@wooden-state-97148 Metaflow doesn't do any explicit argo workflow archiving for you - however, you can set up workflow archiving and Metaflow will obey that. We don't make (m)any assumptions about the nature of your Argo Workflows deployment - so you can customize the Argo Workflows deployment however you would like to.

@wooden-state-97148 we created a metaflow plugin exactly to support extra Argo-Workflows features, i.e. argo artifacts. https://pypi.org/project/sap-ai-core-metaflow/ Beware though, it’s “bound” to the metaflow version 2.7.1 and uses a different sub-command/generator to create an argo workflow template.

Thanks for your guidance.