The human-centric platform for production ML & AI

Outerbounds

Hi, I am a long term user of metaflow with aws batch. Works very well.

I have a new usecase and want to get argo workflows on a on premise k8s cluster working with it. What is the suggested way of bringing aws credentials to workflows in such a setup? Using an artifact? Is it possible to use k8s secrets to bring the credentials to the workflow? How best to do that? Could not find too much documentation on these topics.

good to hear that it has been working well!

yeah, you can use any Kubernetes-standard mechanisms or whatever you have used in the past on-prem. Metaflow isn't picky about it.

<https://github.com/outerbounds/metaflow-with-airflow-minio|these instructions> have helped some folks with on-prem setup, although there isn't much auth-specific there

we are happy to help if you hit any issues