We're in the process of considering kubernetes(EKS) and argo to run metaflow ML jobs. Currently we're running step-functions and AWS Batch.
At the moment, we create multiple AWS Batch Queue jobs, and Computer environment to manage various workloads according to environment such as (dev, prod, gpu, cpu, team).
This allow us to support different teams using a single metaflow platform.
those with experience on kubernetes and argo, how does ArgoCD and EKS manage multiple queue and various compute environment ?