Hello Outerbounds Team:
We (as Roblox) are currently evaluating Metaflow to run on a Kubernetes cluster that is managed by an Istio service mesh. As we scale our usage, we're exploring best practices for securing access to Metaflow's components and UI in a multi-tenant environment.
Our security model for other applications on the cluster leverages the Istio Ingress Gateway with an OIDC provider for end-user authentication. We are interested in understanding how this pattern can be applied to Metaflow.
Specifically, we have a few questions:
1. What is the recommended approach for user authentication and authorization when multiple data scientists are using Metaflow on a shared Kubernetes cluster?
2. For the Metaflow UI, if we expose it via an ingress, have you seen customers successfully place it behind an OIDC-aware proxy for authentication?
3. Are there any reference architectures for integrating Metaflow with a service mesh like Istio, particularly concerning identity propagation for jobs and fine-grained access control for the metadata service?
We are essentially looking to achieve a similar end-user authentication experience as one might find with a platform like Kubeflow, and we would appreciate any guidance or best practices you can share.
Thank you for your time and help.