We're getting Metaflow spun up in Kubernetes. We h...
# ask-metaflow
b
We're getting Metaflow spun up in Kubernetes. We have Metaflow spun up/running in k8s (hooray!), but now we're trying to figure out how we're actually supposed to consume that cluster. The plan is to use Argo Workflows with this, but we're trying to first ensure that the Metaflow chart is working like it's supposed to. We thought we would point
METAFLOW_SERVICE_URL
at the publicly exposed kubernetes service/ingress, add the
@kubernetes
decorator to tell MetaFlow what container to run with what resources and then be off to the races. What seems to be happening is - when kicking off a Metaflow script, it tries to authenticate with Kubernetes, which doesn't succeed as we haven't granted kubernetes access to these ML developers. So - a sanity check - is the
metaflow-service
k8s deployment responsible for managing kubernetes resources? Or is it the script running on the developer's local environment which is responsible for managing/administering kubernetes?
For all I know, the permissions are bizarre because ArgoWorkflow isn't up yet
r
Hi, I'm assuming you are deploying Metaflow on-premise K8S cluster, if that is correct, could you share what guide or documentation did you use to set it up? I have troubles to find proper guide how to setup Metaflow on-premise on official documentation. Thanks
b
Unfortunately, there was no official guide. All of the links kept leading to the AWS terraform documentation working with EKS. We cloned the helm chart, reverse engineered it, fixed it and then deployed it out. We're not exactly sure if it's all correct, but things seem to be running so...maybe?