👋 New to Metaflow, and struggling a bit conceptualizing how we set up a nice ingress path in GCP. I originally asked this on Github, but someone rightly pointed out that having an opinionated ingress pattern in a general terraform model was probably bad practice. So...any tips on how to give my data scientists a good experience accessing Metaflow? Ideally, I'd be able to say something akin to "You can browse to https://metaflow.example.com to see the UI, and use https://metaflow.example.com/api in your local config to send flows." But otherwise, any advice on general "best ingress practices" would be appreciated.

good question. The two main ways would be 1. VPN - you can access Metaflow resources like any other resources inside your private network 2. Some Auth at the edge - I know some folks have set up e.g. basic auth with a shared token using API Gateway on AWS. They must have something similar on GCP

If I can add a load balancer in front, I can easily add IAP (Zero trust auth).

yeah, that should work. The UI and the metadata service are behind a normal HTTP(s) / websocket API, so standard tools should work

I kinda like solving problems like this. Is it reasonable to put them behind one load balancer? Something like:

apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
  name: metaflow-ingress
  annotations:
    # If the class annotation is not specified it defaults to "gce".
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: "gce"
spec:
  rules:
  - http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: ui-backend
            port:
              number: 8083
      - path: /metadata
        pathType: Prefix
        backend:
          service:
            name: metadata-service
            port:
              number: 8080
      - path: /*
        pathType: Prefix
        backend:
          service:
            name: ui-static
            port:
              number: 3000

@average-beach-28850 any advice here?

typical setup is two ingresses, one for UI (ui-backend + ui-static) with IAP style interactive browser authz in front of it, and one for metadata service with some sort of authz based on API keys for programmatic access

Is there any documentation of that "typical" pattern anywhere? I've struggled to find any.

for K8S currently not really but its kind of following the pattern we use for AWS managed services setup like in this diagram https://outerbounds.com/engineering/deployment/aws-managed/terraform/

Thanks!