Hey Metaflow, We are running metaflow in prod on AWS and it is going great. Looking to deploy the UI and get that up and running too, one of the big appeals of the metaflow backend was how easy it was to deploy with the existing cfn templates etc. Is there any plans or documentation for integrating the UI with an already deployed metaflow infrastructure? I can see you have docs, but it feels a lot more daunting than what was involved initially. Thanks for an awesome product 🙂

Hey thanks Ben! Its definitely possible to add UI to the existing infra, and we plan to add more docs on that. The thing with web UIs in my experience is that deploying them usually involves more moving parts like auth and security, that are harder to make one-click, hence the extra complexity. Especially if it is for a production environment. The very basic answer is that one can use the new Cloudformation template to upgrade from the old one. Or deploy the UI using terraform module if you prefer that (our terraform template is more modular than Cloudformation so you can just use the UI template on its own there).

ah yeah, i did see that, thanks I will check it out

yea that's where it gets tricky since you probably don't want to have separate auth system for every piece of software :) this said the latest Cloudformation template does come with auth for the UI out of the box (through AWS Cognito)

ah ok, I didn't know that, I cant see it in the `AWS CloudFormation Deployment`but see it in the cfn template. Is there any docs on it that you can point me towards?

this https://github.com/outerbounds/metaflow-tools/tree/master/aws/cloudformation#optional-metaflow-user-interface-enableui (we're actually overhauling the docs site right now and we'll be adding a lot more content soon)

we have metaflow deployed in kubernetes, in a more "manual" way there is an env var to set basic auth

METAFLOW_SERVICE_AUTH_KEY

so we have that set on the client side and added a nginx on top of the metaflow service container basically the CMD for the docker file (it's just a small modification on top of

netflixoss/metaflow_metadata_service

) looks like

#!/usr/bin/env bash

set -eux

envsubst < /etc/nginx/conf.d/metaflow.conf.template | sed -e 's/§/$/g' > /etc/nginx/conf.d/metaflow.conf

/etc/init.d/nginx start

/opt/latest/bin/python -m services.metadata_service.server

and the nginx template

map §http_x_api_key §is_ok {
    default "0";
    ${AUTH_KEY} "1";
}

server {
    listen ${NGINX_PORT};
    server_name localhost;

    location /ping {
        proxy_bind 127.0.0.1;
        proxy_pass <http://localhost:8080/ping>;
        proxy_set_header Host "localhost";
    }

    location / {
        if (§is_ok !~* "1") {
            return 405;
        }

        proxy_bind 127.0.0.1;
        proxy_pass <http://localhost:8080/>;
        proxy_set_header Host "localhost";
    }
}