https://outerbounds.com/ logo
Join Slack
Powered by
Running `public.ecr.aws/outerbounds/metaflow_metad...
# ask-metaflow
b
Running 
public.ecr.aws/outerbounds/metaflow_metadata_service:2.3.11
and getting
Copy code
| connecting
| booting...
| [Errno 111] Connection refused
| 3
| connecting
| Port reachable 8082
| INFO:AsyncPostgresDB:global:Connection established.
|    Pool min: 1 max: 10
I see in the repo https://github.com/Netflix/metaflow-service there's a migration service running on that port. 1. Is it required to run that migration service separately? a. If so, is there a pre-built image for this service? I didn't see it in the public ECR registry. 2. What is the relationship between the 
Netflix/metaflow-service
and 
outerbounds/metaflow_metadata_service
? a. It seems latest on DockerHub for Netflix is 
v2.4.3
whereas outerbounds image is at 
2.3.11
(noting there are more recents, but they're tagged with a different label (e.g., 
v2.3.8-26-g063a949-obp
) I'm new to Metaflow and trying to get a local docker-compose environment up and running with Argo + Minio and just struggling a little with bridging between Netflix/Outerbounds a bit. Otherwise, this migration thing has been the only snag. It does work if I "up" the 
docker-compose.development.yaml
✅ 1
Further, I think my confusion stems from images available in ECR/Dockerhub for 
metadata_service
and 
metaflow-ui
but not for the 
ui-backend
There is a ui-backend in ECR but there's only 2 versions and 2 years old. Hopefully this helps narrow down on why I'm confused.
b
The official image for the services is published under 
netflixoss
in dockerhub so I would suggest using these as they should be the most recent and stable. It seems that the UI's (frontend part) docker image is published under the Outerbounds ECR. This is provided as an alternative to setting up serving the static assets. About the contents of the images: for the services, we only release one image which contains all three services, 
metadata/migration/ui_backend
and the way it is used in f.ex. the cloudformation templates is by simply varying the Command for the container
For local development the docker-compose should work and you do not need to care about the release images, as everything is built locally. If you want to try mimic a production deployment locally, then I would disregard the existing docker-compose.development setup and try to follow one of the existing templates for how to run the services
and lastly on the role of the migration service. This exists to give more control on how and when to run migrations against the DB during deployments, as it can be upgraded separate from the running services. The service itself can be shut down once the DB schema is up-to-date, and there are nowadays two ways of running the migrations • spinning up the migration service, which exposes API endpoints for triggering the migration, and checking status • alternatively running the 
run_goose.py
inside the migration service, which simply executes migrations immediately and exits. The run_goose script is the default way that migrations are run in the development setup.
For the metaflow-UI frontend, there's three ways to host this • (default) serving the static files through the ui_backend. A version of the static assets are bundled into the backend release • the released Dockerimage, which is mostly for convenience for some platforms • hosting the static files from the release yourself: https://github.com/Netflix/metaflow-ui/releases
b
Thank you for the detailed response @brave-flag-76472, this helps my understanding substantially.
22 Views
Open in Slack
PreviousNext
Powered by