The human-centric platform for production ML & AI

Outerbounds

Hi, has anyone used tags in Metaflow’s Metadata Registry to trigger events like Sagemaker endpoint generation? For example, if you wanted to use metaflow’s metadata registry as a source of truth for a scoring service.

Screen Shot 2022-12-12 at 1.35.50 PM.png

yep, that's a great pattern as it allows you to maintain lineage all the way from real-time predictions served by Sagemaker back to data that was used to train the model, leveraging Metaflow's metadata

you could structure <https://outerbounds.com/docs/deploy-with-sagemaker/|continuous deployments to Sagemaker as outlined here>

instead of deploying every new model unconditionally, you can have a human in the loop to validate each model manually (e.g. by running a bunch of tests and analyzing results) and then promote to production via a tag, <https://outerbounds.com/blog/five-ways-to-use-the-new-metaflow-tags/#tags-and-continuous-training|as outlined here>