The human-centric platform for production ML & AI

Outerbounds

Hi! Our org is currently using metaflow to run scheduled jobs (through AWS step functions), but something we're currently missing is visibility into who deployed which version of the flow, especially when a step fails and we need to debug it.
Do you have any recommendations on tracking lineage of flows deployed as step functions?

if you grep for `metaflow.owner` in the generated state machine definition, that would list who deployed the flow. we can definitely look into lifting this information up into the UI

Got it, is there also a way to track what version of the code was deployed? (Or to deploy it based on a specific version of the repo which the flow is saved in?)

you could do something like `python flow.py step-functions create --tag git-hash:$(git rev-parse --short "$GITHUB_SHA")`

the actual code that is executed is already stored by metaflow in s3, so you don't technically need to rely on git

Oh gotcha, yeah this is mostly to just line up git commits with deployed versions