Hello #C020U025QJK, I have been working with @User , @User and @User to build the Metaflow bot 🤖.  Although the bot is not yet ready, I was very curious to learn about any kind of desired functionality anyone has in mind for the bot. Feel free to ping me personally or on this thread for any ideas.

@hallowed-glass-14538, curious ..what was the purpose of the Metaflow bot ? Is it related to handling notifications (success/failure/etc) ?

Getting to know status of your flow and artifact inspection is a good place to start as a usecase for the bot. It also allows for people to know whats happening with their production workflows even without having to VPN into the network 🙈 .

does the bot have any relation to the “notify operator” ? referencing https://docs.google.com/document/d/1QPX2JhiVf1XPAs2u5j3jdBCGzi_NWfDMjl-LhSvvEx8/edit

excited for this! My wish list would be: 1. relays

@notify

messages for success/failure/etc 2. ability to subscribe to flow execution status updates at various granularities in a slack channel. Thinking something similar to having a slack channel dedicated to a github bot. 3. ability to trigger executions of SFN deployed flows (e.g.

/mf FlowName trigger

) 4. ability to render/pretty-print some common “monitoring” type artifacts from flows. It’d be really cool to be able to persist a matplotlib/seaborn/etc figure in a flow artifact and have it rendered in slack! Pretty-printing small dataframes could also be nice for basic model reporting needs.

@adventurous-gigabyte-81428: re: notify operator :: Not yet fully chalked out the UX but integrations like

@notify

can even be completely separate from the bot. @fresh-laptop-72652: 🙌 So many good ideas. Can you give a little detail about the granularity aspect of

2.

. Do you mean: a notification registration regarding the completion of a

@step

which happens via a slack conversation with the bot ?

For sure! I’m thinking along the lines of a slack subscription more so than a conversation with the bot. I enjoy github’s slack bot and being able to have a dedicated slack channel that’s subscribed to a repo for PRs, comments, etc and can even go as granular as commits (too much imo!) So the way I’m thinking about it, having

@step

level messages would be kinda like commits, probably too granular especially if it’s across a team and multiple flows that have been productionalized. However being able to get messages for SFN flow executions succeeding/failing/etc would be super useful! Continuing on this example, instead of

/github subscribe RepoName reviews comments branches pulls

Metaflow could have something like:

/mf subscribe FlowName successes failures

Could you architect the bot such that it need not be SFN but could be KF Pipelines, Argo, or another orchestration engine?

@cuddly-rocket-69327 We have made sure that the bot has no dependency on SFN. It's powered solely by the metadata service.

This is great! Does it poll the Metadata service or can it be alerted via a callback?

Great question! at the moment it's accessed based on per request basis over polling or callbacks.

The current implementation just pings the metadata service to get the data in response to a user request. At a later stage, it would be great to hook up the

@notify

decorator with the slack bot. cc @damp-lizard-48279