The human-centric platform for production ML & AI

Outerbounds

Hello Everyone ,

I <https://www.youtube.com/watch?v=qk4WPN4H4TA|recently gave a talk> about Metaflow and it's recently <http://github.com/outerbounds/metaflowbot|open-sourced slack bot>. As a demo for the talk, I <https://github.com/valayDave/clip_music_video|created a music video generator> ; The video generator creates videos from text description using Metaflow and OpenAI's Clip model.

As a part of the demo, I even hacked a `download` action to <https://github.com/valayDave/metaflowbot-download-action|download artifacts generated in Metaflow> Flows.

Here is a tiny demo of the bot action.

thanks valay! nice job :slightly_smiling_face:

continuing the conversation from question asked in the video -

what is the best way to monitor cpu usage (and other core metrics) per flow + per step that ya'll have found?

I have seen two approaches being used:
1. Use a standard system monitoring tool like DataDog / CloudWatch / Prometheus - preferably something that can monitor each container separately, not just the host.
2. Use a per-task monitoring sidecar like outlined here <https://github.com/Netflix/metaflow/issues/432>

I'm happy to share more details if these approaches sound appealing