I just discovered Marimo notebooks! Anyone use Ma...
# ask-metaflow
b
I just discovered Marimo notebooks! Anyone use Marimo notebooks with MetaFlow yet? https://marimo.io/
@lively-lunch-9285 check it out.
l
My biggest concern when I saw this originally was the ecosystem Jupiter is so ingrained and everything Like if you open up sage maker or databricks you’ll get a Jupiter experience or something really close to it And data scientist have often put a lot of effort into learning the magic functions and other special tricks for Jupiter
Also, VS code and pycharm have some really nice support for Jupiter
That said, I haven’t tried to do any serious development with Marimo
b
Womp. Rain on my parade man! 🤣 🤣 🤣
f
I think the reactive model would be neat to see in cards, haven't looked closely enough at marimo specifically but I think you can hack cards to work with it or something similar like shiny
b
@flaky-plumber-70709 that’s exactly what I was thinking but didn’t articulate very well. I’d love to get the reactive model working within Metaflow cards!
f
yeah interactivity in cards would be a game changer and opens up some neat possibilities where they could be used in different ways
🙌 1
so the base case would be to use them as they are used currently with some interactivity sprinkled in to explore different facets of run data via some reactive widget
but also allows them to be used more as interactive dashboards - at my last job there was a lot of flows that would email the card contents where really they just wanted to have a static url to give to business users for the latest run's card
b
oh I LOVE that idea!
f
kind of flips the script a bit with metaflow where instead of being a side effect the card is center stage, and the flow becomes a push-button dashboard deployment
I think even more interesting though is that pattern (flows-as-dashboards) combined with event triggering, so can have these fairly reusable interactive dashboards get triggered to build from other runs
b
@flaky-plumber-70709 Yes!!!!! amen brother! Preach! I want this !
f
Interactive cards would be really nice, especially when something fails.
b
@full-kilobyte-32033 amen!
yes!
l
@full-kilobyte-32033 putting on my product hat: Why exactly would an interactive card be nice when something fails? Like, what do you imagine yourself doing with the card in the event of a failure?
f
I think in the failure case a generic card providing context surrounding the error would be nice, sometimes the traceback isn't enough to get everything. Hard for me to come up with a concrete example though. In this case would maybe want something more involved like to trigger another flow to debug the issue / surface relevant info in a card, since metaflow contains all of the environment info you could want in a debug scenario
I build a lot of event driven stuff so triggering other flows is my hammer, see a ton of potential in sharable purpose-built flows that can just compose together
back to the topic though, a while back OB released this repo: https://github.com/outerbounds/metaflow-card-notebook looking at the code it basically runs the notebook -> uses nbconvert to export to html then renders the raw html this pattern should also work w shiny + marimo i think , both run as python files that render html
f
@lively-lunch-9285 our use case for metaflow may not map perfectly onto the general one so not sure how relevant it is, but we're doing piles of program analysis and it's very useful to be able to inspect artifacts after failure in our domain-- you'd be amazed at the crazy code you find out in the real world.
Inspecting those artifacts is often something that you'd really rather do in an environment set up to analyze them as well.
l
^^^ for this: I'd personally like to have some artifact preview / exporation features built into the UI. In ZenML, for example, if you create a dataframe or query a SQL table, a preview of those is readily visible in the pipeline run in their UI. So if you're a manager or less techincal person or just not familiar with the project, it's a really nice help in debugging
I think Metaflow is actually working on this right now ^^^. Basically elevating the status of artifacts in their UI.
🙌 1
🤞 1
f
in your case I think a dedicated app would probably be a better fit (for right now since we're not sure how big a lift interactivity would be to shove into cards), can use the metaflow client to select the run ID and then control what facets are worth looking at. A card could also be used to prepopulate the dashboard w that run ID
f
Yeah, right now we just have a tool that uses the client to pull down artifacts of interest and hope for the best. There's a lot of weirdness around pickle which makes that inconvenient and frankly makes my skin crawl as a security person, but c'est la vie.
(didn't mean to derail the conversation, just putting in my +1)