Hello, I have a question related to describing flows. For experiments or general data processing, there is value in having some description of it, it may be the purpose of running it or the observation after you run it. Currently I have create a card for start step with markdown table where I have a row, Description: Extracting features for dataset X with some new parameters X, Y. And also some tags like dataset name, what features are extracted. But currently i do not see an option other than tags that could be updated after the run. It would be nice to have a notes feature for all runs which are mutable unlike the artifacts of a flow run. I have seen suggestions here for something related to this like - Adding a tag "altered" if the artifacts on S3 were deleted or changed. Another way would be to add the markdown table with the notes at the end step without anything else. That way one could resume the successful run from the end step alone, there by pointing to artifacts of the original run with only the notes in the end step being updated. How are others dealing with it, is it not of interest or are the notes elsewhere? Thank you 🙏

@jolly-afternoon-77975 you can also add notes as doc strings for specific steps. these are captured and made available through the client as well as through the UI

@square-wire-39606 Thanks for the reply, yes thats another way to write some docs, but its not mutable right? I cant update it after the run? I can view all this info via the client but I cant update them right?

that's correct - another mechanism would be to embed an iframe to another wiki inside the cards

thanks for the clarification. Cool so basically adding the notes outside of metaflow. I was hoping there would be an option now or in the future so its in metaflow where I could access and update with the client directly. I guess providing easy external link would be very useful and with the view card option its still easy to view that info even though you cant get the info as a string with the client. That's a surprisingly neat idea, thank you very much thankyou

It's more involved and limited to the UI but in theory I think you could also create a ui-plugin for this functionality, can see a handful of examples in the Outerbounds repo here: https://github.com/outerbounds/mfgui_plugins/tree/main It should be possible to implement add notes at the run or task level but haven't looked too closely at the plugin docs yet so can't say for sure: https://github.com/Netflix/metaflow-ui/blob/master/docs/plugin-system.md

Wow thanks @flaky-plumber-70709, that's something I haven't thought about. I realized the iframe option @square-wire-39606 suggested is pretty neat but quickly realized the problem of authentication, it's pretty straightforward for public notes but for internal repository or something like confluence it's becomes hard. I shall look into this and see how easy it is. Thank you