Hello all,
Any thoughts on Metaflow doing bidirectional artifact tracking?
For example, being able to not only from a given output artifact track back through it's lineage, but from a root/ancestor node search forward to find all descendants?
So I think, currently, users can search backwards to find the hierarchy of artifacts referenced by a run (or ancestors of a run), via manually doing a DFS/BFS traversal.
However, there's a use case where we need to say for a given ancestor bit of data like a certain batch of data, find all the descendent children runs/artifacts/models that are using it. That's the 'other direction' in the bidirectional bit.