Hey folks, we're currently evaluating metaflow and...
# ask-metaflow
f
Hey folks, we're currently evaluating metaflow and have a number of steps that are expensive-to-compute pure functions. Is there a way to reuse the results of those steps between runs?
1
s
yeah, you can take a look at this pattern of resumable computing. The idea is to save results as artifacts and then use the Client API to check what data already exists and only compute the missing parts
f
Very interesting, thanks for sharing
I'm curious about this kind of thing living outside of metaflow proper-- is this just a case where there's no real need to have it live in metaflow, or is there a philosophical/logistical/etc reason why lots of these don't wind up getting contributed back?
s
short answer: no reason why it couldn't be included, and especially documented better technically it's just 40 lines of code using artifacts and Client API, so in that sense it's "just a convenience wrapper" over first-class primitives, which you can easily use without extra support, but not a good reason for not including it I opened an issue to get it included 🙂
f
Awesome, glad to hear that there isn't any kind of barrier there-- I'll move further questions/comments onto that issue if that's the right place?
s
sure, either here or there works
f
Moved over there, happy to move it again PRN.
👍 1