Hey! Is it possible to run multiple steps on the s...
# ask-metaflow
c
Hey! Is it possible to run multiple steps on the same batch? • Instead of running the first step completely on all the batches, joining data, and only then start the second step in batches can we simply run both steps on the same batch sequentially? • Doing so would not require to have all data in memory (required for the joining step). The maximum current memory would be that of the current batch • There’s the workaround to have a single step, but I would prefer a less monolithic approach if possible Thanks a lot!
1