Hey folks, Does anyone have any recommendations o...
# ask-metaflow
a
Hey folks, Does anyone have any recommendations on what is the maximum amount children one can have when using
foreach
on Metaflow? A user wants to process ~2M files, and they're already doing some batching and then each batch will take advantage of
parallel_map
. So wondering what would be a good balance between the horizontal fan out and the batch size?
1