acoustic-van-30942
12/08/2023, 5:22 PMforeach
on Metaflow? A user wants to process ~2M files, and they're already doing some batching and then each batch will take advantage of parallel_map
. So wondering what would be a good balance between the horizontal fan out and the batch size?