I was reading the Fast Data Blog and have a question about the implementation of loading the PyArrow tables, why is it that the implementation leverages a ThreadPool to fetch the files? Doesn’t

S3().get_many()

run in parallel as well?

great question! If you look closely, the files are actually fetched outside

ThreadPoolExecutor

here: https://github.com/outerbounds/fast-data-blog/blob/main/table_loader.py#L27

Ah got it, thanks @straight-shampoo-11124!

great!