Hello, With the help from the threads here, especially the one from @fresh-laptop-72652 (Thanks a lot 🙏), I was able to mount larger volumes to root 🙂 Right after I noticed tmpfs support was added for batch. Is this the recommended way instead of launch templates if all a user care about is storage? As one would still want to use launch templates to modify ECS clean up tasks etc. Any thoughts here. Thanks!

good to hear that you got it working! 🤗

For our usecase, we run an sdk that expects videos in a certain format - on disk, and the videos and artifacts could sum upto 20 GB. So unfortunately I am unable to use the full merits of metaflow.s3 saving data to page cache (I have to move them to specific location with specific name). I shall try to use tmpfs without launch template hope it goes smooth 🙂

it should be very easy: just do

@batch(use_tmpfs=True)

by default it allocates 50% of

@resources(memory=)

to tmpfs but the memory is only used when you store something to the volume

Didn't know about current.tempdir, thank you!

it’s all new so practically no one knows about it 😅 docs coming soon