Has anyone used <S3's intelligent tiering> for the...
# ask-metaflow
q
Has anyone used S3's intelligent tiering for their metaflow s3 bucket? Seems pretty trivial to set up and use which makes the skeptic in me suspicious if there’s something I missed considering.
1
s
It should work but performance might be a bit off
q
Surprisingly AWS guarantees milliseconds access times even for infrequent tiers (as long as you don’t go all the way down to deep archive — which is opt-in), and once accessed the objects are moved back to standard tier. So theoretically it doesn’t seem there would be any perf impact at all
s
🥳 do share what the real world results look like on your end!
1
Looks like Netflix’s metaflow bucket (which holds dozens of petabytes of metaflow artifacts) is on intelligent tiering without any issues
🙌 1