Hi, just doing some exploration work on Metaflow. I'm finding that to successfully run a flow, I needs to be logged in with an aws user/role that has write permission to the Metaflow s3 bucket, else I get an

S3 access denied

error. Just wondering if this is by design or if I have missed something... was under the impression that all the permissions to interact with s3, dynamodb, rds etc are handled by Metaflow as these are part of the Metaflow Infrastructure, and not rely on the user's permission?

yeah, you need permissions to access the bucket by design. If you don’t want to worry about any infrastructure, you can use Metaflow in the sandbox

thanks, that's interesting, it seems all the permission required is just the s3 bucket, everything else works like the metadata still can be written to RDS.

right, it’s because Metaflow talks directly to S3 instead of routing data through any service It’s a deliberate decision that makes Metaflow much more scalable and robust since there are not middle layers handling data, but a downside is that all clients need access to a bucket

ok, thanks for the explanation 👍