in our team we are trying to use MetaFlow with Docker desktop, but can get allowed to use our AWS credentis to access our S3 storage remote, we hit a "error" to remote data on S3, we have trying to add our enviroment variables to the compose file with out any luck 😕 hope some one can assistent me 🙂

I'm getting most of my creds from AWS SSM inside the first step of the flow. Does that help?

@step
    def start(self):
        """
        Start-up: check everything works or fail fast!
        Get all of the parameters from SSM.
        """
        import boto3

        ssm = boto3.client("ssm", region_name="us-west-2")
        namespace = <SSM_NAMESPACE>

        # list of env vars to pull from SSM, assign to self.<param>
        parameters = [
            <labels of your SSM vars>
        ]

        for param in parameters:
            try:
                value = ssm.get_parameter(
                    Name=f"/{namespace}/{param}", WithDecryption=True
                )["Parameter"]["Value"]
                setattr(self, param, value)
            except Exception as e:
                print(f"Error getting parameter {param}: {e}")

        self.next(self.handle_auth)

Yeah but you need some how to get creds to get SSM, our problem is we denied to access S3 bucket, we have tryed both /root/.aws and push it direcly into system variables. local metaflow working but metaflow-ui not working with remote S3 bucket, thats our biggest problem right now 😕

I have an IAM role that has access both to SSM and that s3 bucket.