I try to run sagemaker batch transform via @batch in the cloud but get: Got client error from S3 due to ‘403 Forbidden’ processing getObjectMetadata for bucketName:[blabla], objectKey:[xyz/batch_score_b073a5a9-fddd-4338-bcba-fde649a54952/scoring_files/b073a5a9-fddd-4338-bcba-fde649a54952.csv] Does anyone know, which iam metaflow role needs adjusting and how? my guess is: metaflow-684414486554-batch_s3_task_role-55ff636 thanks!

where exactly does this error show up?

aws/sagemaker/TransformJobs batch-transform-job-2023-06-02-16-28-04-342/i-0228a721aca7f227c-1685723458/data-log so I have a step like below. the job is triggered but fails I presume when the outputs are written …

@batch

@step

def trigger_batch_transform(self):

import sagemaker

from sagemaker.transformer import Transformer

score_files = aws_s3.get_files_in_folder(self.s3_bucket_name, self.score_sub_folder_name)

for score_file in score_files:

sagemaker_session = sagemaker.Session()

input_data = f's3://{self.s3_bucket_name}/{score_file}'

output_path = f's3://{self.s3_bucket_name}/{self.score_results_sub_folder_name}'

transformer = Transformer(

model_name=self.model_name,

instance_count=1,

instance_type='ml.m5.4xlarge',

strategy='MultiRecord',

assemble_with='Line',

output_path=output_path,

base_transform_job_name='batch-transform-job',

sagemaker_session=sagemaker_session,

accept='text/csv'

)

transformer.transform(

data=input_data,

data_type='S3Prefix',

content_type='text/csv',

split_type='Line'

)

transformer.wait()

self.next(self.end)

does it fail when the sagemaker job is submitted or while the sagemaker job is running?

I think so yes. Is it the one attached to metaflow-684414486554-batch_s3_task_role-55ff636?

you may want to grant additional permissions to your default sagemaker role so that it can read/write from that bucket - https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html#sagemaker-roles-create-execution-role

if I run the above which metaflow role runs this? I would think that I have to add an inline policy like this: { “Version”: “2012-10-17", “Statement”: [ { “Effect”: “Allow”, “Action”: [ “s3:GetObject”, “s3:PutObject”, “s3:DeleteObject”, “s3:ListBucket” ], “Resource”: [ “arnawss3:::my-bucket”, “arnawss3:::my-bucket/*” ] } ] } I have these 2 metaflow roles relevant in this context?: metaflow-684414486554-batch_s3_task_role-55ff636 metaflow-684414486554-batch-execution-role-55ff636

great!