Hi folks, I’m running into an issue with using the

S3

client to upload data. Specifically, I am getting an error saying

Error: An error occurred (InvalidToken) when calling the PutObject operation: The provided token is malformed or otherwise invalid.

when I try to upload a pandas DataFrame that’s been converted a CSV encoded in UTF-8. Here is a snippet of my code:

@step
def upload_df_to_s3(self):
    with S3(s3root='<s3://my-bucket/prefix/>') as s3:
        bytes_to_upload = self.df.to_csv(None, index=False).encode()
        s3.put('df.csv', bytes_to_upload)
    self.next(self.next_step)

Any help would be appreciated! Thanks!

it seems like an issue with the aws credentials you might have locally

yep! I can run

aws s3 ls

without errors

and is the region specified for your aws credentials?

yep!

can you try using

aws s3 put-object

command on the CLI to ensure that your credentials are valid?

I did a dry run, of the

cp

command, and it worked