Hello, sorry to create a new thread, but I think this is different than my previous timeout issue. I've noticed that when I use s3.get_many vs. s3.get with files that contain non-english characters in the file name (e.g. Japanese Kanji) I see a perpetual s3 transient error that doesn't resolve. I can confirm that the s3 file exists by doing an s3 ls command and seeing it exactly as I'm providing to the s3.get command. When I use s3.get("filename") it works fine. When I use s3.get_many(["filename"]) I see the following Transient S3 failure which doesn't resolve.

Transient S3 failure (attempt #1) -- total success: 18, last attempt 18/20 -- remaining: 2
Transient S3 failure (attempt #2) -- total success: 18, last attempt 0/2 -- remaining: 2
Transient S3 failure (attempt #3) -- total success: 18, last attempt 0/2 -- remaining: 2
Transient S3 failure (attempt #4) -- total success: 18, last attempt 0/2 -- remaining: 2

@hundreds-rainbow-67050 can you take a look? @future-crowd-14830 can you try it out with a rather old version of metaflow - say 2.5.0 and see if you are still running into the same issue?

@future-crowd-14830 can you give an example filename?

Unfortunately the file names contain private data. I tried with Metaflow version 2.5.0 and the issue does not occur. The 18 are all english characters only in the filename, the 2 contain Japanese Kanji in the file name.

i'm taking a look. meanwhile, can you try with this branch: https://github.com/Netflix/metaflow/pull/2616 it will show what the error is

Not sure if I did it correctly. I used:

uv add git+<https://github.com/Netflix/metaflow.git@npow/s3-log-original-error-code>

But I still don't see any detail, e.g. the output is still:

Transient S3 failure (attempt #1) -- total success: 1, last attempt 1/2 -- remaining: 1
Transient S3 failure (attempt #2) -- total success: 1, last attempt 0/1 -- remaining: 1
Transient S3 failure (attempt #3) -- total success: 1, last attempt 0/1 -- remaining: 1
Transient S3 failure (attempt #4) -- total success: 1, last attempt 0/1 -- remaining: 1
Transient S3 failure (attempt #5) -- total success: 1, last attempt 0/1 -- remaining: 1

not sure about that command.. maybe you need to regenerate the lock?

Tried to use a blank new project,

uv pip install git+<https://github.com/Netflix/metaflow.git@npow/s3-log-original-error-code>

as well but still no further detail on the transient error, same output.

ok i reproduced the problem, the code isnt printed for download (only upload)

Ahh seems like its fixed, both files were downloaded fine now.

no, that case is handled. first part of the filename is a unique hash that doesnt get truncated

Awesome, thank you very much!

will release once it's reviewed by @ancient-application-36103 @dry-beach-38304

Hey @hundreds-rainbow-67050 Just following up to see if this fix is scheduled in a release yet.

sorry this one slipped through the cracks. followed up to get it reviewed

Thank you!