https://outerbounds.com/ logo
Join Slack
Powered by
Hi all, I am starting to use MetaFlow again and D...
# ask-metaflow
h
Hi all, I am starting to use MetaFlow again and DevOps kindly restarted and upgraded metaflow in our AWS. They also provided me with credentials for an AWS profile I use in VSC called 
metaflow
. Alas, when I run some old flow I now get: 2024-03-14 155544.056 [15/bootstrap/49 (pid 63008)] [4f628202-99d3-45ab-8dd0-968c47c65809] Environment bootstrapped. 2024-03-14 155545.810 [15/bootstrap/49 (pid 63008)] [4f628202-99d3-45ab-8dd0-968c47c65809] <flow BlaFlow step bootstrap> failed: 2024-03-14 155546.795 [15/bootstrap/49 (pid 63008)] [4f628202-99d3-45ab-8dd0-968c47c65809] S3 access denied: 2024-03-14 155546.795 [15/bootstrap/49 (pid 63008)] [4f628202-99d3-45ab-8dd0-968c47c65809] s3://metaflow-684414486554-s3-55ff636/metaflow/BlaFlow/data/d0/d02b58387a6fdeb3e003a4526d3e8960d5f6056e 2024-03-14 155546.795 [15/bootstrap/49 (pid 63008)] [4f628202-99d3-45ab-8dd0-968c47c65809] 2024-03-14 155611.460 [15/bootstrap/49 (pid 63008)] AWS Batch error: 2024-03-14 155611.554 [15/bootstrap/49 (pid 63008)] Essential container in task exited This could be a transient error. Use @retry to retry. I assume this has to do with the fact that the AWS 
metaflow
profile I use does not have certain rights? Does this ring a bell? Thanks! I invoke code like so (worked in the past:
Copy code
python3 BlaFlow.py --environment=conda --package-suffixes .sql run \
--p11 1.00 \
--p2 1.2 \
--s3_bucket_name bal --with batch
✅ 1
v
hey Christian meow wave
can you confirm that your AWS user/role has access to the s3://metaflow-684414486554-s3-55ff636/ S3 bucket? The 
S3 access denied
error indicates that it might be an S3 issue
👍 1
h
Thanks will check as soon as I get hold of soon from devops - late here now …
👍 1
So devops made an adjustment and I appear to have got further this time: 2024-03-14 164615.842 [17/create_scoring_data/57 (pid 70503)] [5078f171-434e-4381-88b5-abfa2234fd00] S3 access denied: 2024-03-14 164641.154 [17/create_scoring_data/57 (pid 70503)] AWS Batch error: 2024-03-14 164641.298 [17/create_scoring_data/57 (pid 70503)] Essential container in task exited This could be a transient error. Use @retry to retry. 2024-03-14 164641.298 [17/create_scoring_data/57 (pid 70503)] 2024-03-14 164615.842 [17/create_scoring_data/57 (pid 70503)] [5078f171-434e-4381-88b5-abfa2234fd00] s3://metaflow-684414486554-s3-55ff636/metaflow/BlaFlow/data/be/be4eb098c7b1cf126e45b1e43132abb6444ddb65 2024-03-14 164615.842 [17/create_scoring_data/57 (pid 70503)] [5078f171-434e-4381-88b5-abfa2234fd00] 2024-03-14 164641.526 [17/create_scoring_data/57 (pid 70503)] Task failed. Looks like the same issue. Please not that AWS batch uses Sagemaker batch scoring at this point - maybe needs some more rights?
v
hmm, there's still some 
S3 access denied
at the top
it could be some S3 issue still
8 Views
Open in Slack
PreviousNext
Powered by