2023-03-28 08:52:54.147 [1679978885045759/main/2 (pid 98)] S3 access failed:
2023-03-28 08:52:54.147 [1679978885045759/main/2 (pid 98)] S3 operation failed.
2023-03-28 08:52:54.168 [1679978885045759/main/2 (pid 98)] Key requested: <s3://my-bucket/datastore/myflow/1679978885045759/main/2/0.attempt.json>
2023-03-28 08:52:54.169 [1679978885045759/main/2 (pid 98)] Error: An error occurred (400) when calling the HeadObject operation: Bad Request
2023-03-28 08:52:54.169 [1679978885045759/main/2 (pid 98)]
2023-03-28 08:52:54.169 [1679978885045759/main/2 (pid 98)] Exception ignored in atexit callback: <function Batch.__init__.<locals>.<lambda> at 0x7f3522ba4940>
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] Traceback (most recent call last):
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch.py", line 52, in <lambda>
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] atexit.register(lambda: self.job.kill() if hasattr(self, "job") else None)
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 612, in kill
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] if not self.is_done:
2023-03-28 08:52:54.170 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 576, in is_done
2023-03-28 08:52:54.171 [1679978885045759/main/2 (pid 98)] self.update()
2023-03-28 08:52:54.171 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 532, in update
2023-03-28 08:52:54.171 [1679978885045759/main/2 (pid 98)] self._update()
2023-03-28 08:52:54.851 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 479, in wrapped
2023-03-28 08:52:54.851 [1679978885045759/main/2 (pid 98)] func(*args, **kwargs)
2023-03-28 08:52:54.851 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 520, in _update
2023-03-28 08:52:54.851 [1679978885045759/main/2 (pid 98)] raise err
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/metaflow/plugins/aws/batch/batch_client.py", line 515, in _update
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] data = self._client.describe_jobs(jobs=[self._id])
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 530, in _api_call
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] return self._make_api_call(operation_name, kwargs)
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 960, in _make_api_call
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] raise error_class(parsed_response, operation_name)
2023-03-28 08:52:54.852 [1679978885045759/main/2 (pid 98)] botocore.exceptions.ClientError: An error occurred (ExpiredTokenException) when calling the DescribeJobs operation: The security token included in the request is expired
2023-03-28 08:57:40.244 Workflow failed.