Hi, We have an old Metaflow setup (2.5.0) on AWS, deployed using CloudFormation. Trying to upgrade this, but locally while giving a job using an upgraded Metaflow (2.10), inside Batch we get a s3 access denied error

2023-11-15 09:02:46.980 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]     stdout_lines, stderr = self._s3op_with_retries(
2023-11-15 09:02:46.980 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]   File "/metaflow/metaflow/plugins/datatools/s3/s3.py", line 1697, in _s3op_with_retries
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]     ) = try_s3_op(last_ok_count, pending_retries, out_lines, inject_failures)
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]   File "/metaflow/metaflow/plugins/datatools/s3/s3.py", line 1681, in try_s3_op
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]     raise MetaflowS3AccessDenied(err_out)
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e] metaflow.plugins.datatools.s3.s3.MetaflowS3AccessDenied: s3op failed:
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e] Access denied to URL: <s3://metaflow-stack-metaflows3bucket>-<pvt>/metaflow/conda/conda.anaconda.org/conda-forge/linux-64/tk-8.6.13-noxft_h4845f30_101.conda
2023-11-15 09:02:46.981 [10656/run_on_batch/265393 (pid 13400)] [065eb4d7-9332-4629-be20-5a60aeca9a9e]
2023-11-15 09:02:49.863 [10656/run_on_batch/265393 (pid 13400)] Task failed.
2023-11-15 09:02:49.975 Workflow failed.
2023-11-15 09:02:49.975 Terminating 0 active tasks...
2023-11-15 09:02:49.976 Flushing logs...
    Step failure:
    Step run_on_batch (task-id 265393) failed.

It works on a fresh setup of upgraded Metaflow though. I can compare different Cloudformation templates which was used to deploy, but just wondering if there was a migration guide somewhere, of what changed between different templates