Hello .. I am following the tutorial to use MF on AWS infra. Currently using the suggested cloud formation template. When I run my flow locally, it gets executed. When I changed it to run such that

--datastore=s3

, it also runs successfully. The execution fails when I have

--with batch

. The error message is

2023-02-18 23:42:12.237 Workflow starting (run-id 1676792530723739):
2023-02-18 23:42:25.550 [1676792530723739/start/1 (pid 41117)] Task is starting.
2023-02-18 23:42:27.145 [1676792530723739/start/1 (pid 41117)] [79e6b696-1231-4cc8-ac53-e57cd2d752e8] Task is starting (status SUBMITTED)...
2023-02-18 23:42:31.308 [1676792530723739/start/1 (pid 41117)] [79e6b696-1231-4cc8-ac53-e57cd2d752e8] Task is starting (status RUNNABLE)...
2023-02-18 23:42:33.511 [1676792530723739/start/1 (pid 41117)] [79e6b696-1231-4cc8-ac53-e57cd2d752e8] Task is starting (status FAILED)...
2023-02-18 23:42:34.307 [1676792530723739/start/1 (pid 41117)] Data store error:
2023-02-18 23:42:34.307 [1676792530723739/start/1 (pid 41117)] No completed attempts of the task was found for task 'ProcessFlowDemo/1676792530723739/start/1'
2023-02-18 23:42:34.720 [1676792530723739/start/1 (pid 41117)] 
2023-02-18 23:42:34.901 [1676792530723739/start/1 (pid 41117)] Task failed.
2023-02-18 23:42:35.057 Workflow failed.
2023-02-18 23:42:35.057 Terminating 0 active tasks...
2023-02-18 23:42:35.058 Flushing logs...
    Step failure:
    Step start (task-id 1) failed.

Actual code:

from metaflow import FlowSpec, step
import os

global_value = 5

class ProcessFlowDemo(FlowSpec):
    
    @step
    def start(self):
        global global_value
        global_value = 8
        print(f"Process id: {global_value}")
        print(f"Process id: {os.getpid()}")
        self.next(self.end)

    @step
    def end(self):
        print(f"Process id: {global_value}")
        print(f"Process id: {os.getpid()}")

if __name__ == '__main__':
    ProcessFlowDemo()

As you can there is no permission issue to connect to S3 from Batch. Please LMK if you need additional info to debug this. TIA!