Hi team, I have a similar issue as the OP. When I...
# ask-metaflow
w
Hi team, I have a similar issue as the OP. When I run a simple flow directly from an EC2 instance (4 vCPUs/7 gigs of memory) or a local machine, the flow completes execution within 1 minute 30 seconds. However, when run from Step Functions, the flow takes almost 6 minutes 30 seconds to complete. The batch compute environment has almost double the resources (16 vCPUs/30 gigs of memory) compared to my local machine or the EC2 instance, from which I’m getting better performance. I understand that there is overhead due to cold start, network latency, state transition, etc., but this much disparity in performance puzzles me. As a new user of Metaflow, I apologize in advance for any ignorance on my part. Any help is appreciated. Metaflow was provisioned using cloudformation.