Hi team, I'd be interested in contributing to open source to get Ray running on top of AWS Batch multi-node parallel jobs through Metaflow. I asked the Ray team, and they indicated that it is likely feasible to run Ray on AWS Batch. In my mind, I'm thinking the user could use the

@parallel

decorator to initiate the gang-scheduled cluster, and then maybe do all the Ray configuration through the AWS Batch job definition? Would love a second perspective from Outerbounds on this. These are the docs for installing Ray on on-premise clusters: https://docs.ray.io/en/latest/cluster/vms/user-guides/launching-clusters/on-premises.html#manual-cluster-launcher