Hi everyone, I’m new to Metaflow and looking for some guidance on integrating with Airflow. I’ve set up AWS Managed Metaflow using CloudFormation and AWS Managed Workflows for Apache Airflow, but the challenge I'm currently running into is that Metaflow is currently utilizing AWS Batch, which isn’t compatible with Airflow. Since we’re a startup (under 20 people) without much experience in Kubernetes—and we don’t have any clusters set up—I’m wondering if moving towards Kubernetes is the best path forward or if there’s an alternative integration strategy. If anyone could point me to relevant documentation or share any insights on integrating Airflow with Metaflow, that would be really helpful. Thanks in advance!

Hi, at the moment, our Airflow integration runs workloads on Kubernetes. If you would like to run workloads on AWS Batch, step-functions is a good replacement for Airflow.

Thank you @ancient-application-36103! I'll check out utilizing step functions in the meantime.