green-analyst-32514
02/22/2024, 8:44 PMNot yet supported with Airflow:
The foreach step format_dtype created by step load_datasets needs to have an immediate join step. Step impute_null is invalid since it is a linear step with a foreach. This type of graph is currently not supported with Airflow.
Is support for this type of graph and/or nested foreaches on the roadmap for the Metaflow team? My team is looking to begin deploying/scheduling our Flows, and we want to understand if this capability is being prioritized/worked on in the Airflow integration, or if we should consider using Argo Workflows/Step Functions.