Hi all, It is my understanding that the trigger and trigger_on decorators are only available for argo workflows. My company relies heavily on airflow and thus i would be willing to submit a pull request to enable this feature for airflow. When digging into the airflow pulgin codebase i saw a comment referrencing a commit on the airflow branch that seems like it could be helping in my endaveour, but i was unable to find it. Is anyone able to point me in the right direction ? Thanks a lot for the help

Hey hugo, even though we don't have trigger_on decorators working with Airflow, We have support for ExternalTaskSensors as flow decorators(

@airflow_external_task_sensor

); Does that help ?

Hey Valay, thanks for the reply. I've seen this decorator while browsing the metaflow codebase unfortunately this is not what i am looking for. The external task sensor can only hook on one external dag, but i'd like my flows to trigger on update of multiple datasets (my flows rely heavily on data aware scheduling). The best solution i've found as of now is to disable any scheduling on metaflow and embed every metaflow generated dag into another dag which triggers on dataset update and launches my metaflow dags through a triggerdagoperator. This also allows me to trigger airflow outlets at the end of my flow so that i can launch other flows/dags that might be interested by the output of the flow. This works pretty well but this requires a lot of boilerplate code is a bit hacky. As said above, provided with a bit of guidance i am willing to submit a PR to the metaflow codebase to implement trigger and trigger_on decorators but unfortunalety i am unable to find the commit referenced by the comment in the code. Can you help me out ? Thanks !

@happy-monkey-82439 following up on this - we would be quite interested in exploring data aware scheduling integration within Metaflow. can you help us with what do you have in mind in terms of a potential implementation?

@happy-monkey-82439 this Matryoshka approach is interesting. @square-wire-39606 in addition to the DAS implementation with

airflow create

I wonder if it makes sense to create a new metaflow operator/decorator that runs the metaflow DAG. This way we can let users customize inlets and outlets as Hugo mentions. Also probably other user customizations.