The human-centric platform for production ML & AI

Outerbounds

:wave: hey folks…is there a way to use metaflow with argo’s synchronization (<https://argo-workflows.readthedocs.io/en/latest/synchronization/|docs>) to limit the number of workflows running concurrently?

not today - is the goal to execute at most N executions concurrently where N can be greater than 1?

<@U01U2JMQW5A>  almost, I’d rather N &gt;= 1 (since we might want a flow to only be running one at a time)

<@U01U2JMQW5A> I assume this could be added to the metaflow logic via a argo specific decorator, though right?  Just curious if there is a known limitation in argo/metaflow and thus it’s just a PR away from happening or if there is a bigger issue at play

Just a simple PR away from introducing support for N=1 (via mutex). We would have to think through N &gt; 1 since managing a central counter using config maps can be tricky if you have multiple people working on updating the same workflow template.