Hey all, I am currently facing a weird issue where running on argo workflows with more than 4 branching steps causes error like this, weirdly this is not happening in Step Functions, need some guidance on this thanks: split logic:

@step
    def my_step(self):
        self.next(
            self.branch1
            , self.branch2
            , self.branch3
            , self.branch4
            , self.branch5
            , self.branch6
            , self.branch7
        )

error:

Setting up task environment.
Downloading code package...
Code package downloaded.
Task is starting.
    Data missing:
    Some input datastores are missing. Expected: 7 Actual: 0

https://github.com/Netflix/metaflow/releases/tag/2.12.6 went out yesterday which contains a fix for this issue.

thanks! will test it from my end

the cause was actually that the list compression is not compatible with Argo workflows to begin with, as Argo replaces values for tags

{{}}

during template evaluation/pod scheduling, and we perform the list decompress during runtime, which just results in a bunch of tokens without actual values

right, just a followup question, is there a limit of the input-paths which kinda limits how many splits or artifacts we can have?

the main limiting factor is the size of

ARGO_TEMPLATE

env var on the pods where argo keeps the complete workflow manifest with values. There was an issue in the past with extremely wide foreach steps running into the limit, but even these could hold ~1k step names in there, which is why I'm expecting that any static flow written by hand should not be running against any of the limits.