Hi, I’m new to Metaflow. I’m considering porting an existing ML pipeline to Metaflow, and I’d like to know if it is possible to extract some steps of a flow into a separate class, that I could “call” from inside a step.

👋 Hey Florian. Metaflow "flows" are subclasses of FlowSpec. You can use any kind of inheritance with these subclasses like regular Python. Here is a toy example and a more realistic one of flows that inherit functions from custom classes. Main point is there are many ways you could go about it, since you can run any Python in the Metaflow steps. Hope this helps!

These examples all regular methods that they inherit, not

@step

methods. Do

@step

methods all need to be in the same class? I tried to self.next to a step that is defined in a superclass but I only get the error

Metaflow 2.7.22 executing CombinedFlow for user:florian
Validating your flow...
    Validity checker found an issue on line 36:
    Step start specifies a self.next() transition to an unknown step, data_prep.

see this gist for a code example of what I’m trying to do https://gist.github.com/florianlaws/a5745f8c58aacd575bf6bd9b5595ca10

I see.. separating

@step

unfortunately won't work in the way the gist is showing. each

FlowSpec

needs to have

start

and

end

step.

Do I understand correctly that the entire DAG with all `@step`s from

start

to

end

needs to be in a single

FlowSpec

?

Yes