Is there a way to subclass a Pipeline that has been subclassed from Flowspec, Ex: class MyPipeline(Flowspec), has the steps for the pipeline including start, step1, step2 and end. Now I extend MyPipeline --> class MyExtendedPipeline(MyPipeline): has additional methods. When I try to run MyExtendedPipeline I get the following error: Metaflow 2.12.33 executing MyExtendedPipeline for user:acme Validating your flow... Validity checker found an issue: Add a start step in your flow.

Metaflow does not allow subclassing a FlowSpec subclass. Can you describe the pattern you are hoping to achieve through inheritance?

I have several instances where the pipeline is similar/same where most of it is plumbing - audits/logging/registry setup etc, is common across several models/clients/datasets and offering much of this plumbing as part of a base class (steps predefined) and having the sub classes be concerned about the specific dataset concerns makes sense. Inheritance would give a cleaner implementation but I do realize with FlowSpec using inspect to load up the source it might be an issue. Will likely look at other mechanisms (Composites etc) to achieve a similar effect.

That makes sense. A mixin approach might be useful here:

# flow1.py
from metaflow import FlowSpec, ...
from my_module import BaseMixin

class F1(FlowSpec, BaseMixin):
  ...

# flow2.py
from metaflow import FlowSpec, ...
from my_module import BaseMixin

class F2(FlowSpec, BaseMixin):
  ...

There is a PR right now that may allow this shortly: https://github.com/Netflix/metaflow/pull/2086. It’s been on my plate for a while to look at and I finally did 🙂. Hopefully will be able to tmerge soon.

@crooked-jordan-29960 - Yes you are right, Using a mixin is an approach that could work. I would need to dynamically choose a different implementation based on the dataset/models being tackled. My thoughts are that inheritance naturally solves this where's mixins/composites are valid alternatives. @dry-beach-38304 - Thanks. It's what Im looking for. Will look for the merge to be in a release.

Excited for the full inheritance mode as well. Nice explanations in the PR - fun how it is 50% about the linter 🧹