Hi, Should Metaflow be considered as a job manager meaning it just runs a specific bunch of steps/tasks and thats it? Let me explain what I mean. We are thinking on building a scalable system running on Kubernetes which will process & analyse audio streams with help of a some ML package under the hood and forward results to another pipeline step. But I have concerns about the downtime while the pod is being built before step execution.. Can Metaflow keep pods running forever with pre-loaded ML model to reduce downtime?

As of now, there is always a new pod created per task when Metaflow runs with kubernetes (or via Argo workflows) However, do note that to keep pod startup times low: • Metaflow supports providing a custom docker image for running tasks. So, you could have the model baked into the task docker image. • On any node in the cluster, this docker image gets pulled once at the start and is then cached. So subsequent tasks simply reuse this image.

Is it possible to redefine an existing logic for pod creation and termination in a sub-class (or redefine decorators [haven’t seen it yet]) w/o any problems? I want to add a check if the pod is up and running before starting the task and skip termination logic after finishing the task, but allow the step to be completed as it should be done.