Looking over the documentation for Metaflow, it seems to have the components that allow it to be a production ML model serving platform. But in Production Deployments, this section includes deploying models to a model hosting platform such as Seldon or Sagemaker. Is this just an indication that you can also integrate with external model serving platforms? Bottom line, my question is "Can Metaflow scale sufficiently to be my ML serving production service?"

It depends what you mean by

ML serving production service

and on the components your use case needs. Metaflow's bread and butter is batch workflows, which write predictions to a cache for apps/products to use. You can deploy trained models from a Metaflow workflow to a server that hosts real-time model endpoints too, but there is nothing inside a standard Metaflow deployment that runs a server hosting a real-time model endpoint for you, you'd need to build or buy such a system. This is one reason people use Metaflow with tools like Seldon or Sagemaker endpoints. Outerbounds is also actively exploring inference solutions that are more closely coupled with Metaflow's existing functionality.

Thanks for making this distinction clear. Apart from Seldon and Sagemaker, I see some other deployment possibilities, e.g. Nvidia Triton. Are there docs on what services are supported (or at least available)?

Good q. We could improve the docs along these lines as examples are kind of scattered in ad hoc repos, such as this one for triton inference server. Is there specific service you have in mind? Metaflow is agnostic in this regard, so can get anything going once we pick a direction.

I am looking at a few possibilities and also an internal product that we use for more general compute needs but that is also serving some models. Some other platforms in addition to the ones above are Vertex AI model serving and KServe.

This tutorial shows a basic Sagemaker endpoints example in context, which would be similar to vertex's

aiplatform.Endpoint.create

(docs). KServe is a bit different, but not too much at the python layer. For the least opinionated example, this FastAPI example exists in part to be a quick copy-paste and swap in your model situation.