https://outerbounds.com/ logo
Join Slack
Powered by
I don't know if this is the best place... does any...
# ask-metaflow
c
I don't know if this is the best place... does anyone have some favorite tools for managing and monitoring the deployment aspect on top of metaflow? I know metaflows ethos is productivity and really shines through experimentation all the way through the creation of model artifacts. Historically we've seen great demos on using metaflow with W&B/Comet for experiment tracking... For deployment and monitoring I've seen examples using Seldon, but their docs left a lot to be desired (for me). There are other tools like BentoML, Sagemaker, and more... just curious what are some stacks people have seen as effective in production for a small team?
✅ 1
v
I have heard many folks using Sagemaker's model hosting. On the OSS side, Seldon works nicely
the broad concept of model deployment, or real-time inference in this case, packs a number of different use cases which necessitate different kinds of solutions. Consider these three categories: 1. Large-scale, low-latency, high-SLA inference - imagine real-time ads bidding or hyper-personalized recommendations in a popular consumer-facing product. 2. Medium-scale, medium-latency, high-SLA inference - imagine customer support chatbots, fraud detection, or streaming data use cases. 3. Small-scale, medium-latency, medium-SLA inference - imagine internal dashboards, human-in-the-loop use cases, or control systems with graceful fallbacks. Each of these categories has bespoke engineering and operational concerns. It'd be totally overkill to use a solution tuned for (1) e.g. for (3) - and using a cute Python package meant for (3) for (1) would be hopelessly inadequate.
what's the use case that you have in mind?
❤️ 1
g
Wow the man himself! I’m about 70% through your book currently. This falls squarely in cases two and three. Primarily it’s showing business value for internal stakeholders for the short term through analytics, dashboards, and human in the loop type systems (case 3). My concerns are building solutions that are future proof or at least functional through moderate scale (case 2). Said another way the majority of short term work will be adjunct batch inference and small scale inference for control systems and decision support with advancement and proof of concept for case two in a few quarters. Want to make the right decisions now and stand on the shoulders of work done by others We are resource constrained in terms of external support so we want to make good decisions that will support us through the end of the year and not require significant rework before we can prove business value.
👍 1
c
...It appears I have two accounts and didn't realize it...
v
great! You have a number of easy options then: Seldon, BentoML, or even simple Plotly dash dashboards (or Streamlit) as shown in Chapter 8
everyone here - please chime in and share your experiences if you have used Metaflow with Seldon / BentoML / other model hosting service 👂
🙌 1
c
Thank you for being so generous 🙂
🤗 1
a
My 2cnts, whenever possible, use Azure functions (or similar AWS service) 😄 easy to setup, easy to monitor, very cost efficient, essentially no management overhead. On paper they have a lot of restrictions but in practice they can easily fetch an artifact and run sklearn models. But I wouldn't use these for streaming usecases. Also good experiences with BentoML or just building a FastAPI / Kafka consumer container, deploying the docker images to managed container services like Azure App Service.
🤗 1
🙌 1
Open in Slack
PreviousNext
Powered by