Are there any example architecture diagrams with S...
# ask-metaflow
g
Are there any example architecture diagrams with SageMaker integrations?
1
v
Sagemaker is a portfolio of 10+ services today. Each one of them works differently. One of the most common integrations is with Sagemaker Model Hosting as described here Are there other parts of Sagemaker that you are interested in?
g
Valid point @straight-shampoo-11124. What about integration with SageMaker notebooks and Feature store specifically?
v
Sagemaker notebooks work well, like any other notebooks with Metaflow. You can use the Client API as usual. You just need to make sure that the IAM permissions allow the notebook instance to access Metaflow's S3 bucket and communicate with the service.
👍 1
we have less experience with Sagemaker Feature Store, since there are so many other ways to handle features with Metaflow, but it should work out of the box using their Python API
g
Sounds good @straight-shampoo-11124. Have deployments worked well using Lambda as the "initiator" for Metaflow. So instead of running a .py file I can just run the metaflow python code in lambda to create the step functions and batch processing
v
usually you’d have a (cloud) workstation or a notebook instance where you can develop and execute Metaflow locally. When you are ready to deploy to Step Functions, you can either do it on the workstation or through a CI/CD system. A lambda-based action can work for the latter CI/CD case