https://outerbounds.com/ logo
Join Slack
Powered by
Hi, I see the official metaflow-tools <repo> suppo...
# ask-metaflow
w
Hi, I see the official metaflow-tools repo supports Cloudformation and Terraform to deploy Metaflow. Are there plans to support CDK? I found this repo which has a metaflow CDK but it's unofficial. Somewhat related, the CFN template is using PostgreSQL version 11 in RDS which will run out of support by end of Feb and needs to be updated. I also see the CFN template hasn't been touched in 3 years 😳
f
Also Python 3.7 as a lambda runtime is not really supported anymore.
f
@wooden-dusk-90720 I've been running cdk constructs for metaflow for the last ~3ish years or so, recently moved to eks using https://aws-quickstart.github.io/cdk-eks-blueprints/
w
wdym by moved to eks? like the metaflow service/ui runs on eks?
f
ah no - I mean running on eks vs batch
theres decent tooling going from cfn/terraform -> cdk if you want a place to start in building the constructs from scratch. They end up creating all level 1 cloudformation resources but you can go higher level from there
if deploying on eks then recommend starting w eks blueprints since its batteries included and just have to wire a few things together for the db and metadata/ui
having an official cdk project would be nice since through jsii a construct written in typescript can output to python/dotnet/java/go etc
w
whats the advantage(s) of using eks vs batch?
f
batch doesn't have all of the newfangled event triggering: https://docs.metaflow.org/production/event-triggering
which opens up a ton of powerful event-driven patterns that you don't get on batch, and most new development is probably going to be on kubernetes so it future-proofs yourself
f
Coming back to the original question though, is the Metaflow community going to address the issue in the checked in CFN template?
f
is that really an issue though? you can update the template etc
f
Sure, I can. But I think it would help the community if the checked-in template is up to date.
2 Views
Open in Slack
PreviousNext
Powered by