https://outerbounds.com/ logo
Join Slack
Powered by
Hi all, wondering if you have any suggestions on b...
# ask-metaflow
l
Hi all, wondering if you have any suggestions on best practice of passing secrets to pods? I’ve set it up via environment variables, which is better then keeping them in code but still far from ideal. Is custom image with files storing secrets the only other viable option? I’m using private install with K8s, Argo and MinIO. Thanks!
✅ 1
v
where/how do you store your secrets today?
l
In config files.
v
ok. We are about to release a feature, 
@secrets
that allows you to fetch secrets from AWS Secrets Manager. Another such secrets manager that you might consider is Hashicorp's Vault
l
Great! Would that work for local install provided there is connection to aws?
v
yep!
@ancient-application-36103 can share details if you want to give it a try
l
Thank you! Once released we will give it a shot.
👍 1
a
@little-pizza-78379 you should be able to use it with the latest version of Metaflow today - here are the instructions - https://docs.google.com/document/d/1PQW8ih4IZNxl17ALgnzVcUHLoaJ4BGixQKMNjtqdvVQ/edit#
@secrets - Metaflow OSS
v
your feedback is highly appreciated 🙂
l
Seems straightforward. How do we authorize to the key manager locally and in K8?
I realize that aws must have a doc on that just wonder if you have one that would fit this use case better?
a
You would need to add a policy that allows for read access to secrets in your IAM role that was created for accessing AWS services from K8S and locally.
Also, just re-reading your original question - are you just looking at accessing kubernetes secrets? If so - that will work simply by doing 
@kubernetes(secrets=["secret-1", "secret-2"])
l
We need to provide secrets to pods in K8 cluster.
a
and where are these secrets coming from?
l
Currently we are just storing them in local config file.
Loading at run time.
At least they are not present in environment for all processes to access and we can somewhat control access to files, also not ideal of course.
a
one option is to store them as kubernetes secrets and access them within Metaflow. Another approach is to off load these secrets to an external secrets manager like AWS Secret Manager and access them from within Metaflow
c
Super excited for this release!
excited 1
v
@curved-island-17262 feel free to give it a try 🙂
c
Is it out yet?
Ok just noticed it is
👍 1
l
Thanks for your help! We will try K8 secrets first and then look at AWS.
3 Views
Open in Slack
PreviousNext
Powered by