Hi all, i would like to ask if there's some ideas on using a different secrets backend or how it could best be accomplished. Specifically, at my company, we use HashiCorp Vault, and we’re interested in exploring how we could connect it to metaflow.

it should be somewhat straight-forward to add support. here is an example for a GCP secret manager implementation. we are also looking into introducing support for it imminently.

for the specific mechanism, you could add an extension that adds a secrets provider. Going off the example @ancient-application-36103 pointed to: • create a class

HashiSecretsProvider(SecretsProvider)

or something like that • add a TYPE, let’s say:

hashi-secret-manager

• implement the method

get_secret_as_dict

• using this as a template: ◦ In here, remove everything except config, plugins and toplevel ◦ In the

config

part, you can add any specific configuration value you want to use. You may also want to

get_pinned_conda_libs

to add any required dependencies to use your secrets provider from within a conda/pypi environment ◦ In plugins, add your hashi secrets provider file and then in the init file (typically named

mfextinit_<myorg>.py

, add the following line:

SECRET_PROVIDERS_DESC = ["hashi-secret-manager", ".<path to class>"]

See here for an example in the main repo. ◦ In the toplevel one, you can just modify the init and toplevel file (you don’t need the overrides and can remove mention of it from the init file. Once that package is done, if you install it, Metaflow will pick up your secrets provider. Let me know if you have questions.