Here s a little PR to change the ` environment` decorator to Outerbounds #dev-metaflow

Here's a little PR to change the `@environment` de...

quiet-afternoon-68940

04/04/2024, 2:30 PM

Here's a little PR to change the

@environment

decorator to have

allow_multiple=True

. The main motivation is that it's confusing that an environment decorator defined in code prevents a decorator applied on the CLI from having any effect, even if the sets of variables defined by the two decorators are disjoint. https://github.com/Netflix/metaflow/pull/1785

ancient-application-36103

04/08/2024, 8:38 PM

thanks! @quiet-afternoon-68940 can you help us with the intended use case?

ancient-application-36103

04/08/2024, 8:39 PM

i wonder if introducing support for functions as args to

@environment

will better support the use case of being able to control which env vars are passed to a step

ancient-application-36103

04/08/2024, 8:40 PM

as you pointed out in your PR, handling conflicts can become a bit hairier - as of today, Metaflow code base does not enforce the order of evaluation for decorators by design - so we can't rely on a fixed order as a strict invariant - which can cause unintended issues with different values for the same environment variable getting passed to the step.

quiet-afternoon-68940

04/09/2024, 12:37 AM

I'm not sure this will enable anything we couldn't do without it, but the current behavior has caused quite a bit of confusion and wasted time, as I described here: https://outerbounds-community.slack.com/archives/C02116BBNTU/p1711661547271779

quiet-afternoon-68940

04/09/2024, 12:41 AM

not sure I understand how functions as decorator args would help? and, as an aside, we found a while back that passing functions as arguments to decorators broke the DAG display in the UI, although this could be fixed if it hasn't already been.

Open in Slack

Previous Next