stocky-librarian-57968
11/21/2023, 2:40 PM@kubernetes in my case). I cribbed from what the @environment decorator does, and did something like this:
def runtime_step_cli(self, cli_args, *_args):
if self.attributes["some_attribute"]:
cli_args.env.update({"SOME_ENV_VAR": "SOME_VALUE"})
Then I have a task_pre_step that uses that env var.
However, I'm not seeing SOME_ENV_VAR set in os.environ within the task_pre_step when running in a kubernetes pod. I have confirmed that it is present during step_init while the task is being started up on the local machine, and the values I'm setting using the environment decorator are making it to the pod, but not the ones from my extension.
Any ideas what I'm missing here?quiet-afternoon-68940
11/21/2023, 4:28 PM@environment decorator gets special treatment in various places. When we want a custom decorator to set environment variables, we typically have it modify the environment decorator rather than set the variables directly.quiet-afternoon-68940
11/21/2023, 4:35 PM@environment decorator, adds a bunch of other stuff, and then uses the result in generating a workflow template. I don't think it will find variables set by a custom decorator. https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows.py#L1229-L1436dry-beach-38304
11/21/2023, 5:35 PMdry-beach-38304
11/21/2023, 5:35 PMdry-beach-38304
11/21/2023, 5:36 PMenvironment decorator into your stepdry-beach-38304
11/21/2023, 5:41 PMdry-beach-38304
11/21/2023, 5:41 PMdry-beach-38304
11/21/2023, 5:48 PMstep_init, you can check if you have the decorators (check for one called environment. If it is there, you are all good to go and you can just add to vars in there and it will propagate around
• if it is not, you can do decorators.append(EnvironmentDecorator(attributes={"vars":…}, statically_defined=self.statically_defined) or something like that
I haven’t tested any of this just yet but something like it or close to it should work.stocky-librarian-57968
11/21/2023, 6:46 PMdry-beach-38304
11/21/2023, 7:22 PMrhythmic-beach-70913
11/24/2023, 7:45 AMdef flow_init(self, flow, graph, environment, flow_datastore, metadata, logger, echo, options):
env_config = {
"NAME": "VALUE"
}
my_decorators = [
f"environment:vars={json.dumps(env_config)}",
]
_attach_decorators(flow, my_decorators)