< bulky afternoon 92433> < U01NGJ0GA0J> Unfortunate timing o Outerbounds #dev-metaflow

Join Slack

Channels

ask-metaflow

dev-metaflow

<@U027UBSSBRC> <@U01NGJ0GA0J> Unfortunate timing o...

# dev-metaflow

elegant-beach-10818

08/29/2023, 9:10 PM

@bulky-afternoon-92433 @User Unfortunate timing on this given the recent release, but I found a bug that was introduced in this PR: https://github.com/Netflix/metaflow/pull/1442 TLDR the new annotation logic isn't populating the project, flow_name, or user annotations in kubernetes https://github.com/Netflix/metaflow/issues/1515 Ran out of time today to figure out where in the logic is failing but wanted to let you know

✅ 1

ancient-application-36103

08/29/2023, 10:52 PM

We will roll back the change

bulky-afternoon-92433

08/30/2023, 9:39 AM

rolled back and added a mention to the prior release notes. My guess is that the issue is in relying on

current

inside the kubernetes_decorator scope for populating the system annotations, given that apparently labels were still working, only annotations were not?

bulky-afternoon-92433

08/30/2023, 2:16 PM

motivation for the changes were that we could use the kubernetes decorator as a source for setting the labels/annotations. Apparently during the init phase,

current

is not sufficiently populated though.. One option to consider is if we could populate the necessary properties for the current singleton, f.ex. during the

*_cli.py

entrypoints, but this needs some thought if there are any side-effects to setting project/user info too early.

bulky-afternoon-92433

09/04/2023, 9:41 AM

@elegant-beach-10818 would you have time to look at a fix for this and possibly reintroducing the PR? I might also have some time this week to look into the issue with

current

, but for now the approach would be to see if the problematic system annotations can be inferred in another way inside the kubernetes decorator, and if not, we need to lift those out of the decorator back where they used to be.

elegant-beach-10818

09/05/2023, 5:29 PM

This week is somewhat tight timewise for me. The earliest I can look at it will likely be next week

bulky-afternoon-92433

10/04/2023, 12:55 PM

been extremely busy with other tasks as of late but managed to squeeze out a first-draft for reintroducing this feature. @elegant-beach-10818 if you got any time to spare, feel free to take it for a spin: https://github.com/Netflix/metaflow/pull/1568

bulky-afternoon-92433

10/04/2023, 12:56 PM

also was unable to add you as a reviewer for some reason, but feel free to drop any notes in the PR 🙂

elegant-beach-10818

10/09/2023, 1:25 PM

I feel you on the busy side of things I’ll definitely take a look. Thanks for working on it!

Open in Slack

Previous Next