Hello all please take a look at the following feature propos Outerbounds #dev-metaflow

Hello all -- please take a look at the following f...

melodic-musician-54941

10/25/2024, 6:08 PM

Hello all -- please take a look at the following feature proposal for adding support for

labels

and

annotations

to the

@kubernetes

decorator API. https://docs.google.com/document/d/1OATttdFrwYRHFaq0HC7iwuXApELx1T8p9KAFopYgFLE/edit?usp=sharing

ancient-application-36103

10/25/2024, 6:09 PM

Thanks! Cc @thankful-ambulance-42457 @brainy-truck-72938 for review

👀 1

❤️ 1

melodic-musician-54941

10/28/2024, 3:43 PM

Apologies for not setting the permissions right for the Google Doc -- it should be open now for anyone who follows the above link~

damp-electrician-39888

10/31/2024, 5:21 AM

@melodic-musician-54941 there was also an open issue around adding priority class to the K8s decorator https://github.com/Netflix/metaflow/issues/1752. This would be useful for supporting preemption.

melodic-musician-54941

10/31/2024, 4:36 PM

Adding support for a

PriorityClass

specification would be interesting -- likely a good idea for a follow-on / future PR

melodic-musician-54941

11/04/2024, 5:59 PM

Pinging the thread for visibility -- any further actions required on my end to keep the process going?

ancient-application-36103

11/06/2024, 4:37 AM

We can start on a POC PR!

❤️ 1

melodic-musician-54941

11/07/2024, 4:13 PM

Huzzah! Should I note anything specific on the PR when I submit it?

bulky-afternoon-92433

11/18/2024, 4:18 PM

I took some time to rebase/fix an ancient PR around this issue, which could act as a basis for refinement. You can find it and any commentary on the changes here: https://github.com/Netflix/metaflow/pull/1568

bulky-afternoon-92433

11/18/2024, 4:21 PM

one of the main considerations when it comes to Argo is how to treat

METAFLOW_KUBERNETES_ANNOTATIONS

. The current branch works off the assumption that we want to treat values from the env var as globals, so they are set on a workflow template level, and to any other argo resource we create (such as sensors). annotations/labels explicitly set through the decorator however only apply to immediate task resources, e.g. the pods that run them.

melodic-musician-54941

11/19/2024, 5:13 PM

Appreciate you picking this up again! I hadn't seen this stale PR before, so it makes sense that the work could just continue from there.

melodic-musician-54941

11/19/2024, 5:14 PM

Looking over the PR, everything looks good to me! The updates I laid out in my Google doc appear to be covered in the commits and your resolution of the Argo conflicts wasn't something I had considered, so that's all good too~

Open in Slack

Previous Next