hi, i'm trialing out Metaflow's --notify-on-succes...
# ask-metaflow
a
hi, i'm trialing out Metaflow's --notify-on-success/--notify-on-error. On simulation of trigger this. The pod/tasks is stuck in pending state. Upon checking in kubernetes, to describe more details. We found that the "notification" pod seems to be not being able to be schedule due to toleration/taint that we've applied on our nodes. We do pass the toleration values via the "METAFLOW_KUBERNETES_TOLERATIONS": "[{\"key\":\"function\", \"operator\": \"Equal\", \"value\": \"cia-workloads\", \"effect\": \"NoSchedule\"}]", however, this doesn't seem to work. We did follow the additional instructions in the FAQ section on creating secrets against the service account used - https://github.com/Netflix/metaflow/releases/tag/2.9.1#features.1. However, it still fails in pending state. can i ask, are there specific ways to handle these, thanks ?