bulky-umbrella-55918
05/30/2024, 11:36 AMargo-workflows delete
, the Workflow Template gets deleted, but none of the sensors do. Sensors created by both trigger
and trigger_on_finish
are left behind consuming resources.
I noticed that this is the case only when I use the project
decorator. Without it, the sensors are deleted as they should.
Does anyone know what might be causing this?
I do have permission to delete the sensors. No warnings or errors are shown.
Flow:
from metaflow import FlowSpec, step, project, trigger
@project(name="my_test_project") # commenting this line out "fixes" the issue
@trigger(event="test_airflow_trigger")
class MyTestFlow(FlowSpec):
@step
def start(self):
self.next(self.end)
@step
def end(self):
pass
if __name__ == "__main__":
MyTestFlow()
.metaflowconfig:
{
"METAFLOW_DATASTORE_SYSROOT_GS": "gs://",
"METAFLOW_DEFAULT_DATASTORE": "gs",
"METAFLOW_DEFAULT_METADATA": "service",
"METAFLOW_KUBERNETES_NAMESPACE": "...",
"METAFLOW_KUBERNETES_SERVICE_ACCOUNT": "...",
"METAFLOW_SERVICE_INTERNAL_URL": "http://...",
"METAFLOW_SERVICE_URL": "https://...",
"METAFLOW_KUBERNETES_LABELS": "...",
"METAFLOW_KUBERNETES_CONTAINER_IMAGE": "...",
"METAFLOW_KUBERNETES_CONTAINER_REGISTRY": "...",
"METAFLOW_ARGO_EVENTS_EVENT_BUS": "...",
"METAFLOW_ARGO_EVENTS_EVENT_SOURCE": "...",
"METAFLOW_ARGO_EVENTS_SERVICE_ACCOUNT": "...",
"METAFLOW_ARGO_EVENTS_EVENT": "metaflow-event",
"METAFLOW_ARGO_EVENTS_INTERNAL_WEBHOOK_URL": "http://...",
"METAFLOW_ARGO_EVENTS_WEBHOOK_URL": "https://...",
"METAFLOW_ARGO_WORKFLOWS_UI_URL": "https://..."
}
My system info:
• OS: MacOS
• Architecture: Apple M2
• Python version: 3.9.16
• Metaflow version: 2.9.14
• K8s Python Client Version: kubernetes==26.1.0
• K8s Client Version: v1.27.12
• K8s Server Version: v1.27.11-gke.1062003