Hello folks, I'm using Metaflow with Argo Workflo...
# ask-metaflow
b
Hello folks, I'm using Metaflow with Argo Workflows. When I run
argo-workflows delete
, the Workflow Template gets deleted, but none of the sensors do. Sensors created by both
trigger
and
trigger_on_finish
are left behind consuming resources. I noticed that this is the case only when I use the
project
decorator. Without it, the sensors are deleted as they should. Does anyone know what might be causing this? I do have permission to delete the sensors. No warnings or errors are shown. Flow:
Copy code
from metaflow import FlowSpec, step, project, trigger

@project(name="my_test_project") # commenting this line out "fixes" the issue
@trigger(event="test_airflow_trigger")
class MyTestFlow(FlowSpec):
    @step
    def start(self):
        self.next(self.end)

    @step
    def end(self):
        pass


if __name__ == "__main__":
    MyTestFlow()
.metaflowconfig:
Copy code
{
  "METAFLOW_DATASTORE_SYSROOT_GS": "gs://",
  "METAFLOW_DEFAULT_DATASTORE": "gs",
  "METAFLOW_DEFAULT_METADATA": "service",
  "METAFLOW_KUBERNETES_NAMESPACE": "...",
  "METAFLOW_KUBERNETES_SERVICE_ACCOUNT": "...",
  "METAFLOW_SERVICE_INTERNAL_URL": "http://...",
  "METAFLOW_SERVICE_URL": "https://...",
  "METAFLOW_KUBERNETES_LABELS": "...",
  "METAFLOW_KUBERNETES_CONTAINER_IMAGE": "...",
  "METAFLOW_KUBERNETES_CONTAINER_REGISTRY": "...",
  "METAFLOW_ARGO_EVENTS_EVENT_BUS": "...",
  "METAFLOW_ARGO_EVENTS_EVENT_SOURCE": "...",
  "METAFLOW_ARGO_EVENTS_SERVICE_ACCOUNT": "...",
  "METAFLOW_ARGO_EVENTS_EVENT": "metaflow-event",
  "METAFLOW_ARGO_EVENTS_INTERNAL_WEBHOOK_URL": "http://...",
  "METAFLOW_ARGO_EVENTS_WEBHOOK_URL": "https://...",
  "METAFLOW_ARGO_WORKFLOWS_UI_URL": "https://..."
}
My system info: • OS:
MacOS
• Architecture: Apple M2 • Python version:
3.9.16
• Metaflow version:
2.9.14
• K8s Python Client Version:
kubernetes==26.1.0
• K8s Client Version:
v1.27.12
• K8s Server Version:
v1.27.11-gke.1062003
👀 1
1