https://outerbounds.com/ logo
Join Slack
Powered by
Hi, Im trying to setup a ml workflow with metaflow...
# ask-metaflow
h
Hi, Im trying to setup a ml workflow with metaflow. After the initial flow is completed, im expecting the deployjob to triggered (using 
@trigger_on_finish
decorator), but the i dont see the flow running. In the logs of the initial triggered(manually) flow, i see the event being published.
Copy code
local.user.sln.projectflow-x2t9b-start-3826449337: Setting up task environment.
local.user.sln.projectflow-x2t9b-start-3826449337: Downloading code package...
local.user.sln.projectflow-x2t9b-start-3826449337: Code package downloaded.
local.user.sln.projectflow-x2t9b-start-3826449337: Task is starting.
local.user.sln.projectflow-x2t9b-start-3826449337: project name: local
local.user.sln.projectflow-x2t9b-start-3826449337: project branch: user.sln
local.user.sln.projectflow-x2t9b-start-3826449337: is this a production run? False
local.user.sln.projectflow-x2t9b-start-3826449337: Argo Event (metaflow.local.user.sln.ProjectFlow.start) published.
local.user.sln.projectflow-x2t9b-start-3826449337: time="2024-02-02T14:52:13 GMT+5:30" level=info msg="sub-process exited" argo=true error="<nil>"
local.user.sln.projectflow-x2t9b-start-3826449337: time="2024-02-02T14:52:13 GMT+5:30" level=info msg="no need to save parameter - on overlapping volume: /mnt/out/task_id" argo=true
local.user.sln.projectflow-x2t9b-end-161317768: Setting up task environment.
local.user.sln.projectflow-x2t9b-end-161317768: Downloading code package...
local.user.sln.projectflow-x2t9b-end-161317768: Code package downloaded.
local.user.sln.projectflow-x2t9b-end-161317768: Task is starting.
local.user.sln.projectflow-x2t9b-end-161317768: Argo Event (metaflow.local.user.sln.ProjectFlow.end) published.
local.user.sln.projectflow-x2t9b-end-161317768: time="2024-02-02T14:52:57 GMT+5:30" level=info msg="sub-process exited" argo=true error="<nil>"
I have created an evensource aswell.
Copy code
apiVersion: <http://argoproj.io/v1alpha1|argoproj.io/v1alpha1>
kind: EventSource
metadata:
  name: projectflow
spec:
  service:
    ports:
      - port: 12000
        targetPort: 12000
  webhook:
    # event-source can run multiple HTTP servers. Simply define a unique port to start a new HTTP server
    projectflow:
      # port to run HTTP server on
      port: "12000"
      # endpoint to listen to
      endpoint: /projectflow
      # HTTP request method to allow. In this case, only POST requests are accepted
      method: POST
I see sensor pod aswell. Still the deployjob flow is not running. Is there something im missing?
project_sample.py
Copy code
from metaflow import FlowSpec, step, project, current
Copy code
@project(name="local")
class ProjectFlow(FlowSpec):
    @step
    def start(self):
        print("project name:", current.project_name)
        print("project branch:", current.branch_name)
        print("is this a production run?", current.is_production)
        self.next(self.end)

    @step
    def end(self):
        self.deploy = True
        pass


if __name__ == "__main__":
    ProjectFlow()
post_project_sample.py
Copy code
from metaflow import FlowSpec, step, project, current, trigger_on_finish


@trigger_on_finish(flow="ProjectFlow")
@project(name="local")
class DeployJob(FlowSpec):
    @step
    def start(self):
        print("Can Deploy? - ", current.trigger.run.data.deploy)
        self.next(self.commit_to_github)

    @step
    def commit_to_github(self):
        # Update on git repo for ArgoCD to trigger the deployment for latest model
        for i in range(10):
            if i & 2:
                print("Deploying...", end="")
            else:
                print(f"-{i}", end="")
        self.next(self.end)

    @step
    def end(self):
        self.deployed = True
        pass


if __name__ == "__main__":
    DeployJob()
Copy code
❯ cat ~/.metaflowconfig/config.json
{
    "AWS_ACCESS_KEY_ID": "xxxxx",
    "AWS_SECRET_ACCESS_KEY": "xxxxxxxxxxx",
    "METAFLOW_DATASTORE_SYSROOT_S3": "<s3://metaflow-s3/d1>",
    "METAFLOW_DATATOOLS_SYSROOT_S3": "<s3://metaflow-s3/d1>",
    "METAFLOW_DEFAULT_DATASTORE": "s3",
    "METAFLOW_KUBERNETES_NAMESPACE": "argo-events",
    "METAFLOW_KUBERNETES_SERVICE_ACCOUNT": "default",
    "METAFLOW_SERVICE_INTERNAL_URL": "<http://metaflow-metaflow-service.metaflow-poc.svc.cluster.local:8080>",
    "METAFLOW_SERVICE_URL": "<http://localhost:8080>",
    "METAFLOW_DEFAULT_METADATA": "service",

    "METAFLOW_ARGO_EVENTS_EVENT": "metaflow-event",
    "METAFLOW_ARGO_EVENTS_EVENT_BUS": "default",
    "METAFLOW_ARGO_EVENTS_EVENT_SOURCE": "argo-events-webhook",
    "METAFLOW_ARGO_EVENTS_SERVICE_ACCOUNT": "operate-workflow-sa",
    "METAFLOW_ARGO_EVENTS_INTERNAL_WEBHOOK_URL": "<http://projectflow-eventsource-svc.argo-events.svc.cluster.local:12000/projectflow>"

}
15 Views
Open in Slack
PreviousNext
Powered by