https://outerbounds.com/ logo
Join Slack
Powered by
anyone using metaflow on gcp with argo events? I'v...
# ask-metaflow
g
anyone using metaflow on gcp with argo events? I've configured a flow with @trigger(
date_updated
) and am trying to trigger it via Jupyter notebook -
✅ 1
oops, yeah I updated that to localhost. however it didn't work until I added the following to metaflow/plugins/argo/argo_events.py
Copy code
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
more general question I guess, what is the recommended pattern of deploying argo-events/argo-workflows + metaflow I have metaflow services deployed to the 
metaflow
ns, argo-events and argo-workflows to their own respective namespaces. I would like all metaflow-related pods to be launched within 
metaflow
namespace. when i create my workflow:
Copy code
python3 workflows/event_triggered_flow.py --environment=conda argo-workflows create
it gets created in argo under 
metaflow
ns because of the metaflowconfig. but when I run a jupyter notebook cell (see above) the WF in metaflow does not get triggered, but a workflow I previously created with 
METAFLOW_KUBERNETES_NAMESPACE=argo-events
does. does this mean I have to put everything in one ns? I saw this thread and I guess I'm still wondering what the path forward is
b
https://github.com/Netflix/metaflow/pull/1463 I have been waiting on this for a while. We deploy all the services to one namespace, but the pods run in other namespaces. I think you could put them all in different namespace as long as you have the internal URLs correctly configured.
👀 1
👍 1
g
^^ that would be nice! hope to see it merged soon
a
@ancient-application-36103 @victorious-lawyer-58417 Any timeline for when this PR could be merged in? We have a workaround but having a separate namespace for argo events declared in the metaflow config would be helpful.
a
At the moment, we are looking into setting up a test harness to be able to ensure that there are no regressions with changes of this nature. Once that lands we should be in a good position to merge this in
thankyou 1
b
We ended up just creating one jetstream deployment per namespace and abandoning this PR.
It probably only needs a few tweaks to work tho. The argo workflows code has changed a little but not that much.
6 Views
Open in Slack
PreviousNext
Powered by