Hello! I have recently started using Metaflow and ...
# ask-metaflow
w
Hello! I have recently started using Metaflow and noticed that code snapshots are not being saved to my S3 datastore when I deploy the Flow locally with
argo-workflows create
. I do have access to the S3 datastore from my local machine, so i’m not sure what may be happening with Metaflow. Does anyone have any suggestions for what might be happening?
1
a
@witty-jackal-1438 what is the output of
argo-workflows create --only-json
for your flow?
w
Hi @square-wire-39606, I apologize for the delay here. That command generates a large amount of json text that looks like the Argo workflow template
a
can you paste that output or alternatively what makes you believe that the code snapshots are not being saved as expected?
w
Here is the output. I have checked that bucket and I am seeing the code snapshot saved in the
data/00/
folder, but when i run the workflow on Argo I seem to be missing a step. The start(0) step is missing when i deploy locally, but if i create the workflow from our gitlab CICD runners, the workflow includes the start(0) step
a
what is
start(0)
?
are you able to run any argo workflows on your local setup?
w
Here is an screenshot of a workflow created from a CICD deploy script
And here is one from a local deployment - it’s missing the start(0) step. I’ve just noticed this difference but haven’t been able to figure out why the two deployments would generate different DAGs
I can create an Argo workflow template, then submit it in the Argo UI. No issues there
What i’ve experienced is that changes in my code in the working directory don’t transfer to running the Argo workflow - i need to rebuild the docker image that contains my code and then run the workflow
a
this could be because of different argo versions
w
Is there a recommended Argo version for using Metaflow?
a
we support a very wide range - it seems that your local deployment is also working - the argo UI is just a bit different
the
start(0)
is just a container for executing
start
- you see both in one view and just
start
in another - but it's a well formed correct DAG that is getting executed
👍 1
start(0)
is argo internal detail - it's actually good that in newer versions it's hidden away from the users
👍 1
w
It seems that was just leading me down the wrong path, i did some more testing and was able to verify that code snapshots are working. Thanks for the help clarifying things!