Hi all, I have a question for MetaFlow in the cont...
# ask-metaflow
q
Hi all, I have a question for MetaFlow in the context of experiment tracking (just using the metadata service, without compute and without step functions) if I run
python my_flow.py run
, will the file
my_flow.py
from that run be stored anywhere in the artifacts of that run? Sometimes people do a run without git-committing their code of that run (i.e.
my_flow.py
). Later on, when retrieving the artifacts from a particular run, it would be great if I could retrieve the code that produced the artifacts as well. Otherwise, the alternative solution I would have in mind is to commit the code of every run that is worth reproducing and add the corresponding run-id (e.g.
MyFlow/1675261665594517
) to the git-commit-message. This way the person that wants to reproduce everything will be able to access the code that produced the artifacts from this particular run id. (In this question, I only mention the python file that contains the flow i.e.
my_flow.py
but similarly, it would, of course, be good to retrieve the code from custom python modules that are used by this python file e.g.
utils.py
)
1
s
You might find this helpful - https://docs.metaflow.org/api/client#metaflowcode "`MetaflowCode` includes a package of the user-defined
FlowSpec
class and supporting files, as well as a snapshot of the Metaflow library itself. Currently
MetaflowCode
objects are stored only for `Run`s that have at least one
Step
executing outside the user's local environment."
q
@stale-salesmen-66018 Thank you for the link. However, this makes me sad: "Currently
MetaflowCode
objects are stored only for `Run`s that have at least one
Step
executing outside the user's local environment." Hence, this does not work if I want to use MetaFlow locally. And it does not work if I want to use it with AWS only using the metadata service.
d
Hello. Storing local runs has been something thst we want to do but it hasn’t yet been done.
👍 1