I understand that metaflow typically loads the cur...
# ask-metaflow
e
I understand that metaflow typically loads the current working directory and subdirectories in flows so that they're available. Is there a file size limit where directories won't be loaded in the context? I have a situation where a tensorflow model exists in a directory relative to the flow of
model/tensorflow_model/__various_files__
. Locally I'm able to load this model, but when running on kubernetes that directory is absent. I've confirmed this by using pathlib to list the files in the
model
directory and some smaller python files are there, but other pkl files and the
tensorflow_model
directory are absent
1
https://docs.metaflow.org/scaling/dependencies I think I may have found my answer in that only python files are included:
When Metaflow packages your code for remote execution, any .py files in the directory are included in the distribution automatically
a
yes - you can add
--package-suffixes=.txt,.json
to ensure those get packaged up as well
e
That worked, thanks! Also we're using keras, and longer term will leverage this method: https://outerbounds-community.slack.com/archives/C02116BBNTU/p1680624622388349?thread_ts=1680608119.325749&cid=C02116BBNTU