Hi all!
@square-wire-39606 and
@dry-beach-38304 - thank you both again for fixing the previous FileCache issue I ran into (PR
here). Another question for you.... We're using the Metaflow python API to query artifacts / do analysis from databricks notebooks. I was able to get FileCache to work on databricks use a
volume for tmp files (cannot use local workspace files since it doesn't support POSIX-style functionality like
symlinks).
I'm wondering if there's a way to turn off FileCache with the Metaflow API?
It doesn't make sense to be caching in our context. We're pulling artifacts from S3 and "caching" them back to S3 via a volume. When using the
import metaflow
python from databricks, there's no real concept of a local cache.