The human-centric platform for production ML & AI

Outerbounds

Hey There,
Whilst running metaflow standalone (without any metaflow service and/or s3), the flow of data from steps happens via the read and writes to the file system. Is there a way I can specify a directory which metaflow can then use to read or write from?

yep — by default it will look at stuff in `.metaflow` but you can specify `METAFLOW_DATASTORE_LOCAL_DIR` to be something else if you want.

to be a bit more specific, by default it uses a `.metaflow` directory located in your path or anywhere in the parents.

if you define `METAFLOW_DATASTORE_LOCAL_DIR` to be a relative path, the same behavior will apply. If it is an absolute path, it should just use that.

it will store both the metadata for metaflow  (replacing the metaflow serfvice) as well as the “data” (replacing S3) in that location.

Thanks Romain.
Another followup question, we're running metaflow in a container (the whole graph) and I see that the bootup time for metaflow to start processing the flow is high however once started its quite fast, any way to speed the bootup time?