Hi team! I have a custom image to run my pipelines in AWs, and the place that I have my flows is

/project/pipelines/predict/[name_of_metaflow_flow].py

But I think metaflow creates a directory called

/app/metaflow/[name_of_metaflow_flow].py

Is it possible to maintain the folder structure? Because our symbolic links for our code need to be replicated also to

/project/pipelines/

in order to work

hi bruno! metaflow copies all files in the directory -

/project/pipelines/predict/

to

/app/metaflow/

in the batch container.

for instance, we have:

conf/ # configurations
src/ # code
common/ # ancillery code
pipelines/predict # prediction pipelines
pipelines/train # training pipelines

for the training and prediction pipelines, if we need to run them locally, we have to have symbolic links for src/ and common/ in the prediction and training directories as

ln -s ../../common common

and

ln -s ../../src src

, however we also need to setup in the

pipelines

directory the following symbolic links

ln -s ../common common

and

ln -s ../src src

for the docker image in AWS... how can we setup our project in order to reduce the number of symbolic links while having the training and prediction pipelines separate locations?

are any of these files already baked inside the images?

sorry, what do you mean for "already baked inside the images"?