Hey community, I am having issues with importing local modules. I have no idea why this behaviour appears to be unique to metaflow. I have tried the 

package list

 command to make sure everything I expect is being packaged up. My project structure is as so:

.
├── README.md
├── python
│   ├── db
│   │   ├── __init__.py
│   │   ├── database_adapter.py
│   │   └── database_connector.py

│   ├── sectional_model_flow.py
│   ├── __init__.py
├── requirements.txt
└── tests
    └── unit
        ├── aws
        │   ├── test_ssm.py
        │   └── test_sts.py
        └── db
            ├── test_database_adapter.py
            └── test_database_connector.py

I tried this way because metaflow doesnt like me importing say from sectional_model_flow like so 

from python.db.database_connector import DatabaseConnector

 it gives me a ModuleNotFoundError. So I tried following the instructions here (Netflix/metaflow#175), and this allowed it to run, but now my tests can't run because the import above would change to 

from db.database_connector import DatabaseConnector

What am I missing, why does this behave so differently to normal python implementations?

how do you run the tests?

pytest ./tests/

in the above case if your

PYTHONPATH

points at the root,

from python.db.database_connector import DatabaseConnector

should work in tests and in

sectional_model_flow.py

you can do

from db.database_connector import DatabaseConnector

yes, that is the problem though. There are relative imports in the

database_connector

file as an example ( I left the other files out for simplicity ). So when testing this file it can work for the flow and not the tests

hmm, like

from . import database_adapter

or something like it?

example is another module called aws that has ssm.py So in the

database_adapter.py

it imports it with

from aws.ssm import Ssm

This valid in the flow, but in the test it needs

from python.aws.ssm import Ssm

the basic issue is that Metaflow wants all packages to be under one root directory, so it can package and ship everything when needed. This is

python

in your case, so that should work. You could make

tests

see the same universe by setting

PYTHONPATH=python

but if you don't want to do that, you'd need to change imports to that they are relative and not absolute

nah aws is under python

you could introduce another package under

python

, so you would have

somepkg/db

and

somepkg/aws

then in

db

you can do

from ..aws.ssm import Ssm

yeah cool, thanks for such a quick response and helping me out!

cool! Let me know if it doesn't work 👍