I have a quick question about using custom images. The typical structure of a flow file might look like dependency imports at the top, shared classes/functions, and then the definition of the flow with a series of steps. Each of those steps may not require all the dependencies imported at the top of the file. In an effort to keep images as small as possible, I'd like to define an image per step including only the dependencies that step needs (using a custom image not the pypi or conda decorators). My question is, if I package things up this way, does metaflow try to run all the imports from the top of the file in each step? Or should I structure my steps such that they import the libraries they need?

in my experience you should avoid global imports and treat each step as potentially its own self-contained process / env because it can be actually run in a completely separate / arbitrary runtime -> for example

mypackage_here

can be in the docker image for just this step

def mystep():
   import mypackage_here

Gotcha, ok thanks! I suspected this was the case but wanted to make sure!