Hi. I'm new to Metaflow and I'm trying to get a grip on how environments are handled. Specifically I'm trying to do the following: I have a flow that creates an ML model and I want to be able to execute that flow locally (architecture osx-arm64). Afterwards I might want to use that model in a different flow. In that flow I'd like to use the same environment that was used to create it so that I can be sure that it works the same. This should ideally also work when that flow is deployed on a different architecture (linux-64). The simplest way for making environments easily reusable across flows seems to be via named environments from the netflix extensions package, but it doesn't seem to be possible to resolve two environments on different architectures and give them the same name (correct?). I.e. something like this doesn't result in both of them having the same alias:

metaflow environment resolve --arch osx-arm64 --arch linux-64 --force -r requirements.txt --alias my_test_env

There is the option of using

fetch_at_exec

to determine the name of the named environment at runtime. Then I could resolve the environments for both architectures and give them different aliases, e.g.

my_test_env:<arch>

, and define through runtime information, which is the correct one. I'm just wondering whether this is an appropriate way of doing it. Is there a better way?

what error do you get. That should work (and resolve the environment twice and then apply the same name for it.

The command itself works and builds two environments. They have different full hashes (which is expected, I guess). I can also find both of them by running

metaflow environment show

by supplying the full hashes (and adding --arch linux-64 in one case). But when searching for the alias, only the osx-arm64 version shows up. For the linux version I get

➜ metaflow environment show --arch linux-64 my_test_env 
Metaflow (2.15.18+netflix-ext(1.2.3))

### Environment for 'my_test_env' (arch 'linux-64') was not found

I tried to switch the order of the resolution to see whether the other one would get aliased in that case, but then I get this seemingly unrelated error

➜ metaflow environment resolve --arch linux-64 --arch osx-arm64 --force -r requirements.txt --alias my_test_env
Metaflow (2.15.18+netflix-ext(1.2.3))

    Resolving 2 environments ...Pretty-printed STDOUT:
Could not solve for environment specs
The following package could not be installed
└─ python 3.11.10  is not installable because it requires
   └─ __glibc >=2.17,<3.0.a0 , which is missing on the system.
{
    "success": false
}

whenever you have a chance, could you do

METAFLOW_DEBUG_CONDA=1 metaflow environment resolve — …

and sned me the output. I can try taking a look (likely in the next few days; I am traveling tomorrow so may be a bit slower to resond).

Thank you. Sent a PM with the output.