Hmm... no I think I am just really bad at reading the documentation. There's already a hint in my screenshot. It actually makes sense though, as conda runs in an isolated environment.
What i'm trying to achieve is a custom docker image with base packages that I want to use within a flow step. That would work fine, but not when trying to combine it with @conda or @pypi. But that's even clearly stated in metaflow docs as well. It's one or the other really.
But named environments seem really useful for my use case, so i'm probably going to end up using that