https://outerbounds.com/ logo
Join Slack
Powered by
I am getting an odd GLIBC version error when runni...
# ask-metaflow
f
I am getting an odd GLIBC version error when running metaflow with the netflix extension. It only appears when I use the @pypi decorater and is unrelated to the packages inside of it. My OS is Mac OS 14.1.2 so it doesn't make sense why it tries to look for a GNU C library..
Copy code
Metaflow 2.10.8+netflix-ext(1.1.1) executing TestFlow for user:test
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint not found, so extra checks are disabled.
Bootstrapping Conda environment... (this could take a few minutes)
    Resolving 1 of 2 environments (others are cached) ...    Conda ran into an error while setting up environment.:
    Could not determine maximum GLIBC version
d
Hey @freezing-cartoon-22525 — recently thr handling of glibc changed. You are 100% correct that it should only matter for Linux solves and not for mac so there may be an issue. I’ll look at it later today. To help, would you be able to rerun using METAFLOW_DEBUG_CONDA=1 (env var) and link me to the output.
If not, I’ll also try directly on my mac to see if I can reproduce.
f
d
so looking at the debug log — are you building for remote execution (kubernetes, batch, etc)? It seems to be trying to build for linux-64 as an architecture. There is still definitley a bug (it should default to some version of glibc) but just confirming that this is the expected architecture to build for (ie: if one or more steps executes remotely it will try resolving for linux)
Assuming this is the case, as a workaround until I add a default, you can set this: https://github.com/Netflix/metaflow-nflx-extensions/blob/main/metaflow_extensions/netflix_ext/config/mfextinit_netflixext.py#L122 which should work to inject a version of glibc to use.
It’s just an environment variable. Or you can set it in your metaflow config.
As a reminder, thr name of the environment variable needs METAFLOW_ in front.
if it works for you, I’ll merge it in
basically it’s just specifying a “reasonable” default when cross building.
e
Hi guys, I’m seeing that error too. I’ll plan to test the workaround on Monday
OK, I was delayed by code reviews this week but I have finally tested this workaround by adding the following line to my Metaflow config json to inject the dependency, which I hope looks correct: 
"METAFLOW_CONDA_SYS_DEFAULT_PACKAGES": "{'linux-64': {'__glibc': '2.27'}}"
After saving the changes, I am still seeing the same error message when attempting to run my flow with Kubernetes:
Copy code
Metaflow 2.10.8+netflix-ext(1.1.1) executing TestHCC for user:gracemoore
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint is happy!
Bootstrapping Conda environment... (this could take a few minutes)
    Resolving 1 environment ...    Conda ran into an error while setting up environment.:
    Could not determine maximum GLIBC version
If you have any other ideas for workarounds, I’m all ears 🙏🏻
d
I think it’s because of the bug in #32 (which I should merge I guess). The issue is that it doesn’t read it from the config. You can make the change in PR #32 and then it should work. If not, could you please run with 
METAFLOW_DEBUG_CONDA=1
and send me the output so I can debug. For reference, this is the simple change to make: https://github.com/Netflix/metaflow-nflx-extensions/pull/32/files
e
Hi Romain, thank you, I noticed you merged your changes and I tested them out! My debug logs are attached.
Untitled
My flow runs locally on mac and only fails when I’m running it with kubernetes - still learning here, is it possible my setup there could be the issue?
d
I’ll take a look. The issue is that it doesn’t seem to list 
__glibc
in the 
sys
dependency like it should. Let me see if I can repro.
🙏 1
c
@elegant-plastic-42374 @dry-beach-38304 Did either of you ever figure out what the solution was here? I'm trying to run @batch from a mac.
d
I thought I fixed this. I may have only fixed it internally. I am actually packaging a bunch of fixes here so I’ll see if I can add this one (if it is the one I think it is). Sorry this had slipped my mind.
c
Thanks really appreciate it. The extension is awesome.
d
c
That has fixed the 
GLIBC
issue for me. Thank you so much.
🙌 1
d
ya. I had fixed that one internally but forgot to push. Glad it worked for you. I’ll merge this soon and make another release.
🙌 1
4 Views
Open in Slack
PreviousNext
Powered by