https://outerbounds.com/ logo
Join Slack
Powered by
Anyone know what this issue means? I only see it w...
# ask-metaflow
f
Anyone know what this issue means? I only see it when attempting to include both @conda and @pypi decorators. I am using the Metaflow Netflix extensions
d
could you paste the full error here?
f
pydantic_core._pydantic_core.ValidationError: 1 validation error for VirtualPackageSpec
subdirs.linux-64.packages.__glibc
Input should be a valid string [type=string_type, input_value=2.27, input_type=float]
For further information visit <https://errors.pydantic.dev/2.7/v/string_type>
I seem to have nailed it down to these lines in the 
mfextinit_netflixext.py
(I am trying to run an AWS Batch job on a GPU).
# Packages to add when building for GPU machines (ie: if there is a GPU resource
# requirement). As an example you can set this to:
# CONDA_SYS_DEFAULT_GPU_PACKAGES = {
#     "__cuda": os.environ.get("CONDA_OVERRIDE_CUDA", "11.8=0")
# }
CONDA_SYS_DEFAULT_GPU_PACKAGES = {
"__glibc": os.environ.get("CONDA_OVERRIDE_GLIBC", "2.27")
}
(edited)
d
I mean what’s before that? Where does this pydantic error come from?
I don’t use pydantic anywhere in the extension so I don’t think this is coming from the code in the extension so wondering what is triggering the issue.
f
Apologies, here is the full error:
Metaflow 2.12.5+netflix-ext(1.2.0) executing test_gpu for user:ed
Validating your flow...
The graph looks good!
Running pylint...
Pylint not found, so extra checks are disabled.
Bootstrapping Conda environment... (this could take a few minutes)
Resolving 1 of 2 environments (others are cached) ...No STDOUT
Pretty-printed STDERR:
Traceback (most recent call last):
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/bin/conda-lock", line 8, in <module>
sys.exit(main())
^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1403, in lock
lock_func(
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1111, in run_lock
make_lock_files(
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 317, in make_lock_files
virtual_package_repo = virtual_package_repo_from_specification(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/conda_lock/virtual_package.py", line 260, in virtual_package_repo_from_specification
spec = VirtualPackageSpec.parse_obj(data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/pydantic/main.py", line 1118, in parse_obj
return cls.model_validate(obj)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ed/miniconda3/envs/test-metaflow-gpu/lib/python3.12/site-packages/pydantic/main.py", line 551, in model_validate
return cls.__pydantic_validator__.validate_python(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for VirtualPackageSpec
subdirs.linux-64.packages.__glibc
Input should be a valid string [type=string_type, input_value=2.27, input_type=float]
For further information visit <https://errors.pydantic.dev/2.7/v/string_type>
Conda ran into an error while setting up environment.:
Binary command for Conda '['/Users/ed/miniconda3/envs/test-metaflow-gpu/bin/conda-lock', 'lock', '-f', 'pyproject.toml', '-p', 'linux-64', '--filename-template', '/tmp/conda-lock-gen-tmpfuxo110w', '-k', 'explicit', '--conda', '/Users/ed/miniconda3/condabin/mamba', '--mamba', '--virtual-package-spec', 'virtual_yml.spec']' returned error (1); see pretty-printed error above
d
ah got it. It’s coming from conda-lock. That’s interesting. I didn’t get this error. I have to run to a meeting but here is what you can try for now and I’ll try to take a look later today/tonight: • you can probably just remove the glibc setting and just use the cuda one (ie: uncomment the commented part and comment the one about glibc — if you are doing this from a linux machine reasonably recent, it should be fine and you shouldn’t have any issue with it) • let me know what version of conda-lock you have. I’ll check if there is any recent change that is causing that.
f
I was using the most recent version of conda-lock==2.5.7. Unfortunately using CUDA did not work.
I tried using 
conda-lock==2.1.0
, but I got a message saying 
conda-lock is installed but not recent enough (2.4.0 or later is required) --ignoring
. When I tried 2.4.0, I got the same issue with Pydantic.
d
Could you send me the dependencies you are trying to resolve. I will try to repro and see what I find. I haven’t seen this before and we definitely use the glibc thing. Could you also let me know the version of pydantic that you gave.
Have
f
@pypi(packages={'tensorflow': '2.15.0'})
@conda(libraries={'keras-cv': '0.9.0'})
pydantic 2.8.2
r
I’m having a different, but similar, problem also when trying to use both pypi and conda (on Mac) 
*Metaflow 2.12.7+netflix-ext(1.2.0);mc(0.20.0)* executing *TheMachineHumRgtsSubnat* for *user:i35085*
Project: *human_rights*, Branch: *user.i35085*
Validating your flow...
The graph looks good!
 
Running pylint...
Pylint is happy!
 
Bootstrapping Conda environment... (this could take a few minutes)
Resolving 1 of 2 environments (others are cached) ...No STDOUT
Pretty-printed STDERR:
Usage: conda-lock lock [OPTIONS]
Try 'conda-lock lock --help' for help.
Error: Got unexpected extra argument (virtual_yml.spec)
*Conda ran into an error while setting up environment.*:
Binary command for Conda '['/Users/i35085/.local/bin/conda-lock', 'lock', '-f', 'pyproject.toml', '-p', 'osx-arm64', '--filename-template', '/tmp/conda-lock-gen-tmp4akrlcqf', '-k', 'explicit', '--conda', '/opt/homebrew/Caskroom/miniforge/base/condabin/conda', '--conda', '--virtual-package-spec', 'virtual_yml.spec']' returned error (2); see pretty-printed error above
conda-lock --version
conda-lock, version 2.5.7
d
that one is even weirder. I’ll try to look at both of these today (sorry, i’ve been running late on a bunch of things 😞 )
these errors should be fixed in the next release. Finally got around to fixing it — it was idiotic.
f
Great, thanks for getting back.
5 Views
Open in Slack
PreviousNext
Powered by