Hi again everyone. Are netflix extensions able to download files from a conda s3 channel? Running with Metaflow 2.11.4+netflix-ext(1.1.2) and

METAFLOW_CONDA_DEPENDENCY_RESOLVER=conda

Following package resolve, I'm getting the following error:

Downloading 1(web) + 0(cache) package for arch linux-64 ...debug[conda /home/.../python3.11/site-packages/metaflow_extensions/netflix_ext/plugins/conda/conda.py:1305]: pkg-6.2.0-py_0 -> download <s3://xxxx> to /home/xxx.tar.bz2

done in 0 seconds.

Got the following errors while loading packages:

Error downloading package for 'pkg-6.2.0-py_0': No connection adapters were found for '<s3://xxx/noarch/pkg-6.2.0-py_0.tar.bz2>'

Conda ran into an error while setting up environment.:

Could not fetch packages -- see pretty-printed errors above.

Looks like it's coming from here. At a glance, this code seems to be using requests - which would explain the error message above, so is getting packages this way unsupported? Thanks

That’s a good point. I hadn’t thought of this. I can probably add this support if you need it (The plugin will download from S3 once they have been fetched from the web but yes, I don’t think I had considered fetching initial packages from S3. How urgent is this to add? It should be fairly straightforward so can do it but just asking.

Hi Romain, thanks for your reply. In our case, the packages are in an s3 bucket with auth. I think you've had previous discussions with my co-worker @rhythmic-beach-70913. He's best placed to comment on urgency and might chime in next week, but I think this will be required for us to upgrade metaflow. So anything you can do to implement this would be much welcomed, but I appreciate it may not be a top priority for you. And yes, very happy to test.