Hello, I'm trying to use the new pypi decorator b...
# ask-metaflow
n
Hello, I'm trying to use the new pypi decorator but I'm running into an issue with installing the "sentence-transformers" package. Specifically, I'm running into the following error: "ERROR: Could not find a version that satisfies the requirement sentence-transformers==2.2.2 (from versions: none)" "ERROR: No matching distribution found for sentence-transformers==2.2.2" I'm running it on a M2 MacbookPro. Any help is appreciated!
1
a
@nice-waitress-52063 currently the decorator only supports wheels. sentence-transformers are not published as wheels currently. we are actively working on relaxing this condition.
d
@nice-waitress-52063 , a few threads up there is something similar about the CLIP package. The solution there should also work for you. You can use the bleeding edge decorator versions which do support non wheel packages. The restriction there is no cross architecture (resolving on a mac to execute on batch) but there is a solution for that too in the thread above. The link to the thread is: https://netflix.slack.com/archives/C02116BBNTU/p1697198947028709. Let us know if you have any questions but it should work for you for sentence-transformers. Not in front of a computer so can’t try this minute but if it doesn’t work for you, let me know and I’ll try it out.
On second thoughts that may not work either. I’ll try it out (some pip restriction)
a
another simple way to unblock yourself would be to replace the
@pypi
decorator with the
@conda
decorator - all of your packages are available in the
conda-forge
channel.
n
Thanks for the quick responses! Noted on the wheels-only support. To resolve this, I've migrated to the 'transformers' package instead of 'sentence-transformers' so all good. Looking forward to wheels support :)
d
A quick additional information. I agree with @ancient-application-36103 that if your packages are on conda, that’s your best bet. The reason it doesn’t work right now with pip has to do with a pip restriction more than anything else. Pip requires you to specify that you only consider binary packages (so wheels in essence) if you want to also specify the architecture to resolve for. One option that will work and that some companies do is to have locally built wheels for the packages you need and you can then tell pip to also look there for packages. It’s a bit weird and I am not 100% sure why pip does this though if I had to guess is that because technically you can have some platform specific dependencies which would not show up in a non wheel package like a source tar.
🙏 1