I am trying to use lightgbm as part of a flow scheduled via aws step functions using aws batch and I have to use pypi. How can I avoid this please?

2024-10-18 16:43:27.028 Bootstrapping virtual environment(s) ...
    Pip ran into an error while setting up environment:
    Could not find a binary distribution for lightgbm==4.5.0 
    for the platform linux-64
    
    Note that @pypi does not currently support source distributions

AFIK it is because of lack of libomp? Maybe I could use another docker image for aws batch? Thanks!

you can try the netflix extension which should support this. At least for non cross-platform (it won’t be able to build a distribution if the distribution is platform specific and you are on a mac for example — there is a way around that too but a bit more involved).

Thanks. Are they the official ones or the not yet merged ones. I have to say I struggled with the latter a couple of weeks ago.

send me what env you are trying to resolve and I’ll try it out and let you know. I do have a few pending PRs out — the release of micromamba 2.0 has complicated a lot of things in the conda ecosystem so it has been a bit more complicated to get things all merged in. I haven’t looked at it in a week or two so can take another look

Thanks for the offer. Thus far I now got away with using conda in this specific flow … sorry about the confusion. If I need to use pipy and conda mixed I will get back to you …