https://outerbounds.com/ logo
Join Slack
Powered by
Hi all, I'm getting an error when trying to impor...
# ask-metaflow
a
Hi all, I'm getting an error when trying to import my own custom library that's in the same folder as my flow file. When I do 
python gw_train.py show
, it returns the following error:
Copy code
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/scipy/linalg/_matfuncs_sqrtm_triu.cpython-310-x86_64-linux-gnu.so)
Any insight as to what is causing this error? As a workaround, I suppose I could just create my own custom dockerfile, then bake this custom package into it, and then pass the ecr docker image into 
@batch
. But would be nice to just import this directly and have Metaflow automatically create a code package including my custom libraries.
✅ 1
My flow file looks something like this:
Copy code
from metaflow import FlowSpec, step, batch, conda
from my_model.train_and_evaluate import runStandalone

class GWFlow(FlowSpec):
    @step
    def start(self):
        """
        This is the start of the flow
        """
        self.next(self.load_data)
    
     @conda(libraries={'bidict': '0.22.1', 
                       'torch': '1.11.0', 
                       'torch-geometric': '2.1.0', 
                       'torch-scatter': '2.1.0',
                       'torch-sparse': '0.6.16',
                       'torchvision': '0.12.0',
                       'pandas-profiling': '3.6.6',
                       'graphviz': '0.20.1'
                      }, 
            python='3.8.1'
           )
    @batch(cpu=8, memory=10000)
    @step
    def train(self):
        """
        Train embedder model
        """
            
        print('Train embedder model ...')
        runStandalone(...)
            
        self.next(self.end)
        
    @step
    def end(self):
        """
        This is the end of the flow
        """
        
if __name__ == "__main__":
    GWFlow()
c
have seen this with some versions of scipy conda packages. do you still see the error if you add libstdcxx-ng and libgcc to the libraries dict of your conda decorator?
a
@crooked-jordan-29960 - Unfortunately, yes. I don't think it evaluates the conda package before it starts erroring out, since I'm importing my custom library outside of the 
class
.
c
I see, that is more interesting. Does the error only occur when you use the 
my_model
library in a python file that runs a flow?
this thread also helped me with this error before. upshot was to
Copy code
export LD_LIBRARY_PATH=<PATH TO CONDA /lib DIRECTORY>
a
Yup, I'm afraid so. If I comment out 
my_model
import, no error.
c
Are you able to do 
from my_model.train_and_evaluate import runStandalone
in other Python files?
a
Let me check. I know I can import it directly in my Jupyter Notebook cell
No I cannot. Good catch!
c
ok my hunch then is maybe something in the 
my_model
setup script to do 🤔 will do some searching in ten minutes or so
thankyou 1
a
Oh yay! Your suggestion worked! Thanks so much Eddie.
Copy code
%%bash

export LD_LIBRARY_PATH="/home/ec2-user/anaconda3/lib"
python my_train.py
c
great! is the flow working as well for you now?
a
Will try now
So far so good, thanks again Eddie! Really appreciate it.
🤗 1
Oh spoke to soon...some sort of conda issue:
Copy code
Bootstrapping conda environment...(this could take a few minutes)
    Conda ran into an error while setting up environment.:
    Step: start, Error: command '['/home/ec2-user/anaconda3/condabin/conda', 'create', '--yes', '--no-default-packages', '--name', 'metaflow_GWFlow_linux-64_7b6d74c7d55dd5d2b9a4874a8334691e4d156140', '--quiet', b'python==3.10.6', b'requests==>=2.21.0', b'boto3==>=1.14.0']'
I did specify those in my libraries though...
Copy code
@conda(libraries={'bidict': '0.22.1', 
                      'torch': '1.11.0', 
                      'torch-geometric': '2.1.0', 
                      'torch-scatter': '2.1.0',
                      'torch-sparse': '0.6.16',
                      'torchvision': '0.12.0',
                      'pandas-profiling': '3.6.6',
                      'graphviz': '0.20.1',
                      'boto3': '1.14.0',
                      'requests': '2.21.0'
                     }, 
           python='3.10.6'
          )
c
This error appears to be from the 
start
step, and in the flow pasted above the 
@conda
is only being applied to the 
train
step. A quick option that I tend to do on flows with only a few steps like this is to change this decorator to 
@conda_base
and move to flow-level
Copy code
@conda_base(libraries={'bidict': '0.22.1', 
                      'torch': '1.11.0', 
                      'torch-geometric': '2.1.0', 
                      'torch-scatter': '2.1.0',
                      'torch-sparse': '0.6.16',
                      'torchvision': '0.12.0',
                      'pandas-profiling': '3.6.6',
                      'graphviz': '0.20.1',
                      'boto3': '1.14.0',
                      'requests': '2.21.0'
                     }, 
           python='3.10.6'
          )
class GWFlow(FlowSpec):
a
Okay got it, thanks. The start step is pretty barebones though. There's nothing in it actually besides a pointer to the 
train
step
Ok that did the trick. Thanks a million yet again!
c
anytime!
🙏 1
9 Views
Open in Slack
PreviousNext
Powered by