Hiya folks 🙂
I'm making a metaflow flow right now to train and validate a Markov model!
I tried to create a step that saved my BigQuery client as a class attribute, as per the metaflow documentation, to use it in multiple steps (both for training and for validating the model). I got an error stating that the object is unpicklable. Investigating that error message eventually led me to discover that Metaflow pickles objects intended to be kept as artifacts.
So then I instead instantiated a
new BQ client in each of the steps of the flow that require one. That got me past that error. I then get this at the end of the step that instantiates the model and attempts to save
it as an attribute on the flow class (the model class inherits directly from
object
😞
_pickle.PicklingError: Can't pickle <function root at 0x11e52ccc0>: attribute lookup root on __main__ failed