I have a real head scratcher I have a flow where I m trying Outerbounds #ask-metaflow

I have a real head-scratcher... I have a flow wher...

cool-father-88039

04/25/2023, 11:09 PM

I have a real head-scratcher... I have a flow where I'm trying to read in an artifact from another run to be used in the current run. When trying to access that artifact, I get this error. If I put a breakpoint in the metaflow source code here, I can see that the

name

___getstate___

, which seems like a bug of some kind. The code would be something like

Copy code

self.training_data_run: Run = Flow("TrainingDataGenerationFlow").latest_successful_run
        self.training_data_run_data: MetaflowData = self.training_data_run.data
        self.loaded_data = self.training_data_run_data.loaded_data

Untitled

✅ 1

cool-father-88039

04/26/2023, 1:02 AM

The issue actually stemmed from the fact that I was trying to assign a run to a flow attribute (

self.training_data_run: Run = ...

). Removing this makes the flow succeed. Curiously, I see a custom exception in the source code saying as much, but that exception was never raised for me.

square-wire-39606

04/26/2023, 1:24 AM

we have a fix inflight - https://github.com/Netflix/metaflow/pull/1363

🤔 1

👀 1

cool-father-88039

04/26/2023, 4:12 AM

thanks (I actually came across that and didn't think that it applied to my case; I'm using the same version of Metaflow for the two runs)

square-wire-39606

04/26/2023, 2:43 PM

ideally - we would recommend storing the pathspec as an artifact and recreating the object using the pathspec later

👍 1

8 Views

Open in Slack

Previous Next