We are running into a problem when retrieving artifacts from Outerbounds #ask-metaflow

We are running into a problem when retrieving arti...

shy-refrigerator-15055

04/27/2023, 11:23 AM

We are running into a problem when retrieving artifacts from previous flows, are we doing something wrong here?

Copy code

>>> from metaflow import namespace, Flow
>>> data = Flow("TestSitesFlow/1858/join_metrics/28465").latest_run.data
>>> data
<MetaflowData: name>
>>> data.name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.10/site-packages/metaflow/client/core.py", line 736, in __getattr__
    return self._artifacts[name].data
  File "/opt/homebrew/lib/python3.10/site-packages/metaflow/client/core.py", line 902, in data
    obj = filecache.get_artifact(ds_type, location[6:], meta, *components)
TypeError: FileCache.get_artifact() takes 9 positional arguments but 12 were given

ancient-application-36103

04/27/2023, 1:35 PM

Which version of metaflow are you on?

shy-refrigerator-15055

04/27/2023, 1:44 PM

We might actually be mixing a couple versions, I think the oldest one is 2.7.22

shy-refrigerator-15055

04/27/2023, 1:46 PM

The version shown by the python package running the command above is 2.8.0

square-wire-39606

04/27/2023, 2:23 PM

What is the output of

Copy code

Flow("TestSitesFlow/1858/join_metrics/28465").latest_run.end_task.metadata_dict

better-branch-69556

04/27/2023, 2:29 PM

Copy code

{'runtime': 'dev',
 'python_version': '3.11.2',
 'user': 'tom',
 'metaflow_version': '2.8.3',
 'attempt': '0',
 'origin-run-id': 'None',
 'ds-type': 's3',
 'ds-root': '<s3://metaflow-s3>-*************/metaflow',
 'attempt_ok': 'True',
 'attempt-done': '0'}

square-wire-39606

04/27/2023, 2:34 PM

and what was the version of metaflow used to execute this above command and the original command that errored out?

better-branch-69556

04/27/2023, 2:35 PM

pyproject.toml

has

metaflow = "^2.8.0"

We're also using a custom image, but it has the same version built-in:

Copy code

> docker run --rm -it ********.<http://dkr.ecr.eu-west-1.amazonaws.com/natcap:351117a0758a|dkr.ecr.eu-west-1.amazonaws.com/natcap:351117a0758a> python
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import metaflow
>>> metaflow.__version__
'2.8.3'

3 Views

Open in Slack

Previous Next