https://outerbounds.com/ logo
Join Slack
Powered by
Hi! Has anyone had this error on resume of pipelin...
# ask-metaflow
i
Hi! Has anyone had this error on resume of pipeline:
Copy code
return [self._objects.get(name) for name in names]
AttributeError: 'TaskDataStore' object has no attribute '_objects'
?
✅ 1
a
What do the full logs look like? Also what version of metaflow are you on?
i
Copy code
alidating your flow...
    The graph looks good!
Running pylint...
    Pylint not found, so extra checks are disabled.
2024-11-12 09:19:59.859 Gathering required information to resume run (this may take a bit of time)...
    Internal error
Traceback (most recent call last):
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/cli.py", line 1134, in main
    start(auto_envvar_prefix="METAFLOW", obj=state)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/tracing/__init__.py", line 27, in wrapper_func
    return func(args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 829, in __call__
    return self.main(args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, ctx.params)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 610, in invoke
    return callback(args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/cli.py", line 585, in wrapper
    return func(args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/_vendor/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/cli.py", line 676, in resume
    runtime = NativeRuntime(
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/runtime.py", line 152, in __init__
    self._origin_ds_set = TaskDataStoreSet(
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/datastore/datastore_set.py", line 33, in __init__
    prefetch.update(ds.keys_for_artifacts(prefetch_data_artifacts))
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/datastore/task_datastore.py", line 45, in method
    return f(self, args, kwargs)
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/datastore/task_datastore.py", line 242, in keys_for_artifacts
    return [self._objects.get(name) for name in names]
  File "/home/vkulikov/txt2img-data/.venv/lib/python3.10/site-packages/metaflow/datastore/task_datastore.py", line 242, in <listcomp>
    return [self._objects.get(name) for name in names]
AttributeError: 'TaskDataStore' object has no attribute '_objects'
v '2.12.13'
a
If you upgrade to 2.12.29 and try again, do you see the same issue?
We had a bug fix around 2.12.13 iirc
i
Yes, same issue
👀 1
a
Can you help me with a reproducible example? Would help in identifying the issue
i
Ok, will try. The strange thing it has problems not for all of the runs
I will try to explore the bucket
Open in Slack
PreviousNext
Powered by