Hello all,
How do we reference the input in a nes...
# ask-metaflowa
acoustic-van-30942
03/05/2025, 11:50 PMHello all,
How do we reference the input in a nested foreach? When I call
self.input✅ 1
h
hundreds-rainbow-67050
03/05/2025, 11:51 PMwhats the error youre getting?
a
acoustic-van-30942
03/05/2025, 11:52 PM Copy code
File "/metaflow/metaflow/cage_extractor_flow_v2_extract.py", line 113, in start_cluster
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] batch_id = self.input
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 490, in input
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return self._find_input()
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 568, in _find_input
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] self._cached_input[stack_index] = var[frame.index]
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ~~~^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] KeyError: 4
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp]
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] During handling of the above exception, another exception occurred:
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp]
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] Traceback (most recent call last):
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/cli.py", line 633, in main
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] start(auto_envvar_prefix="METAFLOW", obj=state)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/_vendor/click/core.py", line 829, in __call__
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return self.main(args, kwargs)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/_vendor/click/core.py", line 782, in main
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] rv = self.invoke(ctx)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/cli_components/utils.py", line 69, in invoke
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return _process_result(sub_ctx.command.invoke(sub_ctx))
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/_vendor/click/core.py", line 1066, in invoke
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return ctx.invoke(self.callback, ctx.params)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/_vendor/click/core.py", line 610, in invoke
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return callback(args, kwargs)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/tracing/__init__.py", line 27, in wrapper_func
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return func(args, kwargs)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/_vendor/click/decorators.py", line 21, in new_func
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return f(get_current_context(), args, kwargs)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/cli_components/step_cmd.py", line 167, in step
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] task.run_step(
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/task.py", line 712, in run_step
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] print("%s failed:" % self.flow, file=sys.stderr)
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ~~~~~~~~~~~~~^~~~~~~~~~~
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 906, in __str__
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] index = ",".join(str(idx) for idx, _, _ in self.foreach_stack())
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 539, in foreach_stack
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] return [
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 540, in <listcomp>
2025-03-05 23:48:02.849 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] (frame.index, frame.num_splits, self._find_input(stack_index=i))
2025-03-05 23:48:02.850 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 23:48:02.850 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] File "/metaflow/metaflow/metaflow/flowspec.py", line 568, in _find_input
2025-03-05 23:48:02.850 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] self._cached_input[stack_index] = var[frame.index]
2025-03-05 23:48:02.850 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] ~~~^^^^^^^^^^^^^
2025-03-05 23:48:02.850 [2783/start_cluster/104038 (pid 24313)] [pod t-814bde73-fwqzs-wkxdp] KeyError: 4acoustic-van-30942
03/06/2025, 12:02 AMOh nevermind. Figured it out. The iterator was a dict instead of list