For users that haven't stood up a metaflow UI, how...
# ask-metaflow
p
For users that haven't stood up a metaflow UI, how do you access your logs for a given run_id? We are deploying via Argo on a k8s cluster and we'd like to be able to access the logs of the particular k8s pods on which metaflow deploys certain steps for debugging purposes. Any advice? Apologies if this has been asked before
1
a
all the logs are snapshotted by metaflow and are available from the client - https://docs.metaflow.org/metaflow/client#properties-related-to-tasks
also -
python flow.py logs -h
can help
p
Thanks Savin! Definitely looked at the metrics provided by metaflow natively around the different levels of hierarchy of a flow. These are super useful until they aren't Will try the other suggestion. I'm wondering about access to the raw logs of a run, as well as the logs of the pods on which the run was executed
I see stdout and stderr for task level properties, maybe it's a metaflowconfig issue, but I've previously seen that those are null
s
if you are debugging in the user space, then the runtime logs and task logs that metaflow tracks are useful - these are the same ones that show up in the terminal too.
if you are building your own extensions and hacking on metaflow, then you would need some visibility into the pods from outside metaflow
p
appreciate your pointers, i'm revisiting this issue now. we aren't hacking metaflow with custom extensions. i was trying to retrieve logs by inspecting task level properties but my steps don't seem to have a task object at all and i don't see stdout / stderr attributes for my step. these are the attributes i'm seeing:
Copy code
>>> step.
step.close(        step.gi_code       step.gi_frame      step.gi_running    step.gi_suspended  step.gi_yieldfrom  step.send(         step.throw(
should i be writing flows in a particular way to ensure that my steps have tasks? anything obvious i'm missing?
s
How is the step object constructed?