is there a way to override/add to the printed messages on termination of a flow? one thing i am used to with devops tools is that when a service instance terminates prematurely, the logs typically are very good and we dump certain outputs when an instance is terminated (like a heap dump, or service logs) but with metaflow, i find that when running a big/long flow, and it runs into an errors halfway through (or is stuck processing a task for 1hr that should only take ~5 minutes) , its a bit obscure to track down these errors in the stack trace amongst the other default MF outputs eg... I am running a flow on ~350gb of data, using foreach (not @batch yet). sometimes the flow terminates prematurely, because of some sort of data validation error or OOM error. i'd like to catch those errors and print them out, but i have other console output happening during the flow which pushes the errors im interested in out of scrollable view in the terminal. ideally, id like some way to include print statements before/after the stack trace output when a flow terminates. any ideas on this? a few alternatives i can reason with: • use metaflow UI and observe failures/errors with it • pipe all

metaflow run

console output to some sort of

flow_output.log

file , and view it manually to find the errors im interested in

Metaflow stores task-specific logs in files like

flow_output.log

which you can examine with the

logs

command or using the Client API