Hello! Question about logging to the Metaflow UI: What is the rule when stdout logs get available in the Metaflow UI? I noticed that sometimes it takes until the step finished, even if the step takes very long. Meanwhile it shows "No logs" or "No logs. Logs will be checked again momentarily" It seems it might be different depending on the execution environment / orchestrator? Would love to understand what the behaviour is and if I can influence it. Can I make the logs flush to the Metaflow UI?

Hello! As of today - there are two modes - 1. If the workload is running on a remote node (say on AWS Batch, Kubernetes or SLURM), metaflow pushes the logs to the blob store at regular intervals. Both the console and the UI streams from the blob store to show the logs to the users in realtime 2. If the workload is running locally, the logs are streamed to the user using traditional stdout/stderr and at the end of the workload, they are pushed to the blob store from where the UI is able to pull them. The reason why we don't push logs repeatedly from your local setup is to save on your network bandwidth costs.