Hello, does the metaflow service or metaflow UI produce any operational metrics? Something that we can use in Grafana dashboards for example?

You should be able to get container metrics as usual. There isn’t anything specific that the service publishes but it shouldn’t be too hard to extend it to behave the way you would like it to

@square-wire-39606 do you know which port does it produce those metrics? I couldn’t find any doc or code to configure those metrics

are you running prometheus or any other metrics monitor?

prometheus

Hey Ammar, the service doesn’t “emit” any metrics of its own and so doesn’t even expose any end point for that. I think what Savin meant is that any generic metrics agent like Victoria Metrics agent should be able to get some metrics about container like cpu usage etc. What kinds of metrics are trying to monitor ?

@little-apartment-49355 Hey Valay. So I was looking for metaflow specific metrics like “Number of flows by status”, “Flow step latency”, “Blob latency”, etc

@better-printer-5326 the argo related metrics will only cover argo usage and not provide a full picture. Are there specific statistics that you are looking for?

Yeah, let me put down a few metrics that I think would be useful: 1. Metaflow step metric: emits every time a step is executed with flow-id/step-id, timestamps and status (running, failed, success, etc). From this we can know latency between steps and which steps fail/succeed 2. Metaflow flow metrics: same as 1 but on the flow level with additional information about whether it was a dev flow/argo flow etc. Tells us how many flows are running per hour/day/etc and percentage of failure vs success. 3. HTTP: something like opencensus_io_http_server_completed_count and opencensus_io_http_server_server_latency_bucket to help us know how the metaflow services are responding to requests

on that note, we have support for open telemetry, custom loggers and metrics collectors that can allow you to build out your own solution