Hi team. I am seeing a recurring pattern in our self-hosted UI Backend metaflow service which keeps on crashing due to OOM every couple of hours. When looking at memory metrics it seems, the memory keeps on piling up over time as if there’s some memory leak. Does anyone happen to know what might be up?

there's a cache. How big is your instance?

I only assigned it 2G of memory thinking that a UI backend should not need more than that. Is there a minimum memory requirement? Alternatively, is it possible to reduce the cache size?

try increasing it to, say, 8GB to confirm that it works in general the UI will work more reliably at scale if it's not very resource constrained