Hi all, new to metaflow, got a question regarding ...
# ask-metaflow
w
Hi all, new to metaflow, got a question regarding deploying metaflow to EKS and using RDS/Postgresql as backend storage; We have deployed many metaflow services and metaflow ui to many workspace on EKS for isolation between teams. We have deployed a single RDS/Postgresql and for each metaflow services we create schema. Now issue is that we are hitting number of max connection limit. We have tried to use RDS proxy to reuse connection and reduce number of connection to RDS. However it didn't work. What we see in the log of rds proxy is; "The client session was pinned to the database connection [dbConnection=xxxxx] for the remainder of the session. The proxy can't reuse this connection until the session ends. Reason: SQL changed session settings that the proxy doesn't track. Consider moving session configuration to the proxy's initialization query. Digest: "listen notify". 2023-05-12T131931.906Z [WARN] [proxyEndpoint=default] [clientConnection=xxxx] The client session was pinned to the database connection [dbConnection=xxxx] for the remainder of the session. The proxy can't reuse this connection until the session ends. Reason: SQL changed session settings that the proxy doesn't track. Consider moving session configuration to the proxy's initialization query. Digest: "listen notify"." Question is there way option or feature that we can disable ListenNotify action and any way to configure DB connection number ? More general question, do you have any suggestion how to architect such kind of requirement where you have many metaflow services on EKS and single RDS ? Thank you and apologise for silly question in advance
1
a
@wooden-ghost-87165 what are the isolation guarantees that you are seeking between the teams. Given that there is a shared DB (and assuming tables are shared too) between all the teams, all the UIs will have the same information.
w
@square-wire-39606 we have metaflow service and metaflow ui for each namespace. We have single instance of RDS however, for each metaflow have different schema(tables for each namespace are separated). I just saw (FEATURE_DB_LISTEN_DISABLE) in metaflow ui which may help us to prevent see pinned connection to DB. However some ui function might lost.
👍🏼 1