Scaling PgBouncer to 4x Throughput: A Fleet-Based Approach

PgBouncer is architecturally limited by its single-threaded design. Regardless of the number of available vCPUs on a host machine, a standard PgBouncer instance consumes only one CPU core. In high-concurrency environments, this creates a performance ceiling where the connection pooler becomes the primary throughput bottleneck while the underlying PostgreSQL database and the host hardware remain significantly underutilized.

When a single PgBouncer process approaches its throughput limit, it exhibits high CPU saturation on a single core, leading to increased latency and eventual connection rejection once the max_client_conn threshold is reached. Under these conditions, scaling vertically by adding more vCPUs to the instance provides no performance benefit, as the remaining cores stay idle.

To overcome this limitation, engineers can deploy a fleet of PgBouncer processes, utilizing the SO_REUSEPORT socket option to allow multiple processes to bind to the same port. This configuration enables the Linux kernel to load-balance incoming connections across the entire process fleet. However, implementing a multi-process architecture introduces specific operational challenges:

• Query Cancellation: Postgres cancel requests arrive on independent connections. If a cancel request is routed to a different process than the one managing the active query, the request will fail. Implementing a peering mechanism, where processes communicate to forward cancellations to the correct owner, is required for robust operation.
• Connection Budgeting: Total connection limits, such as max_client_conn and max_db_connections, must be divided across the fleet. Failing to properly partition these limits can lead to oversubscription of the database.

While a single-threaded configuration may be sufficient for low-concurrency workloads, high-throughput systems require this multi-process, peer-aware deployment to fully utilize available hardware. Without these mitigations, the pooler acts as a hard wall, artificially capping throughput long before the database reaches its actual capacity.

PgBouncer is single-threaded per process, meaning a single instance can use only one CPU core regardless of available hardware. On a multi-core machine, this leaves the majority of cores idle while the pooler becomes the bottleneck before the database saturates. To scale horizontally across cores, deploy a fleet of PgBouncer processes, each binding the same TCP port with the SO_REUSEPORT socket option. The kernel then distributes incoming connections across the processes using a hash-based load balancer, preserving a single client endpoint. Clients connect to one address and port, unaware that multiple pooler processes handle their sessions.

Query cancellation introduces a complication. A PostgreSQL cancel request arrives on a new, short-lived connection carrying a cancel key, separate from the connection executing the query. With SO_REUSEPORT, the kernel may route this new connection to a different PgBouncer process than the one holding the session. The cancel then lands on a process that has no knowledge of the query, and the cancellation fails. Peering resolves this: each process maintains awareness of all other processes in the fleet. When a cancel request arrives on the wrong process, it is forwarded internally to the process that owns the session. This ensures cancellation works across the entire fleet regardless of which process receives the request.

Connection budgets must be partitioned across the fleet to avoid oversubscribing PostgreSQL. The configuration parameters max_client_conn and max_db_connections are divided by the number of processes. For example, with 16 processes and a desired total of 1024 client connections, each process is configured with max_client_conn = 64. This per-process limit keeps each instance within safe bounds while the aggregate ceiling scales linearly with the fleet size.

In a production deployment on a 16-vCPU instance, a single PgBouncer process peaked at approximately 87,000 transactions per second and degraded under higher concurrency, using only one core. A fleet of 16 processes on the same instance type reached roughly 336,000 transactions per second, utilizing about 8 cores. The single-process instance consumed under 10% of the machine’s CPU capacity, while the fleet used over 50%, demonstrating effective utilization of available hardware.

• Enable SO_REUSEPORT on the listening socket so all processes share the same port.
• Configure peering between processes to forward cancel requests to the correct owner.
• Divide max_client_conn and max_db_connections by the number of processes.
• Monitor per-process metrics (CPU, connection counts) to detect imbalance or saturation.

Query cancellation in a horizontally scaled connection pooler presents a fundamental routing challenge. A Postgres cancel request arrives on a new, separate connection carrying only a cancel key, not a reference to the process that owns the target session. When the kernel load-balances incoming connections across multiple pooler processes using SO_REUSEPORT, the cancel request can be delivered to any process in the fleet, not necessarily the one holding the query. The receiving process has no local knowledge of the session, so the cancel is silently dropped.

The solution is peering: each process in the fleet maintains awareness of every other process and the sessions they own. When a cancel request lands on the wrong process, that process inspects the cancel key, determines which peer holds the target session, and forwards the request over an internal communication channel. The peer that owns the session then executes the cancellation against its local Postgres connection. This ensures cancellation works across the entire fleet regardless of which process receives the incoming request.

Practical implementation details include:

• Cancel key distribution: Each process broadcasts its active session cancel keys to all peers, or peers query a shared in-memory index on demand.
• Forwarding protocol: A lightweight internal RPC or shared-memory mechanism carries the cancel request from the receiving process to the owning process.
• Timeout handling: If the owning process does not acknowledge the forwarded cancel within a configurable interval, the receiving process returns an error to the client.
• Connection budget splitting: max_client_conn and max_db_connections are divided evenly across the fleet so that aggregate connections never exceed Postgres limits, preventing resource exhaustion that could interfere with cancellation.

For example, in a 16-process fleet using SO_REUSEPORT, a cancel request for a session on process #12 might arrive at process #5. Process #5 looks up the cancel key in its peer table, identifies process #12 as the owner, and forwards the request. Process #12 receives the forwarded request, matches it to the local session, and issues the Postgres cancellation. The client receives a successful cancellation response even though it never connected directly to process #12.

Without peering, a single-process pooler avoids this problem entirely because all sessions and cancel requests land on the same process. However, that design caps throughput at one CPU core. Peering removes the bottleneck while preserving correct cancellation semantics across the fleet.

Connection poolers such as PgBouncer are single-threaded; a single process uses at most one CPU core regardless of available cores. To scale connection handling without oversubscribing the Postgres backend, the fleet must split the connection budget across all processes. This means dividing max_client_conn and max_db_connections by the number of pooler processes so that the aggregate connections never exceed the limits Postgres can safely handle.

Without budget splitting, a single pooler process enforces its own max_client_conn independently. If that limit is set too high relative to Postgres capacity, the pooler can admit more client connections than the database can serve, leading to resource exhaustion. By dividing the budget across the fleet, each process enforces a fraction of the total ceiling, and the fleet as a whole respects the intended cap.

Practical implementation requires three components:

• so_reuseport – All pooler processes bind the same port; the kernel distributes incoming connections across them. Clients connect to a single endpoint.
• Peering – Processes share session state so that query cancellation (which arrives on a new connection) can be forwarded to the correct process.
• Budget division – max_client_conn and max_db_connections are each divided by the number of processes. For example, if the target is 1000 client connections and the fleet has 10 processes, each process enforces max_client_conn = 100. The same logic applies to per-database connection limits.

In a real deployment on a 16-vCPU instance, a single PgBouncer process peaked at ~87k transactions/sec and degraded under load, while a fleet of 16 processes reached ~336k transactions/sec—roughly 4× throughput—because it could use multiple cores. The single process left the machine mostly idle (~9% CPU), whereas the fleet utilized ~52% CPU, demonstrating that budget splitting prevents the pooler from becoming the bottleneck while keeping Postgres safe from oversubscription.

Key recommendations for enterprise engineers:

• Size the fleet proportional to available cores; one process per vCPU is a common starting point.
• Divide max_client_conn and max_db_connections by the number of processes to maintain the aggregate limit.
• Enable so_reuseport on the listening socket and configure peering for cancel forwarding.
• Monitor per-process connection counts and Postgres backend utilization to validate that the budget split is effective.

This approach ensures that resource governance is maintained at the fleet level, not per process, preventing Postgres from being oversubscribed while scaling connection handling horizontally.

Benchmarking was conducted on identical AWS c7i.4xlarge instances (16 vCPUs) with a dedicated Postgres server and a pgbench load generator in select-only, transaction-pooled mode. The test compared a single PgBouncer process against a fleet of 16 processes, all bound to the same port via SO_REUSEPORT with kernel-level load balancing. The fleet also employed peering to forward cancellation requests to the correct process, ensuring proper query cancellation across the group.

PgBouncer is single-threaded per process; a single process can only use one CPU core. Under load, pidstat showed the single process pinned at ~97% CPU on one core while the entire 16-vCPU host remained under 10% utilization. CloudWatch reported 16% CPU utilization for the instance. Throughput peaked at approximately 87,000 transactions per second (TPS) with 64 clients, then degraded to 77,000 TPS at 256 clients as contention for the single core increased.

In contrast, the 16-process fleet scaled throughput to roughly 336,000 TPS at 256 clients—a 4x improvement. The fleet spread work across approximately 8 cores, with in-guest CPU utilization averaging 52% and CloudWatch reporting 60%. The machine was actively used rather than idle. At low concurrency (8 clients), the single process performed slightly better (8,910 TPS vs. 6,450 TPS) because there was no parallelism benefit and the fleet incurred small overhead.

• Single process peak: ~87k TPS at 64 clients, degrading to 77k at 256 clients.
• Fleet peak: ~336k TPS at 256 clients, sustained 320k at 128 clients.
• CPU utilization: single process ~97% on one core, host under 10% (in-guest) or 16% (CloudWatch); fleet ~52% in-guest, 60% CloudWatch.
• Connection budget: max_client_conn and max_db_connections must be divided across processes to avoid oversubscribing Postgres.

SO_REUSEPORT allows multiple processes to bind the same port; the kernel distributes new connections. Peering ensures that cancellation requests arriving on a different process are forwarded to the owning process. The connection budget split prevents any single process from exceeding Postgres limits.

For deployments where the pooler becomes the bottleneck, horizontally scaling PgBouncer across cores using SO_REUSEPORT and peering is recommended. The connection budgets must be adjusted proportionally. This turns the pooler into scalable plumbing rather than a throughput ceiling.

A single-threaded PgBouncer process is constrained by the core it runs on. On a 16-vCPU machine, one process caps throughput long before Postgres exhausts its resources, leaving over 90% of the CPU idle. In ClickHouse Managed Postgres, this bottleneck is eliminated by deploying a fleet of PgBouncer processes scaled to match the available cores. All processes bind the same port using SO_REUSEPORT, which instructs the kernel to distribute incoming connections across the entire fleet. Clients connect to a single endpoint and are unaware of the pooler architecture behind it.

A critical challenge arises with query cancellation. A Postgres cancel request arrives on a new, separate connection. With SO_REUSEPORT, the kernel may route this cancel to a different PgBouncer process than the one managing the original session. The fleet solves this through peering: processes are interconnected, so a cancel received by the wrong process is forwarded to the correct one, preserving standard client cancellation behavior.

The fleet operates in transaction-pooling mode, returning a server connection to the pool immediately after a transaction commits. Connection budgets are divided uniformly across the fleet (e.g., max_client_conn and max_db_connections are split by the process count), preventing any single process or Postgres from being oversubscribed.

Practical evidence from an identical hardware test (16-vCPU c7i.4xlarge, same Postgres and workload driver) shows the performance delta:

• Single process: Peaked at ~87,000 transactions per second, degrading to ~77,000 TPS under 256 clients. CPU utilization remained under ~9% in-guest, leaving nearly 15 cores idle.
• 16-process fleet: Scaled to ~336,000 TPS—roughly a 4x improvement—utilizing ~52% of the box’s capacity (8 cores busy) before Postgres or the load generator became the limiting factor.
• Connection ceiling: A single process enforces its own max_client_conn independently, rejecting new clients once hit. The fleet raises the aggregate ceiling by splitting the budget across processes, allowing higher total concurrency without exceeding safe per-process limits.

The single process performs marginally better at very low concurrency (8 clients), but the fleet dominates under real-world load where that single core becomes the bottleneck.

This fleet-based approach is the default infrastructure in ClickHouse Managed Postgres. Every provisioned server ships with a multi-process PgBouncer deployment, applying SO_REUSEPORT, peering for cancellation, and distributed connection budgets. This ensures that the pooler remains transparent plumbing under load rather than the first capacity constraint.