πScaling for High Concurrency
When thousands of concurrent users hit a Next.js app, bottlenecks appear roughly in this order: (1) DB connection pool exhaustion, (2) server memory, (3) API response time / N+1 queries, (4) edge bandwidth, (5) real-time/websocket limits. Adding servers doesn't help if the database is the wall.
13.1Database (usually the first wall)
- Connection pooling. Serverless functions each open DB connections β pool exhaustion fast. Use PgBouncer or Prisma's pooling / Accelerate. Keep a single global client instance (don't
new PrismaClient()per request):
const globalForPrisma = globalThis;
export const prisma = globalForPrisma.prisma ?? new PrismaClient();
if (process.env.NODE_ENV !== 'production') globalForPrisma.prisma = prisma;
- Kill N+1 queries. Use eager loading (
include/select) instead of a query per relation. Fetch related data in one round trip. - Select only needed fields, and
takea limit β don't over-fetch. - Index strategically. For multi-tenant B2B: index the tenant key (
team_id), status/filter columns, and time columns for range queries. Composite indexes with the most selective column first. Full-text (GIN) indexes for search. - Read replicas. Route heavy reads to replicas, writes to the primary.
- Consider sharding/partitioning only at very large scale (10M+ rows) β by tenant id, or time-partitioning for append-heavy tables.
13.2Caching (multi-layer, recap from Part 5)
Edge/CDN (static, long TTL) β Redis (hot data, short TTL, invalidate on write) β client (react-query). Each layer absorbs load so fewer requests reach the DB.
13.3API design for scale
Cursor-based pagination; partial responses (field selection); request batching; rate limiting to protect the backend; idempotency keys for safe retries on writes.
13.4Real-time at scale
- WebSocket needs sticky sessions (load balancer routes a user to the same instance) β via
ip_hash(Nginx) orsessionAffinity: ClientIP(K8s). Persistent connections can't be freely load-balanced per-message. - Server-Sent Events (SSE) are simpler than WebSockets when you only need serverβclient push (notifications, live updates) and don't need clientβserver over the same channel.
- Batch outbound messages to avoid flooding clients.
- Beyond ~10k concurrent connections, introduce a message broker (Redis Pub/Sub, Kafka, RabbitMQ) or a dedicated realtime service (managed Pusher/Ably, or a Socket.IO cluster with a Redis adapter) so instances can broadcast to each other.
13.5Deployment & scaling
- Horizontal scaling β multiple stateless instances behind a load balancer with health checks.
- Auto-scaling on CPU/memory (K8s HPA), with sensible min/max replicas.
- Multi-region for global latency; route users to the nearest region.
- Managed (Vercel) vs self-hosted (Docker + K8s) β Vercel handles edge/scaling for you; K8s gives control and can be cheaper at scale but you own the ops.
13.6Observability (non-negotiable at scale)
Track latency percentiles (p50/p95/p99, not just averages β averages hide tail pain), error rates, DB pool utilization, cache hit rates, active connections. Tools: OpenTelemetry, Sentry (errors + tracing), Datadog/Grafana. Set alert thresholds and, where possible, auto-rollback on error spikes.
π§ Quiz: Scaling for High Concurrency
1 question1When scaling for high concurrency, which layer is usually the first bottleneck?