More connections did not fix the slow page

A hypothetical page makes many requests. HTTP/2 improves it on a clean network, but one lost packet still delays unrelated responses. Meanwhile, internal gRPC traffic overloads one backend while its peers sit idle.

Both systems multiplex requests. They are blocked at different boundaries.

Follow the transport beneath the request

HTTP/1.1 commonly needs multiple connections for concurrent work; pipelined responses must preserve order. HTTP/2 multiplexes streams with binary framing and HPACK header compression, but TCP delivers one ordered byte stream. A missing TCP segment can delay bytes belonging to several HTTP streams.

HTTP/3 maps HTTP onto QUIC, with QPACK for headers. QUIC integrates transport security and can support connection migration; it still needs path validation and application recovery for a dropped session. 0-RTT resumption raises replay considerations, so do not accept unsafe business effects merely because the connection is fast. These are protocol properties, not a universal latency benchmark. HTTP/3 specification.

One connection can select one backend

An L4 balancer usually chooses a backend for a connection. A long-lived HTTP/2 gRPC channel can then carry many RPCs to that one backend. Add client-side endpoint-aware balancing or an L7 proxy when per-request distribution is needed. Streaming RPCs remain long-lived work with their own placement constraints.

Use connection reuse to avoid repeated setup costs, but budget concurrent streams, flow control, buffers, and dependency timeouts. A single “maximum connections” formula ignores HTTP/2 multiplexing. Measure active requests and queueing separately from sockets.

The gateway earns a shared responsibility

A reverse proxy routes on behalf of backend services. An API gateway often adds authentication checks, admission policies, request shaping, and observability. L4 routing uses transport information; L7 routing can inspect HTTP methods, paths, and headers. More inspection costs resources but enables more policy.

Synthesizing vector architecture diagram...

The gateway must propagate trace context and deadlines, maintain isolated upstream budgets, and define which forwarded identity/client-IP headers are trusted. Public clients must not be able to forge the internal headers your limiter or authorization code trusts. Services still enforce resource-level authorization; gateway authentication is not permission to access every object.

Keep cross-cutting policy coherent without turning the gateway into the database-owning business application. Prevent bypass paths to internal services. Cache verification keys responsibly, refresh unknown key IDs with rate limits, and avoid synchronously querying the identity provider on every request.

Test the complete path

A browser may reach the edge over HTTP/3 while the edge reaches origin over HTTP/1.1 or HTTP/2. Do not label the whole path by the first hop. Test UDP-blocked networks and TCP fallback, protocol negotiation, TLS errors, proxy buffering, backend status translation, and gRPC status codes. Preserve end-to-end request identity so a user-visible failure can be followed across hops.

Mental model: a request crosses several independent queues and trust boundaries. Optimize the one evidence identifies.

Transfer: the slow upload

Source trail: 14-api-gateway-and-reverse-proxy, 19-api-protocols-http1-http2-http3-quic-grpc. See the source map for consolidation and reference-retained material.