Member-only story
Build a TCP Connection Pool From Scratch With Go
Here are some basic ideas on how connections are handled between scalable software systems
My entry task as a junior software engineer was to implement a user registration system with Go.
Sounds simple, right? But, there are performance requirements and design constraints.
Firstly, I am required to place all business logic in a TCP server, and have a client-facing HTTP server interact with it via network connections.
On top of that, I am not allowed to use any Remote Procedure Call (RPC) frameworks like Go’s net/rpc or Google's gRPC. I need to create my own logic to handle connections between the two servers.
Secondly, the entire system is expected to support 1,000 concurrent HTTP requests and 3,000 login requests per second without cache. There need to be 10 million existing users in the database as well.
The key to cracking this entry task is to properly manage the network connections between the HTTP and TCP server. With too few available connections, you’ll end up with slow requests. If too many, your servers will struggle with connectivity problems.
In this article, I will share with you how I built a custom connection pool to achieve the right balance. I will explain the design overview with code snippets. Let’s get started! 🏃♂️
What is a Connection Pool?
A connection pool is a set of cached connections in a server that can be used again in the future.
Usually, opening a new network connection is expensive. Think TCP connection as an example, where the client and server need to perform three-way handshakes. It is an expensive operation that consumes data bandwidth and causes round-trip delays.
With pooling, instead of closing a connection after it is used, we can keep it idle while it waits to handle another request. This saves the resources required to create a new connection.
