Writing High-performance TCP Applications Using the Gain Web Framework
This article introduces a high-performance web framework, Gain, that can be faster than the fastest epoll-based frameworks written in Go since it is built on top of the io_uring API and was designed from the very beginning to be as fast as possible. The article covers a little bit of the basics necessary to use the framework correctly and demonstrates how to write a simple TCP application built on top of it.
What is io_uring
io_uring is a relatively new Linux API developed to handle I/O operations. For performance reasons, it uses an asynchronous programming model.
io_uring is a pair of ring buffers shared by user space and the kernel. The first is used to make requests (submission queue or SQ) and the second is to report the result of their execution (completion queue or CQ). When a user wants to perform an operation, such as reading or writing to a file, it creates a submission queue entry that describes the requested operation and adds it to the tail of the submission queue.
The kernel is then notified using a system call that there are requests in the queue to be processed. The kernel tries to do all the work it has to do and puts the result on the tail of the completion queue under the form of completion queue events.
The user reads them one by one starting from the head of the queue. To reduce the number of system calls, you can add multiple requests before notifying the kernel.
io_uring higher-level abstraction
While to use io_uring you can use a low-level interface it would be very painful in real applications, so a higher-level abstraction layer is needed. There is an official library that provides this — liburing. It is written in the C programming language and is being actively developed all the time.
When creating an application or, as in my case, a library in the Go programming language, we will most likely encounter a performance problem related to the high cost of calling C language functions in Go using the cgo mechanism.
I cared a lot about achieving the highest possible performance, so I decided to write my own abstraction layer for the lower-level io_uring interface. This required a lot of additional work, of course, but it gave me a much better understanding of the mechanisms of the new API. I encourage anyone interested to take a look at the source code published on GitHub.
I tried to make the API of my io_uring abstraction very similar to liburing so that anyone familiar with the official version in C could use io_uring in Go as painlessly as possible. As part of the implementation, I used the syscall, sys/unix, and unsafe packages, which, for example, have many mechanisms to support working directly with the operating system or allow manual allocation of memory that is not managed later by the garbage collector.
Programming model
To make the most of the biggest advantage of io_uring, which is high performance, I had to use a programming model that does not generate a large performance overhead while implementing my library.
Go developers are probably most used to using an abstraction like the net package. The developers of the language have done a good job in designing it so that we get a fairly simple and clear interface that allows us to work with TCP, UDP, and Unix domain sockets protocols. Unfortunately, moving it one-to-one to a library using io_uring would cost too much in terms of performance.
The basic methods for controlling connections are the Read, Write, and Close methods of the net.Conn interface. If we analyze the source code then we will notice that, for example, every attempt to read data involves a system call. However, one of the main objectives of io_uring is to minimize their number. Therefore, the gain.Conn interface has Read, Write, and Close methods, but their behavior is different, and none of them is blocking.
The Read method reads data that Gain has already read from a file descriptor and placed it in the connection’s input buffer. The Write method writes data to the output buffer, which Gain will send asynchronously after taking control of the main loop. The Close method will mark the connection as closed and block the ability to use the connection’s methods, but the actual closing will also happen asynchronously.
If anyone has dealt with the gnet library (if not, I encourage you to familiarize yourself with it), you will probably quickly notice that the main API of the Gain library is very similar. It is based primarily on the EventHandler interface and its methods.
To better understand how to build an application using the Gain framework we will now write a simple application using the TCP protocol. Let’s not make it anything fancy — a simple echo server that will respond to the client with the same data packet it received from it in the request
So let’s write a structure that implements the EventHandler interface.
type EventHandler struct {
server gain.Server
logger zerolog.Logger
overallBytesSent atomic.Uint64
}OnStart
This method is called as soon as the server starts. We can use it to initialize and configure an application written on Gain.
For example, to initialize the logger or start additional tasks in separate goroutines. In our implementation, we will also use it to save a reference to the server.
func (e *EventHandler) OnStart(server gain.Server) {
e.server = server
e.logger = zerolog.New(os.Stdout).With().Logger().Level(zerolog.InfoLevel)
}OnAccept
The OnAccept method is called when the connection is accepted. You should be aware that this method will not work for UDP connections because UDP is a connectionless protocol, so there is no connection-accepting operation in the data transfer flow.
While this method is running, the connection will not yet have data in the input buffer, so trying to read is pointless, but you can already try to send data over the connection (useful for server-first protocol, e.g. TIME).
It is worth remembering that neither this nor the other methods should be blocked for a significant amount of time (with one exception) because they are called in the main loop and will block the processing of io_uring requests.
func (e *EventHandler) OnAccept(conn gain.Conn) {
e.logger.Info().
Int("active connections", e.server.ActiveConnections()).
Str("remote address", conn.RemoteAddr().String()).
Msg("New connection accepted")
}OnRead
The method is called as soon as the bytes are read from the file descriptor. Data can be read by parameter methods. Most often, this will be the method that will contain the most business logic of the application such as a response to a client request. If actions are required that may block the operation of this method for an extended period such as expensive operational I/O then consider configuring Gain so that the method is called in a separate goroutine each time (using a goroutine pool or unbound goroutine).
In our protocol implementation, we read the data sent by the client and send it back.
func (e *EventHandler) OnRead(conn gain.Conn, n int) {
e.logger.Info().
Int("bytes", n).
Str("remote address", conn.RemoteAddr().String()).
Msg("Bytes received from remote peer")
var (
err error
buffer []byte
)
buffer, err = conn.Next(n)
if err != nil {
return
}
_, _ = conn.Write(buffer)
}OnWrite
Fired immediately after writing bytes to the file descriptor. Useful, for example, when you want to log or count the statistics of sent data.
For the purpose of our simple application, let’s assume that after sending data to the client, the connection to the client should be closed.
func (e *EventHandler) OnWrite(conn gain.Conn, n int) {
e.overallBytesSent.Add(uint64(n))
e.logger.Info().
Int("bytes", n).
Str("remote address", conn.RemoteAddr().String()).
Msg("Bytes sent to remote peer")
err := conn.Close()
if err != nil {
e.logger.Error().Err(err).Msg("Error during connection close")
}
}OnClose
Fired as soon as the TCP connection is closed. The second parameter is useful to determine the reason for closing. If the error will be nil, it means that the connection was closed by the server, and was initiated by the application. Other values will mean the connection was terminated by the client or a network error.
Let’s add the condition that after handling enough traffic, our server should shut down.
func (e *EventHandler) OnClose(conn gain.Conn, err error) {
log := e.logger.Info().
Str("remote address", conn.RemoteAddr().String())
if err != nil {
log.Err(err).Msg("Connection from remote peer closed")
} else {
log.Msg("Connection from remote peer closed by server")
}
if e.overallBytesSent.Load() >= uint64(len(testData)*numberOfClients) {
e.server.AsyncShutdown()
}
}Starting the application
We already have our EventHandler implementation ready, so it remains to run our server. This time we won’t dive into the configuration options. To start the Gain server we will use the gain.ListenAndServe method.
Its first parameter is the address where we want our server to listen. For the purposes of our application, the localhost interface will suffice.
The suffix before the address determines what protocol Gain should use. Currently, TCP and UDP protocols are supported. The next parameter is our EventHandler implementation, and the rest of the parameters are optional configurations.
Let’s use them to set the logging level for Gain’s internal logger.
func main() {
runClients()
err := gain.ListenAndServe(
fmt.Sprintf("tcp://localhost:%d", port), &EventHandler{}, gain.WithLoggerLevel(logger.WarnLevel))
if err != nil {
log.Panic(err)
}
}Client-side logic
To test our server, let’s write some simple code for the clients of our protocol. First, let’s wait a while for the server to start up, then connect to it and send test data. Let’s check if the operation was successful, and if so, let’s try to read the server response and verify if it is what we expect.
func runClients() {
for i := 0; i < numberOfClients; i++ {
go func() {
time.Sleep(time.Second)
conn, err := net.DialTimeout("tcp", fmt.Sprintf("localhost:%d", port), time.Second)
if err != nil {
log.Panic(err)
}
n, err := conn.Write(testData)
if err != nil {
log.Panic()
}
if n != len(testData) {
log.Panic()
}
buffer := make([]byte, len(testData))
n, err = conn.Read(buffer)
if err != nil {
log.Panic()
}
if n != len(testData) {
log.Panic()
}
}()
}
}Finally, let’s build and launch our application:
{"level":"info","active connections":2,"remote address":"127.0.0.1:47074","message":"New connection accepted"}
{"level":"info","active connections":2,"remote address":"127.0.0.1:47078","message":"New connection accepted"}
{"level":"info","bytes":4,"remote address":"127.0.0.1:47074","message":"Bytes received from remote peer"}
{"level":"info","bytes":4,"remote address":"127.0.0.1:47078","message":"Bytes received from remote peer"}
{"level":"info","bytes":4,"remote address":"127.0.0.1:47074","message":"Bytes sent to remote peer"}
{"level":"info","bytes":4,"remote address":"127.0.0.1:47078","message":"Bytes sent to remote peer"}
{"level":"info","remote address":"127.0.0.1:47074","message":"Connection from remote peer closed by server"}
{"level":"info","remote address":"127.0.0.1:47078","message":"Connection from remote peer closed by server"}
{"level":"warn","component":"consumer","worker index":0,"ring fd":8,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":1,"ring fd":9,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":2,"ring fd":10,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":3,"ring fd":11,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":4,"ring fd":12,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":5,"ring fd":13,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":6,"ring fd":14,"time":1683456909,"message":"Closing connections"}
{"level":"warn","component":"consumer","worker index":7,"ring fd":15,"time":1683456909,"message":"Closing connections"}Try it yourself
I strongly encourage you to try out Gain (and give it a star if you like it). I would also appreciate any feedback. I would like to point out that Gain is not yet in a stable version, so its API may undergo major changes to meet as many requirements of the Go developers community as possible. Therefore, I also encourage you to discuss whether the current API is clear enough and whether it is sufficient to build any TCP or UDP applications based on Gain.
