Scaling Node.js Applications With Multiprocessing
You might be underutilizing your multi-core server environment without even knowing
Node is a single-threaded JavaScript runtime environment. Being single-threaded has many perks, and it has made Node the go-to back-end development environment for many. However, being single-threaded has its own downsides. With Node, you only have one thread, the main thread, and you have to make sure that it is always available to serve the incoming requests. This is not as easy as it sounds, especially if your application has to serve a larger number of requests or if your application has to perform CPU-intensive tasks at some point. Let’s see how we can tackle this with multiprocessing.
What Is Multiprocessing?
Multiprocessing is a server-side scaling technique in Node that allows you to have more than one Node instance running, ideally one for each of your processor cores. With multiple Node instances, you have multiple main threads, and therefore even if one of the threads becomes occupied or crashes, you still have other threads to serve the incoming requests. That way, your application doesn't look like the following.
You can implement multiprocessing using Node’s built-in cluster module or its child_process module.
Handling CPU-Intensive Tasks
If you’ve been using Node for a while, you must’ve already heard that Node is not a good choice if your application has CPU-intensive tasks. This is because if you run a CPU intensive task that takes a considerably long time to complete, Node will have its single main thread blocked and it will not be receptive to new requests until the task is completed.
The Node child_process module is a good workaround for this issue. It’s ideal when you want to have multiple Node instances following different execution flows to complete a single common task. It allows you to fork a new Node instance (a child process), ideally in another CPU core, and offload the heavy task to the newly forked child process. This keeps the main thread of the parent process free for new requests, and the results of the offloaded task can be communicated back to the parent process once the task is completed.
Let’s see how we can demonstrate this with some code. First I wrote an algorithm to check, in the least efficient way (to mimic a CPU-intensive task), whether a given number is a prime number or not.
Next I coded a simple Node server to execute the above function using a newly forked child process once the relevant API endpoint is called.
Then I benchmarked the server using autocannon by sending eight requests using eight concurrent connections.
As you can see, it only took the server 1.02 seconds to serve the requests. Finally, I coded another server without using multiprocessing and ran the same benchmark on it.
This time it took the server over 74 seconds to serve the same number of requests. That’s a huge drop in performance when compared to the previous implementation with multiprocessing.
There are certain time-consuming tasks, such as I/O operations, network operations, and cryptographic functionalities, that Node manages to execute in an asynchronous non-blocking manner without needing a manual implementation of multiprocessing. To learn more about how Node handles asynchronous tasks, check out my previous article.
Handling a Larger Number of Requests
Being single-threaded can become a limitation when serving a higher number of requests. As the number of requests grows, your server's response time can get longer because all the requests are served using only one thread.
The Node cluster module is a solution to this issue. It allows you to have multiple Node instances following the same execution flow, listening on the same port, which makes it ideal when handling a larger number of requests. Let’s write some code and see how it works.
In clustering, there’s one main process called the master process, and all it does in this application is fork the rest of the worker processes, one for each available CPU core. These worker processes listen for requests on the same port, and the cluster module’s embedded load balancer distributes the requests among the worker processes when they are under high load.
I executed the code above, and to measure if there’s any performance improvement achieved by clustering, I ran a benchmark on the server simulating 100,000 requests coming from eight concurrent connections.
It took the server 6.02 seconds to server 100,000 requests with clustering. Then I implemented another similar server but without clustering and ran the same benchmark on it.
This time it took the server more than 12 seconds to serve the same number of requests. This is a significant drop in performance compared with the previous implementation with clustering, and it can become even more significant in real-world use cases.
Cons of Multiprocessing
The performance boost of multiprocessing comes at a cost. Implementing multiprocessing means running multiple Node instances, and that can consume a lot of memory. Therefore you should always ensure that your server environment can tolerate the memory usage spikes that can occur with multiprocessing.
Multiprocessing, no matter which Node module you use, might require inter-process communication in certain applications, and it can get a little complex to maintain in the long run.
Conclusion
Multiprocessing can be easily implemented using Node’s built-in modules, and it allows you to make the best use of the available server environment. With multiprocessing, you can boost your server's performance when executing CPU-intensive tasks or serving a larger number of requests and improve server availability.
Thanks for reading!
