Member-only story
Distributed Locks With Apache Pulsar
Save time on complex computations
One of the most common challenges that we software engineers have sometimes is how to ensure that only one component in our distributed application is doing a certain computation at a time.
For example, let’s say we are running three nodes in our application and we need to run a scheduled job daily. How can we make sure that only one of the nodes triggers the job? If we were sending an email to customers in this job and the three nodes trigger the task, our customers could receive this email three times! We don’t want that, so what can we actually do?
Some people might say: “Let’s run only one node! Easy!”
Well, not that easy. In most cases we must ensure an adequate level of availability for our service, running just one node would mean that our service would be unavailable if it has a problem.
What we actually need is a way to select some sort of “master node” responsible for this task. Another important aspect to consider is that if our master node fails, this responsibility has to be delegated to one of the secondary nodes immediately to avoid disruptions.
Let’s take a look at what we want to achieve visually to see it clearly.
What we need is a simple way to “elect” one leader who will be responsible for the task, the rest of the nodes will patiently wait for their turn in case their help is needed. These nodes would be in a so-called “dormant” state; they’ll only be awakened if the leader node is lost or becomes unresponsive.
How Could We Solve This Problem?
In some cases, people would decide to implement some fairly complex implementation to make sure that only one of them performs the task.
Some database engines nowadays support a sort of compare&set operation atomically, which could be a reasonable way to solve this…
