Member-only story
Kafka Acks Explained
Visualizing Kafka’s most misunderstood configuration setting
Having worked with Kafka for almost two years now, there are two configs whose interaction I’ve seen to be ubiquitously confused.
Those two configs are acks and min.insync.replicas— and how they interplay with each other.
This piece aims to be a handy reference which clears the confusion through the help of some illustrations.
Replication
To best understand these configs, it’s useful to remind ourselves of Kafka’s replication protocol.
I’m assuming you’re already familiar with Kafka — if you aren’t, feel free to check out my “Thorough Introduction to Apache Kafka” article.
For each partition, there exists one leader broker and n follower brokers.
The config which controls how many such brokers (1 + N) exist is replication.factor. That’s the total amount of times the data inside a single partition is replicated across the cluster.
The default and typical recommendation is three.
Producer clients only write to the leader broker — the followers asynchronously replicate the data. Now, because of the messy world of distributed systems, we need a way to tell whether these followers are managing to keep up with the leader — do they have the latest data written to the leader?
In-sync replicas
An in-sync replica (ISR) is a broker that has the latest data for a given partition. A leader is always an in-sync replica. A follower is an in-sync replica only if it has fully caught up to the partition it’s following. In other words, it can’t be behind on the latest records for a given partition.
If a follower broker falls behind the latest data for a partition, we no longer count it as an in-sync replica.
