Member-only story
Design Pattern of Streaming Enrichment
Understanding the concept of streaming architecture
Last time, we introduced streaming processing. In order to be able to handle batch and real-time data with a more pure infrastructure, we introduced Kafka and the streaming framework.
In this article, we will introduce a common design pattern for streaming, enrichment, and examine what benefits the streaming framework can bring.
What is enrichment? Briefly, it is the implementation of extending the original events to meet the new feature requirements.
Feature Description
Similar features are available on many social media platforms.
- Who visits me
- Others are also viewing
- What is the identity to view me
The first feature is how many people have viewed my profile in a given time period. For example, how many times my profile has been viewed in a week.
The second feature is an advanced version of the previous feature, who else did these people view? For example, 10 people viewed my profile in a week, and they also viewed Elon Musk.
The third feature is a derivative of the first one. What is the identity of these people who viewed…
