I Built a Snappy Static Full-text Search With WebAssembly, Rust, Next.js, and Xor Filters

Fast, lightweight, and tons of fun while building!

TL;DR

• 🦀 There is a rich toolkit for developing WebAssembly with Rust. It’s fun!
• 🤝 WebAssembly and Next.js play fairly well together, but be aware of the known issues.
• 🧑‍🔬 Xor filters are data structures that provides great memory efficiency and fast lookup for value existence.
• 🧑‍🍳 WebAssembly’s performance and code size is not guaranteed. Make sure to measure and benchmark.

I always knew I wanted a full-text search feature for the articles in my portfolio to provide the visitors with quick access to the content they are interested in. After migrating to Contentlayer, it doesn’t seem to be so far-fetched anymore. So I started exploring.

Inspired by tinysearch: A WebAssembly Full-text Search Engine

After some researching, I found a search engine called tinysearch. It's a static search engine built with Rust and WebAssembly (Wasm). The author Matthias Endler wrote an amazing blog post about how tinysearch came about.

I loved the idea of constructing a minimalistic search engine at build time and shipping it in optimized low-level code to the browsers. So I decided to use tinysearch as the blueprint and write my own search engine to integrate with my Next.js static site.

I highly recommend reading tinysearch's codebase. It's very well-written. My search engine's implementation is a simplified version of it. The core logic is the same.

What Does The Search Feature Looks Like?

Very simple:

• Users type anything in the search input.
• Search engine searches the key words in all the contents and finds the most relevant articles.
• UI displays a ranked search result list.