Implementing Multi-Threaded Shared Memory in Rust

Making shared data design pattern work

Michael Hentges
Better Programming
6 min readOct 31, 2022

--

This article is the third in a series related to building a wireless thermostat in Rust, running on a Raspberry Pi — although this one has very little to do with the Raspberry Pi and is more relevant to any multi-threaded application. While creating my application, I implemented a simple and effective design for controlling multi-threaded access to shared memory. You can find the previous articles here:

I also have a few code snippets below, but you can find the entire code repository on GitHub.

First, the Backstory

When designing a multi-threaded application, one of the core considerations is how the different threads share data. There’s always something that the threads share — you’re spinning off threads to work on a shared problem, after all. If nothing else, configuration data or database connection pools have to be shared. There are two main ways of approaching this problem:

  1. Message Passing — channels between threads allow data to be sent and received between them. Message passing is the mechanism that Golang prefers, as the Golang documentation states: “Do not communicate by sharing memory; instead, share memory by communicating.” Rust supports creating channels; you can find more information in the Rust Book’s chapter on message passing.
  2. Shared-State — a set of data identified as a shared state. Each thread access a shared data area through thread-safe protection mechanisms — typically a Mutex. Locks on the data allow you to access the data safely, reading or updating the data as needed, and then the lock is released. Only one thread can read or write the data at a time. The Rust Book’s shared state section describes these mechanisms.

Both approaches work, and Rust has good support for both mechanisms in the standard library. To choose between them, consider the following:

  1. What are the access patterns? Will threads mainly read data, mostly write data, or both?
  2. How will deadlock be avoided? Avoiding deadlock is a primary concern for multithreaded applications — nothing is worse than having a bunch of threads available to do work, and they are all stuck waiting for the other ones to get out of the way.
  3. Performance. While over-engineering for performance is a common fault, having some idea of the performance requirements of your solution is essential. I’m a big fan of the rule: keep the solution as simple as possible, but no simpler.

I decided on a shared state approach for the application I am creating. This decision also implied that my worker threads would work by polling — checking on things periodically instead of receiving an outside message. Here is what led me to that conclusion:

  1. I have three main threads. One is a web interface that allows for reading/writing of the shared state (get temperature, get thermostat setting, set thermostat setting). The web interface is, by definition, a creator of unscheduled activity — things come in based on an external client’s action and are not under the application’s control.
    Second, a pair of background worker threads react to the application’s environment. The first reads the temperature sensor, and the second calculates whether the thermostat should be set on or off and controls the physical relay to make that happen. The thermostat is in a separate thread, as it could turn on/off when the temperature crosses a threshold or we receive a new thermostat setting from the web interface.
  2. I didn’t have to worry about contention with only three threads (assuming a single external client on the web interface). A simple shared state approach wouldn’t run into issues based on too many threads trying to access locks simultaneously.
  3. The logic for determining the thermostat’s state has a time component. We don’t want frequent, short bursts of on/off, which thrash the furnace. We enforce a minimum amount of time that the thermostat will be on or off before switching to another state. Accurately reacting to a message (the temperature reading or thermostat value changes) requires knowing how long it’s been since the thermostat changed.

A shared state made sense based on these requirements. My design enforces a low thread count, so contention was not an issue (a common problem with shared state approaches). I need to keep track of time, so some state data was always required. Implementing two different state data mechanisms in the same application would be added complexity. With each thread doing its own thing and getting/setting shared state independently, we isolate each thread from the others. This choice led to a more straightforward overall application design.

But, as I reviewed the Rust Book’s content on shared-state,
the complexity of managing access from each thread seemed daunting. Plus, sprinkling thread-locking code in the application made it a mess — my nice single-purpose functions now had Mutex locking logic. Also, when accessing data through a Mutex lock dealing with Rust’s borrow checker across threads proved challenging.

A Simpler Approach

To make this manageable, I used an approach I’ve used before on C++ projects — encapsulating the shared data into a separate class and pushing all Mutex logic into the class. Rust doesn’t have classes, but some structs did the trick.

I created a struct to hold all the shared state data in a single place. Then I could deal with a single reference to this struct instead of managing each data element independently.

Then we define a struct that holds an Arc pointer — an Atomic Reference Count pointer to a Mutex that guards our shared data struct. We use this struct to control access to/from our shared data.

We will make many copies of this pointer — they all point to our Mutex, which is how we gain access to our shared memory space. We do this by customizing the Clone() method for AccessSharedData — it looks like the following:

To use this, we first create an instance of our SharedData struct (the simple .new() method isn’t shown above but is straightforward).

Then, we initialize an instance of AccessSharedData.

We then give each thread we spawn() a cloned copy of the AccessSharedData struct. The clone call creates a copy of the Arc pointer, which we then move into the new thread. A similar method passes in a clone of the AccessSharedData struct to the actix_web HttpServer::new() method, so it is also available in HTTP client handlers.

Now that everyone has a copy of the Arc pointer, we need to create a means to use it to read/write our shared data struct. A simple set of getters/setters for each member of our struct handles the task of acquiring a lock, getting/setting the data, and automatically releasing the lock. By putting this logic surrounding the get/set and utilizing the end of the method as a scope boundary to force the lock to be released, we gain control over the access to our data.

When each get/set function returns, the lock is released. This approach also means it is impossible to deadlock around access to the structure — everything is locked and released immediately and prohibits anything else from getting in the way. With all of our locking mechanisms in place, using the shared data in the rest of our application is trivial.

In Summary

This strategy is encapsulation at its finest. Application code uses straightforward calls to methods to get/set the shared data elements whenever needed. Behind the scenes, we manage the Mutex’s locking for that access to be thread-safe and safe from deadlocks. And Rust’s borrow checker is extremely happy that we aren’t allowing direct shared mutable references to our shared data. You could easily extend this design pattern to implement a more robust access pattern.

Putting locking logic inside getters/setters enables our shared data struct to hide the complexities of multi-threaded access — much like the standard library does for thread-safe collections. Using clones of an Arc makes it possible to acquire a handle to the shared memory struct in each thread. These two concepts are the core ideas that make our easy shared data design pattern work.

Sign up to discover human stories that deepen your understanding of the world.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

No responses yet

What are your thoughts?