`Future + Send` Was (Not) Unavoidable

Published: May 9, 2023, 10:02 p.m.

If you've done any async programming in Rust, you will have encountered the problem, where you can't run tokio::spawn, because the future cannot be sent between threads safely. This is a tale about why this problem exists, how it could be solved for good, and why it's not trivial to do without breaking existing code.

Rust Thread Safety

Rust ensures thread safety through 2 really ingenious mechanisms - lifetimes and Send/Sync traits. We won't talk about lifetimes, but we will instead focus on those 2 traits. Here are the core rules surrounding them:

• If T: Send, then you can send T to another thread.
• If T: Sync, then you can have T in multiple threads.
• If &T: Send, then T: Sync.
• Everything is Send, except for pointers and Rc.
• Everything is Sync, except for pointers, Rc, and UnsafeCell.
• If everything in T is Send, then T: Send (if something is !Send, then T: !Send).
• You may manually implement Send and Sync.

Everything is governed by these 7 points. And all you need to do to ensure thread safety is require a Send bound in your arguments, like here:

pub fn spawn<F, T>(f: F) -> JoinHandle<T>where
    F: FnOnce() -> T + Send + 'static,
    T: Send + 'static,

This is incredibly simple yet genius! Where does the problem arise?

Threads Meet Tasks

Rust thread safety hinges upon one assumption - the core unit of execution is a thread. Everything that happens in a thread happens sequentially, and if !Send data is on the thread, it's okay, because we know it's contained within the thread. But what is a thread anyways?

A thread is an OS backed unit of execution. It's how you get CPUs doing multiple things, seemingly at the same time. Everything is actually happening once (or twice, if you count SMT) at a time on a CPU core. However, to build up an illusion of concurrency, the CPU, in conjunction with the OS kernel, are switching those threads around like crazy. Each thread has its own state, its own unique "global" view (to specific processes), and are isolated thanks to the OS and CPU architecture mechanisms. A thread also has per-thread local storage. Great! Now, how do we compare with tasks?

A task in async context is a runtime backed unit of execution. It's how you get threads doing multiple things, seemingly at the same time. Everything is actually happening once at a time on a program's thread. However, to build up an illusion of concurrency, the thread, in conjunction with the async runtime, are switching those tasks around like crazy. Each task has its own state, the same global view to the process, and are isolated merely by compiler safety mechanisms. A task may also have per-task local storage, but it also has access to Thread Local Storage (TLS). Looks fairly similar to a thread, doesn't it? And here's the kicker, I don't have to worry about threads being moved from one CPU core to another! So, why do I have to care about tasks moving from one thread to another?

Why the Send?

We have to remember, that async runtimes still run within the safety rules, that means if entire Future is !Send, then the runtime must stay within the Rust rules, and not move tasks across multiple threads. But, what does make a task !Send?

This results in Send future:

let value = *mutex.lock().await;
let foo = Rc::new(RefCell::new(1));
*foo.borrow_mut() = value;

This results in !Send future:

let foo = Rc::new(RefCell::new(1));
let value = *mutex.lock().await;
*foo.borrow_mut() = value;

The difference is simple - foo is being held across the await point. Very subtle! The idea is that, a multithreaded runtime may move the task to another thread, therefore the only way to ensure that doesn't happen at the wrong time at the typesystem level is to disallow it altogether, therefore the future becomes !Send. This is not good, because it effectively forces you to add Send bounds to your types, and especially in async traits this becomes incredibly verbose and cumbersome to work with. However, it doesn't have to be this way!

AsyncSend - the Lord and Savior

Why don't we extend the core Send/Sync traits with one more - AsyncSend (or TaskSend)? The idea is simple - define AsyncSend as a strict superset of Send, and here are the full rules:

• If T: AsyncSend, then you can send T to another task.
• If T: Send, then T: AsyncSend.
• If T: AsyncSend, then Rc<T>: AsyncSend.
• If everything in T is AsyncSend, then T: AsyncSend.

Then, async runtimes can change their signatures to require AsyncSend, instead of Send, and with slight bit of unsafe, make a promise to the compiler that they hold top level futures and pass them across different threads. Sounds perfect, right?

The Baddie thread_local!

The one thing breaking this idea apart is thread local variables. Consider the following code:

thread_local! {
	pub static FOO: Rc<RefCell<u32>> = Rc::new(RefCell::new(1));
}

async fn do_locked(lock: Mutex<u32>) {
	// Load `FOO` from TLS
	let foo = FOO.with(|foo| foo.clone());
	// Wait for lock, potentially switching threads
	let mut guard = lock.lock().await;
	// Borrow, on another thread already
	*foo.borrow_mut() = *guard;
}

If do_locked was AsyncSend, you could break the safety guarantees, and have 2 of these tasks borrow FOO in different threads. Essentially, thread_local! would wrongfully allow you to leak !Send types across task boundaries. This is a problem, and before going further I strongly encourage you to think of an equivalent to usage of thread_local! in async context, but for regular threads. Take your time... The equivalent would be allowing a thread to access CPU-local data. And here's a handy table showing the equivalents:

Tell me, have you ever thought of accessing per-CPU variables on your program? I guess you can't do that! Okay, if you can't do that, another alternative would be static vars. Does Rust allow you to put !Send types there? No! So does it make sense for an async task to do the equivalent of that? No! So, can we solve it somehow?

Is There Any Hope?

I have a question for you - are you satisfied with the current state of async bounds? If you could break Rust, would you change thread_local! to not work inside async environment? Would you pull the trigger? I doubt we can actually do that, but also, do you think adding where Trait::func(): Send to every async function of the trait is sustainable for the next 10 years? Is there a way around it?

Function Properties

This is a rather complicated idea I have, and it may not be the best solution (I'd actually love to hear your ideas), but it's to have function properties that you can specify - something in-between calling conventions and traits. Here's a hypothetical example:

pub fn func1() !tls + !panic -> u32 {
	// ...
	val
}

fn inferred() -> u32 {
	func1()
}

pub fn func2() !tls -> u32 {
	// ...
	inferred()
}

pub mut FUNC: Option<fn() !tls -> usize> = None;

These properties signify what can and can't happen in a function. If you want to say that the function doesn't panic, you'd add !panic. If you want to say that the function doesn't use TLS, you'd add !tls. These properties would be inferred at the crate level - only public APIs, function pointers, and traits would need to specify this (because changes in properties are breaking from semver perspective). Even then it's optional - these properties are opt-out, meaning if you don't specify them, the compiler assumes the effects can happen. Finally, bridging this to async code, a future that only consists of !tls code and AsyncSend types automatically becomes AsyncSend, and to make things even simpler, one could imply !tls in async traits. This would make async usage very concise, and also trivially buy us an ability to prove lack of panics, which is another topic of interest for many people! In the future, you could even relax the requirements based on the type of TLS var being accessed, in different stages of flexibility:

1. Start with any thread_local! access emitting tls.
2. Do not emit tls, if T: Send.
3. Do not emit tls, if T: !AsyncSend | Send.

Naturally, the third step is very complex to check, so it's quite far off, but the second one could be fairly trivial to specialize!

Conclusion

Rust is a language that got many things right, and it's the reason why many people love it so much! However, with new additions cracks inevitably start to form, and it's not very pleasant to see where async is headed. However, the cracks are small and there ultimately are ways to walk around these problems. The biggest obstacle so far for keeping usability of async code tidy is Thread Local Storage. I propose a possible solution to the problem, in the form of function properties. These properties allow one to prove that a Future is truly thread-independent, which would be instrumental for moving !Send futures across threads.