NullDeref

Releasing SpeedBump on the Play Store

2025-04-08T00:00:00Z

Lessons on Mobile App Promotion I've Learned the Hard Way

2025-04-02T00:00:00Z

Screen Time App Opal Jokes about Being Bought by Meta

2025-04-01T00:00:00Z

How to Reduce Screen Time: The Ultimate Guide

2025-03-07T00:00:00Z

The German language broke my website

2025-03-04T00:00:00Z

My app sucked (but it's okay)

2025-02-26T00:00:00Z

I blasted over a speed bump to create my landing page

2025-02-14T00:00:00Z

New year, new silly app

2025-01-20T00:00:00Z

So far, the year is off to a great start. New Year’s resolutions aren’t my thing, but this time I’ve made an exception. Not so much with specific objectives but rather broad goals. The biggest one is to be more mindful. I want to observe more, reflect, and act:

Sport does me so much good, but in 2024, planning it was a mess. Maybe a swimming club will make it easy to go regularly, for fun. Having a strict schedule also helps keep my self-employed life under control.
Nutrition is crucial if I do more sport. No need to obsess over this, just being more mindful helps.
Reading taught me a lot in 2024. My flat now has a shelf of never-ending unread books for motivation.
The best way to maintain good mental health is to be surrounded by great people. Caring for the valuable relationships makes a big difference.

Such conclusions can only be achieved by sitting down and thinking. I’m both changing my mindset and incorporating new routines. Just keeping in mind the fact that I need to be mindful already has an effect. It’s easy to be busy all the time, but my brain now sometimes tells me to stop for a second. Additionally, things like “morning pages”, meditation and simple prioritization help.

I’m also happy to have quit my job. It’s been quite chaotic, but every step back has been a lesson. Even though I’m still at the start line, my understanding of the problem is more clear now:

Working with so much ambiguity is tough. But my ideas are now more viable and concrete. I don’t want to build either a new phone or yet another app-blocking app with a bazillion features.
Finding a cofounder is equally challenging. Meeting so many people has taught me what a good match looks like. And it’s great to have the contact of such cool individuals for other reasons.

Surprisingly, it’s less stressful. I’m definitely speaking too early, given that my user count is 0. But having control over my schedule gives me peace of mind. The routine feels similar because I still have to do things I don’t like. But the product I’m striving towards is fulfilling, and that makes up for it.

Before committing to a bigger idea, I’m building a silly Android app. “SpeedBump” makes it annoying to use social media and other apps. After a few minutes, cookie banners, rain effects, cats, and more will get in your way. It’s as stupid as it sounds, but also fun. Google now requires developers to find 12 people to test the app, so please opt in here to help me out!

More serious stuff is coming soon, anyhow. I hope SpeedBump isn’t the peak of my career.

I've quit my job to get you off your phone

2024-10-14T00:00:00Z

I never expected quitting a job to be so hard! My time at Lyft has been an enriching journey. Tons of smart people at three very different teams, where I’ve learned how the industry works. The technical challenges were endless: from user-facing features in the app to improving the safety of our routing.

But after two years, I’ve realized that it just wasn’t making me happy. My disconnect with the product was too big, and so much remote work didn’t help, either. The desire to explore grew, with the hope that something else could bring more fulfillment.

I’ve always aspired to build a company, and this was a good moment in life to try. Now or never. So here I am, soon to be jobless, contemplating life, yet excited about what’s next.

My company might fail (in fact, the odds are against me). But I’ll have tried and learned! This decision reassures me I’m making the best out of life, and it frees me of regrets. Mostly. It’s still pretty insane that I quit.

It’s early in my journey, but I can’t stop exploring my area of interest: phones. Or rather, phonesn’t. Not having had the best year of my life, I found myself on the phone too much. The negative effects became obvious, and I want to do something about it.

This space is quite a challenge. Everyone overuses their phone in their own way. You have big companies optimizing their app to get you stuck on it. And often, overuse is a symptom of a deeper problem, where you’d be better off directly fixing that.

Yet, it’s such a huge issue that I’m confident I can build something valuable. You don’t need to be completely addicted to your phone to want to use it less. Among the thousand possible ideas — even manufacturing your own phone —, I might get started with an innovative app to control screentime. It’s frustrating that no existing apps have gone mainstream. Let’s make it fun to reduce your usage.

Loading…

If this is interesting to you in any way, please reach out! Schedule a chat with me or send me a message on LinkedIn, X, Mastodon, or by email. You can vent about how much your phone controls you, share someone you know, or we could work together. Even catching up is a good reason :)

I feel extremely lucky to be in a position to make this decision. Thank you to everyone for your support!

Blindsided by Rust's Subtyping and Variance

2024-09-14T00:00:00Z

Table of Contents

Subtyping and variance is a concept that works in the background, making your life easier without you knowing about it. That is, until it starts making your life harder instead. It’s a good idea to know about it, in case you end up being a fool like me. So let’s take a look at what went wrong, and how it was resolved.

The problem

As part of my Plugin System in Rust series, I was making one of Tremor’s types FFI-compatible. Put simply, instead of using types from the standard library like String, we wanted custom types defined with #[repr(C)]. The crate abi_stable exists for this exact purpose, with an equivalent for the most important types. Theoretically, the task should be as easy as changing the std types in our core enum with theirs:

// Before (simplified)
pub enum Value {
    String(String),
    Object(Box<HashMap<String, Value>>),
    Bytes(Vec<u8>),
}

// After (simplified)
use abi_stable::std_types::{RString, RVec, RBox, RHashMap};

#[repr(C)]
pub enum Value {
    String(RString),
    Object(RBox<RHashMap<RString, Value>>),
    Bytes(RVec<u8>),
}

The Value type is used a lot in the codebase, so this breaking change brought up lots of compilation errors. But for whatever reason, 70 of these errors were related to lifetimes, which I hadn’t changed at all…

Debugging

Instead of a String or Vec, we actually used Cow<'a> for performance reasons. Cow<'a> is a type that can hold either a borrowed or an owned value at runtime. The idea is to use the borrowed one as much as possible, and only if ownership or mutation is needed, the underlying value is cloned (a better explanation can be found here).

Heinz, my mentor at Tremor, managed to reproduce the issue to Cow and its equivalent, RCow. A Playground snippet can be found here.

use abi_stable::std_types::RCow;
use std::borrow::Cow;

// This works
fn cmp_cow<'a, 'b>(left: &Cow<'a, ()>, right: &Cow<'b, ()>) -> bool {
    left == right
}

// This fails to compile
fn cmp_rcow<'a, 'b>(left: &RCow<'a, ()>, right: &RCow<'b, ()>) -> bool {
    left == right
}

It failed to compile with the following error, which didn’t help much. In Rust 1.62.0, they actually improved it to explain what’s going on (shown at the end of the article):

$ cargo b
   Compiling repro v0.1.0 (/home/mario/Downloads/repro)
error[E0623]: lifetime mismatch
  --> src/lib.rs:10:10
   |
9  | fn cmp_rcow<'a, 'b>(left: &RCow<'a, ()>, right: &RCow<'b, ()>) -> bool {
   |                            ------------          ------------
   |                            |
   |                            these two types are declared with different lifetimes...
10 |     left == right
   |          ^^ ...but data from `left` flows into `right` here

For more information about this error, try `rustc --explain E0623`.
error: could not compile `repro` due to previous error

What? Why do lifetimes matter here if it’s just a comparison?

This hinted that the issue was in the underlying library, not my code. RCow is supposed to be a drop-in replacement for Cow. It also had something to do with PartialOrd, which is the trait used for == here. But I couldn’t see a difference in its implementations:

impl<'a, B: ?Sized> PartialOrd for Cow<'a, B>
where
    B: PartialOrd + ToOwned,
{
    #[inline]
    fn partial_cmp(&self, other: &Cow<'a, B>) -> Option<Ordering> {
        PartialOrd::partial_cmp(&**self, &**other)
    }
}

impl<'a, B> PartialOrd<RCow<'a, B>> for RCow<'a, B>
where
    B: PartialOrd + BorrowOwned<'a> + ?Sized,
{
    #[inline]
    fn partial_cmp(&self, other: &RCow<'a, B>) -> Option<Ordering> {
        PartialOrd::partial_cmp(&**self, &**other)
    }
}

There are more libraries providing drop-in replacements for Cow. And for example, beef managed to get it right, somehow. I wasn’t able to reproduce the issue with their version… but why?

impl<A, B, U, V> PartialOrd<beef::Cow<'_, B, V>> for beef::Cow<'_, A, U>
where
    A: Beef + ?Sized + PartialOrd<B>,
    B: Beef + ?Sized,
    U: Capacity,
    V: Capacity,
{
    #[inline]
    fn partial_cmp(&self, other: &beef::Cow<'_, B, V>) -> Option<Ordering> {
        PartialOrd::partial_cmp(self.borrow(), other.borrow())
    }
}

Some progress… or not?

Other traits like PartialEq also caused similar lifetime errors. I was able to fix some by introducing a new lifetime 'b into the trait implementation. This indicated the Rust compiler that comparing objects with different lifetimes is okay:

-impl<'a, B> PartialEq<RCow<'a, B>> for RCow<'a, B>
+impl<'a, 'b, B, C> PartialEq<RCow<'b, C>> for RCow<'a, B>
 where
     B: PartialEq + BorrowOwned<'a> + ?Sized,
+    C: BorrowOwned<'b> + ?Sized,
 {
-    fn eq(&self, other: &RCow<'a, B>) -> bool {
+    fn eq(&self, other: &RCow<'b, C>) -> bool {
         PartialEq::eq(&**self, &**other)
     }
 }

I suddenly got a bit of hope. But this could never work for Ord, which also failed. The Ord trait uses Self for the other parameter, so I can’t just introduce a new lifetime.

impl<'a, B: ?Sized> Ord for RCow<'a, B>
where
    B: Ord + BorrowOwned<'a>,
{
    #[inline]
    fn cmp(&self, other: &Self) -> Ordering {
        Ord::cmp(&**self, &**other)
    }
}

// Implementation in the standard library:
impl<B: ?Sized> Ord for Cow<'_, B>
where
    B: Ord + ToOwned,
{
    #[inline]
    fn cmp(&self, other: &Self) -> Ordering {
        Ord::cmp(&**self, &**other)
    }
}

Discovering the root cause

Some wonderful people on the Rust Discord server helped me understand what was going on. So I started learning more about the so-called “Subtyping and Variance”.

This topic isn’t covered in The Rust Book. We’ll only find it in its more obscure, unsafer brother, The Rustonomicon. This book explains it incredibly well, so I won’t repeat it here. Here are some resources:

“Subtyping and Variance” — The Rustonomicon (an explanation)
“Subtyping and Variance” — The Rust Reference (a cheatsheet)
“Covariance and contravariance” — Wikipedia (the general term)

A couple blog posts take a more practical approach, like “Rust Lifetime Subtype Variance” — Prolific K or “Diving Deep: implied bounds and variance” — lcnr.de. Or if you’re a visual learner, this video from Jon Gjengset might be best for you.

Trying to fix it

The difference between RCow and Cow was the BorrowOwned<'a> trait. For technical reasons, it was being used as a subtrait of ToOwned, and it had to bind to a lifetime 'a. Ultimately, this made RCow invariant over 'a, while Cow was covariant. We want RCow to be covariant for this to work.

 impl<B: ?Sized> Ord for Cow<'a, B>
 where
-    B: Ord + ToOwned,  // in Cow
+    B: Ord + BorrowOwned<'a>,  // in RCow
 {
     #[inline]
     fn cmp(&self, other: &Self) -> Ordering {
         Ord::cmp(&**self, &**other)
     }
 }

Attempt #1: GATs

I had an idea of using Generic Associated Types (GATs). Instead of binding the lifetime to the trait, I could do so to its associated type. Then, I’d be able to use BorrowOwned instead of BorrowOwned<'a>:

impl<T> BorrowOwned for T {
    type RBorrowed<'a> where T: 'a = &'a T;
}

But a section in the Rust Developer Book states that “traits with associated types must be invariant with respect to all of their inputs”. So that still didn’t help make our type covariant.

Note I only found that statement in the book for developers of the compiler! I opened an issue about that in The Rustonomicon, and moved on to something else.

Attempt #2: `transmute`

After many wasted hours, I was tempted to use transmute and call it a day. Here’s what Heinz suggested (trigger warning):

fn compare<'a, 'b>(left: &RCow<'a, str>, right: &RCow<'b, str>) -> Ordering {
    unsafe {
        let right: &RCow<'a, str> = std::mem::transmute(right);
        left.cmp(right)
    }
}

It worked! In theory, it’s safe because both 'a and 'b will live for at least as long as the function does, and we’re returning an owned type.

Ideally, we’d abstract it away by writing a wrapper around RCow with the fix. But that wouldn’t help because invariant relationships are inherited, and the wrapper’s implementation of Ord would still use BorrowOwned<'a>.

struct SCow<'a>(RCow<'a, ()>);  // will still be invariant!

One workaround would be to hide RCow under a *const (). Then, I can pointer-cast back and forth from it. But in this project, I already had too many things backfire. Traumatized, I continued looking for a solution.

Attempt #3: getting rid of `BorrowOwned<'a>`

The best way to not have problems with this trait is to get rid of it. The standard library has ToOwned, which links a borrowed type with its owned counterpart. For example, &str and String. If Cow<B> requires B: ToOwned, then the Cow::Borrowed variant can just hold &B and Cow::Owned can hold B::Owned.

BorrowOwned<'a> roughly did the same thing for types defined in abi_stable, such as RStr and RString:

// standard library
let x: &str = "abc";
let x_owned: String = x.to_owned();

// abi_stable
let x_ffi_safe: RStr<'_> = rstr!("abc");
let x_owned: String = x.to_owned();
let x_ffi_safe_owned: RString = x.r_to_owned();

Note that we need a lifetime in BorrowOwned because the equivalent of &'a str is RStr<'a>. Which is not exactly the same. This is because str is a Dynamically Sized Type (DST), but custom DSTs aren’t supported by Rust.

impl ToOwned for str {  // okay
    type Owned = String;
    // `&self` is `&str`
    fn to_owned(&self) -> String { ... }
}

impl ToOwned for RStr {
    type Owned = RString;
    // `&self` is `&RStr<'a>`, but we want `RStr<'a>`
    // So we can't quite use `ToOwned` here
    fn to_owned(&self) -> RString { ... }
}

So instead of establishing this relationship through a trait, we can introduce a new generic paramter O. B would be the borrowed type, and O the owned one. This is similar to what the cervine crate does, which relaxes the constraints of Cow:

// Before:
#[repr(C)]
enum RCow<'a, B>
where
    B: BorrowOwned<'a> + ?Sized,
{
    Borrowed(<B as BorrowOwned<'a>>::RBorrowed),
    Owned(<B as BorrowOwned<'a>>::ROwned),
}

// After:
#[repr(C)]
enum RCow<B, O> {
    Borrowed(B),
    Owned(O),
}

/// Ffi-safe equivalent of `Cow<'a, T>`, either a `&T` or `T`.
type RCowVal<'a, T> = RCow<&'a T, T>;
/// Ffi-safe equivalent of `Cow<'a, str>`, either an `RStr` or `RString`.
type RCowStr<'a> = RCow<RStr<'a>, RString>;
/// Ffi-safe equivalent of `Cow<'a, [T]>`, either an `RSlice` or `RVec`.
type RCowSlice<'a, T> = RCow<RSlice<'a, T>, RVec<T>>;

Without the BorrowOwned trait, our struct was now covariant over 'a, and the errors disappeared. Rodri, the author of abi_stable ended up proposing the fix that was merged. You can find a simplified version here.

Conclusion

This showcased two gaps in the language:

There were no indications in the error message about the issue being related to “variance”. I had no idea what that was, and it wasn’t covered in the book.
It was very hard to debug the variance of a type, given that they are implicit.

So it’s amazing to hear that starting in Rust 1.62.0, you’re even taken to the documentation. It will still be hard to understand the whole topic, but at least you know where to start!

error: lifetime may not live long enough
  --> src/main.rs:55:5
   |
54 | fn test2<'a, 'b>(left: &RCow<'a, u8>, right: &RCow<'b, u8>) -> Ordering {
   |          --  -- lifetime `'b` defined here
   |          |
   |          lifetime `'a` defined here
55 |     left.cmp(right)
   |     ^^^^^^^^^^^^^^^ argument requires that `'a` must outlive `'b`
   |
   = help: consider adding the following bound: `'a: 'b`
   = note: requirement occurs because of the type `RCow<'_, u8>`, which makes the generic argument `'_` invariant
   = note: the enum `RCow<'a, B>` is invariant over the parameter `'a`
   = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance

I was lucky to have such a great team at Tremor, and an OSS maintainer as helpful as Rodri. You can find all the details of the discussion in the original GitHub issue:

lifetimes with R* types break compared to non R* types rodrimati1992/abi_stable_crates#75

Beyond Dumbphones: Building a Minimalist Yet Functional Phone

2024-06-01T00:00:00Z

Table of Contents

What if there was a company selling phones that, against all odds, optimized how little they were used? There’s a lot of potential here, given that every company is trying to get you hooked on their app.

“Dumbphones” have emerged for this very reason. These are phones whose functionality is intentionally locked down, with the purest examples only supporting calls and SMS. On the other side of the spectrum, we have smartphones. It wouldn’t be far-fetched to say that most people owning one use it more than they’d like.

I’ve seen people around me switching to a dumbphone. A digital detox is not a bad idea; it helps reduce anxiety in the short term. But in the longer term, it’s pretty much impossible. A few apps like WhatsApp or Google Maps can be indispensable, and the rest actually improve our lifes when used reasonably. TikTok can be incredibly addicting, but it’s arguably a great way for people to learn, relax, or connect with others.

Even the founders of the Light Phone — the most popular dumbphone brand — thought it was too restrictive for them, and they have released a more featureful version^[1]. Don’t get me wrong! I admire the company and all the work they do; this is such a small industry that anything helps. But I wonder if there’s something that could become more mainstream.

Besides, as a big fan of second-hand stuff, I’d rather give an old Motorola a second life than bring yet another phone to the world. It’d also be really cool to open and close a retro flip-phone (right???).

So what if we tried to find a middle point? There are two possible approaches:

A dumbphone trying to be a smartphone
Or a smartphone trying to be a dumbphone

I’m a bigger fan of the latter, because most dumbphones don’t give you direct access to an app store. Given how huge the mobile ecosystem is, supporting everyone’s use-cases one by one ends up being very restrictive. Your dumbphone’s operating system could add custom support for Uber, but there may be someone who won’t buy the phone unless it has Spotify. And a password manager. And their banking app. And after implementing all of these, someone will need a damned parking app for their apartment complex (yes, really^[2]).

The question arises: how in the world would you even design a phone? It sounds like the most daunting task, so that’s what I’ll be researching in this series! We’ll get started with broader topics like addiction, building companies, or existing solutions, and eventually get into more details. I’m no expert, so let’s see how far the internet can take us.

Looking at the state of things

I’ve been ~~nerdsniped~~ drawn to this space for some time, learning more about it from different perspectives:

Technology

Firstly, let’s take a step back from “phones that shouldn’t be used”. Can we generalize it to “tech that shouldn’t be used”? Or rather, “tech that doesn’t unnecessarily grab your attention”? Now we’re onto something! That’s what “Calm Technology” is all about, a concept coined by Mark Weiser and John Seely Brown at XEROX Parc in the '90s.

Calm Technology describes design principles that let a piece of technology inform the user without demanding their attention. Anything from hammers to Roombas can be considered “calm”. Researcher Amber Case picked up on this and recently announced an institute to certify products that meet these standards. Also inspired by Mark and John’s original work, Jon Yablonski shares his take and more resources in his website Humane By Design.

The Center for Humane Technology is a more established organization on this topic. This non-profit aims to educate people and companies, raise awareness, and aid in policymaking. It gained popularity in Neflix’s hit documentary The Social Dilemma, which focuses on social media. One of its founders, Tristan Harris, created an internal presentation at Google in 2013 that has aged like fine wine.

On the flip side, Cal Newport writes about what humans can do to stop their attention from being hijacked. He calls it “Digital Minimalism” and also covers managing your attention span in “Deep Work”, or living a more minimalist life in general. This is also discussed on NoSurf, the biggest community on the topic I could find, with even more information^[3].

Negative effects

Is the population aware of problematic smartphone use? Is it even a problem to them? Certainly, there are studies on how phone usage leads to poorer psychological health. But it doesn’t help that there’s no medical consensus nor recognition; there’s correlation, but not clear causation^[4]

Phones have undoubtedly impacted our society. Some changes are good, others, not so much. We fiddle with our phones to avoid eye contact, social media worsens our self-esteem, and we have attention issues. It’s just hard to be bored anymore, which is actually good for you^[5]^[6]. Phones also disrupt our sleep cycles^[7], and cars frequently crash because of them^[8]. Nothing you don’t already know.

There are increasingly more stories about Hikikomori, described as “total withdrawal from society and seeking extreme degrees of social isolation and confinement”. Of course, it’s not limited to Japan. Wikipedia sheds light on the role of technology in this condition:

Although the connection between modern communication technologies (such as the Internet, social media and video games) and the phenomenon is not conclusively established, those technologies are considered at least an exacerbating factor that can deepen and nurture withdrawal.

It even specifically mentions our topic:

The emergence of mobile phones and then smartphones may also have deepened the issue, given that people can continue their addiction to gaming and online surfing anywhere, even in bed.

Some people suffering from migraines or light sensitivity are affected by the screen itself^[9]^[10]. There’s also the topic of phones for children or the elderly, but that’s a huge rabbit hole. Looking at the broader population, nearly 6 in 10 U.S. smartphone users admit that they use their phone too much^[11]. So the good news is that people are at least aware of the issue. That surely means there’s a need for solutions, regardless of science?

Companies

In order to start a company that cares about people’s mental health, it’s sensible to say that its workers shouldn’t be crunching 60 hours per week. I learned a lot about this by reading “It doesn’t have to be crazy at work”, written by the founders of Basecamp. Part of its contents are also available online^[12]^[13]^[14]. You don’t have to agree with everything to admit it makes you rethink how companies are built.

This includes leadership, benefits, planning, hiring, funding… and how to make them more reasonable. One key idea is that starting a business with the goal of changing the world sets the stakes too high from the beginning. The desire to get rich becomes the justification for all-nighters and bad practices, which spreads from founders to employees. Other so-called “Calm Companies” that share this mindset include Transistor.fm^[15]^[16]^[17]^[18], Wildbit^[19], or Convertkit^[20], but Basecamp is the loudest about it.

An important caveat is that we must ensure the company stays true to its values. In this case, that the less screen time, the better (within profitable means). Not that the more revenue, the better. However, this depends on who controls the company, which is often the person who provides the funding.

We can learn this lesson from CouchSurfing, a website that helps you find a couch at someone’s home for a short stay. The company’s goal wasn’t just to “connect hosts to guests”, but to “create lasting positive experiences and relationships between strangers”. And it wasn’t gibberish: their internal metric for success was “positive time had thanks to CouchSurfing” subtracted by “time spent on the website”^[21]. But after a hectic story, they changed to a for-profit and started heading in the opposite direction, fueled by the need for monetization^[22].

What most Calm Companies do is bootstrap, meaning the founders use their own money. They promise never to go public, and Basecamp goes a step further by assuring they will never sell the company^[23]. Unfortunately, that only works for purely software companies; manufacturing hardware means you will need money. The Nothing company spends $30 to $50 million for each new product they develop^[24]. On a “smaller” scale, the Light Phone 2 raised a total of $11.9 million in 2019^[25]. You’d probably do fine with much less, but it shows how hard it’d be to save up that kind of money yourself.

A popular choice for niche hardware is crowdfunding, where the money comes from a bunch of interested people on the internet. Bigme, reMarkable, Light Phone, Minimal Phone, Librem. They all started like this. But from what I’ve learned, the crowdfunded money usually only covers the hardware costs, not so much the upfront design and testing. The successful path seems to be Venture Capital (VC) funding to set up the team and build a prototype, and then crowdfunding to actually manufacture it.

Going back to the Light Phone 2, they raised $3.5M from consumers on IndieGoGo^[26] but also $8.4M in seed funding^[25:1]. Similarly, reMarkable was able to secure $11M from presales but required an essential $10M in seed funding^[27]. It’s hard to avoid VC funding if you aren’t already a billionaire, like in the case of Mudita. This company was founded by Michał Kiciński, who had already succeeded with CD Projekt^[28].

Nowadays, startups raise VC money precisely by asserting they will change the world. Their solution will blow the competition out of the water. Prepare to invest RIGHT NOW or miss the train. It’s just FOMO for investors. How could this work in the context of a Calm Company?

Although harder, I don’t believe it’s impossible. There are funds that leave plenty of freedom to the founders. And a minority of them are specialized in Calm Companies, such as CalmFund (which just paused operations), indie.vc, or tinyseed.

Once you get over the necessity of raising money, there are other ways to preserve the company’s ideals. OpenAI famously failed to do so as a pure non-profit, allegedly because raising money was too hard as just a nonprofit^[29]. So maybe we could take that as a learned lesson. Other ideas are certifications like B Corporation or Social Enterprise, but I’m not sure how effective they are.

You can see how much more I can research about this topic in future posts. And we haven’t even gotten to the phones section yet!

Software

Smartphones actually come with solid features to block apps and minimize screen time in general. However, they don’t seem to be good enough to gain adoption.

Firstly, they aren’t well marketed; most people aren’t aware of their existence. Android calls the features “Digital Wellbeing” and iOS “Screen Time”. Personally, I receive more system notifications about new AI features in my camera than things like this.

Secondly, they aren’t first-party citizens; the tools are there, but they don’t quite integrate seamlessly. The most powerful feature on Android is “modes”, which allows you to switch settings for different situations. For example: when your GPS shows you’re in the library, it can disable Instagram and set the screen to grayscale. Being so powerful, it’s also complicated to configure (and to keep your setup up to date). If the company prioritized reducing screen time over ad revenue, we’d likely see more ideas to improve its adoption.

There are heaps of alternatives on the app marketplaces, although their source code may not be available, and most have in-app payments or ads. Here are some cool features I’ve seen while trying out Android apps^[30]^[31]:

fancy tutorials,
syncing across devices (including your laptop or tablet),
blocking websites (or even features inside an app, like YouTube Shorts),
blocking pre-bundled categories of apps and websites (such as “shopping”),
breathing exercises before opening apps (or having to read a book),
motivational quotes,
forums,
a floating timer indicating total usage on that day,
notification filtering and bundling,
gamification (competing against yourself or friends),
comprehensive statistics,
or having someone else to control your usage.

Not everything is limited to blocking apps; there are also minimal app launchers^[32] or simple productivity timers^[33]. minimalist phone does well in the “seamless experience” department by filtering notifications and taking over your launcher to control how you open apps. I don’t want to do an exhaustive analysis, but just searching “screentime” will already return many results on the app store. It’s worth downloading a few until you find your favorite anyway.

However, manufacturers have it much easier, given that they have full system access over your phone. For instance, the open-source app TimeLimit is an even more configurable alternative to “Digital Wellbeing”. But being external, it needs to start with a long (and worrying) step to grant permissions. This alone is one step too many to make it widespread — I’d argue that even having to install an app is too much.

To improve the user experience, some apps make emphasis on explaining how to use their features. Others avoid it by trying to be smarter; they have your current phone’s usage data, so they already know which apps you use too much. One last approach is to be opinionated and only support a subset of features that may integrate better or have more impact. The Light Phone 2 does this by only providing their limited list of features; if you’re missing one, maybe you’ll get it, but maybe not.

Something else raising the barrier of entry is monetization. Although necessary, some subscription models can be too much. A particularly creative app I liked was Digital Detox , which makes you pay $2 upon failing to meet your phone usage goals.

It’s just great to have so many options and not being locked in to any of them. Different solutions for different people.

Accessories

Some products allow you to disable apps based on physical access. Imagine a keychain with an NFC chip that can restrict apps on your phone. Having to find it and hold it near your phone can help break the habit of opening Instagram automatically, turning it into a conscious decision. These tools can also help transform your phone into a dumbphone: simply block the apps and leave the device at home to fully disconnect.

A couple of options are Brick and UnPluq. They only solve part of the issue, though, and UnPluq follows a subscription-based model that costs 70€ per year. Still, they seem to work well for some folks, which is awesome.

Phones

What would a phone minimizing screen time look like? Many of the popular ones have e-paper displays instead of LCD, which is most commonly seen on e-readers. E-paper feels like real paper, is easier to see under sunlight, and may increase battery time. It doesn’t come without drawbacks, given that it literally moves physical particles in your screen instead of emitting light. You can judge yourself:

Hey, it’s not a good experience for videos, but it doesn’t take 5 seconds per refresh like your crappy 10-year-old Kindle. Knowing how it works under the hood, this sample is impressive to me. Here’s another monitor that recently came out focusing on latency:

E-paper has always had a poor refresh rate. The issue isn’t just that videos are hard to watch — you shouldn’t do it often, anyway. The real problem is that sluggish animations worsen the user experience. Recently, a wave of e-paper products with faster refresh times^[34]^[35] has emerged, so I’m hoping that will improve.

Another charasteristic of most e-paper screens is that they are grayscale. While the absence of colors is linked with reduced addiction^[36], it can also be frustrating. I’ve set my phone to grayscale, and I know how confusing Google Maps can sometimes be without colors. Additionally, charts that rely on color require you to view them on a different device. And I haven’t even tried gaming. One could argue that this is intentional, to get you to use different devices for different purposes. Instead of playing Candy Crush on the train, you might read and wait until you get home to use your PS4.

There’s now color e-paper, with Kobo having released its first e-readers in 2024^[37]. But it does have downsides, such as worse refresh rates or lower contrast ratios. Personally, I’d love to try to embrace the limitations of grayscale. Issues with essential apps like Google Maps could be resolved with custom software. And not having the best experience watching YouTube on your phone might be for good. It’s possible that having a single color like red could improve the user experience by highlighting important items, though.

Nowadays, the biggest brands that go beyond e-readers are HiSense and Boox. However, they aren’t well-supported in the west. Some apps refuse to open, and connectivity only works with certain providers, if at all^[38]^[39]. Boox is known for violating GPL compliance, too^[40]. There are startups releasing similar devices, but they have a long road ahead: Mudita will announce a new phone soon^[41], and Daylight might work on a phone if their $729 tablet is successful^[42]^[43].

For 360€, the Blloc Zero18 was one of the few phones that didn’t use e-paper but wasn’t a dumbphone either. It balanced full functionality with impressive features designed to keep you from opening apps at all^[44]. By default, its screen was grayscale, but tapping the fingerprint sensor would bring back the color. The homepage combined all your chats into a single feed, similar to Beeper, and had interactive widgets for news, notes, or YouTube search. As you can tell from my use of the past sense, though, they ended up ditching the phone. The company shifted focus to just developing the launcher, and they ended up running out of money^[45]. Many employees have since joined Nothing.

Another notable flop^[46] was the YotaPhone. This unique phone featured an additional e-paper screen on the back, marketed for reading and basic tasks. As innovative as it was, you’d have to really like reading to justify spending over $600 for a phone that was otherwise unimpressive^[47]. Unfortunately, it never gained popularity in Europe and was not released in the US^[48]^[49]^[50]^[51].

A simpler approach to consider is what Ghost Mode does. They lock down a Pixel6a with their custom operating system and resell it. In the end, it’s essentially a dumbphone with a nice camera. They don’t need to deal with manufacturing, and the software still has system access for advanced features. I’m only afraid that relying on Google might not be a good idea^[52], but they could switch to a different base phone. Its unpopularity might also have to do with money: at $600, it’s pricey for a dumbphone.

Wrapping up

I hope this topic can eventually be “a thing”. Just like there are movements for “sustainability” or “diversity”, there should also be one for “reduced screentime”. To me, it has a strong relationship with mental health, and there’s a lot to improve in that regard. I love the internet: being able to share this post so easily is wonderful. But what can we do to reduce the bad parts?

We still don’t fully know what this hybrid between a smartphone and a dumbphone looks like. However, this research has taught us many essential things: the relevant organizations, its societal impact, how to build the company, existing software and hardware solutions, and some history.

More details about the design will come later in the series. You can subscribe for free to keep up to date. It’s hard to say yet, but I’d love to build something in this area in the future. Manufacturing a phone sounds crazy, but I’d love to see what can be done :)

Disclaimer: I am not affiliated with any of the companies mentioned in this post. The opinions expressed are my own and are based on my personal experiences and research.

Footnotes

The high hopes of the low-tech phone — The Verge ↩︎
I have to use an app to open my apartment complex parking gate, the app is called Gatewise. My lease does not mention anything about needing a smartphone or the use of any apps for garage access. Street parking is not an option. I just want technological equity — r/dumbphones ↩︎
Digital Minimalism Reading List — r/NoSurf ↩︎
Apple investors say iPhones cause teen depression. Science doesn’t — Wired ↩︎
Why Boredom is Good For You — YouTube, Veritasium ↩︎
Louis CK Embrace Your Loneliness — YouTube, The Impossible Conversation ↩︎
Study: Using Apple’s Night Shift to improve your sleep? Don’t bother — arstechnica (Quote: “it is important to think about what portion of that stimulation is light emission versus other cognitive and psychological stimulations”) ↩︎
Phones Track Everything but Their Role in Car Wrecks — The New York Times (In summary, the exact number is unknown. This NSC report estimates it to be 1.6 million crashes, but it’s not precise and from 2010) ↩︎
LEDStrain Forum ↩︎
Has anyone here been diagnosed with central sensitization and/or relate somehow to my story? (36M, pain started at 33) — r/ChronicPain ↩︎
Americans Have Close but Wary Bond With Their Smartphone — Gallup ↩︎
37signals (the company that owns Basecamp) ↩︎
The 37signals Employee Handbook ↩︎
Signal v. Noise (37signals’ former blog) ↩︎
I’m 40 years old and I finally bootstrapped a SaaS, Transistor.fm, to millions in revenue (with a co-founder!) — r/SaaS ↩︎
Build Your SaaS — transistor.fm ↩︎
What are our values? — GitHub TransistorFM/handbook ↩︎
Justin Jackson (co-founder of Transistor.fm) ↩︎
Wildbit ↩︎
The ConvertKit Team Handbook ↩︎
Distracted? Let’s make technology that helps us spend our time well | Tristan Harris | TEDxBrussels — YouTube, TEDx Talks ↩︎
Paradise lost: The rise and ruin of Couchsurfing.com — Input ↩︎
An obligation to independence — 37signals (the company that owns Basecamp) ↩︎
Nothing CEO Carl Pei on the Phone 2 and the future of gadgets | The Vergecast — YouTube @ 25:49 ↩︎
This credit-card-size phone can do only 3 things and doesn’t have any apps — and it may be the key to freeing us from our smartphones — Business Insider ↩︎ ↩︎
Light Phone 2 — IndieGoGo ↩︎
Remarkable raises $15 million to bring its e-paper tablets to more scribblers — VentureBeat ↩︎
Mudita new technology company co-founder of CD Projekt — eurogamer.pl (archive) ↩︎
Our structure — OpenAI ↩︎
Digital control: AppBlock , Freedom , YourHour , Digital Detox , StayFree , Stay Focused , StayOff , ActionDash , ClearSpace , Refocus , Opal , Jomo , SocialFocus: Hide Distractions, UnTrap for YouTube , BB - Screen Time & App Blocker ↩︎
How to add a timer to social media (Spanish, German) ↩︎
Launchers: Olauncher , minimalist phone , Indistract , Blank Spaces ↩︎
Productivity timers: Forest , Flora , Plantie ↩︎
Daylight debuts world’s first ‘blue-light-free computer’ with a 120Hz LivePaper display — ZDNET (Note: Daylight uses a mix between conventional electrophoretic e-paper and LCD. It feels slightly less like paper, but still improves refresh rate.) ↩︎
“Open-source Eink monitor with an emphasis on low latency” — GitHub Modos-Labs/Glider ↩︎
Will turning your phone to greyscale really do wonders for your attention? — The Guardian ↩︎
Kobo announces its first color e-readers — The Verge ↩︎
Hisense A9 - 1 Week Review — r/eink ↩︎
Why oh why no SIM-card / mobile data support? — Boox Forums ↩︎
GPL Compliance, Onyx Boox — Wikipedia ↩︎
First glimpse of Mudita Kompakt — Mudita ↩︎
The Daylight DC1 is a $729 attempt to build a calmer computer — The Verge ↩︎
Episode #234: Anjan Katta (Founder of Daylight Computer Co), by THE 2AM PODCAST — YouTube @ 37:20 ↩︎
Android in Monochrome? | Blloc Zero 18 - exclusive first look ↩︎
Blloc’s Discord server (more information in the announcements channel) ↩︎
The company behind the dual-screen YotaPhone is bankrupt — The Verge ↩︎
Yotaphone 2 review — TechRadar ↩︎
Dual-Screen YotaPhone Launches in Russia, Europe — PCMag (Release of first generation only in EU, Russia, and Middle East) ↩︎
Dual-Screened YotaPhone 2 Launches in Europe — PCMag (Release of second generation only in EU, Russia, and Middle East) ↩︎
Supply Issues Force Cancellation Of North American YotaPhone 2 Despite Successful Crowdfunding Campaign — Android Police (Release failure of second generation in the US) ↩︎
Dual-screen YotaPhone 3 is finally official and it’s just as kooky as the last two — TechRadar (Release of third and last generation only in China) ↩︎
Killed by Google ↩︎

The bane of my existence: Supporting both async and sync code in Rust

2024-01-14T00:00:00Z

Table of Contents

Introduction

Imagine you want to create a new library in Rust. All it does is wrap up a public API that you need for something else, like the Spotify API or maybe a database like ArangoDB. It’s not rocket science, you aren’t inventing something new or dealing with complex algorithms, so you expect it to be relatively straightforward.

You decide to implement the library with async. Most of the work in your library has to do with performing HTTP requests, which are mostly I/O, so it makes sense (that, and because it’s what the cool kids use in Rust nowadays). You start coding and have the v0.1.0 release ready in a few days. “Neat”, you say, as cargo publish finishes successfully and uploads your work to crates.io.

A couple of days pass, and you get a new notification on GitHub. Someone opened an issue:

How can I use this library synchronously?

My project doesn’t use async because it’s overly complex for what I need. I wanted to try your new library, but I’m not sure how to do it easily. I would rather not fill my code with block_on(endpoint()). I’ve seen crates like reqwest exporting a blocking module with the exact same functionality, could you perhaps do that as well?

Low-level wise, that sounds like a very complicated task. Having a common interface for both async code — which requires a runtime like tokio, awaiting futures, pinning, etc — and regular sync code? I mean, they asked nicely, so maybe we can try. After all, the only difference in the code would be the occurrences of the async and await keywords because you aren’t doing anything fancy.

Well, this is more or less what happened with the crate rspotify, which I used to maintain along with its creator Ramsay. For those who don’t know, it’s a wrapper for the Spotify Web API. To clarify, I did get this working in the end, although not as cleanly as I was hoping; I’ll try to explain the situation in this new article of the Rspotify series.

The first approaches

To give more context, here’s what Rspotify’s client looks like, roughly:

struct Spotify { /* ... */ }

impl Spotify {
    async fn some_endpoint(&self, param: String) -> SpotifyResult<String> {
        let mut params = HashMap::new();
        params.insert("param", param);

        self.http.get("/some-endpoint", params).await
    }
}

Essentially, we would have to make some_endpoint available for both asynchronous and blocking users. The important question here is, how do you do this once you have dozens of endpoints? And how can you make it easy to switch between async and sync for the user?

Good ol’ copy-pasting

This is what was first implemented. It was quite simple and it worked. You just need to copy the regular client code into a new blocking module in Rspotify. reqwest (our HTTP client) and reqwest::blocking share the same interface, so we can manually remove keywords like async or .await and import reqwest::blocking instead of reqwest in the new module.

Then, the Rspotify user just can just use rspotify::blocking::Client instead of rspotify::Client, and voilà! Their code is now blocking. This will bloat the binary size for async-only users, so we can just feature-gate it under the name blocking and done.

The problem was much more clear later on, though. Half the crate’s code was duplicated. Adding a new endpoint or modifying it meant writing or removing everything twice.

There is no way to make sure both implementations are equivalent unless you test absolutely everything. Which isn’t a bad idea either, but maybe you copy-pasted the tests wrong! How about that? The poor reviewer would have to read through the same code twice to make sure both sides look alright — which sounds incredibly prone to human errors.

In our experience, it really slowed down the development of Rspotify, specially for new contributors who weren’t used to this whole ordeal. As a new excited maintainer of Rspotify, I began to investigate other possible solutions.

Calling `block_on`

The second approach consisted on implementing everything on the async side. Then, you just make wrappers for the blocking interface, which call block_on internally. block_on will run the future until completion, basically making it synchronous. You still need to copy the method definitions, but the implementation is written once only:

mod blocking {
    struct Spotify(super::Spotify);

    impl Spotify {
        fn endpoint(&self, param: String) -> SpotifyResult<String> {
            runtime.block_on(async move {
                self.0.endpoint(param).await
            })
        }
    }
}

Note that in order to call block_on, you first have to create some kind of runtime in the endpoint method. For example, with tokio:

let mut runtime = tokio::runtime::Builder::new()
    .basic_scheduler()
    .enable_all()
    .build()
    .unwrap();

This raises the question: should we initialize the runtime in each call to the endpoint, or is there a way to share it? We could keep it as a global (ewwww), or perhaps better, we can save the runtime in the Spotify struct. But since it takes a mutable reference to the runtime, you’d have to wrap it up with Arc<Mutex<T>>, completely killing the concurrency in your client. The proper way to do this is with Tokio’s Handle, which looks like this:

use tokio::runtime::Runtime;

lazy_static! { // You can also use `once_cell`
    static ref RT: Runtime = Runtime::new().unwrap();
}

fn endpoint(&self, param: String) -> SpotifyResult<String> {
    RT.handle().block_on(async move {
        self.0.endpoint(param).await
    })
}

While the handle does make our blocking client faster^[1], there is an even more performant way to do it. This is what reqwest itself does, in case you’re interested. In short, it spawns a thread that calls block_on waiting on a channel with jobs^[2]^[3].

Unfortunately, this solution still has quite the overhead. You pull in large dependencies like futures or tokio, and include them in your binary. All of that, in order to… actually end up writing blocking code. So not only is it a cost at runtime, but also at compile time. It just feels wrong to me.

And you still have a good amount of duplicate code, even if it’s just definitions, which can sum up. reqwest is a huge project and can probably afford this for their blocking module. But for a less popular crate like rspotify, this is harder to pull off.

Duplicating the crate

Another possible way to fix this is, as the features docs suggest, creating separate crates. We’d have rspotify-sync and rspotify-async, and users would just pick whichever crate they want as a dependency, even both if they need to. The problem is — again — how exactly do we generate both versions of the crate? I was unable to do this without copy-pasting the entire crate, even with Cargo tricks like two Cargo.toml files, one for each crate (which was quite inconvenient anyway).

With this idea we can’t even use procedural macros because you can’t just create a new crate within a macro. We could define a file format to write templates of Rust code in order to replace parts of the code like async/.await. But that sounds completely out of scope.

What ended up “working”: the `maybe_async` crate

The third attempt is based on a crate called maybe_async. I remember foolishly thinking it was the perfect solution back when I discovered it.

Anyway, the idea is that with this crate you can automatically remove the async and .await occurrences in your code with a procedural macro, essentially automating the copy-pasting approach. For example:

#[maybe_async::maybe_async]
async fn endpoint() { /* stuff */ }

Generates the following code:

#[cfg(not(feature = "is_sync"))]
async fn endpoint() { /* stuff */ }

#[cfg(feature = "is_sync")]
fn endpoint() { /* stuff with `.await` removed */ }

You can configure whether you want asynchronous or blocking code by toggling the maybe_async/is_sync feature when compiling the crate. The macro works for functions, traits and impl blocks. If one conversion isn’t as easy as removing async and .await, you can specify custom implementations with the async_impl and sync_impl procedural macros. It does this wonderfully, and we’ve already been using it for Rspotify for a while now.

In fact, it worked so well that I made Rspotify http-client agnostic, which is even more flexible than being async/sync agnostic. This allows us to support multiple HTTP clients like reqwest and ureq, independently of whether the client is asynchronous or synchronous.

Being http-client agnostic is not that hard to implement if you have maybe_async around. You just need to define a trait for the HTTP client, and then implement it for each of the clients you want to support. A snippet of code is worth a thousand words (you can find the full source for Rspotify’s reqwest’s client here, and ureq’s here):

#[maybe_async]
trait HttpClient {
    async fn get(&self) -> String;
}

#[sync_impl]
impl HttpClient for UreqClient {
    fn get(&self) -> String { ureq::get(/* ... */) }
}

#[async_impl]
impl HttpClient for ReqwestClient {
    async fn get(&self) -> String { reqwest::get(/* ... */).await }
}

struct SpotifyClient<Http: HttpClient> {
    http: Http
}

#[maybe_async]
impl<Http: HttpClient> SpotifyClient<Http> {
    async fn endpoint(&self) { self.http.get(/* ... */) }
}

Then, we could extend it so that whichever client they want to use can be enabled with feature flags in their Cargo.toml. For example, if client-ureq is enabled, since ureq is synchronous, it would enable maybe_async/is_sync. In turn, this would remove the async/.await and the #[async_impl] blocks, and the Rspotify client would use ureq’s implementation internally.

This solution has none of the downsides I listed in previous attempts:

No code duplication at all
No overhead neither at runtime nor at compile time. If the user wants a blocking client, they can use ureq, which doesn’t pull tokio and friends
Quite easy to understand for the user; just configure a flag in you Cargo.toml

However, stop reading for a couple of minutes and try to figure out why you shouldn’t do this. In fact, I’ll give you 9 months, which is how long it took me to do so…

The problem

Well, the thing is that features in Rust must be additive: “enabling a feature should not disable functionality, and it should usually be safe to enable any combination of features”. Cargo may merge features of a crate when it’s duplicated in the dependency tree in order to avoid compiling the same crate multiple times. The reference explains this quite well, if you want more details.

This optimization means that mutually exclusive features may break a dependency tree. In our case, maybe_async/is_sync is a toggle feature enabled by client-ureq. So if you try to compile it with client-reqwest also enabled, it will fail because maybe_async will be configured to generate synchronous function signatures instead. It’s impossible to have a crate that depends on both sync and async Rspotify either directly or indirectly, and the whole concept of maybe_async is currently wrong according to the Cargo reference.

The feature resolver v2

A common misconception is that this is fixed by the “feature resolver v2”, which the reference also explains quite well. It has been enabled by default since the 2021 edition, but you can specify it inside your Cargo.toml in previous ones. This new version, among other things, avoids unifying features in some special cases, but not in ours:

Features enabled on platform-specific dependencies for targets not currently being built are ignored.

Build-dependencies and proc-macros do not share features with normal dependencies.

Dev-dependencies do not activate features unless building a target that needs them (like tests or examples).

Just in case, I tried to reproduce this myself, and it did work as I expected. This repository is an example of conflicting features, which breaks with any feature resolver.

Other fails

There were a few crates that also had this problem:

arangors and aragog: wrappers for ArangoDB. Both use maybe_async to switch between async and sync (arangors’s author is the same person, in fact)^[4]^[5].
inkwell: a wrapper for LLVM. It supports multiple versions of LLVM, which are not compatible with eachother^[6].
k8s-openapi: a wrapper for Kubernetes, with the same issue as inkwell^[7].

Fixing `maybe_async`

Once the crate started to gain popularity, this issue was opened in maybe_async, which explains the situation and showcases a fix:

async and sync in the same program fMeow/maybe-async-rs#6

maybe_async would now have two feature flags: is_sync and is_async. The crate would generate the functions in the same way, but with a _sync or _async suffix appended to the identifier so that they wouldn’t be conflicting. For example:

#[maybe_async::maybe_async]
async fn endpoint() { /* stuff */ }

Would now generate the following code:

#[cfg(feature = "is_async")]
async fn endpoint_async() { /* stuff */ }

#[cfg(feature = "is_sync")]
fn endpoint_sync() { /* stuff with `.await` removed */ }

However, these suffixes introduce noise, so I wondered if it would be possible to do it in a more ergonomic way. I forked maybe_async and gave it a try, about which you can read more in this series of comments. In summary, it was too complicated, and I ultimately gave up.

The only way to fix this edge case would be to worsen the usability of Rspotify for everyone. But I’d argue that someone who depends on both async and sync is unlikely; we haven’t actually had anyone complaining yet. Unlike reqwest, rspotify is a “high level” library, so it’s hard to imagine a scenario where it appears more than once in a dependency tree in the first place.

Perhaps we could ask the Cargo devs for help?

Official Support

Rspotify is far from being the first who has been through this problem, so it might be interesting to read previous discussions about it:

This now-closed RFC for the Rust compiler suggested adding the oneof configuration predicate (think #[cfg(any(...))] and similars) to support exclusive features. This only makes it easier to have conflicting features for cases where there’s no choice, but features should still be strictly additive.
The previous RFC started some discussion in the context of allowing exclusive features in Cargo itself, and although it has some interesting info, it didn’t go too far.
This issue in Cargo explains a similar case with the Windows API. The discussion includes more examples and solution ideas, but none have made it to Cargo yet.
Another issue in Cargo asks for a way to test and build with combinations of flags easily. If features are strictly additive, then cargo test --all-features will cover everything. But in case it doesn’t, the user has to run the command with multiple combinations of feature flags, which is quite cumbersome. This is already possible unofficially thanks to cargo-hack.
A completely different approach based on the Keyword Generics Initiative. It seems to be the most recent take on solving this, but it’s in an “exploration” phase, and no RFCs are available as of this writing.

According to this old comment, it’s not something the Rust team has already discarded; it’s still being discussed.

Although unofficial, another interesting approach that could be explored further in Rust is “Sans I/O”. This is a Python protocol that abstracts away the use of network protocols like HTTP in our case, thus maximizing reusability. An existing example in Rust would be tame-oidc.

Conclusion

We currently have a choice to make between:

Ignoring the Cargo Reference. We could assume that noone is going to use both sync and async for Rspotify at the same time.
Fixing maybe_async and adding _async and _sync suffixes to each endpoint in our library.
Dropping support for both async and sync code. It’s kind of become a mess that we don’t have the manpower to deal with and that affects other parts of Rspotify. The problem is that some crates that depend on rspotify like ncspot or spotifyd are blocking, and others like spotify-tui use async, so I’m not sure what they’d think.

I know this is a problem that I’ve imposed to myself. We could just say “No. We only support async” or “No. We only support sync”. While there are users interested in being able to use both, sometimes you just have to say no. If such a feature becomes so complicated to deal with that your entire codebase becomes a mess, and you don’t have the engineering power to maintain it, then it’s your only choice. If someone cared enough, they could just fork the crate and convert it to synchronous for their own usage.

After all, most API wrappers and the like only support either asynchronous or blocking code. serenity (Discord API), sqlx (SQL toolkit) and teloxide (Telegram API) are async-only, for example, and they’re quite popular.

Even though it was quite frustrating at times, I don’t really regret spending so much time walking in circles trying to get both async and sync to work. I was contributing to Rspotify in the first place just to learn. I had no deadlines and no stress, I just wanted to try to improve a library in Rust in my free time. And I have learned a lot; hopefully you too, after reading this.

Perhaps the lesson today is that we should remember that Rust is a low level language after all, and there are some things that aren’t possible without a lot of complexity. Anyhow, I’m looking forward to how the Rust team fixes this in the future.

So what do you think? What would you do if you were a maintainer of Rspotify? You can leave a comment below if you like.

Footnotes

[Talk] Rust, the best and worst thing to happen to Tremor

2022-10-18T00:00:00Z

Hello! I’ve recently had the pleasure of giving a talk at this year’s TremorCon. I really enjoy public speaking, so this has been an excellent opportunity to get better at it (let me know your thoughts!). In case you missed it, you can check out the recording here:

You can also find the slides here in pdf or in pptx.

If you want to learn about new events I’m involved in, like this one, you can follow me on Twitter or LinkedIn. Don’t forget to also give Tremor a follow, and to check out the rest of the conference!

Plugins in Rust: Wrapping Up

2022-07-26T00:00:00Z

Table of Contents

Welcome to the final article of this series! Here I’ll showcase some clean-ups and optimizations I may or may not have performed yet, so that our plugin system can get closer to production. I will also run benchmarks for some of these to ensure that the changes are actually worth it.

Benchmarking

Tooling

There are all kinds of ways to measure the performance of a system, and it’s important to take as many of them as possible into account. Here are the ones I used myself:

Raw metrics: Tremor is able to measure its own raw throughput, latency ranges, or standard deviation. They serve as an overall score and are very easy to compare against eachother.
perf.data: the command perf measures the runtime performance thanks to monitoring capabilities in the Linux kernel. It generates a file with the full execution trace, which can be visualized in multiple ways, including:
- Flamegraphs: hierarchical and simple graphs.
- perf report: the integrated CLI, which is more detailed but harder to understand.
High Dynamic Range (HDR) histograms: Tremor’s latency percentiles can be turned into a graph with this site.

There are many other techniques available for benchmarking, though. I wasn’t able to get into lower level performance indicators like cache misses. This part of the project has helped me realize how hard it is to benchmark a program reliably; I would suggest checking out other dedicated learning sources on the topic. You should also tune the process according to your use-case, since it may vary heavily between programs. See “More detailed benchmarking” for more information.

Methodology

My first issue was only having my laptop available for the benchmarks. It’s a decent machine, but it caused greater variance in the results, and it wasn’t very productive, since I had to stop everything I was doing to exclusively run them. The Tremor team helped me out by granting me access to their dedicated benchmarking server; here are the full specs of both machines, for reference:

Machine	CPU	RAM	Disk	Operating System
Laptop (Dell Vostro 5481)	Intel i5-8265U	16 GB @ 2667 MHz	SSD	Arch Linux, 5.18 kernel, 64 bits
Benchmarking Machine	Intel Xeon E-2278G	32 GB @ 2667 MHz	SSD	Ubuntu 20.04, 5.4 kernel, 64 bits

I ended up developing a custom script to run all the tests properly:

It first validates the configuration, including what benchmarks to run and what binaries to compare. It was quite annoying to run into a typo 2 hours into the benchmarks…
It performs a few warmup rounds before the actual benchmarks. This was specially important when using my laptop, since it started to overheat after the first few rounds, degrading its performance.

One way to ensure the variance isn’t too high is to run the exact same benchmark twice. Here’s a histogram for both machines. Unexpectedly, the server ended up having greater variance with this test, but it was good enough to continue anyway.

I would strongly recommend anyone to get a separate machine to run benchmarks. Not only will they be more reliable, but also you will be able to continue working while the benchmarks run.

NOTE: The measurements in this article are performed with the new #[repr(C)] interface, but the plugins aren’t actually loaded dynamically yet. This is because, as the planning specifies, we are on a previous step to actually making the plugins external. The final performance will be slightly worse.

Experiments

We can divide the experiments in two categories:

For tracking: the change is always going to be a good idea, because it may also clean up the code or make it more robust. We just measure its performance impact to report the results and avoid surprises.
For investigation: the change increases the complexity to explicitly improve performance, so we need to make sure it’s worth our time. This improvement may occur directly with the change, or indirectly by helping to come up with new optimizations.

Benchmarking is only essential in the latter, but I thought it would be a great idea in both cases. That way, I can write up a report like this article, explaining the improvements I worked on.

Type conversions

The most important performance concern in the first version of the plugin system had to do with the issues I had found when trying to convert external types to the C ABI. In short, there was an existing type named Value, which I needed to make FFI-safe for the plugin system. However, it was used throughout the entire codebase, and modifying its definition would’ve required a large amount of unrelated changes for just the first iteration.

My temporary fix was to create the copy PdkValue, and convert to Value and back when going through the FFI boundary. PdkValue was only used for the plugin system, making it quite easy to implement in contrast.

An obvious experiment was to fix this problem properly by making Value FFI-safe and removing PdkValue. Unless the performance degraded considerably, this was always going to be a nice change because it would simplify the code.

After the fix, I was able to investigate about the impact of these conversions with some flamegraphs:

With both Value and PdkValue (highlighted in pink are the occurrences of From, Into, and FromIterator, which add up to 9.4% of all the calls in the execution):

With a single Value (the conversion calls add up to 5.2% now, and the highlighted parts are much more sparse and harder to notice):

The flamegraphs pointed out that Tremor was spending roughly 4.2% more of its execution time just converting between Value and PdkValue. This metric is tricky, though. Spending less time on type conversions doesn’t mean that the overall program is faster. The percentages are relative to the specific execution, and it’s possible that the rest of the program just got slower in the case of a single Value.

By checking other metrics we discover that this is more or less the case. The histogram shows that while a single Value improves the latency, the overall throughput actually decreases. This compares a version without the plugin system (connectors branch) vs. first version vs. single Value.

Relative throughput comparison (higher is better):

No plugin system (%)	With `PdkValue` (%)	Single `Value` (%)
100	66.67	63.74

This showcases one of my initial mistakes. My comparison only used the passthrough benchmark, which simply forwards all input events to the output. For a more realistic use-case, I should’ve also tried with other configurations, which may include different payloads or event processing logic. In retrospect, these results should be taken with a grain of salt.

Anyway, it all makes sense to some degree. Converting Value to an FFI-safe type requires using, among others, RHashMap instead of HashMap internally. Tremor’s original HashMap was taken from the halfbrown crate, which was more performant than the standard library version, and even more than abi_stable’s. Converting between types temporarily for the FFI interface might actually be faster than using less performant types everywhere.

Double `Box`

I later came up with an improvement over the previous section. The HashMap within Value was originally stored inside a Box to reduce the overall size of the enum. In the experiment, I just converted it to a RBox and called it a day:

 enum Value {
     // ...
-    Object(Box<HashMap<String, Value>>),
+    Object(RBox<RHashMap<RString, Value>>),
     // ...
 }

However, turns out that unlike RString or RCow, RHashMap isn’t a re-implementation of the underlying type. Writing a hash table from scratch is too complex, so the author just made it an opaque type that wraps the standard library version.

Since RHashMap is an opaque type, it was already on the heap internally thanks to another RBox. Thus, the Object variant was boxed twice, which is unnecessary, since the size of RHashMap is already minimal. Having to allocate every hash table twice was probably costly, and fixing this counted as a clean-up, so it was worth trying.

The subsequent benchmarks showed a slight improvement in performance, though I was still only using the passthrough benchmark at this point. I kept the change because it didn’t make much sense to leave the doubly boxed hash table, and because it slightly improved the readability of the code. Here’s the histogram of a double box vs. a single box. Note that this histogram was recorded on the benchmarking server, while the previous one used my laptop. These tests aren’t meant to be compared between different sections.

Relative throughput comparison (higher is better):

Double `RBox` (%)	Single `RBox` (%)
100	100.82

Hash table optimization

Using a single Value improved the ergonomics of the code by sacrificing a bit of efficiency. We could try to fix this regression by creating an FFI-safe wrapper for the original external hash table implementation, which is what I did.

Tremor used halfbrown both because it’s faster for their usecase, and because it has access to lower level functionality. The raw_entry interface is only available on nightly Rust, while halfbrown exports it in its stable version. raw_entry makes it possible to memoize hashes, enabling an optimization in Tremor’s JSON handling.

NOTE: This actually isn’t a great idea in the long term, because raw_entry isn’t going to make it to a stable release^[1]. The Rust team is working on a different interface, but it will probably be similar enough that updating our raw_entry usage won’t be too much work.

This time I did it right since the beginning and tried multiple benchmarks. The results are much more varied now, and the efficiency can be analyzed from multiple points of view:

Histogram for passthrough, which simply redirects input events from a single source to the output:

Histogram for passthrough-two-inputs, which redirects input events from two different sources to the output:

Histogram for throughput-logging-json, which also implements some event processing logic over JSON data:

Histogram for throughput-logging-msgpack, which also implements some event processing logic over MessagePack data:

Relative performance for number of events processed per second (higher is better):

Benchmark	No plugin system (%)	Halfbrown, No JSON Known Key (%)	Halfbrown (%)	Hashbrown (%)
Passthrough	100	77.39	76.33	70.86
Passthrough Two Inputs	100	70.34	69.78	68.76
Throughput Logging JSON	100	65.57	64.48	69.21
Throughput Logging MsgPack	100	66.66	65.52	70.69
Average	100	69.99	69.02	69.88

They seem to point out that, with the plugin system, hashbrown (the standard hash table) might actually be better than halfbrown. The JSON optimization I mentioned could possibly be worth removing now. But it’s really hard to make a decision just out of this set of benchmarks because of the complexity of the change.

This time, the experiment is only done for investigation. These results must only be taken as an indicator of something worth investigating more in depth; it would be premature to make assumptions so early and remove halfbrown. We would need to also analyze the impact of using hashbrown without the plugin system, and understand why this happens. For example, taking a look at the flamegraphs could help in figuring out the underlying reasoning.

`abi_stable`

We can also take a look at the recurring occurrence of an experiment that isn’t worth pursuing. abi_stable imposes all kinds of overheads, apart from the type validation step before loading a plugin. I wanted to learn more about the Drop implementations, which require accessing to a vtable, and in cases like RBox, dealing with additional logic to ensure everything is safe^[2].

After zooming into specific sections, we can see this overhead by ourselves in the call stack. Here’s the flamegraph with abi_stable’s complex destructors, shown in pink and zoomed in:

However, I overestimated the importance of this problem. Taking a look at the big picture, it’s obvious that optimizing RBox’s implementation of Drop isn’t worth it yet. Here’s the flamegraph with abi_stable’s complex destructors, shown in pink and without zooming in:

I’m definitely taking note of this idea in order to investigate about it in the future for further performance squeezing. But I currently have lots of other more impactful experiments in mind with a higher priority.

Unimplemented ideas

It’s already been a year since I started working with Tremor. I extended the mentorship from three months to twelve so that I could use it as my bachelor’s thesis (see “My Final Year Project”).

But now that I have finally graduated and need to find a full-time job, I won’t be able to continue experimenting with the plugin system. My hope is that with the information in this series, the Tremor team can keep improving the plugin system so that it’s ready for deployment. The same applies to anyone reading this; hopefully you’ll have enough pointers to continue on your own after this.

Here are some ideas worth considering that I haven’t been able to carry out:

More detailed benchmarking

perf can track much more information than just the call stack. For example, perf stat provides statistics like cache or branch misses^[3]. Other profiling tools like Valgrind, AMD μProf, or Heaptrack may provide more specialized insights^[4]. The whole The Rust Performance Book is a great read for more resources and ideas.

Investigate wrapper overhead

The communication between runtime and plugins is handled through the C ABI at a lower level. To improve the ergonomics internally, the interface is accompanied by a set of wrappers, which convert the types to the standard library.

Instead of returning a Result, the following raw function uses the FFI-safe alternative RResult, which doesn’t even work with ? for error handling. This example is quite specific to abi_stable, but what’s important is that we just wrap the base function connect and transform it to an easier to use one.

Low-level interface to communicate between binaries:

#[abi_stable::sabi_trait]
pub trait RawConnector: Send {
    fn connect<'a>(
        &'a mut self,
        ctx: &'a ConnectorContext,
        attempt: &'a Attempt,
    ) -> BorrowingFfiFuture<'a, RResult<bool>>;
}

/// Alias for the FFI-safe dynamic connector type
pub type BoxedRawConnector = RawConnector_TO<'static, RBox<()>>;

High-level wrapper for what essentially is a Box<dyn RawConnector>:

pub(crate) struct Connector(BoxedRawConnector);
impl Connector {
    #[inline]
    pub async fn connect(&mut self, ctx: &ConnectorContext, attempt: &Attempt) -> Result<bool> {
        self.0
            .connect(ctx, attempt)
            .await
            .map_err(Into::into) // RBoxError -> Error::PluginError
            .into() // RResult -> Result
    }
}

Here we just convert the return types, but we can do the same for the parameters or whatever boilerplate is needed for that function. Note that this wrapper only needs to be available in the runtime crate; the plugin is only meant to implement RawConnector and doesn’t need access to Connector.

The problem is that, even though wrappers reduce the much dreaded boilerplate, they can also affect performance-critical parts considerably. connect is only called at the beginning of the program, but on_event is invoked for every single event in the source, making it our hot loop. Any small operation, like a type conversion, will have a much greater impact in there. This experiment consists on iteratively removing wrappers and looking for performance improvements.

Investigate async runtime conflicts

As far as I understand, the runtime and the plugins don’t share the same async runtime state. Every binary has its own thread pool and works independently. It would be best to share everything between runtime and plugins, though it sounds incredibly complicated, and it would most likely require contributions to the async runtimes themselves.

I also wonder what happens when the runtime and the plugin use different async runtimes, even if they are independent in the binaries. Tremor’s core is implemented with async_std, but an external plugin could freely use tokio, for example. I’ve heard that this could break in many ways, since async runtimes heavily rely on global state.

Testing different async runtimes should be easy, so it should at least be done before reaching production just to document the behavior. What I’m not so sure about is how to avoid the conflict, in case it was problematic. The fix would probably be hacky, as this is a somewhat obscure problem.

Benchmark `async_ffi`

Checking how much of a performance impact async_ffi causes sounds like a good idea. It’s actually a quite simple library; all it really does is implement opaque wrappers for the async-related types. But it’s used so often (once per call), that it may end up being noticeable.

I’m sure that if async_ffi ended up being an issue, it could be optimized internally in various ways. Furthermore, we currently use async very liberally. Only using it when strictly necessary could also help reduce the overhead.

Improve error reporting

For simplicity, errors in our interface are reported via abi_stable’s SendRBoxError, which is basically a Box<dyn Error + Send>. It’s really the only way to do it, because a plugin could have any kind of error, and we can’t know them at compile-time.

Using Box<dyn Error> instead of a concrete type makes it really hard to identify and handle specific errors. We could implement our own Error subtrait that provides more plugin-related information. An easier option is to define a more organized data structure with common kinds of errors. Here’s an example of a more organized error type, even if it still uses Box<dyn Error>:

pub struct PluginError {
    err: SendRBoxError,
    kind: PluginErrorKind
}

pub enum PluginErrorKind {
    Internal, // Implementation-defined errors we can't handle
    ConnectionFailed, // Edge cases in `connect`
    // ...
}

Reduce error handling boilerplate

In the plugin system we have to use an FFI-safe type like RResult instead of Result. Since the Try trait isn’t stable^[5], we can’t use the ? operator yet. abi_stable exports the rtry macro as a substitute for ?, but in my experience it introduces noise in the code, making it uglier and harder to maintain.

Depending on how much Try is going to take to stabilize, creating a procedural macro might be worth our time. It would just replace ? with rtry or whatever is configred, which should be somewhat simple to implement. I’m surprised I couldn’t find an existing crate for that. Readability would improve in longer functions with lots of error handling:

+#[try_with(rtry)]
 fn fallible() -> RResult<()> {
-    if rtry!(fallible_op_1()) && rtry!(fallible_op_2()) {
-        rtry!(fallible_op_3());
+    if fallible_op_1()? && fallible_op_2()? {
+        fallible_op_3()?;
     }

     ROk(())
 }

Admittedly, the example above is an oversimplification. In reality, a single function may mix both Result and RResult, or require type conversions. The macro would probably end up being a bit more complex, but it might be worth considering regardless.

Reduce async boilerplate

async_ffi would really benefit from a procedural macro as well. It always requires using async move { /* ... */ }.into_ffi(), which is quite a bit of boilerplate and increases the indentation level by one.

Without the procedural macro:

use async_ffi::{FfiFuture, FutureExt};

#[no_mangle]
pub extern "C" fn work(arg: u32) -> FfiFuture<u32> {
    async move {
        let ret = do_some_io(arg).await;
        do_some_sleep(42).await;
        ret
    }
    .into_ffi()
}

With the procedural macro, especially noticeable in more complex functions:

use async_ffi::async_ffi;

#[no_mangle]
#[async_ffi]
pub async extern "C" fn work(arg: u32) -> u32 {
    let ret = do_some_io(arg).await;
    do_some_sleep(42).await;
    ret
}

I already opened an issue about this with more details for whoever wants to give it a try:

Procedural macro for boilerplate oxalica/async-ffi#12

Improve cross-platform support

As I mentioned in earlier articles, our plugin system will only work on Windows, macOS, and Linux^[6]. It will still compile on other platforms, but possibly with data races in the dynamic linking internals.

Specifically, libloading, which is used by abi_stable, states that its error handling isn’t fully thread-safe on some platforms^[7], such as dlerror on FreeBSD. Its only consequence should be garbage error messages, but I still wouldn’t risk it. There are two ways to approach this:

Quick fix: add a compile-time error for any platform that isn’t explicitly supported. Before adding support for a new platform, we will have to manually ensure that its error handling for dynamic loading is thread-safe.

// Ideally, there should be a comment here with references that state the
// thread-safety of the supported platforms.
#[cfg(not(any(unix, windows, target_os = "macos")))]
compile_error!(
    "This platform isn't currently supported. Please open a ticket on GitHub."
);

Proper fix: fix the data races upstream with an internal mutex. This is what the alternative dlopen does, though in their case they always have the mutex, which is unnecessary in some platforms^[8].

Performance impact of panic handling

abi_stable has to track all panic occurrences so that they don’t propagate through the FFI boundary. Otherwise, as discussed in other articles^[9]^[10], we would be invoking undefined behaviour. async_ffi also has special handling for panics, which overcomplicates the crate a bit.

There are two interesting experiments to try:

Comparing the performance of panic = "abort" and panic = “unwind”. Technically, aborting should optimize away most of the extra code^[11], but it should only be used for research, as it would make debugging in production incredibly hard.
Preparing abi_stable and async_ffi for the stabilization of extern "C-unwind", thanks to which all the additional panic handling could be removed. Unlike aborting, we would still be able to debug errors just fine.

For now, it’s only available on nightly without a certain release date, but it will surely be useful in the future, and we could measure how much panic handling costs. Testing new features like that also speeds up their release, since we could share feedback with the project group.

Use `-Z randomize-layout` to find FFI bugs

The unstable compiler flag -Z randomize-layout randomizes the layout of #[repr(Rust)] types. The Rust ABI is unstable: it explicitly doesn’t specify much about the type layouts, and we must not rely on them. However, in practice they’re usually consistent, at least within the same compiler version^[12], so these errors can be hard to catch without a tool like this.

We use #[repr(C)] for the plugin system, so this flag shouldn’t cause any issues… unless we’re mistakenly interacting with the Rust ABI. In that case, the program would crash in random and unexpected ways, pointing out that there’s something wrong. It certainly won’t be a pleasant debugging experience, but it’s better than having it happen in production.

This may even catch other bugs unrelated to the project. Tremor implements self-referential types and other optimizations, and it’s possible that some of them incorrectly rely on the Rust ABI.

Tremor’s Continuous Integration tests could be run with -Z randomize-layout to ensure that no opaque types export Rust types, even if abi_stable prevents most cases within the plugin system.

Try raw dynamic loading

What I didn’t know at the beginning of this journey is that the hardest part would be making everything #[repr(C)]. Using abi_stable is certainly very useful for types like RVec and to create custom types, but at times I find the library too much.

A few developers, including myself, think that it would be best to have separate libraries for all the utilities abi_stable provides, rather than bundling everything in there. If they became modularized, making it the “community standard” would be easier, and we could have compatible alternatives for different preferences.

It also boils down to just having more support from the community. abi_stable is an incredibly complex library, and I want to give props to the legend rodrimati1992 for creating and maintaining it. But such complexity makes it a scary crate to contribute to, which I know first-hand. Turning it into smaller crates would really help decentralize the work, in my opinion.

Anyway, the point of this experiment is that abi_stable isn’t a hard requirement for the plugin system. Using it definitely makes its implementation easier, but also introduces many overheads. Its main selling points are usability, safety, and backward compatibility, though with enough effort and care, we could write the plugin system by ourselves just with libloading.

It would be super useful to measure how much performance could be squeezed out by removing abi_stable. This doesn’t need to be done over the full plugin system; we could benchmark smaller prototypes, like the ones I host at marioortizmanero/pdk-experiments.

Simplify the interface further

I’m not fully satisfied with what the final interface for the plugin system looks like either. For the first version, all I did was change as little as possible so that we could have a working prototype. But there are many parts that could be simplified with dedicated refactors.

Everything could be reorganized and cleaned up a bit, especially the plugin initialization and the repetitive opaque types. We should continue to minimize the use of complex communication patterns like channels. These improvements can be worked on iteratively, because you will only come up with new ideas over time.

For the long-term future

Our short-term goals focus on either usability or performance. However, there are more ways in which our plugin system could keep evolving afterwards:

Supporting more components in the plugin system. We could keep splitting up the executables into even smaller pieces, continuing to improve the overall flexibility and compilation times.
Refining the user experience by making the plugin loading smarter and easier to use. In the far future, we could even have a centralized repository with all the plugins, so that it’s as easy as possible to download and add them. You can think of crates.io or hub.docker.com.
Improving the developer experience is also essential. Anyone should be able to implement their own plugin, but ours is currently too tied to the internals of Tremor. We could create project templates, properly document everything, write examples, etc.
Security is something we sacrificed when we chose dynamic loading as our base technology. We don’t have a sandbox, which would be essential if a plugin repository were to exist.
Restarting or unloading plugins at runtime. Unloading is almost impossible, given that abi_stable explicitly doesn’t support it^[13]. But restarting could work by simply stopping and leaking an existing plugin, and then loading it again with the same config. It would improve error resilience considerably, as we would be able to restart plugins upon failure (cautiously, or we could run out of memory after leaking too much).

Conclusion

That was all! In summary, our new FFI interface initially slowed down the program by around 36%. In a few iterations, we’ve reduced that number to around 30%, and there are still many ideas left to try.

The complexity of the whole project has turned out to be much greater than we predicted at the beginning. I still remember when we were unaware of the ABI instability and thought we could get away with an interface in pure Rust. Thus, I haven’t been able to attain some goals specified initially, and there’s still a bit of work left before reaching production, mainly related to performance.

However, I’m sure my implementation will serve as a great base for what the plugin system may evolve into in the future! And hopefully, this series and my contributions (see “Open Source Contributions”) will make plugin systems more accessible, even outside Tremor.

Doing all of this in an open source environment has been enormously rewarding. Even if you’re working for a company with propietary software, please try to contribute upstream instead of forking or patching. Try to pay it back to those who save you from so much work, and submit a PR or an issue.

This project has been possible thanks to Darach, Heinz, and Matthias, my mentors in the Tremor team ❤️. Thanks to Wayfair as well for supporting open source, and for funding this project. And one final thanks to the Rust community for motivating me to continue with the series, and for providing such splendid libraries and tools.

Please leave a comment below for any questions or suggestions you may have. Also let me know of new advances on any of the ideas above, so that I can add a link for future readers. I would love to see your plugin system creations and results :)

Appendix A: Open Source Contributions

One of my favorite parts of the project has been contributing so much to all kinds of open source dependencies, so I’ve maintained a list of its occurrences. Some are more important than others, but it’s still a decent metric for my results. This skips the issues or pull requests that:

Contributed nothing (e.g., asking questions or discarded ideas).
Were repetitive (e.g., I made a few identical PRs in Tremor when I was fixing problems with Git).

External Contributions

These include repositories not directly related to Tremor:

Internal Contributions

Here are the issues and pull requests created within Tremor’s repositories, including those for the plugin system and other unrelated improvements:

Appendix B: Other Achievements

Breaking the Compiler

I also managed to break the Rust compiler while working on this plugin system. It may not be as rare as one would think, but for some reason I felt oddly proud to achieve it, so I’ll share it here :)

It’s seemingly related to incremental compilation, and someone had already reported it before. It should be fixed in a future version, and I haven’t come across it again.

LFX Mentorship Showcase

I already shared this in a previous article, but for completeness I’ll repeat it here. This online event made it possible to showcase my work back in January with a quick 15-minute presentation. I couldn’t get into many technical details, but I’m sure it will be useful to anyone considering participating in a LFX Mentorship or in Google Summer of Code.

My Final Year Project

I have finally recently submitted this as my bachelor’s Final Year Project. This document takes a more academic approach, and I rigorously reorganized everything so that even developers unfamiliar with Rust can understand it. The abstract is in English, but unfortunately, the rest is in Spanish due to absurd university rules.

KubeCon + CloudNativeCon 2022

Thanks to the Tremor team, I was also able to presentially attend KubeCon + CloudNativeCon 2022 in Valencia, Spain! It was my first conference and I was very pleasantly surprised by how nice everyone was. I had tons of fun and met smart folk with all kinds of backgrounds. If you’re on the fence about attending something similar, I strongly recommend you to go for it!

Paella! From my LinkedIn profile:

Footnotes

Plugins in Rust: Getting our Hands Dirty

2022-02-11T00:00:00Z

Table of Contents

Welcome to one of the last articles of this series! Previously, we covered how to use external dependencies to lessen the work necessary to implement our plugin system. Now that we know how to actually get started, we’ll implement it once and for all.

I will personally use the crate abi_stable, but the concepts should be roughly the same for any dynamic loading method. Similarly, some of my advice is related to modifying an already existing project of large size. If you’re starting from scratch with a plugin system in mind, it should be the same, just easier in some aspects.

Some prototyping advice

Before anything else, I’d like to give out some suggestions regarding refactoring with Rust, based on my experience. You’ll be introducing a lot of changes, so it would be wise to do some brief planning beforehand — otherwise it’ll take forever with a complex enough codebase. Maybe this will all be obvious to you, or you may prefer taking a different approach. But it certainly worked for me, so it’s worth a shot. I’m personally very satisfied with how much I’ve learned about the process.

The “just make it work” approach

What I personally struggled the most to get drilled into my head is not getting lost in the details. First of all, we just want it to work. I’ll repeat. First of all, we just want it to work. As long as the plugin system runs, the following are completely fine:

Ugly code (non-idiomatic, repetitive, somewhat messy, etc)
Non-performant code
Poor documentation
No tests
No clippy

Or, more specific examples:

If stuck with lifetimes, just slap a .clone() in there and call it a day.
A good ol’ .unwrap() will make your life easier in case of doubt.
You can resort to “dirty” or unnecessary type conversions if necessary.
Just pulling in a new dependency instead of implementing something more tailored to your use-case from scratch will save you a lot of time.
If something that’s not really important is taking you too much time, you can always resort to a // TODO or // FIXME.

I will say that not working on tests until the plugin system is running is definitely arguable for those who prefer to follow a test-driven development. However, I personally didn’t feel the need to write any tests in this case; thanks to Rust’s strong type system, it was mostly a compiler-driven development. My progress basically consisted on making some changes and then trying to make the compiler happy about them, repeatedly. I only moved to the testing step once I was happy enough with the results, and everything seemed to work when running it manually.

Furthermore, premature optimization is the source of all evil. You’re not at a point where that’s important yet. Once you’re done, you can actually spend some time profiling and benchmarking in order to know which optimizations are worth your time. Don’t get me wrong — you can certainly worry about choosing an approach that’s appropriate in terms of performance, which is why I discarded WebAssembly or IPC at the beginning. But definitely give up on trying to avoid a clone that might not actually affect your program speed after all.

My point is that every test, clean-up or optimization you try to make at this stage will very likely end up being wasted effort. You’ll reach a blocker at some point later on, and have to rethink and rewrite lots of stuff. When everything compiles, seemingly runs fine, and you consider it a decent approach, you can actually start working on these points. Just keep note of what can be improved so that you don’t forget.

Note that it is okay to “waste” effort on wrong approaches, because you aren’t really “wasting” it; they’re a necessary step to arrive to the final solution. But it’s painful to delete code you’ve spent time thinking about, so we should at least try to minimize it.

Specific to Rust

Disable the annoying warnings

The first thing I did was remove some warnings the compiler will print when you’re prototyping. By default, a Rust program still compiles with warnings, but in my experience, they’re always in the way when trying to deal with actual errors. For example, I don’t care that an import is unused at all. I’ll fix it once I’m done, but please let me keep trying to “make this work”!

// TODO: disable and cleanup with `cargo fix` automatically once done
#![allow(unused)]
#![allow(dead_code)]

Note that I’m not using #![allow(warnings)], because most warnings will actually be really helpful when refactoring to avoid early bugs. We just want to get rid of the annoying ones, such as unused functions or imports, or dead code that we haven’t cleaned up yet, which are usually harmless.

One month later, I was able to finally compile tremor-runtime with the initial prototype. This screenshot made my day, as simple as it may seem:

Afterwards, I removed these allow statements, ran cargo fix && cargo fmt, and cleaned up a bit. cargo fix is built-in, and will automatically take care of the trivial warnings, such as unused imports, saving lots of time doing cumbersome work.

Don’t touch previous `use` statements

As my Pull Request got bigger and bigger, I started running into problems with Git. The parent branch was also undergoing changes from a team of three people, so it was updated quite often. All of this resulted into frequent conflicts, and having to painfully resolve them many times until I got the hang of it.

Say, you have a number of use statements at the top of your file. After adding some changes related to the plugin system, you now need to import PdkValue as well. You’d usually do:

-use tremor_value::{Value, ValueTrait};
+use tremor_value::{Value, ValueTrait, PdkValue};

The problem is that if the parent branch later adds or removes one of the items in that use statement, you may encounter a conflict. Git isn’t smart enough to handle Rust import statements (although that’d be pretty cool). Multiply this by 100 statements, and you might eventually go crazy. You can keep your sanity by adding the new item in a separate block of uses instead:

-use tremor_value::{Value, ValueTrait};
+
+use tremor_value::PdkValue;

Same thing may happen if tremor_value wasn’t imported and you have to add a new line to the already existing use block:

 use more_stuff::{X, Y, Z};
+use tremor_value::PdkValue;
 use other_stuff::{A, B, C};

In my experience, Git also trips up with this sometimes. You can follow the same advice:

 use more_stuff::{X, Y, Z};
 use other_stuff::{A, B, C};
+
+use tremor_value::PdkValue;

In summary, you can just create a new block after all the previous uses and add your stuff in there. You can clean it up at the end or in a different pull request.

This is harder to do if you’re removing use statements. But if you also followed the previous section about ignoring annoying warnings, it won’t be a problem in the first place. You can just leave the unused imports and clean up when you’re done with the PR.

Defining the plugin interface

The first step that we can do is define the interface of the plugin system, i.e., what a plugin binary must implement in order to be loadable by the runtime. If you’re doing this over an already existing codebase, you’ll probably get tons of errors. We’ll ignore them for now; this is only our first sketch, and you’ll end up changing it a thousand times anyway. Some types in the interface may not exist yet, or they may not be meant to be used for FFI. But it’ll serve us as an initial list of things to work on.

In my case, it first looked as follows. The specifics about how this works with abi_stable are explained in the previous post.

/// This type represents a connector plugin that has been loaded with
/// `abi_stable`. It serves as a builder, making it possible to construct a
/// trait object of `RawConnector`.
#[repr(C)]
#[derive(StableAbi)]
#[sabi(kind(Prefix))]
pub struct ConnectorMod {
    /// the type of the connector
    pub connector_type: extern "C" fn() -> ConnectorType,

    /// create a connector from the given `id` and `config`
    ///
    /// # Errors
    ///  * If the config is invalid for the connector
    #[sabi(last_prefix_field)]
    pub from_config: extern "C" fn(
        id: RString,
        config: ROption<Value>,
    ) -> FfiFuture<RResult<BoxedRawConnector>>,
}

/// Marking `ConnectorMod` as the main module in this plugin. Note that
/// `ConnectorMod_Ref` is just a pointer to the prefix of `ConnectorMod`.
impl RootModule for ConnectorMod_Ref {
    /// The name of the dynamic library
    const BASE_NAME: &'static str = "connector";
    /// The name of the library for logging and similars
    const NAME: &'static str = "connector";
    /// The version of this plugin's crate
    const VERSION_STRINGS: VersionStrings = package_version_strings!();

    /// Implements the `RootModule::root_module_statics` function, which is the
    /// only required method for the `RootModule` trait.
    declare_root_module_statics! {ConnectorMod_Ref}
}

My task was to turn the Connector trait into a plugin. All the Connector implementors were meant to be constructed with ConnectorBuilder, and from that moment on Tremor was able to use them generically with dyn Connector. Thus, I thought the best idea would be to turn ConnectorBuilder into my RootModule under the name ConnectorMod. Once the plugin was loaded, it would be possible to construct the connector with the from_config function. I used abi_stable::sabi_trait’s macro functionality, making dyn usage possible within FFI.

For reference, when I first wrote ConnectorMod, Value wasn’t even #[repr(C)]. I had also added the #[sabi_trait] attribute to the RawConnector trait declaration, but the types used there weren’t #[repr(C)] either. So I had tons of errors everywhere, but that was OK. I would be working on them step by step until it compiled again.

If you’re using libloading directly then you would be implementing the interface via a struct with function pointers instead, and you’d need to store metadata about the plugin with constants. But in the end, it boils down to the same thing; just with different amounts of boilerplate.

Recursively making everything `#[repr(C)]`

Now, this is the actually complicated part. The previous step may have seemed simple, but you might find yourself falling into madness as you realize that you need to make all the types in your interface #[repr(C)], and also all the fields each of these types hold, and so on…

It’s very likely that you’ll eventually find types without an FFI alternative in abi_stable. These will most likely be external types, but things like async are a bit complicated to deal with as well. In the case of Tremor, the most problematic part was the Value type. It’s used to represent a JSON-like payload; roughly defined as follows:

pub enum Value {
    /// Static values (integers, booleans, etc)
    Static(StaticNode),
    /// String type
    String(String),
    /// Array type
    Array(Vec<Value>),
    /// Object type
    Object(Box<HashMap<String, Value>>),
    /// A binary type
    Bytes(Vec<u8>),
}

In order to be able to use Value in the plugin system, it can be converted to:

#[repr(C)]
#[derive(StableAbi)] // Only necessary for abi_stable
pub enum Value {
    /// Static values (integers, booleans, etc)
    Static(StaticNode),
    /// String type
    String(RString),
    /// Array type
    Array(RVec<Value>),
    /// Object type
    Object(RBox<RHashMap<RString, Value>>),
    /// A binary type
    Bytes(RVec<u8>),
}

The first problem arises in the Static variant: StaticNode is a #[repr(Rust)] external type. It’s from our value_trait dependency, and it may hold different basic types: numbers, booleans, or just nothing.

pub enum StaticNode {
    I64(i64),
    U64(u64),
    F64(f64),
    Bool(bool),
    Null,
}

This could be fixed by simply applying the very same procedure again (hence recursively until everything is #[repr(C)]). Here it will finally work because there aren’t any other #[repr(Rust)] types in StaticNode:

#[cfg_attr(feature = "abi_stable", repr(C))]
#[cfg_attr(feature = "abi_stable", derive(abi_stable::StableAbi))]
pub enum StaticNode {
    I64(i64),
    U64(u64),
    F64(f64),
    Bool(bool),
    Null,
}

Since it’s an external library, we’ll have to make a Pull Request and hope that the author is okay with the changes. abi_stable should be optional so that this change is applied only to those that actually need #[repr(C)] in the library. You could also go a step further and differentiate between enabling #[repr(C)] and deriving StableAbi for those that don’t need the latter.

Add support for StableAbi simd-lite/value-trait#14

Overcoming problems with `#[repr(C)]`

Awesome. We got Value working now for FFI. Right? No? Oh. It seems like the compiler strongly disagrees. By changing the variants of Value, a great amount of the code that used it will now fail to compile in multiple ways:

// Won't work because Value::Array holds an RVec now
let value = Value::Array(Vec::new());

That’s the easiest one: we just need to change Vec to RVec and it should be fine. The types in abi_stable are meant to be a drop-in replacement for the ones in std:

let value = Value::Array(RVec::new());

It gets a bit more complicated when the old types are exposed in methods, because you have to decide whether to expand the FFI boundary from the internals of Value to the users of Value. For instance, the Value::Object variant holds a RHashMap now, but the method Value::as_object used to return a reference to a HashMap. You’ll get another error in there, which raises a decision that must be made: returning RHashMap or adding an internal conversion to HashMap.

impl Value {
    // Original code
    fn as_object(&self) -> Option<&HashMap<String, Value>> {
        match self {
            // Problem: `m` is a `RHashMap` now, but the function returns a
            // `HashMap`.
            //
            // Solution 1: change the return type to `RHashMap`
            // Solution 2: convert `m` to a `HashMap` with `m.into()`
            Self::Object(m) => Some(m),
            _ => None,
        }
    }
}

If the return type is changed to RHashMap, almost every caller to as_object in the program will now fail to compile because they expect a HashMap. You’ll have to clean it up one by one and figure out how RHashMap can be used in that case instead.

This can be messy because in order to avoid conversions, your plugin system will infect the entire codebase. You may quickly find yourself propagating the usage of RHashMap everywhere, even when the system isn’t that important. For example, Value also appeared in Trickle’s implementation, the scripting language used to configure Tremor’s pipelines. Having to use RHashMap in there was a bit confusing, and I was modifying lots of files unrelated to the plugin system.
If you perform an internal conversion to HashMap in as_object we’ll avoid all of these errors, at the cost of adding a small overhead. It’s by far the easiest choice, but if Value::as_object is frequently used in, e.g., your hot loop, you may notice a considerable performance degradation.

I already investigated in the past, and the good news is that converting between types in std and abi_stable is $O(1)$. Most of the times it’s equivalent to a pointer cast or a match. So here’s when the “just make it work” approach is useful: we’ll just keep the FFI boundary minimal and add conversions as early as possible. After we’re done, we’ll see if there are any performance issues, and then work on them.

Reaching `#[repr(C)]` blockers

That was my first attempt at making Value FFI-compatible, and unfortunately, it didn’t end there. Converting from std to abi_stable is a relatively painless experience; their usage intends to be the same. The only issue I found in that regard is that some methods from std weren’t available in abi_stable yet because it’s not updated as regularly. Usually, you can just copy-paste the implementation from std into abi_stable's and create a new Pull Request, which is what I did a few times:

Add support for .keys() and .values() in RHashMap rodrimati1992/abi_stable_crates#58

Implement Index for slices and vectors rodrimati1992/abi_stable_crates#59

Support for f32 and f64 rodrimati1992/abi_stable_crates#61

Implement ROption::as_deref rodrimati1992/abi_stable_crates#68

Implement RVec::append rodrimati1992/abi_stable_crates#70

Implement ROption::{ok_or,ok_or_else} rodrimati1992/abi_stable_crates#82

However, this stops being as “easy” when you have to convert from an external library to abi_stable. I lied at the beginning of the article: the declaration of Value was an oversimplification. For performance reasons, Tremor actually uses halfbrown's implementation of a hash map instead of std::collections::HashMap.

NOTE: halfbrown is based on hashbrown, which was, in fact, merged into the standard library at some point^[1]. Although with this plugin system we’re suffering the consequences of not having a stable ABI, seeing that it enables things like that makes me less bitter.

halfbrown has some additional functionality over std’s implementation. Some of it is actually available on Nightly, but for that reason it’s not meant to be in the stable RHashMap either. This extra functionality is used in Tremor for example with raw_entry. There is an optimization for JSON handling that consists on memoizing the hash of a known item in a map in order to access its value directly with it^[2]. After switching to RHashMap, this becomes an impossible task.

Even if I managed to fix the hash map mess, the same story repeats itself for Cow. Tremor uses beef's Cow instead of std::borrow::Cow because it’s faster and more compact, at the cost of a slightly different usage.

There are a few possible ways to approach these kinds of issues:

Approach 1: Avoid the type in the first place

As always, we’ll try to follow the “just make it work” advice in here. It’s a perfectly valid solution to just comment out the optimizations and add a // TODO above so that they can be reviewed later. You might be asking for too much complexity in your plugin system; limiting yourself to the functionality in std may be more than enough for now. You’ll now see that it’s always possible to fix this properly. It just might be too much effort at the moment.

In Value’s specific case, it seems like removing the optimizations that are giving us trouble is the easiest way to fix this problem. And it would, if removing code wasn’t also tricky when the optimizations aren’t performed transparently to the hash map functionality. After trying to remove them I eventually gave up because I considered it was getting out of hands with so many changes.

These are 120 errors after attempting to remove the optimizations, most of them about lifetime hell (more about this at the end of the article):

Approach 2: Implement a wrapper

Another possibility is to write a wrapper for halfbrown. Opaque types, for instance, may be used to wrap the functionality of an underlying type that’s not FFI-safe, as I covered in previous articles. This is what abi_stable does in its external_types module for crates like crossbeam or parking_lot.

However, as you may see with the already existing examples, implementing wrappers can be quite a cumbersome task. And even after you’re done you’ll have to keep them up to date, so this will increase your maintainance burden. halfbrown and beef are somewhat complex libraries, so I decided this wasn’t the best choice at that moment for Value. I did use this approach a lot in other cases, so I’ve included an example in a later section.

Approach 3: Re-implement with `#[repr(C)]` from scratch

Similar to implementing a wrapper, but on steroids. It might seem like overkill, but as far as I know it’s the only choice in some scenarios, because we can make sure the type is as performant as it can get. In Value’s example, the problematic types are part of optimizations, so writing a wrapper for them may have a performance hit and render them useless (e.g., if we used opaque types we would introduce at least a mandatory pointer indirection).

If this part of the project is important enough, or you consider that there’s enough manpower, then it might not be such a bad idea to create a new implementation with your use-case in mind. It’s actually why Tremor’s Value was created in the first place; simd_json::Value wasn’t flexible enough for them, so they decided to define their own version. Same thing could be applied for your plugin system.

Approach 4: Simplifying the type at the FFI boundary

The last idea I came up with was the easiest one: creating a copy of Value meant to be used only for FFI communication, PdkValue:

Since it’s a new type, we won’t run into the breaking changes I showed in the Avoid the type in the first place section, and it’s quite easy to implement:

#[repr(C)]
#[derive(StableAbi)]
pub enum PdkValue {
    /// Static values (integers, booleans, etc)
    Static(StaticNode),
    /// String type
    String(RString),
    /// Array type
    Array(RVec<PdkValue>),
    /// Object type
    Object(RBox<RHashMap<RString, PdkValue>>),
    /// A binary type
    Bytes(RVec<u8>),
}

We don’t need to write any additional methods for the new PdkValue, only its conversions to and from the regular Value. This would be equivalent to, instead of passing a Vec to the interface, replacing it with a *const u8 for the data and a u32 for the length. We’re just simplifying the types at the FFI boundary, and then maybe converting them back for access to the full functionality.

The problem in my case is that these conversions are now $O(n)$ instead of $O(1)$, because I have to iterate the objects and arrays in order to convert its inner values as well:

impl From<Value> for PdkValue {
    fn from(original: Value) -> Self {
        match original {
            // No conversion needed; `StaticNode` implements `StableAbi`
            Value::Static(s) => PdkValue::Static(s),
            // This conversion is cheap
            Value::String(s) => PdkValue::String(s.into()),
            // This unfortunately requires iterating the array
            Value::Array(a) => {
                let a = a.into_iter().map(Into::into).collect();
                PdkValue::Array(a)
            }
            // This unfortunately requires iterating the map and a new
            // allocation
            Value::Object(m) => {
                let m = m.into_iter().map(Into::into).collect();
                PdkValue::Object(RBox::new(m))
            }
            // This conversion is cheap
            Value::Bytes(b) => PdkValue::Bytes(conv_u8(b)),
        }
    }
}

// Same as before, but inversely
impl From<PdkValue> for Value {
    fn from(original: PdkValue) -> Self {
        match original {
            PdkValue::Static(s) => Value::Static(s),
            PdkValue::String(s) => Value::String(s.into()),
            PdkValue::Array(a) => {
                let a = a.into_iter().map(Into::into).collect();
                Value::Array(a)
            }
            PdkValue::Object(m) => {
                // No Box dereference move magic with RBox, we call `into_inner`
                let m = RBox::into_inner(m);
                let m = m
                    .into_iter()
                    .map(|Tuple2(k, v)| (k.into(), v.into()))
                    .collect();
                Value::Object(Box::new(m))
            }
            PdkValue::Bytes(b) => Value::Bytes(conv_u8_inv(b)),
        }
    }
}

And then we can use it like:

// This is implemented in the plugin. It will work because `PdkValue` is
// FFI-safe.
pub extern "C" fn plugin_stuff(value: PdkValue) {
    let value = Value::from(value);
    value.do_stuff()
}

// This is implemented in the runtime
fn runtime_wrapper(value: Value) {
    plugin_stuff(value.into());
}

Surprisingly easy to get working! Two problems, though:

Usability: the snippet of code above shows that the conversions introduce some noise in our code, as opposed to having a single #[repr(C)] Value. It could be worse, but it can get annoying when you have many Values in your runtime and have to pass them to the plugins.

This is actually trivial enough to simplify with a macro — perferrably procedural — at the cost of introducing more complexity. We’re currently following “just make it work” so we’ll leave that for the future.
Performance: passing the value to the plugin and converting it back in there means we’re iterating the data twice. With experiments that I’ll include in the next article, I found out that these conversions make up 5 to 10% of Tremor’s execution time. Which is less than I expected, but still not good enough for production.

For the first version of the plugin system, this is the solution that I ended up using for Value. Most of the time I spent was just trying the different available approaches, and this one won in simplicity by a huge margin. After being done, I would be able to decide if the performance hit was bad enough, and then maybe switch to a different one. Creating PdkValue wasn’t a big time investment, so it wouldn’t really feel like a waste.

Asynchronous plugins

`async` functions

I’ve always wanted to avoid communication primitives between plugins and runtime other than plain synchronous calls. But this might be inevitable if your program uses asynchronous programming heavily, which is the case of Tremor. Anyhow, turns out that using async in FFI isn’t that complicated!

In the previous post I introduced the async_ffi crate, which exports FFI-compatible Futures. It’s quite easy to use; here’s an example from the docs:

use async_ffi::{FfiFuture, FutureExt};

#[no_mangle]
pub extern "C" fn work(arg: u32) -> FfiFuture<u32> {
    async move {
        let ret = do_some_io(arg).await;
        do_some_sleep(42).await;
        ret
    }
    .into_ffi()
}

The types in async_ffi implement Future, so invoking that function is as easy as usual: just adding .await after the function call.

It’s admittedly a bit ugly to use async move { }.into_ffi() everywhere, specially because it increases the indentation in one level. But that’s something that can be fixed with a procedural macro in the future:

Procedural macro for boilerplate oxalica/async-ffi#12

The only problem I found was that the futures didn’t implement StableAbi, so it wasn’t possible to use them with abi_stable. It took me a while to understand the crate, but it’s nothing a Pull Request can’t fix:

Support for abi_stable oxalica/async-ffi#10

One concern here may be performance. I imagine that it’s not a huge problem because the crate is actualy quite small and only introduces some pointer juggling. I will confirm this in the next post with some benchmarks, though.

Channels

Tremor also needs channels for asynchronous communication. For example, a connector may need to indicate the runtime that the connection has been lost at any point of its execution. We can’t delay that until the next synchronous call from the runtime because we don’t know when that might happen, and we want to keep Tremor low-latency.

We have three options here:

Option 1: Use `abi_stable`’s alternatives

Turns out abi_stable includes an FFI-safe wrapper for crossbeam. We could just switch the usage of Sender<T> to RSender<T> and that’s it.

Problem: Tremor actually uses asynchronous channels, such as async_std::channel, so it wasn’t as easy as changing to crossbeam. We want to be able to poll for events without blocking the thread.

Option 2: Callbacks

If your use-case is simple enough, callbacks might be sufficient. I experimented a bit with them and it’s a good alternative if you:

Don’t need to listen for events actively (calling rx.recv())
Don’t need to access much data other than the callback’s parameters. Note that it’s a function and not a closure, so you’d need globals, and that can get nasty real quick.

The type definition in the shared crate:

#[repr(C)]
pub struct ConnectorContext {
    id: i32,
    callback: extern "C" fn(i32),
}

The callback definition and the plugin loading in the runtime:

// This will run a few plugins concurrently.
pub fn run() {
    // You still have access to some resources in the runtime, but don't
    // abuse it!
    static COUNT: AtomicI32 = AtomicI32::new(0);
    extern "C" fn callback(x: i32) {
        println!("raw callback invoked! {x:?}");
        COUNT.fetch_add(1, Ordering::Relaxed);
    }

    let mut handles = Vec::new();
    for id in 0..NUM_THREADS {
        handles.push(thread::spawn(move || {
            let ctx = ConnectorContext { id, callback };
            plugin_fn(ctx);
        }))
    }

    for handle in handles {
        handle.join().unwrap();
    }

    println!("Final count: {}", COUNT.load(Ordering::Relaxed));
}

The functionality implementation in the plugin:

// This only invokes the callback with its own ID.
pub extern "C" fn plugin_fn(ctx: &ConnectorContext) {
    // Asynchronous communication!
    thread::spawn(move || {
        (ctx.callback)(ctx.id);
    });
}

You can even wrap the function pointer up so that the usage can remain as sender.send(value):

pub struct Sender<T> {
    callback: extern "C" fn(T),
}

impl<T> Sender<T> {
    pub fn new(callback: extern "C" fn(T)) -> Self {
        Self { callback }
    }

    pub fn send(&self, t: T) {
        (self.callback)(t);
    }
}

However, the use-cases for this are very limited. If you don’t really need to access anything in the runtime you might as well just have a regular function in the shared crate. And if you need more complex functionality you might be better off with a regular channel.

Option 3: Opaque types

We can resort to opaque types for anything we can’t remove or simplify. This is what I did in order to have asynchronous channels available on the plugins.

The interface in the shared crate, using abi_stable:

#[abi_stable::sabi_trait]
pub trait SenderOpaque: Send {
    /// Send a message to the runtime
    fn send(&self, reply: Reply) -> BorrowingFfiFuture<'_, RResult<()>>;
}

/// Alias for the FFI-safe sender, boxed
pub type BoxedSender = SenderOpaque_TO<'static, RBox<()>>;

The plugin loading in the runtime:

impl SenderOpaque for async_std::channel::Sender<Reply> {
    fn send(&self, reply: Reply) -> BorrowingFfiFuture<'_, RResult<()>> {
        async move {
            self.send(reply)
                .await
                .map_err(|e| RError::new(Error::from(e)))
                .into()
        }
        .into_ffi()
    }
}

fn runtime() {
    // Non FFI-safe type
    let (tx, rx) = async_std::channel::unbounded();

    // Maybe multiple times in different asynchronous tasks
    let reply_tx = BoxedSender::from_value(tx, TD_Opaque);
    library.plugin(reply_tx);

    // Waiting for events
    while let Ok(reply) = rx.recv().await {
        println!("Got a reply from a plugin: {reply:?}");
    }
}

The functionality implementation in the plugin:

pub extern "C" fn plugin(sender: BoxedSender) {
    // Asynchronous communication!
    task::spawn(async move {
        sender.send(Reply::Fail).await.unwrap()
    });
}

Loading plugins

Loading plugins in the runtime was actually the easiest part. abi_stable takes care of most of the work; the only thing that was missing for me is a way to find the plugins. This greatly depends on your program: you may hardcode the location, have them passed as CLI arguments, etc.

In the case of Tremor, we wanted to make it possible to configure the directories where the plugins may be saved. Thus, I introduced a new environment variable TREMOR_PLUGIN_PATH. It’s equivalent to PATH in the sense that the directories are separated by colons.

Once the runtime has a list of what directories may contain plugins, it can look for them recursively with the crate walkdir. It’s highly customizable and remarkably easy to use:

/// Recursively finds all the connector plugins in a directory. It doesn't
/// follow symlinks, and has a sensible maximum depth so that it doesn't get
/// stuck.
pub fn find_recursively(base_dir: &str) -> Vec<ConnectorMod_Ref> {
    WalkDir::new(base_dir)
        // No symlinks are followed for now
        .follow_links(false)
        // Adding some safe limits
        .max_depth(1000)
        .into_iter()
        // Ignoring permission errors
        .filter_map(Result::ok)
        // Only try to load those that look like plugins on the current platform
        .filter(|file| {
            file.path()
                .extension()
                .map(|ext| ext == env::consts::DLL_EXTENSION)
                .unwrap_or(false)
        })
        // Try to load the plugins and if successful, add them to the result.
        // Not being able to load a plugin shouldn't be fatal because it's very
        // likely in some situations. Errors will just be printed to the logs.
        .filter_map(|file| match ConnectorMod_Ref::load_from_file(file.path()) {
            Ok(plugin) => Some(plugin),
            Err(e) => {
                log::debug!("Failed to load plugin in '{:?}': {}", file.path(), e);
                None
            }
        })
        .collect()
}

Separating runtime and interface

So far I’ve been assuming the following (ideal) structure for the plugin system:

The runtime crate, which loads and runs the plugins.
The common crate, with the interface shared between the plugin and the runtime.
The plugin crates, with the loadable functionality.

This is essential in order to actually improve compilation times, which is one of the main objectives of our plugin system. There are two ways to look at compilation times:

For runtime development
For plugin development

In both cases, we want to compile only either component. If we’re developing a plugin, it makes no sense to be forced to compile the runtime as well, because we aren’t actually changing it. And if we’re working on the runtime, we don’t want to re-compile the functionality from the plugins.

If we just separate the runtime from the plugins, we can achieve the first point. The functionality is now in separate plugin binaries, so the runtime will have considerably lower compilation times.

But we also want to improve plugin compilation times, and if the interface is in the same crate as the runtime, we’ll have to compile both. The shared parts should be written in a separate crate.

The problem is that if you aren’t starting from scratch, it’s very likely that you won’t have a common crate. You’ll just have a single binary crate with everything: runtime, plugins, and common. This is exactly what happened to me with Tremor as well.

For now, I just have the runtime and the interface in the same crate. External plugins then have the entire runtime as a dependency. Which, as I said, is suboptimal, but it works. Moving so much stuff into a separate common crate would mean a ton of conflicts, so I would rather leave it for whenever the first iteration is merged. Here’s the “just make it work” structure for the plugin system:

Getting closer to deployment

Preparing the release for the plugin system has always sounded like a quite complicated task. I’ve never wanted to introduce too many changes at once because otherwise it will become very hard to review and merge into the main branch. This is why I decided to schedule the release in four separate iterations:

Define the new interface and use it internally: the plugin system should be as minimal as possible. The program can be converted to support plugins, but while keeping them in the same binary, for simplicity. The plugin loading functionality can be left as a proof-of-concept for now. Still, this release could maybe include a couple external plugins for testing purposes.
Separate the runtime from the interface: what I talked about in the previous section can actually be done now. Plugins should be able to depend only on the common crate, rather than on the runtime. We may also run into many Git conflicts while doing this, but it should be a much faster process, since the crate separation is all we have to worry about now.
Actually make the plugins external: since the in-tree plugins are implemented with the new interface, making them actually external should be trivial. It’ll just require re-organizing the repository with the new crates, fixing the build system, and similars.
Polishing for deployment: last improvements before the release, extensive documentation, evaluating the final results, etc.

Step 2 and 3 can be interchanged as desired, but I thought that the sooner we separate the crates, the better. Plugin development should be much faster after that step for everyone else.

Conclusion

This has been my road to implementing the first version of Tremor’s plugin system. I’m still only on the first step of the schedule, but hey, it works! The next and hopefully last article will cover the final cleaning up, testing and benchmarking required to make it ready for deployment, hopefully in v0.12 :)

I will also work on properly making Value #[repr(C)], instead of also having PdkValue. In retrospect, creating PdkValue was a great decision at that point: these 120 errors I got when trying to make Value #[repr(C)] were related to a nasty bug in RCow’s implementation. Changing from Cow to RCow is broken in some cases because RCow is invariant. For those that don’t know what that means, don’t worry, as I will be releasing another article that explains everything once it’s been fixed. If you’re interested, you can follow this issue in the meanwhile, and hope that you don’t run into it:

lifetimes with R* types break compared to non R* types rodrimati1992/abi_stable_crates#75

After I’m fully done I will also reorganize this series a bit and make it an easier read. I have been writing these articles as I learned how the plugin system could be implemented, so there might be some outdated or repetitive statements in previous articles. If you have any suggestions you can leave them here:

Reorganize rust plugins series once I'm done marioortizmanero/nullderef.com#50

For those interested, I recently gave a quick talk about the whole project in the 2022 LFX Mentorship Showcase. Unfortunately, it was just a 15 minutes presentation, so I couldn’t get into many technical details, but it covers how the whole experience has been so far, and what I’ve learned:

Footnotes

Plugins in Rust: Reducing the Pain with Dependencies

2021-11-08T00:00:00Z

Table of Contents

Previously in this series, I covered how the plugin system could be implemented from scratch. This is a lot of work if you’re dealing with a relatively large codebase and therefore a complex interface in your plugin system, so let’s see how we can make our lives easier. I’ve been wanting to try abi_stable for this since the beginning, which was specifically created for plugins. But we aren’t really locked to that crate, so I’ll show other alternatives as well, which can even be combined to your liking.

Handy tools for our Plugin System

Async with the C ABI

In a previous post I mentioned that async was not supported in abi_stable. While this is true, because there is no FFI-safe Future in the crate, it’s certainly possible, and it might be of interest later on.

Matthias recently let me know about the async_ffi crate, which lets us do exactly that. It exports the type FfiFuture<T>, which provides the same functionality as Box<dyn Future<Output = T> + Send>:

// This is how regular async works: the first function is practically equivalent
// to the second.
async fn example() -> String {
    read_file().await
}
fn example() -> impl Future<Output = String> {
    async {
        read_file().await
    }
}

// For FFI-safe interfaces there can't be generics involved, so the future is a
// concrete type instead of a trait. This conversion from `Future` to
// `FfiFuture` can be done with `into_ffi`.
fn example() -> FfiFuture<String> {
    async move {
        read_file().await
    }
    .into_ffi()
}
// `FfiFuture<T>` implements `Future<Output = T>`, so it can be awaited as usual
async fn user() {
    example().await
}

Someone asked for this feature in abi_stable back in 2019, but noone seemed interested enough to implement it at that time, so maybe in the future.

LCCC

The Lightning Creations Compiler Collection provides a set of frontends and backends with a uniform intermediate representation for multiple programming languages, including Rust.

This means that they’ve written their own standard library with the C ABI, which is exactly what we need. It’s much simpler than Rust’s standard library, but it includes the most popular types your library may use: HashMap, Vec, String, Box, etc. The source code is quite nice to read in comparison to std, which often includes lots of procedural macros and various forms of astral magic.

It’s not too popular right now, and it’s still Work In Progress, but it serves as an example of what we’re looking for in this article. We just want to simplify our lives by having a #[repr(C)]-compatible standard library so that we don’t have to write it ourselves. If all you need is something simple like LCCC, consider this library or a similar one.

Safer FFI

If you don’t like any of the solutions listed in this article, and you’re going to end up writing the plugin interfaces by hand, you might be interested in safer_ffi.

All this crate provides is a set of procedural macros to make FFI interfacing an easier and safer task. With it, you’ll be able to get rid of lots of extern "C" and unsafe instances in your code, which can get out of hands in larger codebases. Its documentation is excellent, you can check out its book for more information.

CGlue

In my last post, I was brought up the cglue crate by its own creator. It takes a very interesting approach, achieving ABI stability through opaque types.

An opaque type is simply one for which you don’t know its concrete layout. There’s no #[repr(C)] needed at all, because one can only interact with it via void pointers and its associated vtables.

cglue’s README showcases the following snippet of code, and the repo even includes an example of a plugin system.

use cglue::*;

// One annotation for the trait.
#[cglue_trait]
pub trait InfoPrinter {
    fn print_info(&self);
}

struct Info {
    value: usize
}

impl InfoPrinter for Info {
    fn print_info(&self) {
        println!("Info struct: {}", self.value);
    }
}

fn use_info_printer(printer: &impl InfoPrinter) {
    println!("Printing info:");
    printer.print_info();
}

fn main() -> () {
    let mut info = Info {
        value: 5
    };

    // Here, the object is fully opaque, and is FFI and ABI safe.
    let obj = trait_obj!(&mut info as InfoPrinter);

    use_info_printer(&obj);
}

cglue is limited to just generating FFI-safe trait objects, trying to make the whole process as straightforward as possible. You could say that cglue covers just a subset of what abi_stable does, because most of this is also available in abi_stable through the sabi_trait procedural macro, which I’ll explain later. It’s possible to combine both crates, which is something cglue plans to do in the future. cglue offers the following benefits over sabi_trait^[1]:

It’s possible to generate bindings for C/C++, which means that plugins can be written in languages other than Rust.
You can define trait groups, even with optional traits.

Neither of these are particularly useful for my use-case, but if any of these features interests you, definitely take a deeper look. It’s actively maintained and constantly being improved; the documentation is great and the author frequently uploads updates to his personal blog.

Miri

Miri is an interpreter for Rust’s mid-level intermediate representation. This doesn’t help us with the plugin system per se, but since it’s very likely that we’re going to end up writing unsafe code, it’s good to know about it. That’s exactly what Miri is used for: detecting undefined behavior, such as using uninitialized data or use-after-frees.

I was going to use Miri from the beginning, but since I’ll be using abi_stable for now, there will be no unsafe code involved. If I end up having to resort to it, I’ll try to add Miri to Tremor’s workflow (mainly their Continuous Integration).

cbindgen

For the first steps with dynamic loading I think the C/C++ binding generator cbindgen will help us understand what’s going on under the hood. We can take a look at the generated headers and see how it works internally. Unfortunately, it fails to run for the abi_stable crate:

(...)
WARN: Skip abi_stable::CONST - (...)

thread 'main' panicked at 'RResult has 2 params but is being instantiated with 1 values', src/bindgen/ir/enumeration.rs:596:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This probably has to do with the following warning found in cbindgen’s documentation:

NOTE: A major limitation of cbindgen is that it does not understand Rust’s module system or namespacing. This means that if cbindgen sees that it needs the definition for MyType and there exists two things in your project with the type name MyType, it won’t know what to do. Currently, cbindgen’s behaviour is unspecified if this happens. However, this may be ok if they have different cfgs.

If you’re using something else like cglue, this will work without issues. But after letting the maintainers of abi_stable know about this in an issue, they pointed out that this was expected and that they don’t plan on supporting cbindgen because it would take too much effort. Understandable, so let’s move on.

Working with `abi_stable`

I will personally use abi_stable because it seems like the easiest choice for now, and the one that meets my needs best. Not only does it provide a standard library defined with the C ABI, but also lots of other macros and utilities specially useful for plugin systems. With it, I won’t need a line of unsafe, and I’ll avoid reinventing the wheel in many instances.

Once the plugin system is fully functional with abi_stable, I might consider using something more hand-crafted. This switch won’t be too complicated, since our interface will already be #[repr(C)], which is the most troublesome part. All we’d have to do is remove a few procedural macros, switch the abi_stable types, and load the plugins manually with something like libloading. The only thing I want right now is a plugin system that works, and then we can maybe focus on trying to make it available in other languages, making it more performant, or whatever.

So let’s start comparing abi_stable with my experiments in the previous post using raw dynamic linking. I’ve created the abi-stable-simple directory in the pdk-experiments repository. I’ll be taking a look at the already implemented examples for abi_stable in order to make the learning experience smoother. The base structure for a plugin system with abi_stable is the same as always: a crate for the plugin, another for the runtime, and common, with the shared interface.

Versioning

abi_stable states this regarding versioning:

This library ensures that the loaded libraries are safe to use through these mechanisms:

The abi_stable ABI of the library is checked, Each 0.y.0 version and x.0.0 version of abi_stable defines its own ABI which is incompatible with previous versions.

Types are recursively checked when the dynamic library is loaded, before any function can be called.

In summary, abi_stable itself is far from being permanently backward compatible, but it automatically makes sure that its versions are compatible when running the plugin. While it doesn’t exactly stick to semantic versioning, it’s good enough for us.

The version checking for the entire common crate is already implemented, i.e., we can’t try to mix different versions that aren’t compatible. We could still add a version string for each kind of plugin if more fine-grained control is needed, as described in the previous post.

Loading plugins

abi_stable plugins are structured in modules, which can help us split up our functionality into smaller independent pieces. There must always be a root module that initializes the entire library and provides metadata such as the name or the version strings. Then, we can have submodules to organize the functions exported by the library nicely.

Furthermore, the StableAbi trait in abi_stable indicates that a type is FFI-safe. It contains information about the layout of the type, and it can be derived automatically. Each item in abi_stable’s standard library (RStr, RSlice<T>, RArc<T>, etc) implements this trait, and it’s used to make sure the types are compatible when loading the plugin.

This also introduces the concept of prefix types. When a type derives StableAbi and has the #[sabi(kind(Prefix(...)))] attribute, two more types are generated:

<name>_Prefix, which contains all the fields up to the #[sabi(last_prefix_field)] attribute in the original type.
<name>_Ref, which is a pointer to <name>_Prefix that can actually be passed through the FFI barrier safely.

Prefix types are needed to guarantee some kind of individual versioning to avoid breakage in future patches. It will let us add more fields to the module after the last_prefix_field attribute in patch (0.0.x) updates. Moving this attribute requires a backward-incompatible version bump. Prefix types are often used for modules and vtables.

For now, I’ll just have a single root module and call it MinMod, exporting the min function:

// Using the stable C ABI
#[repr(C)]
// Deriving the `StableAbi` trait, which defines the layout of the struct at
// compile-time:
// https://docs.rs/abi_stable/0.10.2/abi_stable/derive.StableAbi.html
#[derive(StableAbi)]
// Marking the struct as a prefix-type:
// https://docs.rs/abi_stable/0.10.2/abi_stable/docs/prefix_types/index.html
#[sabi(kind(Prefix))]
pub struct MinMod {
    /// Initializes the state, which will be passed to the functions in this
    /// module. I'll explain more about the state later on.
    pub new: extern "C" fn() -> State,

    /// Calculates the minimum between two integers. This is the last defined
    /// field for the current version. If we try to load fields after this, all
    /// of them will be an `Option`.
    #[sabi(last_prefix_field)]
    pub min: extern "C" fn(&mut State, i32, i32) -> i32,
}

Most of the loading functionality is already handled by abi_stable. The module we’re exporting implements the RootModule trait, which includes functions to load the plugin, such as RootModule::load_from_file or RootModule::load_from_directory:

// Marking `MinMod` as the main module in this plugin. Note that `MinMod_Ref` is
// a pointer to the prefix of `MinMod`.
impl RootModule for MinMod_Ref {
    // The name of the dynamic library
    const BASE_NAME: &'static str = "min";
    // The name of the library for logging and similars
    const NAME: &'static str = "min";
    // The version of this plugin's crate
    const VERSION_STRINGS: VersionStrings = package_version_strings!();

    // Implements the `RootModule::root_module_statics` function, which is the
    // only required implementation for the `RootModule` trait.
    declare_root_module_statics!{MinMod_Ref}
}

When loading directories, it makes the following decisions by default (though we could change them if we wanted to):

It does so non-recursively, i.e., only checking the immediate files in the given directory.
The name of the library must be the RootModule::BASE_NAME in lowercase, according to the Operating System’s defaults. For example, in Linux our plugin would be libmin.so, and on Windows it’d be min.dll.

This means that we should add the following parameter to the plugin’s Cargo.toml file:

[lib]
# This way, the shared object will be saved as `abi_stable` prefers, for example
# `libmin.so`.
name = "min"

Finally, this is what the runtime may look like:

pub fn run_plugin(path: &str) -> Result<()> {
    let plugin = MinMod_Ref::load_from_directory(path.as_ref())?;
    println!("Loading plugin {}", MinMod_Ref::NAME);

    // First we obtain the function pointer. This is not an `Option` because
    // `new` is defined before `min`, the last prefix field.
    let new_fn = plugin.new();

    // We initialize the plugin, obtaining a state.
    let mut state = new_fn();

    // Same for the `min` function
    let min_fn = plugin.min();

    println!("initial state: {:?}", state);
    println!("  min(1, 2): {}", min_fn(&mut state, 1, 2));
    println!("  min(-10, 10): {}", min_fn(&mut state, -10, 10));
    println!("  min(2000, 2000): {}", min_fn(&mut state, 2000, 2000));
    println!("final state: {:?}", state);

    Ok(())
}

Executing the plugin-sample implementation:

$ make debug-sample
Loading plugin min
initial state: State { counter: 0 }
  min(1, 2): 1
  min(-10, 10): -10
  min(2000, 2000): 2000
final state: State { counter: 3 }

Handling state

Regular Rust

As we saw in the previous example, we need some kind of generic State type that each plugin can implement with their own data. In regular Rust, we’d do as follows. See the full code here.

trait State: Debug {}

// Remember that we can't use generics, so we need `dyn`, either by itself as a
// reference, or in a box.
type StateBox = Box<dyn State>;

fn usage(state: &mut StateBox) {
    println!("state debug: {:?}", state);
}

Interface types

Unfortunately, we already know that regular dyn is not FFI-safe. I covered how it’s possible to work around it with pointers, but here we can resort to abi_stable’s safer and more convenient alternatives. Here’s one of them (see the full code here):

#[repr(C)]
#[derive(StableAbi)]
// An `InterfaceType` describes which traits are required when constructing
// `StateBox` and are then usable afterwards.
#[sabi(impl_InterfaceType(Debug, PartialEq))]
struct State;

// A trait object for `State`
type StateBox = DynTrait<'static, RBox<()>, State>;

// It can then be used easily like this
fn usage(state: &mut StateBox) {
    println!("state debug: {:?}", state);
}

Here we first declare a State interface type. Note that even though it’s defined as a struct, this is a translation of the previous snippet of code, so it acts as the empty “trait”. But all it does is establish Debug and PartialEq as its supertraits and give access to them; you can’t really add custom methods to the trait.

Unlike dyn, this even works with supertraits that aren’t object-safe. Thus, we can use something like PartialEq. Its main disadvantage is that it’s limited to a set of 21 hardcoded traits, so it might not be enough for us.

Trait objects

If we want something more akin to traits on Rust, we can use #[sabi_trait]. The trait has to be object-safe, and by default there’s no support for PartialEq in the list of supertraits, so I’ll remove it. See the full code here.

#[sabi_trait]
pub trait State: Debug {
    fn counter(&self) -> i32;
}

// A trait object for the `State` Trait Object
pub type StateBox = State_TO<'static, RBox<()>>;

// It can then be used easily like this
pub fn usage(state: &mut StateBox) {
    println!("state debug: {:?}", state);
    println!("state counter: {:?}", state.counter());
}

As its documentation explains, this still has a limited number of possible supertraits, but at least it lets us require functions as usual, and it even works with default implementations.

Error handling

abi_stable is just a wrapper over libloading after all. It doesn’t include a sandbox, so if the plugin developer was a malicious actor, they’d have full access to the computer the runtime is being executed on. Other popular plugin systems such as nginx’s or apache’s suffer from the same issues, for reference.

However, I think it’s not so bad to assume that no bad actors will be involved here. A sandbox would be mandatory if we were working on something like Solana (one of the main users of eBPF in Rust), which basically executes random code from the internet. But with Tremor we can assume that the plugins come from trusted sources because they’re installed and configured manually by the user.

There are some additional security measures that could be implemented in the future, like checking the integrity of the plugins and verifying they come from a trusted source before loading them. Of course, if we could afford to have a sandbox it’d definitely be the best way to do it, but we’ve already seen in this series that it’s currently not really viable for this use-case.

Still, we trust that the plugin developer has good intentions, but not necessarily that they know what they’re doing. We should make fatal errors as hard as possible to happen so that Tremor isn’t constantly crashing. The fewer pitfalls, the better.

The full source for the example that’s supported to work is here. Let’s see a few ways in which the plugin could go wrong:

Version mismatch

The versions of the common library are checked automatically. In case there’s a mismatch in those considered incompatible (changes in x.0.0 or 0.x.0), this is what will show up. See the full code here.

$ make debug-versionmismatch
Error when running the plugin:

(...)

Error:incompatible package versions
Expected:
    0.2.0
Found:
    0.1.0

We can absolutely catch this error gracefully and continue with the execution of the runtime, just like with raw dynamic loading. It’s even easier because it works out of the box.

Missing fields and wrong types

The layout of every type is recursively checked before trying to use them to make sure they are compatible. Unlike raw dynamic loading, these errors can be caught gracefully, which is a huge plus (it used to segfault). See the full code here.

$ make debug-wrongtype
Error when running the plugin:
Compared <this>:
    --- Type Layout ---
    type:PrefixRef<'a, MinMod>
    (...)
To <other>:
    --- Type Layout ---
    type:PrefixRef<'a, MinMod>
    (...)

0 error(s).

0 error(s)inside:
    <other>

    (...)

Layout of expected type:
    --- Type Layout ---
    type:MinMod
    (...)

Layout of found type:
    --- Type Layout ---
    type:MinMod
    (...)

(...)

The error message is way too long to show here, but it basically shows the entire layout tree of the types that don’t match for each of its versions (runtime vs plugin). For this example, I changed the State trait to use a boolean instead of an integer counter, and the message describes it perfectly: their sizes, alignments, and types differ in the trait’s methods.

Panicking

Panicking trough the FFI boundary is undefined behaviour; we aren’t guaranteed that the plugin will abort. It may just continue its execution in a completely invalid state, which is scary. But turns out abi_stable properly handles this for us! It will use what it calls an AbortBomb to even print out the line and file where it happened. This is publicly available through the macro extern_fn_panic_handling. See the full code here.

$ make debug-panic
Loading plugin min
initial state: State { counter: 0 }
thread '<unnamed>' panicked at 'This will crash everything', src/lib.rs:26:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

file:src/lib.rs
line:24
Attempted to panic across the ffi boundary.
Aborting to handle the panic...

If we panic in the plugin it won’t be undefined behaviour anymore because abi_stable already makes sure the panic doesn’t reach the FFI boundary.

Panicking and FFI

As we’ve already seen, plugins cannot panic across the FFI boundary under any circumstance^[2]. If we aren’t using something like abi_stable, every single function we export in the plugin should wrap its contents in catch_unwind in order to be able to panic.

Unwinding is a process in which all local objects are destroyed, properly calling the destructors in the thread in order to continue execution safely^[3]^[4]. Knowing this is something taken for granted when taking a look at documentation about exceptions in Rust, but it wasn’t so clear to me at the beginning.

For example, the following snippet will panic after creating the vector. If panics were configured to abort, the contents of the vector wouldn’t be freed at all; the program would just end abruptly, and the cleaning up would be left to the Operating System. But if it unwinds, Rust will call Vec’s destructor, freeing its allocated memory properly, making it possible to continue the execution of the program.

{
    let data = vec![1, 2, 3];
    panic!("oh no!");
    println!("My data: {:?}", data); // Unreachable
}

In a typical usage of Rust, a panic usually means that your program writes some scary message to stdout and then ends. This is because unwinding is propagated and it may end up finishing the execution of the program if it’s not stopped. But that’s exacty what catch_unwind is for:

let result = panic::catch_unwind(|| {
    let data = vec![1, 2, 3];
    panic!("oh no!");
    println!("My data: {:?}", data); // Unreachable
});

// This will run just fine and print out `true`
println!("Did it panic? {}", result.is_err());

Rust makes it very clear that catch_unwind is not intended for regular error handling (you have Result for that). But in our case we are almost forced to use it in order to not invoke undefined behaviour when panicking through the FFI boundary. Every single function in the FFI interface that has a possibility of panicking should use it so that the panic doesn’t try to propagate. And this is quite tricky because even things like addition may cause a panic (overflow in debug mode).

Let’s see what else can we do about panicking:

Aborting

The simplest way to do it would be to just configure plugins to abort on panic instead of unwinding. This is possible with the panic = "abort" option in the plugin's Cargo.toml`. It will still show the panic message, but the execution will be completely stopped by an abort:

$ cargo r -q
thread 'main' panicked at 'Oh no!', src/main.rs:2:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
zsh: abort (core dumped)  cargo r -q

This is sound because the entire program’s execution ends before reaching the FFI boundary. The problem is that cleaning up will never happen, and that although there’s a hack you can use in your common library to make sure the plugin is compiled with `panic = “abort”, it’s only available on nightly until this is merged:

Pluggable panic implementations (tracking issue for RFC 1513) rust-lang/rust#32837

`C-unwind`

This problem is something the Rust devs are aware of, and that they’re trying to fix. It has been proposed under the “C-unwind” ABI string. Just like how you currently use extern "C", if we used extern "C-unwind", we’d get more guarantees about what happens when a thread panics. More information here:

The most relevant things this feature offers us is:

Support for unwinding through the FFI boundary.
A guarantee that even with extern "C", panicking is not undefined behavior, it’ll just abort (except for some very specific cases). Switching between “abort” and “unwind” for the panic option in Cargo.toml is always sound.

Unfortunately, it’s moving somewhat slowly, and I’m not quite sure when this will be ready. In the meanwhile, we’ll need to use something else to ensure no undefined behaviour occurs in our plugin system.

`AbortBomb`

abi_stable does this in a pretty clever way: it creates an AbortBomb struct at the beginning of the function, which contains its filename and line of code. If something panics and unwraps, AbortBomb’s destructor will be called, which aborts the program. Otherwise, mem::forget is called for the AbortBomb at the end of the function, which will avoid calling its destructor and the function will be able to end successfully.

Note that even though mem::forget is called, no memory is actually being leaked, because the filename is a 'static str — which lives for the entirety of the program — and the line number is an integer, which will be in the stack and doesn’t need fancy destructors.

This approach is completely fine and works great, but it aborts the whole plugin system, so you can’t recover from it at all. In the case of Tremor, if a plugin panics, from a logical standpoint it doesn’t make much sense to continue the execution because there’s a piece missing in the pipeline. It couldn’t continue anyway… Right? Well, we could actually load the plugin that panicked again and use that instead for the remainder of the program. But since our plugin system doesn’t support unloading, we’d be leaking memory, and if the plugin keeps panicking it’d eventually crash.

Recovering from a plugin panicking is definitely viable, and it might be an interesting feature for the future. Unfortunately, it’s a lot of work to make sure it works properly, and it’s not really an objective for the first implementation, so for now I’ll just use abi_stable’s solution.

Recovering with `catch_unwind`

As I explained in the beginning, catch_unwind can be used to detect and stop unwinding panics. One way to notify the runtime that a plugin has panicked so that it can act accordingly would be to use an enum equivalent to Option<T>:

#[repr(C)]
#[derive(Debug, StableAbi)]
pub enum MayPanic<T> {
    Panic,
    NoPanic(T)
}

MayPanic is a type that only returns the original value if the function finished without panicking. Since the contents returned by catch_unwind are just dyn Any and don’t provide much value for us, they’re discarded and the Panic variant is empty. The panicking information will be printed automatically as output anyway (or whatever is configured with the panic hook). We will use it in FFI contexts, so it also implements StableAbi and it’s #[repr(C)].

I didn’t want to use Result for this because panic errors should be treated differently from a regular error. Apart from the fact that panic::catch_unwind returns a Box<dyn Any>, which doesn’t implement Error, panics happen when the plugin reaches an unrecoverable state and cannot continue. We really have to make sure this is handled differently from a regular error, so having the type safety of a different type can help.

It implements From<thread::Result<T>>, so it can simply be used like this:

fn plugin_stuff() -> MayPanic<Whatever> {
    panic::catch_unwind(|| {
        // Code goes here
    })
    .into()
}

Ideally, MayPanic could be accompanied by a #[may_panic] procedural macro that adds this boilerplate automatically to the function it’s attached to. Additionally, it could come with a #[may_not_panic] variant that attaches the #[no_panic] macro from the no-panic crate to make sure the statement is true at compile time. However, no-panic isn’t too reliable, so perhaps it could be opt-in with something like #[may_not_panic(enforce)].

Something that complicates this whole thing considerably is the concept of exception safety. Unfortunately, catch_unwind isn’t as easy to use as just slapping your code into its closure/function, as there are some types that aren’t considered unwind safe. You can read more about that here, but I won’t get into more details because we aren’t going to use MayPanic in our own plugin system anyway.

Type conversions

It’s important to know the complexity of conversions from and to abi_stable types. If Vec<T> → RVec<T> wasn’t $O(n)$ it might be worth avoiding it altogether.

This means that I should spend at least a bit of my time on understanding how the abi_stable types are implemented and making sure this isn’t the case. In std, the definition of Vec is actually quite simple if we remove most of the noise:

// A non-null pointer to `T` that indicates ownership.
pub struct Unique<T: ?Sized> {
    pointer: *const T, // The data itself
    _marker: PhantomData<T>, // Indicating that we own a `T`
}

// Low level type related to allocation
pub struct RawVec<T> {
    ptr: Unique<T>,
    cap: usize,
}

pub struct Vec<T> {
    buf: RawVec<T>,
    len: usize,
}

It’s mostly self-explanatory; a Vec<T> is a pointer to T with a set capacity and length. What about abi_stable’s implementation?

#[repr(C)] // Notice this, so that it's FFI-safe
#[derive(StableAbi)] // This trait marks `RVec` as FFI-safe, with info about its layout
pub struct RVec<T> {
    pub(super) buffer: *mut T,
    pub(super) length: usize,
    capacity: usize,
    vtable: VecVTable_Ref<T>,
    _marker: PhantomData<T>,
}

Yup, basically the same, but packed inside a single struct. The single difference is that we have a field with the vtable. The conversion between these types is written with a macro, but if expanded, it looks like this:

impl<T> From<Vec<T>> for RVec<T> {
    fn from(this: Vec<T>) -> RVec<T> {
        let mut this = std::mem::ManuallyDrop::new(this);
        RVec {
            vtable: VTableGetter::<T>::LIB_VTABLE,
            buffer: this.as_mut_ptr(),
            length: this.len(),
            capacity: this.capacity(),
            _marker: PhantomData,
        }
    }
}

The only “weird” part is the usage of std::mem::ManuallyDrop, which is simply a wrapper that indicates Rust to not call the destructor of its contents automatically. In this case it’s basically a less error-prone std::mem::forget, as its docs explain. Thanks to it, the memory from the Vec won’t be dropped when this function ends, and its pointer ownership can be safely moved into RVec, with no copying.

This happens for every type I checked in abi_stable, including RSlice<T>, which contains a reference to a slice, RStr, which is just a RSlice<u8>, and RString, which is just a RVec.

Thread safety

abi_stable uses libloading, whose error-handling is not fully thread-safe on some platforms, such as dlerror on FreeBSD^[5]^[6]. It’s fully thread-safe on Linux^[7], macOS^[8], and Windows^[9], so for Tremor specifically we don’t have to worry about this. But if your programs supports other Operating Systems, you might want to check their manuals one by one in order to make sure.

However, for the first version of our system this won’t be a problem at all. For simplicity’s sake, loading plugins after the startup will not be implemented yet, and we’ll do it sequentially. But it’s good to know it for the future.

Performance

I first tried to write these benchmarks with cargo nightly’s implementation. However, since it’s so basic, not updated regularly, and requires nightly, I moved to criterion, which I quite liked after using it for another post.

First, we can take a look at already implemented plugin systems in order to have an idea of the performance hit we’ll experience in Tremor. This is what we should expect once our system is polished and ready for deployment:

nginx reports 20% slower startup times and up to a 5% slowdown in their execution times^[10].
This article explains that the only performance difference is saving the resolved address of the symbol in a table the first time, and then it’s just a couple more instructions to access it. Also, obviously, the fact that the compiler can’t optimize parts of the code (e.g., inline function calls).

These are the results of the benchmarks I wrote, on my not-so-fast laptop:

dynamic setup           time:   [652.53 ns 654.72 ns 657.34 ns]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

abi_stable setup        time:   [30.386 ns 30.477 ns 30.575 ns]
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe

dynamic runtime         time:   [1.8814 ns 1.8878 ns 1.8947 ns]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe

abi_stable runtime      time:   [3.2155 ns 3.2325 ns 3.2494 ns]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild

native runtime          time:   [817.39 ps 819.33 ps 821.38 ps]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

Note that the benchmarks still don’t represent a real usage of Tremor; it’s just using the plugin I described in this post with the min function. But we can more or less analyze the performance differences between abi_stable and raw dynamic loading — I doubt it’s worth implementing the final version with both methods just to run some benchmarks.

The loading times aren’t so important for performance because they only happen once at the beginning of the program. But abi_stable’s way of recursively checking the types in the plugins is not free; the difference with raw dynamic loading should be quite noticeable. But somehow, in my benchmarks abi_stable was way faster. What??

It turns out that abi_stable just leaks the library when it’s loaded to prevent a user-after-free. And since it won’t be unloaded anyway, it’s not much of a problem in terms of leaking memory. The library will be saved into a static variable (of type LateStaticRef), and the next times it’s loaded the initial value will be reused. So in my bencharks for abi_stable, loading only actually happens once, and for dynamic loading it happens for every iteration.

Once the library is loaded, it seems that using dynamic loading versus static linking is quite bad, being more than twice as slow. This is understandable; the problem with the native benchmark was, and most likely still is, that the Rust compiler is too smart. If I called min with fixed parameters — say 10.min(3) — it was optimized away, so I had to write a more intricate example that was different for each loop. Furthermore, using tools like sabi_trait instead of a void* almost doubles the execution time again.

Conclusion

We’ve learned a lot about abi_stable and the overall state of dynamic loading in Rust. We’ll definitely avoid a lot of work thanks to these dependencies. It’s not as bad as I thought; there’s plenty of tools for each use-case, though most are admittedly only in early stages.

Hopefully, the performance degradations we’ve found won’t be as noticeable in the final version of the system. We’ll use sabi_trait only when loading the library instead of for each call. And having a more complex use-case will probably avoid such incredible optimizations in the native code. You can find the full statistical reports in the criterion-reports directory of the repository.

In the next article, I’ll cover the different caveats I’m finding as I try to actually implement the plugin system on Tremor, and the different ways in which they can be approached.

Footnotes

Designing an API Client in Rust: New RSpotify Version a Year Later

2021-10-13T00:00:00Z

Table of Contents

This article in the Rspotify series describes my journey of basically rewriting the entirety of this Rust library; around 13 months of work (in my free time), starting at September 2020, up until October 2021. I think this has given me enough experience for an article regarding API design on Rust, and perhaps those who attempt the same in the future can take some ideas and save time. Specially considering how much I’ve gone in circles and how much time I’ve “wasted” implementing things that ended up being discarded.

The story

If you don’t care about RSpotify’s specific story, you can jump to the “Making the API HTTP-client agnostic” section, but I think knowing a bit the motives for this rewrite is interesting for other open source maintainers in the same situation as me. This all started when I wanted to use a Rust Spotify API wrapper for one of my projects at that time, Vidify. I went to crates.io and looked for “Spotify”, and all I found was rspotify (the most popular one), aspotify (an asynchronous-only alternative), and a bunch of other outdated or incomplete libraries.

The author of aspotify had made a brand-new library from scratch, because as he commented in its release, “[rspotify’s] API in general I found hard to use and confusing”. Honestly, depending on the state of the original library this is the easiest choice because you don’t have to worry about backwards compatibility or old code.

A year back I had also helped with the first steps of tekore, an alternative to spotipy, the most popular Spotify library for Python out there. The problem was that back then, Paul, spotipy’s creator and maintainer, was absent, and the library really needed a push to upgrade from Python 2 and improve the overall interface and user-friendliness, which had become pretty poor.

A few devs, including Felix, tried to start from scratch. Felix was the most dedicated, and Paul eventually granted him permissions to upload new spotipy releases, meaning that he could rewrite the entire thing under the same name, and upload it to PyPi (Python’s official third-party repositories) if he wanted to. The problem was that spotipy had a good amount of users by that point. A couple of them ended up stopping Felix from completely breaking backward compatibility, so he made it into a brand-new library instead and left the maintainance tasks to the new devs in charge. Tekore still is, in my very biased opinion and at the time of this writing, a much nicer to use library than spotipy. But it has taken a lot of time to get a user base starting from scratch, and it’s still not as popular as spotipy by far.

Having gone through this, I also wanted to see if it would be possible to give RSpotify a face wash without anyone complaining. RSpotify was still in beta – version 0.10 – so you can’t expect to have a stable API, but that’s far from a full rewrite. At the very least I wanted to talk with Ramsay, the original RSpotify developer, and see what his thoughts were. I proposed some big changes like rewriting the blocking module and Kestrer wrote up an issue with ~80 issues he had found when using the library.

I think it’s fair to release the new version 0.11 as such, rather than as a new library, for the following reasons:

Ramsay, the author, also wanted to help.
The fundamentals of the library would remain the same: being the most user-friendly feature-full library for the Spotify API.
It was considered a necessary evil. In my opinion, the library was doomed without a big rewrite. Otherwise, it’d eventually become really hard to maintain and work with. I admit that said breaking changes could’ve been smaller, but I thought that, while I’m at it, I might as well make it as close as possible to a perfect Rust interface.

Also, note that I am in no way associated with Spotify (though open to offers :P). I’m just a Computer Science student whose sole goal was to use Rust for a non-trivial case, from start to finish. I was somewhat experienced with the Spotify API, so I decided RSpotify was a good choice.

Now that most of the stuff I talk about in this article is finished, you can read about the changes in detail in the CHANGELOG. The following sections will talk about what I learned about API Design, and some tips for those writing one in Rust.

Making the API HTTP-client agnostic

RSpotify is now HTTP-client agnostic, which means that it can work with whichever HTTP library the user configures without adding much overhead. For now, we support ureq and reqwest^[1]. The non-trivial part about this is that the HTTP client can be either blocking (ureq) or asynchronous (reqwest).

While it’s fully implemented in the new version, it’s the only part that isn’t finished yet and that’s subject to change. This has to do with the way we support both async and sync HTTP clients under the same trait. In order to have a common interface for both kinds of programming, we use the maybe_async crate, which lets us switch between them with a feature.

The trait used as a base for any HTTP client is implemented with async. That way, if the maybe_async/is_sync feature is disabled in the Cargo.toml, the trait and its implementations remain the same, and if it’s enabled, all the occurrences of async and .await are removed. It’s also possible to add special cases where removing async/.await isn’t enough. This works perfectly, but unfortunately breaks a rule in Cargo’s feature system: features must be strictly additive, and maybe_async’s feature is a toggle – you can’t have both a sync and an async client at the same time.

Since we don’t follow that rule, those who depend on RSpotify with both async and sync (directly or indirectly) won’t be able to compile their project. I won’t get into much detail because I’m writing an entire separate post dedicated for this issue that I’ll publish once this whole ordeal is fixed. We’re keeping track of it in #221 for now, for those who want to learn more.

For now, I’ll just explain how it works for clients of the same type: either synchronous or asynchronous. Since the usage for an HTTP client in an API is often very basic, i.e., you only need to make requests and get the results, we can cover its usage under a single interface. For example:

pub trait HttpClient: Send + Default {
   type Error;

    fn get(
        &self,
        url: &str,
        headers: Option<&Headers>,
        payload: &Query,
    ) -> Result<String, Self::Error>;

    fn post(
        &self,
        url: &str,
        headers: Option<&Headers>,
        payload: &Value,
    ) -> Result<String, Self::Error>;

    // etc
}

Then, we can add an implementation for each of our supported clients:

pub struct UreqClient {}

impl HttpClient for UreqClient {
    type Error = ureq::Transport;

    #[inline]
    fn get(
        &self,
        url: &str,
        headers: Option<&Headers>,
        payload: &Query,
    ) -> Result<String, Self::Error> {
        let request = ureq::get(url);
        let sender = |mut req: Request| {
            for (key, val) in payload.iter() {
                req = req.query(key, val)
            }
            req.call()
        };
        self.request(request, headers, sender) // Internal more complex handler
    }

    // etc
}

And we can finally make our client generic over its internal HTTP client:

pub struct Spotify<Http: HttpClient> {
    http: Http,
    // etc
}

impl<Http: HttpClient> Spotify<Http> {
    pub fn endpoint(&self) -> String {
        let headers = todo!();
        let payload = todo!();
        self.http.get("/some/endpoint", headers, payload)
    }
}

Beware that this introduces a good amount of additional complexity which is probably unnecessary for your own API wrapper. But this was definitely something interesting for RSpotify: some crates that already depend on us like ncspot or spotifyd are blocking, and others like spotify-tui use async. I thought I might as well try, and I’ve finally figured out how to make it work, even for both async and sync.

We implement all of this in the crate rspotify-http, which I plan on moving into a separate crate for the whole community to use once it’s working as I want it to. I think this is a pretty neat feature for an API client that will hopefully become easier to implement in the future (and first of all work properly).

Finding a more robust architecture

Another key refactor I worked on for RSpotify was its architecture. The Spotify API in particular has multiple authorization methods that give you access to a different set of endpoints. For example, if you’re using client credentials (the most basic one), then you can’t access an endpoint to modify the user’s data; you need OAuth information. This used to work with the builder pattern, following this structure (though not exactly the same):

// OAuth information
let oauth = SpotifyOAuth::default()
    .redirect_uri("http://localhost:8888/callback")
    .scope("user-modify-playback-state")
    .build()
    .unwrap();

// Basic information
let creds = SpotifyClientCredentials::default()
    .client_id("this-is-my-client-id")
    .client_secret("this-is-my-client-secret")
    .build()
    .unwrap();

// Obtaining the access token
let token = get_token(&mut oauth).unwrap();

// The client itself
let spotify = Spotify::default()
    .client_credentials_manager(creds)
    .token_info(token)
    .build()
    .unwrap();

// Performing a request
spotify.seek_track(25000, None).unwrap();

I wanted something more tailored towards our specific application. I think the builder pattern is great, but it might become too verbose or confusing:

Do we really need it for Credentials, which always takes the same two parameters?
Which authorization method are we using above again? Currently, it’s possible to call seek_track after having followed an authorization process that doesn’t give access to it. And since we’re mixing all of them under the same client it quickly becomes a mess, having many Option<T> fields that are only Some for specific authorization methods. So, what if we have a Spotify client for each authorization method?
Wouldn’t it be nice to have some type safety, too? The unwrap hurts my eyes.

After removing the builder pattern and being more explicit about the authorization method that’s being used, this is more or less what we get:

// OAuth information
let oauth = OAuth::new("http://localhost:8888/callback", "user-read-currently-playing");
// Basic information
let creds = Credentials::new("my-client-id", "my-client-secret");
// The client itself, now clearly with the "authorization code" method
let mut spotify = AuthCodeSpotify::new(creds, oauth);

// Obtaining the access token
spotify.prompt_for_token().unwrap();

// Performing a request
spotify.seek_track(25000, None).unwrap();

And if the user wants something more advanced, they can always write this:

let oauth = OAuth {
    redirect_uri: "http://localhost:8888/callback",
    state: generate_random_string(16, alphabets::ALPHANUM),
    scopes: "user-read-currently-playing",
    ..Default::default()
};

It’s sufficient to use the regular initialization pattern for this case because we don’t even need validation. If we did, we could always just add a few setters or checks before its usage and we’re done. Ask yourself: do you really need the builder pattern? In this case we certainly didn’t.

The most complicated part of the refactor is having a client for each authorization method, and making sure the user can only call those endpoints they have access to. There are many ways to approach this, I just had to decide which one was the best. I gave this a lot of thought^[2]^[3].

Having multiple clients seems trivial with inheritance, with a base from which they can extend. In Rust, we could follow the typical “composition over inheritance” principle:

pub struct EndpointsBase {
    http: Rc<Http> // Shared with the rest of the endpoints
}
impl EndpointsBase {
    pub fn endpoint1(&self) { self.http.get("/endpoint1") }
    pub fn endpoint2(&self) { self.http.get("/endpoint2") }
    // etc
}

pub struct EndpointsOAuth {
    token: Token,
    http: Rc<Http>
}
impl EndpointsOAuth {
    pub fn endpoint3(&self) { self.http.get_oauth("/endpoint3", self.token) }
    pub fn endpoint4(&self) { self.http.get_oauth("/endpoint4", self.token) }
    // etc
}

pub struct AuthCodeSpotify(EndpointsBase, EndpointsOAuth);
impl AuthCodeSpotify {
    pub fn authenticate(&self) { /* ... */ }

    pub fn base(&self) -> &EndpointsBase { &self.0 }
    pub fn oauth(&self) -> &EndpointsOAuth { &self.1 }
}

The user can then write spotify.base().endpoint1() or spotify.oauth().endpoint3() to access the endpoints in their different groups. However, all of them have to share a single HTTP client and other information such as the config or the token, so we have to use something like Rc. We can improve this by taking ideas from aspotify, another popular crate for the Spotify API, which groups up the endpoints by categories. Their endpoint groups take a reference to the client itself instead, which is pretty neat and works just as well. Here is a simplified version of this Rust Playground:

pub trait Spotify {
    fn get_http(&self) -> &Http;
    fn get_token(&self) -> &Token;
}

pub struct EndpointsBase<'a, S: Spotify>(&'a S);
impl<S: Spotify> EndpointsBase<'_, S> {
    pub fn endpoint1(&self) { self.0.get_http().get("/endpoint1") }
    pub fn endpoint2(&self) { self.0.get_http().get("/endpoint2") }
    // etc
}

pub struct EndpointsOAuth<'a, S: Spotify>(&'a S);
impl<S: Spotify> EndpointsOAuth<'_, S> {
    pub fn endpoint3(&self) { self.0.get_http().get_oauth("/endpoint3", self.0.get_token()) }
    pub fn endpoint4(&self) { self.0.get_http().get_oauth("/endpoint4", self.0.get_token()) }
    // etc
}

pub struct AuthCodeSpotify {
    pub http: Http,
    pub token: Token
}
impl Spotify for AuthCodeSpotify {
    fn get_http(&self) -> &Http { &self.http }
    fn get_token(&self) -> &Token { &self.token }
}
impl AuthCodeSpotify {
    pub fn authenticate(&self) { /* ... */ }

    pub fn base(&self) -> EndpointsBase<'_, Self> { EndpointsBase(self) }
    pub fn oauth(&self) -> EndpointsOAuth<'_, Self> { EndpointsOAuth(self) }
}

However, you might think using just spotify.endpoint1() instead of spotify.base().endpoint1() is more suitable for your particular API client. The only way to do that would be to delegate every single endpoint manually into the main client. Some people use Deref and DerefMut in order to automatically do it, but that’s a common anti-pattern^[4].

I tried different approaches, and my favorite ended up being a trait-based interface. All you need is a couple traits with the endpoint implementations, which require a getter to the HTTP client or similars. Here is a simplified version of this Rust Playground:

pub trait EndpointsBase {
    fn get_http(&self) -> &Http;

    fn endpoint1(&self) { self.get_http().get("/endpoint1") }
    fn endpoint2(&self) { self.get_http().get("/endpoint2") }
    // etc
}

pub trait EndpointsOAuth: EndpointsBase {
    fn get_token(&self) -> &Token;

    fn endpoint3(&self) { self.get_http().get_oauth("/endpoint3", self.get_token()) }
    fn endpoint4(&self) { self.get_http().get_oauth("/endpoint4", self.get_token()) }
    // etc
}

pub struct AuthCodeSpotify(Http, Token);
impl AuthCodeSpotify {
    pub fn authenticate(&self) { /* ... */ }
}
impl EndpointsBase for AuthCodeSpotify {
    fn get_http(&self) -> &Http { &self.0 }
}
impl EndpointsOAuth for AuthCodeSpotify {
    fn get_token(&self) -> &Token { &self.1 }
}

This way, as long as the user has these traits in scope, they can access the endpoints with just spotify.endpoint1(). We can make that easier by including a prelude in the library with these traits, so that all the user has to do is use rspotify::prelude::*. Another big advantage this provides is that it’s extremely flexible. The user can declare their own client and implement its internal functionality themselves, while still having access to the endpoints, which is the boring task that probably doesn’t need customization. And even if they wanted to, they could just override the trait implementation.

The main issue with the trait-based solution is that you can’t use -> impl Trait in trait methods as of Rust 1.55^[5]. We unfortunately need these, specially with asynchronous clients, because async trait methods are -> impl Future after all. For now, we can work around it by erasing the types with the async-trait crate. Supposedly, this will be temporary until GATs are implemented, which isn’t too far off^[6].

Both of these solutions also make it hard to have private functions in the base client, because the shared parts are in a trait. We don’t really want the user to have access to the methods get or get_oauth. It’s defined in the client/trait because it’s useful for every client, but for the end user it’s just noise in the documentation. This isn’t that much of a big deal because you can just declare the item with #[doc(hidden)] so that it doesn’t appear in the documentation.

So yeah, there are no perfect solutions, but these are two of the best ones I could find. The choice is up to the designer of the library and their needs. Having multiple clients let us implement PKCE Authentication for RSpotify quite easily, so it’s worth it in the end anyway. Our final architecture looks like this (diagram by Ramsay):

Configuration

Runtime over compile-time

There are a few parts of the Spotify client that can be customized by the user. Previously, these were just fields of the main client, but since we now have multiple clients, it might be worth moving into a separate struct to avoid duplication.

Anyhow, one of our fails was attempting to use features instead of the Config struct for configuration, on the assumption that features would be more performant. Which one is faster?

if self.config.cached_token {
    println!("Saving cache token to the file!");
}

#[cfg(feature = "cached_token")]
{
    println!("Saving cache token to the file!");
}

Turns out that both of these are usually compiled to the same machine code anyway. Since self.config.cached_token is most times specified as a constant, optimizing it away is one of the more basic tasks a compiler can do. Features are drastically less flexible and harder to use than runtime variables, so before introducing one you should really think about it. Apart from the fact that you obviously can’t use features at runtime (which is a possible use-case here), they are applied globally, so you can’t have two different clients, one with cached tokens and another without them. In order to take this decision I actually wrote an entire article about it, so check it out if you want more details.

Even though it’s basic, I keep forgetting about this: don’t get obsessed with performance. As you add new features to the crate, it’s completely natural that some overheads are introduced here and there. And even then, they might not even be noticeable. First of all get that new feature working. Then, measure the real effect on performance. And finally, if it’s more than you expected, then actually think about optimizing it.

One correct usage would be our new cli feature. We have some utilities for command-line programs, such as prompting for the user’s credentials. However, not everyone needs these, such as servers, and it introduced the webbrowser dependency and a few unnecessary functions. So we decided to move this into a separate feature for those interested, which is disabled by default.

Sane defaults

On the topic of configuration, it’s important to have sane defaults as well. This is highly subjective, but I prefer to do as little as possible under the hood without the user knowing about it. When initializing a client we used to automatically try to read from the environment variables. If that didn’t work then we tried to use the default values or we just panicked in the builder:

let creds = SpotifyClientCredentials::default() // this reads the env variables
    .client_id("this-is-my-client-id")
    .client_secret("this-is-my-client-secret")
    .build()
    .unwrap();

This is a pretty useful feature, but we can’t be sure the writer/reader of the code knows about it, and it could potentially cause unintended behaviour. Instead, we can just have a default method that does nothing special, which is what the user would expect, and also from_env, which explicitly tells us what it does:

let creds = SpotifyClientCredentials::from_env() // this reads the env variables
    .client_id("this-is-my-client-id")
    .client_secret("this-is-my-client-secret")
    .build()
    .unwrap();

Flexibility

Taking borrowed/generic parameters

Friendly reminder: generally, it’s better to take a &str than a String in a function^[7]^[8]. The same thing applies to the owned type Vec<T>; it’s probably a better idea to take a &[T] instead, or the even more fancy impl IntoIterator<Item = T>. The last option makes it possible to pass iterators to the function without requiring a collect, which not only is more user-friendly, but also avoids a memory allocation. Its only downside is that the function signatures become a bit uglier, and all the consequences of using generics. Either of these options are fine, really, so it’s up to you.

Optional parameters

Similarly, if the functions in your library frequently include optional parameters (i.e., of type Option<T>), you might want to consider other ways to handle them. In our case, we were using generics with Into<Option<T>> in order to not have to wrap the parameters in Some when passing them to the function, but it wasn’t consistent. We finally agreed that using plain Option<T> was good enough because it simplifies the function definition in the docs and it’s less magic^[9]. But the important part is that we made it consistent; the decision itself between Into<Option<T>> or Option<T> wasn’t that important. After doing research about this topic, I wrote up an article with more details here, in case you want to learn more.

Splitting up into multiple crates

Another cool idea that promotes flexibility is separating the wrapper into multiple crates. In RSpotify, we now have a total of four of them (credits to Ramsay for the diagram):

rspotify-http: the multi-HTTP client abstraction, which I plan on making more generic and moving into a separate crate for everyone to use^[10]
rspotify-macros: a small crate with macros
rspotify-model: the full model for the RSpotify crate
rspotify: the implementation of the clients

The most important one here is splitting up the wrapper into the model and the clients. The model is generic enough that it can be used by any client, even outside RSpotify. Some users have to implement their own custom clients for different reasons, and pulling our model helps to avoid lots of complexity and maintainance work^[11]. It can also be shared with other public crates, such as aspotify, and join forces in keeping the model up to date^[12].

Documentation

Introducing how to use the crate

This might be obvious to some, but it isn’t enough to document every single public item in your library. You also have to introduce the user how to work with it in the top-level documentation. Some ideas:

List the goals, current and future features of the crate, and things you don’t plan working on. Perhaps also add a comparison with similar crates; these are usually super helpful.
Write a small getting started guide, explaining the most important items in the crate and what they do.
Add some notes about the architecture of your crate. This is specially useul to those who want to contribute. For RSpotify, Ramsay created the diagrams included in this article, and added more details in the README.
Explain the Cargo features in your crate and how to use them.
Make sure you have a few examples working. It’s the easiest way to get started, in my opinion.

Helping users upgrade

Since this change was going to break so much code, I wanted to make sure that the upgrade is as less painful as possible. This can be achieved in many ways:

Make sure you prove why these breaking changes are actually necessary. It will feel like less of a waste of time to the user.
Include a changelog, either as an indepent file, or in the release notes. In RSpotify, we make habit of adding a new line to the changelog for every release that includes a new feature or breaking changes. To be honest, in our case it’s turned out quite messy because we had so many changes, but in a regular update it should be nicer to read.
It might be a good idea to create an issue in your repository where you provide help directly to those who try to upgrade and have problems with it.

Macros

Macros in Rust are pretty cool! But you don’t want to overdo them either. In rspotify we frequently had to build hashmaps or JSON objects; at least once per endpoint. Some of the parameters in the endpoints were mandatory, and others optional (passed as an Option):

let mut params = Query::with_capacity(3);
params.insert("ids", ids);
params.insert("limit", limit.to_string());
if let Some(ref market) = market {
    params.insert("market", market.as_ref());
}

I first tried to simplify this by using macros to their full strength, so my initial attempts looked like this:

let params = build_map! {
    ids,
    limit => limit.to_string(),
    optional market => market.as_ref(),
};

Or this:

let params = build_map! {
    ids,
    limit => limit.to_string(),
    Some(market) => market.as_ref()
};

Yes, they are very concise and we remove a lot of boilerplate, but they’re bad for two reasons:

There’s too much magic going on:
- They turn the ids/limit/market identifiers into a string with stringify! and use that as the key for the hashmap insertion.
- In the expression to the right of an optional parameter, its value isn’t treated as an Option anymore; there’s a hidden if let Some(market).
The syntax is weird. In order to understand them correctly, you’d probably have to look up their documentation and read it first.

The final design still reduces the boilerplate needed in each endpoint considerably, but there’s no magic going on. It’s basically the same as a regular hashmap builder macro like you’d find on maplit, and the macro doesn’t hide anything:

let params = build_map! {
    "ids": ids,
    "limit": limit.to_string(),
    optional "market": market.map(|x| x.as_ref()),
};

Anyhow, we might remove it in the future, since this syntax will soon work as well^[13]:

HashMap::from([
  (k1, v1),
  (k2, v2)
]);

Other goodies

Some new features we added to RSpotify that might be of interest specifically for other web API wrappers:

Cached and self-refreshing tokens

Cached tokens are automatically saved into a file, encoded for example in JSON, and then attempted to be loaded again when restarting the application.

Before making a request, self-refreshing tokens check if they are expired, and in that case perform the re-authorization process automatically.

Implement cache token and refresh token ramsayleung/rspotify#223

Type-safe wrappers for ID types

In the Spotify API, items such as artists or tracks are identified by a unique ID string. The URI is the ID, but prefixed by its type, for example spotify:track:4cOdK2wGLETKBW3PvgPWqT.

Many endpoints previously took the URI parameters as a String. That meant we had to manually check that their type were what we were expecting, and also that they were valid (they’re usually made up of alphanumeric characters).

Instead, we now have an Id trait and structs that implement it, like ArtistId or TrackId, keeping its type known at compile time and also at runtime with dyn Id. If you take a TrackId as a parameter, then you already know its type, and that its contents are valid, so you’re ready to use it.

Initial id type proposal ramsayleung/rspotify#161
Fix IDs v4 ramsayleung/rspotify#244

Automatic pagination

Many API servers have paginated replies for large lists. Instead of sending a huge object, it splits it up into multiple packets, and sends them one by one along with an index to the position in the list. Then, the user can stop requesting them at any time and potentially only end up using a portion of that originally huge object.

In Rust, this can be abstracted away very naturally with iterators in sync programs, and streams for async. The latter can be implemented easily in your crate thanks to async_stream.

Add unlimited endpoints ramsayleung/rspotify#124

Simplify wrapper model objects

Due to how JSON works, sometimes an object will always have a single field:

{
    "many_artists": [
        {
            // ..
        },
        // ...
    ]
}

In that case, instead of just deserializing that object with serde and returning it to the user, you can just return that one field in the object:

#[derive(Deserialize)]
struct ArtistCollection {
    many_artists: Vec<Artists>
}

// Before
fn endpoint() -> Result<ArtistCollection> {
    let response = request();
    serde_json::from_str(response)
}

// After
fn endpoint() -> Result<Vec<Artists>> {
    let response = request();
    serde_json::from_str::<ArtistCollection>(response).map(|x| x.many_artists)
}

The way to reduce wrapper object ramsayleung/rspotify#149

Measuring the changes

Since this release changed so much stuff and took so long, I wanted to get a detailed comparison between v0.10 and v0.11 for different aspects of the library – not just performance.

The full source for these benchmarks is available at the marioortizmanero/rspotify-bench repository. Note that I had to apply a small patch to the v0.10 version because by now it didn’t work correctly.

Statistics

Some parts of RSpotify can be analyzed statically, such as the lines of code that will need to be maintained, or its number of dependencies. Here’s an example as of October 12, 2021:

Version	Rust LoC	Dependencies in tree	Dependencies in tree (all features)
0.10.0	11281	132	141
master	7525	101	123

The Lines of Code in the old version were quite bloated because of the blocking module, which was a copy-paste of the async client. Still, these were lines that needed to be maintained, so they count just as much. On the other hand, we now have a much more extensive set of tests and new features that add up. In total, we have about 33% less lines to be maintained.

The number of dependencies has decreased both by default and with all the features enabled. We cleaned up a lot of them and tried to keep the defaults leaner. Since the new version adds more features such as PKCE, we even had to add new dependencies like sha2, but it’s still a clear win.

Execution time

The execution benchmarks use Criterion, with a total of 100 iterations on my Dell Vostro 5481 laptop, or more specifically, Intel i5-8265U (8) @ 3.900GHz. The full reports are available in the report directory of each benchmark.

Taking a look at the Criterion reports, it seems that the Spotify API doesn’t intentionally slow down responses when it’s being “spammed”, so it should be fine in that regard:

Note that comparing the blocking clients for now is unfair, because instead of using reqwest::blocking, now it’s ureq. Furthermore, the async and sync versions can’t be compared either, since the former requires setting up the tokio runtime and a bunch of other stuff.

The asynchronous clients in both versions should give a rough idea of the actual differences, though you can tell it’s just a quick benchmark, and the results shouldn’t be taken too seriously in the first place. Here’s an example as of October 12, 2021:

Version	Debug Compilation Time (s)	Release Compilation Time (s)	Benchmarking Time (ms/iter)	Release Binary Size (MB)	Release Stripped Binary Size (MB)
0.10, blocking	72.7	126.2	271.3	9.9	4.9
0.10, async	72.2	115.7	428.0	11	5.2
0.11, blocking (ureq)	38.5	55.9	503.6	7.3	2.6
0.11, async (reqwest)	51.014	86.594	432.49	8.5	4.0

I also wanted to reflect the compilation time, since it’s a possibility that we have less dependencies, but of larger size. The results show that this isn’t the case, since it takes 29% less time to build in debug mode, and 25% less time in release mode.

In terms of execution time, I didn’t expect it to be any better. Even though the architecture and implementation is cleaner, some of the new features introduce noticeable overhead. For example, now that we have automatically refreshing tokens, the Token has to be saved in an Arc<Mutex<T>>, which means we’re locking and unlocking at least once per request. Still, the difference is negligible: just a 1% increase.

The cleanup and all these dependencies we removed mean that the resulting binary is also smaller, and by a lot: there’s a 23% decrease in its size.

Special thanks

This release has been possible thanks to @ramsayleung, @kstep, @hellbound22, @Qluxzz, @icewind1991, @aramperes, @Sydpy, @arlyon, @flip1995, and @Rigellute.

I’m specially grateful towards Ramsay, who apart from contributing many of the features I listed here, read and reviewed every single one of my issues and pull requests. I’ve learned how important it is to have a second opinon, and someone else who proofreads everything before you merge dumb stuff into master. Note that I did proofread my own ideas and pull requests, but there are some things that you just don’t notice on time, as much as you try to. This is a problem that I think is particularly relevant in open source. I personally had worked on projects alone most of the time, and the difference is huge. I would suggest everyone to try to join forces with at least one more person when working in side projects.

That’s all! I hope this post provided insightful information and that you learned something from it. Remember that you can leave a comment at the bottom in case you want to discuss it.

Lots of love,
Mario

Footnotes

Plugins in Rust: Diving into Dynamic Loading

2021-10-05T00:00:00Z

Table of Contents

In the last article of this series I wrote some simple experiments of plugins both with WebAssembly and dynamic loading. After discarding Wasm for this specific plugin system, I wanted to try to get a more realistic example with dynamic loading and work towards the final implementation.

Since the Tremor team suggested that I began by implementing connectors, we’ll first have to learn more about them. Matthias ran me through their current state in a meeting, which I’ll try to summarize. We won’t need to know much for this specific article, though.

Learning more about connectors

First of all, Tremor is an event processing system for unstructured data. One of its many usages may be to:

Receive logs from different applications in your business.
Filter, transform, and mix them up following the same structure.
Send all the now structured logs to your database.

This currently works with onramps/sources, offramps/sinks and pipelines:

An onramp specifies how Tremor connects to the outside world (or pipeline) in order to receive from external systems, such as TCP, periodically or PostgreSQL.
An offramp specifies how Tremor connects to the outside world (or pipeline) in order to publish to external systems, such as stdout, Kafka or ElasticSearch.
A pipeline is a set of operations (transformation, aggregation, dropping, etc) through which events can be routed.

The thing is that some onramps may not only want to receive from external systems, but also respond to them directly, acting like an offramp, and vice-versa. This is currently implemented with what’s called “linked transports”, and it’s specifically useful for some onramps and offramps like REST and websocket, where the protocol already provides facility for responding to events with a single connection, for example with an ACK.

Basically, connectors are just a way to abstract over both onramps and offramps under the same concept, including linked transports. As the time of writing this article they’re still being implemented by Matthias in the connectors branch of tremor-rs/tremor-runtime, but their interface, defined with the Connector trait, is somewhat stable.

It’s important to keep the plugin interface as simple as possible. The communication details should be left to the runtime, so that the plugin can be simplified to just exporting a number of synchronous functions. With this we can avoid passing some complex types (async, channels, etc) between the runtime and plugin, which can be impossible if you have to maintain ABI stability (abi_stable doesn’t even support async).

Once this lean plugin interface is defined, we can create some kind of wrapper in the runtime (a manager, in Tremor terms) that handles communication and other similar tasks. This exact same thing is done by other crates such as rdkafka, which is based on the C library rdkafka-sys, and implements a higher-level asynchronous interface on top of it.

About Tremor

As always, these articles include a first section with content specific to Tremor that you might want to skip. Unfortunately, with time this series will become more and more specific to Tremor; after all I’m just reporting my progress on their project. Still, having a step-by-step walkthrough for a real-life Plugin System will surely be helpful to those attempting to do the same.

My next steps

In the first meeting we discussed the work I had exposed in my last update. Despite the complications (being forced to use #[repr(C)]), the team liked where the plugin system was going.

They suggested me to start with connectors for the real-life example, even though they were incomplete because Matthias was still working on them. The best way to do this would be to copy the bare minimum from Tremor’s repository and try to get the simplest Proof of Concept working.

In previous meetings we had discussed the possibility of having generics in the interface, but that turned out to not be necessary at all. The Connector trait had a workaround to avoid generics with SinkManagerBuilder.

On software engineering

At the end of the first meeting, Darach gave some very interesting advice for my software engineering career, so I took note of it and reflected for a bit:

As you get more experience in the field, you talk more and code less. The positions you’re in become more about team management than programming. It’s good to remember that software engineering isn’t just coding. Also that with time, your personality changes, and you have to keep adapting.
Team building isn’t about getting a group of people to carry the exact same tasks in the same way. Everyone is different; you’ll have to discover the strengths and weaknesses of each member and figure out how to mix them up. The best teams are often very heterogeneous, and it’s pretty clear to me that this is the case with Tremor as well.
Don’t care about what others say about you (the don’t worry rule). Don’t let “You’ll never end up being X”, “You’re bad at Y” and similars ever affect you.
Coding is mentally exhausting and burnout is a very common thing. Take good rest, breathe, and have fun. Taking a break from programming is a good idea from time to time.

I’ve personally experienced burnout myself, so I know this first hand. You may immerse yourself too much in computers or coding (specially under a pandemic that restricts how much you can go out). Finding a hobby outside that is incredibly helpful.

How Tremor works

After starting to write a first prototype for connectors and failing because I didn’t know what I was doing, I decided to step back and try to understand in detail how Tremor works. Once I had that covered, I could try to simplify the plugin system as much as possible in order to keep my sanity.

I jumped into the codebase of tremor/tremor-runtime and tried to figure out how it was structured, also with the help of the team later on. Tremor is loosely based on the actor model. Quoting Wikipedia:

[The actor model treats the] actor as the universal primitive of concurrent computation. In response to a message it receives, an actor can: make local decisions, create more actors, send more messages, and determine how to respond to the next message received. Actors may modify their own private state, but can only affect each other indirectly through messaging (removing the need for lock-based synchronization).

It doesn’t use a language (e.g., Erlang) or framework (e.g., bastion, maybe in the future) that strictly follows the actor model, but it often re-implements the same patterns manually. Tremor is currently implemented with asynchronous programming, which means that instead of threads we’ll be working with tasks, a higher level concept. From the async-std documentation:

An executing asynchronous Rust program consists of a collection of native OS threads, on top of which multiple stackless coroutines are multiplexed. We refer to these as “tasks”. Tasks can be named, and provide some built-in support for synchronization.

We could summarize this with the sentence “Tremor is based on actors running in separate tasks which communicate asynchronously via channels”. The main actor is called the World. It contains the state of the program, such as the available artifacts (repositories) and the running ones (registries), and it’s used to initialize and control the program.

I’ll try to follow what Tremor does in order to get a connector running with the help of a few diagrams. The following diagram showcases what happens when a World is created. This introduces the concept of Managers, which simply are actors in the system that wrap up some functionality.

Managers help decouple the communication and the implementation of the underlying functionality. They are also useful to remove some boilerplate when initializing the components, such as creating the communication channel or spawning the component in a separate task.

Generally, there’s one manager per artefact type, which helps with their initialization process, and then there’s one manager per running instance, handling their communication details.

Once all the managers are initialized, Tremor currently registers all the built-in artifacts in a “hardcoded” way with register_builtin_types. But after the plugin system is implemented, this will happen dynamically, i.e., Tremor will automatically look for DLL/SO files in its configured directory and try to register all the plugins it can find. The user may additionally request a specific plugin to be loaded while Tremor is running.

Note that the initialization of the connectors is done in two steps: first they’re registered, which just means that the connector is now available for loading (they’re added to the repository). The connector doesn’t actually start running until a binding is created with it, for example with launch_binding, which will remove it from the repository and add it to the registry, with the currently running artifacts.

connectors::Manager contains all the connectors running in Tremor, which we’ll now try to understand:

Since it’s a multistep process (it’s actually more complicated than registration + creation), the first part of it already provides the tools to initialize the connector (mainly the builder). When the connector needs to start running because it’s been added to a binding in the pipeline, the builder helps to construct it generically with the previously provided configuration details. Finally, it’s moved into a task of its own, so that it may communicate with other parts of Tremor.

Now that we have a connector running, let’s see how it’s split up into the source and sink parts. In a very similar way, a builder is used to initialize the underlying source, sink, or both, and then a new task is spawned for them.

A manager is also created for each instance of source/sink, which will handle the communication with other actors. This way, the source and sink interfaces can be kept as simple as possible. These managers will receive connection requests from the pipeline and then redirect or read from it.

The main difference between sinks and sources currently is that the former can also reply to messages within the same connection. This is useful to acknowledge the package (“Ack”) or to notify something has failed in the sink (“Fail” for a specific event, “CircuitBreaker” to completely stop data from being sent).

Codecs and preprocessors are involved here both at the source and sink levels. In the source part, the data is transformed or split up through a chain of preprocessors and then the codec is applied. For the sinks, the inverse process is followed: the data is first encoded into bytes with the codec, and then a series of post-processors are applied to the raw binary data.

Some connectors are based on streams. They are equivalent for example to TCP streams, which help to group up messages and avoid mixing them up. They are manually started and ended via messages, and the manager saves their state in a field called states (since for instance preprocessors may need to keep a state). If a connector doesn’t need this, such as the metronome, it may simply specify DEFAULT_STREAM_ID as the stream ID always.

After the full interface of connectors is done, I could implement the two following connector plugins:

Blackhole is used for benchmarking. It takes measurements of the end to end times of each event traversing the pipeline and at the end prints an HDR (High Dynamic Range) histogram.
Blaster replays a series of events specified in a file, which is specially useful for performance testing.

Both of these are relatively simple and will be helpful to benchmark the plugin system later on. But that isn’t really important right now; I first need to get it working, and then I can care about performance.

Taking a look at eBPF first

In the previous articles I mostly considered using either WebAssembly or Dynamic Loading. What I didn’t even know about is eBPF, “a revolutionary technology with origins in the Linux kernel that can run sandboxed programs in an operating system kernel”. However, similarly to WebAssembly, its usage has been expanded to user-space applications. eBPF defines a set of bytecode instructions that may be ran by a virtual machine anywhere, similarly to how Wasm works.

There are multiple active crates for eBPF in Rust. libbpf_rs, redbpf and aya are specific to the Linux Kernel. solana_rbpf is a virtual machine, so it only works for user-space. The maintainers of the latter use it to safely run apps on the blockchain, and their crate seems to be a fork of the now abandoned (?) rbpf. This recent talk at LPC 2021 explains the situation of eBPF in Rust quite well (mainly for Aya, so it’s mostly related to the Linux Kernel).

Unlike WebAssembly, you don’t necessarily need to serialize or write to an intermediate memory. Since you fully control how the virtual machine works, the runtime could implement a custom sandbox that simply checks for the read/written addresses in the plugins to make sure they aren’t out of bounds, while still sharing the same memory space. So in terms of performance, Tremor itself could use it — though there’s still the penalty of interpreting plugins instead of running them natively.

The problem in this case is that, for what I’ve found, Rust support leaves to be desired. Most people seem to use C for eBPF and I think it shows; the number of tutorials/guides/articles about eBPF on Rust is incredibly small. There’s no official target to compile Rust to eBPF, and the only user-space runtime we can use is rbpf and its derivatives. Looking for information about this topic was somewhat frustrating, specially because the search results are mixed up with kernel-only BPF, which is not relevant to us.

It doesn’t really seem like the best choice right now, in my opinion. We would have to write almost everything about the plugin system from scratch, including the sandbox itself (allowing only different sets of syscalls, bounds checking, etc). It would be considerably more cumbersome than using something like abi_stable. Maybe in the future it’d be worth considering it in detail and running some benchmarks, but for now I think dynamic loading is still the clear winner for Tremor. Still, I’m surprised by how flexible eBPF seems to be, and how it’s possible to avoid the memory barrier problem found in Wasm.

Cheers to Dr. Florentin Rochet for letting me know about this technology — though he’s considering switching to WebAssembly for his project. He’s currently using it to research pluggable anonymous protocols like Tor, which would allow patches to their code to happen at runtime^[1]^[2]^[3]^[4]. This makes it faster to fix vulnerabilities until it’s properly updated upstream, among other things. Pretty cool :)

Getting deeper into dynamic linking

Now that we definitely know how to approach the plugin system, we have two choices: using raw dynamic linking with the C ABI and libloading, or trying out the abi_stable crate. I suggest we do both. We’ll most likely end up using the latter because it should be easier and safer, but it’s still a very good idea to know how abi_stable works under the hood.

In the previous article I created a dynamic-simple experiment in examples to the pdk-experiments repository. In this one we’ll try to get an implementation that’s closer to what we need for connectors, so I’ll call the new experiment dynamic-connectors.

Versioning

In order to get more advanced things running, we should figure out how to properly embed metadata in the plugin. In order to export any type, we already know that it must be FFI-safe. But there’s something else of great importance: versioning. In order to safely load the plugin, one must ensure that the versions of the common crate match — or at least that they’re compatible — for both the runtime and the plugin. Here’s an example of how this could go wrong if we don’t save information about versioning:

// plugin implementation
pub mod common {
    // This is the declaration for the plugin data in version 0.1
    #[repr(C)]
    pub struct PluginData {
        pub name: &'static [u8],
        pub new: unsafe extern "C" fn() -> State,
    }
}

#[no_mangle]
pub static PLUGIN_DATA: common::PluginData = common::PluginData {
    name: b"test",
    new
};

// runtime implementation
pub mod common {
    // And this is the same type, but in version 0.2
    #[repr(C)]
    pub struct PluginData {
        pub name: &[u8],
        pub new: unsafe extern "C" fn() -> State,
        // NOTE: this field is new here!
        pub connect: unsafe extern "C" fn(&mut State) -> bool
    }
}

fn main() -> Result<(), anyhow::Error> {
    unsafe {
        let library = Library::new(path)?;

        let data = library
            .get::<*const common::PluginData>(b"PLUGIN_DATA")?
            .read(); // !!! UNDEFINED BEHAVIOUR !!! What will `data.connect` be?
    }

    Ok(())
}

In the code above, we can see that, even though both versions of PluginData are FFI-safe, their layouts aren’t the same, because the last one has a new field. When trying to read PLUGIN_DATA, undefined behaviour will occur (most likely accessing to an invalid memory address).

Every plugin should at least export the version of common it uses, and the runtime should check it before anything else.

Specifically, the type used to export the version has to be:

FFI-safe, so &str or CStr are discarded (the latter is a Rust wrapper and not #[repr(C)]).
Stable. abi_stable::RStr won’t work either because the versions for abi_stable might mismatch, since we’re reading the symbol before knowing that. Its layout must be always the same.

Thread-safe (implement Sync). If we wanted to use something like *const c_char, the compiler would throw the following error, because it’s a pointer:

error[E0277]: `*const i8` cannot be shared between threads safely
 --> src/lib.rs:4:1
  |
4 | pub static VERSION: *const c_char = b"0.1.0\0".as_ptr() as _;
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `*const i8` cannot be shared between threads safely
  |
  = help: the trait `Sync` is not implemented for `*const i8`
  = note: shared static variables must have a type that implements `Sync`

Instead, we can use a function that returns the string:

#[no_mangle]
pub extern "C" fn get_version() -> *const c_char {
    b"0.1.0\0".as_ptr() as _
}

Finally, there are multiple ways to handle versioning within the runtime, depending on how fine-grained (but also more error-prone) it should be:

The simplest way possible: both version strings must be strictly the same.
The plugin system could take advantage of semantic versioning. Only differences in the major version (X.0.0) would be incompatible. The problem in this case is that this is kept track of manually, and it’s possible that a breaking change is introduced by mistake.
Since there are actually many kinds of plugins (connectors, codecs, etc), rather than checking the version for the entire common crate, there could be a version per type of plugin. If a change in the common crate only modifies structures for codec plugins, the rest of the plugins would still work.

Loading plugins

Another complicated topic is plugin distribution and management. In order to make it easier for the user, plugins should be found and loaded automatically. But how exactly should this work? I’ll explain a few ideas.

Firstly, the plugins can be found automatically by searching one or more user-configurable directories. For instance, in Tremor’s case we could use the environment variable TREMOR_PATH. Once we have a list of directories where we should look for plugins there are two ways to do it:

Only checking the immediate files in the directory
Recursively, which is more convenient but might cause issues if the node is too deep. If the user specified / as a directory, the runtime would most likely crash unless we used something efficient like fd or added a depth limit (which is probably the most sensible choice here).

Once we’re traversing a directory, we have to figure out which files are plugins and which aren’t. The easiest way to do it is with file extensions, but this introduces the problem of cross-compatibility. Dynamic libraries usually have a different extension name for each Operating System: Windows uses .dll, Linux and FreeBSD use .so, and macOS uses .dylib, as specified by std::env::consts::DLL_EXTENSION. It would make sense that our runtime only tried to load plugins with their respective extensions.

However, these extensions are just conventions; we could just enforce a single extension name, as libloading suggests. It might be easier if we just used .module for everything, for example. In order to make them even more convenient, it’d be nice if they also worked for all of these Operating Systems within a single file. Apparently, this is called a “Fat binary” and it was used in the past, but it’d be extremely complicated to get working now^[5], so we’ll just forget about it.

Additionally, the Tremor plugin system requires that plugins can be loaded both at initialization time and at runtime. There is a decision to be made in here about how the latter should work:

Manually: after adding the new plugin to the configured directories (or specifying its full path), the user would input in some way that it should be loaded (for example with the CLI tool).
Automatically: the runtime could detect whenever a new plugin is added to the list with a crate like notify. Most Operating Systems have some way to get a notification whenever a file or directory changes. In case a new file was added to any of the configured directories, the runtime could try to load it. This way, it’d work with no user interaction, other than adding the file to one of the directories.
A combination of both: if the directories configured to look for plugins can’t be changed at runtime it might be interesting to also let the user manually load plugins in specific paths.

Handling state

Most plugins will want to keep some kind of state between calls to its interface. For example, the TCP connector will need to keep its socket after its initialization in order to send or receive messages. This means that most of them will follow the following pattern:

let state = plugin.new();
plugin.something(&mut state);

The state is first created with a new function that initializes everything as needed, and then a mutable reference is passed to its functions. The main problem here is, if each plugin is going to have its own type of state, what’s the function signature of Plugin::something, defined in common?

Generics in plugins?

In a regular Rust project we’d just make Plugin::something generic over a common trait that all states should implement. Unfortunately, generics in plugins are fundamentally impossible. In Rust, monomorphization turns generic code into specific code by filling in the concrete types that are used when compiled^[6]. Plugins are loaded at runtime, so they may want to use types the compiler didn’t generate code for.

It’s really easy to prove in Rust with the following example. We’ll try to load an external function with generics:

extern "C" {
    fn foo<T>(_: T);
}

This results in the following error:

error[E0044]: foreign items may not have type parameters
 --> src/lib.rs:2:5
  |
2 |     fn foo<T>(_: T);
  |     ^^^^^^^^^^^^^^^^ can't have type parameters
  |
  = help: replace the type parameters with concrete types like `u32`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0044`.

Interestingly enough, the compiler lets you export generic functions declared in Rust:

extern fn foo<T>(_: T) {}

This confused me in the beginning; it made me think generic functions through FFI were somehow be possible. But as described in the original issue that allowed them, they’re only supported to pass callbacks to C functions.

Note that generics in plugins do work for lifetimes. This will compile:

extern "C" {
    fn foo<'a>(_: &'a str) -> &'a str;
}

Even though lifetimes and generics share the same syntax, in the case of lifetimes they are only annotations for the Rust compiler; monomorphization is not applied.

If you want to know more about this topic, I’d suggest watching this video by Jon Gjengset.

`dyn` in plugins?

The alternative to generics is often using trait object types with dyn. Again, will that work for plugins? Let’s try:

pub trait PluginState {}
pub extern fn foo<T>(_: &dyn PluginState) {}

Compiling…

warning: `extern` fn uses type `dyn PluginState`, which is not FFI-safe
 --> src/lib.rs:2:25
  |
2 | pub extern fn foo<T>(_: &dyn PluginState) {}
  |                         ^^^^^^^^^^^^^^^^ not FFI-safe
  |
  = note: `#[warn(improper_ctypes_definitions)]` on by default
  = note: trait objects have no C equivalent

Nope. dyn is strictly part of the Rust ABI, so it’s not stable for our plugin system.

The C way

There are two popular ways to approach this in C:

Globals, but they are hard to deal with in concurrent programs
void*, which is a pointer with no associated type^[7]

For safety’s sake, let’s see how the second one works. This pattern is used for example in PulseAudio^[8], in which callbacks pass a void* parameter for user data. Here’s a simpler program:

#include <stdio.h>
#include <stdlib.h>

// The state of the plugin
typedef struct {
    int counter;
} plugin_state_t;

// Exported by the plugin, initializes the state
void* new() {
    plugin_state_t* plugin_state = malloc(sizeof(plugin_state_t));
    plugin_state->counter = 0;
    return (void*) plugin_state;
}

// Exported by the plugin, which takes a pointer to its state
void something(void* state) {
    // We know the runtime used `new` to initialize the state, so we can cast it
    // back to its original type.
    plugin_state_t* plugin_state = (plugin_state_t*) state;

    printf("Current state: { counter = %d }\n", plugin_state->counter);
    plugin_state->counter++;
    printf("Final state: { counter = %d }\n", plugin_state->counter);
}

int main() {
    // We initialize the plugin, which returns its state
    void* state = new();
    // When calling anything from the plugin we pass its state
    something(state);
    // Don't forget!
    free(state);
}

This does work perfectly, and we could port it to Rust as a straightforward solution. However, it has the following inconvenients:

It’s very unsafe. We’d need to add some kind of wrapper/macro for the plugin developers to avoid invoking undefined behaviour.
We know nothing about the state. A void* can’t enforce Debug being implemented, nor any base other method or trait that might be of interest to us.

Based on how this works, we can try to extend it by implementing intheritance-based polymorphism manually. This blog post by Michael F. Bryan’s covers the topic extremely well.

Here’s how our previous example would look like, which could be translated to Rust with no problems whatsoever:

#include <stdio.h>
#include <stdlib.h>

// The base plugin type
typedef struct base_state_t {
    void (*print)(struct base_state_t *);
} base_state_t;

// The state of the plugin, child of the above type
typedef struct {
    base_state_t base;
    int counter;
} plugin_state_t;

// The implementation of `print` for the `plugin_state_t` child
void print(base_state_t* state) {
    plugin_state_t* plugin_state = (plugin_state_t*) state;
    printf("Current state: { counter = %d }\n", plugin_state->counter);
}

// Exported by the plugin, initializes the state
base_state_t* new() {
    base_state_t base = {print};

    plugin_state_t* plugin_state = malloc(sizeof(plugin_state_t));
    plugin_state->base = base;
    plugin_state->counter = 0;
    return (base_state_t*) plugin_state;
}

// Exported by the plugin, which takes a pointer to its state
void something(void* state) {
    // We know the runtime used `new` to initialize the state, so we can cast it
    // back to its original type.
    plugin_state_t* plugin_state = (plugin_state_t*) state;
    plugin_state->counter++;
}

int main() {
    // We initialize the plugin, which returns its state
    base_state_t* state = new();
    // When calling anything from the plugin we pass its state
    state->print(state);
    something((void*) state);
    state->print(state);
    // Don't forget!
    free(state);
}

The main difference in the code is the new base class plugin_base_t. It defines a single function print that should be implemented by its children, and it could also include other fields that would be inherited. Casting between base_state_t and plugin_base_t is explicitly allowed by the C standard as long as the base class is the first member in the struct, so this is sound.

This covers all of our necessities. The only remaining problem is that it’s still quite unsafe to use. Thankfully, we can avoid most user errors by using the crate thin_trait_object, which provides a very flexible procedural macro to automatically write all the necessary boilerplate in Rust.

Error Handling

I’ve created a few more plugins to see how this approach reacts to some common errors. Since in the end we aren’t using a sandbox, I wonder what kind of errors we can’t recover from.

The full source for the example that’s supported to work is here. Let’s see a few ways in which the plugin could go wrong:

Missing fields

The plugin-missing directory contains an empty plugin. It doesn’t export any fields at all, like the name or the version. This one is already handled by libloading, actually. When using library.get("name"), if “name” is not exported by the shared object, the following error will show up. See the full code here.

$ make debug-missing
Error when setting up the plugin: plugin-missing/target/debug/libplugin_missing.so: undefined symbol: get_name

Version mismatch

After implementing the versioning system, we can see how these kinds of errors can be caught safely. See the full code here.

$ make debug-versionmismatch
Initializing plugin versionmismatch
Version mismatch. Aborting.
Error when setting up the plugin: version mismatch: 0.0.0 incompatible with 0.1.0

Wrong type

Libloading assumes the type that’s being loaded is correct. If for example the plugin exported the get_name function, but it returned an integer instead of a string, we’d be in undefined-behaviour-land:

Users of this API must specify the correct type of the function or variable loaded. Using a Symbol with a wrong type is undefined.

Ignoring this will cause an unavoidable segfault. See the full code here.

$ make debug-wrongtype
Segmentation fault (core dumped)

Wrong address

Unfortunately, there’s not much we can do about out of bounds pointers. If the plugin exports, e.g., the name with a null pointer, we’ll just get a segmentation fault. See the full code here.

$ make debug-wrongaddress
Segmentation fault (core dumped)

In order to avoid this, the runtime could manually check that the pointer isn’t zero, the usual value for null. But the same would still happen if the pointer’s value was 1 instead of 0. And even if it was within bounds, it could just point to garbage anyway.

Panicking

NOTE: This seems to be Work in Progress, and panicking through the FFI boundary will be allowed after [#![feature(c_unwind)](https://rust-lang.github.io/rfcs/2945-c-unwind-abi.html)] is implemented.

Panicking is not supported in the C ABI; it’s considered undefined behaviour^[9]. If a plugin panics, the entire program will most likely abort. Plugin developers should wrap every single exported function in catch_unwind in order to not crash the entire runtime when something goes wrong. See the full code here.

$ make debug-panic
Segmentation fault (core dumped)

Full implementation

The example at dynamic-connectors approaches the topics covered in this section in the simplest of ways, while still implementing a working plugin system. More specifically:

Versioning requires an exact match between the version of common in the plugin and the runtime.
The plugins are manually loaded given a directory.
The runtime looks for plugins in the immediate files of the directory, i.e., non-recursively.
The state is passed as a void pointer, rather than trying to use inheritance.

Most of these are just decisions to be made by the designer of the system. I chose to go for the easiest options so that we can focus on abi_stable sooner.

I did implement a declarative macro to make plugin-writing easier and less error-prone, just to see how it’d work. It takes care of most of the boilerplate, which basically consists on creating functions for the name, kind, and version returning *const c_char, and exporting the plugin data struct. For the curious, it’s defined in the common directory.

// sample usage of the macro
define_connector_plugin! {
    name: "metronome",
    data: ConnectorPlugin {
        new,
        something,
        is_sink: false,
        is_source: true
    }
}

The plugin system supports multiple types of plugins (connectors, codecs, etc), so there’s actually a specific macro and data structure for each of them.

Apart from the examples listed in the “Error Handling” section, I’ve created a proper plugin that is supposed to work, with the name plugin-metronome. It was supposed to implement the metronome connector, but I’ve decided to just leave that for the next post. I’d rather spend my time writing the real-life example with the abi_stable version, because it’s what we’ll end up using.

Conclusion

This article has covered a lot of questions that one may encounter when trying to use dynamic loading for a plugin system. It’s definitely a complicated task with lots of decisions to make, and plenty of pitfalls. This is why I’d love to try abi_stable in detail, which would let us do the same things but without a line of unsafe.

In my opinion, although abi_stable is a very large crate and somewhat hard to learn, most of the problems this post exposes are greatly simplified thanks to it. In the next article I’ll see the differences between both approaches.

Footnotes

Plugins in Rust: Getting Started

2021-09-05T00:00:00Z

Table of Contents

Welcome to the second article of my “Plugins in Rust” series! Here I will try to actually write some simple code of what a plugin system might look like, and try to answer any questions that may arise while doing so.

Even though the project is specific to Tremor, I try to keep it relatively generic so that it’s useful to future rustaceans interested in plugin systems. So if you don’t really care about Tremor’s specific case, you can skip to the next section.

About Tremor

Getting to know the team

My first official meeting with the Tremor team took place in the beginnings of August. I had already been lurking in their Discord server before, and I had talked with them a few times, so I more or less knew what to expect. But I was still surprised by their welcomingness and how refreshing their dynamics were.

Starting to work on Tremor for the first time with such a large task at first sounds daunting. I was somewhat overwhelmed by that and felt quite some pressure on me. But their words helped me de-stress about it; finishing the project would of course be hard, but no need to worry about things you can’t control yet.

I particularly liked the three “rules” of their workflow:

Have fun
Take good notes
Don’t worry

My mentors were Matthias, Darach, and Heinz. They seemed like very different kinds of people; something they knew and took advantage of. Decision-taking is hard; it requires good communication and an open mind. I also took note of this quote (roughly) from them:

When I have an idea about something, I just post it on the Discord server and wait for it to be destroyed by someone else. If it doesn’t, maybe it’s worth considering.

— Darach

Also note that while writing this post I quite literally broke my leg (tibia & fibula). The team was very supportive, and I took some rest until I went through surgery. Which is another reason as to why this article has taken me longer than usual to release. Thanks a lot to all of them ❤️.

Getting to know other mentees

In the first reunion I was suggested to meet the other LFX mentee at that moment, @aryan9600 on Discord, which seemed like a good idea. I’d seen him a couple of times and knew he’d been contributing to Tremor even before the mentorship.

Aryan turned out to be a very nice colleague! He was very enthusiastic about the project, and he explained me his progress in detail. In summary, he was starting to add gRPC support to Tremor, and wasn’t sure how to approach it so that new interfaces could be added outside the compilation time. He had multiple ideas on how to implement this, but the one that interested me the most was via Wasm or dynamic linking, which was exactly what I was trying to do as well for the system. This meant I’d probably spend more time with him, as our projects were more interconnected than I thought.

Learning about the internals of Tremor

I’ve always known that the original intent for the system was the typical architecture via dynamic linking, but I wanted to explore the viability of Wasm or other alternatives before ruling it out. Would this idea be destroyed by someone in the end, or does it stand a chance? In order to know that I’d first need to find more specific requisites by defining the basic interfaces of the system. Which is very hard at first because you need to know more about the internals of Tremor and how it works.

Luckily, Matthias spent the second meeting explaining to me how Tremor works, with the help of some drawings, taking a look at the code, and running it. He also taught me the novel concept of a connector, which is just an abstraction over both sources and sinks. The thing was that connectors were still Work In Progress, and Matthias needed to know how the system would work to modify the Connector interface. Mainly how to combine plugins and generics, which is what I’ll try to figure out in this article.

Back when I talked to Aryan we ended up asking ourselves the same exact things. Turns out that the generated code with tonic for gRPC uses generics, which might not be compatible with dynamic loading.

The first steps

Up until now I had mostly investigated about the technologies that could be used for the system, but I was suggested that I should first take a look at how the interfaces would actually look like. That way it would be easier to take the final decision for the system’s technology, according to the requirements for these interfaces. In summary, the following questions arose:

What’s the performance hit of the system?
Can we avoid serialization?
Is it possible to use generics in the system?
Otherwise, is it possible to use dyn in the system?
If all else fails, what can be done to represent a “generic” trait object in the interface Tremor exports for plugins?

This meant I had to do more research. I spent a few days reading different articles and forum discussions, but it’s a rabbit hole 🐰! A few links I recommend checking out:

Michael F. Bryan has put a lot of effort into teaching how dynamic loading and Wasm work in Rust. Thanks a lot to him!

Experimenting with dynamic loading

ABI unstability, it’s much worse than it seems

I recently had this very late realization about ABI stability in Rust. Up until this point I thought that even though Rust’s ABI is unstable, one could dynamically load a library safely as long as both the library and the main binary were compiled with the same exact compiler/std/etc version. I had read this many times in forums like this one on Reddit and in blogposts such as “Plugins in Rust”, so I assumed it was true.

But turns out that the ABI may not only break between compiler versions, but also compiler runs. rustc doesn’t guarantee that a layout is going to be the same in different executions. This is proved by rust-lang/compiler-team#457, the new unstable -Z randomize-layout flag. It’s pretty much self-explanatory: it randomly reorders repr(rust) layouts for testing purposes. The same thing could happen in the future without an explicit flag; an optimization may cause the repr(rust) layouts to change between compilation runs. It’s briefly mentioned in the Rust reference as well:

Type layout can be changed with each compilation. Instead of trying to document exactly what is done, we only document what is guaranteed today.

Props to the devs at the #black-magic channel in Rust’s Discord server, who helped me understand this. Specially Yandros and Kixiron, both of them very respectable contributors to the Rust compiler/community.

This basically means that we are forced to stick to the C ABI with #[repr(C)], and that we should use abi_stable in order to have access to a stable std library as well, instead of re-implementing everything ourselves from scratch. On the positive side, it means that plugins could be implemented in any language, but that wasn’t important for Tremor’s case since the beginning anyway.

Getting a simple example running

I’ve created the pdk-experiments repository, where I’ll write various examples of how the plugin system might work. The first experiment is in the dynamic-simple directory, with a small example of how to dynamically load plugins with Rust.

We first need a crate called common, which defines the interface shared by the plugin and the runtime in the main binary. In this case it’s just a pointer to a function with the C ABI. We can specify the ABI with extern "C", or simply extern, as "C" is already its default value^[1]. To keep it simple it’ll just compute the minimum between two integers:

// in common/src/lib.rs
pub type MinFunction = unsafe extern fn(i32, i32) -> i32;

With it, the plugin crate may export its own implementation. In this case I’ll declare a static variable, but the example showcases how extern may work as well. Since we want to use the C ABI, we’ll have to specify crate-type as cdylib in our Cargo.toml. Note that #[no_mangle] is necessary so that the variable’s name isn’t mangled and we can access it when dynamically loading the library.

// in plugin-sample/src/lib.rs
#[no_mangle]
pub static with_static: MinFunction = min;

pub extern fn min(a: i32, b: i32) -> i32 {
    a.min(b)
}

Finally, the main binary can load the library with libloading, which requires a bit of unsafe. I was looking forward to using a different library because of how easy it seems to end up with undefined behaviour in that case. I found out sharedlib was abandoned, as no commits had been made since 2017, leaving dlopen as the only alternative. Which was updated two years ago as well, but their GitHub repo seemed somewhat active in comparison.

For now, I’ll just use libloading for being the most popular crate, and perhaps I’ll consider using dlopen in the future. In terms of relevant features and performance they’re pretty close anyway^[2]. Here’s what the code looks like:

// in src/main.rs
fn run_plugin(path: &str) -> Result<(), libloading::Error> {
    unsafe {
        let library = Library::new(path)?;
        let min = library.get::<*mut MinFunction>(b"plugin_function\0")?.read();
        println!("Running plugin:");
        println!("  min(1, 2): {}", min(1, 2));
        println!("  min(-10, 10): {}", min(-10, 10));
        println!("  min(2000, 2000): {}", min(2000, 2000));
    }

    Ok(())
}

We can run it with the following commands (though the Makefile in the repo will do everything for you):

$ cd plugin-sample
$ cargo build --release
$ cd ..
$ cargo run --release -- plugin-sample/target/release/libplugin_sample.so
Running plugin:
  min(1, 2): 1
  min(-10, 10): -10
  min(2000, 2000): 2000

Cool! This raises a few questions that I should learn more about:

Since we’re using the C ABI, is it perhaps best to declare the bindings in C? The common crate I introduced earlier could just be a header.
There are many options to configure crate-type as a dynamic library. What are they and which one should I choose?
I faintly remember that rlib files are Rust-only objects with additional metadata for things like generics. Could that possibly work at runtime? As in, is there an equivalent to COM in Rust, or maybe like JAR files in Java?

Generating bindings

The public interface for the plugins can be written either in Rust (thanks to extern "C") or directly in C. There are two commonly used projects when writing bindings:

rust-bindgen generates Rust bindings from C code
cbindgen is the opposite; it generates C bindings from Rust code.

Some examples of its usage:

hyper is a crate completely written in Rust that exposes C headers for compatibility, so it uses cbindgen to generate them automatically.
pipewire_rs exposes the interface of PipeWire, written in C, so that it’s also available from Rust, thanks to rust-bindgen.

Since we’re going to write the plugin system in Rust, the most appropiate choice would be to use Rust for the interface as well. And if we wanted to make the plugin interface available to other languages — which is not a concern right now — it’d be as “easy” as setting up cbindgen.

`crate-type` values

There are two ways to configure dynamic linking with the crate-type field in the crate’s Cargo.toml:

dylib
cdylib

Once again, this difference has to do with the ABIs in the dynamic library^[3]. cdylib is meant for linking into C/C++ programs (so it strips away all functions that aren’t publicly exported), and dylib is meant for Rust libraries.

When compiling the previous example with dylib, the resulting shared object for the plugin has a size of 4.8Mb, whereas with cdylib it’s just 2.9Mb. So while both of these will work for our C ABI, cdylib is clearly the more appropiate choice.

`rlib` files

rlib is another value for crate-type to generate Rust static libraries, which can then be imported with extern crate crate_name. But since rlib files are static libraries, they can’t be loaded at runtime, so they’re of no use in a plugin system.

Here’s a crazy idea though: What if the rlib files were dynamically loaded as plugins with the help of MIRI? I recently learned about it, and quoting its official documentation:

[MIRI is] an experimental interpreter for Rust’s mid-level intermediate representation (MIR). It can run binaries and test suites of cargo projects and detect certain classes of undefined behavior.

You can use Miri to emulate programs on other targets, e.g., to ensure that byte-level data manipulation works correctly both on little-endian and big-endian systems.

Hmm. Could it possibly be used to interpret Rust code? In some way this would be very similar to using WebAssembly, but theoretically with less friction, as MIR is specific to Rust and plugin development would be as easy as in the case of dynamic loading with Rust-to-Rust FFI. A few things to consider:

Is this even possible?

The Rust compiler itself uses MIRI to evaluate constant expressions^[4] via the rustc_mir crate. But taking a quick look it seems to be created specifically for the compiler, at a very low level, and without that much documentation. Plus, it’s nightly-only. It does seem possible, but I wasn’t able to get a simple example working.
Is MIR stable?

MIR is unfortunately unstable^[5], so we’d have the same incompatibility problems between plugins and the main binary.
Is the overhead of MIRI worth it?

Considering the previous answers, no, but it was cool to consider and learn about :)

Can we use WebAssembly for this?

I also tried to write a simple example of how plugins would work with WebAssembly. They are available in the wasmer-simple and wasmtime-simple directories. It took me considerably more effort to understand and get running than with dynamic linking, even following Free Masen’s guide and Wasmer’s official documentation. But at least I didn’t have to write that much unsafe (it’s still needed to load or store data from Wasm’s virtual memory, which I’ll explain later on).

The following snippet is what the plugin would look like. Note that this time we use pub without even considering static to export a pointer to the function. Wasm does have support for globals, but since handling complex types — those other than numbers, a function in this case — isn’t trivial, it’s not worth it.

// in plugin-sample/src/lib.rs
#[no_mangle]
pub fn with_extern(a: i32, b: i32) -> i32 {
    a.min(b)
}

For the runtime, since it’s a simple example I’ll try both Wasmer and Wasmtime. For more complex things I’ll just use Wasmer, since in an earlier post it seemed like the best alternative. I also wanted to try wasm_plugin, which includes procedural macros to help reduce the overall boilerplate (which will be more important later on), but it seems to be abandoned since 2019. It’s interesting to try the lower-level interface of the runtimes myself to learn more about it anyway, so I’ll just use that for now. If we ended up using Wasm for Tremor I’d try updating and maintaining wasm_plugin to keep the code boilerplate-free.

// in src/main.rs (wasmer)
pub fn run_plugin(path: &str) -> Result<(), Box<dyn Error>> {
    // For reference, Feather also reads the plugins with `fs::read`:
    // https://github.com/feather-rs/feather/blob/07c64678f80ff77be3dbd3d99fbe5558b4e72c97/quill/cargo-quill/src/main.rs#L107
    let module_wat = fs::read(&path)?;
    let store = Store::default();
    let module = Module::new(&store, &module_wat)?;
    let import_object = imports! {};
    let instance = Instance::new(&module, &import_object)?;

    println!("Running plugin:");
    let min_extern: NativeFunc<(i32, i32), i32> = instance.exports.get_native_function("with_extern")?;
    println!("  min(1, 2): {}", min_extern.call(1, 2)?);
    println!("  min(-10, 10): {}", min_extern.call(-10, 10)?);
    println!("  min(2000, 2000): {}", min_extern.call(2000, 2000)?);

    Ok(())
}

// in src/main.rs (wasmtime)
pub fn run_plugin(path: &str) -> Result<(), Box<dyn Error>> {
    let engine = Engine::default();
    let module = Module::from_file(&engine, &path)?;
    let mut store = Store::new(
        &engine,
        WasmState {},
    );
    let imports = [];
    let instance = Instance::new(&mut store, &module, &imports)?;

    println!("Running plugin:");
    let min_extern = instance.get_typed_func::<(i32, i32), i32, _>(&mut store, "with_extern")?;
    println!("  min(1, 2): {}", min_extern.call(&mut store, (1, 2))?);
    println!("  min(-10, 10): {}", min_extern.call(&mut store, (-10, 10))?);
    println!("  min(2000, 2000): {}", min_extern.call(&mut store, (2000, 2000))?);

    Ok(())
}

For a simple example they’re pretty much the same. Running them:

$ rustup target add wasm32-wasi
$ cd plugin-simple
$ cargo build --target wasm32-wasi --release
$ cd ..
$ cargo run --release -- plugin-sample/target/wasm32-wasi/release/plugin_sample.wasm
Running plugin:
  min(1, 2): 1
  min(-10, 10): -10
  min(2000, 2000): 2000

AFAIK Rust has multiple compilation targets for Wasm. Which one should I be using?
What’s the difference between the wasmer crate and wasmer_runtime?
What about exporting types more complex than an i32? Is it possible to export a struct that implements a specific trait?

WebAssembly targets

Wasmer docs don’t mention this much because targets are related to the plugin, rather than the runtime. But Wasmtime’s book does include a section about compiling Rust to WebAssembly:

wasm32-wasi - when using wasmtime this is likely what you’ll be using. The WASI target is integrated into the standard library and is intended on producing standalone binaries.

wasm32-unknown-unknown - this target, like the WASI one, is focused on producing single *.wasm binaries. The standard library, however, is largely stubbed out since the “unknown” part of the target means libstd can’t assume anything. This means that while binaries will likely work in wasmtime, common conveniences like println! or panic! won’t work.

wasm32-unknown-emscripten - this target is intended to work in a web browser and produces a *.wasm file coupled with a *.js file, and it is not compatible with wasmtime.

So basically what we need is wasm32-wasi. By the way, even though Wasmer was my initial choice, to be honest I found Wasmtime’s docs to be much more detailed and well-organized.

`wasmer` vs `wasmer_runtime`

This was confusing for me at first, since both crates seem to have a very similar interface and almost the same set of authors. Some tutorials used wasmer, others wasmer_runtime.

The difference seems to be that wasmer_runtime was updated about a year ago, while wasmer got bumped to v2.0.0 just two months ago. The last release of wasmer_runtime is v0.17 (v0.18 seems to be yanked), and the first one of wasmer is v0.17 as well, so my bet is that wasmer_runtime is the name of the crate their team used previously, and they eventually deprecated it in favor of wasmer.

I’ve opened an issue upstream so that this is hopefully more clear to future users.

More complex types

The WebAssembly specification only defines integers and floating point as its supported types^[6]. There are three ways to handle non-trivial types like structs or enums:

Option 1: The Interface Types Proposal

This proposal for WebAssembly defines the binary format for encoding and decoding the newly supported types, and specifies a set of instructions to transform the data between WebAssembly and the outside world. Note that this proposal is not meant to define a fixed representation of e.g., a string in Wasm, it attempts to allow representation-agnostic high-level value types.

These new high-level value types are called interface types. The current proposal defines them as:

Floating point of 32 and 64 bits
Signed and unsigned integers of up to 64 bits
Single characters
Lists (a string would be a list of characters)
Records (basically like Rust structs)
Variants (basically like Rust enums or unions, it would enable Option, Result, etc)

Additionally, language-neutral interfaces for WASI can be defined with the experimental witx files^[7], which makes it easy to define a common ABI in order to interact from different programming languages^[8]^[9]. They look like this:

(use "errno.witx")

;;; Add two integers
(module $calculator
  (@interface func (export "add")
    (param $lh s32)
    (param $rh s32)
    (result $error $errno)
    (result $res s32)
  )
)

Please refer to the proposal itself for more detailed information; it’s very well explained.

The main problem is that this proposal is still at Phase 1. It’s still actively being worked on, and its specification is far from stable.

At the plugin level the wasm_bindgen crate seems to be ideal, and it already works well. It’s a very simple procedural macro that can be added to the exported functions in the plugin in order to automatically add support for Interface Types.
The wiggle crate can be used to generate Rust code from witx files. For the previous snippet of code, wiggle’s macro will generate a Calculator trait and the defined types in another module named types.
The runtimes, however, don’t work with Interface types:
- Wasmtime did support this in the past until their implementation was removed after being considered outdated. As this issue indicates, it still hasn’t been updated.
- Wasmer has the wasmer_interface_types crate, but with a similar story; it’s outdated. There’s this issue as a continuation of Wasmtime’s, which explains the situation.

In the end I wasn’t able to get Interface Types working, nor I considered them worth my time, as it’s still too early.

Option 2: The hacky-but-working way

Via pointers and a shared memory. The user has to first construct and serialize the complex types, and then save them into Wasm’s memory, which can be accessed directly by the runtime or the plugin with pointers. This is what Feather or Veloren do, in case you want more details.

Not only does this require a serialization and deserialization step and writing/reading all the data from memory, but also it’s very cumbersome to use and easy to mess up. It’s somewhat trivial though, so a procedural macro like the now outdated wasmer-plugin could simplify it. For now, bincode can be used for the serialization steps manually.

I did try this and got it working by following Free Masen’s guide, but it wasn’t worth exploring in depth because it requires a [de]serialization step Tremor can’t afford.

Option 3: Separate executables

Another way I’ve seen to work around this is by using Wasm as an executable instead of a library. Zellij does this, and communicates with the plugins via the standard input and standard output streams.

Unfortunately, this also requires [de]serializing and has the overhead of writing/reading all the data from stdin and stdout.

Conclusion

Ultimately, this has forced me to discard WebAssembly for Tremor’s Plugin Development Kit. Performance is really important in this specific case, and Wasm doesn’t offer a solution to efficiently pass data between the host and the plugin. For most use cases Wasm still may be of interest, so hopefully I’ve included enough links and ideas so that you can get started if it’s the way you want to go.

Wrapping up

Today we’ve learned a lot of things both about dynamic loading and WebAssembly. Unfortunately, I’ve had to discard the latter for Tremor’s case, but it may be of interest to projects that aren’t so focused on performance. Or maybe by the time you, the reader, have finished this article, the issues I found are already stabilized and broadly used, which would make Wasm an even more appealing choice.

In the next post, I will try to get a more advanced example of a plugin system with dynamic loading. The goal is to simulate the end-product by writing it as close as possible to how it would be used in real life. This way, we will also be able to benchmark it against a statically linked implementation and analyze the performance overhead we’ll end up getting.

I did write a few benchmarks for the examples provided in this article, but they aren’t fair at all nor representative of a real-life situation. They simply load the plugin and run the min function once, or in the case of dynamic loading twice (one with static and another with extern). And since they’re very simple examples, the Wasm ones don’t include the [de]serialization part that may introduce even more overhead. Knowing that, the results are the following, which is more or less what I was expecting:

test test::dynamic_simple ... bench:     139,702 ns/iter (+/- 34,699)
test test::wasmer_setup   ... bench:     967,633 ns/iter (+/- 203,933)
test test::wasmtime_setup ... bench:     988,500 ns/iter (+/- 363,244)

I hope you enjoyed reading this post and that you learned something from it! You can leave any suggestions in the comments below.

Footnotes

Why you shouldn't obsess about Rust "features"

2021-07-06T00:00:00Z

Table of Contents

Rust makes it very easy to express conditional compilation, especially thanks to Cargo “features”. They’re well integrated into the language and are very easy to use. But one thing I’ve learned by maintaining RSpotify (a library for the Spotify API) is that one shouldn’t obsess over them. Conditional compilation should be used when it’s the only way to solve the problem, for a number of reasons I’ll explain in this article.

This might be obvious, but to me, it wasn’t so clear back when I started using Rust. Even if you’re already aware, it might be a good reminder — maybe you forgot about it in your latest project, and you added an unnecessary feature.

Conditional compilation isn’t anything new; C and C++ have been doing it for a long time now. So this principle can also be applied in these cases. However, in my experience, it’s much easier to work with conditional compilation in Rust, meaning that it’s more likely to be misused.

The Problem

I went through this dilemma when deciding how to configure cached tokens in RSpotify. Said library gives you the possibility of persistently managing the authentication token via a JSON file. That way, when the program is launched again, the token from the previous session can be reused. One doesn’t have to follow the full auth process again — that is, until the token expires.

Originally, this was going to be a compile-time feature named cached_token. I didn’t give it much thought; why would one need the code to save and read the token file if you just don’t need it? The easiest way to do that is using a feature you can just toggle in your Cargo.toml.

I later worked on another very similar feature, refreshing_token. When optionally enabled, the client would automatically refresh expired tokens. As this pattern appeared more and more in the library, I wanted to make sure its design was optimal. Once I took a deeper look, I began to find the many inconveniences of Cargo features:

They’re inflexible: you can’t have a client with cached tokens and another without them in the same program. It’s a library-wide thing, so you either enable them or you don’t. Obviously, they’re not configurable at runtime either; a user might want to choose what kind of behaviour to follow while the program is running.
They’re ugly: writing #[cfg(feature = "cached_token")] is much more weird and verbose than a plain if cached_token.
They’re messy: features are hard to manage in the codebase. You can find yourself in the Rust equivalent of an #ifdef hell very easily.
They’re hard to document and test: Rust doesn’t provide a way to expose the features of a library. All you can do is list them manually in the main page of the docs. Testing is also harder because you have to figure out what combinations of features to use in order to cover the entire codebase, and apply them whenever you want to run the tests.

All of these are supposedly outweighed by just being guaranteed that the binary won’t have code you don’t need. But how true is that, really? And how important is it?

An Alternative

Turns out that one of the simplest optimizations a compiler can implement is the propagation of constants. This, in combination with the removal of dead code, can result in exactly the same effect as features, but in a more natural way. Instead of adding features to configure the behaviour of your program, you can do the same with a Config struct. You may not even need a struct if it’s just a single option to be configured, but that way it’s future-proof. For example:

#[derive(Default)]
struct Config {
    cached_token: bool,
    refreshing_token: bool,
}

You can then modify your client in order to optionally take the Config struct:

struct Client {
    config: Config
}

impl Client {
    /// Uses the default configuration for the initialization
    fn new() -> Client {
        Client {
            config: Config::default(),
        }
    }

    /// Uses a custom configuration for the initialization
    fn with_config(config: Config) -> Client {
        Client {
            config,
        }
    }

    fn do_request(&self) {
        if self.config.cached_token {
            println!("Saving cache token to the file!");
        }
        // The previous block used to be equivalent to:
        //
        // #[cfg(feature = "cached_token")]
        // {
        //     println!("Saving cache token to the file!");
        // }

        if self.config.refreshing_token {
            println!("Refreshing token!");
        }

        println!("Performing request!");
    }
}

Finally, the user can customize the client however they want in the code itself in a very natural way:

fn main() {
    // Option A
    let client = Client::new();

    // Option B
    let config = Config {
        cached_token: true,
        ..Default::default()
    };
    let client = Client::with_config(config);
}

Proving that you end up with the same code

Thanks to the awesome Compiler Explorer, we can make sure this compiles as we expect it to with this snippet:

It seems that as of Rust 1.53, for values of opt-level greater or equal than 2, the code for the deactivated features doesn’t even appear in the assembly (it’s easy to see by taking a look at the strings at the end). cargo build --release configures opt-level to 3^[1], so it shouldn’t be a problem for production binaries.

And we aren’t even using const! I wonder what will happen in that case. With this slightly modified snippet:

Hmm. We actually get the same results. The generated assembly is exactly the same, and the optional code is optimized away only starting at opt-level=2.

The thing is that const just means that its value may (and not must) be inlined^[2]^[3]. Nothing else. So we still don’t have anything guaranteed, and inlining isn’t enough to simplify the code inside the function.

So for what I’ve investigated it seems to be best to just not worry about it and use a variable instead of const. It looks better and gets the same results.

You can probably afford the overhead anyway

Even if the previous optimization wasn’t implemented, would the optional code cause any harm in the final binary, really? Are we overengineering the solution, as always? Truth is, the optional code for cached/refreshing tokens isn’t even that much bloat.

It depends, of course, but binary bloat isn’t that much of a problem for higher level binaries, in my opinion. Rust already statically embeds its standard library, its runtime, and a ton of debug info in each binary, which sums up to around 3 MB. And the only overhead you may get at runtime is a branch.

Conclusion

Sometimes you just have to use conditional compilation; there’s no way around it. You might be dealing with platform-specific code or want to reduce the number of dependencies of your crate, in which cases features are super helpful.

But that wasn’t RSpotify’s case; conditional compilation was definitely not the way to go. When you’re about to introduce a new feature to your crate, think to yourself, “Do I really need conditional compilation for this?”.

Neither cached_token nor refreshing_token follow the usual reasoning as to why a feature might be added. They don’t give access to new functions/modules. They don’t help get rid of optional dependencies. And they certainly aren’t platform-specific features. They just configure the behaviour of the library.

In order to avoid this, perhaps the naming for features could have been different? Enabling support for cached tokens certainly sounds like a “feature”, while OS-specific code doesn’t really seem like one. I also find it confusing sometimes, and Google agrees with me in this one. Looking for information related to Rust features might return completely unrelated stuff just because the result has the word “feature” but meaning “an attribute or aspect of the program”. Kind of like how you have to google “golang X” instead of “go X” because otherwise it doesn’t make sense. But whatever, it’s too late for my opinion anyway.

Anyhow, I hope you learned something new, or that this was at least a good reminder! If you have any suggestions please leave them in the section below :)

Footnotes

Plugins in Rust: The Technologies

2021-05-17T00:00:00Z

Table of Contents

Welcome to the “Plugins in Rust” series! During the next months I’ll be involved in a project with Tremor, for which I need to implement a Plugin System. The goal is to maximize modularity and reduce Rust’s wild compilation times. The implementation will end up being done with dynamic loading, but I will cover all the alternatives first to make sure it’s the best approach for my use-case.

In this first article I will analyze the different ways in which our plugin system may be implemented in Rust. Note that I’m no expert in the topic; my objective is to merely collect all the information I’ve found so far and present it as a summary, with enough resources for the reader to learn more on their own and make their own decisions.

The Problem

This is actually part of my Google Summer of Code project for Tremor. While this will specifically cover Tremor’s use case, I’m sure many other applications in the future will have similar requirements for their own program. And considering the few articles I found about plugins in Rust, this series may be helpful to the community.

NOTE: Google didn’t end up allocating a slot for me in the GSoC, but I’ll still be able to continue through a LFX Mentorship.

After showing the proposal to the Tremor team, I was given an important suggestion: in this case there isn’t really a need to be able to write plugins from multiple programming languages; it’s not worth the additional complexity. It would definitely be interesting but with this I can focus on the main problems I discovered in my proposal: the ABI instability, and safety.

Besides that, the rest of the main points to consider remain the same:

Mandatory: Being able to load/unload the plugins both at start-time and at run-time.
Mandatory: Cross-platform support
Mandatory: Low overhead
Mandatory: Available at least from Rust
Extra: Safety
Extra: Backwards compatibility
Extra: Not much work to port from the existing implementations

Safety Concerns

I’ve been wanting to take a look at the possible safety concerns a plugin system could bring to Tremor since the initial proposal.

NOTE: Depending on the approach that is taken for the Plugin System, implementing solutions to some of the points I make below can be really complex, so I wouldn’t work on them for the initial deadline. For now, it’s something we have to be aware of.

Unsafe Code

Many of the technologies that can be used to implement a Plugin System work with unsafe code, like FFIs or IPC Memory Sharing. This isn’t necessarily a problem if it’s self-contained and thoroughly reviewed, but we lose some safety guarantees provided by Rust, increasing the maintenance cost of the library.

It takes considerably more work to make sure the implementation is sound, even with tools like MIRI, which I plan on integrating into Tremor if I do end up having to use unsafe.

Error Resilience

Rust doesn’t protect its users from leaking memory. In fact, it’s as easy as calling mem::forget. The thing is that if a plugin leaks memory, the entire process also does, meaning that Tremor’s performance could be affected by incorrectly developed plugins.

This doesn’t happen just with memory leaks; a plugin could abort or panic and crash the entire thing very easily.

Ideally, Tremor could detect plugins that aren’t performing well and stop them before it’s too late. The core of the program could continue running even when a plugin fails, and perhaps warn the user about its malfunctioning, for optimal error resilience.

Remote Code Execution via Plugins

This was a problem with Internet Explorer, for example. It used COM and ActiveX, which implemented no sandboxing at all and were running directly on the user’s machine, allowing malware to be included in plugins with the ability to execute arbitrary code on the host machine. This can be less of a problem when installing only trusted plugins via digital signatures, but it’s still a huge risk.

In the case of Tremor plugins this is a similar problem. The end user — those who will add plugins to their configuration — is a developer, who should be more wary of what they’re including in their own projects. But the truth is that it isn’t any better.

I would compare the situation more specifically to how package managers like npm work (or anything that deals with dependencies, really). The entire infrastructure is usually based on trust; noone’s really stopping you from uploading a malicious package/crate for remote code execution or to steal data^[1]^[2]. Plugins are like dependencies in this case; they have full access to the host’s computer, and thus shouldn’t be trusted by default.

An improved approach to Node + npm would be something like Deno, which is a secure by default JavaScript/TypeScript runtime. This is enabled thanks to sandboxing, and requires the developer to manually toggle, e.g., access to the filesystem or to the network. This is no panacea because people might end up enabling whatever permission is required by the dependency they want to install without thinking about it, but it’s similar to unsafe: at least it makes you aware that you might mess up.

One could argue that their program is realistically going to be running in a Virtual Machine or a container most of the times, where this may not matter so much. But should you really assume that? Should user safety rely on the fact that the machine is running on an isolated kernel? Containers, by the way, introduce a much bigger overhead in comparison to some sandboxing methods. And even if the system as a whole is isolated, there is still a possibility for internal leaks: the Postgres plugin has access to whatever is being used by the Apache Kafka plugin, which may have sensitive logs you don’t want anyone to see.

Backward Compatibility

There needs to be some kind of version handling within this project. The core of Tremor may change with frequency, which will break plugins based on previous versions. If the plugin receives a struct from the core, but the struct had one of its members removed in a new version, it will result in undefiend behaviour.

Possible Solutions

The easiest fix for backward compatibility issues I can come up with is to serialize and deserialize the data with a flexible protocol rather than using its binary representation directly. If something like JSON is used to communicate between the core and its plugins, adding a field to a message won’t break anything, and removing one can be done via a deprecation process. Unfortunately, this introduces some overhead the application may not be interested in.

Other more involved methods for binary representations include^[3]:

Reserving space in the struct for future use.
Making the struct an opaque type, requiring function calls to get the fields.
Giving the struct a pointer to its “version 2” data (opaque in “version 1”).

Avoiding Breakage

There are times when breakage is inevitable. Tremor may want to rewrite part of its interface or finally remove a deprecated feature without being afraid of breaking all the plugins previously developed.

For that, the plugins must embed a bit of metadata about the different versions of rustc/interface/etc it was developed for. Then, when it’s loaded by Tremor, it may check if they’re compatible, rather than breaking in mysterious ways.

Possible Approaches

The following are the most viable technologies that could be used as the base of the system. Some of them won’t match the requirements I mentioned earlier at first glance, but it’s a good idea to at least consider all of them. I haven’t written a line of code yet, so if an approach were to catch someone’s eye we could investigate more about it. I will rate each of the alternatives in a scale from 1 to 5 (approximately) in order to easen the decision-taking.

Scripting Languages

Plenty of projects use scripting languages to extend their functionality at runtime, like Python, Ruby, Perl, Bash, or JavaScript. Most notably, Vim created its own scripting language, Vimscript, in order to be fully customizable, and NeoVim is now pushing Lua as a first-class language for configuration. Even Tremor itself has the interpreted query language Troy for configuration.

Lua is seen in game development; it’s a quite simple language with a very performant JIT implementation, which in any case I think would be the best option here. It could be embedded into our runtime (it’s only 281 kB compiled!^[4]) and used to load plugins at either start-time or run-time. And knowing it’s used in games, which are obsessed with performance, it might not be that much of a big deal in that regard.

NOTE: There are languages specifically designed to extend Rust’s capabilities, which you might be interested in, but I’ll just simplify this part by going for Lua.

Rust has libraries like rlua which provide bindings for interoperability with Lua. rlua in particular seems to focus on having an idiomatic and safe interface, which is rare for a bindings library and good news, though it seems to be currently semi-abandoned, and forked by mlua. Unfortunately, after digging a bit the Rust ecosystem for Lua bindings doesn’t seem to be as mature as I’d like for a project this big; there’s still some work to do to reach more stability.

Lua gets extra points in safety. It’s possible to sandbox it by blocking whichever functions you don’t want users to access^[5] (though it’s “tricky and generally speaking difficult to get right”). Similarly, one can also handle resource exhaustion issues within Lua programs. Not to mention that this wouldn’t require unsafe at all with an optimal set of bindings.

Anyhow, the main deal-breaker with scripting languages in this case is that it would be extremely complicated to port everything in Tremor so that it can be used from Lua. For new projects this could perhaps be something interesting, but not if the entirety of the already existing plugins have to be rewritten.

Still, it’s a pretty interesting option for new projects, as you have ABI stability guarantees, solid safety overall, and it’s very straightforward to use.

5/5 Cross-platform support
4/5 Low overhead
3/5 Rust availability
5/5 Safety
5/5 Backwards compatibility
0/5 Ease of porting existing implementations

Inter-Process Communication

Another possibility for plugins is to define a protocol for Inter-Process Communication, turning your program into a server that extends its capabilities by connecting to external plugins. For instance, most text editors use this method to support the Language Server Protocol, which uses JSON-RPC.

There are three ways to do IPC, which I’ll briefly list below. Performance-wise, this graph shows a comparison of the overhead of each of them^[6]:

Option 1: Based on Sockets

Sockets are the “worst”-performing alternative in the previous chart, but they’re so common and easy to use in most languages that it’s worth taking a look at. Using relatively lightweight protocols like Protocol Buffers, the performance would be close to passing raw structs, but with improved backwards/forwards compatibility^[7]. JSON would probably not make that big of a difference in terms of performance either. This would make it possible to write a plugin in any language as well — including Rust — as long as there’s an implementation of the protocol available. But there’s still noticeable overhead when communicating via sockets; sending and receiving the messages can be much costlier than just calling a function, even if this happens on localhost.

This alternative is much more interesting than Scripting Languages for Tremor’s specific case: we don’t have to completely rewrite everything, since Rust can still be used, and implementing the protocol to communicate between the Tremor core and its plugins should be as easy as #[derive(Serialize)] for sending and #[derive(Deserialize)] for receiving.

As to safety, separate processes imply that malfunctioning plugins don’t affect Tremor directly, and the system basically consists on implementing servers, which require no unsafe at all and has much more popularity and support in Rust. It’s still hard to properly sandbox the plugins, though.

Overall, I consider this a very solid solution, with its main drawback being performance. I can’t really guess the effect this would have in Tremor’s speed, so I would love to create a quick benchmark when I get to implement the first prototypes to see if it’s actually the best choice, if Tremor devs think it’s worth my time.

5/5 Cross-platform support
3/5 Low overhead
5/5 Rust availability
3/5 Safety
5/5 Backwards compatibility
5/5 Ease of porting existing implementations

Option 2: Based on Pipes

Pipes have always been fairly popular specifically on Unix systems, and enable Inter-Process Communication with less overhead than sockets. They are made to be ran on the same machine, which is exactly what we need. The terminal file manager nnn uses this approach: plugins can read from a FIFO (Named Pipe) to receive selections from nnn (lists of files or directories) and act accordingly.

The rest is basically the same as with Sockets, maybe with extra points for performance, and less for Rust availability, since there don’t seem to be any reliable libraries for pipes (maybe interprocess or ipipe). But really, are libraries necessary at all? The std library has support for cross-platform pipes when executing external commands for stdin, stdout, and stderr, which most times is enough. The plugin can just use stdin to receive messages and stdout to send them. If that’s enough for your case then it’s vastly simplified.

5/5 Cross-platform support
4/5 Low overhead
4/5 Rust availability
3/5 Safety
5/5 Backwards compatibility
5/5 Ease of porting existing implementations

Knowing that the plugins are intended to be on the same machine as the core of Tremor, there’s no need to actually send and receive messages. One can share memory between multiple processes and send notifications to receive updates. The performance is comparable to using FFI, since the only overhead is the initial cost from setting up the shared pages, having regular memory access afterwards^[8].

This feature heavily depends on the system’s kernel, so it may hurt the “Cross-Compatibility” requirement. We have libraries like shared_memory + raw_sync in Rust that wrap all the OS implementations under the same interface, but admittedly, they don’t seem anywhere near as popular as most of the other alternatives. Not to mention that the examples for shared_memory do use unsafe, and a lot of it.

Maybe if it was easier to use this would be a good idea, but IPC shared memory doesn’t seem to be any better than FFIs overall.

5/5 Cross-platform support
5/5 Low overhead
2/5 Rust availability
2/5 Safety
3/5 Backwards compatibility
5/5 Ease of porting existing implementations

Dynamic Loading

This is possibly the least weird way to implement a Plugin Development Kit, i.e., it’s the most popular method I’ve seen outside of Rust. A Foreign Function Interface can allow us to directly access resources in separately compiled objects, even after the linking phase with dynamic loading. It’s one of the fastest options available because there’s no overhead at all after dynamically loading the library.

The main library for this is libloading. There’s also the less popular dlopen and sharedlib, with some small differences^[9]. It seems to be a lower-level implementation for any kind of FFI that requires unsafe for almost everything — what I was expecting. Based on it there’s dynamic_reload, which is very interesting in order to “live reload” the plugins when they are recompiled. This would be useful for the development process of the plugins, since it also handles unloading the same plugin seamlessly, but that’s not a goal for this project, so I don’t plan on using it. And the dlib crate provides macros to make the library loading simpler, based on libloading.

Creating an internal interface for Tremor through C is a lot of work. But it’s unfortunately the only way to do it, since Rust-to-Rust FFI is not stable. It has awful safety by relying heavily on unsafe with plenty of caveats^[10]^[11], including subtle differences between Operating Systems^[12] — although dlopen seems to be better in that regard^[13]. Since it doesn’t have a sandbox, plugins can abort Tremor’s core execution when panicking/leaking memory/similars.

I recently discovered the abi_stable crate, which guarantees stability in Rust’s ABI unofficially and helps a lot with the binary compatibility aspect.

It includes FFI-safe alternatives to many of the types in the standard library, and even external ones (namely crossbeam, parking_lot and serde_json). This works by implementing a StableAbi trait that guarantees its FFI-safety, which may be done automatically with one of its procedural macros. Internal ABI stability is also guaranteed with macros like #[sabi(last_prefix_field)], which would allow Tremor to add fields to existing structs without breaking backwards compatibility.

Fortunately, this crate has a few very detailed examples one can look at to better understand how it works, and it’s exceptionally well documented. If I’ve understood it correctly, some of its inconvenients are the following:

You have to use the types from abi_stable instead of std for the values passed through FFI.
The whole crate seems huge and would probably add considerable complexity to this FFI method.
It’s worth mentioning that library unloading is a non-feature; although it’s not going to be implemented for this project, it might in the future.

While it’s a fascinating concept and look forward to seeing how it evolves, I’m not a big fan of having to resort to it. And the fact that it’s unofficial and not that popular doesn’t give me full confidence that this will still work in 5 years, or that it won’t be outdated/abandoned.

More people have tried writing Rust FFIs in the past, so we can take a look at existing tutorials in order to see their experience:

The one and only Amos wrote an extremely detailed blog post on fasterthanlime here, specifically about live reloading Rust — a closely related topic.
Michael Bryan made a guided introduction to Plugins in Rust here, and also wrote a tutorial for his unofficial Rust FFI book here.
@zicklag, who had read Michael’s article, tried it by himself in order to add a plugin system to Amethyst, and posted this tutorial. When he shared the post on the official Rust forum, it was accompanied by this demotivating comment, after failing^[14] to implement it for Amethyst:

Unfortunately I found that dynamic linking doesn’t actually work in Rust across different versions of Rust, and the technique for plugins also failed, even inside the same version of Rust, when I tried to compile an app with other dependencies like Amethyst. That leaves the technique outlined in the tutorial not very practical for real applications.

The closest thing I’ve found to accomplish something similar is [abi_stable].

He also added later on:

It could very well be possible [to use WebAssembly here]. It wouldn’t be exactly the same workflow, but I’ve considered using Wasmtime or CraneLift, which Wasmtime is built on, to Run Wasm modules as plugins.

He didn’t have time to end up doing so, so we’ll have to investigate ourselves.

So, more or less:

3/5 Cross-platform support
5/5 Low overhead
5/5 Rust availability
1/5 Safety
0/5 Backwards compatibility (maybe 5/5 if using abi_stable)
5/5 Ease of porting existing implementations

WebAssembly

Now, this is what I wanted to emphasize in this article! Turns out WebAssembly isn’t limited to web development anymore; it’s slowly evolving into a portable binary-code format. As far as I know, this should be like a mix between Dynamic Loading and Scripting Languages, with a stronger focus in stability and portability. Here’s what Wikipedia has to say about it:

WebAssembly (sometimes abbreviated Wasm) is an open standard that defines a portable binary-code format for executable programs, and a corresponding textual assembly language, as well as interfaces for facilitating interactions between such programs and their host environment. The main goal of WebAssembly is to enable high-performance applications on web pages, but the format is designed to be executed and integrated in other environments as well, including standalone ones.

So to clear it up, Wasm is an assembly language, and WASI is a system interface to run it outside the web. The latter is extremely well explained in this article by Mozilla, I suggest giving it a read for more details. This one is also very nice to read and explains the isolation system it provides, specifically.

The two main points WebAssembly offers are, in a nutshell:

When compiled, it doesn’t need to know what Operating System is being targeted. This is handled by the runtime, and the binary itself is fully portable.
In order to handle untrustworthy programs, it implements a sandbox. With that, the host can limit exactly what a program has access to.

WASI is just a standard, so it has multiple runtimes available. The most popular ones are coincidentally written in Rust: wasmtime and wasmer. Both use the Cranelift backend to generate the WebAssembly machine code, though wasmer seems to also support LLVM and others. Then, the runtime can be used to run the generated .wasm binary as a CLI or a library. This also means that plugins could be written in any language that compiles to WebAssembly, though it is frequently harder than compiling to native code. Many Rust libraries don’t support WebAssembly, so it could be tricky depending on the situation.

The differences between the two runtimes aren’t that big. You can read this wiki article for more details, including examples, but I particularly liked this quote:

Just based on what they demonstrate, wasmer is more focused on embedding wasm in your native program, while wasmtime is more focused on executing standalone wasm programs using WASI. Both are capable of both, it just seems a matter of emphasis.

The article also includes a ~~not reliable at all~~ benchmark, which can serve us as a way to compare its performance with the native code you’d get with e.g., Dynamic Loading. It estimates that Wasm is a bit less than an order of magnitude slower than native code, and the same applies to memory usage. A more thorough benchmark was done in libsodium that shows better results: Wasm can be just about 3 times slower than native code. Do note that this depends on the runtime that’s being used, and it may improve in the future, as WebAssembly is just 4 years old.

There’s a whole series on how to make a Plugins System with Wasmer here, which will come in handy to know what to expect. The usability doesn’t actually seem to be that good, since by default you can only use integers, floating-point or vectors^[15] as parameters when calling Wasm plugins. For more complex types, you have to resort to encoding and decoding via a crate like bincode, although most of the boilerplate can be reduced with procedural macros or a wrapper like wasm_plugin, and this opens up the possibility of using serialization with support for backwards compatibility within Tremor. The last part of the series is the most interesting one, as it includes a real-world example, with a version of the final code in this repository.

All in all, WebAssembly seems to win against Dynamic Loading in terms of security by not needing unsafe at all and including sandboxing by default, at the cost of efficiency. This is up to the managers of the project and what they consider more important.

5/5 Cross-platform support
3/5 Low overhead
5/5 Rust availability
5/5 Safety
4/5 Backwards compatibility
3/5 Ease of porting existing implementations

Prior Art

It’s very important to take a look at projects that have already done this in the past in order to learn from their mistakes and not start from scratch.

Here’s a list of some of the libraries I found with Plugin Systems, specifically written in Rust:

cargo, mdbook: both have an extension system via CLI commands. Adding a subcommand to either of these utilities is as easy as creating a binary with a fixed prefix (e.g., cargo-expand), and if it’s available in the $PATH when running cargo, it will be possible to invoke the plugin with cargo expand as well.

This is a very interesting approach, specially because of how simple it is to use. Cargo doesn’t seem to need to communicate with the extension at all, but mdbook does use stdin to receive messages and stdout to send them, via serialization. So it’s basically the IPC Based on Pipes approach.
zellij: a terminal workspace with “a plugin system allowing one to create plugins in any language that compiles to WebAssembly”.

This is an extremely valuable resource in case the WebAssembly option is chosen, as it’s very similar to what Tremor needs. One can even subscribe to events in order to simulate the traits in Tremor that currently use async.

After trying it out and seeing its source code, it seems to work with a wasm binary that acts as a standalone program, where the communication takes place via stdin and stdout, serializing and deserializing with serde. zellij includes a few plugins by default, like the status bar, or a file manager. Very neat architecture!

Other WebAssmbly-based plugin systems: Veloren, Feather.
xi: a now abandoned modern text editor. Its plugins, described here in detail, are based on JSON RPC.

Text editors overall are very interesting, because they must be built with extensibility in mind and thus have to implement some kind of plugin system.
- Visual Studio Code is based on Electron, so it has a renderer process for the frontend, and a main process with Node. Its extensions share the same event loop, which has been an issue for a long time in case an extension blocks temporarily, causing lag.
- Eclipse’s plugin architecture is based on Java classes^[16] loaded at runtime and configured with XML manifests.
bevy: a very promising game engine whose features are implemented as plugins. Most times they are loaded at compile-time, but the bevy::dynamic_plugin allows this to happen at runtime. It uses libloading internally, with actually very little code.

Conclusion

This article has covered quite a few ways to approach a Plugin System. The final choice depends on what trade-offs the project wants to make. Most of them require sacrificing some level of performance for safety or usability. Here’s a drawing that very roughly compares the main methods:

There’s never a single answer in programming: how much performance are you willing to lose in exchange for safety and usability? Is that performance actually measurable, or is it just hypothetical? Don’t forget that this depends on the use case, so make sure you run a couple benchmarks if the resulting overhead may be important for your program.

You can join the discussion at Reddit if you have any additional suggestions or comments, or leave a comment below.

Footnotes

Optional parameters in Rust

2020-10-10T00:00:00Z

Table of Contents

Optional or default parameters are a very interesting feature of some languages that Rust specifically doesn’t cover (and looks like it won’t anytime soon). Say your library has lots of endpoints like so:

fn endpoint<T1, T2, T3, ...>(mandatory: T1, opt1: Option<T2>, opt2: Option<T3>, ...);

In this case, when you call endpoint, you have to use endpoint(mandatory, None, None, ...), or endpoint(mandatory, Some(val1), Some(val2), ...), instead of the more intuitive endpoint(mandatory) or endpoint(mandatory, val1, val2). Other languages like Python have named arguments, which make optional parameters natural and easier to read: endpoint(mandatory, opt1=val1, opt2=val2), while also allowing them to be written in any order.

Even without official support, there are lots of different ways to approach them in Rust, which is what this blog post tries to analyze. My goal is not to show which one is the “best” option, but to exhaustively showcase the different ways they can be approached, and the ups and downs of each of them.

Introducing an example

Let’s start with a typical web API wrapper library. These often require a client struct to hold authentication fields and such:

#[derive(Default, Debug, Clone)]
struct APIClient;

This client will have various endpoints which can be used to send requests to the server. We will use a function with the actual implementation and some type aliases so that the following snippets are easier to read:

use std::error::Error;

/// Some value that the endpoint will return
#[derive(Debug, Default)]
struct Value;
type ReturnedValue = Result<Value, Box<dyn Error>>;

/// The actual code for the endpoint inside the client
impl APIClient {
    fn actual_endpoint(name: &str, opt1: Option<u32>, opt2: Option<i32>) -> ReturnedValue {
        println!("params: {} {:?} {:?}", name, opt1, opt2);

        Ok(Default::default())
    }
}

A) Using `Option<T>`

The simplest way to do this would be to just use the actual_endpoint function signature. Sometimes the best solution is the simplest, and your project just might not need more complex approaches. Do consider if anything more elaborate is actually necessary.

For the sake of this example, it would look like this:

impl APIClient {
    pub fn approach_a(&self, name: &str, opt1: Option<u32>, opt2: Option<i32>) -> ReturnedValue {
        self.actual_endpoint(name, opt1, opt2)
    }
}

let api = APIClient {};
let param2: u32 = 324;

api.approach_a("option", Some(param2), Some(1234))?;
api.approach_a("option", None, None)?;

Upsides

Simplest to understand and implement.

Downsides

Multiple optional parameters require lots of None and Some.
Parameter names unknown when reading the code, which is specially annoying with None values, since these don’t have context to know what they are for.

B) With `Into<Option<T>>`

A variation of the previous approach consists on using Into<Option<T>> as the generic value for the optional parameters (impl Into<Option<T>> can be used as well). This way, Some isn’t needed when the optional parameters are specified:

impl APIClient {
    pub fn approach_b<T1, T2>(&self, name: &str, opt1: T1, opt2: T2) -> ReturnedValue
    where
        T1: Into<Option<u32>>,
        T2: Into<Option<i32>>,
    {
        self.actual_endpoint(name, opt1.into(), opt2.into())
    }
}

api.approach_b("into_option", param2, 1234)?;
// This still works
api.approach_b("into_option", Some(param2), Some(123))?;
api.approach_b("into_option", None, None)?;

Upsides

Some isn’t required, which makes it slightly easier to read.

Downsides

Multiple optional parameters still require lots of Nones.
No parameter names either.
More complex function signatures, and might not be too “idiomatic”.
Requires generics, 2^N copies of this function may be generated, where N is the number of optional parameters.

C) With a custom struct

Another option is to create a struct that holds the parameters and use that instead. The complexity is still relatively simple, and it can work out well if the API has functions with repetitive function signatures. This will serve as a base for the following approaches as well:

let call1 = params::ApproachC {
    name: "builder".to_string(),
    opt1: Some(param2),
    opt2: Some(123),
};
api.approach_c(&call1)?;

We can even use ..Default::default() to initialize the rest of the parameters with their default values:

let call2 = params::ApproachC {
    name: "builder".to_string(),
    ..Default::default()
};
api.approach_c(&call2)?;

The implementation looks like this:

mod params {
    /// We derive `Default` to be able to initialize with
    /// `..Default::default()`.
    #[derive(Default)]
    pub struct ApproachC {
        pub name: String,
        pub opt1: Option<u32>,
        pub opt2: Option<i32>,
    }
}

impl APIClient {
    pub fn approach_c(&self, data: &params::ApproachC) -> ReturnedValue {
        self.actual_endpoint(&data.name, data.opt1, data.opt2)
    }
}

Upsides

Some APIs might feel more natural this way, if the combination of these parameters as a group makes sense.
The struct can be reused in different calls.

Downsides

Multiple optional parameters require lots of None and Some.
Way more verbose.
Can be difficult to scale, since it needs a struct and a function per endpoint.

D) With the builder pattern

The previous approach can be improved by using the builder pattern for the parameters, so that building the parameters is simpler and more pretty to look at:

let call1 = params::ApproachDBuilder::default()
    .name("builder")
    .opt1(param2)
    .opt2(2134)
    .build()?;
api.approach_d(&call1)?;

let call2 = params::ApproachDBuilder::default()
    .name("builder")
    .build()?;
api.approach_d(&call2)?;

In this case, we use the derive_builder crate to make the implementation less repetitive:

mod params {
    #[derive(Default, Builder)]
    pub struct ApproachD {
        #[builder(setter(into))]
        pub name: String,
        #[builder(setter(strip_option), default)]
        pub opt1: Option<u32>,
        #[builder(setter(strip_option), default)]
        pub opt2: Option<i32>,
    }
}

impl APIClient {
    pub fn approach_d(&self, data: &params::ApproachD) -> ReturnedValue {
        self.actual_endpoint(&data.name, data.opt1, data.opt2)
    }
}

Upsides

It’s the most popular pattern in Rust for optional parameters.
The struct can still be reused in different calls.
No None or Some at sight.
Optional parameters are now associated to their name, which makes it easier to read.

Downsides

Somewhat verbose.
In our example it can be difficult to scale, since it needs a struct and a function per endpoint.
More overhead, both at runtime and compile-time (specially if macros are used).
Constructing the values may fail. Thus, the documentation has to specify very clearly which parameters are mandatory and which aren’t.

E) Endpoint-oriented interface

Here’s a different take: what if the API was endpoint-oriented instead of client-oriented? Starting from the previous approach, we could make the call inside the endpoint struct itself instead of in the API’s client by adding it to ApproachE, or by overriding the build method from derive_builder for a slightly less verbose version:

ApproachEBuilder::default()
    .name("endpoint-oriented")
    .opt1(param2)
    .opt2(1111)
    .call(&api)?;
ApproachEBuilder::default()
    .name("endpoint-oriented")
    .call(&api)?;

And its implementation:

#[derive(Default, Builder)]
#[builder(build_fn(private))]
struct ApproachE {
    #[builder(setter(into))]
    pub name: String,
    #[builder(setter(strip_option), default)]
    pub opt1: Option<u32>,
    #[builder(setter(strip_option), default)]
    pub opt2: Option<i32>,
}

impl ApproachEBuilder {
    pub fn call(&self, client: &APIClient) -> ReturnedValue {
        let data = self.build()?; // This might fail!
        // `actual_endpoint` would have to be at least pub(in crate) in this
        // case.
        client.actual_endpoint(&data.name, data.opt1, data.opt2)
    }
}

Note: this is assuming the client contains necessary information to make the requests, like a reqwest::Client or authentication details. But for example, ureq can perform calls without a client instance, just by calling ureq::get and similars. In that case, the API could just not have a client at all, and the call method wouldn’t require a reference to the client.

Upsides

Fits perfectly for some specific APIs.
No None or Some needed.
Just as readable as the previous case.
The struct may be reused.
Simple to use and implement, since it doesn’t need declaring both a function in the client and a parameters struct, only the latter.

Downsides

Still relatively verbose, might not be compatible with some APIs.
Slight builder pattern overhead, and can also fail to construct the value.

F) Hybrid derive pattern

Back to the client-oriented API. We can remove some disadvantages of the builder pattern by using a different approach and possibly a custom implementation instead of just using derive_builder.

The client will now have a method that calls ApproachBuilder::default() and whatever is necessary to start building the endpoint. Mandatory parameters are added to that function’s signature, so that the build can never fail. This also avoids passing the client in call(&api), since we can do that inside the method. This is how it would look like:

api.approach_f("hybrid-derive-pattern")
    .opt1(param2)
    .opt2(2222)
    .call()?;

And an implementation by wrapping derive_builder:

impl APIClient {
    pub fn approach_f(&self, name: &str) -> ApproachFBuilder {
        ApproachFBuilder::default().client(self).name(name)
    }
}

/// Not meant to be used directly, only within `APIClient`.
#[derive(Default, Builder)]
#[builder(build_fn(private), pattern = "owned")]
pub struct ApproachF<'a> {
    #[builder(setter(strip_option), default)]
    client: Option<&'a APIClient>,
    #[builder(setter(into))]
    pub name: String,
    #[builder(setter(strip_option), default)]
    pub opt1: Option<u32>,
    #[builder(setter(strip_option), default)]
    pub opt2: Option<i32>,
}

impl ApproachFBuilder<'_> {
    pub fn call(self) -> ReturnedValue {
        let data = self.build().unwrap(); // This should never fail
        data.client
            .unwrap()
            .actual_endpoint(&data.name, data.opt1, data.opt2)
    }
}

derive_builder doesn’t know that build() can never technically fail because the mandatory parameters are provided inside the wrapper, so it’s kind of a waste to use this macro. It also forces us to create the wrapper because the builder always has to be initialized with Builder::default(). With a custom implementation, we could have this new method integrated in the builder itself instead, like Builder::new(A, B, C).

Thus, it might be better to use other crates like typed-builder, or just a custom implementation.

Upsides

Although mandatory parameters don’t have a parameter name now, it’s still quite readable. It’s also much less verbose.
No None or Some needed.
Can’t fail to initialize the endpoint value, so it’s “safer”.
A custom implementation would avoid runtime overhead.

Downsides

Currently a bit hacky to implement, which makes it much more complex, specially because there’s no existing macro that can simplify this specific variation of the builder pattern (that I know of). It requires both a function and a struct as well, so the implementation can be quite lengthy.
Still some compilation-time overhead.

G,H) Grouping up endpoints

Another possible approach based on the previous approach of the builder pattern consists on grouping up all or some of the endpoints under a single struct. They will share the optional parameters, which is useful to avoid declaring a struct with optional parameters for each endpoint we have, and makes a lot of sense for some APIs.

The call method could be removed in place of approach_f, and the optional parameters would go first. This would require a different order for the optional parameters. Here’s an example if all the endpoints shared the same optional parameters:

api.opt1(param2)
    .opt2(2222)
    .approach_f("group-builder-pattern")?;

And if we used different groups it would look like this:

api.group()
    .opt1(param2)
    .opt2(2222)
    .approach_g("group-builder-pattern")?;
api.group().opt2(2222).approach_h("builder-from-scratch")?;

Here’s an implementation with derive_builder:

impl APIClient
    pub fn group(&self) -> GroupBuilder {
        GroupBuilder::default().client(self)
    }
}

#[derive(Default, Builder)]
#[builder(build_fn(private), pattern = "owned")]
pub struct Group<'a> {
    #[builder(setter(strip_option), default)]
    client: Option<&'a APIClient>,
    #[builder(setter(strip_option), default)]
    pub opt1: Option<u32>,
    #[builder(setter(strip_option), default)]
    pub opt2: Option<i32>,
}

impl GroupBuilder<'_> {
    pub fn approach_g(self, name: &str) -> ReturnedValue {
        let data = self.build().unwrap();
        data.client
            .unwrap()
            .actual_endpoint(name, data.opt1, data.opt2)
    }

    pub fn approach_h(self, name: &str) -> ReturnedValue {
        let data = self.build().unwrap();

        // This endpoint doesn't need `opt1`. It can either be ignored, or
        // be an error. I hate silent errors, though.
        if data.opt1.is_some() {
            panic!("opt1 isn't needed for enpoint_h")
        }

        data.client.unwrap().actual_endpoint(name, None, data.opt2)
    }
}

As the approach_h endpoint indicates, the shared optional parameters don’t actually have to be strictly the same. As long as they are related, they can share the same group, and some extra verifications can be added in the final call to make sure the user is using it properly.

Upsides

Basically the same as the hybrid builder pattern, but with an easier implementation, and it might fit perfectly for some APIs that have clearly established groups of endpoints.

Downsides

It’s a bit odd, specially because the order is inverse to what you’d expect. And as it’s based on the hybrid builder pattern, it may still be hacky to implement and require compilation-time overhead.

I) Macros

Rust macros support variadic arguments, which make it possible to create a macro with named parameters, like foo!(1, c = 30, b = -2.0). Ideally, we want a macro that generates the macros for us, which does sound crazy. I wanted to at least try how existing crates approached this, and only found named and duang, which haven’t been updated in years, and probably for good. I tried duang with Rust 1.47 but got some unexpected errors, so we can assume there are no crates that support this yet. It definitely sounds like a fun challenge, if it’s still possible to implement.

Conclusion

Whew! That took more than I expected. Some of these endpoints might be unnecessarily complicated or straight up weird. But I hope this was as a good showcase of the different ways optional parameters can be approached in Rust, and that reading this served as a learning experience. I look forward to seeing new crates in the future that simplify these approaches.

The code for the different approaches can be found here. Bear in mind that there are a lot of different ways to implement the approaches, as I explained in this post. You can discuss it at the reddit thread.

Disclaimer: this post was originally in https://vidify.org/blog/rust-parameters/. I’ve moved it to my personal blog.

NullDeref

Releasing SpeedBump on the Play Store

Lessons on Mobile App Promotion I've Learned the Hard Way

Screen Time App Opal Jokes about Being Bought by Meta

How to Reduce Screen Time: The Ultimate Guide

The German language broke my website

My app sucked (but it's okay)

I blasted over a speed bump to create my landing page

New year, new silly app

Want to receive an email when I post?

I've quit my job to get you off your phone

Want to receive an email when I post?

Blindsided by Rust's Subtyping and Variance

The problem #

Debugging #

Some progress… or not? #

Discovering the root cause #

Trying to fix it #

Attempt #1: GATs #

Attempt #2: transmute #

Attempt #3: getting rid of BorrowOwned<'a> #

Conclusion #

Want to receive an email when I post?

Beyond Dumbphones: Building a Minimalist Yet Functional Phone

Looking at the state of things #

Technology #

Negative effects #

Companies #

Software #

Accessories #

Phones #

Wrapping up #

Want to receive an email when I post?

Footnotes

The bane of my existence: Supporting both async and sync code in Rust

Introduction #

The first approaches #

Good ol’ copy-pasting #

Calling block_on #

Duplicating the crate #

What ended up “working”: the maybe_async crate #

The problem #

The feature resolver v2 #

Other fails #

Fixing maybe_async #

Official Support #

Conclusion #

Want to receive an email when I post?

Footnotes

[Talk] Rust, the best and worst thing to happen to Tremor

Want to receive an email when I post?

Plugins in Rust: Wrapping Up

Benchmarking #

Tooling #

Methodology #

Experiments #

Type conversions #

Double Box #

Hash table optimization #

abi_stable #

Unimplemented ideas #

More detailed benchmarking #

Investigate wrapper overhead #

Investigate async runtime conflicts #

Benchmark async_ffi #

Improve error reporting #

Reduce error handling boilerplate #

Reduce async boilerplate #

Improve cross-platform support #

Performance impact of panic handling #

Use -Z randomize-layout to find FFI bugs #

Try raw dynamic loading #

Simplify the interface further #

For the long-term future #

Conclusion #

Appendix A: Open Source Contributions #

External Contributions #

Internal Contributions #

Appendix B: Other Achievements #

Breaking the Compiler #

The problem

Debugging

Some progress… or not?

Discovering the root cause

Trying to fix it

Attempt #1: GATs

Attempt #2: `transmute`

Attempt #3: getting rid of `BorrowOwned<'a>`

Conclusion

Looking at the state of things

Technology

Negative effects

Companies

Software

Accessories

Phones

Wrapping up

Introduction

The first approaches

Good ol’ copy-pasting

Calling `block_on`

Duplicating the crate

What ended up “working”: the `maybe_async` crate

The problem

The feature resolver v2

Other fails

Fixing `maybe_async`

Official Support

Conclusion

Benchmarking

Tooling

Methodology

Experiments

Type conversions

Double `Box`

Hash table optimization

`abi_stable`

Unimplemented ideas

More detailed benchmarking

Investigate wrapper overhead

Investigate async runtime conflicts

Benchmark `async_ffi`

Improve error reporting

Reduce error handling boilerplate

Reduce async boilerplate

Improve cross-platform support

Performance impact of panic handling

Use `-Z randomize-layout` to find FFI bugs

Try raw dynamic loading

Simplify the interface further

For the long-term future

Conclusion

Appendix A: Open Source Contributions

External Contributions

Internal Contributions

Appendix B: Other Achievements

Breaking the Compiler

LFX Mentorship Showcase

My Final Year Project

KubeCon + CloudNativeCon 2022

Some prototyping advice

The “just make it work” approach

Specific to Rust

Disable the annoying warnings

Don’t touch previous `use` statements

Defining the plugin interface

Recursively making everything `#[repr(C)]`

Overcoming problems with `#[repr(C)]`

Reaching `#[repr(C)]` blockers

Approach 1: Avoid the type in the first place

Approach 2: Implement a wrapper

Approach 3: Re-implement with `#[repr(C)]` from scratch

Approach 4: Simplifying the type at the FFI boundary

Asynchronous plugins

`async` functions

Channels

Option 1: Use `abi_stable`’s alternatives

Option 2: Callbacks

Option 3: Opaque types

Loading plugins