Rust

Subtyping and variance is a concept that works in the background, making your life easier without you knowing about it. That is, until it starts making your life harder instead. It’s a good idea to know about it, in case you end up being a fool like me. So let’s take a look at what went wrong, and how it was resolved. 1. The problem As part of my Plugin System in Rust series, I was making one of Tremor's types FFI-compatible. Put simply, instead of using types from the standard library like String, we wanted custom types defined with #[repr(C)]. The crate abi_stable exists for this exact purpose, with an equivalent for the most important types. Theoretically, the task should be as easy as changing the std types in our core enum with theirs: ...

My journey trying to support both asynchronous and blocking code in the Rspotify library. I got it working in the end, but Cargo didn’t quite like it.

Hello! I’ve recently had the pleasure of giving a talk at this year’s TremorCon. I really enjoy public speaking, so this has been an excellent opportunity to get better at it (let me know your thoughts!). In case you missed it, you can check out the recording here: You can also find the slides here in pdf or in pptx. ...

Welcome to the final article of this series! Here I’ll showcase some clean-ups and optimizations I may or may not have performed yet, so that our plugin system can get closer to production. I will also run benchmarks for some of these to ensure that the changes are actually worth it. 1. Benchmarking 1.1. Tooling There are all kinds of ways to measure the performance of a system, and it’s important to take as many of them as possible into account. Here are the ones I used myself: ...

Welcome to one of the last articles of this series! Previously, we covered how to use external dependencies to lessen the work necessary to implement our plugin system. Now that we know how to actually get started, we’ll implement it once and for all. I will personally use the crate abi_stable, but the concepts should be roughly the same for any dynamic loading method. Similarly, some of my advice is related to modifying an already existing project of large size. If you’re starting from scratch with a plugin system in mind, it should be the same, just easier in some aspects. ...

Previously in this series, I covered how the plugin system could be implemented from scratch. This is a lot of work if you’re dealing with a relatively large codebase and therefore a complex interface in your plugin system, so let’s see how we can make our lives easier. I’ve been wanting to try abi_stable for this since the beginning, which was specifically created for plugins. But we aren’t really locked to that crate, so I’ll show other alternatives as well, which can even be combined to your liking. ...

This article in the Rspotify series describes my journey of basically rewriting the entirety of this Rust library; around 13 months of work (in my free time), starting at September 2020, up until October 2021. I think this has given me enough experience for an article regarding API design on Rust, and perhaps those who attempt the same in the future can take some ideas and save time. Specially considering how much I’ve gone in circles and how much time I’ve “wasted” implementing things that ended up being discarded. ...

In the last article of this series I wrote some simple experiments of plugins both with WebAssembly and dynamic loading. After discarding Wasm for this specific plugin system, I wanted to try to get a more realistic example with dynamic loading and work towards the final implementation. Since the Tremor team suggested that I began by implementing connectors, we’ll first have to learn more about them. Matthias ran me through their current state in a meeting, which I’ll try to summarize. We won’t need to know much for this specific article, though. ...

Welcome to the second article of my “Plugins in Rust” series! Here I will try to actually write some simple code of what a plugin system might look like, and try to answer any questions that may arise while doing so. Even though the project is specific to Tremor, I try to keep it relatively generic so that it’s useful to future rustaceans interested in plugin systems. So if you don’t really care about Tremor’s specific case, you can skip to the next section. ...

Rust makes it very easy to express conditional compilation, especially thanks to Cargo “features”. They’re well integrated into the language and are very easy to use. But one thing I’ve learned by maintaining RSpotify (a library for the Spotify API) is that one shouldn’t obsess over them. Conditional compilation should be used when it’s the only way to solve the problem, for a number of reasons I’ll explain in this article. This might be obvious, but to me, it wasn’t so clear back when I started using Rust. Even if you’re already aware, it might be a good reminder — maybe you forgot about it in your latest project, and you added an unnecessary feature. ...

Welcome to the “Plugins in Rust” series! During the next months I’ll be involved in a project with Tremor, for which I need to implement a Plugin System. The goal is to maximize modularity and reduce Rust’s wild compilation times. The implementation will end up being done with dynamic loading, but I will cover all the alternatives first to make sure it’s the best approach for my use-case. In this first article I will analyze the different ways in which our plugin system may be implemented in Rust. Note that I’m no expert in the topic; my objective is to merely collect all the information I’ve found so far and present it as a summary, with enough resources for the reader to learn more on their own and make their own decisions. ...

Optional or default parameters are a very interesting feature of some languages that Rust specifically doesn’t cover (and looks like it won’t anytime soon). Say your library has lots of endpoints like so: fn endpoint<T1, T2, T3, ...>(mandatory: T1, opt1: Option<T2>, opt2: Option<T3>, ...); In this case, when you call endpoint, you have to use endpoint(mandatory, None, None, ...), or endpoint(mandatory, Some(val1), Some(val2), ...), instead of the more intuitive endpoint(mandatory) or endpoint(mandatory, val1, val2). Other languages like Python have named arguments, which make optional parameters natural and easier to read: endpoint(mandatory, opt1=val1, opt2=val2), while also allowing them to be written in any order. ...