Macros Are Complex

60 Days Until I Can Walk

Macros are one of the features of Rust that I completely misjudged when initially getting to grips with the language. In the book they are a single section of a chapter on “Advanced Features” towards the end. So I glossed over this section and started writing my ray caster. Big mistake.

Macros in Rust are awesome! But they are not trivial to learn in the same way that C macros are. I was hoping to write a proper article on Rust macros this weekend, but after working through the material over yesterday and today, I’ve decided that the best thing to do would actually be to give myself one more week to really study them and then write the article next weekend. There are a number of very direct applications of macros in the Fourteen Screws code-base that will make excellent case studies—the floating point arithmetic and trigonometry lookup tables to name just two.

But what have I learned so far.

Macros In C

First a quick look at macros in C, which is where my understanding of macros comes from. In this language, macros are little more than string substitution. If, for example, I define a MIN macro to return the min of two values and use it as shown:

#define MIN(a, b) a < b ? a : b

int main() {
	return MIN(0, 1);
}

Then the C compiler will literally replace the call to MIN in main with the code given in the macro definition, yielding:

#define MIN(a, b) a < b ? a : b

int main() {
	return 0 < 1 ? 0 : 1;
}

However, this can cause problems. Consider the following:

#include <stdio.h>

#define MIN(a, b) a <= b ? a : b

int main() {
	int a = 3;, b = 4;
	fprintf(stdio, "%d\n", MIN(++a, b));
	return 0;
}

In the expansion of the macro above, we get the following code:

#include <stdio.h>

#define MIN(a, b) a <= b ? a : b

int main() {
	int a = 3;, b = 4;
	fprintf(stdio, "%d\n", ++a < b ? ++a : b);
	return 0;
}

If you run this, you may be surprised to find that the value returned by MIN is 5, not 4. This is because the increment operation is triggered twice in the macro expansion.

There are plenty of other ways that this simple string substitution can bite us. Consider another example below using a MULTIPLY macro:

#include <stdio.h>

#define MULTIPLY(a, b) a * b

int main() {
	fprintf(stdio, "%d\n", MULTIPLY(3 + 2, 10));
	return 0;
}

You may expect this to print the number 50. In actuality, the result will be 23. Why is this? Let’s expand the macro:

#include <stdio.h>

#define MULTIPLY(a, b) a * b

int main() {
	fprintf(stdio, "%d\n", 3 + 2 * 10);
	return 0;
}

Because of order of operations, multiplication is performed before addition. In the expanded macro, the string 3 + 2 is simply substituted for the token a in the macro. Hence 2 * 10 is resolved first, then 3 is added to the result.

The solution to this problem in C is to wrap all arguments of a macro in parenthesis as shown below:

#include <stdio.h>

#define MULTIPLY(a, b) ((a) * (b))

int main() {
	fprintf(stdio, "%d\n", MULTIPLY(3 + 2, 10));
	return 0;
}

Now our code substitution will look like the following, and we will get the result we expect.

#include <stdio.h>

#define MULTIPLY(a, b) ((a) * (b))

int main() {
	fprintf(stdio, "%d\n", ((3 + 2) * (10)));
	return 0;
}

But it should be clear where this is error prone and potentially introduces bugs. Not to mention the fact that if an intermediate variable is declared within a macro, then it may pollute the scope of the function in which it is called.

So macros in C can be risky business, and generally the recommendation is to avoid overusing them. They do have their place, but don’t go nuts.

Macros In Rust

Macros in Rust are a whole different story. First, Rust defines two different classes of macro, with the latter being further broken down into three distinct types. These are:

• Declarative Macros
• Procedural Macros:
  • Function-like
  • Derive
  • Attribute

The idea here is that the whole Rust language is available to you, even at compile time! Using macros you can write Rust code which will be executed by the compiler and can modify your code base on-the-fly. A brief overflow of these types of macros follows.

Declarative Macros

Of the types of macros available, declarative macros seem to be the simplest and most similar to C macros. These macros define a section of code which is more-or-less substituted into your code-base at compile time. A common example used to illustrate this is the vec! macro, which will be familiar to anyone who has spent some time writing Rust. An implementation of this simple macro could look something like the following:

macro_rules! vec {
	( $( $x:expr ),* ) => {
		{
			let mut temp_vec = Vec::new();
			$(
				temp_vec.push($x);
			)*
			temp_vec
		}
	};
}

Let’s break this down. First, the macro_rules! keyword indicates that we are about to define a macro, here given the name vec. The body of the macro itself is actually rather like a match statement. A pattern is given for the macro inputs and a corresponding arm dictating what code should be inserted for specific matches. In the above example $x:expr indicates that we are expecting an expression (variable, constant, etc.) as input. This is further wrapped in a $( ) expression, followed by ,*. This means that the input will be a comma separated list of expressions. The * indicates that we may have zero or more comma separated values after the first argument.

If our inputs match this arm, then the macro will expand to the code shown. A new vector temp_vec will be declared. The input values are then iterated over and individually pushed into the vector. After the last input is handled, the newly created vector is returned.

Unlike in C, this macro is properly scoped, and does not suffer from the issues highlighted in the simple string substitution examples shown above. Generating code in this way may be an effective way to tidy up our code base without introducing the runtime overhead of calling a function.

Procedural Macros

Procedural macros are considerably more complex. They must be declared in their own crate with the type

[lib]
proc-macro = true

However, these macros are incredibly powerful. Unlike declarative macros, which are basically just expanded inside your code, procedural macros are given access to the token stream parsed by the compiler while processing your code. You can manipulate this token stream and tell the compiler how to reinterpret your code.

Basically, you can rewrite your code on-the-fly as it is being compiled! How cool is that!?!

Function-like procedural macros are exactly what they sound like. They are macros which look like functions. The example given in The Book is the following code which parses an SQL statement and returns an SQL query object:

let sql = sql!(SELECT * FROM posts WHERE id=1);

#[proc_macro]
pub fn sql(input: TokenStream) -> TokenStream {

Note how the SQL macro receives it input as a token stream. The SQL statement is not a string, it is code and within the sql macro definition we can write our own code to interpret this code. This is an incredibly powerful feature of Rust.

Derive macros (for example Debug or PartialEq), allow you to generate new code which may be attached to a struct or enum definition. For example, when we add the #[derive(Debug)] definition to a struct, what we are telling the Rust compiler to do is attach function definitions to our struct which will allow them to be printed by the debug output in println.

Finally, attribute-like macros are similar to derive macros except that they may also be attached to functions.

Fully understanding this type of macro is complicated enough that I do not feel confident writing an article about them based on the reading I have done this weekend. Rather I am going to point to a series of incredibly helpful resources which I will be using over the next week to try and really get to grips with this too.

First and foremost is David Tolnoy’s Procedural Macros Workshop, a GitHub repository from Rust Latam conference 2019,k which provides a series of exercises for learning about procedural macros. I will be working on this while watching a series of videos by Jon Gjengset in which he works to solve the problems proposed by David.

Additionally, I will be using The Little Book of Rust Macros, originally by Daniel Keep, but since adopted and expanded upon by a group of volunteers.

Conclusion

So this week was a bit slow, work-wise, and as a result I haven’t managed to get the full article written that I wanted this weekend. This is a pity, but I’m going to cut myself a bit of slack and use this lull as motivation to work a little harder next week. My plan is to work on macros and ultimately integrate them into the Fourteen Screws engine as part of The Great Refactor. Hopefully the refactor itself will be complete by the end of the week.