Fast Arduino Audio Control Processing with Numerical Operation Specialization

In my last post, I added the ability to read analog control knobs for the tremolo pedal that Steve implemented. However, the code in that post only read the raw values from the analog-digital converter (ADC), whereas Steve's original C++ code did some transformations on those values to normalize or rescale them. In this post, I add these features to our high-level audio programming language, Bell, and show how we compile Bell numerical value transformations into efficient C++ code.

Our target for Bell code compilation is the Ultraviolet virtual machine, which is implemented on Arduino in C++. It provides access to the Arduino Audio library. Ultraviolet is an unusual virtual machine design in that while most virtual machines (and the real processor architectures they are based on) have instruction sets that focus heavily on doing numerical computations, which in higher-level functional languages are often represented as pure functions. Ultraviolet, however, is designed for Effective Computing, where the focus is primarily on side effects. Therefore, it doesn't really need instructions for doing numerical computations in pure functional form, as these computations are not considered to be effects.

This may be confusing for those familiar with traditional virtual machine architectures, so I would like to offer an alternative way to look at Ultraviolet, which is that you can also think of it like an operating system for the Arduino platform. An operating system is somewhat similar to a collection of low-level libraries that you used to build applications. All of these libraries are bundled into a kernel, and your application interacts with the kernel code through an API where the functions in that API are referred to as "system calls" (syscalls). Check out this example of Linux syscalls, for instance. All interaction between your program and the world happens through syscalls, because on the kernel has the right code to interact with things like physical storage, displays, keyboards, etc. Your application is doing mostly computations, with the equivalent of effects happening in the kernel, driven by syscalls. This is the same with the Ultraviolet virtual machine, it provides all of the effects and your application handles the computations. In C, syscalls look like normal function calls, but they are compiled into a form than normal functions since the application code does not include an implementation for the function as it is handled by the kernel. The Bell language does something similar, with effects being compiled into message dispatches, but we have flexibility to decide what to do when we compile computations.

In order to implement Steve's tremolo effect controller with analog knobs, we need to do some normalization and rescaling, which are computations. Normalization is where you take the input range, which for the analog knobs is an integer from o t0 1023, and you convert it to a standard range, such as a floating point number from 0 to 1.0. The tremolo controller uses this for volume, for instance. Rescaling is similar, but instead of ending up with a number from 0 to 1.0 you end up with a number in a new range. The tremolo controller uses this to convert an analog knob value into the range of 85 to 1000, representing the number of ms of delay to use. This is an arbitrary range that Steve thinks sounds good. This is a key consideration in making knobs for pedals, make most of the range of the knob sound pretty good, maybe letting the extreme high and low sound a little weird for people that are into that. You pretty much always want to rescale or normalize knobs and not just use the raw value.

When implementing math operations (our first in Bell, surprisingly!), there is one major decision that comes up: boxed or unboxed number types. Unboxed types are what you have in the underlying implementation language, in our case C++ numbers such as int and float. Boxed types are an abstraction provided by the virtual machine and provide uniform semantics with the rest of the virtual machine operations. For instance, we implemented integer math as an Ultraviolet module that responds to messages, then that would be a boxed type. The advantage of boxed types is the uniform semantics. We already have a whole message dispatch system. If we create messages for integer math then we're done. The compiler won't care if we're doing integer math or configuring the audio code, it's all just messages. The downside is the conversion boxed and unboxed types. We can't ever get rid of the unboxed types since ultimately our implementation platform needs them to implement the underlying C++ operations that would happen when we received a message requesting a math operation. So we would need to first convert all of our unboxed numbers to boxed numbers, send a message, and then in the message receiver convert back from boxed to unboxed in order to actually do the operation, and then box up the return value. People often complain about the performance of boxed types, especially when doing operations on large arrays of boxed numbers. I am not that worried about performance. However, I decided to go with unboxed numbers for now because it allowed me to continue implementing the Bell compiler quickly rather than pausing to write an integer psuedo-effect for Ultraviolet. It also keeps Ultraviolet focused only on effects, which I think makes a strong philosophical point because it is so unexpected.

Implementing math with only unboxed types in the compiler requires specializing all math operations. This means that instead of compiling math operations into general-purpose message sends like everything else we've dealt with so far, the compiler emits specific C++ code for each and every math operation. Fortunately, there is not actually all that much math available in the base C++ (excluding libraries) and we only need a couple of operations currently for our analog knob implementation.

Here is an example of some new math syntax in Bell:

function main speed -> int : speedKnob rescale 0 1023 85 1000
function main volume -> float : volumeKnob normalize 0 1023

There are a couple of things to note here. First, I had to add return types to function, this is done with "->". This is so that I can generate C++ functions for each Bell function. Our previous examples of functions returned nothing, so the return type is optional. We could definitely make this syntax unnecessary by implementing type inference, which is not all that hard, but again I wanted to keep my momentum up on implementing the Bell compiler and not pause to write a type inference engine. Hopefully we will have time to circle back and add that in the future. Also, if you compare this Bell code to Steve's C++ tremolo code, it doesn't look the same. This is because I made normalize and rescale first class operations whereas Steve implemented them manually. The compiled code ends up pretty much the same, of course, because they have to achieve the end result, but I thought this added enough elegance to be worthwhile. As I said above, rescaling and normalization are very common operations for this use case.

Here is the C++ code that is generated:

int Main::speed()
{
  return int((float)(speedKnob - 0) / (float)(1023 - 0)) * (1000 - 85) + 85;
}

float Main::volume()
{
  return (float)(volumeKnob - 0) / (float)(1023 - 0);
}

As you can see, we don't use any message dispatch at all, not even any function calls. We unwrap what looks like message sends in Bell into specialized C++ code implementing each operation. This requires the compiler to do more work, but also makes the virtual machine less complicated.

There are a couple of other notes that I wanted to leave you with about implementing math operations and boxed/unboxed number types. The first is that while I think either boxed or unboxed is totally fine, the worst plan is what Java did: have both. Boxing and unboxing is something that the compiler should worry about and not the language that you are actually using to write applications. Having both just means that the application developer has to write all of the boxing/unboxing code instead of having it automatically generated by the compiler. Not cool. In Bell, numbers look just like boxed types, and the compiler converts them into unboxed types. Way cooler.

The second note that I wanted to leave you with is Steve's preferred solution to this problem, which is also what is done in Smalltalk-80 and various LISP implementations. It might sound similar to the thing I say not to do above, but it is entirely different: have one numeric type that supports both boxed and unboxed numbers. This gets technical quickly, and I'm not currently implenting it, so understanding this bit is optional at this point, but the essential trick is that a C++ pointer and a C++ integer (int) are generally the same size. So by typecasting in C++, you can have one value that might be an integer or might be a pointer. You can "tag" it with its type (int or pointer) by using 1 of the available bits as a flag for which it is. I believe this approach is generally called a "tagged union". Of course, this requires special handling while doing math. You need to take the flag off before you do math and then put the flag back when you're done. You also need the compiler to special case all math operations, just like I had to do with all unboxed numbers. However, with this approach sometimes your special case math code gets lucky and can just delegate to the general-purpose message dispath code. One of the really nice things about this approach is that you can support both small integers (normal ints) and big integers (BigInts). Since all of your math code needs to do some special case handling anyway, you can have it handle the conversion between small and big integers whenever it is necessary in the compiler and virtual machine, allowing your application programming language to treat small and big integers as interchangeable. Very cool! I'd love to get around to doing this, but there's lots more to do on more fundamental features, so this is another "nice to have" feature.

In summary, I have added the ability for Bell functions to now do math, specifically for the use case of transforming analog knob values into useful ranges for controlling audio effects. We can now control the example tremolo effect and set all of its parameters from values read off of control knobs. These functions that can do math are also the foundation for being able to implement the tremolo effect entirely in Bell. However, to implement an audio effect we need to go beyond integer math, or even floating point math. We need to be able to do operations on arrays of numbers. That's a whole different situation, for which I have some wild ideas that I'd like to try out, so that will be the focus of the next post.