How kabelsalat works

This site explains how kabelsalat works under the hood. You don’t need to know this to use it!

Overview

The understand how kabelsalat works, we need to understand how the user code is transformed to produce audio in the end. These are the basic steps:

User evaluates code
The code creates a Node representing the audio graph
The Node is compiled to an optimized chunk audio callback code
The code is sent to the audio thread
The audio thread creates the required AudioNode’s and runs the code for each audio sample

From Code to Graph

Let’s look at this example:

If you run the above patch, the log method will output this:

{
  "type": "Sine",
  "ins": [
    {
      "type": "n",
      "value": 200,
      "ins": []
    }
  ]
}

Here we see the basic structure of a Node in kabelsalat:

type: which kind of node is it?
ins: which Node’s are connected to it?

We can also see that the number passed to sine is converted to a Node of type n, which is the constant number node. It is special because it also has a value property.

The clever thing about this data structure is that you only need to know the last Node (before .out) to know the whole graph!

Method Chaining vs Function Calling

kabelsalat relies heavily on method chaining, which is only syntax sugar for regular function calls. These 2 variants are equivalent:

mul(sine(200), 0.5);
// is equal to
sine(200).mul(0.5);

This will work for any node (except out, which is a special case). More generally, you can say

a.x(b) = x(a,b)

This syntax sugar reduces the level of parenthesis nesting alot and is also more natural way to express arithmetic:

3 * 4; // infix notation
// =
n(3).mul(4); // method chaining
// =
mul(3, 4); // polish notation

So you could say the method chaining syntax acts as an infix operator replacement.

Compiling the Graph

In the next stage, we pass the graph to the compiler, which turns it into a sequence of instructions that runs well on the audio thread. Before actual compilation there is preprocessing step:

flatten the Nodes
apply topological sort

Flatten Node’s

This step takes the last Node and turns it into an Array of Node’s where, the ins are replaced with indices:

This is the console output:

[
  // 0
  {
    "type": "mul",
    "ins": ["1", "3"]
  },
  // 1
  {
    "type": "Sine",
    "ins": ["2"]
  },
  // 2
  {
    "type": "n",
    "ins": [],
    "value": 200
  },
  // 3
  {
    "type": "n",
    "ins": [],
    "value": 0.5
  }
]

This representation of the graph contains the same info, but structured in a way suitable for…

Topological Sort

Topological Sorting makes sure the Nodes are ordered so that nodes without dependencies are first.

This gives us:

["3", "2", "1", "0"]

Reordering the flattened nodes in this order:

[
  // 3
  {
    "type": "n",
    "ins": [],
    "value": 0.5
  }
  // 2
  {
    "type": "n",
    "ins": [],
    "value": 200
  },
  // 1
  {
    "type": "Sine",
    "ins": ["2"]
  },
  // 0
  {
    "type": "mul",
    "ins": ["1", "3"]
  },
]

We can now observe that the ins of each Node preceed the Node itself, which means it’s time to generate some code..

Compilation

The compiler can now generate a chunk of code, where each line roughly corresponds to the value of one Node:

The compiler gives us this compiled unit:

{
    "src": "const n1 = nodes[0].update(200, 0); /* Sine */\nconst n0 = n1 * 0.5; /* Sine * n */\nreturn [(0*0.3), (0*0.3)]",
    "ugens": ["Sine"]
}

Let’s make the src more readable:

const n1 = nodes[0].update(200, 0); /* Sine */
const n0 = n1 * 0.5; /* Sine * n */
return [n0 * 0.3, n0 * 0.3];

This is now the actual code that runs on the audio thread.

The n variable numbers correspond to the original indices of the topologically sorted Array.
To shorten the output, the compiler inlines arithmetic Node’s (add, mul,…) and constant numbers (n).
For AudioNode’s like Sine, the compiler inserts an update call with its inputs as arguments (they could also be variables).
In the end we just get 2 numbers for the left and right speaker.

Running the Compiled Code

The unit returned by the compiler is sent to our GraphWorklet which hosts an instance of AudioGraph. The AudioGraph creates an AudioNode for each type in the ugens Array. Each AudioNode has the ability to keep its own state, for example, Sine will keep track of its phase. The compiled src is used as the body of the _genSample function, which is ultimately called in GraphWorklet.process for each sample.

Credits

This is the basic mechanism! It is heavily influenced by the amazing noisecraft by Maxime Chevalier-Boisvert. The audio DSP code is mostly borrowed as is, but I’ve written a new compiler + implemented feedback and multichannel expansion + removed UI related features. You can read more about the specific changes in this discussion

Other influences include genish.js and SuperCollider. Also thanks alot to Bubo for his invaluable feedback!

What I haven’t described so far:

how feedback is resolved
how multichannel expansion works
how the viz works

I might write about it in the future