Little Pi's Tech Music Tips

Many AIs that talk or sing don't make sound directly: they first draw a "photo of sound," the mel spectrogram, and a neural vocoder turns it back into audio. A spectrogram puts time on the horizontal axis and pitch on...

Type "lazy jazz piano on a rainy day" into an AI music site and you get music, but how does AI know which sound goes with which sentence? Meet CLAP (Contrastive Language-Audio Pretraining), inspired by OpenAI's 2021...

At the movies a helicopter seems to fly right over your head, and now streaming services and headphones advertise Dolby Atmos and spatial audio. How is that different from old surround sound? From two-channel stereo to...

On the subway or a plane, you press the noise cancelling button and the rumble suddenly drops: your headphones are using sound to destroy sound. Little Pi first separates two kinds of quiet: passive isolation, where ear...

X-Codec only got one sentence in episode 36, so this episode Little Pi takes it apart. Codecs like EnCodec and DAC were built for compression: they care whether the rebuilt sound sounds the same, not what it means, like...

Episode 47 corrected that Stable Audio Open Small gets its speed mainly not from distillation but from adversarial post-training (ARC): no teacher model, just a judge whose job is finding faults. The adversarial idea...

Distillation is a hot word in the language model world, and music generation AI uses it too, but the same word means two different jobs. It comes from a 2015 paper by Geoffrey Hinton and colleagues: a big teacher model...

Episode 38 said diffusion models wipe noise away step by step until music appears. But how many steps, and how much noise per step? Little Pi explains the noise schedule and sampling steps. The noise schedule is the...

Episode 38 said Stable Audio Open does diffusion inside a compressed space. Where is that space? Little Pi introduces the latent space and the autoencoder: an encoder squeezes sound down, a decoder rebuilds it, and the...

When AI writes bar nine, how does it know to look back at bar one? Little Pi unpacks the attention mechanism, mentioned briefly in episode 37. Older models worked like a game of telephone, passing information along step...

Why does AI use several codebooks to turn sound into tokens? Little Pi unpacks RVQ, Residual Vector Quantization, mentioned briefly in episode 36. The encoder cuts sound into short slices, turns each into a vector, and...

You type "jazz saxophone" and get a piano piece? Little Pi introduces CFG, Classifier-Free Guidance, the knob that makes AI follow your prompt. The older classifier guidance (Dhariwal and Nichol, 2021) needed a separate...

Same prompt, two different songs? Little Pi takes apart the dice behind AI music generation. At every step, an autoregressive model gives a list where every candidate has a probability (each MusicGen codebook has 1,024...

Can melodies evolve? Little Pi introduces the genetic algorithm, which John Holland turned into a computer method in 1975 based on Darwin's survival of the fittest. Start with a random crowd of melodies, where every...

How should four codebooks take turns? Little Pi takes a deep dive into MusicGen's delay pattern. EnCodec produces 50 frames per second, and with RVQ each frame has four codebooks, so every second has 200 tokens to...