It started with Audiosurf
Growing up, my internet connection crawled along at 56 kbit/s. This was dial-up: a modem connected through the telephone line, with a chorus of beeps and screeches every time it dialled in. Even getting a 3G USB modem barely helped. Where I grew up, the mobile signal was still stuck on 2G/EDGE.
Luckily, I got Igromania every month. It was a Russian gaming magazine with news, reviews and a cover disc packed with demos and useful software. With internet that slow, those discs were a big part of how I discovered games. The April 2007 issue has stayed with me ever since.
Mostly because of the brilliant song that played in the disc's main menu: Seven Wiser's Sick.
Anyway, enough nostalgia. Here's where this is going.
I can't remember which issue it was, but one day I read about Audiosurf.
The idea felt fresh: load any song and it becomes a road. Parts of the music, such as the drums, create coloured blocks that you collect into matching groups of three or more.
I wanted to ride through every song on my computer. Music was easy to come by: friends were always sharing flash drives and CDs.
So I left Download Master running for a day and a half, praying the SIM card in my modem wouldn't run out of credit. Somehow, all 300 MB arrived and I finally got to play.
I loved spending time with my own playlist that way. Even then, I saw it as the starting point for a kind of game I wanted to explore.
Rhythm games
You probably know a few of the famous examples.
Audiosurf gives any piece of music a physical shape. Each song changes the experience in its own way. So I'd narrow this down to a particular kind of rhythm game:
Equalizer rhythm games
Remember those old music visualizers?
Music as a game system
When you think about it, Audiosurf is an interactive equalizer.
That got me thinking about which other genres had already tried this, and which combinations were still missing.
Turn-based games driven by music? Done.
Shooters driven by music? Done.
Racing games driven by music? Done.
...
I considered strategy games. Three minutes later I realised how hard that would be to put together. Though I'm sure someone has tried it.
Then I remembered the game we were all hooked on three or four years ago: Vampire Survivors.
And its little brother, Brotato.
Eventually I decided to mix all of it together.
A music visualizer, a roguelike and a survivor game.
Coming up with that is one thing. Actually making it work is another.
The prototype that played itself
The engine was an easy choice: Godot.
Everything else took a lot more thought:
- What should react to the music, and how?
- How do I break music down into useful aspects?
- Which music? A soundtrack made for the game, or any song?
- How will the player feel progress, and what will that progress look like?
I had rough answers to the first three within a couple of days. You probably thought of something similar. The fourth question turned out to matter much more than it first seemed.
In Vampire Survivors, levelling up is a pause and a breather. But this game runs to music. Stopping everything every ten or fifteen seconds would ruin that.
For the first prototype, I probably picked the daftest possible solution:
- The player chooses any song.
- The analyser breaks it down by intensity and frequency into aspects such as drums, bass and vocals. I started with three.
- Each weapon is assigned to an aspect slot.
- It fires when that aspect triggers.
- The player stands still.
- Enemies run towards the player and weapons automatically target the nearest one.
- Experience is collected automatically.
- Level-ups automatically improve a random stat.
- The player hopes they survive the first ten seconds.
Here's what that looked like.
This prototype only had three weapons.
A beam, a lightning strike and a basic orb. Fake lyrics also flew out and dealt damage.
I turned on invincibility for the demonstration. Otherwise I'd have died in the first ten seconds.
It sort of worked. You put on a familiar song, the screen lights up at the right moments and enemies pop on a kick drum. Your brain goes: 'Oh, the music is actually playing along with me.' More or less. But watching and playing are different things. The character stood still, weapons chose their own targets, and I mostly watched the equalizer.
I'd made a music screensaver. More enemies, more guns and more particles wouldn't bring back the missing part: feeling that I was the one trying to survive this song.
The idea still seemed worth pursuing. The implementation, less so. I scrapped that version, identified the main problems and moved on.
Giving the player control
I kept two rules that still form the core of the game:
- The player always plays to their own music.
- Weapons automatically fire to different aspects of that music.
But now the player would move and decide which weapons to assign to which aspects. Upgrades would happen between battles in a Brotato-style shop. A run would be a playlist of N songs, with stronger enemies at each stage and a boss at the end. I also needed to improve the visuals and settle on a visual language.
With that in mind, I started rebuilding.
First, I let the player move and watched what happened.
I'm not entirely sure why I didn't quit right there. Something about it still felt promising.
After a bit of sulking, I carried on.
- Let the player assign weapons to musical aspects.
- Rebuilt the music analyser from scratch.
- Added a combo system where pressing a button to the drums increased a damage multiplier. For some reason.
- Brought back levelling up during combat, this time on the 1, 2 and 3 keys. Also for some reason.
- Added some visual work to the menus and combat.
It started to look vaguely like fun.
But upgrading during combat and tapping a button to build a combo both felt awkward. The visuals were rough, and the analyser was splitting aspects in strange ways.
So, another round of changes:
- Removed button-based combat upgrades. Enemies now drop bonuses instead.
- Removed the combo button.
- Polished the visuals and built a proper main menu.
- Added a roguelike run mode.
- Put shops between battles, where crystals earned in combat buy stats and other upgrades.
A different song this time.
Visually and mechanically, it was finally starting to resemble a game. That was encouraging. I also felt less embarrassed showing it to friends, and their first feedback helped me keep going.
Finding a visual language
I'll skip over some of the less exciting details. The short version:
I spent two weeks improving the visuals and adding detail. I rewrote effects that generated geometry procedurally to use shaders instead. In that build, it gave me roughly ten times the frame rate and much better stability.
The game finally had a visual language. A simple one, but I think the VHS and retro look suits it.
I added metaprogression using resources earned during runs: cassettes and vinyl records.
Then I added personalization, which changes how your battles look.
Weapon styles, backgrounds, the player character and more.
Here's how that changes a battle.
The run format was now: choose your songs, play through them, earn currency for visual customization and permanent upgrades.
I also added two super abilities.
Storm.
And a rewind that creates a copy repeating your actions from the last five seconds.
On death, time and the song rewind, then the character revives with a hit across the entire arena.
How the game reads music
The track analyser finally came together too. It deserves a closer look.
The analyser processes a song in advance, so the game doesn't have to guess its rhythm during combat.
A Python script uses Demucs to separate drums, bass, vocals and other instruments. It then identifies BPM, beats, accents, note changes, sustained sounds, intensity and sections such as quiet passages, build-ups and drops.
Lyrics can also be aligned using WhisperX locally, but that takes around five to ten minutes per track, so it's a fallback. First the game looks for synchronized lyrics through LRCLIB. It searches using the filename and chooses the result whose duration is closest to the song. LRC timestamps become word events in the track map.
If no synchronized lyrics are available online, the game offers local vocal recognition with WhisperX. You can review and fix the words in an editor. That lets text-based weapons fire on actual spoken words rather than roughly following the vocals.
The result is saved as a JSON track map. During play, Godot plays the ordinary audio file while the Conductor follows that map and emits musical events. Drums, bass, vocals and the other aspects become a firing schedule for the weapons.
The first analysis takes a few minutes. Once cached, subsequent plays start straight away and follow the same musical timing.
And if something was detected incorrectly, there's an editor.
I was already thinking about Early Access, but the game loop still felt too simple. So I started reworking it again.