Performing in the Metaverse (AWE Asia 2023)

I gave this talk on the AWE Asia main stage in August 2023, on the subject of performing live music and visuals inside VRChat- how the whole grassroots scene works, who’s actually showing up, and why. Full video’s below, but here’s the write-up for anyone who’d rather read than watch 25 minutes of a bald man pacing a stage.

https://www.youtube.com/watch?v=Xs9wOI1B09E

How I got here

Funny enough, this whole talk is a bit of a homecoming. Long before Dell XR Studios, before Lucasfilm, before any of the day job stuff, I was VJing live visuals for touring acts- Panasonic MX50 vision mixer, a pile of computers, someone handing me the desk at the Big Day Out in 2002/2003 and saying “figure it out.” I told that story on stage with the actual photo from back then still on the screen behind me. Twenty-plus years later I’m doing more or less the same job, just the venue moved into a headset.

I’d been quiet on the live circuit for a while- COVID happened, I’ve got a receding hairline now instead of hair, and I was in the middle of moving to Boston (I flew out three days after giving this talk). What pulled me back in wasn’t a booking, it was boredom during lockdown- I started going to virtual clubs in VRChat just to see what was actually happening there, and kept going.

The how

VRChat is the platform, though none of this is exclusive to it. The recipe on the performing side is close to what I was running in 2003, just with a VR headset bolted on: a real DJ setup (decks, MIDI controller, a one-handed keypad off to the side so I can switch video clips blind), Traktor for the actual mixing, Resolume for the visuals, OBS to stream it out, and a VRCDN account so the stream can get pumped into the club at low enough latency that hitting a cue and hearing it land are only about two seconds apart (not perfect but close enough). The visuals I’m running are footage I made back in 2003- 640x480 QuickTimes, extremely low res, extremely glitchy, and that’s kind of the point.

The avatar is a scientist model from Half-Life 2 I rigged myself in Blender, and the venue is a Half-Life 2 level I decompiled and rebuilt as a DJ stage- lighting rigs that respond to the music, pattern and colour all live-controllable, and yes, you can pick up a crowbar as Gordon Freeman and start throwing it at people mid-set. People do. It’s part of the bit. I like running the old scientist model specifically because everyone else in the room tends to be a sleek sexy anime avatar, and I’m the bald middle-aged guy who showed up as a Half-Life NPC with an AOL logo on my back.

Full body tracking is the other half of the “how”- most people are running SteamVR trackers, which means real money and trackers whose batteries die mid-session, so you’ll see people’s legs just lock up. There are cheaper inertial disc trackers coming out from other manufacturers too, phone-camera tracking is starting to show up too. None of it’s perfect yet, and the glitches themselves become part of the culture- first time you see someone lying flat on the ground in a club you assume their tracker died- but usually they’re just asleep (really, they do that).

The who

There’s a real spread of users, not just headset owners: desktop users who show up as a static avatar and never move, just there to have the music and the community running in the background while they work. Quest users (aka “Questies”), who make up something like half of everyone there, but can’t see the live streams at all because of a protocol limitation- they’re in the room, just watching a different version of it. SteamVR users tethered or standalone. And then full body tracked users on top of that, the ones actually dancing with full natural motion.

Typical age is 18 to 24, which makes me the old man in the room by a wide margin- they call people my age boomers. North America runs the busiest timezone, but there are solid pockets in Japan and Europe too. Most people are there for exactly the reason you’d go to any club: to hang out, dance, and talk to people. VRChat pulls 30,000 concurrent users on a slow day and up to 75,000 including Quest users on a busy one, all inside an open platform built on open standards, which is the actual reason any of this works- nobody’s gatekeeping who gets to build a world or bring a body into it.

The emergent behaviour is where it gets genuinely strange in a good way. Clone dancing, where a song drops and the whole crowd spontaneously copies whatever avatar someone nearby is wearing- I’ve watched a room turn into a dozen identical throat-singer avatars because that’s what one person happened to be running. Phantom touch, where people reach out and hold a hand or a shoulder for long stretches, because the gesture itself builds a real sense of presence even with zero haptic feedback. Gender presentation gets loose fast- female avatar, male voice, nobody blinks. And there’s a whole deaf community that uses VRChat specifically to practice and teach sign language, which is not a use case I’d have guessed walking in.

The why

Bringing this to an AWE stage instead of leaving it as a hobby was the actual point of the talk: what the metaverse people keep talking about in the abstract is already fully alive in specific corners like this one, and it’s alive because it’s open- open worlds, open standards, and people building whatever they want out of it rather than waiting for a platform to hand them a feature. From the day-job side at Dell XR Studios, watching what peripherals this crowd actually uses versus what they wish existed is a genuinely useful signal for where the hardware needs to go next- multimodal input, presence, edge compute, all the stuff that’s still half-finished. It’s a rough preview of what Web3 was supposed to be, minus the token speculation.

I closed the talk with footage from a Shelter event- an LA club/label that runs live hybrid shows a couple of times a year, tying a real venue to a virtual one at the same time. That’s the actual horizon here: not replacing the physical gig, tying the two together properly.

Q&A afterward was mostly about where full body tracking goes next- cheaper inertial trackers, better calibration, maybe optics eventually. My honest answer on stage was that the funniest thing to me is watching a room full of people with thousands of dollars of tracking gear get more excited about someone turning on a plain webcam than about the tracking itself. Says something about what people actually want out of presence.