Building a Playable 3D Guitar in React

I wanted to put a 3D guitar on my portfolio.

A normal model viewer would have been enough: load a GLB, add orbit controls, give it decent lighting, and let people inspect it.

But I play guitar. A guitar that only rotates felt wrong.

So the experiment slowly became a browser instrument. You can pluck individual strings, strum across them with the cursor, switch between chords, use keyboard controls, and let a small song sequencer play chord progressions automatically.

The interesting part was not rendering the model. It was making the visual object, pointer interaction, music state, and audio clock behave like one system.

You can play the guitar experiment here, or browse the rest of my interactive experiments.

The 3D model was only the starting point

The guitar begins as a GLB model loaded with Three.js. I render it inside React with React Three Fiber, which lets the 3D scene live alongside the rest of the component state and interaction logic.

The model itself came from Dzenly on Sketchfab under CC BY 4.0. I kept that attribution directly on the experiment page instead of treating the asset as if I made it.

The first version could have stopped there.

GLB model
  -> camera
  -> lighting
  -> orbit controls

That gives you a viewer. It does not give you an instrument.

The strings needed to become separate interactive objects. They needed positions, frequencies, fret behavior, pointer hit detection, and visible vibration. Once that happened, the imported model became more like the body of the interface than the entire interface.

That distinction changed how I thought about the experiment. I was no longer trying to make a realistic 3D render. I was trying to make something that responded correctly enough to feel playable.

I separated the guitar into systems

Trying to put every behavior into one Three.js scene component would have become difficult to reason about quickly.

The implementation ended up with a few distinct responsibilities:

• the viewer owns modes, selected chords, song state, keyboard input, and the audio instance
• the scene owns the model, string placement, fret state, and pointer interaction
• each string owns its geometry and animation
• the audio class owns synthesis and the Web Audio graph
• chord and song data stay outside the rendering code

That separation matters because the same string can be triggered in several ways.

A user can click or drag across it. A keyboard key can play it. Easy mode can change its frequency through a selected chord. Auto mode can advance the chord without touching the pointer at all.

All of those paths eventually need to describe the same musical event instead of implementing their own version of a string.

The strings are geometry, not part of the imported model

I render the playable strings separately using Three.js line geometry.

That gives me control over the points along each string. When a string is triggered, its geometry can move over time instead of relying on an animation baked into the original model.

The important visual trick is simple: the endpoints stay attached while points between them move. The displacement decays, so the string settles back into place.

It is not a physical simulation of an actual steel string. I did not need one.

For this experiment, the goal was perceptual correctness. The pointer crosses a string, the string moves immediately, a note appears, and sound follows. That feedback loop matters more than simulating every force acting on a guitar.

I have found this useful in interactive work generally: realism is expensive, but responsiveness is often what makes an interaction believable.

The sound is synthesized in the browser

I initially needed something more useful than playing one prerecorded audio file for every interaction.

Each string has a frequency, and the browser synthesizes the sound with the Web Audio API. The current implementation combines a few short triangle-oscillator voices with slightly different frequency ratios, a low-pass filter, an amplitude envelope, and a small burst of noise at the attack.

That is still an approximation. It does not sound like a carefully sampled acoustic guitar, and I do not pretend that it does.

But a single clean oscillator sounded too electronic. The extra harmonics, tiny detuning, fast attack noise, and decay make the interaction read more like a plucked object without introducing a library of recorded samples.

Muted strings use a different path: a short filtered noise burst instead of a pitched oscillator. That gave me a lightweight way to represent the percussive "chuck" of a muted string.

The nice part is that all of this stays inside the browser. There is no audio backend involved.

Timing became a different problem from animation

The moment I added automatic chord progressions, normal UI timing stopped being good enough.

JavaScript timers are useful for updating interface state, but I did not want the musical timing to depend entirely on when the browser happened to run a callback. The Web Audio API exposes AudioContext.currentTime, a separate audio timeline specifically useful for scheduling playback. MDN describes it as the context's elapsed audio time and documents it as a clock for scheduling and visualization.

So auto mode uses the audio clock as the source of timing.

The scheduler checks ahead by a small window and advances the next musical event before it becomes due. The UI can then update around those scheduled times instead of pretending a setInterval callback is the beat itself.

This follows the same general lookahead pattern MDN documents for browser sequencers: periodically inspect the audio clock, schedule events slightly ahead, and let the audio timeline handle precise playback.

MDN's Web Audio scheduling guide was useful here because it separates two jobs that are easy to mix together: scheduling audio and drawing the current state of the interface.

That separation also made the beat indicator easier to reason about. The dots are visual feedback. They do not control the music.

Chords are data, not six special cases

Easy mode lets the player choose a chord and then interact with the guitar normally.

I did not want if (chord === "C") logic spread through the scene. Chords are represented as data describing what happens to each string: open, fretted, or muted.

From that, the scene can determine the fret position and the audio layer can determine the resulting frequency.

That same representation is reused by auto mode. A song is essentially a BPM plus a sequence of chord IDs. The sequencer advances through those IDs, while the guitar already knows how to render and play each selected chord.

This is one of the smaller architecture decisions in the experiment, but it removed a lot of potential duplication.

song
  -> chord ID
  -> chord definition
  -> string fret/mute state
  -> rendered string + frequency

The UI, scene, and audio code do not each need their own interpretation of what a C chord means.

Interaction had to win over camera controls

There was another conflict that only appears once a 3D object becomes interactive.

Dragging is normally useful for orbiting the camera. Dragging across the guitar is also how I wanted strumming to work.

Those two interactions cannot own the same pointer movement at the same time.

The viewer therefore tracks whether the guitar itself is being interacted with and temporarily disables orbit controls during that interaction. When the gesture ends, camera rotation becomes available again.

It is a small detail, but without it the experience feels broken immediately: trying to strum would rotate the entire instrument underneath the cursor.

This is the kind of problem I like about browser experiments. The difficult part is often not the headline technology. It is deciding which system owns an input at a particular moment.

I prefer experiments that teach me something reusable

The guitar now sits beside a vinyl player, Polaroid camera, and cassette player in my experiments page.

I do not expect these to become products. That is not why I build them.

They give me a place to work on interaction problems that normal portfolio pages rarely create: 3D scenes, pointer gestures, audio graphs, animation timing, model loading, keyboard controls, and the boundary between React state and an imperative rendering system.

The guitar happened to combine more of those problems than I expected.

I started with a model I wanted people to rotate. I ended up thinking about fret positions, synthesized attacks, audio clocks, strumming gestures, camera ownership, and how much physical realism an interface actually needs.

That is a better outcome than a model viewer.