Media Servers II

A Program That Runs Once Per Pixel

A shader is a small program compiled to run directly on the GPU: a vertex shader runs once per vertex (typically deciding where a point ends up on screen), a fragment shader runs once per pixel (typically deciding that pixel's color), and both run for every vertex or pixel simultaneously across thousands of GPU cores. The canvas below is running an actual compiled GLSL fragment shader — the classic demoscene "plasma" effect, built from nothing but summed sine waves evaluated at every pixel, every frame, on your GPU right now.

The whole visible pattern is just sin(x) + sin(y) + sin(x+y) + sin(√(x²+y²)), evaluated at every point and mapped to color — no textures, no geometry, no lookup images, just trigonometry running fast enough, at enough points, to look like liquid light. It's the same wave equation from Universal Waves, doing pixel duty instead of describing sound or light.

The Projector Doesn't Know It's Hitting a Corner

Part One's corner-pin demo warped a flat image to fit a flat (if tilted) rectangle. Real architectural projection mapping is harder: the surface is a real 3D object — a building with multiple faces at different angles and depths — and the projector is a real point in 3D space throwing light along diverging rays. Move the projector, and every face's distortion changes independently, in a way flat 2D warping can't capture; the software has to model the actual camera projection matrix (a real light source's position, direction, and lens field-of-view) and compute, per pixel, exactly where each face's texture should land.

Try it: move the projector

Drag the red dot to move the virtual projector. Watch the grid "light" on each face of the building stretch and skew independently — the actual geometry of projecting from a single point onto a 3D object with multiple faces.

Real-world calibration usually automates this with structured light: the system projects a sequence of known patterns (stripes, grids, gradients) onto the surface, films the result with a camera, and solves for the exact 3D geometry and projector position that would produce what the camera just saw — turning "where exactly is this building, relative to this projector" into a problem a computer can answer in seconds instead of an installer manually nudging keystone sliders for an hour.

Real Waves, Built From the Same Wave Equation

Trace a media server's most literal wave-to-3D pipeline: ocean rendering. A Gerstner wave — first described mathematically in 1802 — displaces each point on a flat grid both vertically and horizontally, tracing a circular (trochoidal) path rather than a simple up-down bob, which is what gives real Gerstner waves their sharper crests and broader troughs^[1]. Real-time computer graphics has used exactly this since 1986, and the modern standard technique — formalized in Jerry Tessendorf's widely-cited 2001 course notes — sums just 4 to 8 of these waves, at different directions, wavelengths, and amplitudes, directly in a vertex shader to produce a convincing ocean surface in real time^[1].

Try it: build an ocean from sine waves

At 1 wave, you're looking at a single, obviously-fake corrugated sheet. By 6–8, with each wave contributing a different direction and wavelength, the same grid of 3D points reads as genuinely turbulent, unrepeating open water — the entire illusion built from nothing but sine functions and vector addition, per vertex, per frame.

A Shape Is Just a Wave With Extra Dimensions

A microphone's 1D signal decomposes into a sum of sine waves at different frequencies — that's the Fourier transform, covered from the audio side in Music Is Mathematics. The exact same idea, one dimension up, decomposes a 2D image into 2D spatial frequencies (the basis of JPEG compression), and one dimension up again, decomposes the surface of a 3D shape — or the pattern of light arriving from every direction around a point — into spherical harmonics: the 3D, spherical-surface equivalent of a Fourier series, and the real technique behind fast, realistic real-time lighting in modern game and film rendering. It's the same transform each time. Only the number of dimensions changes.

Try it: build a shape from pure sine waves

A square wave, like any repeating shape, can be built entirely from a sum of sine harmonics — the 1D version of exactly the same decomposition spherical harmonics do in 3D.

A Grid of Arrows Is Also a Wave

A vector field assigns a direction and magnitude to every point in space — wind speed across a map, water flow through a pipe, or a purely mathematical field built for a visual effect. Curl noise — taking the curl (a measure of local rotation) of a smoothly-varying noise field — produces a vector field that never has any "sources" or "sinks," which makes it the standard technique for believable, turbulent-looking particle flow: smoke, ink in water, or a purely abstract flow-field artwork. The particles below don't know about "the field" as a global shape; each one just reads the direction at its current position and moves, every frame — a vector field is nothing more than an enormous set of tiny, local wave-like oscillators, sampled at your particle's exact location.

Two Chemicals, Diffusing, Are a Media Server

The Gray-Scott model simulates two virtual chemicals — one "feeds" the system, the other "kills" the first — diffusing across a grid and reacting where they meet. Depending on just two numbers (the feed and kill rates), the exact same equation produces wildly different patterns: coral-like branching, mitosis-like splitting spots, fingerprint whorls, or traveling worms — the same class of self-organizing chemical wave introduced in Universal Waves' chapter on chemical waves, and the mathematical basis of naturally-occurring Turing patterns in real animal markings.

Every Family From Universal Waves Has a Real Technique Here

None of these connections are loose metaphors — each is a real, named technique used in production creative-technology work.

The Idea Underneath Everything on This Page

A color is a vector with 3 components (red, green, blue). A single point on a 3D mesh usually carries a position, a normal, a UV coordinate, and a color — a vector with 10 or more components. A 512-bin audio spectrum, sampled once, is a single vector living in a 512-dimensional space. An AI image model's internal representation of a picture is a vector with thousands of dimensions. None of these are metaphors for vectors — they are, mathematically, exactly vectors, full stop, just living in spaces too large to draw.

And a wave, the moment you sample it at a finite number of points — which is the only way a computer can ever store one — becomes exactly the same kind of object: a vector, with one number per sample. The Fourier transform, the spherical-harmonic decomposition, the FFT running under the plasma shader above — every one of them is nothing more exotic than rotating that vector into a different, more useful set of coordinate axes, the same way you might rotate a 2D point's (x, y) coordinates into a rotated frame. That single idea — a wave is a vector, and a transform is a change of basis — is the actual mathematics connecting every demo on this page, and everything on Universal Waves, to each other.

1. Simulating Ocean Water — Jerry Tessendorf, SIGGRAPH course notes, on Gerstner/trochoidal waves and the sum-of-sines technique.
2. Reaction-Diffusion Tutorial — Karl Sims, on the Gray-Scott model and its named pattern presets.