Please note: This code comes with no warranty, nor support, whatsoever. None. Zip. Nada. If your talking robots become self-aware and enslave humanity, then I will not be held responsible. But if you're in the mood for tinkering, here's how it's strung together:

• The sound is generating using Linear Predictive Coding ("LPC").
• First, some Python code: The analyze_lpc.py script analyzes phonemes.dat (which is just a headerless version of phonemes.aif). Individual phonemes are separated by moments of silence, so the script splits the sound file on those. Each phoneme is converted to LPC data, using code that I ported from the rt_lpc project. I felt like I understood the mathematics 3 years ago, but I doubt I could explain it today.
• Now, in Flash: Launch the DictCompressor application, and watch the trace messages. Click the screen to open the browser window, then select your cmudict___.txt pronouncing dictionary. (You can obtain the latest CMUdict here.) Flash will convert this to a (smaller) cmudict.dat file, which is what LPCsynth.swf loads.
• LPCsynth is the application that talks. The LPCSynthHarness.sayItNow() method creates an array of LPCFrames, which are "spoken" in the sampleData() method.  This was never intended for public distribution, so the code is not exactly stellar (the talking bit should be extracted into its own class).

Is this interesting? Did your Flash Player become self-aware after hearing its own voice? Let me know!

It's funny, originally this was intended for the controlzinc.com website. The robot voice would sing as you clicked, crooning about your mousing habits. I still can't decide if that idea was brilliant, or terrible.

More »

Watch: regular, or HD version. (Vimeo, 4:00)

The clips have a bias towards art and animation, because: 1). It shows well in a video, 2). This reel was created for an application to an artist's colony. (More on that later.)

The intro & outro animations (the flying moths) were created specifically for the reel. They're my favorite clips, yet they took less than a day to assemble. The moths were left-over from an old project (you'll see an experimental Too Many Moths interface within the video), and I felt the convergence motion was a strong opener.

The moths were composited in Flash. They were arranged semi-randomly, then the image was "baked" onto their wings & body. I flew them backwards, and recorded their positions on each frame. Then I reset the moths and flew them forward, again tracking their flight paths.

Actually, this was done in memory, so it was very quick. My laptop needed 3 seconds to arrange, bake, & move 120 moths. Then the harness presented me with the frozen frame, so I could evaluate how well the moths covered the images. If the coverage was bad, I hit a key to build new moths from scratch.

When I found a nice arrangement, I hit another key which stepped through the animation, and rendered each frame at a ridiculously high resolution (2160x1440, to cope with scaling problems in Adobe Premiere, unfortunately). The frames were encoded as .PNG's and uploaded to a PHP script running on localhost, which saved the 200+ images to disk. The intro took about 20 minutes to render & export. The outro, being more complicated, took over an hour.

Looking at the reel, I'm happy to present so many types of work — animation, audio programming, visual design, interactives, games, kiosks, websites. These four years have been amazing.

More »

• Import AIFF files
• Audition and edit formant sequences in real time
• Export .syx files for the Yamaha FS1R

Click to launch. Press the space bar to play the sound.

You can push your formant sequence to the Yamaha FS1R, using software such as K_Take's FS1R Editor. Click the "Save .syx" button, and follow the instructions in K_Take's documentation. This is a lot of fun, and breathes new life into the FS1R.

This project became much deeper than anticipated! The code includes FFT analysis (thanks Gerry Beauregard), pitch detection, a formant detection algorithm, and an AIFF parser to read AIFF files. The interface was a challenge to design and implement, and there are still many unfinished features (see the issues list).

My energy is shifting to other work, so I'll enhance fseq-flash when time permits.

More »

The Korg Radias synthesizer was released in 2006, discontinued after a 4-year run, and seems to have polarized all its users. Some dismiss it as "digital crap", others hail it as a "future classic". Who is right? Are these people snobby analog purists? Is the other group Korg's minions, paid to drum up favor for this little silver beast?

I purchased the rack-mount model, the Radias-R, in 2006. I was relatively new to the world of "virtual analog" synthesis, and the sound of the Radias seduced me because it was so different. Think: bright, smooth, icy, crystalline, electrical.

There are many reasons to appreciate this synth. I'll summarize a few, because I want to skip to the heart of this post: The Radias contains 64 sampled waveforms ("SynthPCM"), and they're are a lovely, creative bunch, with lots of rich, cybernetic character. (Someone enjoyed designing these, I bet.) The comb filter is surprisingly useful and rich. Korg's free editing software is a joy to use.

Other features are likable, despite a few flaws: The 39 (!) knobs on the front panel don't rotate endlessly. The insert effects are impressive, but some of the best effects (the amp simulators) are restricted to monoaural output. The unison & stereo spread features are great, but you can't target them via the modulation matrix (yet you can twist those knobs in realtime).

There's also a snappy vocoder, which can vocode live input, or record input as "formant motion" and play it back. Unfortunately, recording is difficult, and must be triggered on the hardware. You can't scrub, or change the formant motion speed. The Radias can store 16 formant motions, but only 8 presets are provided. I hate to admit this, but most of them are tacky... (One of the presets speaks, "Yo! Everybody in the house, say 'Raaaaadias!'")

How Does It Sound?

The most important feature of the Radias -- which you will either love or hate -- is the character of its sound. My opinion is that anyone who describes the Radias as "deep" or "warm" is wrong, just plain wrong. The Radias is the opposite of warm, fuzzy analog. It is deeply cold, in an air-sanitizing sort of way. If iciness can be sexy, this is it.

The sound has a certain sonic distance, as if the phase of one speaker is inverted (but of course, it's not). The basic waveforms seem a bit sterile, even after summoning some analog trickery (detuning the oscillators, etc.) The sound is so smooth that it's practically transparent. This synth sits well in the background. With its bright character, I've been able to drop it into many mixes, and it magically sits well with minimal equalization. But my attempts to feature the Radias as the frontman have never felt satisfactory.

How did Korg craft this particular sound? I ran some experiments to figure this out. Here is my conclusion:

In most cases, the waveforms are not synthesized in realtime. They're pre-sampled.

The Formant provides a clue

The Radias has 9 oscillator types, including a Formant oscillator. (A formant is similar to a bandpassed sawtooth wave.) Here's the spectrum of a formant produced by the Radias, with the narrowest formant width available. (This spectrum was analyzed in Ableton Live.) This looks acceptable, you can see the prominent peak around 800hz:

However, when we increase the formant width, instead of producing a wider hump ... The peak splits into two peaks, with the same narrow widths as before:

By increasing the formant width further, the peaks move even farther apart:

Although the sound is pleasant, this is not a true formant. This behavior indicates that the Radias' "formant" is actually composed of two waveforms. These waveforms have been pre-sampled at various resonant peaks. The Radias produces its "formant" by playing two of these samples simultaneously.

The Sawtooth misbehaves

Another Radias oscillator type is the Sawtooth wave. When played at mid-range and high frequencies, the Radias delivers the expected sound -- a series of overtones marching up to infinity:

But something undesirable happens when we move down the keyboard. When playing lower notes, the high-end treble frequencies roll off:

Further down, it sounds even more muffled:

The Sawtooth waves are not being generated in realtime. They, too, are sampled. This sort of frequency roll-off implies that a sample is being played at a low frequency, and interpolated (a.k.a. "anti-aliased") to smooth the output. I'm not sure which window function Korg chose for the interpolation, but it is the driving force behind the Radias' character, as it filters every sound. Also note that the other fundamental waveforms (Square, Triangle and Sine) are sampled as well.

I suspect that waveform modulations (such as square wave's pulse width modulation) may be pre-sampled as well, since they exhibit the same roll-off. By sweeping the pulse width, the Radias may be interpolating between waveforms (like using wavetables). I can't seem to produce an audio demo to support this claim, so the wavetables may have hundreds of samples in them, too many for the human ear to discern.

However, the two effects based on frequency modulation ("Cross" and "VPM") do modulate in realtime. They produce appropriate spectrums, even when the oscillators run at low frequencies.

The Radias features a "morphing" filter, which can move between two filter shapes (for example, a lowpass and a highpass). This is probably achieved by processing the incoming signal with two "cardinal" filters (LPF24, LPF12, HPF, or BPF), and mixing their results together. I can't prove this, but the spaces between the cardinal points seem a little dull, as if phase artifacts are creeping into the treble, or the anti-aliasing artifacts are doubled.

These experiments helped me understand how the Radias acquired its sound. Short, single-cycle waveforms are looped and anti-aliased, resulting in smooth, hygenic timbres. Some of the longer samples (in the "SynthPCM" group) have more density, but they're still filtered by the anti-aliasing algorithm, which makes them sound airy and hollow. The Unison and Analog Detune features can add density, but in my experience, they don't add much thickness or richness, as it's merely playing multiple copies of the same smoothed waveform.

Hopefully this post doesn't read like Radias-bashing. I was more interested to discover Korg's tricks than anything else. The Radias is suited to some very specific genres and roles, and I wouldn't recommend it to everyone. Still, the sweet GUI editor makes this the most "fun", accessible synth in my collection, and I keep returning to the Radias for this reason alone.

To hear some creative Radias patches, check out Jexus' great YouTube demos: part 1, part 2.

More »

Filtered Noise Sequencer

Drag and resize the blue blocks to change the filter frequency and width.

This sequencer is not using expensive bandpass filters. The oscillators are sine waves, which are frequency modulated with white noise. It may not sound inherently musical, but you can produce great hihats, bass thuds, and airy pitched noises.

Here's the source code. (Requires Flash CS5 to compile.) Have fun!

More »

A Curious Lack of Light by tooManyMoths

More »

First, do you know Lev Grossman? He's an incredibly talented author who recently toured Portland. If you haven't read his book The Magicians, then stop whatever you're doing and procure a copy immediately. Without trying to spoil anything, the major college in the book is named the Brakebills College for Magical Pedagogy. Lev saw the Brakebills T-shirt that I designed for my sweetheart's birthday present:

Long story short, my Brakebills shirts are now officially endorsed by the author! Please buy twenty of them, or thirty if the spirit moves you. Zazzle's print quality is great. The colors are vivid, and the ink has survived many washings. Rabid fans are buying and even customizing them! I love the internet.

Lev blogged some of my other work, too ("the guy who does this has the enviably fake-sounding name of Zach Archer"). It's true, I have an awesome pro wrestler name.

Second, my new iPhone app has landed in the App Store:

Twang is a handheld guitar. It's easier to play than a real guitar, and is very expressive. Instead of using audio samples, Twang uses physical modelling techniques to create a more natural, dynamic sound. No two plucks are identical. Watch my grainy first video if you disbelieve.

In the next version of Twang, left-handed people will be able to switch Twang's orientation, and serious musicians can dampen or mute strings with their fingers. And probably more! This version is already in development, and may be submitted in a week or two? Follow Control Z, Inc on Twitter if you have a ravenous thirst for updates!

More »

Here's something from the vaults. Aquasound was built with these requirements in mind:

• Generate sounds that aquatic animals might make
• Sounds can be "combined" somehow
• Sounds can emote

This was never used in production. I wonder if I could turn this into something? Like a paid iPhone app? ;)

Double-click the envelopes to add/remove control points. Drag lines up & down to change their curviture. The best feature is the "Combine With" dropdown, which splices the current sound with your selection. Also the "Emote" menu will play sounds with different expression.

The audio algorithm is reverse-engineered from my beloved FS1R. I generated formants in two ways (toggle the "Tonal" checkbox to hear both), the "atonal" version is closer to ring modulation than actual formants. It's more fun if you don't understand what the controls are doing, but if you insist: Pitch controls the overall pitch of the sound. Freq controls the center frequency of the formant (like a bandpass filter). LFOFreq and LFOWeight control a low-frequency sine wave, which can be applied to other controls via their "___LFOAmt" curves. Amp is amplitude, Width is formant width (think: width of the bandpass filter), Skirt adds distortion. Each voice has two formant generators, check "Formant Active" to enable them.

May all your bloops and crackles be happy ones!

More »

This took about 5 weeks to develop. Meanwhile, I've started another app, and I envision releasing Metal Mouth 2.0 in a few months, with more voices, and the ability to record audio.

More »

Here's my first attempt at LPC speech in Flash: (click & explore)

It's great, except for one tiny problem: It sounds horrific. Can you feel the cold, robotic love? This voice will stalk your nightmares.

The phonemes were derived from an unrehearsed recording of my voice. I'm confident that it can be improved. Note that direct LPC encodings of my voice, such as this one, sound more acceptable.

EDIT: I made an iPhone version, "Metal Mouth", with lots of features. Here it is on YouTube and the iTunes Store!

EDIT #2: The source code is available here.

More »

Like the era-defining DX7, the FS1R is an FM Synthesizer, but it boasts a massive 8 operators per voice, compared to 6 in the DX. And the FS1R sports a new toy, Formant Synthesis, capable of mimicking voices, human and otherwise! Waves and formants can modulate each other in 88 different configurations. Top that off with LFOs, filters, on-board effects... It's so flexible, and so complicated. So much power.

The FS1R was the last of Yamaha's FM synthesizers, and that's no accident. Even by 1998 standards, this thing dwells in a deep abyss of user-unfriendliness. Without a GUI, you must program 1,000+ parameters using 15 tiny buttons on the front panel. Fortunately, K_Take has developed FS1R Editor Software which lets you load/save patches, and twiddle the internals in real time (thanks K_Take!)

The unit sold poorly and was discontinued after 2 short years (long before a proper GUI emerged). I didn't know the FS1R existed, until Tavvv of the Braindead Monkeys mentioned it might be something my brain would enjoy -- Weird vocal effects! Technical crazy-sauce! Aural blasphemy!

Thus, the hunt was on. After two months, I had a single nibble -- a unit appeared on eBay, and I became its adoptive parent. (These units are scarce; only two units are listed on eBay at the time of this writing, and they're not getting any cheaper.) Was it worth it? Good question. If I had to describe this thing in one word, it would be: "Uhnnnhhhhh ... !!!"

After three months of use, I'm starting to comprehend this beast. With 8 operators (waves/formants) per voice, you must choose one of 88 algorithms. Here's a screenshot of a few (in K_Take's application):

Each operator has a zillion parameters: pitch and amplitude envelopes, a "skirt" (which makes the sound fuzzier), and wave shapes (sine, sawtooth-esque, square-ish, formants) which have another zillion parameters of their own. Also each voice has 8 "unvoiced" operators, which generate pitched noise. These imbue its "speech" with sibilance, and thus, consonants; A zillion more parameters.

Everything I just listed is actually quadrupled, as you control 4 voices per program. And you can use "formant sequences" to make the operators fly around and "talk", morphing into vowels and phonemes.

Yeah, it's pretty hectic. This is a saturation point of some sort. This must be what happens when synths fly too close to the sun. Here are some sound samples that demonstrate the madness. If you're brave, here's a medley of my FS1R creations spanning 3 months, oldest to newest.

What the hell can you do with this device, then? I tried using the FS1R in a "lead" role for months; carrying the melody, or standing in front of the other instruments, snarling aggressively at the listener. That's not what the FS1R excels at. It sports an impressive collection of waveforms, but compared to other synths I've used, the waves sound soft, and clean. Even with the "skirt" maxed out, the FS1R can't produce a sawtooth wave, or a square wave, or anything big and phat that resonates your rib cage. FM synthesis can sound harsh on other platforms (the Yamaha TX81Z; the Sega Genesis) but the fidelity of the FS1R is apparently too good. You can't make this thing scrape, or scream.

This leaves you with a few options -- you can use it as a "background" instrument, or as a smooth pad. This is where the FS1R shines. It produces excellent textures that liven up a mix if they're used ornamentally. The FS1R could become my "secret weapon", adding that undefinable something to my sound universe. At least, I hope so. FM synthesis is hard to program because you don't really know what will happen when you twist that knob; little tweaks can send a patch spinning. The FS1R constantly surprises me, and that makes it cool. My best sounds come from building a patch using one algorithm (out of 88), and then clicking the other algorithms, arranging the operators into something that I couldn't have premeditated.

It has filters and on-board effects, too. Honestly, I've barely touched these, I'm still internalizing the other stuff. The filter seems unexciting; it's transparent and color-less, although it does offer three lowpass slopes (12, 18 and 24 db/octave), which is nice. The effects include several flavors of chorus & reverb, delays, dynamics processors, and two distortions (although they're soft too, and neuter the sound more than give it teeth).

My wish list for a sequel (the "FS2R"?):

• New modulation types between operators -- ring modulation and hard sync. That way, "the kids" can play it in "the clubs".
• More LFOs. You get 2 per voice, and 1 is reserved for the filter. More are needed!
• Formant sequences should be editable, or better yet, recordable. The manual states that sequences cannot be edited, but of course they can if you're willing to hack it a bit.

Meanwhile, the FS1R will keep me busy with its steep learning curve. At this point, I have rare moments of clarity where I can plan, and (mostly) construct the sound I want. What will I produce after 3 more months? I have no idea, none at all. We shall see.

P.S. Thomas Gruber explains how the FS1R's DSP works, and it's spiffy.

More »