Please note: This code comes with no warranty, nor support, whatsoever. None. Zip. Nada. If your talking robots become self-aware and enslave humanity, then I will not be held responsible. But if you're in the mood for tinkering, here's how it's strung together:

• The sound is generating using Linear Predictive Coding ("LPC").
• First, some Python code: The analyze_lpc.py script analyzes phonemes.dat (which is just a headerless version of phonemes.aif). Individual phonemes are separated by moments of silence, so the script splits the sound file on those. Each phoneme is converted to LPC data, using code that I ported from the rt_lpc project. I felt like I understood the mathematics 3 years ago, but I doubt I could explain it today.
• Now, in Flash: Launch the DictCompressor application, and watch the trace messages. Click the screen to open the browser window, then select your cmudict___.txt pronouncing dictionary. (You can obtain the latest CMUdict here.) Flash will convert this to a (smaller) cmudict.dat file, which is what LPCsynth.swf loads.
• LPCsynth is the application that talks. The LPCSynthHarness.sayItNow() method creates an array of LPCFrames, which are "spoken" in the sampleData() method.  This was never intended for public distribution, so the code is not exactly stellar (the talking bit should be extracted into its own class).

Is this interesting? Did your Flash Player become self-aware after hearing its own voice? Let me know!

It's funny, originally this was intended for the controlzinc.com website. The robot voice would sing as you clicked, crooning about your mousing habits. I still can't decide if that idea was brilliant, or terrible.

More »

If only Mixing Secrets For The Small Studio had existed 10 years ago, my music would have been impeccable! (Well, I like to think so.) This book is a magical tome for anyone who records or produces music on a budget. It's packed with big reveals, and explains the science behind each mixing technique. Forget the accumulated hit-or-miss wisdom of the internet; after reading this book, I found that I could produce substantially better mixes immediately. That's amazing. (My mixes still aren't great, but I'm working on it!)

Here are my favorite takeaways from the 20 chapters. I'm writing this to lock these concepts in my head. I'm skimming lots of material, because there's so much valuable information packed into this book, I can't possibly recap all of it.

Part 1: Hearing and Listening

My Rokit KRK-8 studio monitors are distorting my mixes. They're "ported," meaning they have an open-air port cut into the body. The manufacturer claims that this helps the air move freely. As a consequence, this creates noisy air turbulence, and muddies the bass frequencies. That's bad. The book suggests plugging the ports with socks; the sound might become clearer! Try it, listen carefully, trust your ears...

The goal of soundproofing your studio is to tame reverberant frequencies. You don't want an echoless space. Also, stereo is overrated, try mixing in mono. If this advice sounds heretical, consider the typical consumer's listening environment: Computer speakers, stuffed in a bookcase, clear across the room. Car speakers, blaring inside a noisy, reflective space. Restaurant speakers, bolted to the ceiling in semi-random locations. Your mix should sound good in mono, because most people will never give it a proper stereo listening. And there will always be room reflections in any listening space, so don't completely Nerf your room. Allow your walls to sweeten the sound.

Headphones are a special exception. The book cautions that some headphones have inadequate bass response (as do some studio monitors, of course). Music may sound oddly "clean" without the natural reflections of a room. Personally, I love modern headphones. I bought a pair of Beyerdynamic DT 770 headphones this summer. It was the single best musical purchase I've ever made. They have clear, deep bass response, which rattles my skull a bit. The treble is impressive, and it reaches very high into the range of human hearing. I can finally hear the full audio spectrum (unlike running my Rokit KRK-8's through a Mackie mixer), so the headphones have been a godsend for me. I can't imagine returning to my KRK-8's in my oddly-shaped studio space.

The biggest takeaway from this section is learning how to use other songs as reference material. Create a workflow where you can rapidly switch between your mix, and several reference tracks, to keep your ears fresh. Listen to how your mix compares with the professional's. The author repeats this throughout the book: It is possible to create stunning mixes in a small studio environment. Checking your work with professional mixes is crucial.

Part 2: Mix Preparation

It's okay to use pitch correction, rhythm correction (such as Ableton Live's time markers), recreate the bassline on a synthesizer and mix it in, or rearrange the band's song. Oh, yes indeed. They're paying you to produce good music, and if that means fixing their mistakes, and using smoke and mirrors, then so be it!

Consider the overall flow of the song. Each instance of the chorus could have a different intensity. Perhaps they'll get (perceptably) louder, except for the "drop chorus" at the end: Play the chorus once, with some instruments cut out, then play another chorus with everything driving the song home.

You may need to "mult" recorded tracks, which means splitting them up into separate layers, with different processing for each layer. The rhythm guitar should subtly slip into the background when the vocalist is singing, for example. Now that I'm aware of multing, I can't help but hear it used everywhere!

Part 3: Balance

Every layer will need a high-pass filter, probably no steeper than 18db/octave. Raise the filter frequency until the track feels like something is missing, then lower the frequency a bit. This will liberate the precious low-frequency spaces, so the kick drum and bass won't have to fight the other instruments in a bath of bass mud.

Now it's time to balance the levels. Mute every track. Slowly unmute the trocks, one at a time. Start with the most important section; start with the most important instrument. Adjust the track's volume so it fits with the mix, then don't touch it afterwards. After you've leveled every track, your mix should now sound pretty good.

Compression is primarily used to stabilize the balance of individual layers. Some tracks will have a steady balance without compression, and that's okay! Leave them alone. Other tracks may need one compressor to reduce momentary spikes in amplitude; another compressor to help even out the amplitudes of individual notes; etc. Try using parallel compression (aka "New York compression"), sending a track through a heavy compressor on a second channel, which will help even out the sound yet preserve its original character.

Expanders (and gates) are typically not essential. If they're used, put them before compressors in your effects chain. If your drum beat needs more punch, try sending the track to a parallel channel, gating it (to isolate just the attacks), and applying EQ and a little distortion before mixing it back in.

Equalization ("EQ") is a broad topic. Before delving into it, I'll disagree with one point up front: The book claims that any EQ plugin is about as good as any other. Most modern DAWs are equipped with plugins that provide low-pass, high-pass, shelving, peaking, and notch filters. My opinion is that some EQ plugins sound substantially better than others. I have never enjoyed the sound of Ableton Live's EQ 8, I feel that it sucks precious high-end frequencies and detail out of anything it touches. EQ 3 is better, although it's less flexible, and the filter curves are steeper, which is not always what you want. Shameless product placement: My new favorite is Tone2's FilterBank3.

To EQ the mix, mute every track, then unmute and EQ them in order of importance. You're trying to avoid frequency masking, a phenomenon where layers obscure each other. Avoid graphic EQ, and try shelving filters before using peaks. Use low "Q" values, so your filters will have wide, smooth curves.

Bass instruments: Try adding high-end brightness to the bass guitar. Try reducing the kick drum around 400hz, creating a hole for the bass guitar to punch through.

If a track's dynamics aren't doing a good job, apply EQ before the dynamics. (It's not uncommon to use both pre- and post-dynamics EQ, although this is more difficult to wrap one's head around.)

Linear EQ should generally be avoided, since it smears the clarity & definition of transients (momentary bursts of sound, such as drum attacks).

And, this chapter answered an old riddle for me: Why are we told to use EQ cuts, and avoid EQ boosts? Because when you apply EQ to a range of frequencies, that frequency range suffers phase alterations. This can rob sounds of their clarity. As you boost phase-altered frequencies, you're emphasizing muddier, out-of-phase frequencies. So a high-shelf cut (with a gain after it) really is preferable to the "equivalent" low-shelf boost. In a multi-mic environment, EQ boosts can be catastrophic, since instruments will leak into neighboring microphones with slight delays (i.e. different phases!), and may induce bad comb filter effects. Thank you, Mike Senior. I salute you for your clear explanation.

This surprised me: distortion is often used as a mixing tool! EQ can only modify frequencies that already exist, whereas distortion filters can add brightness and sparkle to the high-end. If your distortion plugin lacks a wet-dry control, the book suggests using parallel processing, by sending the track to an effects channel, and mixing it back in.

Multiband dynamics plugins have their uses, but are "mostly just an extension of what we've already covered in previous chapters." If you're clever, there are ways to achieve multiband dynamic effects without a dedicated plugin. For instance, if your bass guitar notes have uneven sustains, try extracting the lowest frequencies using a low-pass filter, and then compressing those, and mixing them back in. Watch out for a "hollow" quality, which indicates that phase artifacts are entering the mix.

Side-chained dynamics can be useful. The book explains "ducking," which is akin to extreme side-chained compression: When the vocalist is singing louder than a certain threshold, the guitar "ducks" a fixed amount. The brilliant thing is, if you don't have a ducking filter, you can create one using a side-chained gate: Send the guitar through a gate (side-chained to the vocals), and then invert the output. When gate opens, the inverted signal mixes with the original guitar channel and reduces the overall amplitude. Amazing. You can get creative, too: Try adding a linear-phase high-pass filter to the gate's output, so only the high frequencies duck out when the vocals are active. You get the idea.

Part 4: Sweetening to taste

Reverb, chorus, and delays cannot salvage a bad mix. However, they can sweeten a good mix, and make it great.

My big takeaway from the chapter on reverbs is that you can move instruments away from the foreground, into the background by sending them through a common "blend reverb." The reverb should have about 10-20ms of predelay. Reverbs should be CPU-intensive, as the better ones use lots of simulated reflections, and this requires more CPU power.

My big takeaway from the chapter on delays is that reverbs are overrated! Delays offer the same benefits as reverbs, but occupy less "space" in the mix.

The chapter on stereo enhancement is extensive, but the main point is that stereo width is more important than stereo panning. There are a number of ways to achieve stereo width effects using plugins (even rotary speaker plugins are discussed!). If the guitarist plays the same riff repeatedly, try hard-panning two instances of the riff, left and right. Try an M&S plugin, which splits a layer into the sum and stereo difference; pan the sum to the center, and apply all the crazy effects you want to the difference. Try adding some stereo "room tone," which can be a recording of tape hiss, background noise in a room, or the crackling of a vinyl record. Try duplicating the vocals, pitch shifting them a few cents apart, and panning them hard left & right.

Finally, there's a chapter on the "endgame," which is final compression, equalization, and automating individual notes & transients by hand. The big shocker for me was that a little hard clipping is acceptable. This is used in commercial music, and it can add a little sparkle to the sound. That's wild.

Definitely, definitely reference your mix with other people's tracks. Try sitting in silence for a minute, and hearing the song in your mind. Now play your mix; does it sound like what's in your head?

The book recommends a myriad of useful plugins, including shareware and freeware! The pages are also sprinkled with creative advice from professional producers.

Really, if you produce music at home, you're insane if you skip this book. It really is that enlightening. Get it! Use it! Everyone deserves to make high-quality music, and it can be done on the cheap with today's technology.

More »

I paid $99 for a Midines in the year 2008, and... I have still not received it. I have sent Chris Kann at least a dozen emails, and never received a single reply. In 2008, I did track him down on IRC -- he mentioned that he was going through some hard times, but now it is 3 years later, and he has been completely silent.

I wondered if I offended him personally, but it turns out that many, many, many, many (etc) people have sent money, and received nothing. A critical mass of people complained to PayPal, who finally shut down his account, so he switched to using Google Cart Service this year.

If I receive a Midines, then I'll be happy to give it a proper review! In the meantime, I urge everyone to use their own research & judgment before sending money to strangers.

Meanwhile, whether you own a Midines or not, you can make NES-style chiptunes for free. I created an Ableton Live Pack with all 5 NES waveforms: Download it here. You can also download the raw AIFFs. Here's a silly track that demonstrates how it sounds:

NES Instruments by Too Many Moths

The 5 waveforms are: 1-bit noise (the drum beat), a 4-bit triangle wave (the bass), and pulse waves at three different widths: 50%, 25%, and 12.5%. Here's how the pulses look, next to the noise:

The triangle wave is a bit tricky. The NES provides 4-bit volume control for the other channels, but the triangle wave always plays at maximum volume. In my Live Pack, I generated a triangle wave using Operator, downsampled it using Redux, and then filtered the high frequencies to curb the Redux artifacts. This was a subjective process, I tweaked the filter until it sounded correct.

I recommend altering the instrument envelopes, to create smooth attacks and gradual delays. Also, try using vibrato and tremelo (changing the pitch or amplitude with an LFO). This will improve the character of the instruments immensely.

If you want a more authentic sound, try limiting yourself to 2 pulse channels at a time, since that's all the NES can support. You can also send the instruments through convolution reverb, using a small speaker impulse. This may sound more like an 1980's television speaker. Enjoy!

More »

Try it. Click to generate new attractors.

The attractor coefficients are still chosen randomly. But now, attractors that explode/collapse are rejected. Also, attractors that create "boring" shapes (by drawing the same pixels repeatedly) are discarded. It's a little slow, but I'm sure the speed could be improved using Pixel Bender.

Also, here's the source code. (Compile with Flash CS5.)

More »

Watch: regular, or HD version. (Vimeo, 4:00)

The clips have a bias towards art and animation, because: 1). It shows well in a video, 2). This reel was created for an application to an artist's colony. (More on that later.)

The intro & outro animations (the flying moths) were created specifically for the reel. They're my favorite clips, yet they took less than a day to assemble. The moths were left-over from an old project (you'll see an experimental Too Many Moths interface within the video), and I felt the convergence motion was a strong opener.

The moths were composited in Flash. They were arranged semi-randomly, then the image was "baked" onto their wings & body. I flew them backwards, and recorded their positions on each frame. Then I reset the moths and flew them forward, again tracking their flight paths.

Actually, this was done in memory, so it was very quick. My laptop needed 3 seconds to arrange, bake, & move 120 moths. Then the harness presented me with the frozen frame, so I could evaluate how well the moths covered the images. If the coverage was bad, I hit a key to build new moths from scratch.

When I found a nice arrangement, I hit another key which stepped through the animation, and rendered each frame at a ridiculously high resolution (2160x1440, to cope with scaling problems in Adobe Premiere, unfortunately). The frames were encoded as .PNG's and uploaded to a PHP script running on localhost, which saved the 200+ images to disk. The intro took about 20 minutes to render & export. The outro, being more complicated, took over an hour.

Looking at the reel, I'm happy to present so many types of work — animation, audio programming, visual design, interactives, games, kiosks, websites. These four years have been amazing.

More »

• Import AIFF files
• Audition and edit formant sequences in real time
• Export .syx files for the Yamaha FS1R

Click to launch. Press the space bar to play the sound.

You can push your formant sequence to the Yamaha FS1R, using software such as K_Take's FS1R Editor. Click the "Save .syx" button, and follow the instructions in K_Take's documentation. This is a lot of fun, and breathes new life into the FS1R.

This project became much deeper than anticipated! The code includes FFT analysis (thanks Gerry Beauregard), pitch detection, a formant detection algorithm, and an AIFF parser to read AIFF files. The interface was a challenge to design and implement, and there are still many unfinished features (see the issues list).

My energy is shifting to other work, so I'll enhance fseq-flash when time permits.

More »

The Korg Radias synthesizer was released in 2006, discontinued after a 4-year run, and seems to have polarized all its users. Some dismiss it as "digital crap", others hail it as a "future classic". Who is right? Are these people snobby analog purists? Is the other group Korg's minions, paid to drum up favor for this little silver beast?

I purchased the rack-mount model, the Radias-R, in 2006. I was relatively new to the world of "virtual analog" synthesis, and the sound of the Radias seduced me because it was so different. Think: bright, smooth, icy, crystalline, electrical.

There are many reasons to appreciate this synth. I'll summarize a few, because I want to skip to the heart of this post: The Radias contains 64 sampled waveforms ("SynthPCM"), and they're are a lovely, creative bunch, with lots of rich, cybernetic character. (Someone enjoyed designing these, I bet.) The comb filter is surprisingly useful and rich. Korg's free editing software is a joy to use.

Other features are likable, despite a few flaws: The 39 (!) knobs on the front panel don't rotate endlessly. The insert effects are impressive, but some of the best effects (the amp simulators) are restricted to monoaural output. The unison & stereo spread features are great, but you can't target them via the modulation matrix (yet you can twist those knobs in realtime).

There's also a snappy vocoder, which can vocode live input, or record input as "formant motion" and play it back. Unfortunately, recording is difficult, and must be triggered on the hardware. You can't scrub, or change the formant motion speed. The Radias can store 16 formant motions, but only 8 presets are provided. I hate to admit this, but most of them are tacky... (One of the presets speaks, "Yo! Everybody in the house, say 'Raaaaadias!'")

How Does It Sound?

The most important feature of the Radias -- which you will either love or hate -- is the character of its sound. My opinion is that anyone who describes the Radias as "deep" or "warm" is wrong, just plain wrong. The Radias is the opposite of warm, fuzzy analog. It is deeply cold, in an air-sanitizing sort of way. If iciness can be sexy, this is it.

The sound has a certain sonic distance, as if the phase of one speaker is inverted (but of course, it's not). The basic waveforms seem a bit sterile, even after summoning some analog trickery (detuning the oscillators, etc.) The sound is so smooth that it's practically transparent. This synth sits well in the background. With its bright character, I've been able to drop it into many mixes, and it magically sits well with minimal equalization. But my attempts to feature the Radias as the frontman have never felt satisfactory.

How did Korg craft this particular sound? I ran some experiments to figure this out. Here is my conclusion:

In most cases, the waveforms are not synthesized in realtime. They're pre-sampled.

The Formant provides a clue

The Radias has 9 oscillator types, including a Formant oscillator. (A formant is similar to a bandpassed sawtooth wave.) Here's the spectrum of a formant produced by the Radias, with the narrowest formant width available. (This spectrum was analyzed in Ableton Live.) This looks acceptable, you can see the prominent peak around 800hz:

However, when we increase the formant width, instead of producing a wider hump ... The peak splits into two peaks, with the same narrow widths as before:

By increasing the formant width further, the peaks move even farther apart:

Although the sound is pleasant, this is not a true formant. This behavior indicates that the Radias' "formant" is actually composed of two waveforms. These waveforms have been pre-sampled at various resonant peaks. The Radias produces its "formant" by playing two of these samples simultaneously.

The Sawtooth misbehaves

Another Radias oscillator type is the Sawtooth wave. When played at mid-range and high frequencies, the Radias delivers the expected sound -- a series of overtones marching up to infinity:

But something undesirable happens when we move down the keyboard. When playing lower notes, the high-end treble frequencies roll off:

Further down, it sounds even more muffled:

The Sawtooth waves are not being generated in realtime. They, too, are sampled. This sort of frequency roll-off implies that a sample is being played at a low frequency, and interpolated (a.k.a. "anti-aliased") to smooth the output. I'm not sure which window function Korg chose for the interpolation, but it is the driving force behind the Radias' character, as it filters every sound. Also note that the other fundamental waveforms (Square, Triangle and Sine) are sampled as well.

I suspect that waveform modulations (such as square wave's pulse width modulation) may be pre-sampled as well, since they exhibit the same roll-off. By sweeping the pulse width, the Radias may be interpolating between waveforms (like using wavetables). I can't seem to produce an audio demo to support this claim, so the wavetables may have hundreds of samples in them, too many for the human ear to discern.

However, the two effects based on frequency modulation ("Cross" and "VPM") do modulate in realtime. They produce appropriate spectrums, even when the oscillators run at low frequencies.

The Radias features a "morphing" filter, which can move between two filter shapes (for example, a lowpass and a highpass). This is probably achieved by processing the incoming signal with two "cardinal" filters (LPF24, LPF12, HPF, or BPF), and mixing their results together. I can't prove this, but the spaces between the cardinal points seem a little dull, as if phase artifacts are creeping into the treble, or the anti-aliasing artifacts are doubled.

These experiments helped me understand how the Radias acquired its sound. Short, single-cycle waveforms are looped and anti-aliased, resulting in smooth, hygenic timbres. Some of the longer samples (in the "SynthPCM" group) have more density, but they're still filtered by the anti-aliasing algorithm, which makes them sound airy and hollow. The Unison and Analog Detune features can add density, but in my experience, they don't add much thickness or richness, as it's merely playing multiple copies of the same smoothed waveform.

Hopefully this post doesn't read like Radias-bashing. I was more interested to discover Korg's tricks than anything else. The Radias is suited to some very specific genres and roles, and I wouldn't recommend it to everyone. Still, the sweet GUI editor makes this the most "fun", accessible synth in my collection, and I keep returning to the Radias for this reason alone.

To hear some creative Radias patches, check out Jexus' great YouTube demos: part 1, part 2.

More »

Filtered Noise Sequencer

Drag and resize the blue blocks to change the filter frequency and width.

This sequencer is not using expensive bandpass filters. The oscillators are sine waves, which are frequency modulated with white noise. It may not sound inherently musical, but you can produce great hihats, bass thuds, and airy pitched noises.

Here's the source code. (Requires Flash CS5 to compile.) Have fun!

More »

A Curious Lack of Light by tooManyMoths

More »

(click to launch)

The next version of Twang will record & save songs (this is done), and share them in some capacity (a bit more complicated). So that's where the Twang Player comes in. The Flash version looks like it wants to be touched & strummed. I need to revisit the design and convey that Twang Player is just a music box, you can't compose anything in the browser! ...Yet.

There are performance issues on some of my machines, too. This version uses Vector.<Number> objects to handle data, but it looks like ByteArray or even BitmapData structures are the way to go.

More »

First, do you know Lev Grossman? He's an incredibly talented author who recently toured Portland. If you haven't read his book The Magicians, then stop whatever you're doing and procure a copy immediately. Without trying to spoil anything, the major college in the book is named the Brakebills College for Magical Pedagogy. Lev saw the Brakebills T-shirt that I designed for my sweetheart's birthday present:

Long story short, my Brakebills shirts are now officially endorsed by the author! Please buy twenty of them, or thirty if the spirit moves you. Zazzle's print quality is great. The colors are vivid, and the ink has survived many washings. Rabid fans are buying and even customizing them! I love the internet.

Lev blogged some of my other work, too ("the guy who does this has the enviably fake-sounding name of Zach Archer"). It's true, I have an awesome pro wrestler name.

Second, my new iPhone app has landed in the App Store:

Twang is a handheld guitar. It's easier to play than a real guitar, and is very expressive. Instead of using audio samples, Twang uses physical modelling techniques to create a more natural, dynamic sound. No two plucks are identical. Watch my grainy first video if you disbelieve.

In the next version of Twang, left-handed people will be able to switch Twang's orientation, and serious musicians can dampen or mute strings with their fingers. And probably more! This version is already in development, and may be submitted in a week or two? Follow Control Z, Inc on Twitter if you have a ravenous thirst for updates!

More »

Here's something from the vaults. Aquasound was built with these requirements in mind:

• Generate sounds that aquatic animals might make
• Sounds can be "combined" somehow
• Sounds can emote

This was never used in production. I wonder if I could turn this into something? Like a paid iPhone app? ;)

Double-click the envelopes to add/remove control points. Drag lines up & down to change their curviture. The best feature is the "Combine With" dropdown, which splices the current sound with your selection. Also the "Emote" menu will play sounds with different expression.

The audio algorithm is reverse-engineered from my beloved FS1R. I generated formants in two ways (toggle the "Tonal" checkbox to hear both), the "atonal" version is closer to ring modulation than actual formants. It's more fun if you don't understand what the controls are doing, but if you insist: Pitch controls the overall pitch of the sound. Freq controls the center frequency of the formant (like a bandpass filter). LFOFreq and LFOWeight control a low-frequency sine wave, which can be applied to other controls via their "___LFOAmt" curves. Amp is amplitude, Width is formant width (think: width of the bandpass filter), Skirt adds distortion. Each voice has two formant generators, check "Formant Active" to enable them.

May all your bloops and crackles be happy ones!

More »

Here's my second batch of image reconstruction using Times New Roman characters. The algorithm is a brute-force affair: New characters are colored & positioned randomly. Any characters that make the canvas look more like the original image are saved. And that's about it. Oh, and the font sizes start large (5120pt) and end small (10pt), so that fine details have a chance of survival.

Time lapse of the Mona Lisa reconstruction on YouTube, with silly music.

My first batch uses a different algorithm. Each canvas allocates a certain quantity of letters, and progressively mutates them, trying to mimic the original image as closely as possible. This technique is more akin to image compression. This batch is still in progress, it's very slow. I'll post these when they're ready!

More »

This took about 5 weeks to develop. Meanwhile, I've started another app, and I envision releasing Metal Mouth 2.0 in a few months, with more voices, and the ability to record audio.

More »

I spent a couple hours generating strange attractors in Flash, just a simple 2D version for now. Click to play:

Click the black region to generate new polynomial coefficients and redraw. You will have to click many times to generate something interesting. That's the nature of fractals, I'm afraid. Some coefficients are automatically thrown out if the drawing exceeds a certain size. Unfortunately, the inverse is not true: the code isn't smart enough to trash any drawings that shrink to microscopic size.

I believe that you can stabilize any coefficients by scaling the values of each coefficient, gradually nudging them larger/smaller until the drawing is stable. I'll try this when I get more time. I've been gung-ho on my first proper iPhone app, trying to finish it before Christmas! Stay tuned...

Also, here's the source code for the strange attractor harness! Enjoy.

More »

Today I noticed that if you right-click on yesterday's SWF and show the redraw regions, you can see that it's redrawing the contents of the entire stage, even though I put the scene in a scrollRect. Is it seriously rendering a scene that's thousands of pixels wide before displaying it ?!?!?!? Oh, no. No they DIDN'T.

Today I ported the scene to the Away3D rendering engine. Here's the result:

It's beautiful, and provides access to low-level drawing routines, light sources, normal maps, ... It was speedy at first, then slowed down considerably when I added the glowing floors. (Each glow is 16+ triangles right now, for various reasons including: I can't render objects in my own custom order.) This makes yesterday's version look performant, I'm reluctant to admit.

Possible next steps:

• Reduce the native 3D rendering area, see if performance improves?
• Grow beyond Flash, embrace the future and try Unity 3D?
• Dump this project, finish that iPhone game I started, make a million dollars in 2 weeks?

More »

Click to set the focus. Use the arrow keys to move. Touch blocks to illuminate.

I'm disappointed with two things:

1). How much time is required to create a 3D engine, even a grid-based one like mine. I've been wrestling this project for 4+ hours every day, for a week. I feel like I must be lagging behind, but there are ten thousand things that will go wrong when developing in 3D. The paradigm is uniquely punishing, there are always edge cases where some polygons aren't drawn correctly. This project hasn't been a joy.

Also:

2). Flash's native 3D is not suited for a high-performance application like this one. It would be fine if I was only spinning a few DisplayObjects in space. However, the scene above displays up to 125 Bitmaps simultaneously. (Light all 25 bulbs (3 Bitmaps each), stand in the corner facing them, and the 25-segment walls.) 125 Bitmaps would be child's play in OpenGL. But after you light a few blocks, Flash Player chokes pretty hard.

Here's another version that uses a BlendMode on the lightbulbs. It looks great, but its performance is even less acceptable.

Here's an early version that uses my own 3D computations, and the Graphics API. Also it has a limited field of view, which I widened for the latest builds. The performance is surprisingly high. I abandoned my custom 3D when I reached this point; drawing lines around each cube face was expensive, so I switched to Bitmaps, and the native 3D.

The cube faces are set to width & height of 100. However, the bitmaps are higher resolution, a 200x200 region is shown. They're being downsampled at 100x100 before they're rendered, not by my choice.

At runtime, I get periodic warnings like these:

Warning: 3D DisplayObject will not render. Its dimensions (8238, 1628) are too large to be drawn.

What?! How is this happening? I swear that any blocks behind the camera are being removed from the Stage. (Actually, this is difficult to verify. If I shrink the scene, Flash magically applies the 3D perspective with a weird projection, and distorts everything in Lovecraftian dimensions.) Please, Adobe, tell me that you're not rendering the scene at 8000 pixels wide, then scaling it down to my 700x400 window, frame after frame?

Also note that you, the developer, are responsible for drawing the DisplayObjects in the correct depth order (farthest to nearest), Flash doesn't handle it automatically. This is known as "2.5D", and it's wildly inconvenient.

So, I'm pretty disappointed with Flash 10's native 3D. Even with my limited 3D experience, I dislike how it renders the scene (I'm not alone in this) and the performance is obviously sub-par. This technology will not bring 3D games to the web, it cannot.

I need to decide whether to endure its shortcomings for 4 more weeks, or if I should abandon this project altogether. There are moments when you realize you've outgrown something you used to love, and this may be one of mine.

More »

Here's my first attempt at LPC speech in Flash: (click & explore)

It's great, except for one tiny problem: It sounds horrific. Can you feel the cold, robotic love? This voice will stalk your nightmares.

The phonemes were derived from an unrehearsed recording of my voice. I'm confident that it can be improved. Note that direct LPC encodings of my voice, such as this one, sound more acceptable.

EDIT: I made an iPhone version, "Metal Mouth", with lots of features. Here it is on YouTube and the iTunes Store!

EDIT #2: The source code is available here.

More »

(You will need Flash Player 10.)

Ten days have elapsed since I shared this version with friends (who are unwittingly being used as play testers). It's time for a "wrap-up" meeting, because I want to examine what went wrong/right, and instead of a meeting it's a blog post:

The tile engine has existed for half a year. Toaster Bro began its life as "Roomhack", one of the pet projects Miles and I dreamed up -- could we create a game that was Turing complete? In the tradition of Robot Odyssey, we wanted to create an experience that was both programmable and entertaining.

During the development of Space Kitty, we used its script engine to change the rules of the game at runtime. Once, we configured a cat to act as an "attack cat", which chased (and kicked) the player. We discovered new modes of gameplay -- how long can you say aloft? Can you slingshot yourself around a dense star? (Ultimately, we ran out of time before we could make the Kitty-verse bigger. We're currently discussing a Space Kitty sequel.)

The script engine whet our appetite -- what if the player could modify the rules of the game itself? What happens to the gameplay? Can it remain challenging and fun? We talked about this for days, weeks... Hashing out ideas for an "ultimate" hacker game. I think we're still talking about it.

I imagined such a game would resemble a dark, apocalyptic 3/4 view basement crawl, reminiscent of:

So I built the tile engine. The plot & aesthetic of the game were still unknown. Tavvv challenged me to invent a "missing" console from the NES-SNES era, devise hardware specs for it, and develop the game in accordance to the hardware specs. Constraints fuel creativity, right? I settled on SNES-era hardware, with a custom color engine, and improved sound. Television artifacts & blur could be exaggerated to make the game creepier; bosses would have extra blur and distortions, so you couldn't get a clear image of them. The lurking horror. This concept was eventually dropped, once I realized that I'm not ready to develop a Turing-complete game... yet. I'll come back to this project in the future, I'm sure.

"The tile game" evolved into a shooter, much like Smash TV but with a heavy-handed political bent. That still didn't feel right, even with "a spider-tank with Richard Nixon's head" in tow. In the middle of this confusion, we realized that putting hackable terminals in a tile-based game detracted from the appeal of both. Is there a way to allow hacking, within the context of a tile engine? From this fray emerged the Toaster Bro mechanic. I rolled a quick proof-of-concept. It seemed like we were on to something.

Then the game sat in stasis for a few months, while I worked and tended to life. I wanted to finish one solid level, and I imagined a 4-week timeline:

• Week 1: Engine Malarky. Moving between rooms, easier wire drawing, electricity & fire shouldn't hurt you, remove crufty code.
• Week 2: Levels & Art.. Finalize the plot & characters, somewhat. Replace stand-in art with "real" art. On-screen text. Create a bunch of rooms.
• Week 3: Audio, splash screen, boss fight, begin testing.
• Week 4: Music, ???

This seemed like a good plan. Work began, but here's how things really went down:

Week 1: Plugged lots of holes in the engine, but a single week wasn't enough to tackle everything. There are major structural implications when you modify your game's core. Still no way to change blocks at runtime, yet. I need a scripting engine, but I want one with infix notation (Space Kitty used various postfix notation languages, before Miles settled on an implementation of the PostScript language I think? It made my brain bleed.) I didn't allot time for creating a scripting engine, but clearly there was a need for it.

Week 2: I'm not sure how the 8-bit NES aesthetic won out, but I liked it because it entailed fewer artistic decisions. The aesthetic had an entire era of gaming backing it. This seemed like a wise choice, because I'm not an accomplished artist, nor designer. I'm partially colorblind, in fact. Could I still cobble together pixel art?

I bought a Wacom tablet and books on animation, hoping to produce an acceptable look & feel. By the way, nearly every graphic in the game has been redrawn at least once. You should have seen the original sideways animation for Toaster Bro, it took a few tries to make him look non-crippled.

So I had a decent lead on the art, but bugs kept popping up. Fundamental pieces of the puzzle were still missing (no scripting engine, no on-screen text, player can't die).

Week 3: I caught a bad cold, so I spent most of this week supine on the couch. I forced the "game" on a few people, and collected their feedback. All the responses were valuable, and people raised issues that I hadn't considered. One tester said he couldn't view the entire window, he was on a MacBook with its standard (small) screen. Oops! Sorry about that... I reduced the console's "pixel" size.

The upshot of being sick was having lots of quiet time, during which I (slowly, groggily) puzzled out the scripting engine. Here's what the scripting language looks like now, this is extracted from the first room on the Dark Chocolate level:

function clubDoor_electricity {
	i.isDoorOpen = true;
	call o.setBlock '__' 7 4; // empty space
	call o.setBlock '__' 7 5;
	call o.playSound 'doorOpen';
	call o.emotePlayer 'happy' 75;
}
function clubDoor_stop {
	if(i.isDoorOpen == true) {
		i.isDoorOpen = false;
		call o.setBlock '$L' 7 4; // lock
		call o.setBlock '$L' 7 5;
		call o.playSound 'doorClose';
		call o.emotePlayer 'sad' 75;
	}
}

The interpreter supports a large syntax, including: loops, switch/case statements, tons of operators, enterFrame yields, userInput yields, recursion ... You get the idea. Variables are always referenced through a few specific objects, right now these are: thread, interpreter, owner (this is the Room that created the Interpreter instance), level, and global (or game). (These are all dynamic objects except for owner.) The script engine is decoupled from the code, I wrote a harness for it.

Also the 8-bit NTSC artifacts happened during week 3, somehow.

At this point, the game had a mere 3 rooms, all unsightly geometric slogs used for testing. Uh oh.

Week 4: Still a bit sick, and my real-world work schedule poked its head in, and needed some attention. The list of bugs & issues was huge; 30 items, including toughies like art and program flow.

I forced myself to draw the dreaded sprite animations. Infrequently, this was relaxing and zen-like. But in the wrong state of mind, it was taxing and disheartening. I'm using the palette swap trick from Space Kitty, so I stared at grotesque red, green and blue images all day, and felt sorry for myself. Definitely out of my league.

Also I hacked together a level editor using Flex components, accessible through a magic keystroke. It's buggy and non-intuitive, but it's better than editing the room layouts as raw ascii! Here's how the final room in Mom's Kitchen is stored:

[P,d [P,d [P,d [P,d [P,d [f,d ..,d ..,d ..,d ..,d [d,d [P,d [P,d [P,d [P,d [P,d
[P,d [P,d [P,d [P,d [P,d [f,d ..,d ..,a ..,a ..,d [d,d [P,d [P,d [P,d [P,d [P,d
[j,d [P,d [P,d [P,d [P,d [F,d ..,d ..,a ..,a ..,d [D,d [P,d [P,d [P,d [P,d [k,d
[J,d [h,d [h,d [h,d [h,d [i,d ..,d ..,d ..,d ..,d [g,d [h,d [h,d [h,d [h,d [K,d
[f,d ..;n ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;p [d,d
[f,d ..;q ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d ..;q [d,d
[f,d ..;q ..,d ..,a ..,a ..,a #a,a #a,a #a,a #a,a ..,a ..,a ..,a ..,d ..;q [d,d
[f,d ..;q ..,d ..,a ..,a ..,a #a,a #a,a #t,a #a,a ..,a ..,a ..,a ..,d ..;q [d,d
[f,d ..;q ..,d ..,a ..,a ..,a __,a ..,a ..,a ..,a ..,a ..,a ..,a ..,d ..;q [d,d
[F,d ..;q __,d __,d __,d __,d ..,d ..,d ..,d ..,d ..,d ..,d ..,d __,d ..;q [D,d
[i,d ..;r ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;o ..;s [g,d

I resolved to ride my bike every day, to avoid getting sick again. So far so good, but now I ride to coffee shops and spend a metric crapload of money dining out of the house. The hidden price of wellness...

Week 5: The list grew even longer. Lots of bugs, lots of missing features that I didn't consider -- the heart containers, the health bar. Dark Chocolate's level was half done. The entity behaviors were unstable; the damage system was finicky, every Entity required too much configuration to properly interact with others. This needed an overhaul. I'd started thinking about unit testing, which I'd never tried before.

I drew Dark Chocolate, but with no behavioral code, he didn't move. Fun fact: I saw his animation in a dream.

Week 6: Dark Chocolate's level was finished, mostly. The boss fight took a day and a half to get right. You still couldn't "win" a level. I always joked that the goal was to plug in a toaster. Hmmmmm...

Weeks 7-9: Sound design, on-screen text, Mom's kitchen (the training level), giant toasters. Redrawing sprites. Bigger croissants (the laser-guards). Spotlights. Refactoring lots of code. The scripting engine is versatile, but not easy to use. In week 7, it was much clunkier than the syntax listed above. There were no switch statements, you had to write a list of consecutive if statements instead.

Days passed. The spit 'n polish phase took forever. I sympathize with whoever said, "your code is always 85% done". It sure felt that way.

Also I finally implemented unit testing. I tried ASUnit first, but it's not very Flash-like; I believe it's a direct port of JUnit (as evidenced by the Java code in its comments). If I'd realized FlexUnit was still in development, then I would have jumped on it, but... Long story short: I wanted to display failed tests on the stage, and ASUnit made that difficult. So I threw out the ASUnit code, kept my custom TestCase subclasses, and wrote my own Flash-centric versions of a TestRunner and TestCase (they both extend Sprite).

I can compile Toaster Bro in two ways: I have a Main.as class which will compile in the Flash IDE. But there's also a Roomhack.mxml wrapper which creates a "developer" build, integrating the testing framework and level editor. When the developer build is launched, it runs every TestCase. If a TestCase fails, and if it has added children, then it's displayed on the stage, with diagnostic messages & an execution trace. Failed TestCases can be dismissed with a click. I'm not "in love" with unit testing yet, perhaps because writing the tests after the code was tedious. I'm sure it will save my butt in the future, tho.

Week 10: Toaster Bro alpha is unleashed on the world!

Overall, feedback has been positive. Like Space Kitty, most people want the game to be longer. The negative feedback has been very helpful, pointing out specific problems which aren't obvious to me, since I view the application as a whole now.

So, after writing this gigantic treatise, I figured I'd have some insights on how to improve the process. And you know what? I don't.

Gamasutra had an excellent article on rapid prototyping, and their process echoes what we experienced on Toaster Bro and Space Kitty: Creativity can't be scheduled, and you'll spend a huge chunk of time germinating the idea in your head. We had 2 months to develop Space Kitty; we spent almost exactly 1 month dreaming up the idea. After that, the initial prototype took a day or two, and we were rolling.

With Toaster Bro, nearly half of the "serious" development time went towards polishing the game and making it presentable. This was a surprise, even though most of my interactive work has followed this pattern, too. The functional code is fairly easy to develop, but then you must meet the clients' standards, and hammer the UI to find the rough spots.

Sound effects gave Toaster Bro an air of legitimacy. The difference was like night and day. Without sound effects, it was a shaky graphics demo; with sound effects, it felt alive. Hard to explain unless you've witnessed it.

Testers will comment on all the broken windows they find. I still hear about the missing sound effects in Toaster Bro (and yes, there are plenty!) But that's okay, their feedback is always valuable. Occasionally they'll have solid ideas on how best to fix a broken window, too.

What happens now? I have to produce more levels, art, and script. I'd like to believe that my productivity will ramp up, with so much of the infrastructure and assets completed. Sadly, I don't think it works that way. I have big ideas for the remaining Toaster Bro levels, and these involve intense scripting. The second room of Mom's Kitchen (where Toaster Bro learns how to use a wire, via 4 simple dialogs) is 131 lines of script, and it's still not done -- Mom should rush in and cheer after you activate the toaster, right? I can only imagine the length of the equivalent code in AS3. I want big effects & easter eggs. It will be difficult to develop and test these.

Whatever skills you have, I recommend bringing these into your projects, and emphasizing them, even leaning on them. I produce groovy electronic music, so I made a bunch of chiptunes early on, to help center my thinking. Some aren't useable for Toaster Bro, they're too active -- they have the pace of Mega Man, rather than Zelda. It was good to realize that, it helped shape my thinking.

More to come?...

More »