Watch a video of our performance.

I was lucky enough to receive a Leap Motion Controller. It acts like a short-range Kinect for your hands, tracking the position of each individual finger. The Leap’s sensors are fast, and spookily accurate. I love it.

For my first “real” Leap project, I planned to use the device on stage, and sculpt a block of virtual clay. Initially I was targeting Flash + Stage3D (aka OpenGL), using a Python script to grab the Leap data and send it to Flash through a socket.

In the end, I switched to Cinder, on the assumption that its performance would be superior. In retrospect, Flash + Stage3D would have performed fine. Even in Cinder, my laptop only managed 15-20 FPS. Instead of switching to the native C++ Leap library, I kept the (slower?) Python socket, which may have been unwise. It was also my first Cinder project, so progress was slow.

Still, I regret nothing!!! The performance was a blast. A momentary “ooooohhh” passed through the crowd when they saw the Leap in action. I was concerned about stage lights interfering with the Leap’s sensors, but the Leap was unaffected by the lights.

For the surface of the clay, I’m storing a heightmap in memory. I treat the tips of the fingers as spheres. When the spheres move below the heightmap, some of the height is distributed to neighboring cells. Then the clay is drawn as a triangle mesh. Overall, the effect is a bit rough, but it definitely conveys the feeling of sculpting solid material.

Download the source code. I included a few sample images and videos to get things rolling:

1. Copy the Leap Python SDK files to the /python folder. (Unfortunately, I cannot provide these files. Registered Leap developers can download them from the Leap website.)
2. Start the Python server: ./python/server.py
3. Open and run the project in Xcode: xcode/Clay.xcodeproj
4. Keys: 1 starts the next background video, ` stops all video. 2 toggles silly matrix transforms. 3 toggles liquid mode. 4 resets the surface of the clay. 0 selects a random image, [ and ] display the prev/next image.

[Cross-posted to dorkbotpdx.org!]

(Here’s the .swf version.) There was a lot of nice feedback. Random people would spot it and exclaim, “That’s awesome!”

A secret view from inside the lab!

Things I learned from this experience:

Keep it obvious. The tiny bright spot in the corner of the video is an LED sign displaying a phone number. Anyone who called was treated to this soundscape:

Guess how many people called? … Zero! By my estimation, the typical art-walker gazed at the Math+Heart graphics for a few seconds, hardly enough time to notice the tiny phone number.

A different approach would be a phone-centric interactive, such as Mall of America: A Toll Free Audio Exhibition. Their phone number is displayed in massive yellow lettering! The back-end could be developed with Twilio, or a similar group that provides interactive phone service.

Don’t expect 60 FPS from Flash’s display list. Ultimately, I did achieve 60 FPS on one of my computers (with the occasional dropped frame, alas) but I invested an unhappy amount of time, fiddling & optimizing, to reach that point. (Flash’s OpenGL graphics, a.k.a. Stage3D, are more performant. But I didn’t attempt this, for fear of texture memory issues.)

Motion design tips. The mathematical glyphs are essentially large particles, which enter at random speeds. This “swarm” felt best when the random speeds ranged between 50% and 100% of some maximum value. Any ranges wider than 50% broke the cohesive effect, and felt sloppy.

I experimented with randomizing the easing equations, for instance the rotation could twist at either an exponential or quartic rate. This was surprisingly frustrating to watch. The glyphs behave almost like a unified fluid, so it hurts when they move according to different rules.

Also bad: Using different tween times & delays for the four properties: X, Y, scale, and rotation. I tried using tweens to look like a phantom hand was hurriedly slapping the glyphs down on the canvas. This felt weaker than the smooth, unified motion that was used in the final version.

To achieve movement that was both organic and emotive, I could have tried filming myself acting out the motion with slips of paper, then traced that motion. (Or developed rules to mimic that motion. It could work! Maybe!)

Simple variations can be very effective. The space packing algorithm has two modes. It switches between packing space that’s visible on the screen, and packing a larger space including a region below the bottom edge. When it switches to the second mode, it suddenly has a tall, uncluttered space to fill. The result is a texture that constantly varies between tight square-shaped glyphs, and taller ones. The contrasting modes look good!

From top to bottom:

1). Rainbow: Spectral analysis of the track. I’m using Balkansky & Loop Stepwalker – Cicatriz for this example, because it’s noisy and highly compressed, presenting a good challenge to the algorithm.

2). Red: The differences between adjacent spectral frames. This should help isolate the moments when drum sounds (transients) pop. Pseudo-code: abs( frame[x][y] – frame[x-1][y] )

3). Cyan: The total energy of each frame in (2), with heavier weights applied to higher frequencies. This attempts to compensate for equal-loudness effects: Higher frequencies are typically perceived as being “louder” than lower frequencies at the same amplitude. So, most music is mixed with higher frequencies having less spectral energy. (You can see this in layer (1): The spectral analysis shows the bass range (near the top) is much brighter than the treble).

4). Blurry cyan: Layer (3) is smeared using convolution. This smooths out any spectral artifacts and transients.

5). Yellow: Layer (4) is subtracted from layer (3). Only positive values are lit. Energetic spikes stand out amidst the surrounding noise of their neighbors.

6). Magenta: Searches for the greatest energies in layer (5). These are the beats! Frames that surround the beats are zeroed-out, ineligible for beat detection. The search is repeated until all beats above 10% of the maximum spike’s energy are identified. Congratulations! You have found the beats.

Now the trick is to determine the tempo and rhythm. My solution was to begin at the strongest beat. Then measure the distance forward to the next beat. Using this distance, repeatedly step forward through the landscape of beats, and compare the two “measures” to see if the rhythms correlate.

Then run this again with a distance of 2 beats forward; then 3 beats; etc.

In this enthusiastic screenshot, a very strong rhythm has emerged when the program steps a distance of 9 beats forward (or backward) from the strongest beat. The rhythm of “Cicatriz” is largely composed of 8 quick hihats (sixteenth notes), followed by 1 big half note snare, so this result seems to make sense.

And there it is! You have found the rhythm. Divide the sampling rate (44100) by this distance to get the tempo. Actually, the resulting number will be very small; multiply by 16, 32, 64… until a reasonable BPM emerges. (“Cicatriz” is 70bpm.)

Source code is here. This proof of concept was developed in Flash, which is a decent tool for rapid prototyping, and it comes with a huge library of classes out of the box. This comes with absolutely no warranty! I recommend not using this in mission-critical situations. The Queen Mary plug-in set may be a more robust choice.

The YouTube trailer would probably look something like this:

I hacked Angel Harp together in my spare time. Four long months! The plan was to finish by Halloween of 2011, but it took considerably longer than expected. The synthesis was completed in one week, the sound effects in another week. Standing on the shoulders of Twang, Angel Harp produces somewhat-realistic tones (like an actual harp! Complex filtering!) And it has 3+ dozen strings, for serious plucking power!

And, the graphics… Let’s talk about that.

Once the Halloween deadline became improbable, I decided to hack each feature until it was “good enough.” If any feature became an eyesore, then I’d revisit it — either for version 1.0, or a future release. The clouds were redone a couple times. I had grand plans for the harp itself, using an (awful, buggy) harp modeling tool; in a future version, you can draw your own harps, and skin them with fancy materials, I think.

How do you like those smiles? The facial expressions were sketched in 30 seconds, all of them. In my mind, the enormous half-circle grins and grimaces were very temporary. However, as friends were subjected to the ‘Harp prototypes, it became clear that people loved the primitive scribbled angels, and they wanted to keep them. Beware the things you bring to light.

I have mixed feelings about the facial expressions. Actually, I love them. They’re so happy. It’s almost a parody of top-selling app mascots; googly-eyed characters, managing diners and flying over the outback, are positively leering. Like they haven’t tasted human flesh for millennia, and then suddenly YOU show up.

I worry that prospective buyers won’t risk a couple bucks on Angel Harp, because it might look a bit hastily designed. (Even though it feels reactive and expressive, the screenshots can’t communicate that.)

Angel Harp was a conscious attempt to entertain all ages, and still deliver something unexpected and quirky. Everyone who interacts with Angel Harp loves it, but it’s a big, intimidating knot of weirdness to figure out. Is it an instrument? Why are there angels? Why do they squeal so much?

Frustrating admission: I must create apps that are more familiar to the average person (or a certain subgroup of insane musical geniuses).

Not-so-frustrating admission: I’m proud of the Devil Harp’s carousel of effects. The reversed audio, the tube amp distortion, and the fuzz box. Don some headphones, and lose yourself in the Devil Harp’s ambient spookiness. It’s nice.

Career-building achievement: I greatly expanded my OpenGL knowledge.

True tales of bits and bytes: The sound effects are stored in one continuous file, which sounds like this, and can cause hysteria:

Culinary trivia: Coffee is a wonderful drug. It powered both recording sessions. My arms flapped wildly, as I voiced every angel and demon.

Creepy outtake: Angel Harp mostly “designed itself,” but an early version of the Hell world was genuinely upsetting:

Will there be more Control Z apps in the future: I expect so! There’s one music app that I’m itching to create, but it’s a massive undertaking. Work, play, life, action, rest… I’m still finding the right balance. We shall see.

Is it better to create simple apps, which have one function: Almost certainly. I always want to go beyond the limits, and tinker with new, unprofitable things. This is my curse. If you’ve figured out how to manage this affliction, let me know.

View Angel Harp at the App Store!

Please note: This code comes with no warranty, nor support, whatsoever. None. Zip. Nada. If your talking robots become self-aware and enslave humanity, then I will not be held responsible. But if you’re in the mood for tinkering, here’s how it’s strung together:

• The sound is generating using Linear Predictive Coding (“LPC”).
• First, some Python code: The analyze_lpc.py script analyzes phonemes.dat (which is just a headerless version of phonemes.aif). Individual phonemes are separated by moments of silence, so the script splits the sound file on those. Each phoneme is converted to LPC data, using code that I ported from the rt_lpc project. I felt like I understood the mathematics 3 years ago, but I doubt I could explain it today.
• Now, in Flash: Launch the DictCompressor application, and watch the trace messages. Click the screen to open the browser window, then select your cmudict___.txt pronouncing dictionary. (You can obtain the latest CMUdict here.) Flash will convert this to a (smaller) cmudict.dat file, which is what LPCsynth.swf loads.
• LPCsynth is the application that talks. The LPCSynthHarness.sayItNow() method creates an array of LPCFrames, which are “spoken” in the sampleData() method.  This was never intended for public distribution, so the code is not exactly stellar (the talking bit should be extracted into its own class).

Is this interesting? Did your Flash Player become self-aware after hearing its own voice? Let me know!

It’s funny, originally this was intended for the controlzinc.com website. The robot voice would sing as you clicked, crooning about your mousing habits. I still can’t decide if that idea was brilliant, or terrible.

If only Mixing Secrets For The Small Studio had existed 10 years ago, my music would have been impeccable! (Well, I like to think so.) This book is a magical tome for anyone who records or produces music on a budget. It’s packed with big reveals, and explains the science behind each mixing technique. Forget the accumulated hit-or-miss wisdom of the internet; after reading this book, I found that I could produce substantially better mixes immediately. That’s amazing. (My mixes still aren’t great, but I’m working on it!)

Here are my favorite takeaways from the 20 chapters. I’m writing this to lock these concepts in my head. I’m skimming lots of material, because there’s so much valuable information packed into this book, I can’t possibly recap all of it.

Part 1: Hearing and Listening

My Rokit KRK-8 studio monitors are distorting my mixes. They’re “ported,” meaning they have an open-air port cut into the body. The manufacturer claims that this helps the air move freely. As a consequence, this creates noisy air turbulence, and muddies the bass frequencies. That’s bad. The book suggests plugging the ports with socks; the sound might become clearer! Try it, listen carefully, trust your ears…

The goal of soundproofing your studio is to tame reverberant frequencies. You don’t want an echoless space. Also, stereo is overrated, try mixing in mono. If this advice sounds heretical, consider the typical consumer’s listening environment: Computer speakers, stuffed in a bookcase, clear across the room. Car speakers, blaring inside a noisy, reflective space. Restaurant speakers, bolted to the ceiling in semi-random locations. Your mix should sound good in mono, because most people will never give it a proper stereo listening. And there will always be room reflections in any listening space, so don’t completely Nerf your room. Allow your walls to sweeten the sound.

Headphones are a special exception. The book cautions that some headphones have inadequate bass response (as do some studio monitors, of course). Music may sound oddly “clean” without the natural reflections of a room. Personally, I love modern headphones. I bought a pair of Beyerdynamic DT 770 headphones this summer. It was the single best musical purchase I’ve ever made. They have clear, deep bass response, which rattles my skull a bit. The treble is impressive, and it reaches very high into the range of human hearing. I can finally hear the full audio spectrum (unlike running my Rokit KRK-8′s through a Mackie mixer), so the headphones have been a godsend for me. I can’t imagine returning to my KRK-8′s in my oddly-shaped studio space.

The biggest takeaway from this section is learning how to use other songs as reference material. Create a workflow where you can rapidly switch between your mix, and several reference tracks, to keep your ears fresh. Listen to how your mix compares with the professional’s. The author repeats this throughout the book: It is possible to create stunning mixes in a small studio environment. Checking your work with professional mixes is crucial.

Part 2: Mix Preparation

It’s okay to use pitch correction, rhythm correction (such as Ableton Live’s time markers), recreate the bassline on a synthesizer and mix it in, or rearrange the band’s song. Oh, yes indeed. They’re paying you to produce good music, and if that means fixing their mistakes, and using smoke and mirrors, then so be it!

Consider the overall flow of the song. Each instance of the chorus could have a different intensity. Perhaps they’ll get (perceptably) louder, except for the “drop chorus” at the end: Play the chorus once, with some instruments cut out, then play another chorus with everything driving the song home.

You may need to “mult” recorded tracks, which means splitting them up into separate layers, with different processing for each layer. The rhythm guitar should subtly slip into the background when the vocalist is singing, for example. Now that I’m aware of multing, I can’t help but hear it used everywhere!

Part 3: Balance

Every layer will need a high-pass filter, probably no steeper than 18db/octave. Raise the filter frequency until the track feels like something is missing, then lower the frequency a bit. This will liberate the precious low-frequency spaces, so the kick drum and bass won’t have to fight the other instruments in a bath of bass mud.

Now it’s time to balance the levels. Mute every track. Slowly unmute the trocks, one at a time. Start with the most important section; start with the most important instrument. Adjust the track’s volume so it fits with the mix, then don’t touch it afterwards. After you’ve leveled every track, your mix should now sound pretty good.

Compression is primarily used to stabilize the balance of individual layers. Some tracks will have a steady balance without compression, and that’s okay! Leave them alone. Other tracks may need one compressor to reduce momentary spikes in amplitude; another compressor to help even out the amplitudes of individual notes; etc. Try using parallel compression (aka “New York compression”), sending a track through a heavy compressor on a second channel, which will help even out the sound yet preserve its original character.

Expanders (and gates) are typically not essential. If they’re used, put them before compressors in your effects chain. If your drum beat needs more punch, try sending the track to a parallel channel, gating it (to isolate just the attacks), and applying EQ and a little distortion before mixing it back in.

Equalization (“EQ”) is a broad topic. Before delving into it, I’ll disagree with one point up front: The book claims that any EQ plugin is about as good as any other. Most modern DAWs are equipped with plugins that provide low-pass, high-pass, shelving, peaking, and notch filters. My opinion is that some EQ plugins sound substantially better than others. I have never enjoyed the sound of Ableton Live’s EQ 8, I feel that it sucks precious high-end frequencies and detail out of anything it touches. EQ 3 is better, although it’s less flexible, and the filter curves are steeper, which is not always what you want. Shameless product placement: My new favorite is Tone2′s FilterBank3.

To EQ the mix, mute every track, then unmute and EQ them in order of importance. You’re trying to avoid frequency masking, a phenomenon where layers obscure each other. Avoid graphic EQ, and try shelving filters before using peaks. Use low “Q” values, so your filters will have wide, smooth curves.

Bass instruments: Try adding high-end brightness to the bass guitar. Try reducing the kick drum around 400hz, creating a hole for the bass guitar to punch through.

If a track’s dynamics aren’t doing a good job, apply EQ before the dynamics. (It’s not uncommon to use both pre- and post-dynamics EQ, although this is more difficult to wrap one’s head around.)

Linear EQ should generally be avoided, since it smears the clarity & definition of transients (momentary bursts of sound, such as drum attacks).

And, this chapter answered an old riddle for me: Why are we told to use EQ cuts, and avoid EQ boosts? Because when you apply EQ to a range of frequencies, that frequency range suffers phase alterations. This can rob sounds of their clarity. As you boost phase-altered frequencies, you’re emphasizing muddier, out-of-phase frequencies. So a high-shelf cut (with a gain after it) really is preferable to the “equivalent” low-shelf boost. In a multi-mic environment, EQ boosts can be catastrophic, since instruments will leak into neighboring microphones with slight delays (i.e. different phases!), and may induce bad comb filter effects. Thank you, Mike Senior. I salute you for your clear explanation.

This surprised me: distortion is often used as a mixing tool! EQ can only modify frequencies that already exist, whereas distortion filters can add brightness and sparkle to the high-end. If your distortion plugin lacks a wet-dry control, the book suggests using parallel processing, by sending the track to an effects channel, and mixing it back in.

Multiband dynamics plugins have their uses, but are “mostly just an extension of what we’ve already covered in previous chapters.” If you’re clever, there are ways to achieve multiband dynamic effects without a dedicated plugin. For instance, if your bass guitar notes have uneven sustains, try extracting the lowest frequencies using a low-pass filter, and then compressing those, and mixing them back in. Watch out for a “hollow” quality, which indicates that phase artifacts are entering the mix.

Side-chained dynamics can be useful. The book explains “ducking,” which is akin to extreme side-chained compression: When the vocalist is singing louder than a certain threshold, the guitar “ducks” a fixed amount. The brilliant thing is, if you don’t have a ducking filter, you can create one using a side-chained gate: Send the guitar through a gate (side-chained to the vocals), and then invert the output. When gate opens, the inverted signal mixes with the original guitar channel and reduces the overall amplitude. Amazing. You can get creative, too: Try adding a linear-phase high-pass filter to the gate’s output, so only the high frequencies duck out when the vocals are active. You get the idea.

Part 4: Sweetening to taste

Reverb, chorus, and delays cannot salvage a bad mix. However, they can sweeten a good mix, and make it great.

My big takeaway from the chapter on reverbs is that you can move instruments away from the foreground, into the background by sending them through a common “blend reverb.” The reverb should have about 10-20ms of predelay. Reverbs should be CPU-intensive, as the better ones use lots of simulated reflections, and this requires more CPU power.

My big takeaway from the chapter on delays is that reverbs are overrated! Delays offer the same benefits as reverbs, but occupy less “space” in the mix.

The chapter on stereo enhancement is extensive, but the main point is that stereo width is more important than stereo panning. There are a number of ways to achieve stereo width effects using plugins (even rotary speaker plugins are discussed!). If the guitarist plays the same riff repeatedly, try hard-panning two instances of the riff, left and right. Try an M&S plugin, which splits a layer into the sum and stereo difference; pan the sum to the center, and apply all the crazy effects you want to the difference. Try adding some stereo “room tone,” which can be a recording of tape hiss, background noise in a room, or the crackling of a vinyl record. Try duplicating the vocals, pitch shifting them a few cents apart, and panning them hard left & right.

Finally, there’s a chapter on the “endgame,” which is final compression, equalization, and automating individual notes & transients by hand. The big shocker for me was that a little hard clipping is acceptable. This is used in commercial music, and it can add a little sparkle to the sound. That’s wild.

Definitely, definitely reference your mix with other people’s tracks. Try sitting in silence for a minute, and hearing the song in your mind. Now play your mix; does it sound like what’s in your head?

The book recommends a myriad of useful plugins, including shareware and freeware! The pages are also sprinkled with creative advice from professional producers.

Really, if you produce music at home, you’re insane if you skip this book. It really is that enlightening. Get it! Use it! Everyone deserves to make high-quality music, and it can be done on the cheap with today’s technology.

I paid $99 for a Midines in the year 2008, and… I have still not received it. I have sent Chris Kann at least a dozen emails, and never received a single reply. In 2008, I did track him down on IRC — he mentioned that he was going through some hard times, but now it is 3 years later, and he has been completely silent.

I wondered if I offended him personally, but it turns out that many, many, many, many (etc) people have sent money, and received nothing. A critical mass of people complained to PayPal, who finally shut down his account, so he switched to using Google Cart Service this year.

If I receive a Midines, then I’ll be happy to give it a proper review! In the meantime, I urge everyone to use their own research & judgment before sending money to strangers.

Meanwhile, whether you own a Midines or not, you can make NES-style chiptunes for free. I created an Ableton Live Pack with all 5 NES waveforms: Download it here. You can also download the raw AIFFs. Here’s a silly track that demonstrates how it sounds:

NES Instruments by Too Many Moths

The 5 waveforms are: 1-bit noise (the drum beat), a 4-bit triangle wave (the bass), and pulse waves at three different widths: 50%, 25%, and 12.5%. Here’s how the pulses look, next to the noise:

The triangle wave is a bit tricky. The NES provides 4-bit volume control for the other channels, but the triangle wave always plays at maximum volume. In my Live Pack, I generated a triangle wave using Operator, downsampled it using Redux, and then filtered the high frequencies to curb the Redux artifacts. This was a subjective process, I tweaked the filter until it sounded correct.

I recommend altering the instrument envelopes, to create smooth attacks and gradual delays. Also, try using vibrato and tremelo (changing the pitch or amplitude with an LFO). This will improve the character of the instruments immensely.

If you want a more authentic sound, try limiting yourself to 2 pulse channels at a time, since that’s all the NES can support. You can also send the instruments through convolution reverb, using a small speaker impulse. This may sound more like an 1980′s television speaker. Enjoy!

Try it. Click to generate new attractors.

The attractor coefficients are still chosen randomly. But now, attractors that explode/collapse are rejected. Also, attractors that create “boring” shapes (by drawing the same pixels repeatedly) are discarded. It’s a little slow, but I’m sure the speed could be improved using Pixel Bender.

Also, here’s the source code. (Compile with Flash CS5.)

Watch: regular, or HD version. (Vimeo, 4:00)

The clips have a bias towards art and animation, because: 1). It shows well in a video, 2). This reel was created for an application to an artist’s colony. (More on that later.)

The intro & outro animations (the flying moths) were created specifically for the reel. They’re my favorite clips, yet they took less than a day to assemble. The moths were left-over from an old project (you’ll see an experimental Too Many Moths interface within the video), and I felt the convergence motion was a strong opener.

The moths were composited in Flash. They were arranged semi-randomly, then the image was “baked” onto their wings & body. I flew them backwards, and recorded their positions on each frame. Then I reset the moths and flew them forward, again tracking their flight paths.

Actually, this was done in memory, so it was very quick. My laptop needed 3 seconds to arrange, bake, & move 120 moths. Then the harness presented me with the frozen frame, so I could evaluate how well the moths covered the images. If the coverage was bad, I hit a key to build new moths from scratch.

When I found a nice arrangement, I hit another key which stepped through the animation, and rendered each frame at a ridiculously high resolution (2160×1440, to cope with scaling problems in Adobe Premiere, unfortunately). The frames were encoded as .PNG’s and uploaded to a PHP script running on localhost, which saved the 200+ images to disk. The intro took about 20 minutes to render & export. The outro, being more complicated, took over an hour.

Looking at the reel, I’m happy to present so many types of work — animation, audio programming, visual design, interactives, games, kiosks, websites. These four years have been amazing.

• Import AIFF files
• Audition and edit formant sequences in real time
• Vowel- and function-drawing tools
• Export .syx files for the Yamaha FS1R

Click to launch. Press the space bar to play the sound.

You can push your formant sequence to the Yamaha FS1R, using software such as K_Take’s FS1R Editor. Click the “Save .syx” button, and follow the instructions in K_Take’s documentation. This is a lot of fun, and breathes new life into the FS1R.

This project became much deeper than anticipated! The code includes FFT analysis (thanks Gerry Beauregard), pitch detection, a formant detection algorithm, and an AIFF parser to read AIFF files. The interface was a challenge to design and implement, and there are still many unfinished features (see the issues list).

My energy is shifting to other work, so I’ll enhance fseq-flash when time permits.

Filtered Noise Sequencer

Drag and resize the blue blocks to change the filter frequency and width.

This sequencer is not using expensive bandpass filters. The oscillators are sine waves, which are frequency modulated with white noise. It may not sound inherently musical, but you can produce great hihats, bass thuds, and airy pitched noises.

Here’s the source code. (Requires Flash CS5 to compile.) Have fun!

A Curious Lack of Light by tooManyMoths

(click to launch)

The next version of Twang will record & save songs (this is done), and share them in some capacity (a bit more complicated). So that’s where the Twang Player comes in. The Flash version looks like it wants to be touched & strummed. I need to revisit the design and convey that Twang Player is just a music box, you can’t compose anything in the browser! …Yet.

There are performance issues on some of my machines, too. This version uses Vector.<Number> objects to handle data, but it looks like ByteArray or even BitmapData structures are the way to go.

First, do you know Lev Grossman? He’s an incredibly talented author who recently toured Portland. If you haven’t read his book The Magicians, then stop whatever you’re doing and procure a copy immediately. Without trying to spoil anything, the major college in the book is named the Brakebills College for Magical Pedagogy. Lev saw the Brakebills T-shirt that I designed for my sweetheart’s birthday present:

Long story short, my Brakebills shirts are now officially endorsed by the author! Please buy twenty of them, or thirty if the spirit moves you. Zazzle’s print quality is great. The colors are vivid, and the ink has survived many washings. Rabid fans are buying and even customizing them! I love the internet.

Lev blogged some of my other work, too (“the guy who does this has the enviably fake-sounding name of Zach Archer”). It’s true, I have an awesome pro wrestler name.

Second, my new iPhone app has landed in the App Store:

Twang is a handheld guitar. It’s easier to play than a real guitar, and is very expressive. Instead of using audio samples, Twang uses physical modelling techniques to create a more natural, dynamic sound. No two plucks are identical. Watch my grainy first video if you disbelieve.

In the next version of Twang, left-handed people will be able to switch Twang’s orientation, and serious musicians can dampen or mute strings with their fingers. And probably more! This version is already in development, and may be submitted in a week or two? Follow Control Z, Inc on Twitter if you have a ravenous thirst for updates!

Here’s something from the vaults. Aquasound was built with these requirements in mind:

• Generate sounds that aquatic animals might make
• Sounds can be “combined” somehow
• Sounds can emote

This was never used in production. I wonder if I could turn this into something? Like a paid iPhone app? ;)

Double-click the envelopes to add/remove control points. Drag lines up & down to change their curviture. The best feature is the “Combine With” dropdown, which splices the current sound with your selection. Also the “Emote” menu will play sounds with different expression.

The audio algorithm is reverse-engineered from my beloved FS1R. I generated formants in two ways (toggle the “Tonal” checkbox to hear both), the “atonal” version is closer to ring modulation than actual formants. It’s more fun if you don’t understand what the controls are doing, but if you insist: Pitch controls the overall pitch of the sound. Freq controls the center frequency of the formant (like a bandpass filter). LFOFreq and LFOWeight control a low-frequency sine wave, which can be applied to other controls via their “___LFOAmt” curves. Amp is amplitude, Width is formant width (think: width of the bandpass filter), Skirt adds distortion. Each voice has two formant generators, check “Formant Active” to enable them.

May all your bloops and crackles be happy ones!

Here’s my second batch of image reconstruction using Times New Roman characters. The algorithm is a brute-force affair: New characters are colored & positioned randomly. Any characters that make the canvas look more like the original image are saved. And that’s about it. Oh, and the font sizes start large (5120pt) and end small (10pt), so that fine details have a chance of survival.

Time lapse of the Mona Lisa reconstruction on YouTube, with silly music.

My first batch uses a different algorithm. Each canvas allocates a certain quantity of letters, and progressively mutates them, trying to mimic the original image as closely as possible. This technique is more akin to image compression. This batch is still in progress, it’s very slow. I’ll post these when they’re ready!

This took about 5 weeks to develop. Meanwhile, I’ve started another app, and I envision releasing Metal Mouth 2.0 in a few months, with more voices, and the ability to record audio.

I spent a couple hours generating strange attractors in Flash, just a simple 2D version for now. Click to play:

Click the black region to generate new polynomial coefficients and redraw. You will have to click many times to generate something interesting. That’s the nature of fractals, I’m afraid. Some coefficients are automatically thrown out if the drawing exceeds a certain size. Unfortunately, the inverse is not true: the code isn’t smart enough to trash any drawings that shrink to microscopic size.

I believe that you can stabilize any coefficients by scaling the values of each coefficient, gradually nudging them larger/smaller until the drawing is stable. I’ll try this when I get more time. I’ve been gung-ho on my first proper iPhone app, trying to finish it before Christmas! Stay tuned…

Also, here’s the source code for the strange attractor harness! Enjoy.

Today I noticed that if you right-click on yesterday’s SWF and show the redraw regions, you can see that it’s redrawing the contents of the entire stage, even though I put the scene in a scrollRect. Is it seriously rendering a scene that’s thousands of pixels wide before displaying it ?!?!?!? Oh, no. No they DIDN’T.

Today I ported the scene to the Away3D rendering engine. Here’s the result:

It’s beautiful, and provides access to low-level drawing routines, light sources, normal maps, … It was speedy at first, then slowed down considerably when I added the glowing floors. (Each glow is 16+ triangles right now, for various reasons including: I can’t render objects in my own custom order.) This makes yesterday’s version look performant, I’m reluctant to admit.

Possible next steps:

• Reduce the native 3D rendering area, see if performance improves?
• Grow beyond Flash, embrace the future and try Unity 3D?
• Dump this project, finish that iPhone game I started, make a million dollars in 2 weeks?