After my tinkerings with sound output and processing, my Friend Aidan got inspired to embark on a more ambitious project of creating a simple music sequencer and invited me to participate.  A large portion of our free hours since then have been devoted doing that.  We’ve been really impressed – even without particular attention to efficiency, it seems that even Flash can take pretty much everything we throw at it!

Both have us have tinkered around in music sequencers for over a decade  now.  We started with ReBirth RB338 and Jeskola Buzz, so we’re very familiar with the expectations for what needs to go into this type of app.  It has been fascinating to shift from simply using a sequencer to considering how it should actually work!  There are so many questions around application design that have remained theoretical until now, where we’ve run across them almost by accident. There are some questions that are much harder than they would first appear – When something is a dial vs. when it can be a slider, how to guide a user through the most important feature of an application etc.  Other things that people often get wrong are laid bare as lazy mistakes – Load/Save effects and a failsafe when the application gets too processor-intensive, say.

Anyway, do not be fooled by its sleek, refined appearance! This application is very much a work in progress, and part of the fun of working on it is deciding what happens next.  If you have any ideas, let us know!

(this is what it should look like)

Warning: Setting the squareness below 0 will result in a much louder sound than anything else, so be aware that you might need to turn the volume down!

I’ve been reading a textbook called Human-Computer Interaction. It’s an excellent and extensive primer on the subject for someone who hasn’t studied it formally in the past, although at nearly 8 years old it can’t be expected to be up-to-date on everything in the field. At first I thought that one of those areas was the use of sound in HCI, though when I thought more about it, I realized that sound is almost always an afterthought – in games, and especially in applications.

One of the reasons that sound-based UI isn’t explored more is that an interface that uses pre-recorded sound samples is limited in the number of ways that those samples can be used to convey information. Sound can be generated on the computer, though, so I decided to look at where more fine-grained feedback could be given to a user through sounds that can be modified at will to reflect even minor changes. In order to do that I had to put together a system that would be able to generate a wide range of sounds that can take a range of parameters, and what you see before you is the initial result of that exploration. Source code can be provided if desired. I’m hoping to use this generative sound output in the near future – stay tuned!

NOTE: This requires Flash player 10.1, a very recent update – if it isn’t working, chances are you’ll have to grab it from here.  Due to feedback issues you might want to mute your system while it’s playing.

UPDATE: This also suffers from the ‘default input device’ problem that my webcam experiments experience – in order to get this to work, you’ll need to open the ‘settings’ panel by right-clicking and selecting the microphone icon.

Select this section and check that the right mic is selected.

I just got back from AnimFX 10 this weekend, and it was a fantastic experience!  It was a great opportunity to learn about what Zynga thinks about their games, how Popcap manages to polish the way it does and what the pipeline for making a film like Avatar looks like.  I’ve taken lots of notes on the sessions and will do a more complete write-up of the sessions that I have something to say about, but for now I’d like to thank the speakers and the GAV NZ Trust for putting on such a great show, and all the attendees from all around NZ and Australia -it’s fantastic to know that there is such a talented and interesting community of people here working on such fun stuff!

Since the first time I started thinking about game design (admittedly only in about 2004), I have been interested in what happens when you break convention.  How do you make a first-person shooter from the perspective of a blind person?  How can you play a game without moving a mouse or pressing any buttons?  What do we have access to that we could try to read information from that people can use in a game? The bandwidth, or amount of information that can be transmitted back and forth between a computer and a person is determined by the Input-Output systems available (IO).  In future, we will hopefully have access to facial recognition, galvanic skin response and, eventually, brainwaves – although at this point it’s anybody’s guess how to make sense of them!

In the meantime, without bolting extra sensors on to a computer system, the range of input mechanisms is somewhat more limited.  We can at least play around with what’s generally available, though.   One thing that I’ve always been interested in is making use of voice input to play a game – and not necessarily a game about singing, either.  The way I see it, the voice can represent at least three different values at once (pitch, volume and ‘envelope’ – basically the vowel sound).  In theory, you’d be able to control each aspect independently, and have a continuous, three-dimensional mode of input even before you touched a button!   If people got precise with it, it could be a way of learning how to form certain sounds, develop a sense of rhythm, or learn perfect pitch – It’s a long shot but there’s only one way to find out!

Spectrogram of Dolphin Clicks, Whistles, and whines (Wikipedia)

As a starting point, the relatively recent update of Adobe Flash Player to version 10.1 has exposed the actual waveform data that gets taken in from a microphone feed.  This means that we can start trying to look at that wave data and look not just at the volume, but everything about it.  I’ve made a super-simple prototype that takes in the voiceprint, picks the loudest overtone and lets us see it.  In this case I have made it specify the altitude of an hilarious cat, which is trying to catch all the goodies it can.  Whistle high to push the cat-balloon up, low to make it go down low, and anywhere in between to reach the height that a pitch corresponds to.  I’d love to hear what you think of the concept!

By default, Mac computers have a number of ‘microphone devices’ attached like ‘firewire’ – make sure the selected mic is something like ‘built-in’.

A couple of things that come to me from the limited testing I have done:

1. Quite a lot of people can’t actually whistle.  The choice to use whistling as a driver is not out of malice, but because it’s such a ‘pure’ waveform. I’m going to try and increase the sensitivity of the analysis so that singing would work too.

2.  I got quite a lot better at estimating pitch after playing the thing for 5-10 minutes.  Having such a direct mapping, with visual feedback about how close an attempt is to the actual goal seems to make it a lot easier to estimate my error as a whistler.

3.  I can’t make it to more than about 25 points without cracking up from the silliness of it all.  With a little more variety in the goals and the kinds of interactions and I think we’ll be on to a winner!

It was an idle thought, until the other night when I found I couldn’t sleep, so I came up with this model for a paper split pin, and I decided to try it out this afternoon.. the result is what you see before you!

It turned out to take a lot longer than I had anticipated – the pins do work and they are quite tight, but you need to assemble them with a pair of tweezers.  That said, though, it was only printed on 90GSM inkjet paper.. I would like to try it on heavier stock to see if it makes it easier. In the meantime though, If you’d like to try for yourself, though, The PDF Download is here: Brandel Zachernuk’s Paperguy v0.01.  If anyone finds a use for it or makes any modification, I’d love to see it!

The possibilities are terrifying.

I like Unity and I like being able to play with the webcam as input.  Unfortunately, Webcam support within Unity – in particular, webcam support in the browser plugin for Unity – isn’t really possible out of the box.  I’ve written a (somewhat naive) implementation using the Aquiris u3dobject framework – click here to see it in action! (I should also warn you that it seems to be somewhat (read:extremely) unstable when the video stream is being passed through – On Windows it seems to run at 10fps at an absolute maximum with the video stream on, on Mac it fares even worse.. fun to see though!)

A guy at Doncaster College has managed to get a similar system up and running, although it doesn’t seem to work for people behind a proxy. (Any similarity between their solution and mine is because they’re both based on the Aquiris sample system. The Doncaster people seem to be using a localconnection to get the texture transferred, which sounds promising;  Mine is just passing things through the web page, and transferring the pixel data in an extremely sub-optimal way.  Each pixel’s value, for example Ox FF FF FF (white) is converted to the ASCII string “255:255:255″, passed through the JS and converted to Unity-native colour values (1.0f,1.0f,1.0f).  It’s roundabout but I found that there were incompatibilities in the byte-array representations of bitmaps between Flash and Unity, and since the middle JS stage is created as a script, so the string has to be valid there too.

It’s still a far cry from the the totally direct form of Webcam input that appears to be possible with a commercial license – I’ll definitely be looking into how that one works.  Even so, it represents a terrific way to leverage some of the functionality of Flash without having to re-create it in Unity – whether that functionality be webcam support or related to other aspects of Flash’s functionality like p2p network connections, microphone support or having a reasonable hook into local storage for a persistent session.  I’m hoping to make good use of this bridge soon!

The Car

I have the mk.2  car back!  It went together very well, though it was perhaps too tight in some places – the short loops and forks that hold the drive wheels to the gears needed to be shaved a little.  I added 0.1mm to the perimeter of all the shapes, and on thinking about it, it seems likely that it should have been more like 0.08 mm or so.  Needing to make adjustments that tiny is a strange business indeed!

Anyway, here is the finished product in all its lasered glory:

This time there were only two things amiss:  First, I didn’t account for the width of the AA batteries in the bottom compartment in addition to the holder for the power transfer gear, so it can only hold 1x AA in the bottom bay and 2x AA in the top – although you could probably pack a bunch of those squishy Lithium ion batteries in with no problem.  The other problem lies in trying to make the design too cramped – on each side, one wheel is attached directly to the motor, and one wheel is attached via a wooden axle, to a point behind the motor that powers the opposite side.  The current design is so compact, though, that there isn’t really enough room behind the motor for a solid fixture, and it has a tendency to fall off.

Widening the design by 3mm would solve both of those problems and the whole plan would stay well within the 181 x 181 mm limits set by Ponoko though, so I’m pretty pleased with the design.  Ideally I’d like to be able to keep the base design small enough that it can form the basis of a miniature robotics platform, and be able to have room on a P1 board to put holders for sensors or a small robotic arm. Stay tuned!

Arduino Enclosure

I also needed to make an enclosure for the Arduino Duemilanove as part of another project I’ve been tinkering with. After looking online I found surprisingly few examples anywhere – in particular a total absence of  compact, fully-enclosing cases, so I designed one and got it cut.  At first I was going to get it made in 3mm MDF as well but Heather suggested that an acrylic would be more pleasing and I agree.  It looks like this:

There are pass-through holes for the three holes in the Duemilanove board,  so it could be bolted to another object.  I offset the pats for this design by 0.1mm as well, and unfortunately, acrylic is a lot more brittle than MDF, which has meant a couple of small parts snapping off when I tried to force them.  Other than that, though, the design is perfect and the tightness has meant it keeps together really well!

The other good thing about sticking the Arduino in an enclosure is that you can stick other things to it – in this case I wanted to guarantee the orientation and position of a compass module on top of the unit, which is encased in the small box on top.

Once I made the enclosure design (and was able to verify that it actually works), I decided that it would be worth looking into making the interlocking parts more automatically.  I was inspired by Jared Tarbell’s Cubes to start thinking about how to generate the map automatically.  It’s still very much a sketch, but the principle is straightforward enough to automate most – if not all- of the process.

Lay out the lines of a shape by clicking, adjust the tooth width with the slider and hit ‘clear’ to clear the shape.  Enjoy!

The complexity of the thing you simulate doesn’t have to be much either. The term “computer simulation” often evokes imagery of simulating enormous and complex systems – weather patterns or the turbulence flow from a large-scale atomic detonation – but simulation can be useful on a more modest scale too. In University, a friend of mine threw together a blackjack simulation.  He wanted to see the patterns of how well a virtual player does when it follows hard-and-fast rules – refusing cards over a total of 16, 18 and so on.* The first programmable computer, the Z3, ran at a staggering 10 Hertz – roughly one billionth of the speed of a modern computer. Even at that speed, though, it brought the science of aviation forward by a decade through simulating wing flutter at rates of execution that were previously impossible to reach.

The example I often use to explain my fascination with simulation is this: Imagine a child’s sand-pail at a beach. Imagine the light dusting of sand it gets from a long day of use. The smoothness at the joints, the build-up inside any detailing and under the lip. Now imagine taking the bucket away and leaving the sand – the suggestion of form but not the form itself. Next to impossible in reality, but in a simulation, once you have the base action down, setting things to on or off is the easiest thing in the world.

I’ve been using this example of the sand and the bucket for years now, and recently I decided to actually do something about it – make a sand simulation that can show you what I mean, rather than just telling people about it. Here it is! Just move the mouse around on the play field to shift the sand. If you want extra options for playing with the sand fall or changing the pen size, open the options panel. Many thanks to Aidan of Shade and Prime for the fantastic elevation visualizer.

iPhone app, anyone?

* While it might seem like this is a less-than-useful endeavour to use computers for, I’m sure we can all agree that it is better to test a virtual player losing virtual money than trying it at your local casino. In fact, after finding that there is no foolproof approach to blackjack, he gave up on it altogether and started putting his money in the stock market… you win some, you lose some.

I’m not sure whether this is re-creating functionality that other people have already put into papervision, but it was a lot of fun! There are some issues with shaded materials, but so far I have bitmap- and colour materials displaying and it’s looking good.  You can try it for yourself!  Like the pale text says – click in the window to add a vertex, press space to close the shape.  It seems to work even on pretty low-powered machines, so give it a try!