tnSuperman tanktn oParticles uExtrusion j3d sandy flExtrusion fuelyoursenses meshCam bufferSnake dMap2 dMap pv2 cursorFire opticks faceGame FPSnake pv preOpticks painty youtuber fractalFIngers 2dbals

November 13, 2010

Developing Broadband IO

Filed under: doodles,Games — Tags: , , , , , , , , , , , — Brandel Zachernuk @ 12:11 pm

This is the game!  No keys, no mouse – just whistle to move the cat up and down.  The higher the pitch, the higher the balloon, and the lower the pitch, the lower.  Try to catch the yellow dots and see how far you can get!

NOTE: This requires Flash player 10.1, a very recent update – if it isn’t working, chances are you’ll have to grab it from here.  Due to feedback issues you might want to mute your system while it’s playing.

Get Adobe Flash player

UPDATE: This also suffers from the ‘default input device’ problem that my webcam experiments experience – in order to get this to work, you’ll need to open the ‘settings’ panel by right-clicking and selecting the microphone icon.

Select this section and check that the right mic is selected.

I just got back from AnimFX 10 this weekend, and it was a fantastic experience!  It was a great opportunity to learn about what Zynga thinks about their games, how Popcap manages to polish the way it does and what the pipeline for making a film like Avatar looks like.  I’ve taken lots of notes on the sessions and will do a more complete write-up of the sessions that I have something to say about, but for now I’d like to thank the speakers and the GAV NZ Trust for putting on such a great show, and all the attendees from all around NZ and Australia -it’s fantastic to know that there is such a talented and interesting community of people here working on such fun stuff!

Since the first time I started thinking about game design (admittedly only in about 2004), I have been interested in what happens when you break convention.  How do you make a first-person shooter from the perspective of a blind person?  How can you play a game without moving a mouse or pressing any buttons?  What do we have access to that we could try to read information from that people can use in a game? The bandwidth, or amount of information that can be transmitted back and forth between a computer and a person is determined by the Input-Output systems available (IO).  In future, we will hopefully have access to facial recognition, galvanic skin response and, eventually, brainwaves – although at this point it’s anybody’s guess how to make sense of them!

In the meantime, without bolting extra sensors on to a computer system, the range of input mechanisms is somewhat more limited.  We can at least play around with what’s generally available, though.   One thing that I’ve always been interested in is making use of voice input to play a game – and not necessarily a game about singing, either.  The way I see it, the voice can represent at least three different values at once (pitch, volume and ‘envelope’ – basically the vowel sound).  In theory, you’d be able to control each aspect independently, and have a continuous, three-dimensional mode of input even before you touched a button!   If people got precise with it, it could be a way of learning how to form certain sounds, develop a sense of rhythm, or learn perfect pitch – It’s a long shot but there’s only one way to find out!

Spectrogram of Dolphin Clicks, Whistles, and whines (Wikipedia)

As a starting point, the relatively recent update of Adobe Flash Player to version 10.1 has exposed the actual waveform data that gets taken in from a microphone feed.  This means that we can start trying to look at that wave data and look not just at the volume, but everything about it.  I’ve made a super-simple prototype that takes in the voiceprint, picks the loudest overtone and lets us see it.  In this case I have made it specify the altitude of an hilarious cat, which is trying to catch all the goodies it can.  Whistle high to push the cat-balloon up, low to make it go down low, and anywhere in between to reach the height that a pitch corresponds to.  I’d love to hear what you think of the concept!

By default, Mac computers have a number of ‘microphone devices’ attached like ‘firewire’ – make sure the selected mic is something like ‘built-in’.

A couple of things that come to me from the limited testing I have done:

1. Quite a lot of people can’t actually whistle.  The choice to use whistling as a driver is not out of malice, but because it’s such a ‘pure’ waveform. I’m going to try and increase the sensitivity of the analysis so that singing would work too.

2.  I got quite a lot better at estimating pitch after playing the thing for 5-10 minutes.  Having such a direct mapping, with visual feedback about how close an attempt is to the actual goal seems to make it a lot easier to estimate my error as a whistler.

3.  I can’t make it to more than about 25 points without cracking up from the silliness of it all.  With a little more variety in the goals and the kinds of interactions and I think we’ll be on to a winner!

1 Comment

  1. That’s positively hilariously insane. I only wish it worked on my Mac…

    Comment by Tom K — November 14, 2010 @ 7:36 am

RSS feed for comments on this post. TrackBack URL

Sorry, the comment form is closed at this time.

Powered by WordPress