Natal, MotionPlus and the bright new frontiers of HCI

June 11th, 2009

When I started to catch up on what happened at E3 recently, I was dumbstruck. Between Microsoft’s Natal and Sony’s yet-unnamed motion control, we are seeing quite a turnaround. Motion control is the new bandwagon to jump on - and everyone seems to want a piece.

Maybe not everyone.  Gamers seem to be in violent opposition to the idea of changing input methods.  On Slashdot, Gametrailers and other forums I still see people insist that the two-thumb controller is the best input method for platformers, a mouse the best method for Realtime Stragety (RTS), and a mouse and keyboard for First Person Shooter (FPS) games.

Gaming in 1990

Gaming in 1990 (click to enlarge)

I find that reaction disappointing.  For a start, It’s probably not true.  Prior to Quake, most ‘gamers’ (though they did not exist in the same numbers or have the same recognition as a demographic) insisted that a mouse was an unnecessary and fiddly complication to the task of moving a character around, and that looking up or down could be done easily enough entirely with the keyboard. As strange as it seems, gamers and technophiles in general appear to be a very conservative lot.*

Second, even if it were true, and it is a possibility - it is only because the RTS, FPS and platformer genres have been forced to work as effectively as they can the available systems.  They’ve been designed with these input methods in mind, so of course they’re going to seem like the only option available.

Gaming in 2000

Gaming in 2000 (click to enlarge)

When you change the range of input devices, though, the options broaden.  It’s not hard to imagine a shooter where the player can lean around corners, duck projectiles or operate machinery with a little more nuance than just a ‘use’ button.  With RTS games, we could issue two orders simultaneously with each hand, draw out formation plans with gestures or guide the attitude a subordinate AI officer commands with through a facial expression.

While I’m glad that “Motion Control” appears to be the bandwagon to jump on at present, I urge developers, gamers, and the public at large to see motion gaming as a ‘gateway drug’ to better and weirder things. We can turn this into a watershed moment in history not just for gaming but for computing in general, where we decide that the main challenges in computing today are with Human-Computer Interaction (HCI), rather than computation or processing.

Gaming in 2010

Gaming in 2010 (click to enlarge)

One relatively recent breakthrough in Artificial Intelligence is the realization that agents are far, far more effective if they are properly embedded in the environment in which they operate.  That means having both consistency of - and plenty of - sensory information. Problems involving vision become (comparatively) simple when a number of camera views are added and combined as input, the problem of balance is made much easier if acceleration and force data is collected directly from multiple points rather than modelling ‘expected’ values internally.

It follows, then, that the task of making sense of what a user wants from a computer would be made easier by giving it more information.  What may be less obvious is that it works the other way around, too.  The more sensory input from a system we have, the more sense the system is going to make, and the more immersive that system becomes.  The “next big thing” in mobile devices after touch-screen interfaces is tactile feedback from interfaces.  The Nokia Haptikos display will try to simulate the press of a button by shaking on confirmation.

I welcome the shift in emphasis of the console vendors.  We need to ignore the grumbling from current gamers and game developers, who are very stuck in their ways.  These detractors are ignoring the trends in gaming that have led us to this point, and where these trends indicate that gaming will go.  I hope that by the time I have kids who are old enough to lecture, I can tell them to go outside and play a videogame.

Gaming in 2020

Gaming in 2020 (click to enlarge)

* As an aside on the conservatism of technophiles, I remember that, at the ripe old age of 10, I was incensed at the suggestion that people were installing “sound cards” and other add-on hardware into IBM-PC compatible computers.  PCs were supposed to be generalized computing devices! If you wanted sound that badly, you should wait until the CPU could process it all in software! If I had heard that people were going to then go on and make cards that dealt exclusively with 3d graphics, I probably would have passed out from sheer indignation.  I was an unusual child.

On Simulations and Agency

April 27th, 2009

During my time at Clocktower Games, I had an on-going debate with Henry, the writer.  He was of the opinion that realism in games - creating accurate simulations of water, ragdoll physics, gravity or any other natural phenomenon was a pointless waste of time.  These things already exist in reality, so why bother making what would inevitably be a poor copy of them?

Over the course of this Opticks project, I’ve been wondering a lot about that - after adding in lensing and additive/subtractive light systems it started to get important.  The next step is chromatic aberration - making red split from green and green from blue - but what’s the point of this? How far is it worth going?

Fiat Lux!

Fiat Lux!

I think the answer is this:  When you create an artificial system that mimics the laws of nature, you’re in control of when the laws are the same and when the laws are different.  In reality, mirrors are never perfect reflectors - they might reflect 99.99% of light (many do) - but if you bounce light between them indefinitely then it’s eventually going to dissipate.

In Opticks, I can make a perfect reflector - even a mirror that reflects 110% of the light that falls on it.  I don’t know why I’d want to right now, but the point is that I can.

And the errors in simulated reality can look so funny!

And the errors in simulated reality can look so funny!

Arbitrarily altering the simulated reality doesn’t even need to be useful, either - it can be done just for the enjoyment of experiencing an altered reality.  If you want to change your perception of a thing, you can either change your perception, or change the thing.  Drugs do the former,  simulations allow me to do the latter.  This way, I can take screenshots of what I see!

Lenses in the Opticks engine

April 22nd, 2009

Sorry for the lack of word, I moved to China and it’s all a bit exciting.  More updates to Opticks-

* Additive colours - beams and lasers whose paths overlap now add to one another, so that the resulting colours are as they should be.

* Limited to Flash Player 10 (necessary for performance and loading & saving)

* Loading and saving to file - I will attach some level files soon!

* Lenses! - This is a little tricky.  Use the rotate tool to orient the lens into place, and the distance from the lens will give you the focal length (negative or positive).  It’s a little buggy with beams, but seems to work perfectly with lasers so far.

With all of that, I give you lenses:

Things on the to-do list:

* Integrate a proper panelling system, so that panels, tooltips, instructions, level load/save dialogs etc. can be managed reasonably.  Is the Flex stuff the right way to do it? I’m not sure.

* Create a way to alter the colour of lenses / beams / mirrors etc.  This will come when the paneling system is sorted out.

From here,  I’ll probably be returning to more on what to actually do with something like this re: education and games.  I hope the discussion will be interesting!

Opticks update

March 25th, 2009

So I’ve been working on the idea of the level editor, and here’s what I’ve got so far. Expect loading and saving to be happening soon, as well as thin lenses and hopefully colour manipulation. Times are sweet!

A few words of explanation / warning:
* You have to select an object before you move it. Marching ants will appear around it.

* To select an object or objects, drag a rectangle over them with the select (cursor) tool.

* Two mirrors intersecting will still cause the thing to fall over.

But enough of that - here it is!

Beam Opticks!

March 9th, 2009

I’m very very sorry for not updating but I’ve got it!  Here is the proper workings for beams to work.  I haven’t ensured that mirrors don’t overlap, but this causes the thing to largely fall over, so avoid it.

(Move to a mirror to select, click on either side to rotate, ctrl+drag to move.)

What’s next from here?  Implementing prisms, which will require implying a medium (and relative refractive index) then hopefully thin lenses should come fairly naturally out of that.

What kind of game will this eventually make?  I’m hoping something along the lines of Fantastic contraption but including things that require lasers and other beams of light.  I expect to be able to burn things soon!