I’m not sure whether this is re-creating functionality that other people have already put into papervision, but it was a lot of fun! There are some issues with shaded materials, but so far I have bitmap- and colour materials displaying and it’s looking good.  You can try it for yourself!  Like the pale text says – click in the window to add a vertex, press space to close the shape.  It seems to work even on pretty low-powered machines, so give it a try!

Of course, it’s great when GTA and WoW do it because you know that any investment you make in your character customization is going to be worth it.  Each new installment of GTA offers gameplay for dozens of hours, and you can play WoW, for better or worse, for as long as you can stand it without running out of things to see.  In games of this calibre, the new customization options are a welcome sight, and I enjoy discovering what new options are available as time goes on.

In a new game – particularly one without the backing of recognizable names – customization screens can have the opposite effect.  Without knowing a little bit of what I’m in for, it’s hard to muster up the enthusiasm to define my character.  If it’s a requirement, which it often is, I make rash decisions in a show of contempt for the system that end up making my avatar look something of a mess.  The remainder of my play experience is then realized through this hash of a character, and ends up tarnishing my experience of the whole game.  A difficult-to-implement feature, that is often considered a must-have for social games, can end up ruining it.

So customization screens as a first port-of-call are annoying, but so is a lack of customization options.  It can be made an optional, progressive task, as in GTA: San Andreas, but most games can’t afford to demand the sheer amount of time that GTA can.  In addition, games in a highly-connected, social space have to make this a requirement in order to make concurrent players distinct, so it has to be done, and done before a lot of the fun can begin – so the question we’re left with is “How do you give people a meaningful custom avatar without forcing them into a customization screen?”

Web development to the rescue

I’ve done time as a web developer before, and one of the technologies that has made the job much easier since the 1990s is the development of  Cascading Style Sheets, or CSS.   The Cascading part refers to a neat technique where more general definitions are are inherited by more specific ones, unless otherwise specified.  Basically this means if I want all text to be red, I say “All text should be red”.  Then, headings, subheadings, body and caption text will all be red – unless I want caption text to be green, in which case I say “All text should be red but caption text should be green”.  This has a significant advantage over defining each piece of text colour by hand in that it’s smaller to load, faster to write and easier to change if you decide you want to go with a slightly less gross colour scheme.

..So?

What I would like to borrow from CSS here is the idea that definitions can start out as vague, and become more specific with time – not just definitions of web pages, but definitions of character customization too.  Say I often have pale skin, wear glasses, grey shorts and a red-and-blue T-shirt (I do).  It wouldn’t be a bad place to start for every character I start out as to possess these properties,  and if I want to refine them further, I could.  If a game demanded a character that was an elephant instead of a humanoid, I might decide that I would prefer it not to wear glasses, but I’m not likely to resent the attempt, particularly if it was done automatically.

To an extent, Nintendo already does this with the concept of the portable ‘Mii’, which can represent you in a number of different games.  However, Nintendo requires that the Mii be presented indentically in all the games it appears in, which either limits the aesthetic of the game to the standard set by the Mii art, or jars with the artistic style employed in the game.  On the other extreme is Second life, where there are next to no restrictions on what you can do to your avatar or to your island.  The results of that aren’t pretty either.  I’d suggest we not go there.

I think it would be possible to find something in the middle.  If players could define a character appearance once. with some kind of open standard, then it would be possible to use that as a basis for any new character in a game.  If those parameters were made available to game developers, they could be integrated into the game in a manner that was in-keeping with the artistic direction of the game.  Players get avatars that they can immediately identify with, without having to invest time in building them,  Developers get a recognizable source of customization data that they can interpret how they like.  Everybody wins!

A few months back I started playing through all of the main (free) Social Games available – I went through most of the games by the two major figures in the space, Playfish and Zynga, and a handful of the offerings from other competitors. I’m hoping to turn this into a series of posts on my observations through these games.

Why can’t you play nicely with the others?

One thing that struck me about these Social Games is how little social interaction actually occurs.  Mafia Wars, the most popular social game, puts you in the position of a mafia boss.  Friends of yours who also subscribe to the game comprise your mafia.  You are able to request assistance on specific jobs from your friends, give them gifts and swap special ‘collectible’ items with them – but participation goes no further than this.  All that distinguishes one friend from another is their name, avatar and experience level.

In fact, for a price, you are able to enlist random strangers into your ranks, who behave no differently to your peers.   Equally, you are treated in an identical manner – your name used as a largely meaningless token in your friend’s Mafia engagements.

So?

To an extent, this is an inevitable part of gaming – if I create a game-world with a limited complexity of representation, some features of my personality will be rendered irrelevant.  The fact that I like long walks on the beach is likely to be insignificant in a game about intergalactic space battles, just as my intergalactic space tactics are likely to be unimportant in a game about long walks on the beach.  In Mafia wars, though, not even my views on the Mafia make the cut – not so much as a D&D-style moral alignment or request to sacrifice progress toward one goal for another.  Likewise, any charitable acts I engage in – sending out an “Energy Pack” or giving a gift to my peers – carries no consequence to me.  The items are not available to me in the first place other than to give to others.  Without any opportunity cost there is no decision for me to make, and no way to make my mark or distinguish myself from other players.  When I play, no aspect of how you play has any impact on my game’s representation of you, and vice versa.

How it normally works

Most multiplayer games have been based on either symmetrical (largely turn-based) or synchronous play.  Symmetrical games like Chess require that for every move that you make,  I make one too.  Synchronous games like Warcraft 3 and Halo  rely on the fact that we are both given the same opportunity to act – they assume that every player is giving the game their full attention,  so that any difference in impact comes down to a difference in skill.   It’s not possible to rely solely on these techniques for creating a multiplayer experience in the social space –  I may rack up ten times the average play-time of my friends in a week.  A symmetrical, round-robin approach would mean that nine-tenths of my time in the game would be spent waiting for others to catch up, while a synchronous model would require that I be online at the exact same time as my friends in order to collaborate.

These traditional methods for creating a multi-player experience may fail in a larger social context, but they were  also designed for a different quantity of players.  Chess requires exactly two players to allow a game to proceed, and games like Warcraft or Halo require between 2 and 16 players.  A typical social game has players in the thousands – Mafia Wars has membership in the tens of millions.

The second difference is in the length of engagement.  A Counterstrike match  is generally around 35 minutes, and a game of Warcraft 3 can last between 30 and 120 minutes.  Mafia Wars and Pet Society offer a single, persistent game experience which lasts indefinitely, often with players racking up 30 minutes or more every day of the week.

Options

The differences between normal videogames and “Social Games” do close off some possibilities for how to establish a shared experience, so while we can’t rely on symmetrical play patterns from a core set of players, we can make use of much longer time-scales and a vastly larger pool of players to draw on.

Rather than a single, one-to-one sequence of interaction, social games can make use of the wider network of contacts – engaging in multiple parallel ‘quests’, each one requiring roughly equal participation from its members, where the number of quests allows the player to scale their level of involvement.  If I play 2 hours a week, I might participate in 3 quests,  where someone who spends 10 hours will join 20.

Another approach is to let the game play itself – or at least to for friends to interact with a simulated version of the player,  which is refined over the course of their play-time.  Say I’m predictably good-aligned in most games (I am.) – If a friend of mine requests my assistance on a mission which requires some morally dubious action,  It should be possible for my approximated self to decline any requests to participate.  If it is necessary at some later date to request the services of a character with such dull and unflinching moral fortitude, My virtual character may offer his services up (or at least ask me directly whether to do so).

That’s weird.

While breaking the convention of a game following a single, coherent thread, or providing a simulated copy of me and my behaviours sound like strange approaches, this is essentially already happening in these social games.  Many avid “Social gamers” are registered in several services – and often several accounts – suggesting that thy feel that the maximum level of engagement available under a single game / account is insufficient.  Likewise,  any time a character bearing my name, experience level and portrait appears in a social game, it is effectively a simulation of me.  It just so happens that the simulation is  so poor that no two characters with the same level of experience can be considered behaviourally distinct.

What do we get out of this?

Using either (or both) of these approaches, we can come closer to having an actual social game experience.  By relaxing the requirements for symmetrical participation, we can play alongside a number of friends – and be playing with them, rather the  poor approximation we have now.  By relaxing the requirement for synchronous play through improving that approximation, we open up new opportunities for participation.

Not the most advanced CSS experiment by any stretch of the imagination, but it’s a start!

The following is the outcome:  Make sure you click inside the window to get it started, the keys are as follows:

Q: Toggles making the camera turn with the snake, so the head is always pointing down.

W: Toggles moving with the snake, so the head is always center screen.

E: Toggle turn amount (90º or continuous)

R: Get points (increase length)

Arrow Keys: Turn snake.  In continuous-turn mode, or in rotate-with-snake mode, left and right turn CCW and CW,  and in right-angle,  non-rotate mode, the arrow keys behave like a normal snake game.

What do you think of it?  How do the changes in presentation affect your feelings on the game?  Any reactions would be most welcome.

Personally, I’m pretty impressed by how different a follow-mode snake feels.  You could imagine navigating a much wider world, or even a looping one where subtle differences creep in over time. Making the game focused on the player (rather than the level) makes me think of it as a much more character- or story-focused environment.

I was also very surprised by just how sickening the motion is in first-person mode.  After messing around with it for about three hours today,  I think I’m going to need a lie-down.

LSO (Wikipedia) stands for ‘Local Shared Object’, and is a native  interchange format which is integrated into flash at the very core.  It’s also very old – They first appeared in Flash Player 6 / Flash MX,  so they enjoy near-universal support.  You create and edit them by calling

var lso:SharedObject = SharedObject.getLocal("local","/");

lso.data["message"] = "Hello, World (of Local SharedObjects!)";

The parameters here, “local” and “/” are less important than maintaining consistency.  They refer to how the locally-saved SharedObject data will be stored – something we don’t really need to concern ourselves with.    All we need to care about now is using this lso.data object as the container for the persistent data we want to store.  If a user leaves your website or closes the flash player window, the data is still stored in the same form as it was prior to closure.  It’s locally stored per-user on a machine so if the same user goes to the same window, then the flash files will have access to the same LSO.  The Wikipedia article suggests that some have gripes with this, since it’s a data store that isn’t flushed along with the other private data in a browser, but as long as you’re using it for good then it’s not the end of the world.  This has the upside that there is more limited room for abuse of things like number-limited offers – since nobody really knows about what LSOs are and how they work,

The other great thing about this system is that the objects are shared, so that any flash files (again, on the same domain and for the same user, to prevent misuse)  will have access to the same data.  So in FuelYourSenses I have about 5-6 separate .SWF files communicating with one another indirectly – told about new data through Event dispatches, and then reading the new data from these agreed-upon Shared Objects.

RTMP (Wikipedia) : Real-Time Messaging Protocol

RTMP is another one of Flash’s native tricks for multimedia streaming.  It’s a proprietary protocol, which means in order to get the full functionality out of it you need to buy the Flash Media Server, but for our purposes we were happy to use the very excellent opensource version or the Media Server,  Red5.  It’s a Java-based implementation that apparently only contains a portion of the full Media Server functionality, but it was more than sufficient for our needs.  If you’re interested in Media streaming (Audio, video, both or neither(!)) then it would be well worth your time to download Red5 and start experimenting.  The Demo files show you all you need to know to start recording and rebroadcasting streams as soon as you can get a server that you have permission to run a Java application on.  Once you have a video feed up, say, from a webcam or a locally stored .flv file, you just publish to the RTMP server and you’re done!  I was shocked.

EC2 (Wikipedia, Official site):  Elastic Compute Cloud

Amazon EC2 is a place where you can hire a (virtual) server by the hour.  You have full and unfettered access to machines that run either Linux or Windows,  with a number of pre-installed options for software provided by Amazon and other third parties.  We were initially unsure about the level of traffic the site would attract, whether one server would be sufficient or whether we would need to scale up.  Because of this, and the entirely reasonable concern that QTel had of letting a rogue developer have unfettered access to a server in their data center,  we decided early on to manage the audio/video streaming with a solution we could 1.  have complete control over without freaking anyone out, and 2.  scale without having to divert existing servers or purchase new ones.

EC2 isn’t the easiest thing in the world to work with, but there’s a lot of good help out there.  We learned a lot from this very useful Youtube tutorial.  We did as the tutorial says and used ElasticFox, Amazon’s Firefox plugin, to manage the whole process – without it I think it would be just about impossible.  Between that,  Amazon’s documentation and our undergrad dabbling in Linux we were able to select a flavour of Linux, install Apache, Java and Red5 .  The default installations are seriously minimal so you’ll have to install a lot of things that you would have expected to come built-in.

I hit an irritating stumbling block in getting a Linux instance up and communicating with my Windows desktop:  if you’re using puTTY rather than a straight SSH command in OS X / Linux, you’ll need to convert the keypairs as generated by Amazon,  into puTTY’s native format for handling credentials.  This is done with puTTYgen.  Aside from that and making sure that you have opened the appropriate ports for SSH and other traffic to the server, it was relatively straightforward.

All in all it was a fascinating project to jump into,  and I’d like to thank ADabisc and QTel for the opportunity to work on it.  The site went live on November 22, and after a pretty modest start it is now up to 10,000 Pageviews, and has broken 1,000 Daily views for the past 3 days!  Not bad for the 16 days or so it has been up.  My blog has had 1,800 pageviews in the 14 months it has been online,  so it has been by far the mos-viewed piece of work I have ever created.  The numbers are exciting but bring with them their own worries.

A lot of these guys have flickr accounts that show off some of their more entertaining screenshots as well, so I browsed through them too – it’s equal parts amusing and disturbing that the straight-up photography that we do*  ends up so strikingly similar – the sun at a certain angle, a piece of bark which must have looked spectacular at the time but looking somewhat nondescript on screen. I think there are some recurring images and patterns that we just like.

Anyway, I digress.  On Quasimondo I found a fantastic and reasonably lightweight face detection library.  I’ve been hankering after one of these for years.  I tried it, and it works reasonably well – better than I would expect in flash, certainly. This must be the lib that oaxoa used in his Rorshach mask – Mr. Doob too.  They both applied a large opaque decal on top of the face region, which looks great.  I wanted to try something different though, so I started to think about what you could use this face data for.    It might be possible to get some Johnny Lee stuff happening – I’ll need to play around with it a little more to decide if it can hold up to that – but what I ended up with is a 3d variant of Worm or Chopper or whatever you want to call it.  Here’s how to play:

Navigate through the space and collect as many of the tokens as you can – They were going to be carrots, and I started going down that road before realizing it was something of a Maxi-min, so they’re stuck as orangey-red triangular pyramids.  Here goes!

It’s not the smoothest of interfaces, but I’m sure it can be improved upon, even if I have to drop Flash for something more robust.  I recently noticed that the NeHe openGL resources are still being maintained, which is extremely tempting – especially since working in FlashDevelop has made me much more comfortable  with Visual Studio and “Real Programming”, although Ian has tempted me greatly with the promise of a genuine visual programming environment in Quartz Composer. We’ll have to see once I get some real downtime.

What do you think?  What would you do with this new face-detecting superpower?  I’d love to know!

* I consider myself part of this fraternity of flash dabblers, or would at least like to join them in due course.  I must learn their secret handshake!

One thing I hadn’t thought of was the fact that, on a laptop with a monitor-fixed webcam, in a room where the monitor is the only light,  The relief map does a reasonable job of being a depth map!  The luminance might map to the square of the height or something but you can see from the image above how well it works.

You can try it for yourself if you like – Note the usual applies with the webcam… If you can see what it’s like in a darkened room, do try it! I’d love to see what other people come up with!

(mouse controls rotation, buttons toggle params – try the relief map with the original bitmap laid on top for freaky 3d-ness!)

Update 2009-12-26

It is possible to make a 3d depth map in realtime, although there are a couple of hurdles to jump through. Even on a reasonable computer, the highest poly count you can expect out of Flash at present is about 5,000 triangles, so the resolution of the mesh used in this Flash experiment is 64 x 48.

If you were to do the same thing in a language that gave you access to DirectX or openGL you can expect to have poly counts in the millions, so one vertex per pixel is an easily achievable goal.

The next part is more complicated: Getting a color channel to look like a depth channel. The experiment above is using the red channel as the depth map. This will yield a real depth map, but only under these two conditions:

1. The light must originate from (close to) the camera. That way, the brighter a point is, the closer to the camera it is. For objects that are reasonably distant – say, 50 cm away, it is sufficient for the light to be within 5cm, but the closer the lights are, the better. You can buy some webcams with a built-in ring of LEDs – these are ideal.

2. All the objects must exhibit Diffuse / Lambertian luminance. Not as complicated as it sounds: Basically, when an object has a mirrored or a shiny surface, its brightness depends on both how it is lit and the camera position (where you are looking at it.) When you change your point of view, you change the brightness of the surface. There are commercially-available Lambert materials like Spectralon, but an unfired white clay will do just as well.  In fact, you could probably dust an object in any white powder – something like flour or baking soda,  light it under the right conditions and see what it looks like.

If anyone tries this I would love to see the results!  Under the two conditions above,  you should be able to use my mesh experiment above to make  meshes that show depth.  If you’re interested in some of the lower-level stuff in this subject, it’s part of Computer Vision, specifically Shape from Shading. As with any vision project, the best port-of-call is the openCV project.

Click in the panel to get focus, then use left and right to steer your accumulation buffer-derived snake to victory.  Good luck!

Remember that if your camera isn’t showing up, you can open the settings panel to make sure it’s enabled and that it’s the right camera. On Macs especially, there are a number of cameras installed by default and it might not be auto-selecting the right one.

Also, only one program is allowed to have the webcam at a time, so you’ll have to make sure that nothing else is using it.

A little explanation:  The top left is the raw video feed, the top right is the difference between the current frame and the last one.

the bottom left is an accumulation of those differences over the past ~30 frames, and the bottom right is the actual effect – try moving around to see how it works.  The button in the middle lets you toggle between using the difference or the accumulated difference as the displacer.

Enjoy!