The first day it was publicly announced it was linked on Reddit, and it ended up getting 1,600 visitors who collectively viewed  twenty-eight thousand pages.   By the end of the night, though, the system that had been used to manage it  had been overcome by a malicious script and the flow of the experience had been destroyed.  Over the next weekend my friend Aidan and I rebuilt a tighter  system using a database. It meant that we could limit the speed that people posted at, but that meant we could also tell who posted them. From there we decided that we could also let people vote on which posts they liked.

While the site lost the initial momentum from the exploit, the improved system captured people’s attention. In the four months it has been online, it has had 3,600 visits, but shown 55,000 pages and the average visitor stays on the site for five minutes. By comparison the average visit on a website is seven seconds – it’s hard to overstate how distracted the average internet user is.

The data that I needed to collect to make the system work also makes for intriguing analysis. With individual ratings you can see which posts are the most popular and which are the most contentious. You can compare the score distribution of all the posts with the subset containing a specific keywords. At present the corpus of data is relatively small so it’s hard to determine clear trends. I’m hoping to expand the system to other places to see how they vary. It’s going to be interesting!

The first two accusations are reasonable – there are a lot of stupid comments out there, and they are very repetitive.  They are so hated by some that someone has developed a Firefox Add-on that will strip comments out of the page before you can even be offended by them.  On the other hand, though, people are very stupid and repetitive.  There’s often a surprising honesty to the statements that people make on youtube. It is suggested that the monumental scale of the youtube ‘community’ means that individuals are effectively anonymous, which liberates users from any self-censorship that would occur from fear of shame or other punishment.  It’s not all honesty, since people are more likely to be offensive if they know they can get away with it, but it’s a refreshingly different place to look for views and opinions in.

So while they may be stupid, they’re still worth looking at.  The issue of quantity is more of a practical issue, though.  For example, I keep tabs on  a video called “The Most Pathetic Baby Panda Ever”. It has almost 5 million views and 15,500 comments.   Even if you only dedicated 5 seconds to each comment, you have to spend over 21 hours studying them. The Evolution Of Dance has almost 250,000 comments -   almost two weeks’ worth of study.  While they’re interesting, they’re not that interesting.

Conveniently, though, this is exactly the kind of thing that the study of data mining is supposed to deal with.  In addition, the fact that the comments appear to be stupid (or simple,) and highly repetitive works in our favour. The first question I would try to ask of these comments is “what’s a typical comment?”  and toward that end, I have created the Youtuber.