[arg_discuss] [meta] Stats on ARGs

[adam]

Christy wrote:
 > Hello All,
 >
 > I've recently posted an article about stats on ARGs. It is a lengthy
 > expansion of my previous short and somewhat confusing post. I think this
 > data, plus Adrian's stats, provides some helpful information about ARGs.

Love the idea, and I think it has the potential to be as valuable as 
MMOGChart. But ARGs are not MMOG's ;). And they seem to be a LOT harder 
to quantify :(.

So...the first question is: what are we trying to achieve here?

When SirBruce started the MMOG chart (which now even has its own website 
- mmogchart.com), it was looking at a very homogeneous genre/revenue 
model where the number of bona fide subscribers could be directly 
equated to several things: popularity, reach, commercial success (all 
cash came from subscriptions back then).

One thing it's been particularly valuable for is quantifying the number 
of people who play MMOG's and the extent to which the subscription model 
is sustainable in the long-term. It has provided a running commentary on 
the issue "do new games create new lifelong subscribers, or do they 
steal them from existing games?".

(the answer has been tending more and more powerfully towards "yes they 
create new MMOG-players", in case you were wondering)

Sadly, given the many directions ARG's have expanded in simultaneously, 
it seems it just ain't that easy for us :(...

Personally, I like the emergence of ARG-to-ARG comparisons because I 
hope they'll soon fulfill several uses:

  - comparing what works and what doesn't as measured in footfall 
(rather than, say, how raving the reviews were on Unfiction)
  - comparing the pareto penetration of ARG's into disparate markets 
(e.g. do Audi buyers flock to forums but shun playing the actual game, 
where Unfiction players flock to the game but shun non-unfiction forums?)
  - providing investors and business partners with cold, hard, facts on 
the successes of previous ARG's, in undisputable figures (they're 
accustomed to being given technically correct but grossly misleading 
figures, and tend to reject anything you don't/can't substantiate)
  - a metric to measure your performance against others in the market - 
both to satisfy commercial partners and just for personal gratification

Perhaps the basic need here is to measure the reach of an ARG in each of 
the main media in which it operates, and even within each media to have 
a breakdown by several measurements (pareto comparison), as Adrian 
suggests, (http://www.mssv.net/archives/000705.shtml), e.g. for web:

  - traffic ranking (approximately verifiable through alexa)
  - google hits ("gives a good indication of buzz", as Adrian put it)
  - number of sites (verifiable through domain lists from player sites)
  - unique visitors overall (verifiable by trusted partners only, 
through httplogs and decent log-analyzer software)
  - unique visitors per day, avg (verifiable through httplogs to trusted 
partners)
  - unique visitors per day, peak (verifiable through httplogs to 
trusted partners)

No, I don't think those are great measures. But the first three can be 
done publically *for any site*, with or without the PM's involvement, 
and are better than nothing. (the last three I added as wishlist items - 
if I were evaluating someone else's ARG on behalf of a potential 
business partner, those are the first three I'd ask for, under NDA ;))

I think verification is particularly important with these measures. The 
MMOGChart initially was just taken from press releases from exuberant 
marketing depts - who for various legal reasons I believe were not 
allowed to lie, representing as they were large assets of public 
companies? - but could be approximately checked by any subscriber, using 
empirical data from the game servers that, in almost every game, 
broadcast the number of current players.

They needed to know the ratio of players online to actual subscribers, 
but figures for this were regularly thrown about by different developers 
and tended to be similar across all games (Lineage being the only 
notable exception IIRC).

> 
> Perhaps this information can be developed by the members of this list for
> the ARG SIG site? Also, please note, my intention with the list is to propel


So, I've been wondering whether the best way forwards is a matrix of 
measurements, one set for each core medium the game plays out through.

For instance, referencing several different ARGs liberally here ;), you 
might have columns for each of:
  - web traffic
  - radio-play listeners
  - online-game registered players
  - live-event participants
  - live-event spectators
  - forum unique posters (sum of unfiction + in-game forum - on the 
assumption most people will post to one but can't be bothered to 
simultaneously run the same conversations in the other!)
  - ...etc

> ARGs, not cause any offence to PMs. I hope I have not done the later.

A touchy subject, fraught with other difficulties too - such as: how do 
you compare something like ILB (strict time limit, coinciding with major 
international product launch) with something like PXC (onrunning game, 
picks up more players every month)?

e.g. the number-of-players figure for PXC has jumped another 20% since 
you furst looked :), and every time you post, the figure will be 
invalidated sooner or later :(.

For MMOG's it was easy - they were all aiming to be everlasting games, 
so it made sense to plot their figures on a graph of 
time-vs-subscribers; would similar charts make sense for ARGs? Could we 
even get month-by-month data?

(for some things, like approximate traffic, yes, thankfully. But that's 
one of the rougher measures in the first place :( ).

Adam