Imagine yourself naked. Put on some Enya and turn down the lighting. Relax.

Now, imagine sticking your head into a scorching hot oven, and simultaneously plunging your bare feet into a freezer. On average (or ‘statistically speaking’), you should feel just fine. Of course, this may not actually be the case. (Kids – please do not try this at home.)

Statistics, metrics and measurement are not the pure sciences they may seem to be.

So when it comes to ‘defining and measuring the success’ of your interactive or digital project, you need to think very carefully about where you put the goalposts, and indeed, if you even actually need goalposts. Is there one goalpost? Or several? Is there a hierarchy? Is there an order in which they need to be met?

Sometimes people have a very clear idea of what they want to achieve, other times they have given it little thought, sometimes they’ve defined success in a narrow or incorrect way, and other times, success is such an intangible term (some projects are just like that) that raw ‘metrics’ (in isolation) just don’t adequately determine the success or failure of a project.  

If you haven’t figured out what your project is about, slap on some Enya, lie back and figure it out.

Enya. Rumoured to be opening for Megadeth and Slayer in November. I've never seen our programmers so excited.

1. MARMOTS: First up, the easy ways to define success

Obviously, some projects have a nice, tidy, quantitative metric that is used to determine a project’s success. For example:

• Sales
• Leads (people who might become sales)
• Conversions (turning leads into sales)
• Subscribers
  

These are some of the more conventional business metrics that people will use when defining commercial projects such as an ecommerce or online shopping site.

Generally, they’re pretty easy things to measure. And there are plenty of diagnostic tools to help you do so.

Let’s say you’re in the business of selling marmots (you should be fully dressed by now, by the way). But let’s imagine that marmots aren’t so hot right now. They’re, like, totally 2008. Everyone wants wombats.

You want to increase marmot sales so you run a clever campaign to reignite public interest in marmots (“Brad Pitt has a Marmot! You need one too!”), you have an online call to action (www.getmymarmot.com) and you can measure how many people click on the ‘Buy Marmot!’ button and then complete a transaction. Bingo. You have a measurable campaign. It’s so neat and tidy that Virgos the world over are high-fiving each other (and immediately washing their hands afterwards of course).  

Brad Pitt. Sure, he scrubs up ok and has 81 kids. But does he have a wombat?

2. METRICS: Harder ways to define success

Google Analytics is one of the more popular tools that you can use to measure the success of your website.

As well as measuring your own leads, sales, conversions, subscribers etc, you can use Google Analytics to drill down into a bunch of other metrics such as returning visitors (as opposed to one-time visitors), page views per visit, time spent on each page, time spent on your website, common website pathways, referrals (where your users are coming from) and so forth that may help you to paint a picture of ‘success’. Plus it’s free, really comprehensive and is easy to use.

Google Analytics. The de facto standard in website analytics reporting.

You can also delve further into a whole bunch of additional Search Engine metrics which are really nerdy and tell you things like ‘Google bot net frequency’. This is basically a measure of how often the Google search engine Hoovers up your website and throws its vital statistics into their complex search engine algorithms and (ultimately) search results. Check out this post if you want to learn more about Search Engine metrics.

And finally, there are also all the lovely touchy-feely ‘social media metrics’, online reputation or ‘buzz’ metrics such as bookmarks, tweets, views, likes, shares, reviews, friends, followers, blogs, news mentions and other ‘moments of user-generated content’ (and distribution) that might be part of your success equation. If your project is all about ‘buzz’, these are the metrics you need to clearly articulate before you begin. Also check out Google Trends or (Twitter) Trending Topics which are other interesting forays into the levels of online interest currently circulating around any given topic.

More effective than a fire drill? Just mention ‘Bot Net Frequency’ at your next social function and watch people run for their lives in fear.

3. MAYHEM: What do I do with my Business Case?

So, where does this leave you when confronted with a boss, Minister or project sponsor who wants to know the ROI or success of your project? Confused? Tired? Excited? Afraid? Obviously, measuring success can start to get quite tricky. Take just one example from the slew of metrics and options canvassed above. How do you measure the impact of a single comment? Google Analytics may be telling you that your new website is awesome but there’s a comment from an influential user or blogger that says your website is about as appealing as wet bread. What is the value (or cost to you) of that comment? If you want ‘user generated content’ or ‘customer feedback’ as a result of your project, have you figured out who is going to collate and assess all of this material?

Sometimes you don’t even need to do anything to attract attention. There are plenty of examples of corporates vs citizens in PR disasters that have been ‘estimated to have cost millions in lost revenue’ etc. A while back, United Airlines mishandled a passenger’s precious guitar. The passenger penned a song (United Breaks Guitars) about the whole affair that was a viral hit. Recently, Microsoft bungled a product launch with a glorious Photoshop fail. If it’s getting harder and harder to define and quantify success, how do we even begin to measure failure?

You must define success – even if it feels flakey

If you’re spending money, it’s highly likely that someone, somewhere in your organisation is going to ask what happened to it. So make sure you complete this sentence and tattoo it on your forehead:

“This project will be successful when it has [done what?], [to/for whom?], [by when?].”

It seems ridiculously simple. But it’s amazing how many projects get underway without this task having been completed.

This can cover a wide range of scenarios. For example:

“My new online badger shop will be successful when I have sold 88 badgers to new customers by Christmas 2012.”  

“Our art installation will be successful when at least 15% of all exhibition visitors have attached a digital photo of themselves to the giant digital panda installation or videoed themselves doing the giant panda snuggle.”

“Our museum interactive will be successful when over 50% of all people who try it complete the mission to catch the falling marmots.”

Do you know what you are trying to do? “We need a new website!” is just not good enough. Nor is: “We need a cool new website.” Nor is: “We want something that’s really going to get people talking.” Are you: selling, informing, persuading, dissuading, encouraging, creating, sharing or simply telling?

If it is at the more intangible end of the spectrum, like ‘brand building’ or ‘telling people something’ or ‘providing public information’, that’s absolutely fine, but what do you want people who come encounter your ‘project’ to do about it. Respond? Argue? Laugh? Cry? Think? Upload? Download? Exercise? Write? Think differently? Again, it might seem self-evident, but try and put some parameters around it. You may be surprised at how it helps you to focus and clarify your project’s reason for being.

And if you think that woolly success measures like ‘laughing and crying’ seem absurdly weak when facing an ROI inquisition from a table-thumping CEO, consider this: way back in 2007 Sony Computer Entertainment’s (Europe) President, David Reeves, stated that one of his objectives for the new PS3 slate was to create games that “make people cry”. Now, I’m sure they had other measures of success as well – flogging a lot of units, for example. But, they were acknowledging that sometimes the intangible characteristics of success are the most enduring and the most powerful.

Again, from Phil Harrison (Sony): “We’re working towards it because we think it’s important to have an entertainment experience pressing emotional buttons.”

No, it’s not the launch of Vista. These are real live human beings having a memorable experience. Does your project warrant the inclusion of emotional metrics?

In my experience, if you can be functional, simple and artful all at the same time, you’re often on to a winner.

And now, after telling you how important it is to provide definition I wish to officially play the contradiction card, and tell you not to get too carried away with your metrics.

Design vs Data. Art vs Science. Marmots vs Wombats. Is there too much measurement?

Art vs Science. Why are they so often framed as polar opposites in the spectrum of human cognition? Da Vinci didn’t have an MBA. Or Photoshop. And never MCed at his mate's 21st.  

There is, of course, often a natural tension between the ‘cold hard truth’ of what the data is telling you (defining success), and what you, your users and your team believe in ‘your heart’ the correct course of action is. But when do you ‘trust your instincts’?

Luke Skywalker didn’t ask his Twitter followers if Princess Leia may have been his sister before he snogged her, he just went with his instincts. In hindsight, maybe he should have Googled it first. Anyway, back to my point...

Douglas Bowman was a leading visual designer who left Google after feeling that their obsession with data and analysis was killing any small amount of human creativity and aesthetic sensibility. His blog outlines the sources of his frustration (testing 41 shades of blue to see which one performs better etc); debates over whether a column should be 3, 4 or 5 pixels wide and so on. I can completely understand his frustration (and I’m no designer). In his words, “I won’t miss a design philosophy that lives or dies strictly by the sword of data.” He is now Creative Director at Twitter.

Interesting. It's hard to argue with Google and the volume of data that is used to justify everything they do. Douglas Bowman did. Apparently, he woke up with a horse's head in his spreadsheet. Nasty.

In a world awash with data, metrics, analysis, measurement and real-time feedback, nobody’s really figured out how to bottle success. And I hope they never will. People change. Technology changes. Content changes. Trends come and go. Expectations, expertise and competence are always in a state of flux. The best you can do is define success in an intelligent, meaningful, measurable and realistic manner.

And if you do these things, you’ll have a better chance of achieving it:

•    use the diagnostic tools that are freely available
•    talk to your users frequently
•    test and refine what you do
•    embark on projects that are small and scalable
•    prepare to fail
•    prepare to fix the things that will go wrong
•    read, listen and watch widely what others are doing and improve on it
•    surprise people
•    define success as best you can
•    define how (and if) you are going to measure success

Don't let your project be ruled by metrics. Make sure it's guided by them. Good luck.

Further reading: Top 5 Web Analytics Apps that are better than Googles Analytics - anyone out there using these?