RMM

Every day we find new and exciting examples of organisations using social technologies, but amidst the “wow factor” of engaging customers/prospects through funky widgets or clever campaigns, there remains that big question – how can we track and measure this stuff?

What’s often referred to as social media monitoring is of huge importance to us at RMM, be it to help our customers track shifting brand sentiment over time or gather quantifiable research to support a client pitch.

Therefore, to keep our finger on the pulse, we’re now in the habit of annually reviewing a selection of monitoring tools to understand what’s happening in the market and to ensure we recommend the most appropriate tools to our customers.  Our most recent review was conducted in December 2009.

This review takes the form of a comparitive test of a selection of the established, leading tools and new entrants to the market-place.  One of our clients provided a live test environment as they were looking to review their current buzz research partner. Our objectives were as follows:

• To understand how well the respective buzz tools meet our, and our clients’, needs
• To understand which tools provide the best value for money

The test helped confirm the following hypotheses:

• There are a number of ways in which different business departments can make use of a monitoring tool. Understanding the underlying objectives for monitoring activity is fundamental
• Different tools work best in different scenarios, depending on the client’s research objectives and project criteria (for example cost and frequency of use)
• A combination of tools often works best

More details after the jump.

We evaluated a range of factors for each tool, including the volume of results returned, capability to rank and influence and authority, reporting and the level of customer service we received during our trial, to help us compare and contrast.

One of the most frequent objectives of social media monitoring is to enable customer response to mentions of a brand, product or service – i.e. what’s happening right now?  We found that combining a tool such as Sysomos Heartbeats or Sentiment Metrics with a good old-fashioned Google Alerts and Twitter search will provide a scalable solution that scrapes a decent breadth of the web to help keep on top of what the world is saying.

For other clients, there’s a greater focus on comparison analysis and research across a wide range of topics, often looking back historically. For this, we found that tools such as SM2 Techrigy provide considerable flexibility in helping extract and monitor trends over time between user-defined categories.

Another use of monitoring tools, particularly for an agency like ourselves, is to support ad hoc research to provide a client with a snap shot of what’s being said about their brand, for example to support a sales pitch. For a cost effective “toe in the water”, SM2 Techrigy’s free service or Radian6’s seven day profile package (if the data is being used for pitch purposes only) can cater for such a requirement.

Of course, great functionality is one thing, but value for money is always something that requires careful consideration. We found two main cost models – based either on the number of search phrases that can be setup, or the capacity of search results. To make the correct choice, it’s again essential to ensure a clear understanding of the client’s needs and objectives and to run some testing of what search volumes are likely. For example, if the client wants to search under a large number of different terms we found BrandWatch, Techrigy and Radian6’s pricing models work well. When a client wishes to search a smaller number of high volume search terms, then Sysomos and Sentiment Metrics pricing models seemed better suited.

While the majority of tools come in at around the £350-£400 a month mark, we also checked out the budget end of the market with tools such as Viralheat. Such tools may lack the analytical and reporting capability of their more expensive counterparts, but at £10/month, provide far more structure and organisation of search results than a plain Google search.

Of course, it’s essential to bear in mind that these tools won’t bring in every single mention of a brand, product or service. As far reaching as they attempt to be, each covers only a proportion of the web, which means rather than taking results as absolute, one should look at extracting trends, comparisons and patterns.  A similar consideration should also be made regarding automated sentiment scoring. In our experience, without considerable manual input over a longer period of time, it’s hard to attain high levels of accuracy - the machines haven’t quite replaced human judgement just yet.

So, whilst we favoured certain tools in respect of value, flexibility or power of analysis, it’s the objectives of the organisation (or agency) which remain at the heart of the decision. We’ll certainly be keeping a close eye on new and developing products over the year, but we’d also like to hear about your experience with these types of tools.

Please note: a full version of our December 2009 report (with anonymous data) is available to our clients upon request.