We recently carried out a survey of conversation monitoring agencies as part of our work on a report for Contagious, some free extracts of which can be viewed here.
In recent years several companies have emerged to provide insights into the levels of online conversational activity around their clients. These companies use a widely varying mix of complex measurement technologies and processing methods to offer different approaches to conversation monitoring.
We surveyed 10 of the most prominent monitoring companies in the UK and US to compare the services on offer. The survey looks at each company’s background and successes, what it measures and where, how the resulting data is analysed and reported, and how much each service costs.
The full collection of responses from the agencies was too big to be published in the report so I’m taking this opportunity to give you access to it here. Click on the file name to be taken to a full-screen, downloadable version.
I’ve outlined below some interesting patterns and trends which we’ve drawn from the agencies’ responses. What is clear from the various processes, solutions and explanations is that this is a rapidly maturing method of research, and that the only sure-fire way of finding which supplier is right for you is to trial one or more of the services.
History and success
Our survey set includes a mixture of companies, some of which started before the dotcom boom period, and others that have sprung up since 2004. All of the companies have strong client lists, suggesting no run-away industry leader, although Nielsen BuzzMetrics’ response noted that it was named as the market share leader in the Outsell Social Media Market Intelligence Report, December 2007.
Methodology – measurement
All of the companies approach conversation measurement in slightly different ways and with differing levels of complexity. Some focus on explaining what they measure, others on how. One, Umbria, offers an additional feature that helps uncover trends in the genders and ages of those speaking by using NLP algorithms. As far as measuring influence in conversation is concerned, this again is done in different ways, but normally taking into account multiple factors. Measuring influence is a core part of the service offered by Onalytica, Magpie, Nielsen BuzzMetrics, VML SEER and 1000heads.
Methodology – analysis and reporting
All but three of the companies use fully automated processes to analyse data; Nielsen, MotiveQuest and 1000heads all draw particular attention to the additional level of human analysis as a core part of their product. 1000heads is the only company surveyed that performs more manual work than automated.
For data cleansing, all companies use at least part-automated processes but do so in very different ways. MotiveQuest, Nielsen BuzzMetrics, Dow Jones & Company, Umbria and 1000heads mentioned the use of additional manual filtering.
Where presentation of data is concerned, there is an even split between the use of offline methods, such as PowerPoint, and online methods, such as dashboards.
Coverage
Most companies monitor a varying but sizeable part of the blogosphere, as well as other selected social media environments (eg message boards and social networks). A number of companies (BuzzLogic, Magpie, Dow Jones & Company, MotiveQuest and Cymfony) draw attention to the fact that they monitor some degree of mainstream media. Onalytica and SEER go one step further and seemingly spread their nets over almost the entire web. Onalytica states that it monitors ‘everything freely and publicly available online’ while SEER states that it ‘aggregates activity from any online location that doesn’t require a password’.
Pricing
Costs are expressed in a number of formats, but the answers of those companies who offered a price for a single research dip range from £2,500 to £15,000 ($5,000-30,000) – with what you receive for your money varying by supplier. Only two suppliers work primarily on a subscription basis – BuzzLogic offering a yearly subscription, and Magpie monthly.
Hi Ben:
Thanks for the mention – happy to provide more information about MotiveQuest.
FYI we have developed a measure of online advocacy – and correlated it to sales. (What you called “referral statistics”) We can track advocacy related to a brand over time.
Tom O’Brien
MotiveQuest LLC
i know i’m on holiday and all, but the contagious report looks sooooo pretty…
Thanks Ben – good stuff and i agree with Dan – very nice looking report. Is this something you plan to do regularly? I think it would be useful as things are happening so fast in this area.
Best
Giles Palmer
Brandwatch (Magpie)
Hi Ben,
Let us know the next time you are updating the report as we’d love to take you through what Radian6 is doing in social media monitoring as well. We have a quickly growing number of agencies and now corporations using our solution in North America, Europe and Australia.
Cheers.
David Alston
Radian6
Giles – things are indeed happening very fast and it would be helpful to mark the rising tideline on a regular basis. This survey is part of the one-off special report for Contagious but that doesn’t negate the idea that we could run such an exercise again.
I see from Will McInnes’s blog that there is now a Camp advancing your recent discussions on the standardisation of social media measurement. This is really encouraging stuff. Benchmarking and standardisation are mentioned as key issues – hopefully we can assist each other in achieving these goals.
If we run another survey, I would be very interested to know what criteria might be reviewed beyond those we covered this time. Any suggestions would be most welcome.
You forgot KDPaine & Partners that has been measuring social media since 1995, before most of these companies were even a gleam in their founders eyes. We started monitoring newsgroups for Epson and Procter & Gamble in the mid 90s and now conduct both formative and evaluative research in social media for the likes of Georgia Tech, Georgia Pacific, Raytheon, Juniper Networks and Mothers Against Drunk Driving. Most of the people on this list are only doing automated tracking which is only about 75% accurate. Most coporations, including McDonalds, Microsoft and Facebook don’t feel that’ s adequat and therefore insist on a hybrid solutions of humans and automation.
Katie and David,
Thanks for introducing yourselves and I’m sorry we missed you from our radar. We whittled out a relatively short shortlist for the survey but even our full list could not have been entirely comprehensive.
There’s clearly some interest in a similar evaluation in the future. If we undertake another review we shall endeavour to start out with a wider set of companies.
Even with the ten agencies we looked at, it was very difficult to make clear comparisons between the offerings of each company in the survey, as they varied so much. Hopefully this will become clearer if the industry moves towards more common services and metrics.
Ben -
Thank you for considering the SEER methodology as part of your review! The report looks fantastic – comprehensive, and replete with create acronyms. I love it.
Since we spoke, SEER has begun to marry qualitative social metrics (I like to call them “squishy metrics”) with quantitative measurements from media and analytics, resulting in 360-degree optimization across client marketing plans.
An exciting time, integrating social media efforts with overall planning for marketers…
Camille Lauer
VML – SEER
“Squishy metrics”? Nice. So you’re attempting to nail the ever elusive perfect balance of crispy and chewy. It seems that the ultimate analysis would have to optimise both qualitative and quantitative data, filtered in some really natural way, like Google does for search.
[...] behave in social media, what the future might look like and a survey of the various online conversation monitoring options that are [...]
Hi Ben,
I’ve just found this post after publishing The Social Web Analytics eBook 2008 (www.socialwebanalytics.com). Wish I’d found it beforehand… it would have been great to include comment from your good self.
You’ll note from the title of the ebook that I’m planning to update it every year, and of course the conversation is ongoing, so let’s keep in touch.
Cheers, Philip.
Philip,
What a shame that neither of us turned up on the other’s radar while conducting our respective research. It would have been most interesting to have swapped notes at the time.
The conversation is indeed ongoing, in fact it’s hard to publish anything current as new tools and businesses keep emerging at an accelerating pace. Having taken a quick glance just now, I’m looking forward to reading your eBook. It’s interesting to see that you too have examined the free, public tools next to the commercial services, the atagonistic balance between the two is intriguing.
I’m sure we’d be delighted to discuss some possible input on your next edition.
[...] very thorough summary of the major players in the online conversation tracking market [...]
Ben this was great stuff. Once we’re up and running, I’ll send you our info for mediasphere360 from MediaBadger. We’re taking a slightly different approach in a different way…ah, enough said!
thanks again!
BrandIntel (www.brandintel.com) is another company/market leader to consider in this category.
hey ben – thanks for the roundup and the report snippets. very helpful. bit skeptical myself about some of the more qualitative tools, but am researching like mad and hoping to find the right balance of hard numbers vs fuzzier stuff that looks at reputation…. anyways – thanks again. keep up the darn fine work : )
Further to your excellent report, I’d point out Nathan Gilliat, an independent researcher and analyst who’s covered this space in great depth.
http://net-savvy.com/executive/
Interestingly, there are over 150 such monitoring companies worldwide, around 63 of them in the U.S. and Canada.
I think some interesting changes are looming towards monitoring; in areas as yet unconsidered.
Hi Ben,
This is indeed an interesting piece of work but missed out EmPower Research. We are a Media Services firm which provides for social media analysis as well as monitoring to corporates located in US, UK/ Europe and APAC.
Do let me know if and when you update your research. I would be glad to provide more detailed information on EmPower’s offerings.
Cheers.
Hitika
Ben,
Thank you for this posting.
We would be delighted to further explain what we do at Digimind for monitoring buzz, e-reputation in the event you intend to update your research.
Regards, Chris.
PS: We have a white-paper available about e-reputation that explains in length various techniques we use for analyzing and visualizing concepts discussed on blogs, forums and other digital sources.
Good post and thanks for the comparison table – v useful.
Best,
Jon M
http://www.theappleofmyi.com
[...] A slightly older but well-done and complimentary report (with pricing info) on 10 leading online conversation monitoring tool can be found on Ryan MacMillan blog. [...]
Hello,
I am French student of Paris, and I am doing a study about the best social media agencies in UK.
Does anybody here could help me to find any report or ranking about the 10/20 best agencies which practice : conversation monitoring, e-reputation, community management ?
Thanks a lot in advance !
Violaine