By Kathryn Korostoff | founder of Research Rockstar | @ResearchRocks.

At this week’s AMA Market ResearchConference, I saw that one of the Monday sessions was titled, “Social Media and Research.” Now that’s a title that piques my interest. Unfortunately I was unable to travel this week, so I decided to follow the twitter stream to pick up any nuggets this session would offer.

But first, let me as you a question. When you hear the phrase “Social Media and Research,” what comes to mind?

·      Are you thinking about brand and sentiment monitoring tools like TNS Cymfony (http://www.cymfony.com/), Nielson’s BuzzMetrics (http://en-us.nielsen.com/) and Crimson Hexagon (http://www.crimsonhexagon.com/home/)?

·      Are you thinking about how social media sites are a forum for discussing market research topics?

·      Maybe you are thinking about the rich community platforms that brands use to facilitate customer feedback?

·      Perhaps you think about how company blogs can be part of internal market research portals—to further the sharing and application of market research results?

All of these are great examples of how social media can impact market research.  And each could easily be the topic of its own conference.

So I was intrigued when I saw that the Twitter stream for this session focused on conversation scraping. Now I am talking about Twitter here—I am sure the session covered other topics, but this was the angle that the crowd twittered about…so obviously it had impact.

Conversation scraping is part of the overall web scraping phenomenon that has been around for a few years now.  You may have seen articles about web scraping being used by law enforcement professionals to help solve crimes. The software for web scraping involves some scripting functions that find and capture specified Web content, and stores the scraped data in a database.

Imagine focusing such a tool on your brand name or your product category. Imagine using it to focus on specific competitors. You could also design the process so that the scraping focuses on the web sites you have already qualified as being relevant to your target market—so that you are scraping from a high-quality pool of conversations.

Obviously this is related to the powerful analytic tools from Cymfony, BuzzMetrics, Crimson Hexagon and others; these firms tend to promote their analytics and dashboards—the stuff that makes all of the raw data actually useful. But there are many other tools available for people who are willing to forego the analytics for a simpler focus on the manipulation of what is scraped, and how. But remember: you get what you pay for. There is a lot of spam and other low-quality web content—so beware of what you scrape and how you enforce quality.

The bottom line: scraping and analyzing conversations from social media sites is certainly feasible today using any of a broad set of tools—from the no-frills variety all the way up to business-ready, dashboard metrics generators.