The next two to three years Big Social will still be about web-analytics, visitor tracking and making sense of all the data coming from social media feeds. Central to these activities will be the tracking of sentiment, identifying influencers and building rich data profiles and locating patterns in human behavior. But are we getting better at these activities and if so, in what way? Could we come up with a mathematical approach to analyzing humans? A formula that can accurately know our preferences?
In a paper by Alon Halevy, Peter Norvig and Fernando Pereira, the authors present a important argument in this context:
“Perhaps when it comes to natural language processing and related fields, we’re doomed to complex theories that will never have the elegance of physics equations. But if that’s so, we should stop acting as if our goal is to author extremely elegant theories, and instead embrace complexity and make use of the best ally we have: the unreasonable effectiveness of data.”
They argue and validate that if a dataset is large enough, it works just as a ‘actual’ formula. You could design a complex model to calculate how many people get the flue, however wading through Google search results delivers the same or even better results. Another example is a strategy for setting a price on used products. One could come up with a economic model that calculates the right pricing strategy for each product, but leveraging years of data from a site like Ebay probably is more accurate. In fact, if there is a need for text interpretation, one is better of performing a statistical calculation on the internet then trying to analyse sentences and deriving meaning of this interpretation.
This might sound trivial, but is means that data is capable of providing answers that until recently could only be found with the help of complex models. On top of that answers are now available without the necessity of knowing the model behind the answer. It sort of feels like using a calculator to solve the root of two, without knowing wat the root actually is. Is Big Social becoming smarter then all the sociologists in the world?