MIT prof knows Twitter trends better than Twitter


An MIT professor has developed an algorithm that can predict, hours in advance, when #Bieber will inevitably start trending again on Twitter, but the machine-learning algorithm could also have usages beyond social news — if the computational power is made available.

Associate professor Devavrat Shah and his student, Stanislav Nikolov, developed the new algorithm as a way to improve upon traditional machine learning, which starts with a hypothesis, and then tries to find matches to that hypothesis while testing how accurately the hypothesis tracks.

What makes Shah and Nikolov’s algorithm different, according to a release from the Massachusetts Institute of Technology, is that it’s nonparametric, meaning that the model structure is determined from the data, not by humans in advance.


Social media is an ideal test case for the data analysis, since what and when items are being published and measured are so clearly defined, but the Larry Hardesty writes that algorithm could be used to predict “the duration of a bus ride, ticket sales for films, maybe even stock prices.’’

Or, as more data-driven start-ups like Uber and AirBnB try to make the most of their own in-house big data, the likelihood someone will hail a cab or crash on your futon.

I reached out to Shah to learn more about potential applications, but you can ask him yourself next week: He’s speaking next Friday at MIT’s “News at Noon’’ session, which is free and open to the public.

Loading Comments...