by Cindy Atoji Keene
The data scientist has been called the sexiest job of the 21st century. The emergence of this role – an elusive blend of software engineer, statistician, and business analyst – is akin to a cowboy trying to harness the wild frontier of “Big Data.” As organizations wrestle with the volumes of information generated by mobile devices, social computing, the cloud, and more, data scientists help turn data into insights that companies can use.
Big Data will add 1.9 million IT jobs in the U.S. alone by 2015, with a majority of those hires happening in Boston and Silicon Valley. Some of those data scientists are brilliant, quirky and nocturnal, said Andrew Schwartz, Lattice Engine’s co-founder and data scientist, who lives on a sailboat on Boston Harbor; rides a bright green recumbent bike to work, and does his best work between midnight and 4 a.m. “To really understand a problem, it helps to be completely isolated with my computer. I have to sit and stare at it because it takes me a while to get everything in my brain. The longer I can work and understand the problem, the more productive I am,” said Schwartz, who created Lattice Engines to help businesses make more informed decisions about sales and marketing opportunities. Companies, for example, can make predictions about which customers and prospects are most likely to buy products, based on data about purchase history and customer service records, as well as outside sources such as LexisNexis, tax records, and other databases.
A: Is there an art to data science?
A: Data science is a mixture of business analysis, analytics modeling, and data management, and you have to know how to look at large amounts of data to help businesses gain a competitive edge. It’s like a scientist painting a picture: you have to do some analysis and sketch out what the main figures are, while also teasing out the fine-grain details, like choosing your colors. A data scientist needs to be methodical but also make guesses.
A: Data scientists can look at data and spot trends. What trends have you discovered?
Q: In a previous company, for example, I worked to help a casino optimize their slot machines. We did some controlled experiments to see how customers responded to payoffs, or winning the jackpot. We discovered that the size of the ‘take’ is not as important as the frequency and distribution, meaning that people are very responsive to hearing someone else win, even if it’s just a small amount of money. They are more likely to stay around and play if they hear other people hit the jackpot.
Q: How did you become a data scientist?
A: I have a Ph.D. in Math from Harvard. Like everyone else, I went off and taught. I was standing in front of a calculus class of 150 students when I realized that none of the students really wanted to be there, and no one would ever use what I was teaching. After classes ended, I went sailing for three months with a programming book, and by the end of my trip, I knew how to program.
Q: What are some buzzwords among data scientists?
A: Hadoop is a system to put break up problems and put together results; Hive is a tool to make queries and build up a particular analysis. You also might be using SAS or R as a statistic library. The other big buzzwords are machine learning, regression analysis, and a nice one, bagging and boosting.
Q: What’s the future of the data scientist?
A: We are right now in the early stages of data science, so for the next five to 10 years, we’re going to be seeing a lot of projects opening up that are addressing the types of problem that you need Big Data to handle. In the next 15 years, as the basic footprint of Big Data spreads into a range of domains, then it will be time to find commonality and pull together a single Big Data optimization platform.
Q: Give us an analogy of how big Big Data is.
A: Big can be very big. It’s equivalent to the total of every email sent or received by Gmail for the past four years; or every financial transaction by every bank; or a reading of all the information on every person on every flight in the US for the past three years.
Q: Are you using data analysis to buy a new boat?
A: Yes, I am looking for a slightly bigger boat, so I’m following the classified ads and doing a data analysis for pricing, including how long a boat has been on the market, and the relationship between price, space, and retail value. I’m spending all my time trying to correct my models of analysis. My boat broker hates me.
The author is solely responsible for the content.
about this blog
e-mail your question
Meet the Jobs Docs
Patricia Hunt Sinacole is president of First Beacon Group LLC, a human resources consulting firm in Hopkinton. She works with clients across many industries including technology, biotech and medical devices, financial services, and healthcare, and has over 20 years of human resources experience.
Elaine Varelas is managing partner at Keystone Partners, a career management firm in Boston and serves on the board of Career Partners International.
Cindy Atoji Keene is a freelance journalist with more than 25 years experience. E-mail her directly here.
Peter Post is the author of "The Etiquette Advantage in Business." Email questions about business etiquette to him directly here.
Stu Coleman, a partner and general manager at WinterWyman, manages the firm's Financial Contracting division, and provides strategic staffing services to Boston-area organizations needing Accounting and Finance workforce solutions and contract talent.
Tracy Cashman is a partner and the general manager of the Information Technology search division at WinterWyman. She has 20 years of experience partnering with clients in the Boston area to conduct technology searches in a wide variety of industries and technology.
Paul Hellman is the founder of Express Potential, which specializes in executive communication skills. He consults and speaks internationally on how to capture attention & influence others. Email him directly here.