The data scene in NYC

I love the NYC data scene. It is vibrant, healthy and welcoming.

Eight months ago I left San Francisco and moved to New York. I had come very close to taking an offer in DC and had been struggling weighing the pros and cons of DC versus NYC. I had previously lived in DC so knew the area, Virginia schools are very good and the cost of living significantly lower than the New York area, all important considerations when you have a young family. Warby Parker (my current employer) had set me up to have a chat with DJ Patil who at the time was data scientist in residence at Greylock Partners. We had an honest and open discussion of these issues and he asked, "That is a great offer but what happens if it doesn't work out? New York has a rich and vibrant data scene. It is going to be stimulating and there are lots of opportunities." He was right of course.

DC has changed significantly since I left in 2007. There are many more startups and networks for entrepreneurs and Harlan Harris and co have done an amazing job bringing together the data community under the umbrella of Data Community DC. The reality, however, is that there are relatively few opportunities. The two big tech companies, AOL and Living Social, are both a mess. There are many other data-related positions, if you have security clearance, and the majority of other positions tend to be with small consulting firms that service the government and Department of Defense.

Contrast this with New York:

• New York is the place to be for advertising and media. 
• New York is the place to be for fashion. 
• New York is the place to be for finance. 

The tech scene is rich too and while I know that this is a gross oversimplification there is somehow something very tangible about the startups here. They are more likely to sell real goods and services, physical goods from Etsy, Rent the Runway or Birchbox. In other words, eCommerce is thriving. In addition, under Bloomberg's initiatives, the city is investing heavily in data science and statistics per se with new institutes and a campus on Roosevelt Island.

At the O'Reilly Strata + Hadoop World conference last week there was an interesting panel "New York City: a Data Science Mecca." On the panel were Yann Le Cunn (NYU), Chris Wiggins (HackNY/Columbia) and Deborah Estrin (Cornell NY Tech). Yann Le Cunn is the Director of the newly opened Center for Data Science, a multi- and inter-disciplinary research institute that plans to churn out 50 data science Masters students per year as well as host a PhD program, all of which will have strong ties to the local tech scene. Similarly, Columbia's new Institute for Data Science and Engineering will be hiring 30 new faculty positions, taking up shop in a new 44k square foot building and have an industrial affiliate program. Finally, Cornell will be moving to Roosevelt in 2015 with a broader program than just data science that covers computer science and operations research, all skills the feature in the data science world. The panel made the point that New York is such a great place to be for data because of the density of the ecosystem. On a tiny island with great public transport you have a huge conglomeration of finance, media and advertising companies, other organizations such as Mt Sinai (who recently hired Jeff Hammerbacher---the very person who coined the term "data scientist" with DJ Patil), a suite of world class universities who are investing faculty, buildings (highly significant given than land is precious) and have both research and training foci, and finally the city itself. Yann also made the point that the density of jobs and other organizations here make it easier to attract part-time students. 

The data-related Meetup scene is very strong too. (Meetup is based in NYC.) You could go to a packed and interesting data-or data-tech related meetup almost every night. DataKind are based here too. One of the most prominent data scientists, Hilary Mason (now data scientist in residence at Accel Partners) is based here. Strata just took place last week. In Fashion Week, 596 people attended "Fashion Tech: Demos & Drinks" that showcased the local fashion related tech companies. I couldn't attend. Why? Because I was attending a DataGotham event, another important data conference that brings the data community together. Later this month is pyData. You get the idea. 

NYC is a fruitful mix of data-research, data practitioners and a strong data community. I am happy to be both here and at Warby Parker. Oh, and the team that I would have joined in DC recently imploded. I dodged a bullet. Thanks DJ.