Big Data

For the last few decades, it seems like I work on a project every few years that stretches my expectations of how much data can be effectively processed (starting in the 1980s: processing world-wide seismic to detect underground nuclear explosions, all credit card phone calls to detect theft, etc.)

I was in meetings three days last week with a new customer and as we talked about their current needs I made mental notes of what information they are probably not capturing that they should because it is likely to be valuable in the future in ways that are difficult to predict.

To generalize a bit, every customer interaction with a company's web sites should be captured (including navigation trails through web sites to model what people are personally interested in), every interaction with support staff, every purchase and return, etc.

Amazon has set a high standard in user modeling with Amazon suggests for products that you might want to buy. Collecting data on your customers should not make them feel creepy about interacting with your company but rather should make them feel important and well-serviced.

I am a voracious reader both for fun and to continue a life long education. I have found myself shifting some of my reading attention from computer science to statistics, math, and general business intelligence. (That said, I still read several computer science books per month).

I was disappointed that I could attend the Strata 2011 data conference but I did enjoy watching the keynote speech videos: fairly good material and worth some viewing time.