[ Drew Conway and Jake Porway of Data Without Borders. Designated the "James Bond and Harry Potter of data" by Alistair Croll -- photo by Pinar Ozger for O'Reilly Media ]
The first keynote of day 2 at the Strata Conference leads with a rather pointed question: with the data revolution upon us and unprecedented analytical power at hand, are the results of our pursuits personally fulfilling? Even programmers and data scientists with the best intentions often seem to end up developing tools that "make already comfortable lives ever so slightly more comfortable." Meanwhile, NGO's and nonprofits with decades of first-hand experience (not the kind you can google), are clambering for technical expertise to make use of their information resources. Essentially, what Jake Porway and Drew Conway of Data Without Borders have done is put two and two together.
This type of collaborative model is already in play for the United Nations team that built Hunchworks, a platform to post and verify hypothesis about potential crisis emerging worldwide (no affiliation to DWB). Chris van der Walt is communications strategy, Sara Farmer manages the systems and technology, and Dane Peterson brings in essential UX ethnography skills and hand-drawn powerpoint charm. Both initiatives make a smart decision about minimizing participation costs -- Hunchworks through crowd sourcing, while DWB harnesses good will and free time to participate in weekend-long "data dives," which place computer programmers, data analysts, and researchers together at the same table.
[ Hunchworks interface ]
For those few whose hearts are not moved by pure motivation to do public good without compensation, perhaps the announcement of the Heritage Provider Network $3 million prize caught their eye. Within 3 years, the goal is to develop a predictive algorithm that could identify patients who will be admitted to a hospital within the next year. The size of the reward should be telling of the seeming impossibility of solving this entrenched social, biological, and densely bureaucratic problem, which makes me think this is exactly the kind of challenge to set.
The winner of the Tableau data visualization prize, Steve Wexler, made his own public health contribution with a rather cheeky interactive dataset showing STD cases in Texas, proud purveyor of abstinence-only sex education:
[ Stills of Steve Wexler's winning data visualization ]
The Tableau contest's crowd favorite, John Boeckenstedt, gave form to some interesting trends in higher education:
[ Cost of attendance (x) and admittance rate (y); orange = public; blue = private; circle size corresponds to percent of undergraduates receiving institutional aid. ]
At a more intimate level, Anne Wright, of BodyTrack, shares her engineer's approach to self-diagnosis and better health when all the conventional tests fail to. Her experiences with explorable data for projects at NASA translated into the experimentation with several monitoring devices to self-track her daily behaviors and identify what was making her feel unwell. Although some of the tools and graphic models were rough around the edges at first, it shows the power of collecting your own observations and modeling it to explore trends in order to find the stories to tell yourself.
Enlightening as the speakers were, the spaces in-between formal presentations were often an even more captivating scene: business cards changing hands, a conviviality possible only around like-minded people, and a few "what recession?" jokes cracked in good humor. Big sponsors and private businesses held a dominating presence, but a number of people I spoke to represented nonprofits and academic institutions: a representative from UNESCO trying to better engage audiences and communicate education data for the Millenium Development Goals, or a history professor from UC Berkley who wants to help the social sciences catch up with big business statistical analysis.
Having seen a slice of what "big data" can do for the business community, I look forward to watching these public interest groups adopt and grow with the same tools.