Data for the cause: here’s how big data is used to solve social problems

4 min reading
Data for the cause: here’s how big data is used to solve social problems
Data for the cause: here’s how big data is used to solve social problems


The education community would welcome more students graduating successfully from schools. Health professionals would like greater access to affordable health coverage. Environmentalists would like to balance energy consumption. And governments aspire to public policies with demonstrable results that meet the priorities of citizens. All sectors seek better performance, and there have never before been so many experts in data use and data mining to offer new and better insights to help them achieve it.

Last summer, more than 400 data scientists, government representatives, non-governmental organizations, business persons and academics met in New York during an event organized by Bloomberg, Data for Good Exchange, to analyze causes where Big Data is causing a positive impact, or may do so in the future.

Organizations like Microsoft, Johns Hopkins University, Facebook, Google, UNICEF, Planet Lab, MIT Climate CoLab, Cornell Tech, the NGO DataKind and the City of New Orleans accompanied Bloomberg, the financial intelligence giant, to share how they are encouraging the use of big data for causes for the common good.

How can Big Data impact health policy?

"Sixty-five percent of deaths in the world are not recorded. Not the cause of death but the death itself. And about a third of births are not registered," asserted Dr. Kelly Henning, from the public health area of Bloomberg Philanthropies, during the event. Steps are already being taken to consolidate a network of 20 countries, with major cities from around the world and with international organizations to register data on causes of mortality in detail and then do the hard part: use the data to make better decisions.

Dr. Henning explained that data are used in three ways: to establish priorities (e.g. to combat obesity, that "new tobacco" that is already a prominent cause of mortality in middle-income countries), to locate places where it can have more impact (such as Mexico, where there is already a pilot program to alleviate this problem) and to measure daily progress. "The data are not to provide a care service or to deliver health care directly. Our program is a policy shop”, she explained.

How is data mining used to optimize philanthropic activity?

What are the major foundations that fund health programs in Ghana? Is too much philanthropic effort overlapping on the same problem rather than diversifying it? Which NGOs should we partner with to have a greater impact on improving educational access for the Hispanic community in an American city?

The Foundation Center applies (and sells) Big Data to answer questions like these. This organization collects, cleans, analyzes and displays data of more than 140,000 foundations worldwide, of 3.6 million scholarships each year and of 372,000 recipients of funds. With this information, it develops products and payment services that, they say, "help other foundations to make strategic decisions about their financing, find partners and understand the impact of their programs on communities and sectors." 


Mobility, an area where Big Data is more consolidated

Waze, an Israeli navigation and mobility service acquired by Google, also presented its Connected Citizens during this event, which is an initiative to facilitate the free exchange of data between citizens and governments on traffic congestion and incidents.

The differential approach of this “panopticon for traffic” compared to other navigation systems is that it relies more on real-time data and on information generated by the user, and is already used in cities like Rio de Janeiro, Los Angeles and Jakarta. Of course measuring well whether this system to share data can be used to verifiably improve mobility conditions in the short term is another story.

Data for Good Exchange also served as an introduction to, a project that will be released soon. It is a dynamic panel to explore data on the city of New York on indicators (demographic, economic and welfare: health, housing, infrastructure, etc.) that are important to their communities. This initiative by Measure of America will have a narrative layer to answer questions like whether neighborhoods with a higher immigrant population have, for example, more or less access to preventive medicine, to take one case.

It also featured Detroit Water Project, a nonprofit platform that helps put more than 900 families from that city and from Baltimore who cannot pay their water bills into contact with more than 10,000 donors from around the world to make payments on their behalf directly to the public utility company. How do they use the data? To prevent families who depend on social handouts from their water being cut off due to defaulting on payments.   

Is the same "data science" valid for both profit-orientated and non-profit projects?

An analysis of 72 North American data science for good initiatives led to identify that all but three had the support of a federal agency or governmental body. This body either defines the problem or provides its own resources and data to then use the results, such as the New York Fire Department with this initiative.

Another study on Big Data for civic causes highlights that while private sector data scientists mostly use their own collected data as raw material – produced in-house by their companies – those who work for social causes tend to rely on external data and should be more cautious, because their data often have less control or imbalances.

In all cases, experts suggest that the factor necessary for the insights of these "data science for good" projects to have a real impact depends primarily on the extent to which the project is (from the outset) part of the decision-making process of institutions that are already having an impact. This would apply to projects such as Scraping Websites To Collect Consumption And Price Data, by DataKind for the World Bank, which is based on views to locally measure inflationary changes in Africa. Or the use that official bodies such as the Mayor's Office of Data Analytics (MODA) of New York or the Office of Performance and Accountability of New Orleans now make of big data to measure how their policies are performing.

To find more information on this subject:

– Definitive Guide to doing Data Science for Social Good

– Five principles for applying data science for social good

– Unity is strength: Big Data plus cloud computing


It may interest you