Big Data and Social Science: A Practical Guide to Methods and Tools (Hardback) book cover

Big Data and Social Science

A Practical Guide to Methods and Tools

Edited by Ian Foster, Rayid Ghani, Ron S. Jarmin, Frauke Kreuter, Julia Lane

© 2016 – Chapman and Hall/CRC

356 pages | 60 Color Illus.

Purchasing Options:$ = USD
Hardback: 9781498751407
pub: 2016-08-09
eBook (VitalSource) : 9781315368238
pub: 2016-09-15
from $49.95

FREE Standard Shipping!
Comp Exam Copy


Both Traditional Students and Working Professionals Acquire the Skills to Analyze Social Problems.

Big Data and Social Science: A Practical Guide to Methods and Tools shows how to apply data science to real-world problems in both research and the practice. The book provides practical guidance on combining methods and tools from computer science, statistics, and social science. This concrete approach is illustrated throughout using an important national problem, the quantitative study of innovation.

The text draws on the expertise of prominent leaders in statistics, the social sciences, data science, and computer science to teach students how to use modern social science research principles as well as the best analytical and computational tools. It uses a real-world challenge to introduce how these tools are used to identify and capture appropriate data, apply data science models and tools to that data, and recognize and respond to data errors and limitations.

For more information, including sample chapters and news, please visit the author's website.


"This book builds a nice bridge connecting social science and big data methodology. Big data such as social media and electronic health records, empowered by the advances in information technology, are an emerging phenomenon in recent years and present unprecedented opportunities for social science research. This book was written by pioneering scientists in applying big data methods to address social science problems. As shown by numerous examples in the book, social science could benefit significantly by embracing the new mode of big data and taking advantage of the technical progress in analysing such data. If you work in social science and would like to explore the power of big data, this book is clearly for you. Indeed, if you do not have previous experience in dealing with big data, you should read this book first, before implementing a big-data project.

As indicated by the title, this book acts as a practical guide and targets readers with minimum big data experience, hence it is very hands-on. … It covers all necessary steps to finish a big data project: collecting raw data, cleaning and preprocessing data, applying various modelling tools to analyze the data, evaluating results, protecting privacy, and addressing ethical problems. … All the important topics concerning big data are covered, making this book a good reference that you should always keep on your desk."

— Guoqiang Yu, Virginia Tech, in Journal of the American Statistical Association, July 2017

"…In summary, although there is a growing number of books related to social science and big data, this volume contains several non-trivial aspects which make it worth to have in the library, possibly along with other similar textbooks as a good complement to them."

—Stefano M. Iacus, University of Milan, in Journal of Statistical Software, June 2017

"This is a well-written book and showcases a good number of examples and applications to demonstrate how the methods are actually used in real life situation using real datasets. Further, topics at hand are motivated by social science data. … The chapters are nicely structured, well presented and motivated by data examples. The main strength of the book is that it still offers a good number of applications that are based on real datasets emerging from social science perspectives. The book will be useful to students, practitioners, and data analyst in the respective fields. The editors did a very good job introducing the book, it aims and goals, intendent audience, clarifying underneath concepts and phrases, a must read before moving to other chapters."

—S. Ejaz Ahmed, in Technometrics, April 2017

"Economists and Social Scientist have a lot to learn from Machine Learning, and Engineers have a lot to learn from Econometricians and Statisticians. This two way sharing is long overdue and it is time to start the conversation. This book is a tour-de-force for anyone interested in participating in such a discussion."

Roberto Rigobon, Society of Sloan Fellows Professor of Applied Economics, MIT

"This ambitious sweep through data science techniques provides an invaluable introduction to the toolbox of big data methodologies, as applied to social science data. It provides tremendous value not only to beginners in the field, but also to experienced data scientists wishing round out their knowledge of this broad and dynamic field."

Kenneth Benoit, Department of Methodology, London School of Economics and Political Science

"Most social scientists would agree that ‘big data’ – the term we use to encapsulate the huge amount of electronic information we generate in our everyday lives – provide the potential for path-breaking research not just into our economic, social, and political lives but also the physical environment we create and inhabit. However, few have the knowledge, or critically, the tools that equip them to realize this potential. This book provides a bridge between computer science, statistics, and the social sciences, demonstrating this new field of ‘data science’ via practical applications. The book is remarkable in many ways. It originates from classes taught by leading practitioners in this area to federal agency research staff, drawing in particular upon the example of a hugely successful project that linked federal research spending to outcomes in terms of patents, job creation, and the subsequent career development of researchers. By making these workbooks accessible, the book takes the novice on a step-by-step journey through complex areas such as database dynamics, data linkage, text analysis, networks and data visualization. The book is a treasure trove of information. It leads the field in the important task of bringing together computer science, statistics, and social science. I strongly recommend that all social scientists with an interest in ‘big data’ immerse themselves in this book."

Peter Elias CBE, University of Warwick

"The explosive growth in big data and in new technologies to analyze these data is transforming the practice of research in a variety of fields. Foster, et al. provides a well-timed, valuable guide to the new methods and tools associated with big data that can be used to address critical research questions in the social science field. The breadth of the material is impressive, providing a comprehensive summary of the methods and tools as well as practical guidance for their use. A key feature of the guide is the use of a case study to illustrate how big data techniques can be used to address a research question from beginning to end of the project, including providing examples of computer code targeted to specific steps in the project. Any researcher will find this unique guide to be useful, and it is essential reading for any social science practitioner that wants to use the best available data to conduct influential research in the near future."

Paul Decker, President and CEO, Mathematica Policy Research

"The typical statistics pedagogy has changed. In the past, textbooks assumed that data was hard to obtain, but neatly organized in a single file. Today, data is very easy to obtain from a number of data sources, often very messy, and analysts are now responsible for organizing it in addition to deriving useful insights. Foster, Ghani, Jarmin, Kreuter, and Lane have assembled a book that gives a pointed overview of tools to facilitate the entire digital lifespan of data in this era of analytics. Big Data and Social Science gives an evenhanded look at the myriad of ways to obtain data--whether scraping the web, web APIs, or databases--to conducting statistical analysis to doing analysis when your data cannot fit on a single computer. Meanwhile, they provide sound, diligent advice on pitfalls that still, and will always, exist. A book like this is useful for social scientists, experienced statisticians, econometricians, and computer programmers who want to see the tools available to them. It will also be a helpful text for a budding data scientist who wants a fairly technical preview of the landscape."

Tom Schenk Jr., Chief Data Officer, City of Chicago

"In Big Data and Social Science, the authors have deftly crafted one of the very best "how-to" books on big data that researchers, enterprise analysts, and government practitioners will find equally valuable. From Nodes, to Edges, to Arcs, the book takes the reader along a near-perfect path to understanding the fundamental elements of constructing a practical and realistic model for Big Data Analysis that any organization can execute by simply following the path outlined in this book. Elegant in its simplicity, Big Data and Social Science is one of those books that every research group and data-analysis team will want to have on their reference shelf."

Tom Herzog, Former Deputy Commissioner, NY State Department of Corrections and Community Supervision

Table of Contents


Why this book?

Defining big data and its value

Social science, inference, and big data

Social science, data quality, and big data

New tools for new data

The book’s "use case"

The structure of the book


Capture and Curation

Working with Web Data and APIs


Scraping information from the web

New data in the research enterprise

A functional view

Programming against an API

Using the ORCID API via a wrapper

Quality, scope, and management

Integrating data from multiple sources

Working with the graph of relationships

Bringing it together: Tracking pathways to impact



Acknowledgements and copyright

Record Linkage


Introduction to record linkage

Preprocessing data


Record linkage and data protection





DBMS: When and why

Relational DBMSs

Linking DBMSs and other tools

NoSQL databases

Spatial databases

Which database to use?



Programming with Big Data


The MapReduce programming model

Apache Hadoop MapReduce

Apache Spark



Modeling and Analysis

Machine Learning


What is machine learning?

The machine learning process

Problem formulation: Mapping a problem to machine learning methods



Practical tips

How can social scientists benefit from machine learning?

Advanced topics



Text Analysis

Understanding what people write

How to analyze text

Approaches and applications


Text analysis tools



Networks: The Basics


Network data

Network measures

Comparing collaboration networks



Inference and Ethics

Information Visualization


Developing effective visualizations

A data-by-tasks taxonomy




Errors and Inference


The total error paradigm

Illustrations of errors in big data

Errors in big data analytics

Some methods for mitigating, detecting, and compensating for errors



Privacy and Confidentiality


Why is access at all important?

Providing access

The new challenges

Legal and ethical framework






Workbook details



About the Editors

Ian Foster is a professor of computer science at the University of Chicago as well as a senior scientist and distinguished fellow at Argonne National Laboratory. His research addresses innovative applications of distributed, parallel, and data-intensive computing technologies to scientific problems in such domains as climate change and biomedicine. Methods and software developed under his leadership underpin many large national and international cyberinfrastructures. He is a fellow of the American Association for the Advancement of Science, the Association for Computing Machinery, and the British Computer Society. He received a PhD in computer science from Imperial College London.

Rayid Ghani is the director of the Center for Data Science and Public Policy, research director at the Computation Institute, and senior fellow at the Harris School of Public Policy at the University of Chicago. His research focuses on using machine learning and data science for high-impact social good and public policy problems in areas such as education, healthcare, energy, transportation, economic development, and public safety.

Ron S. Jarmin is the assistant director for research and methodology at the U.S. Census Bureau, where he oversees a broad research program in statistics, survey methodology, and economics to improve economic and social measurement within the U.S. federal statistical system. He is the author of many papers in the areas of industrial organization, business dynamics, entrepreneurship, technology and firm performance, urban economics, data access, and statistical disclosure avoidance. He earned a PhD in economics from the University of Oregon.

Frauke Kreuter is a professor at both the University of Maryland and the University of Mannheim. She is also head of the Statistical Methods Group at the Institute for Employment Research in Germany. Among her over 100 publications are several textbooks in survey statistics and data analysis. She established the International Program in Survey and Data Science and is a fellow of the American Statistical Association. She received a PhD from the University of Konstanz.

Julia Lane is a professor at the NYU Wagner Graduate School of Public Service and the NYU Center for Urban Science and Progress. She is also an NYU Provostial Fellow for Innovation Analytics. She co-founded the UMETRICS and STAR METRICS programs at the National Science Foundation, established a data enclave at NORC/University of Chicago, and co-founded the Longitudinal Employer-Household Dynamics Program at the U.S. Census Bureau and the Linked Employer Employee Database at Statistics New Zealand. She is the author/editor of 10 books and the author of over 70 articles in leading journals, including Nature and Science. She is an elected fellow of the American Association for the Advancement of Science and a fellow of the American Statistical Association. She received a PhD in economics from the University of Missouri.

About the Series

Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Machine Theory
MATHEMATICS / Probability & Statistics / General