Big Data Analytics Strategies for the Smart Grid: 1st Edition (Hardback) book cover

Big Data Analytics Strategies for the Smart Grid

1st Edition

By Carol L. Stimmel

Auerbach Publications

256 pages | 43 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781482218282
pub: 2014-07-25
SAVE ~$22.00
eBook (VitalSource) : 9780429173301
pub: 2016-04-19
from $55.00

FREE Standard Shipping!


By implementing a comprehensive data analytics program, utility companies can meet the continually evolving challenges of modern grids that are operationally efficient, while reconciling the demands of greenhouse gas legislation and establishing a meaningful return on investment from smart grid deployments.

Readable and accessible, Big Data Analytics Strategies for the Smart Grid addresses the needs of applying big data technologies and approaches, including Big Data cybersecurity, to the critical infrastructure that makes up the electrical utility grid. It supplies industry stakeholders with an in-depth understanding of the engineering, business, and customer domains within the power delivery market.

The book explores the unique needs of electrical utility grids, including operational technology, IT, storage, processing, and how to transform grid assets for the benefit of both the utility business and energy consumers. It not only provides specific examples that illustrate how analytics work and how they are best applied, but also describes how to avoid potential problems and pitfalls.

Discussing security and data privacy, it explores the role of the utility in protecting their customers’ right to privacy while still engaging in forward-looking business practices. The book includes discussions of:

  • SAS for asset management tools
  • The AutoGrid approach to commercial analytics
  • Space-Time Insight’s work at the California ISO (CAISO)

This book is an ideal resource for mid- to upper-level utility executives who need to understand the business value of smart grid data analytics. It explains critical concepts in a manner that will better position executives to make the right decisions about building their analytics programs.

At the same time, the book provides sufficient technical depth that it is useful for data analytics professionals who need to better understand the nuances of the engineering and business challenges unique to the utilities industry.


This book provides an in-depth analysis that will help utility executives, as well as regulators, investors, large power users and entrepreneurs, understand some of the tectonic changes coming to an industry that from the outside can seem impervious to change. Making sense of a chaotic future, Carol charts a path where everyone can benefit.

–Amit Narayan, PhD, CEO, AutoGrid

After more than a century providing a mission-critical resource to consumers around the world, traditional energy providers are realizing the power of big data and predictive analytics. Not only will adopting these technologies improve the transmission and distribution of energy, it more importantly will enable providers to adopt a services mentality. In her exceptional book, Carol examines these trends and breaks down very complex topics into prose that is easy to understand. I highly recommend this book to anyone in the energy industry looking to grow and evolve their business.

–Adrian Tuck, CEO, Tendril

Carol Stimmel defines utility data analytics as the application of techniques within the digital energy ecosystem that are designed to reveal insights that help explain, predict, and expose hidden opportunities to improve operational and business efficiency and to deliver real-world situational awareness. She then provides the framework, methodology, insight, and experiential observations to help utilities conceive, plan, implement, enhance, and sustain the imperative smart grid analytics required to achieve the inexorable change taking place in the energy delivery ecosystem. Volume, velocity, variety, and value—the characteristics ascribed to ‘big data’ will aptly characterize the reader's and practitioner's view of Ms. Stimmel's book.

–Ivo Steklac, GM Residential & Commercial Energy Solutions, SunPower Corporation

The author has done an excellent job of leveraging her experience in the industry and her strong technical background to create a book that is a very easy-to-read, useful tool for anyone trying to get started in applying big data analytics to the utility industry. She not only provides the reader with a solid base knowledge and background but provides solid examples of how data analytics can be applied within a utility environment and the advantages that can be gained by doing so.

–Ron Gerrans, CEO, Genus Zero and former CEO, E Source

Table of Contents


Putting the Smarts in the Smart Grid

Chapter Goal

The Imperative for the Data-Driven Utility

Big Data: We’ll Know It When We See It

What Are Data Analytics?

The Data Analytics Infrastructure

Starting from Scratch

Mind the Gap

Culture Shift

A Personal Case Study

Ouija Board Economics

Business as Usual Is Fatal to the Utility

To Be or Not to Be

Finding Opportunity with Smart Grid Data Analytics

Building the Foundation for Data Analytics

Chapter Goal

Perseverance Is the Most Important Tool

"It’s Too Hard" Is Not an Answer

Building the Analytical Architecture

The Art of Data Management

Managing Big Data Is a Big Problem

The Truth Won’t Set You Free

One Size Doesn’t Fit All

Solving the "Situation-Specific" Dilemma

The Build-Versus-Buy War Rages On

When the Cloud Makes Sense

Change Is Danger and Opportunity

Transforming Big Data for High-Value Action

Chapter Goal

The Utility as a Data Company

Creating Results with the Pareto Principle


The Business of Algorithms

Data Classes

Just in Time

Seeing Intelligence

Remember the Human Being

The Problem with Customers

The Transformation of the Utility

Bigger Is Not Always Better

Assessing the Business Issues

Start with a Framework


Applying Analytical Models in the Utility

Chapter Goal

Understanding Analytical Models

What Exactly Are Models?

Warning: Correlation Still Does Not Imply Causation

Using Descriptive Models for Analytics

Using Diagnostic Models for Analytics

How Diagnostic Tools Help Utilities

Predictive Analytics

Prescriptive Analytics

An Optimization Model for the Utility

Toward Situational Intelligence

Enterprise Analytics

Chapter Goal

Moving Beyond Business Intelligence

Energy Forecasting

Asset Management

Demand Response and Energy Analytics

Dynamic-Pricing Analytics

Revenue-Protection Analytics

Breaking Down Functional Barriers

Operational Analytics

Chapter Goal

Aligning the Forces for Improved Decision-Making

The Opportunity for Insight

Adaptive Models

Focus on Effectiveness

Visualizing the Grid

Distributed Generation Operations: Managing the Mix-Up

Grid Management

The Relationship Between Standards and Analytics

Resiliency Analytics

Extracting Value from Operational Data Analytics

Customer Operations and Engagement Analytics

Chapter Goal

Increasing Customer Value

Customer Service

Advanced Customer Segmentation

Sentiment Analysis

Revenue Collections

Call Center Operations

Utility Communications

What’s in It for the Customer?

Enhanced Billing and Customer-Facing Web Portals

Home Energy Management

Strategic Value

Analytics for Cybersecurity

Chapter Goal

Cybersecurity in the Utility Industry

The Threat Against Critical Infrastructure

How the Smart Grid Increases Risk

The Smart Grid as Opportunity for Dark Mischief

The Role of Big Data Cybersecurity Analytics

Predict and Protect

Cybersecurity Applications

Proactive Approaches

Global Action for Coordinated Cybersecurity

The Changing Landscape of Risk


Sourcing Data

Chapter Goal

Sourcing the Data

Smart Meters


Control Devices

Intelligent Electronic Devices

Distributed Energy Resources

Consumer Devices

Historical Data

Third-Party Data

Working with a Variety of Data Sources

Data Fusion

Big Data Integration, Frameworks, and Databases

Chapter Goal

This Is Going to Cost

Storage Modalities


Network-Attached Storage

Object Storage

Data Integration

The Costs of Low-Risk Approaches

Let the Data Flow



Hadoop Distributed File System

How Does This Help Utilities?

Other Big Data Databases


In-Memory or Main Memory Databases

Object-Oriented Database Management Systems

Time Series Database Servers

Spatial and GIS Databases

The Curse of Abundance

Extracting Value

Chapter Goal

We Need Some Answers Here

How Long Does This Take?

Mining Data for Information and Knowledge

The Process of Data Extraction

When More Isn’t Always Better

Running for Performance

Hadoop: A Single-Purpose Batch-Data Platform?

Stream Processing

Complex Event Processing

Process Historians

Avoid Irrational Exuberance

Envisioning the Utility

Chapter Goal

Big Data Comprehension

Why Humans Need Visualization

Walking Toward the Edge

The Role of Human Perception

Preattentive Processing

The Utility Visualized

Advancing Business Intelligence

High-Impact Operations

Improving Customer Value

Making Sense of It All

A Partnership for Change

Chapter Goal

With Big Data Comes Big Responsibility

Abandon All Hope, Ye Who Enter Here?

Privacy, Not Promises


Data Management


Privacy Enhancement

Enabling Consent

Data Minimization

The Role of Metadata

The Utility of the Future Is a Good Partner


About the Author


Carol L. Stimmel began working with "big data analytics" in 1991 while hacking code and modeling 3D systems for meteorological research—years before that combination of words ever became buzzword compliant. In those 23 years, she has spent the last 7 focusing on the energy industry, including smart grid data analytics, microgrids, home automation, data security and privacy, smart grid standards, and renewables generation. She has participated in emerging technology markets for the majority of her career, including engineering, designing new products, and providing market intelligence and analysis to utilities and other energy industry stakeholders.

Carol has owned and operated a digital forensics company, worked with cutting-edge entrepreneurial teams; co-authored a standard text on organizational management, The Manager Pool; and held leadership roles with Gartner, E Source, Tendril, and Navigant Research. She is the founder and CEO of the research and consulting sustainability company, Manifest Mind, LLC, which brings rigorous, action-based insight to advanced technology projects that create and maintain healthy ecosystems for people and the environment. Carol holds a BA in Philosophy from Randolph-Macon Woman’s College.

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Database Management / General
COMPUTERS / Database Management / Data Mining
COMPUTERS / Information Technology
TECHNOLOGY & ENGINEERING / Power Resources / Electrical