A large electric utility generating 15-minute interval reads from millions of smart meters produces data volumes that dwarf most enterprise systems. The challenge is not storage; cloud and on-premises warehouses handle the volume. The challenge is governance: ensuring that the right data, at the right quality level, reaches the right application without mixing validated billing data with raw, unvalidated telemetry.
The Data Sources in a Utility Environment
Understanding the data problem starts with mapping the sources:
AMI interval data. Electric meters from Itron and Landis+Gyr report consumption every 15 minutes. Before this data is used for billing, it must pass through the MDM (Oracle MDM or SAP MDG-U) for VEE: validation, estimation, and editing. Downstream of VEE, the data is fit for the CIS. Upstream of VEE, it is raw and should not be used for revenue calculations.
SCADA telemetry. Substations and transmission assets generate analog measurements (voltage, current, power flow) and digital status points (breaker positions, transformer tap positions) at one-to-four-second intervals. This data belongs in the historian (OSIsoft PI is the dominant platform) and the EMS/ADMS, not in the billing system.
CIS transactional data. Oracle CC&B, SAP IS-U, and Cayenta CIS store customer accounts, rate schedules, billing cycles, payment history, and service orders. This is structured relational data with a well-defined schema. Analytical queries against production CIS databases create performance risk; a reporting replica or data warehouse extract is standard practice.
GIS and network model data. Asset attributes, network topology, and service point locations are used in analytics for loss analysis, load forecasting by feeder, and capital planning.
The Architecture: From Source to Analytics
A typical utility data architecture routes each source to an appropriate staging layer before analytics:
The MDM provides validated interval data to a time-series database or data warehouse. Cloud platforms on AWS and Azure both host large-scale time-series workloads. SAP HANA, used in SAP S/4HANA Utilities deployments, provides in-memory analytics against IS-U billing data and FI-CA financial records without requiring a separate warehouse for those domains.
Historian data from SCADA remains in the OSIsoft PI ecosystem for operational analytics (frequency deviations, transformer loading trends) and can be replicated to a cloud data lake for longer-term capacity planning queries.
CIS data typically replicates nightly to a data warehouse where it can join with AMI reads for customer analytics: identifying high-consumption outliers, flagging potential meter faults, and supporting demand response targeting.
Practical Use Cases
Non-technical loss detection. Comparing transformer load (from SCADA) against the sum of meter reads on that transformer’s secondary (from AMI) identifies distribution losses. Gaps beyond expected technical loss levels indicate metering problems or theft. This analysis runs at scale on a data platform, not inside the CIS.
Rate design analytics. Utilities designing time-of-use or tiered rates need interval data by customer class. That analysis requires joined CIS and AMI data in a warehouse environment. SAP IS-U supports rate simulation tools, but the underlying consumption analytics typically happen in a separate layer.
Demand forecasting. Accurate short-term load forecasting for the next 24 to 48 hours requires historical interval data, weather inputs, and calendar data. This feeds economic dispatch and wholesale market scheduling. In ERCOT, the Texas deregulated market, load forecast accuracy directly affects settlements.
Data Governance: The Underinvested Layer
Governance is the most frequently skipped investment in utility data programs. The specific failure mode is mixing unvalidated AMI reads with validated billing data in a shared analytics layer. An analyst who pulls “consumption data” without knowing its VEE status will produce incorrect results. A data catalog that tags data by source, VEE status, and intended use is a prerequisite for reliable analytics.
Access control matters too. CIS data contains customer personal information and financial records subject to state privacy regulations and tariff confidentiality requirements. A single analytics platform handling both SCADA telemetry and CIS customer data needs role-based access that mirrors the separation required by policy. See the modern software solutions overview for how platform vendors are approaching this in their cloud offerings. For architecture guidance on your specific CIS or ERP environment, AvanSaber can help.