10 min read2,126 words

Correlating AMI Last-Gasp Messages to Network Features

An advanced metering infrastructure deployment turns every service point into an outage sensor: when the power fails, each meter spends its last stored energy sending a “last-gasp” message before it goes dark. The signal is invaluable and treacherous in equal measure. A real feeder outage produces a burst of thousands of gasps within seconds, but those messages traverse a mesh network with per-hop delay and per-device clock skew, so they arrive out of order and smeared across time. A single meter can also gasp on its own — a blown service fuse, a tripped house breaker, a dying meter battery — with no network outage behind it at all. Correlating these messages naively spawns phantom outages from clock skew and dispatches crews to non-events. This procedure windows the bursts by event time, snaps each meter to its service point, and rolls confirmed points up to a feeder, implementing the correlation contract from SCADA and AMI telemetry integration for the highest-volume feed in outage routing and impact automation. The feeder rollup depends on the containment relationships defined in asset hierarchy design for water and electric.

Environment Prerequisites

Lock the following before correlating a live last-gasp feed. The dominant risk here is volume: an approach that works on a handful of messages collapses under a storm burst of tens of thousands.

  • Python runtime: Python 3.11 in an isolated environment so windowing and tolerance behavior stay reproducible.
  • Dependencies: pandas>=2.0 for vectorized time-bucketing, geopandas>=1.0 and shapely>=2.0 for the meter-to-service-point snap, and networkx>=3.0 for the feeder rollup over the asset hierarchy.
  • Service-point layer: Service points as a point layer with a stable meter-id-to-service-point mapping where it exists, and geometry for the spatial fallback, exported from a reconciled snapshot on a currently validated topology.
  • Feeder hierarchy: The service-point-to-transformer-to-feeder containment resolved into a lookup or a directed graph, so confirmed points aggregate to a parent feeder deterministically.
  • CRS and tolerance: A single authoritative projected EPSG code shared by the meter register and the GIS, aligned per CRS alignment and geodetic transformations, and a snap tolerance at design accuracy (0.1–0.5 m).
  • Skew budget and thresholds: A measured worst-case skew across the AMI mesh to size the event-time window, and a per-feeder corroboration threshold below which a burst stays provisional.

Schema-Aware Validation Protocol — Run Before Correlation

Most last-gasp correlation errors come from the setup, not the messages. Work this ordered checklist first; the earliest item is the most frequent culprit.

  1. Size the window to the real mesh skew, not a guess. Measure the spread between event time and arrival time across a known past outage’s burst. A window narrower than that spread shatters one outage into several; the resulting phantom swarm is the single most common failure.
  2. Prefer the id mapping, snap only as fallback. A meter id resolves to its service point exactly; a spatial snap is an approximation used only when the mapping is missing. Confirm the mapping table is current before trusting the fallback, and pin the snap tolerance to design accuracy so a fallback never crosses to a neighbor’s service point.
  3. Confirm event time is populated and monotonic per meter. Meters that report only arrival time, or that reset their clocks, corrupt the window bucket. Reject or quarantine any message whose event time is absent or implausibly ahead of arrival.
  4. Set the corroboration threshold from feeder density. A dense urban feeder needs more corroborating gasps than a rural lateral before an outage is confirmed. A threshold set too low reintroduces single-meter phantoms; too high delays detection of a genuine small outage.
  5. Verify the feeder hierarchy resolves every service point. A service point with no parent feeder cannot roll up and silently vanishes from the affected-area count. Validate the containment lookup covers the full service-point set before correlating.
Last-gasp burst windowed, snapped to service points, and rolled up to a feeder with phantom suppression On the left, a burst of last-gasp messages from many meters enters an event-time window. Inside the window, meters snap to service points, which roll up through the asset hierarchy to a single confirmed feeder outage on the right when the corroboration threshold is met. Below, a lone last-gasp from one isolated meter falls short of the threshold and is held as a provisional, suppressed phantom rather than a confirmed outage. Last-gasp burst many meters Event-time window skew budget Snap to service points id, then spatial Feeder outage confirmed threshold met one meter Below threshold no corroboration Provisional phantom suppressed

Minimal Reproducible Implementation

The correlator below is the centerpiece. It takes a batch of last-gasp messages as a pandas DataFrame, buckets them into event-time windows sized to the mesh skew budget, snaps each meter to its service point (preferring the exact id mapping and falling back to a spatial snap within tolerance), rolls the confirmed points up to their parent feeder through the asset hierarchy, and applies a per-feeder corroboration threshold so isolated single-meter gasps stay provisional. It handles missing columns, unmapped meters, and empty batches without halting, and returns a structured result the detection layer and audit log both read.

from __future__ import annotations

import logging
from dataclasses import dataclass, field

import geopandas as gpd
import networkx as nx
import pandas as pd
from shapely.geometry import Point

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")


@dataclass
class CorrelationResult:
    """Structured outcome of one last-gasp correlation cycle."""

    confirmed_feeders: dict[str, int] = field(default_factory=dict)  # feeder -> count
    provisional: list[str] = field(default_factory=list)            # suppressed points
    unmapped: int = 0
    windows: int = 0


def _snap_meter(row: pd.Series, service_pts: gpd.GeoDataFrame,
                id_to_sp: dict[str, str], tol_m: float) -> str | None:
    """Resolve a meter to its service point: exact id first, spatial fallback."""
    sp = id_to_sp.get(row["meter_id"])
    if sp is not None:
        return sp
    distances = service_pts.geometry.distance(Point(row["x"], row["y"]))
    idx = distances.idxmin()
    if float(distances.loc[idx]) > tol_m:
        return None
    return str(service_pts.loc[idx, "SP_ID"])


def correlate_last_gasps(
    messages: pd.DataFrame,
    service_pts: gpd.GeoDataFrame,
    id_to_sp: dict[str, str],
    hierarchy: nx.DiGraph,
    window_seconds: int = 45,
    tolerance_m: float = 0.25,
    threshold: int = 3,
) -> CorrelationResult:
    """Correlate a last-gasp burst to confirmed feeder outages.

    Windows messages by event time, snaps meters to service points, rolls up
    to parent feeders, and suppresses feeders below the corroboration
    threshold. Returns a structured CorrelationResult.
    """
    result = CorrelationResult()
    required = {"meter_id", "event_time", "x", "y"}
    if not required.issubset(messages.columns):
        raise KeyError(f"messages missing columns: {required - set(messages.columns)}")
    if messages.empty:
        logging.info("empty last-gasp batch; nothing to correlate")
        return result

    df = messages.copy()
    df["event_time"] = pd.to_datetime(df["event_time"], utc=True, errors="coerce")
    df = df.dropna(subset=["event_time"])  # drop meters with no usable event time

    # 1. Bucket by event time so one outage's burst clusters together.
    epoch = df["event_time"].astype("int64") // 10**9
    df["window"] = epoch // window_seconds
    result.windows = int(df["window"].nunique())

    for window, group in df.groupby("window"):
        # 2. Snap each meter to a service point (id first, spatial fallback).
        group = group.copy()
        group["sp_id"] = group.apply(
            _snap_meter, axis=1,
            service_pts=service_pts, id_to_sp=id_to_sp, tol_m=tolerance_m,
        )
        result.unmapped += int(group["sp_id"].isna().sum())
        confirmed_pts = group["sp_id"].dropna().unique()

        # 3. Roll service points up to their parent feeder via the hierarchy.
        feeder_counts: dict[str, int] = {}
        for sp in confirmed_pts:
            if sp not in hierarchy:
                continue
            # Walk ancestors to the feeder node (kind == "feeder").
            feeder = next(
                (n for n in nx.ancestors(hierarchy, sp)
                 if hierarchy.nodes[n].get("kind") == "feeder"),
                None,
            )
            if feeder is not None:
                feeder_counts[feeder] = feeder_counts.get(feeder, 0) + 1

        # 4. Apply the corroboration threshold; suppress phantoms.
        for feeder, count in feeder_counts.items():
            if count >= threshold:
                result.confirmed_feeders[feeder] = (
                    result.confirmed_feeders.get(feeder, 0) + count
                )
            else:
                result.provisional.extend(
                    f"{feeder}@{window}" for _ in range(count)
                )

    logging.info(
        "correlated %d window(s): confirmed=%d feeder(s), provisional=%d, unmapped=%d",
        result.windows, len(result.confirmed_feeders),
        len(result.provisional), result.unmapped,
    )
    return result

The design turns two failure modes into explicit, observable outcomes. Windowing on event time — bucketing epoch // window_seconds rather than sorting by arrival — keeps a skewed, out-of-order burst together, so a single feeder outage surfaces as one confirmed event instead of a scatter of small ones. The corroboration threshold makes phantom suppression a policy rather than an accident: a lone gasp from a blown house fuse lands in provisional and never dispatches a crew, while a genuine burst clears the threshold and confirms. Preferring the id mapping over the spatial snap keeps resolution exact where the data allows and reserves the tolerance-bounded fallback for the gaps, so a fallback can never quietly attach a meter to a neighbor’s service point.

Production Deployment Pattern

Promote the correlator from notebook to control with the same discipline the rest of the outage pipeline uses.

  1. Cache the service-point layer and hierarchy. Load service_pts and the hierarchy graph once and refresh them on the topology-validation cadence, never per batch, so correlation keeps pace with a storm burst instead of blocking on reads.
  2. Feed a durable, replayable stream. Consume last-gasp batches from the same durable queue the SCADA fault-event consumer uses, so a restart replays committed offsets and a burst buffers rather than drops.
  3. Tune the window and threshold per region. Store the skew budget and corroboration threshold as per-feeder parameters under version control, and re-derive them from observed outages rather than hard-coding one global value.
  4. Hand confirmed feeders to impact analysis. The confirmed feeder set drives computing affected-customer counts with NetworkX and the critical-facility checks in automating critical-facility outage notifications.
  5. Persist an immutable audit record. Log each cycle’s confirmed, provisional, and unmapped counts alongside the window and threshold parameters and library versions, so the detection decision is reconstructable during a reliability review.

Conclusion

Last-gasp messages are the densest and noisiest outage signal a utility receives, and correlating them well is the difference between an accurate affected-area picture and a phantom swarm. By windowing bursts on event time, snapping meters to service points by id with a tolerance-bounded fallback, rolling confirmed points up to their feeder, and gating on a corroboration threshold, this correlator turns a smeared flood of gasps into confirmed feeder outages and suppresses the single-meter false alarms that would otherwise waste crews. Cache the layers, replay from a durable stream, tune the window and threshold per region, and the correlation becomes a dependable input to impact analysis rather than a source of noise.

Related pages